Development Inside Kubernetes Part 2: Traefik Dashboard Setup

In Part 1 of the series, I showed the setup of Rancher Desktop. This part is all about Traefik and using it to handle ingress. I am going to show how to setup ingress by exposing the dashboard.

If you are not familiar with ingress controllers like Traefik, it's real value comes with having many servers. Modern software now runs multiple services on the same machine. It used to be you had one application that was a monolith. Software has become more discrete such that an application can be composed of multiple services. Each one requiring https and port 443. All of the services cannot listen on the same port. Instead, Traefik will refer to each service by name and divert "traffic" accordingly.

Setup Middleware

Middleware is used by Traefik to handle things like auth, url rewriting, headers, etc... Routes can depend upon them for configuration, so I am going to set it up first. The middleware I am going to use for my dashboard are url rewriting. I have two. One adds /dashboard when I go to /. The other replaces http for https

Here is the middleware.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: dashboard-rewrite
  namespace: kube-system
spec:
  addPrefix:
    prefix: /dashboard
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata: 
  name: redirect-permanent
  namespace: kube-system
spec:
  redirectScheme:
    permanent: true
    scheme: https

Applies with

kubectl create -f middleware.yaml

Ingress Routes

This is the preferred method for managing ingress. Before we setup https, we will use http. I have set the entrypoint to web for http. I added 2 rules. One for / and another for /api. There is middleware added for the first rule which rewrites / for /dashboard.

Let's talk about dashboard-traefik.$CLUSTERIP.sslip.io for a second. There are a few things to unpack. First $CLUSTERIP is a token I plan to replace with the IP address of my cluster. If you want to know your cluster ip, just go to your rancher nodes and it will display the ip address.

Let's say my ip address is 10.10.10.10. This makes the domain name dashboard-traefik.10.10.10.10.sslip.io. For those in the back, sslip.io is DNS service that resolves the ip address for whatever <ip>.sslip.io subdomain. That means dashboard-traefik.10.10.10.10.sslip.io will resolve to 10.10.10.10.

This is my ingress.yaml.tmpl

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard-secure
  namespace: kube-system
spec:
  entryPoints:
    - web
  routes:
    - kind: Rule
      match: Host("dashboard-traefik.$CLUSTERIP.sslip.io")
      services:
        - kind: TraefikService
          name: api@internal
      middlewares:
        - name: dashboard-rewrite
    - kind: Rule
      match: Host("dashboard-traefik.$CLUSTERIP.sslip.io") && PathPrefix("/api")
      services:
        - kind: TraefikService
          name: api@internal

export CLUSTERIP=10.10.10.10
cat ingress.yaml.tmpl | envsubst > ingress.yaml
kubectl create -f ingress.yaml

I can test this is working with

curl -si http://dashboard-traefik.$CLUSTERIP.sslip.io  | head -1

Cert-Manager

cert-manager is used in kubernetes to manage certificates for apps in kubernetes. I am going to use it for self-signed certificate now, but later I will use it go get a free certificate from Let's Encrypt.

First, to install cert-manager I need to first setup a values file.

installCRDs: true
extraArgs:
  - --dns01-recursive-nameservers=1.1.1.1:53,9.9.9.9:53
  - --dns01-recursive-nameservers-only
podDnsPolicy: None
podDnsConfig:
  nameservers:
    - "1.1.1.1"
    - "9.9.9.9"

The purpose of settings here are to ensure the hosts DNS is not used by the cluster when resolving for certificates.

helm install \
        cert-manager jetstack/cert-manager \
        --namespace cert-manager \
        --create-namespace \
        --values=cert-manager-values.yaml

Self-signed Certificate

I need an issuer to issue certificates. I want one that will issue self-signed certificates cluster wide.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned
spec:
  selfSigned: {}

I install with

kubectl create -f selfsigned.yaml

Now I can self-sign a certificate. This will create a certificate with a secret in dashboard-crt

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: dashboard
  namespace: kube-system
spec:
  dnsNames:
    - 'dashboard-traefik.$CLUSTERIP.sslip.io'
  issuerRef:
    kind: ClusterIssuer
    name: selfsigned
  secretName: dashboard-crt

I install with

cat selfsigned-certificate.yaml.tmpl | envsubst > selfsigned-certificate.yaml
kubectl create -f selfsigned-certificate.yaml

Now that I have a self-signed certificate, I can setup https. I have updated the IngressRoute for https.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard-secure
  namespace: kube-system
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host("dashboard-traefik.$CLUSTERIP.sslip.io")
      services:
        - kind: TraefikService
          name: api@internal
      middlewares:
        - name: dashboard-rewrite
    - kind: Rule
      match: Host("dashboard-traefik.$CLUSTERIP.sslip.io") && PathPrefix("/api")
      services:
        - kind: TraefikService
          name: api@internal
  tls:
    secretName: dashboard-crt

cat ingress.yaml.tmpl | envsubst > ingress.yaml
kubectl delete ingressroute traefik-dashboard-secure -n kube-system
kubectl create -f ingress.yaml

I can test this is working with

curl -ksi http://dashboard-traefik.$CLUSTERIP.sslip.io  | head -1

I have to use -k because our certificate is self-signed.

Next, I am going to add another ingress controller for http. This one adds the redirect-permanet middleware which will redirect all http traffic to https.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard-http
  namespace: kube-system
spec:
  entryPoints:
    - web
  routes:
    - kind: Rule
      match: Host("dashboard-traefik.$CLUSTERIP.sslip.io")
      services:
        - kind: TraefikService
          name: api@internal
      middlewares:
        - name: dashboard-rewrite
        - name: redirect-permanent
    - kind: Rule
      match: Host("dashboard-traefik.$CLUSTERIP.sslip.io") && PathPrefix("/api")
      services:
        - kind: TraefikService
          name: api@internal
      middlewares:
        - name: redirect-permanent

cat ingress-http.yaml.tmpl | envsubst > ingress-http.yaml
kubectl create -f ingress-http.yaml

I can test this is working with

curl -si http://dashboard-traefik.$CLUSTERIP.sslip.io  | head -1

Which returns a status 302 to show it's redirecting.