DevOps Blog - Nicolas Paris

Istio with Let's Encrypt Example

KubernetesIstio

It's not that hard to setup Istio with let's encrypt. This will work for as domain name you need.
This post is based on Kubernetes, Helm, Laravel, PHP-FPM, Nginx, GitLab the DevOps Way, it goes deeper on the istio/let's encrypt part. If your interest on the big picture, the post might give you more information.

Make sure you have istio and cert-manager installed.

istioctl install --set profile=default
#kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v0.11.1/cert-manager.yaml
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.3/cert-manager.yaml

Let's create the issuer first, simply check the documentation.

# certificate-issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
  namespace: kube-system
spec:
  acme:
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: xxx@gmail.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-prod
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
        ingress:
          class: istio

Check that is up & running.

$ kubectl get ClusterIssuer -n istio-system
NAME               READY   AGE
letsencrypt-prod   True    82d

You need a gateway

# gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: cluster-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - www.url.com
    - www.url.net
  - port:
      number: 443
      name: https-url-1
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: "url-1-certs" # This should match the Certificate secretName
    hosts:
    - "www.url.com" # This should match a DNS name in the Certificate
  - port:
      number: 443
      name: https-url-2
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: "url-2-certs" # This should match the Certificate secretName
    hosts:
    - "www.url.net" # This should match a DNS name in the Certificate

let apply and check the result

$ kubectl apply -f gateway.yaml
$ kubectl get gw
NAME              AGE
cluster-gateway   9d

This should add a service in the istio-system namespace. In the Google Cloud case, it will add a load balancer with an external IP adresse. You can check the following.

$ kubectl get svc -n istio-system
NAME                        TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)                                      AGE
cm-acme-http-solver-t67d9   NodePort       10.32.xx.xx   <none>         8089:30608/TCP                               16d
istio-ingressgateway        LoadBalancer   10.32.xx.xx    35.201.x.x   15021:32411/TCP,80:31500/TCP,443:30929/TCP   82d
istiod                      ClusterIP      10.32.xx.xx   <none>         15010/TCP,15012/TCP,443/TCP,15014/TCP        82d

Now, let's instanciate the certificate himself, you dont need to have the up and running service to make it work. But you do need the gateway properly configure.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: url-1-certs
  namespace: istio-system
spec:
  secretName: url-1-certs
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  commonName: www.url.com
  dnsNames:
  - www.url.com

This will give you the certificate

$ kubectl get certificates -n istio-system
NAME                               READY   SECRET                             AGE
url-0-certs               True    url-0-certs               12d

If somethings goes wrong, check the describe, it will show object you have to search for. Here is some example.

$ kubectl describe certificates/url-0-certs -n istio-system
$ kubectl get order -n istio-system
NAME                                                STATE     AGE
url-0-certs-x26mj-1472136003               valid     12d
url-1-certs-zkkn4-4225748097               valid     11d

Maybe the most usefull object to get information on why your let's encrypt has failed is inside the challenges

$ kubectl get challenges -n istio-system
NAME                                                      STATE     DOMAIN                     AGE
url-0-certs-wlmd4-3390807151-264327372   pending          www.url.fr   16d

The will find information inside the describe:

$ kubectl describe challenges/url-0-certs-wlmd4-3390807151-264327372 -n istio-system
Name:         url-0-certs-wlmd4-3390807151-264327372
Namespace:    istio-system
Labels:       <none>
Annotations:  <none>
API Version:  acme.cert-manager.io/v1
Kind:         Challenge
# ...
# ...
Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for HTTP-01 challenge propagation: failed to perform self check GET request 'http://url.fr/.well-known/acme-challenge/m8dtAEELdfQICpIza3h714Y-xxx': Get "http://noyon.nci-expeditions.fr/.well-known/acme-challenge/m8dtAEELdfQICpIza3h714Y-xxx": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  State:       pending

This is an explaination.

We are not done, is missing the virtual service and the service.

# virtualservice.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: myapp
spec:
  hosts:
  - www.url.com
  gateways:
  - cluster-gateway
  http:
  - match:
    - uri:
        prefix: "/"
    route:
    - destination:
        host: myapp.default.svc.cluster.local
        port:
          number: 80

---
apiVersion: v1
kind: Service
metadata:
  name: myapp
spec:
  type: NodePort
  selector:
    app: myapp
  ports:
      # By default and for convenience, the `targetPort` is set to the same value as the `port` field.
    - port: 80
      targetPort: 80

I'm not showing the deployment but it is as usual.

Force redirection from HTTP to HTTPS

The tricky part is to issue the certificate on the first place, you can get the redirection working you can issue a temporary certificate with an annotation.
This will work even if the /.well-known/acme-challenge/ is forced to an https redirection.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: url-1-certs
  namespace: istio-system
  annotations:
    cert-manager.io/issue-temporary-certificate: "true"
spec:
  secretName: url-1-certs

And the forced redirection.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: cluster-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - www.example.com
    tls:
      httpsRedirect: true

As I said in the introduction, this is a deeper explaination for a longer and broader post on Kubernetes, Helm, Laravel, PHP-FPM, Nginx, GitLab the DevOps Way.
Hope it can help someone.