Kubernetes Self Signing and Trusting your CA
What's in This Post
Here are some quick steps to assist you in creating a private CA within Kubernetes that you can then trust on the Linux hosts within your perimeter.
Install cert-manager
The instructions for installing cert-manager are quite straight forward, so I wont include any further details here.
Create the CA
cert-manager provides some very good instructions on creating a self-signed CA. However I prefer to use a single Cluster Issuer. Using different issuers might allow an organisation to tailor certificate policies per namespace, which inturn might be used by separate organisations. In my case, my lab replicates a single organisation operational model, so a common set of policies covering all certificates in the k8s cluster makes a lot more sense.
1---
2apiVersion: cert-manager.io/v1
3kind: ClusterIssuer
4metadata:
5 name: selfsigned-cluster-issuer
6spec:
7 selfSigned: {}
8---
9apiVersion: cert-manager.io/v1
10kind: Certificate
11metadata:
12 name: my-lab-root
13 namespace: cert-manager # Default cert-manager namespace for cluster wide resources
14spec:
15 isCA: true
16 commonName: my-lab-selfsigned-ca
17 secretName: my-root-secret
18 privateKey:
19 algorithm: ECDSA
20 size: 256
21 issuerRef:
22 name: selfsigned-cluster-issuer
23 kind: ClusterIssuer
24 group: cert-manager.io
25---
26apiVersion: cert-manager.io/v1
27kind: ClusterIssuer
28metadata:
29 name: my-lab-root-issuer # using the self signed issuer caused problems with importing the crt into linux
30spec:
31 ca:
32 secretName: my-root-secret
The example above only differs from the cert-manager documented example by creating a ClusterIssuer.
Updating Trust Anchors
Now that we have a CA that will be issuing certificates to endpoints in Kubernetes, we'll need to add the root certificate into the trust store of any clients. The certificate is stored in base64 form within the my-root-secret we created above. We can see the certificates and private key by looking at the contents of the secret:
1user@k8s-master:~$ kubectl get secret my-root-secret -n cert-manager -o yaml
2
3apiVersion: v1
4data:
5 ca.crt: 1234567890ABCDEFG=
6 tls.crt: 1234567890ABCDEFG=
7 tls.key: abcdefg1234567890=
8kind: Secret
9metadata:
10 annotations:
11 cert-manager.io/alt-names: ""
12 cert-manager.io/certificate-name: my-lab-root
13 cert-manager.io/common-name: my-lab-selfsigned-ca
14 cert-manager.io/ip-sans: ""
15 cert-manager.io/issuer-group: cert-manager.io
16 cert-manager.io/issuer-kind: ClusterIssuer
17 cert-manager.io/issuer-name: selfsigned-cluster-issuer
18 cert-manager.io/uri-sans: ""
19 creationTimestamp: "2021-10-26T23:13:17Z"
20 name: my-root-secret
21 namespace: cert-manager
22 resourceVersion: "12975422"
23 uid: abcd2039-5866-42e0-b078-63457c6e6c22
24type: kubernetes.io/tls
We only need the ca.crt value in order to update our clients' trust stores. We'll also need to decode the output from base64. We can do all that with a single line:
1user@k8s-master:~$ kubectl get secret my-root-secret -n cert-manager -o yaml | grep ca.crt | cut -d ':' -f 2 | sed -e 's/^[ \t]*//' | base64 -d > my-ca.pem
2user@k8s-master:~$ cat my-ca.pem
3-----BEGIN CERTIFICATE-----
4MIIABCDEFG1234567890abcdefgABCDEFG1234567890abcdefgABCDEFG123456
5ABCDEFG1234567890abcdefgABCDEFG1234567890abcdefgABCDEFG123456789
6-----END CERTIFICATE-----
Trust the CA on your clients
To update your Centos hosts to trust the newly created CA you'll need to update the ca-bundle on each host.
Copy the CA certificate pem file to each client's trust anchor source directory. Then use the update-ca-trust extract command to force a rebuild of the bundled certificates to now include your CA.
1user@my-host01:~$ cp my-ca.pem /etc/pki/ca-trust/source/anchors/
2user@my-host01:~$ update-ca-trust extract
We can check to see if the new CA has been added by looking at the updated bundle file:
1user@my-host01:~$ head /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
2# my-lab-selfsigned-ca
3-----BEGIN TRUSTED CERTIFICATE-----
4MIIABCDEFG1234567890abcdefgABCDEFG1234567890abcdefgABCDEFG123456
5ABCDEFG1234567890abcdefgABCDEFG1234567890abcdefgABCDEFG123456789