Custom domains

This article describes how you can use your own domains with Bunnyshell.

📘

The recommended solution is to use the popular open-source solution ExternalDNS to handle the actual DNS records creation and removal.

Use your own domain in bunnyshell.yaml

The first step is to instruct Bunnyshell to allow using your own domain when creating hosts (which translates to creating Ingresses). You just need to specify the selfManagedDns attribute with the true value under the host.

A simple example is provided below:

components:
    -
        kind: Service
        name: nginx
        dockerCompose:
            image: 'nginx:latest'
            ports:
                - '80:80'
        hosts:
            -
                hostname: '*.myowndomain.com'
                path: /
                servicePort: 80
                selfManagedDns: true

This way, Bunnyshell will not create the DNS record for the hostname, but leave the DNS creation to a 3rd party - ExternalDNS in this case.

Installing ExternalDNS in your cluster

In order to complete the setup and also have the actual DNS records created, you need ExternalDNS installed in your Kubernetes cluster and configured/integrated with your DNS provider.

The list of available DNS providers is available on ExternalDNS's Github repository.

Install ExternalDNS for CloudFlare

You can use a Helm Chart provided by Bitnami to easily install ExternalDNS.

helm upgrade --install external-dns-bunnyshell \
  --create-namespace \
  --namespace=external-dns \
  --set "sources[0]"=ingress \
  --set provider=cloudflare \
  --set policy=sync \
  --set cloudflare.apiToken=Z9-n1s**********************WwM-gVoRiYnk \
  --set cloudflare.proxied=true \
  --set txtOwnerId=k8s-aws-se-cluster \
  --set "domainFilters[0]"=bunnydemo.com \
  oci://registry-1.docker.io/bitnamicharts/external-dns

You just need to:

• replace cloudflare.apiToken
• set the txtOwnerId to your cluster's name
• set domainFilters[0] to your actual domain

For the full list of parameters supported by the Helm Chart, consult its documentation.

📘

Bunnyshell will integrate ExternalDNS as a cluster add-on in the upcoming period.

SSL certificates

If you already have the SSL/TLS certificates for the custom domain used in Bunnyshell, you can save it in a K8s Secret in the environment namespace and reference it in the component host config

components:
    -
        kind: Service
        name: nginx
        dockerCompose:
            image: 'nginx:latest'
            ports:
                - '80:80'
        hosts:
            -
                hostname: 'nginx.myowndomain.com'
                path: /
                servicePort: 80
                selfManagedDns: true
                k8s:
                    ingress:
                        tlsSecretName: my-custom-domain-cert

This will translate into tls config on the Ingress k8s resource

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
    name: nginx-0
...
rules:
    - host: nginx.myowndomain.com
      http:
      ...
tls:
    - hosts:
          - nginx.myowndomain.com
      secretName: my-custom-domain-cert

Or you can use cert-manager to automatically create certificates for the domains. In this case the name of the TLS Secret is not so important anymore, you can even leave it empty string and Bunnyshell will choose a name for it and cert-manager will create it with the correct value, but it's important to add also an annotation on the host to specify the CertificateIssuer to be used

components:
    -
        kind: Service
        name: nginx
        dockerCompose:
            image: 'nginx:latest'
            ports:
                - '80:80'
        hosts:
            -
                hostname: 'nginx.myowndomain.com'
                path: /
                servicePort: 80
                selfManagedDns: true
                k8s:
                    ingress:
                        tlsSecretName: ''
                        annotations:
                            cert-manager.io/cluster-issuer: letsencrypt-prod

See below how to install cert-manager and how to create a ClusterIssuer.

Installing cert-manager in your cluster

Following the installation steps from cert-manager the easiest way to install is using kubectl and the Helm Chart . Check the Helm chart latest version and change it in the example commands below, which are using the v1.16.1 version.

Before installing the chart, you must first install the cert-manager CustomResourceDefinition resources. This is performed in a separate step to allow you to easily uninstall and reinstall cert-manager without deleting your installed custom resources.

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.crds.yaml

Then install the chart with the release name cert-manager in the cert-manager namespace:

helm repo add jetstack https://charts.jetstack.io --force-update

helm install cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.16.1 \
  jetstack/cert-manager

Then create a ClusterIssuer with one of existing cert-manager issuer integrations . The following example uses the Let's Encrypt issuer.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
    name: letsencrypt-prod
spec:
    acme:
        # The ACME server URL
        server: https://acme-v02.api.letsencrypt.org/directory
        # Email address used for ACME registration
        email: <your_email_address_here>
        # Name of a secret used to store the ACME account private key
        privateKeySecretRef:
          name: letsencrypt-prod
        # Enable the HTTP-01 challenge provider
        solvers:
            - http01:
                  ingress:
                      class: bns-nginx

The name of the ClusterIssuer above is the one you should add on host.k8s.ingress.annotations.cert-manager.io/cluster-issuer.

Also the example above uses the IngressClass bns-nginx which is the one created by the Bunnyshell Ingress Controller Add-on. If you are not using this add-on change the class name with the one you are using.

🚧

Caution

The are some rate-limits for Let's Encrypt (and for the other issuers too), so in Bunnyshell this configuration is suitable for production environments, or long lived stages (e.g. one stage per dev, or a sales demo stage, ...), but definitely NOT for ephemeral environments, where you will quickly reach the weekly certificates limit, or even the requests limit. For ephemeral environments the recommended setup is to use a wildcard certificate *.mydomain.com and most probably the DNS challenge in the ClusterIssuer (instead of the http01 one)

Also because of the rate-limits above, you should use the interpolation with care for the host.k8s.ingress.tlsSecretName, if it's an always changing value, the Secret will be recreated by cert-manager, but this means a request, and there is also a Duplicate Certificate limit in Let's Encrypt.

Also Let's Encrypt has a limit of 10 labels per domain, including the TLD, so make sure your custom domains comply (e.g. label8.label7.label6.label5.label4.label3.label2.label1.domain.com)