Ingress for Docker Compose

📘
By default Ingress created in KubernetesManifest or Helm components will not be interfered with. Bunnyshell handles only the Ingresses with ingressClassName: bns-nginx.
This article refers mostly to Docker-compose components:Application, Database, Service and StaticApplication.

If you want to learn why Docker Compose is unsuitable for production and how Bunnyshell can help you transition from docker-compose to Kubernetes, read this article.

Introduction

In Kubernetes, Ingress is the resources that can expose in internet an application deployed in the cluster. If you have a web application in a Pod in cluster, you need a Service that receives cluster traffic and routes it to the Pod on a specific port, and an Ingress that receives internet traffic and routes it in cluster to a specific Service and port.

Bunnyshell automatically creates for the Pod a Service resource, if the component exposes at least one port (component.dockerCompose.ports) and for the Service an Ingress resource foreach host of the component (component.hosts). Each host need to specify the hostname, path and servicePort. Optionally you can add k8s with extra configs for Ingress, such as annotations and TLS secrets, but pay attention that if you change the k8s.ingress.className, then Bunnyshell will skip creating its DNS records. Read more about URLs in Bunnyshell .

Kubernetes Cluster Requirements

To be able to receive internet traffic and route it to different Ingress resources, the cluster needs to have installed an Ingress Controller . The best way is to have also an IngressClass, to group the Ingress resources, and instruct the Ingress Controller to manage only the Ingresses from a specific IngressClass. This way you can have simultaneously in cluster groups of Ingresses, with different common configurations, and they can even be managed by different Ingress Controllers.

Bunnyshell needs to have its own, dedicated IngressClass named bns-nginx.

If you already have an Ingress Controller installed in your cluster and want it to manage also the Bunnyshell's IngressClass, then you can just create the IngressClass and instruct your IngressController to manage it.

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: bns-nginx
spec:
  controller: example.com/ingress-controller # replace with your existing ingress controller
  #parameters:                   # optionally other parameters 
  #  apiGroup: k8s.example.com
  #  kind: IngressParameters
  #  name: external-lb

Ingress Controller Requirements

Bunnyshell can work with many ingress controllers, but it has some expectations from it.

It must ensure Ingress host+path uniqueness at cluster level. If you have in cluster 2 Ingresses which both want to serve the same hostname and path, then the behaviour is unpredictable. Which one will win? What if you have cloned the prod environment, with the prod URLs, to make a stage, and now the URLs are served by the stage environment? It's better the ingress controller to reject the duplicate ingress. Bunnyshell does not make cross environments hosts validation, especially if they are selfManagedDns, so this must be an ingress controller feature.
It must accept creating each Ingress independently, without another central/master resource. Bunnyshell deploys each component individually, and it's very common that many components uses URLs under the same hostname, with different paths. It should be able to just create the required Ingress without having to also create/maintain a central resource (for example one per hostname), where you have to register each individual Ingress under that hostname. This is complicated and prone to race condition errors.
It must expose the LB IP on the Ingress object. At deploy, Bunnyshell inspects bns-nginx Ingresses and wait for them to have allocated IPs, and optionally it creates a DNS record pointing to that IP. If at least one Ingress does not receive an IP the deploy fails.
It must accept SSL traffic. Bunnyshell uses Cloudflare as DNS provider for the bunnyenv.com domains, and the traffic between Cloudflare and cluster is secured too. The cluster may present a self signed certificate, configured once per controller, as Bunnyshell does not set tlsSecret on each Ingress resource.
It must use Ingress resources. Bunnyshell creates Ingress resources, watches them to receive IP address, cleans them up when no longer needed. Ingress controllers typically leverages their full capabilities through custom CRDs, but Bunnyshell cannot implement all of them, so it sticks to Ingress resources, so the ingress controller should support the features presented here through the Ingress resources.

📘
Ingress Controller Add-on
Bunnyshell can help you install the Ingress Nginx Controller through the Ingress Controller Add-on, with a universal recipe, which works on any cluster.
Just go to Bunnyshell > Integrations > Clusters then Cluster > Add-ons > Add Add-on > Ingress Controller.
⚠️ Ingress Nginx is a retired Kubernetes project no longer maintained. This means it will not receive any more features and security updates. For this reason we recommend using another ingress controller.

📘
NGINX Ingress Add-on
Bunnyshell can help you install the NGINX Ingress Controller through the NGINX Ingress Add-on, with a universal recipe, which works on any cluster.
Just go to Bunnyshell > Integrations > Clusters then Cluster > Add-ons > Add Add-on > NGINX Ingress.

Use the Helm charts to install Ingress Nginx Controller

If you don't have an Ingress Controller installed in cluster, then you can install the Ingress Nginx Controller which will also create the IngressClass.

⚠️ Ingress Nginx Controller is a retired Kubernetes project no longer maintained. This means it will not receive any more features and security updates. For this reason we recommend using another ingress controller.

To manually install the Ingress Controller, we recommend using the Helm below.

The following example is for the chart version 4.12.1.

Write the following values.yaml file:

controller:
  ingressClass: bns-nginx
  ingressClassResource:
    name: bns-nginx
    controllerValue: "k8s.io/ingress-nginx"
  service:
    type: LoadBalancer
    externalTrafficPolicy: Local
  config:
    proxy-real-ip-cidr: "173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/13,104.24.0.0/14,172.64.0.0/13,131.0.72.0/22,2400:cb00::/32,2606:4700::/32,2803:f800::/32,2405:b500::/32,2405:8100::/32,2a06:98c0::/29,2c0f:f248::/32" # see https://www.cloudflare.com/ips/
    use-forwarded-headers: "true"
    proxy-buffer-size: "128k"
    proxy-buffers-number: 4
    proxy-body-size: "250m"
    strict-validate-path-type: "false"
  resources:
    requests:
      cpu: "100m"
      memory: "300Mi"
    limits:
      cpu: "500m"
      memory: "500Mi"
  admissionWebhooks:
    createSecretJob:
      resources:
        requests:
          cpu: "100m"
          memory: "20Mi"
        limits:
          cpu: "100m"
          memory: "20Mi"
    patchWebhookJob:
      resources:
        requests:
          cpu: "100m"
          memory: "20Mi"
        limits:
          cpu: "100m"
          memory: "20Mi"

defaultBackend:
  enabled: "true"
  image:
    registry: registry.k8s.io
    image: ingress-nginx/nginx-errors
    tag: v20220916-gd32f8c343@sha256:09c421ac743bace19ab77979b82186941c5125c95e62cdb40bdf41293b5c275c
  resources:
    limits:
      cpu: "100m"
      memory: "128Mi"
    requests:
      cpu: "100m"
      memory: "128Mi"

Depending on the cloud provider, you may have to add in the values.yaml above some extra values:

controller:
  service:
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"

# nothing extra to add for Microsoft Azure

# nothing extra to add for Google Cloud Platform

controller:
  service:
    annotations:
      service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "false"
  config:
     use-proxy-protocol: "false",
     proxy-real-ip-cidr: "10.0.0.0/8"
  admissionWebhooks:
     timeoutSeconds: 29

controller:
  service:
    annotations:
      loadbalancer.openstack.org/keep-floatingip: "true"
      service.beta.kubernetes.io/openstack-internal-load-balancer: "false"
  config:
    use-proxy-protocol: "false"
    proxy-real-ip-cidr: "10.0.0.0/8"

controller:
  service:
    annotations:
      service.beta.kubernetes.io/scw-loadbalancer-use-hostname: "true"
      service.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v1: "false"
      service.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v2: "false"
  config:
    use-proxy-protocol: "false"
    proxy-real-ip-cidr: "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10"

# Check the capabilities of your cloud provider
# if it can automatically provision a LoadBalancer and an IP for the Kubernetes cluster.
# If it cannot, check here more considerations https://github.com/kubernetes/ingress-nginx/blob/main/docs/deploy/baremetal.md
# But probably you will have to add:

controller:
  service:
    type: NodePort
  publishService:
    enabled: "false"

Then install the chart:

helm upgrade --install \
   --repo https://kubernetes.github.io/ingress-nginx \
   ingress-nginx ingress-nginx --version 4.12.1 \
   --namespace=ingress-nginx --create-namespace \
   -f /values.yaml

Use the Helm charts to install NGINX Ingress Controller

If you don't have an Ingress Controller installed in cluster, then you can install the NGINX Ingress Controller which will also create the IngressClass.

To manually install the Ingress Controller, we recommend using the Helm below.

The following example is for the chart version 2.5.2.

Write the following values.yaml file:

controller:
    service:
        type: LoadBalancer
        externalTrafficPolicy: Local
    config:
        entries: 
            # see https://www.cloudflare.com/ips/
            set-real-ip-from: '173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/12,172.64.0.0/13,131.0.72.0/22,2400:cb00::/32,2606:4700::/32,2803:f800::/32,2405:b500::/32,2405:8100::/32,2a06:98c0::/29,2c0f:f248::/32' 
            real-ip-header: X-Forwarded-For
            real-ip-recursive: 'True'
            proxy-buffer-size: 128k
            proxy-buffers: '4 128k'
            client-max-body-size: 250m
            ssl-redirect: 'False'
            redirect-to-https: 'False'
            port-in-redirect: 'False'
    defaultTLS:
        # obtain them with:
        # openssl req -x509 \
        #   -newkey rsa:2048 \
        #   -sha256 \
        #   -days 365 \
        #   -nodes \
        #   -keyout origin.key \
        #   -out origin.crt \
        #   -subj "/CN=NGINX self-signed"
        cert: LS0tLS1CRUdJTi...
        key: LS0tLS1CRUdJTi...
    wildcardTLS:
        # the same ones as above
        cert: LS0tLS1CRUdJTi...
        key: LS0tLS1CRUdJTi...
    resources:
        requests:
            cpu: 100m
            memory: 300Mi
        limits:
            cpu: 500m
            memory: 500Mi
    ingressClass:
        name: bns-nginx
        create: true
        # if you want this class to be also the default IngressClass in cluster
        # setAsDefaultIngress: true
    reportIngressStatus:
        enable: true

Depending on the cloud provider, you may have to add in the values.yaml above some extra values:

controller:
  service:
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"

# nothing extra to add for Microsoft Azure

# nothing extra to add for Google Cloud Platform

controller:
  service:
    annotations:
      service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "false"
  config:
    entries:
      proxy-protocol: "False",

controller:
  service:
    annotations:
      loadbalancer.openstack.org/keep-floatingip: "true"
      service.beta.kubernetes.io/openstack-internal-load-balancer: "false"
  config:
    entries:
      proxy-protocol: "False"
      set-real-ip-from: "10.0.0.0/8"

controller:
  service:
    annotations:
      service.beta.kubernetes.io/scw-loadbalancer-use-hostname: "true"
      service.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v1: "false"
      service.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v2: "false"
  config:
    entries:
      proxy-protocol: "False"
      set-real-ip-from: "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10"

# Check the capabilities of your cloud provider
# if it can automatically provision a LoadBalancer and an IP for the Kubernetes cluster.
# See all values at https://docs.nginx.com/nginx-ingress-controller/lts/install/helm/parameters/
# But probably you will have to add:

controller:
  service:
    type: LoadBalancer
    externalTrafficPolicy: Local
    # if you need to specify the LB IP
    # externalIPs: [1.2.3.4]

Then install the chart:

helm upgrade --install \
   --repo https://helm.nginx.com/stable \
   nginx-ingress nginx-ingress --version 2.5.2 \
   --namespace=nginx-ingress --create-namespace \
   -f /values.yaml