Kubernetes Cluster Changes

When connecting a Kubernetes cluster, Bunnyshell applies the following changes to the cluster:

add the access control resources
install an Ingress Controller.

The changes are documented in detail below.

ClusterRoles:

bunnyshell:events-read-access - required to read events from cluster. The events are displayed in the real-time logs section of the Bunnyshell UI.
bunnyshell:read-access - required to read real-time logs from pods/services. The logs messages are shown in the real-time logs section of the Bunnyshell UI.
bunnyshell:remote-development-v2 - required to enable remote development.

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: bunnyshell:events-read-access
rules:
  - verbs:
      - get
      - list
      - watch
    apiGroups:
      - ''
    resources:
      - events
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: bunnyshell:read-access
rules:
  - verbs:
      - get
      - list
      - watch
    apiGroups:
      - ''
    resources:
      - pods
      - pods/attach
      - pods/exec
      - pods/log
      - pods/status
      - pods/portforward
      - services
      - configmaps
      - serviceaccounts
      - secrets
      - events
      - replicationcontrollers
      - persistentvolumeclaims
      - namespaces
      - namespaces/status
      - resourcequotas
      - resourcequotas/status
  - verbs:
      - get
      - list
      - watch
    apiGroups:
      - apps
    resources:
      - '*'
  - verbs:
      - get
      - list
      - watch
    apiGroups:
      - batch
    resources:
      - jobs
      - cronjobs
  - verbs:
      - get
      - list
      - watch
    apiGroups:
      - extensions
    resources:
      - '*'
  - verbs:
      - get
      - list
      - watch
    apiGroups:
      - metrics.k8s.io
    resources:
      - pods
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: bunnyshell:remote-development-v2
rules:
  - verbs:
      - create
      - patch
      - delete
    apiGroups:
      - ''
    resources:
      - pods/attach
      - pods/exec
      - pods/log
      - pods/status
      - pods/portforward
      - persistentvolumeclaims
      - secrets
  - verbs:
      - patch
      - update
    apiGroups:
      - apps
    resources:
      - deployments
      - statefulsets
      - daemonsets

Ingress Controller (Nginx)

Bunnyshell will install ingress-nginx controller and provision the bns-nginx IngressClass to be used by Ingresses configured for environment components.

It uses the ingress-nginx Helm chart, version 4.10.0, with the following chart configuration:

controller:
  admissionWebhooks:
    createSecretJob:
      resources:
        limits:
          cpu: 100m
          memory: 20Mi
        requests:
          cpu: 100m
          memory: 20Mi
    patchWebhookJob:
      resources:
        limits:
          cpu: 100m
          memory: 20Mi
        requests:
          cpu: 100m
          memory: 20Mi
  config:
    custom-http-errors: 502,503
    proxy-body-size: 250m
    proxy-buffer-size: 128k
    proxy-buffers-number: 4
    proxy-real-ip-cidr: 173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/12,172.64.0.0/13,131.0.72.0/22,2400:cb00::/32,2606:4700::/32,2803:f800::/32,2405:b500::/32,2405:8100::/32,2a06:98c0::/29,2c0f:f248::/32
    use-forwarded-headers: "true"
  ingressClass: bns-nginx
  ingressClassResource:
    controllerValue: k8s.io/bns-ingress-nginx
    name: bns-nginx
  publishService:
    enabled: "false"
  resources:
    limits:
      cpu: 500m
      memory: 500Mi
    requests:
      cpu: 100m
      memory: 300Mi
  service:
    type: NodePort
defaultBackend:
  enabled: "true"
  extraVolumeMounts:
  - mountPath: /www
    name: custom-error-pages
  extraVolumes:
  - name: custom-error-pages
    secret:
      items:
      - key: 502.html
        path: 502.html
      - key: 503.html
        path: 503.html
      secretName: custom-error-pages
  image:
    image: ingress-nginx/nginx-errors
    registry: registry.k8s.io
    tag: v20220916-gd32f8c343@sha256:09c421ac743bace19ab77979b82186941c5125c95e62cdb40bdf41293b5c275c
  resources:
    limits:
      cpu: 100m
      memory: 128Mi
    requests:
      cpu: 100m
      memory: 128Mi

Some values in the configuration can vary with the cluster provider.

The controller.config.proxy-real-ip-cidr are the Cloudflare IPs.