Volumes for Docker Compose

Suggest Edits

📘

PersistentVolumeClaims created in KubernetesManifest or Helm components will not be interfered with. They will work exactly as defined.

This article refers to Docker-compose components only: Application, Database, Service.

Introduction

Bunnyshell enables you to define persistent volumes and attach them to nodes/pods using the volumes key.

A PersistentVolume (PV) is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using a StorageClass.

It is a resource in the cluster just as how a node is a cluster resource. PVs are volume plugins like Volumes, but have a lifecycle independent of any individual Pod that uses the PV. Read more about Volumes on the official Kubernetes documentation platform.

Types of Bunnyshell volumes

There are two types of volumes users can define in Bunnyshell:

  • Disk volumes: persistent volumes with ReadWriteOnce access mode.
    Such volumes can be attached to a single Kubernetes node.
  • Network volumes: persistent volumes with ReadWriteMany access mode.
    Such volumes can be attached to multiple Kubernetes nodes.

Requirements

Disk and Network volumes

  • Are persistent
  • Can have configurable sizes and names
  • Support subpaths on mount

❗️

Warning

Volumes that are not claimed by an environment / component will be removed.

For all persistent volumes, the file mode will be 0777. File modes are set by init containers. Each pod has an init container that modifies permissions for all volumes.

📘

Before the pod container starts, the init container executes a chown 0777 on all those volumes.

 

Specifications

A volumes key may be added at both the environment level and the component level. Below we will list the volumes for each of the cases.

At an Environment level, Volumes are declared, while at the Component level, they are used (mounted).

 

Declaring volumes in Environments

At the Environment level, volumes are created for the Docker-compose Components (Application, Database, Service) to use.

  • name: the name of the volume. This field is mandatory;
  • size: the size of the volume. This field is mandatory;
  • type: the volume type mapped to Kubernetes cloud volume types. The following options are available:
    • disk: can be attached to a single Node;
    • network: can be attached to multiple Nodes.

📘

Important

Volumes are isolated at the environment level.

This means that if two bunnyshell.yaml configs declare volumes with the same name, Bunnyshell will add the unique ID of their respective environments to their name, making them unique.

 

Using volumes in Components

At the Component level, the volumes property will contain claims to volumes defined at the environment level. It contains the following properties:

  • name: the name of the volume claimed. This field is mandatory;
  • mount: the path where to mount the volume in the component. This field is mandatory;
  • subPath: Path of what to mount into the component.

Volumes will be mounted inside Components based on this spec.

 

Validations and constraints

Environment level

  • Volume names must be unique. Two volumes can not have the same name;
  • The name must consist of lower case alphanumeric characters. It can also contain hyphens -, but it must start and end with an alphanumeric character (e.g. 123-abc).
  • The volume type must be either disk, or network;
  • The volume size must be greater than zero (any float number is accepted);
  • The volume size syntax must be composed of two (case sensitive) parts : float and unit. Unit is a memory unit like Gi. 'KB', 'MB', 'GB', 'TB', 'b', 'Gi';
    An environment volume must be used by a container, otherwise Bunnyshell will show errors.

Component level

  • The name must belong to one of the volumes declared at the environment level, otherwise Bunnyshell will show errors;
  • The mount field must include a directory path where to mount the volume, in linux format (e.g. /var/mnt/vol1)
  • The name must consist of lower case alphanumeric characters. It can also contain hyphens -, but it must start and end with an alphanumeric character (e.g. 123-abc).
  • Two volumes declared at the component level can not be mounted in the same path. The mount path must be unique in list.

 

Docker-compose import

All volumes declared in docker-compose will be interpreted as persistent volumes and will have a default size of 1Gi after parsing. These volumes can be found at environment level (declared) or requested by Docker-compose components, pods, init containers or sidecar containers.

📘

The volumes attribute from docker-compose.yaml has a different definition compared to the volumes attribute from bunnyshell.yaml.

Here's an example of a volume declared in docker-compose.

services:
  nginx:
    image: nginx
    volumes:
      - database_volume

And this is how it will be transformed in bunnyshell.yaml.

kind: ...
...
components:
  - 
    name: nginx
    volumes:
      -
        name: database_volume
        mount: /var/db
volumes:
  -
    name: database_volume
    type: disk
    size: 1Gi

 

Conversion from Docker-compose - Considerations

  • Bunnyshell ignores volumes that are not used by any services (root directive identation 0 in docker-compose.yaml file).
  • Bunnyshell ignores bind and tmpfs Volumes, as well as any local bind that does not use a volume defined under the global key volumes.
  • Volume names will be random values when only the target is defined, as a string.
  • If the StorageClass was already added in a cluster that was previously connected to Bunnyshell, the user must wait until the cluster is verified to determine whether or not a needed storage class exists.
  • If a volume is attached to several separate services in docker-compose.yaml, or if it is mounted several times inside the same service, it will be migrated as a network volume.
  • The default size is 1Gi.
  • When adding an application that has a volume name already existing inside the environment, a random string will be added to the Volume name to make it unique.

 

Example Manifest

# all volumes are persistent
# if you don't need persistance, don't use a mount path
#     - the only thing that might not make sense here, is breaking out of the container resource pool
#     - I.E. backend php with 1GB disk space having an external mount of 5G
volumes:
  -
    type: network
    # means ReadWriteMany
    name: logs-all
    size: 10Gi
  -
    type: disk
    # means ReadWriteOnce
    # passes validation if used by only 1 ServiceComponent
    # passes validation if used by any Init/Sidecar
    name: data-mysql
    size: 3Gi
components:
  -
    name: mysql
    volumes:
      # now we have a persistent database
      # what we lack is a way to "kickoff" the database
      - name: data-mysql
        mountPath: /var/lib/mysql
    pod:
      init_container:
        # this init_container needs to:
        #   1. be able to init mysql data
        #   2. detect if init has already took place
        #   3. not force the volume to become a network volume
        - name: init-mysql-data
          mounts:
            - name: data-mysql
              mountPath: /var/lib/mysql
  -
    name: nginx
    volumes:
      - name: logs-all
        mount: /var/log/nginx
        subPath: /nginx
  -
    name: monitor
    volumes:
      - name: logs-all
        mount: /opt/logs
    pod:
      sidecar_container:
        - name: log-rotate
          volumes:
            - name: logs-all
              mountPath: /opt/logs

Kubernetes Cluster Requirements

In order to be able and create volumes (PersistentVolumeClaims) for Docker-compose components, Bunnyshell needs to have its own, dedicated StorageClasses created in the Kubernetes cluster.

📘

A StorageClass provides a way for administrators to describe the "classes" of storage they offer.
Different classes might map to quality-of-service levels, or to backup policies, or to arbitrary policies determined by the cluster administrators.
More information on the StorageClass concept is available on the official Kubernetes documentation.

Creating a Storage Class in Amazon Web Services

Creating a Storage Class in Microsoft Azure

Creating a Storage Class in Google Cloud

Creating a Storage Class in DigitalOcean

Creating a Storage Class in DT Cloud

Updated 5 months ago