CronJobs for Docker Compose

Suggest Edits

Oftentimes, your applications have periodic tasks which need to be ran in order to do some async / batch processing. Usually, these are modelled as Cron Jobs.

Bunnyshell supports running multiple CronJobs for each of your Applications, Services and Databases.


Defining a CronJob

CronJobs are defined in Bunnyshell by configuring a cronJobs attribute for any Docker Compose component.

The best way to demonstrate how CronJobs can be configured, we'll take some examples. For simplicity's sake, git* attributes, as well as the hosts attribute were omitted, so component configuration is kept to a minimum.

Let's assume we have an Application named webapp and we add some CronJobs to it.

Let's also say you have a bin/console send:emails command which can send emails, and you want to trigger it to run every 5 minutes.


Basic example

The most simple example for adding a CronJob looks like this, and involves only adding the cronJobs attribute on the Component, only with name, schedule and command attributes.

components:
  - 
    kind: Application
    name: webapp
    ...
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
    cronJobs:
      - name: send-emails
        schedule: '*/5 * * * *'
        command:
          - /bin/sh
          - '-c'
          - 'bin/console send:emails'

Volumes

If your Application also has Volumes attached, you can choose whether or not these will be attached to the Pod running the CronJob as well.

Two volumes are mounted in the main component, webapp and below some CronJob examples are provided, which demonstrate how you can:

  • omit the volumes altogether
  • include only some volumes, or
  • include all volumes (default behaviour)

using the volumes attribute on the CronJob.

components:
  - 
    kind: Application
    name: webapp
    ...
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
    volumes:
      -
        name: media
        mount: /var/media
      -
        name: artefacts
        mount: /var/artefacts
    cronJobs:
      - name: send-emails-no-volumes
        schedule: '*/5 * * * *'
        command:
          - /bin/sh
          - '-c'
          - 'bin/console send:emails'
        volumes: false # no volume included

      - name: send-emails-all-volumes
        schedule: '*/5 * * * *'
        command:
          - /bin/sh
          - '-c'
          - 'bin/console send:emails'
        volumes: true # all volumes included; default value, can be ommitted

      - name: send-emails-some-volumes
        schedule: '*/5 * * * *'
        command:
          - /bin/sh
          - '-c'
          - 'bin/console send:emails'
        containers:
          - nginx
        volumes:
          - artefacts # only "artefacts" volume included, "media" not attached
volumes:
  -
    name: artefacts
    type: network
    size: 1Gi
  -
    name: media
    type: network
    size: 1Gi

📘

Note

Because a volume can be simultaneously mounted in a running Pod of a Deployment and in a Pod of a CronJob, and these can be scheduled on different cluster nodes, we limit the usage of cronJobs only with volumes of type network


Kubernetes Resources

Usually, running separate dedicated commands require significantly less resources to run than the container running the web app - so, in order to be mindful of utilizing resources, these can be specified on a per-CronJob basis, using the resources attribute. Its syntax is Docker compose compatible, and the same as when defining the main component resources.

components:
  - 
    kind: Application
    name: webapp
    ...
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
    cronJobs:
      - name: send-emails-low-resources
        schedule: '*/5 * * * *'
        command:
          - /bin/sh
          - '-c'
          - 'bin/console send:emails'
        resources:
          limits:
            cpus: '2'
            memory: '256M'
          reservations:
            cpus: '0.50'
            memory: '128M'

Execution Timeout

Sometimes it's important to make sure the cronJob does not run more than a maximum time, or if the Pod it's not even scheduled in that time, to cancel the run entirely. This can be accomplished by setting the executionTimeout seconds on the cronJob, by default there is no timeout.

Sidecar containers

You may have the need to initialize your application by performing some tasks before your application (container) actually starts. For this, you would use InitContainers.

Other times, besides your main application, you may have additional containers which need to run alongside your main application's container - take for example, sending logs and the other for pushing metrics. These would be implemented using SidecarContainers.

Let's take an example which provides one InitContainer for building the application (eg: running npm run build) and two SidecarContainers, one for logging and the other for pushing metrics.

The InitContainer and SidecarContainer components are defined first; these are similar to Application components, except they are not deployed directly, but their configuration is used only when attached to another Application (or Service / Database) Component, using the pod attribute.

Using the containers attribute, you can control which containers will be included in the CronJob's Pod, but please keep in mind that the bare minimum, the main container must be included, in this case webapp - it bears the name of the components itself.

components:
  -
    kind: InitContainer
    name: build-app-init
    ...
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
  -
    kind: SidecarContainer
    name: logging-sidecar
    ...
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
  -
    kind: SidecarContainer
    name: metrics-sidecar
    ...
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
  - 
    kind: Application
    name: webapp
    ...
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
    pod:
    	init_containers:
      	-
        	from: build-app-init
          name: build-app
      sidecar_containers:
        -
          from: logging-sidecar
          name: logging
        -
          from: metrics-sidecar
          name: metrics
    cronJobs:
      - name: send-emails-no-containers
        schedule: '*/5 * * * *'
        command:
          - /bin/sh
          - '-c'
          - 'bin/console send:emails'
        containers:
          - webapp

      - name: send-emails-all-containers
        schedule: '*/5 * * * *'
        command:
          - /bin/sh
          - '-c'
          - 'bin/console send:emails'
        # not specifying the "containers" attribute will include all of them

      - name: send-emails-some-containers
        schedule: '*/5 * * * *'
        command:
          - /bin/sh
          - '-c'
          - 'bin/console send:emails'
        containers:
          - webapp
          - build
          - logging

Rich configuration example

A very complex configuration example is presented below, combining all options into a single example.

components:
  -
    kind: InitContainer
    name: build-app-init
    gitRepo: git@...
    gitBranch: master
    gitApplicationPath: /
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
  -
    kind: SidecarContainer
    name: logging-sidecar
    gitRepo: git@...
    gitBranch: master
    gitApplicationPath: /
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
  -
    kind: SidecarContainer
    name: metrics-sidecar
    gitRepo: git@...
    gitBranch: master
    gitApplicationPath: /
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
  - 
    kind: Application
    name: webapp
    gitRepo: git@...
    gitBranch: master
    gitApplicationPath: /
    dockerCompose:
      build:
        context: .docker
        dockerfile: Dockerfile
    volumes:
      -
        name: media
        mount: /var/media
      -
        name: artefacts
        mount: /var/artefacts
    pod:
    	init_containers:
      	-
        	from: build-app-init
          name: build-app
      sidecar_containers:
        -
          from: logging-sidecar
          name: logging
        -
          from: metrics-sidecar
          name: metrics
    hosts:
      - hostname: 'app-{{ env.base_domain }}'
        path: /
        servicePort: 8080
    cronJobs:
      - name: webapp-rich
        schedule: '*/5 * * * *'
        command:
          - /bin/sh
          - '-c'
          - 'bin/console send:emails'
        containers:
          - webapp
          - build
          - logging
        volumes:
          - artefacts
        resources:
          limits:
            cpus: '2'
            memory: '10000M'
          reservations:
            cpus: '0.50'
            memory: '5000M'
        executionTimeout: 180

volumes:
  -
    name: artefacts
    type: network
    size: 1Gi
  -
    name: media
    type: network
    size: 1Gi

Updated 5 months ago