Service deployment with Docker

The goal of this document is to describe the recommended way of deploying a service to lund.fsfeurope.org with Docker. With this procedure, the service is automatically deployed and updated on lund. For this documentation we assume that the service is simple enough to fit in one Docker container and that the service is accessible through HTTP.

General Idea

We have one server that hosts our containers. We have a Continuous Integration system, Drone, that builds images and run containers locally on the lund.fsfeurope.org server. Drone listens for commits in a git server, each commit triggers the creation of a Ansible containers that will build the container and run it. We have a HTTP reverse proxy container that watches container creation and create virtual host dynamically to route the HTTP traffic to the containers.

Here are the necessary steps to use the CI to deploy a service:

1. Build a Dockerfile recipe that builds a Docker image for the service.

2. Create a Ansible playbook that will build and run the container.

3. Create a Drone file that will run the Ansible playbook so the Docker container is deployed on lund.

4. Configure Drone to listen for git commits on the git server.

5.Add a SSH_KEY secret on Drone so it can access to the server.

6. Commit

Dockerfile

The Dockerfile that builds the service should have 4 main items:

1. Selection of the base image with a FROM statement.

2. List of the ports that will be exposed so Docker knows what ports are used in the container. Use the EXPOSE statement

3. The actual build of the service. Install the service dependencies and the service itself

4. A CMD statement so the service is run automatically when the container starts.

An example Dockerfile is provided below:

# Base image selection
FROM debian:9

# List of ports that will be exposed to the outside world
EXPOSE 3000

# Installing the services and dependencies
ENV DEBIAN_FRONTEND=noninteractive LANG=en_US.UTF-8 LC_ALL=C.UTF-8 LANGUAGE=en_US.UTF-8
RUN [ "apt-get", "-q", "update" ]
RUN [ "apt-get", "-qy", "--force-yes", "upgrade" ]
RUN [ "apt-get", "-qy", "--force-yes", "dist-upgrade" ]
RUN [ "apt-get", "install", "--no-install-recommends", "-qy", "--force-yes", \
      "ca-certificates", \
      "perl", \
      "git", \
      "uuid-dev", \
      "sqlite3", \
      "build-essential", \
      "cpanminus" ]
RUN [ "apt-get", "clean" ]
RUN [ "rm", "-rf", "/var/lib/apt/lists/*", "/tmp/*", "/var/tmp/*" ]
RUN ["cpanm", "--notest", "DBI", "DBD::SQLite", "Mojolicious", "UUID" ]
RUN adduser --quiet --disabled-password --shell /bin/bash --home /home/perlscript --gecos "User" perlscript
USER perlscript
WORKDIR /home/perlscript
RUN git clone https://git.fsfe.org/vincent/mailman-reconfirm.git
WORKDIR /home/perlscript/mailman-reconfirm

# CMD to run the service on container start
CMD perl reconfirm.pl daemon -l http://0.0.0.0:3000

Here is the Dockerfile reference.

Ansible playbook

Next, we need to create an Ansible playbook. The playbook is responsible for building the image with the Dockerfile and running the container.

To run the container, there are mandatory options:

The docker volumes. This specifies the parts of the container that will be
- shared as mount points with the docker host. Usually this is for the container's data directories.
The environment variables. The environment variables are important because
- they are used by the reverse proxy container to configure the subdomain used in the virtual host created automatically.
The exposed ports. This is used by the reverse proxy container to route the
- traffic to the correct ports inside the service's container.

An example is provided below:

---
- hosts: lund.fsfeurope.org
  remote_user: root

  tasks:
    - name: build image mailman reconfirm
      command: docker build -t mailman-reconfirm https://git.fsfe.org/vincent/mailman-reconfirm.git

    - name: run the container
      docker_container:
        name: mailman-reconfirm
        image: mailman-reconfirm
        state: started
        restart: yes
        exposed_ports:
          - 3000
        env:
          VIRTUAL_HOST: mailman-reconfirm.fsfe.org
          LETSENCRYPT_HOST: mailman-reconfirm.fsfe.org
          LETSENCRYPt_emaIL: vincent@fsfe.org
        volumes:
            - "/srv/mailman-reconfirm/data:/home/perlscript/mailman-reconfirm/data/:rw"

Here is the Ansible docker reference.

Drone configuration

Drone is the Continuous Integration and Continuous Delivery software we use. We need to tell Drone to build the image and run the container.

Add this file to your project and call it .drone.yml:

pipeline:
  deploy:
    image: williamyeh/ansible:debian8
    secrets: [ ssh_key ]
    commands:
      - mkdir /root/.ssh && echo "$SSH_KEY" > /root/.ssh/id_rsa && chmod 0600 /root/.ssh/id_rsa
      - ssh-keyscan -H lund.fsfeurope.org >> ~/.ssh/known_hosts
      - ansible-playbook playbook.yml -i lund.fsfeurope.org,
    when:
       branch: master

This will create a Ansible Docker container that will build connect locally to lund.fsfeurope.org with SSH and run the Ansible playbook.

Next, got to https://drone.fsfe.org/ and use your FSFE login. Click on the hamburger menu in the upper right corner and select "Repositories". Then activate the CI for your repository.

One last think is to add the lund server ssh key secret. Connect to lund as root and run the following commands:

cd ~
. drone.cli.sh
./drone secret add --image=williamyeh/ansible:debian8 --repository <repository path> --name SSH_KEY --value @/root/.ssh/id_rsa

Here <repository path> must be something like <your git username>/<your repository name>.

If that does not work, please make sure the DRONE_* env variables are set correctly. You do so by typing env | grep DRONE_. Please make sure that the DRONE_TOKEN belongs to a user that is owner of the repository.

Commit

You can do a test commit to see if it deploys the container correctly.