Setting up Jenkins Docker to dynamically create Docker slaves (Docker-in-Docker)

Note: this is part 4 in a series of Docker Tutorials.

Problem

You want to set up your dockerized Jenkins server via Jenkins Configuration as Code (JCasC) such that it dynamically creates Docker slaves whenever it executes a job.

Solution

TLDR: Bitbucket code.

This can be achieved with the docker-plugin and the corresponding YAML code block in your JCasC configuration file.

Implementation

Continuing where we left off, we need to

add the docker-plugin to our plugins.txt file
extend our initial Dockerfile by installing the Docker Commandline Interface
bind-mount /var/run/docker.sock into our Jenkins Docker container so that we can make use of the Docker daemon running on the host machine
add the docker-plugin settings to our JCasC jenkins.yml configuration file

We will skip step 1 because it is straight forward.

Dockerfile

Step 2 leads to the following Dockerfile

FROM jenkins/jenkins:lts

USER root

ARG USER_GID
ARG USER_ID

# RUN groupmod -g ${USER_GROUP_ID} jenkins
RUN usermod -u ${USER_ID} -g ${USER_GID} jenkins \
    # chown ${REF} since we changed the UID
    && chown -R jenkins ${REF}

# install docker-cli and its dependencies
RUN apt update && apt install -y lsb-release \
    software-properties-common \
    apt-transport-https \
    && rm -rf /var/lib/apt/lists/*

RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add - \
    && add-apt-repository "deb [arch=amd64] \
    https://download.docker.com/linux/debian $(lsb_release -cs) stable" \
    && apt update && apt install -y docker-ce-cli \
    && rm -rf /var/lib/apt/lists/*

USER jenkins

# copy plugins.txt to the $REF/init.groovy.d directory
# that is already set up by the base jenkins image
COPY plugins.txt ${REF}/init.groovy.d/plugins.txt
# install all plugins listed up there
RUN install-plugins.sh < ${REF}/init.groovy.d/plugins.txt

WORKDIR $JENKINS_HOME

We just added the 2nd and 3rd RUN block, which are adaptions of the original Docker documentation. Also, note that we install only the command line tools, since we will bind-mount the host's docker daemon (often also referred to as docker-in-docker). However, beware that this also introduces some security issues. You might want to google up on it.

docker-compose.yml

Step 2 is accomplished by adding /var/run/docker.sock to the volumes block of our docker-compose.yml:

version: "3.7"

services:
  jenkins:
    build:
      context: .
      args:
        USER_GID: $USER_GID
        USER_ID: $USER_ID
    environment:
      CASC_JENKINS_CONFIG: $CASC_JENKINS_CONFIG
      JAVA_OPTS: -Djenkins.install.runSetupWizard=false
    ports:
      - 9080:8080
    volumes:
      - $HOME/data/jenkins:$JENKINS_HOME
      - ./bashrc:$JENKINS_HOME/.bashrc
      - ./jenkins.yml:$CASC_JENKINS_CONFIG
      - /var/run/docker.sock:/var/run/docker.sock

Note: depending on your hosts settings, you might have to create a docker group in your container, add the user jenkins to add and chgrp docker /var/run/docker.sock inside your container after the first boot-up. This could be done with a one-liner as root user as following:

# create docker group if necessary, add jenkins user
# and chgrp /var/run/docker.sock
getent group docker || groupadd docker && usermod -aG docker jenkins \
      && chgrp docker /var/run/docker.sock

Reminder: you can enter your Jenkins container as root user via docker exec -it -u root <container_name> bash.

jenkins.yml

Finally, we only have to set up the docker-plugin settings. We will make use of the light-weight jenkins/jnlp-slave:alpine Docker image from the official Jenkins registry. There is also other ones, e.g. jenkins/slave, or you could even specify your own image. Feel free to choose the image that suits you best:

clouds:
    - docker:
        dockerApi:
          dockerHost:
            uri: "unix://var/run/docker.sock"
        name: "docker"
        templates:
          - labelString: "deploy"
            name: "jnlp"
            dockerTemplateBase:
              image: "jenkins/jnlp-slave:alpine"
            pullStrategy: PULL_LATEST
            remoteFs: "/home/jenkins"
            connector: attach

The labelString corresponds to the label specified in your Jenkins pipelines. So our whole jenkins.yml looks now as following:

jenkins:
  systemMessage: "Jenkins configured automatically by Jenkins Configuration as Code plugin\n\n"
  authorizationStrategy:
    roleBased:
      roles:
        global:
          - name: "admin"
            description: "Jenkins administrators"
            permissions:
              - "Overall/Administer"
            assignments:
              - "admin"
          - name: "readonly"
            description: "Read-only users"
            permissions:
              - "Overall/Read"
              - "Job/Read"
            assignments:
              - "authenticated"
  securityRealm:
    local:
      allowsSignup: false
      users:
        - id: "admin"
          password: "admin"
  clouds:
    - docker:
        dockerApi:
          dockerHost:
            uri: "unix://var/run/docker.sock"
        name: "docker"
        templates:
          - labelString: "deploy"
            name: "jnlp"
            dockerTemplateBase:
              image: "jenkins/jnlp-slave:alpine"
            pullStrategy: PULL_LATEST
            remoteFs: "/home/jenkins"
            connector: attach
unclassified:
  location:
    url: "http://127.0.0.1:9080/"
  shell:
    shell: "/bin/bash"

Note: without connector: attach there will be a Configuration As Code.init java.lang.NullPointerException exception:

jenkins_1  | 2020-01-25 15:03:45.966+0000 [id=27]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed ConfigurationAsCode.init
jenkins_1  | java.lang.NullPointerException
jenkins_1  | 	at com.nirima.jenkins.plugins.docker.DockerTemplate.hashCode(DockerTemplate.java:698)

Weirdly enough, this setting is not displayed if you inspect the Configuration as Code in the Jenkins Backend via Manage Jenkins > Configuration as Code > View Configuration: http://localhost:9080/configuration-as-code/viewExport. So it seems you can not simply rely on setting up your Jenkins via the GUI first before, inspecting the JCasC settings and copy pasting them into your YAML file.

Result

Above configuration leads to the following Cloud settings in the Jenkins backend under Manage Jenkins → Configure System:

and the following Docker Agent templates

We can test our new setup by creating a simple Pipelinjob, like so:

#!/usr/bin/env groovy

pipeline{

    agent{
        label "deploy"
    }
    
    stages{
        stage('hello world'){
            steps{
                sh "echo hello world"
            }
        }
    }

}

Executing the job leads to the following output:

Simple hello-world pipline run on a docker slave

Getagged mit:
Docker Tutorial Jenkins-Docker-Tutorial