One of the primary goals of the JBoss product team is to ship a product that we're confident can be effectively supported for many years.  When a bug is discovered we need to be able to quickly reproduce it, create a fix, and then build and ship a new binary package which contains that fix.  This means that when a bug is reported, we need to not only be able to quickly find the relevant source code, we also need information related to the environment and any build commands/parameters needed to build and run that project.


We're always looking for ways to improve our build environment, which is currently based on the Fedora build system called KojiLinux Containers in general, and specifically Docker seem to be gaining popularity, and they have the potential to provide a great environment for reproducible builds.  Why are linux containers potentially great for creating reproducible builds?  For a few reasons:

  1. Build system images can be configured as needed (Java, Maven, gcc, etc)
  2. Each image provides a clean environment in which a build can be run.
  3. They are relatively lightweight (compared to a full VM) and can be started up quickly
  4. They can be saved/tagged to provide a record of the resulting build output


So, enough of the background information, let's see how to get this set up so you can start experimenting for your own use cases.


Configuring the Docker Host


The first step is to set up the server which is going to be hosting the docker containers.  This could be your local workstation, or you can set it up on separate server.  For this example we'll be using Fedora 20 as the Docker Host operating system, but you can easily use Ubuntu or another Linux distro.


Note: several of the examples use "sudo" to execute privileged commands.  If you are using Fedora, and don't want to use sudo, you may need to add yourself to the "wheel" group.

$ usermod -G -a wheel <username>


The Fedora package is called "docker-io" to avoid conflicts with another package called "docker":


$ sudo yum -y install docker-io


If you run into problems during this install, you can take a look at the more detailed instructions on the Docker Web Site.


Next, we need to configure the docker host to accept incoming connections over TCP (the default is a local unix socket) so that Jenkins can communicate with docker.  On Fedora, the easiest way to configure this is by editing the systemd startup file for docker located in /usr/lib/systemd/system/docker.service

On the line that starts with "ExecStart=", add the parameter " -H tcp://".  This tells docker to listen for incoming commands on port 4243 on the local system.  After modification, the line should look something like this:


ExecStart=/usr/bin/docker -d --selinux-enabled -H fd:// -H tcp://


Next, you can use systemd to start the docker daemon.


$ sudo systemctl start docker


You may also want to add your user account to the "docker" group so that you can run docker without sudo access.


$ usermod -G -a docker <username>


Creating the Jenkins Slave Docker Image


The next step is to create a docker image which meets the basic requirements to be a Jenkins slave.  A simple Jenkins slave requires sshd, Java, and a user account for Jenkins to connect.  There are plenty of existing images which meet these requirements, but for completeness, we'll build our own from scratch.


First, create a Dockerfiles directory with a subdirectory called fedora-jenkins-slave.


mkdir -p Dockerfiles/fedora-jenkins-slave


Next, we need to create two files.  One is the Dockerfile, which contains the instructions for building a Docker image.  Then, we'll need a simple supervisord.conf file to manage sshd startup when we run the image (docker images are often intended to run a single process/service and therefore do not contain process management configuration such as systemd or upstart)


The example dockerfile based on the default fedora image installing sshd, Java, and a "jenkins" user.


FROM fedora:latest

RUN yum -y update
RUN yum -y install openssh-server
RUN yum -y install supervisor
RUN yum -y install java-1.7.0-openjdk

RUN echo "root:password" | chpasswd
RUN useradd jenkins
RUN echo "jenkins:jenkins" | chpasswd

RUN mkdir -p /var/run/sshd
RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N ''
RUN sed -i 's|session    required|session    optional|g' /etc/pam.d/sshd

RUN mkdir -p /var/run/supervisord
ADD supervisord.conf /etc/supervisord.conf

CMD ["/usr/bin/supervisord"]


Most of the Dockerfile should be pretty self-explanatory.  The "FROM" line says that we want to start with the latest base Fedora image.  The "RUN" lines run a few commands inside the image to get sshd, java, and supervisor installed and configured.  And the "EXPOSE" line tells docker that port number 22 should be exposed to incoming connections.

A Complete Reference to the Dockerfile is available on the Docker site.

Next, create a file called "supervisord.conf" in the same directory as the Dockerfile.  This file will be copied into the docker image during the build.



command=/usr/sbin/sshd -D


This file just tells supervisord to start up sshd whenever the image is started.


Now, build the image and give it a name such as fedora-jenkins-slave.


$ docker build -t my-fedora-jenkins-slave .

This may take a few minutes, so you'll have to be patient


Now we're ready to run the image and make sure it's working correctly.


$ docker run -t -i my-fedora-jenkins-slave


You can see that the docker container is running using the "docker ps" command.


$ docker ps


This will give you a list of the currently running docker containers, including a unique identifier which will be needed for the next step.

CONTAINER ID    IMAGE                                     COMMAND               CREATED                STATUS               PORTS             NAMES

68f1ed77d3e9      my-fedora-jenkins-slave:latest     /usr/bin/supervisord   About a minute ago   Up 59 seconds       22/tcp              stupefied_bell

Next, use the container ID to get more information about the running container.


$ docker inspect 68f1ed77d3e9
$ docker inspect 68f1ed77d3e9 | grep IP

The second command shows the virtual IP address of the running container which we can use to test the ssh connection.  If everything is working, you should be able to log into the container using the username and password "jenkins".


$ ssh jenkins@


If the login is successful, you can also check that the correct version of Java is installed.

$ java -version

Congratulations, your Docker cloud is now ready!  The last step is to tell Jenkins to start using it.


Configuring Jenkins

The final part of this process is to install and configure Jenkins with the Docker Plugin.  If you have a Jenkins server available, feel free to skip the Jenkins installation step and go directly to the Docker Plugin configuration.


Installing Jenkins

The super easy way to install Jenkins is by downloading and running the war file directly.  Otherwise, if you want to use a specific application server to host Jenkins, see the Container specific setup.  Download the Jenkins war file from the main Jenkins site, then set the path to your Jenkins home directory, and start the Jenkins server.


export JENKINS_HOME=/opt/jenkins
java -jar jenkins.war


You should be able to access the Jenkins UI using a browser to port 8080 (http://localhost:8080/).


Configuring the Jenkins Docker Plugin

The next step is to download and install the Jenkins docker plugin.  Go to "Manage Jenkins"->"Manage Plugins", then click on the "Available" tab.  Find the "Docker Plugin" and install it.  Restart Jenkins if necessary.


Next, locate the Jenkins cloud configuration under "Manage Jenkins"->"Configure System".  Towards the bottom of the page, you should find an option to "Add a new cloud" and the "Docker" option will be available.  The docker URL should point to the IP address of your docker host.  The  ID field determines which container image will be used for for the slaves, and the Labels field allows your Jenkins projects to use this docker cloud provider.



If everything is set up correctly, the "Test Connection" button should return the version of Docker running on the host.  The final step is to configure your Jenkins project to use the Docker cloud.



The project is now ready to run.  If everything is set up correctly, Jenkins should start up a new Docker container, run the build, and then shut down the container.




Well, that was a lot of work, but hopefully if you followed all the steps, you were able to see it working.  The integration between Jenkins and Docker is still relatively new, so it can take some effort to get them working nicely together.  The ability to start up a new Jenkins slave quickly from a clean image has a lot of potential when it comes to creating reproducible builds.  And features like tagging the docker image upon build completion (described on the Jenkins docker plugin site), have a lot of potential both for auditing, and for easily reproducing a build failure in a local environment.



Related Links


Some additional links that helped greatly while working on this.