One of the primary goals of the JBoss product team is to ship a product that we're confident can be effectively supported for many years. When a bug is discovered we need to be able to quickly reproduce it, create a fix, and then build and ship a new binary package which contains that fix. This means that when a bug is reported, we need to not only be able to quickly find the relevant source code, we also need information related to the environment and any build commands/parameters needed to build and run that project.
We're always looking for ways to improve our build environment, which is currently based on the Fedora build system called Koji. Linux Containers in general, and specifically Docker seem to be gaining popularity, and they have the potential to provide a great environment for reproducible builds. Why are linux containers potentially great for creating reproducible builds? For a few reasons:
- Build system images can be configured as needed (Java, Maven, gcc, etc)
- Each image provides a clean environment in which a build can be run.
- They are relatively lightweight (compared to a full VM) and can be started up quickly
- They can be saved/tagged to provide a record of the resulting build output
So, enough of the background information, let's see how to get this set up so you can start experimenting for your own use cases.
Configuring the Docker Host
The first step is to set up the server which is going to be hosting the docker containers. This could be your local workstation, or you can set it up on separate server. For this example we'll be using Fedora 20 as the Docker Host operating system, but you can easily use Ubuntu or another Linux distro.
Note: several of the examples use "sudo" to execute privileged commands. If you are using Fedora, and don't want to use sudo, you may need to add yourself to the "wheel" group.
$ usermod -G -a wheel <username>
The Fedora package is called "docker-io" to avoid conflicts with another package called "docker":
$ sudo yum -y install docker-io
If you run into problems during this install, you can take a look at the more detailed instructions on the Docker Web Site.
Next, we need to configure the docker host to accept incoming connections over TCP (the default is a local unix socket) so that Jenkins can communicate with docker. On Fedora, the easiest way to configure this is by editing the systemd startup file for docker located in /usr/lib/systemd/system/docker.service
On the line that starts with "ExecStart=", add the parameter " -H tcp://0.0.0.0:4243". This tells docker to listen for incoming commands on port 4243 on the local system. After modification, the line should look something like this:
ExecStart=/usr/bin/docker -d --selinux-enabled -H fd:// -H tcp://0.0.0.0:4243
Next, you can use systemd to start the docker daemon.
$ sudo systemctl start docker
You may also want to add your user account to the "docker" group so that you can run docker without sudo access.
$ usermod -G -a docker <username>
Creating the Jenkins Slave Docker Image
The next step is to create a docker image which meets the basic requirements to be a Jenkins slave. A simple Jenkins slave requires sshd, Java, and a user account for Jenkins to connect. There are plenty of existing images which meet these requirements, but for completeness, we'll build our own from scratch.
First, create a Dockerfiles directory with a subdirectory called fedora-jenkins-slave.
mkdir -p Dockerfiles/fedora-jenkins-slave
Next, we need to create two files. One is the Dockerfile, which contains the instructions for building a Docker image. Then, we'll need a simple supervisord.conf file to manage sshd startup when we run the image (docker images are often intended to run a single process/service and therefore do not contain process management configuration such as systemd or upstart)
The example dockerfile based on the default fedora image installing sshd, Java, and a "jenkins" user.
FROM fedora:latest MAINTAINER Foo Bar "firstname.lastname@example.org" RUN yum -y update RUN yum -y install openssh-server RUN yum -y install supervisor RUN yum -y install java-1.7.0-openjdk RUN echo "root:password" | chpasswd RUN useradd jenkins RUN echo "jenkins:jenkins" | chpasswd RUN mkdir -p /var/run/sshd RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N '' RUN sed -i 's|session required pam_loginuid.so|session optional pam_loginuid.so|g' /etc/pam.d/sshd RUN mkdir -p /var/run/supervisord ADD supervisord.conf /etc/supervisord.conf EXPOSE 22 CMD ["/usr/bin/supervisord"]
Most of the Dockerfile should be pretty self-explanatory. The "FROM" line says that we want to start with the latest base Fedora image. The "RUN" lines run a few commands inside the image to get sshd, java, and supervisor installed and configured. And the "EXPOSE" line tells docker that port number 22 should be exposed to incoming connections.
A Complete Reference to the Dockerfile is available on the Docker site.
Next, create a file called "supervisord.conf" in the same directory as the Dockerfile. This file will be copied into the docker image during the build.
[supervisord] nodaemon=true [program:sshd] command=/usr/sbin/sshd -D
This file just tells supervisord to start up sshd whenever the image is started.
Now, build the image and give it a name such as fedora-jenkins-slave.
$ docker build -t my-fedora-jenkins-slave .
This may take a few minutes, so you'll have to be patient
Now we're ready to run the image and make sure it's working correctly.
$ docker run -t -i my-fedora-jenkins-slave
You can see that the docker container is running using the "docker ps" command.
$ docker ps
This will give you a list of the currently running docker containers, including a unique identifier which will be needed for the next step.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
68f1ed77d3e9 my-fedora-jenkins-slave:latest /usr/bin/supervisord About a minute ago Up 59 seconds 22/tcp stupefied_bell
Next, use the container ID to get more information about the running container.
$ docker inspect 68f1ed77d3e9 $ docker inspect 68f1ed77d3e9 | grep IP
The second command shows the virtual IP address of the running container which we can use to test the ssh connection. If everything is working, you should be able to log into the container using the username and password "jenkins".
$ ssh email@example.com
If the login is successful, you can also check that the correct version of Java is installed.
$ java -version
Congratulations, your Docker cloud is now ready! The last step is to tell Jenkins to start using it.
The final part of this process is to install and configure Jenkins with the Docker Plugin. If you have a Jenkins server available, feel free to skip the Jenkins installation step and go directly to the Docker Plugin configuration.
The super easy way to install Jenkins is by downloading and running the war file directly. Otherwise, if you want to use a specific application server to host Jenkins, see the Container specific setup. Download the Jenkins war file from the main Jenkins site, then set the path to your Jenkins home directory, and start the Jenkins server.
export JENKINS_HOME=/opt/jenkins java -jar jenkins.war
You should be able to access the Jenkins UI using a browser to port 8080 (http://localhost:8080/).
Configuring the Jenkins Docker Plugin
The next step is to download and install the Jenkins docker plugin. Go to "Manage Jenkins"->"Manage Plugins", then click on the "Available" tab. Find the "Docker Plugin" and install it. Restart Jenkins if necessary.
Next, locate the Jenkins cloud configuration under "Manage Jenkins"->"Configure System". Towards the bottom of the page, you should find an option to "Add a new cloud" and the "Docker" option will be available. The docker URL should point to the IP address of your docker host. The ID field determines which container image will be used for for the slaves, and the Labels field allows your Jenkins projects to use this docker cloud provider.
If everything is set up correctly, the "Test Connection" button should return the version of Docker running on the host. The final step is to configure your Jenkins project to use the Docker cloud.
The project is now ready to run. If everything is set up correctly, Jenkins should start up a new Docker container, run the build, and then shut down the container.
Well, that was a lot of work, but hopefully if you followed all the steps, you were able to see it working. The integration between Jenkins and Docker is still relatively new, so it can take some effort to get them working nicely together. The ability to start up a new Jenkins slave quickly from a clean image has a lot of potential when it comes to creating reproducible builds. And features like tagging the docker image upon build completion (described on the Jenkins docker plugin site), have a lot of potential both for auditing, and for easily reproducing a build failure in a local environment.
Some additional links that helped greatly while working on this.