Developing with Jenkins as code
Why
Having your job’s configuration and creation as code is awesome (at least in my opinion), but it comes with a downside : you can wreck you whole CI process with a little change that introduces a bug. And since you don’t want to experiment on your production server, you must have a sandbox. Maintaining a seond server is an option, but then it does not scale if your team has to work on the same parts of the server. It is a problem that we faced at my current job, and we solved it by having each SRE (and even developer if they want to) that needs/wants to contribute to the repository create and roll his own jenkins instance.
Docker to the rescue !
Docker is not a silver bullet, let’s say it once and for all. But it has its perks (otherwise it would probably not have gained so much attention), and here what we care about is the “run everywhere” feature, backed with the docker-compose
tool for the ease of use.
Let’s get started with the docker-compose.yml
file to set grounds :
version: '2'
services:
jenkins:
build: docker/master
ports:
- "8080:8080"
- "50000:50000"
volumes:
- ~/.jenkins_home:/var/jenkins_home
- .:/home/jenkins/jobs-dsl
- /var/run/docker.sock:/var/run/docker.sock
environment:
JAVA_OPTS: "-Dhudson.model.ParametersAction.keepUndefinedParameters=true"
ADMIN_USERNAME: "admin"
ADMIN_PASSWORD: "jenkinsadminpassword"
networks:
- jenkins-net
worker:
build: docker/worker
volumes:
- ~/.jenkins_worker:/home/jenkins
ports:
- "2222:22"
networks:
- jenkins-net
networks:
jenkins-net:
we will use 2 containers here, one being the master and the other one being a worker. The current dir is mounted into the master’s container to be read while working on it
Here is the master Dockerfile
(located in a docker/master
subfolder) stripped down to a minimal version :
FROM jenkins:2.46.3
USER root
RUN apt-get update && apt-get install -y awscli jq apt-transport-https ca-certificates gnupg2
RUN echo "dash dash/sh boolean false" | debconf-set-selections && DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
RUN adduser jenkins staff
USER jenkins
COPY plugins.txt /usr/share/jenkins/plugins.txt
RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/plugins.txt
ADD seed-job/ /usr/share/jenkins/ref/jobs/_seed-job
ADD nodes/ /usr/share/jenkins/ref/nodes
It uses the “official” jenkins image as a base, and then add the plugins (how to list them), a seed job to create the jobs and the worker node. I added these files at the end of this post.
Once you have run docker-compose
up and everything has been built, you can head to localhost:8080
and enjoy your whole new jenkins. Run the seed job on the master
branch and you’re set. While developing, if you use a branch per feature, then build your work branch to get the results in your development environment.
With this setup we came down to very few fixes that relate to having stuff not previously tested on the CI config side, even if it is not perfect. But it really does help not to break things and deliver quality work and PRs.
Additional files
seed job config file :
<?xml version='1.0' encoding='UTF-8'?>
<project>
<actions/>
<description></description>
<keepDependencies>false</keepDependencies>
<properties>
<hudson.plugins.disk__usage.DiskUsageProperty plugin="disk-usage@0.28"/>
<com.sonyericsson.rebuild.RebuildSettings plugin="rebuild@1.25">
<autoRebuild>true</autoRebuild>
<rebuildDisabled>false</rebuildDisabled>
</com.sonyericsson.rebuild.RebuildSettings>
<hudson.model.ParametersDefinitionProperty>
<parameterDefinitions>
<hudson.model.StringParameterDefinition>
<name>sha1</name>
<description></description>
<defaultValue>master</defaultValue>
</hudson.model.StringParameterDefinition>
</parameterDefinitions>
</hudson.model.ParametersDefinitionProperty>
</properties>
<scm class="hudson.plugins.git.GitSCM" plugin="git@3.0.0">
<configVersion>2</configVersion>
<userRemoteConfigs>
<hudson.plugins.git.UserRemoteConfig>
<url>/home/jenkins/jobs-dsl</url>
</hudson.plugins.git.UserRemoteConfig>
</userRemoteConfigs>
<branches>
<hudson.plugins.git.BranchSpec>
<name>${sha1}</name>
</hudson.plugins.git.BranchSpec>
</branches>
<doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
<submoduleCfg class="list"/>
<extensions/>
</scm>
<canRoam>true</canRoam>
<disabled>false</disabled>
<blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
<blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
<triggers/>
<concurrentBuild>false</concurrentBuild>
<builders>
<javaposse.jobdsl.plugin.ExecuteDslScripts plugin="job-dsl@1.51">
<targets>**.groovy</targets>
<usingScriptText>false</usingScriptText>
<ignoreExisting>false</ignoreExisting>
<ignoreMissingFiles>false</ignoreMissingFiles>
<removedJobAction>DELETE</removedJobAction>
<removedViewAction>DELETE</removedViewAction>
<lookupStrategy>JENKINS_ROOT</lookupStrategy>
</javaposse.jobdsl.plugin.ExecuteDslScripts>
</builders>
<publishers/>
<buildWrappers/>
</project>
worker node config file :
<?xml version='1.0' encoding='UTF-8'?>
<slave>
<name>worker</name>
<description></description>
<remoteFS>/home/jenkins</remoteFS>
<numExecutors>3</numExecutors>
<mode>NORMAL</mode>
<retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/>
<launcher class="hudson.plugins.sshslaves.SSHLauncher" plugin="ssh-slaves@1.11">
<host>worker</host>
<port>22</port>
<credentialsId>jenkins</credentialsId>
<maxNumRetries>0</maxNumRetries>
<retryWaitTime>0</retryWaitTime>
</launcher>
<label></label>
<nodeProperties/>
</slave>
work node Dockerfile :
FROM debian:jessie
RUN apt-get update && apt-get install -y awscli jq apt-transport-https ca-certificates gnupg2 openssh-server openjdk-7-jre-headless git make curl
RUN useradd jenkins
RUN echo "jenkins:jenkins" | chpasswd
RUN mkdir -p /var/run/sshd
RUN mkdir -p /home/jenkins ; chown jenkins: /home/jenkins
CMD ["/usr/sbin/sshd", "-D"]