Jenkins works

I was dabbling around with Jenkins in my Nomad cluster lately. In this post I quickly share my experiences and what I learned along the way.

#cicd #coding #jenkins #nomad

You may ask why I spend time to setup a good old Jenkins while everyone else seems to jump on some newer CI systems (GitLab, Forgejo, etc.)? Well, as said, it's still good old Jenkins and I assume it will be around for some more time.

To run Jenkins agents on a Nomad cluster I followed Ola Ogunseghas instructions and example code here:

It involves installing the Nomad plugin for Jenkins and configuring this “Nomad cloud” (that's how the integration is called in Jenkins) with a template for the Jenkins agents.

Obviously, I wanted to integrate Jenkins with my Git repos. The most straight forward way seemed to use post-receive hooks to nudge Jenkins on every push. This has worked fabulously so far.

Even though the runners are spawned as Nomad jobs, I still wanted to run other Docker containers in the pipeline. This is where it got very confusing for me, because of the different Docker plugins for Jenkins. Most notably, there exist at least these two plugins:

I decided to go with the former “docker-workflow” plugin because I already deployed the Jenkins agents as Nomad jobs. docker-workflow can run arbitrary containers from any Docker image, whereas the docker-plugin needs to be based on the Jenkins inbound-agent image to be able to connect to the Jenkins server.

I wanted the containers that are launched from Jenkins to be contained in a another users namespace. There exists the option to rebuild the inbound-agent with user-supplied attributes for uid/gid, but since I wanted to modify the image anyways I simply forked and built my own agent images, mostly inspired by the example for running Docker inside the agent.

At this stage, my tooling was sophisticated enough I could run a simple gitleaks container on each push to scan for secrets. I'm always afraid to publish secrets accidentally (it happened to me before).

Furthermore, I needed to establish some kind of build process, so I can also build images with Jenkins and directly push them to my local image registry. After some reading, I discarded the thought to use Kaniko because it still requires nodes in the native architecture of the respective build for multi-architecture builds.

Therefore, I followed RedHat best-practices to integrate Buildah, which is also the tool I use to build multi-arch container images locally on my laptop.

If you are interested in more example code, here the link to some of the key components:

I'm not finished with playing around, because I still haven't figured out some things fully yet (e.g., how to properly use jlink in multi-arch builds) and because I'm also curious how other members of the community use Jenkins on Nomad.

🛜 RSS | 🐘 Mastodon | 💬 IRC