Run Jenkins and Jenkins agents on Docker

I have managed a Gitlab instance for a couple of years, but for some organizations, Gitlab is overkill. For some people, Gitea is enough. However, Gitea does not have production-ready CI/CD yet.

Fortunately, it’s possible to link Jenkins to Gitea. Here’s how to do it. In this post, we will first configure Jenkins to use agents in Docker.

It is not recommended to run pipelines on the Jenkins host. You can run a static container or let Jenkins spin up containers on the fly. We will do both. We will connect to the containers using an SSH key.

First, create a SSH key pair using ssh-keygen -t ed25519 -f jenkins.id_ed25519 -C jenkins.

Build a custom image from jenkins/ssh-agent or jenkins/inbound-agent as shown on this github repository. I have decided to use jenkins/ssh-agent and add python packages.

FROM jenkins/ssh-agent:alpine-jdk17

RUN apk -U --no-cache upgrade && \
apk -U --no-cache add python3 py3-pip py3-pytest

For the static container path, use the following service in your docker-compose.yml file. Copy the public key generated above in the JENKINS_AGENT_SSH_PUBKEY environment variable.

  agent:
    build: jenkins_ssh-agent-alpine-jdk17-python3
    restart: always
    environment:
      - JENKINS_AGENT_SSH_PUBKEY=ssh-ed25519 xxxxxxxxxxxxx jenkins
    volumes:
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 1000M
    memswap_limit: 1000M

Start the container using docker compose up -d agent.

As a Jenkins admin, go to Manage Jenkins > Nodes and Clouds and click New Node then follow these steps:

1. Set a name (“agent-python“), select “Permanent Agent” and click Create.
2. Suit the number of executors to your needs.
3. Set the Remote root directory to /home/jenkins.
4. Set a label such as “python“
5. Select “Launch agents via SSH” as the Launch method
6. Enter the name of the service as host (here, “agent“)
7. Add new credentials of the kind “SSH Username with private key“, set the username to jenkins, paste the private key and public key, click Add.
8. Select the credentials you have just created.
9. Select Non verifying Verification Strategy.
10. Click Save.

You should see in the agent logs it accepted a connection using a public key and in Jenkins web interface, the agent should appear connected with its status.

You should now be able to run a pipeline using this agent.

Next, we will give the ability to Jenkins to start containers on the fly, for each pipeline. To achieve this, Jenkins needs to talk to the docker daemon. You can either talk through the docker socket or using TCP.

If you choose to use the socket, you need to mount it in the container as a volume and set the container as privileged. I didn’t want this so I went for the TCP method.

Docker can listen on multiple IP addresses and use TLS to secure the TCP socket. The docker documentation explains how to setup a PKI to enable TLS on the TCP socket as well as authenticate the clients using a SSL certificate. I will therefore not cover the setup of the CA and assume you saved the files in /etc/docker/ssl.

To make docker daemon listen on TCP sockets on Ubuntu, you need to override the ExecStart option of the systemd unit. Edit the systemd unit using systemctl edit docker.service , then type the following. You can add as many IP addresses as you need.

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://127.0.0.1:2376 -H tcp://1.2.3.4:2376 -H tcp://6.7.8.9:2376 --containerd=/run/containerd/containerd.sock --tls=true --tlsverify=true --tlscacert /etc/docker/ssl/ca.pem --tlscert /etc/docker/ssl/server-cert.pem --tlskey /etc/docker/ssl/server-key.pem

Then, (re)start docker daemon: systemctl restart docker. At this point, only clients presenting a valid certificate will be able to connect to your daemon. This is enough security for me. If you need more, look into the access authorization plugin.

It’s time to connect Jenkins to your Docker host. Go to Manage Jenkins > Plugins > Available plugins and install Docker Pipeline and Docker plugin.

Then, go to Manage Jenkins > Nodes and Clouds > Clouds and click Add new cloud. Then follow the steps below.

1. Set a name like “localhost“
2. In Docker Host URI, set one of the values you configured for Docker daemon above, such as “1.2.3.4:2376“
3. Add a new credentials of the kind X.509 Client Certificate.
4. Paste the client private key, client certificate and CA certificate.
5. Tick the Enabled check box.
6. Set a reasonable value for Container Cap such as 10, depending on your docker host resources.
7. Click Docker Agent templates and then Add Docker Template.
8. Set a label reflecting your template, such as “python“.
9. Tick the Enabled check box.
10. Set a friendly name such as docker-ssh-agent-jdk17-python.
11. Set the docker image to the name of the image built earlier.
12. Set an environment variable for the public key to accept (see above), such as JENKINS_AGENT_SSH_PUBKEY=ssh-ed25519 xxxxxxxxxxxxx jenkins.
13. Set Remote File System Root to /home/jenkins.
14. Set the connect method to Connect with SSH.
15. Use the credentials defined earlier to connect with SSH private key.
16. Because the container needs to start and it can take a couple of seconds, set the maximum number of retries to 2 and the seconds to wait between retries to 10. This will ensure the ssh server has started before the second connection attempt.
17. Tick the Remove volumes check box.
18. Set the pull strategy to Pull once and update latest.
19. Click Save.

Create a simple pipeline and set the label to any you set above and run the build. Both should succeed.

And that’s it.