How to run ‘Jenkins pipeline completely in local’ with Local GitHub repo and Local docker image registry.

Let’s discuss about a Jenkins pipeline where you need to pull a base docker image from your docker registry, clone your GitHub repo with all your changes and you want to mount this repo into the docker container to create an environment to run your test script.

Here I will explain how to run a ‘completely local Jenkins Pipeline’ set up with docker image and local git repo.

Let’s start from the scratch!!

Install docker if it is not already installed. You can follow the below steps to install docker (Ubuntu 20.04)

sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu focal stable" | sudo tee /etc/apt/sources.list.d/docker.list
sudo apt update
sudo apt install -y docker-ce
sudo systemctl start docker
sudo systemctl enable docker
docker --version

Refer the below GitHub repo for the source file reference:

sidharthvpillai/Jenkins-docker-local: A reference repo for Medium blog for running Jenkins with local repo and local docker registry. (github.com)

Step 1: Create a sample docker image.

Here we are creating an image with Ubuntu base image and python installed on it.

# Use a Linux distribution (Ubuntu in this case) as the base image
FROM ubuntu:20.04
USER root
# Set the working directory inside the container
WORKDIR /app
# Update the package index and install Python 3 and pip
RUN apt-get update && \
    apt-get install -y python3 python3-pip
# Add your all other dependencies here

Build the above docker file.

sudo docker build -t my-image .
sudo docker images

This will create a sample docker image named my-image. We need to push this into local docker registry for the Jenkins to pull that from the Pipeline.

For running the local registry, we need to use registry:2 image:

sudo docker run -d -p 5000:5000 --restart=always --name registry registry:2

Now we are ready with the local registry, then we need to push our local image into the local registry.

sudo docker tag my-image localhost:5000/my-image
sudo docker push localhost:5000/my-image

We are now ready with the docker image. We need to create a git repo that contains the script to run which will be later mounted to the docker container inside the Jenkins Pipeline. Assuming that the git is already installed.

mkdir ~/sample-repo
cd ~/sample-repo
git init

Add a python script in the repository.

print("hello world")

Create a Jenkinsfile with the below content to run the helloworld.py file in pipeline. The pipeline will first clone the repo, pull the image from registry and then it will start the container by mounting the repo changes into the container. Later it will run the helloworld.py script.

pipeline {
    agent any
    stages {
        stage('Setup') {
            steps {
                script {
                    sh "docker stop container || true"
                    sh "docker rm container || true"
                    def dockerImage = 'localhost:5000/my-image'
                    docker.image(dockerImage).pull()
                    def containerId = docker.image(dockerImage).run("--user root --rm -it -v ${pwd()}:/mnt --name container")
                }
            }
        }
        stage('Run Script') {
            steps {
                sh "docker exec container /bin/bash -c 'python3 /mnt/helloworld.py'"
            }
        }
    }
    post {
        always {
            cleanWs()
            sh "docker stop container || true"
            sh "docker rm container || true"
        }
    }
}

Commit the changes.

git add .
git commit -m "first commit"

Step 2: We will now start setting up the Jenkins Server using docker.

Create a Dockerfile with the following content to create a custom Jenkins image which contains docker (Since we are using docker inside Jenkins)

# Use the official Jenkins LTS image as the base image
FROM jenkins/jenkins:lts

# Install Docker CLI and client tools
USER root
RUN apt-get update && apt-get install -y apt-transport-https ca-certificates curl gnupg lsb-release
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
RUN echo "deb [signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list
RUN apt-get update && apt-get install -y docker-ce-cli

# Switch back to the Jenkins user
USER jenkins

Build docker image.

sudo docker build -t custom-jenkins .
sudo docker images

It will list down the created docker image.

Create a directory to mount to Jenkins docker container so that the Jenkins data won’t be lost in restart:

mkdir /var/jenkins_home

When we run the Jenkins container, we will be mounting this directory so that the Jenkins data won’t be getting lost when we restart the server.

Start the Jenkins container with the below command. Make sure that the sample-repo path is given properly (We have created the sample-repo in ~ directory in the above step, if you cloned the repo, change the directory accordingly).

sudo docker run -d --restart=always -p 8080:8080 -p 50000:50000 \
  -v /var/jenkins_home:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v ~/sample-repo:/mnt \
  -e JAVA_OPTS=-Dhudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true \
  --name jenkins \
  custom-jenkins

ALLOW_LOCAL_CHECKOUT=True is used for allowing the Jenkins to clone the local repository.

Now you will be able to access the Jenkins server in http://localhost:8080

Troubleshoot:

Give permissions for the directory /var/jenkins_home

sudo chmod 7777 /var/jenkins_home

Run the below command to get the initialAdminPassword and use that password to setup the admin user in Jenkins.

docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword

Select “Install Suggested Plugins” from the options and it will take some time to install the plugins. Once it is done, fill up the details and start.

Go to “Manage Jenkins” > “Plugins” > “Available Plugins” > Search “Docker Pipeline” > Install and restart the Jenkins Server.

This plugin is required for us to run docker inside the Jenkins.

Step 6: Create pipeline from Jenkins Server.

Click on ‘New Item’ for creating new pipeline.

Enter a name, select ‘Pipeline’ and click ‘OK’.

On the next page scroll down and select ‘Pipeline Script from SCM’ from dropdown.

Select “Git” from “SCM” dropdown, give the path and branch name and script path and “save”.

Now you are ready for “Build Now” to start a Jenkins build with the local Repo and it will run the helloworld.py script in pipeline.

Troubleshoot:
If you are facing a permission issue with docker.sock file, try a workaround to change the permission of docker.sock file.

sudo chmod 777 /var/run/docker.sock

And try “Build Now” again to run the pipeline.

You will get a successful build with Hello world message printed into the console.

You have successfully created a Jenkins build pipeline with local git repo and local docker image. Now you can create your own repository and image by modify corresponding Jenkinsfile and Dockerfile accordingly.

:-)

Sidharth V | LinkedIn