Integrate Jenkins with GitHub

https://resources.github.com/whitepapers/practical-guide-to-CI-with-Jenkins-and-GitHub/ https://docs.github.com/en/github/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks https://docs.github.com/en/rest/reference/repos#statuses https://www.jenkins.io/doc/book/pipeline-as-code/ https://www.jenkins.io/doc/book/pipeline/jenkinsfile/ https://plugins.jenkins.io/github/#GitHubPlugin-GitHubhooktriggerforGITScmpolling https://github.com/alexanderkasten/test_runtime/blob/c5131ce6251a9317b34de8c6619a9cf29df41e01/Jenkinsfile

According to about-GitHub-webhooks and https://www.blazemeter.com/blog/how-to-integrate-your-github-repository-to-your-jenkins-project , please add GitHub webhook for Jenkins first.

User and Role

If user have Access Denied, then you need to grant the user proper role. In Configure Global Security, choose Jenkins’ own user database and Authorization.

My Goal I want to make sure all the Django tests passed before a GitHub reviewer can merge it into repository’s base branch. Otherwise, block the merge. Like this:

I want to make it by Using a Jenkinsfile.

Jenkins Login You need to have an account to login to: https://newjenkins.your-orgci.com/ or https://jenkins.your-orgci.com/ . Please ask your admin for it.

Create Pipeline After you have login, click New Item , them enter an item name like project-qa and select Pipeline , then click Save.

Configure Jenkins The Configure URL for this job would be https://jenkins.your-orgci.com/job/project-qa/configure .

General Check GitHub project , enter https://github.com/your-org/project to Project url.

Check This project is parameterized. Click Add Parameter, select String Parameter, then fill ref in name, fill a_custom_branch_name to Default Value. Like this:

Build Triggers Check Generic Webhook Trigger For Post content parameters, fill ref in Variable, fill $.ref in Expression. Like this:

Fill MyTokenForProject in token .

Fill Hi there, the ref is $ref in Cause.

Check Print post content and Print contributed variables.

Click Save.

Copy the URL like https://jenkins.your-orgci.com/generic-webhook-trigger/invoke?token=MyTokenForProject and open it in browser. You should see {“jobs”:{“project-qa”:{…}}}.

Add GitHub Webhook Visit https://github.com/your-org/project/settings/hooks, click Add webhook , paste the previous link to the Payload URL. Set Content type as application/json. For Which events would you like to trigger this webhook? , please only check Pushes. Save it.

You should see the first event ping in the Recent Deliveries has been delivered successfully. If not, you can go back and check. You can click the Redeliver button to test it until it is successfully delivered.

Optional filter Fill ^refs/heads/PROJECT-.*$ in Expression.

Fill $ref in Text. Like this:

Pipeline In Configure - Pipeline panel, select Pipeline script from SCM for Definition.

Select Git for SCM.

Fill https://github.com/your-org/project.git in Repositories. Since it is a private repository, we will need a Credentials. This is one of the ways to generate the Credentials:

GitHub token as Credentials Go https://github.com/settings/tokens and click Generate new token.

Check the Repo and Workflow (these two options are Jenkins admin told me to check) in Select scopes, click Generate token.

Copy the token and send this token to Jenkins admin. Ask him to add the Credentials for you.

After Jenkins admin has added the Credentials, then go back to Jenkins Configure - Pipeline panel, and select the the Credentials.

Go back to Jenkins pipeline.

For Branches to build, fill $ref in Branch Specifier.

Fill Jenkinsfile in Script Path.

Uncheck the Lightweight checkout.

Click Save. Like this:

Jenkinsfile Simplest Jenkinsfile Create a Jenkinsfile under your repo’s root folder. The Jenkinsfile can be like this:

void setBuildStatus(String message, String state) { step([ $class: “GitHubCommitStatusSetter”, reposSource: [$class: “ManuallyEnteredRepositorySource”, url: “https://github.com/your-org/project.git”], contextSource: [$class: “ManuallyEnteredCommitContextSource”, context: “ci/jenkins/build-status”], errorHandlers: [[$class: “ChangingBuildStatusErrorHandler”, result: “UNSTABLE”]], statusResultSource: [ $class: “ConditionalStatusResultSource”, results: [[$class: “AnyBuildResult”, message: message, state: state]] ] ]); }

pipeline { agent { label ‘project’ } stages { stage(‘Build’) { steps { echo ‘================= Building… ===================’ } } stage(‘Test’) { steps { echo ‘================= Testing… ===================’ } } stage(‘Deploy’) { steps { echo ‘================= Deploying…. ===================’ } } }

post {
    success {
        echo '================= Success! ==================='
        setBuildStatus("Build complete", "SUCCESS");
    }
    aborted {
        echo '================= aborted! ==================='
        setBuildStatus("Build aborted", "FAILURE")
    }
    failure {
        echo '================= failure! ==================='
        setBuildStatus("Build failed", "FAILURE")
    }
} }

After you have committed this file and pushed to GitHub by git push origin PROJECT-573 , you will see the build is running.

If you click link of build number, you can see something like this:

You need to make sure the Changes is the commits which you have just made, the Revision is the right head commit and the branch in the last line is just the branch you want to build.

Troubleshooting You may want to check the Console Output if the build is failed.

Here we saw a Jenkinsfile not found error. That is because there are many other branches which didn’t contain a Jenkinsfile yet.

Agent label You can see there is an

pipeline { agent { label ‘project’ } } in Jenkinsfile.

You can go https://jenkins.your-orgci.com/computer/ to select one of the Node like: https://jenkins.your-orgci.com/computer/ubuntu-b/configure , fill project in Labels and click Save.

Then your build will always be run on Node ubuntu-b.

Set build status For

pipeline { post { success { setBuildStatus(“Build complete”, “SUCCESS”); } } } The setBuildStatus will update the GitHub commit status. So you should see this:

If you didn’t see it, please check to Console Output, if you see [Set GitHub commit status (universal)] SUCCESS on repos [], please consult this page.

Run Django test in Docker To run a Django test, we will need to install Python and PostgreSQL first. I tried to do it before but finally quit since it is a bit complex and I don’t have the permission to access to Jenkins EC2. Finally I decided to run it in a docker container.

The full Jenkinsfile would be:

void setBuildStatus(String message, String state) { step([ $class: “GitHubCommitStatusSetter”, reposSource: [$class: “ManuallyEnteredRepositorySource”, url: “https://github.com/your-org/project”], contextSource: [$class: “ManuallyEnteredCommitContextSource”, context: “ci/jenkins/build-status”], errorHandlers: [[$class: “ChangingBuildStatusErrorHandler”, result: “UNSTABLE”]], statusResultSource: [ $class: “ConditionalStatusResultSource”, results: [[$class: “AnyBuildResult”, message: message, state: state]] ] ]); }

pipeline { agent { label ‘project’ } environment { POSTGRES_CREDS = credentials(‘project-postgres-creds’) } stages { stage(‘Test’) { steps { echo ‘================= Testing… ===================’ echo “The target ref is ${params.ref}” script { checkout scm def postgres_host = ‘db’ def postgres_db = ‘postgres’

                docker.image('postgres:11').withRun("--env POSTGRES_USER=$POSTGRES_CREDS_USR --env POSTGRES_PASSWORD=$POSTGRES_CREDS_PSW --env POSTGRES_DB=${postgres_db}") { c ->
                    docker.image('postgres:11').inside("--link ${c.id}:${postgres_host}") {
                        /* Wait until postgreSQL service is up */
                        sh "while ! pg_isready --host ${postgres_host} --username $POSTGRES_CREDS_USR --dbname ${postgres_db}; do sleep 1; done"
                    }
                    docker.image('python:3.8').inside("--link ${c.id}:${postgres_host}") {
                        /*
                         * Run some tests which require PostgreSQL, and assume that it is available on
                         * the host name `${postgres_host}`
                         */
                        withEnv(["HOME=${env.WORKSPACE}"]) {
                            sh '''
                                cd api
                                cp .env.dev.example .env
                                export ENVIRONMENT=dev
                                pip install -r requirements.frozen
                                python3 manage.py migrate
                                python3 manage.py test -v 3
                            '''
                        }
                    }
                }
            }
        }
    }
}

post {
    always {
        cleanWs()
    }
    success {
        echo '================= Success! ==================='
        setBuildStatus("Build complete", "SUCCESS");
    }
    aborted {
        echo '================= aborted! ==================='
        setBuildStatus("Build aborted", "ABORTED")
    }
    failure {
        echo '================= failure! ==================='
        setBuildStatus("Build failed", "FAILURE")
    }
} }

The ./api folder is a Django project.

Add PostgreSQL connection credentials In the Jenkinsfile, we can see:

POSTGRES_CREDS = credentials(‘project-postgres-creds’) When we connect to PostgreSQL database, we will need a username and password.

Please ask Jenkins admin to add project-postgres-creds as credentials ID, the credentials kind is User with password, your_custom_postgres_user as credentials username, your_custom_postgres_password as credentials password.

Your Python code’s database connection code should use the same configuration defined in this Jenkisfile. FYI:

Build default branch Once you have done the previous steps, you may also want to build the default branch (e.g: master).

So you will need to change the Optional filter expression as this:

The trouble is that when you do a Rebase and mergein GitHub

, it will send two push events at the same time.

The key props of this payload are:

ref: refs/heads/PROJECT-595 before: ‘ccedc4ef29af1a5b7433cdeb8890b234f03fa9c5’ after: ‘0000000000000000000000000000000000000000’ deleted: true commits: [] head_commit: The push to master is what we what to build. The other push event indicating that the branch is deleted. But it will cause a Jenkins build which it will always fail. The reason of the failure is that the cleanWs() in Jenkinsfile need a revision which cannot be found in the deletion push event.

So our best bet is to prevent this push event from triggering build. But I didn’t make it. I think it is not a big deal for it to fail. It seems no hurt.