Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Category: Jenkins (page 1 of 2)

Jenkins – How To Manually Upgrade Jenkins Plugin To Specific Versions

Introduction

Jenkins Plugin Manager will enable plugin installation of the latest version. Sometimes we might want to install plugin of a specific version, instead of the latest. To do that, we can do the following:

Steps

  • Older versions of plugins can be found here: https://updates.jenkins.io/download/plugins/
  • The plugins are in .hpi file extensions
  • Download the version you want and put it on the Jenkins server
  • Plugins are stored in /var/lib/jenkins
  • Backup – you might want to backup the current version of the plugins before upgrading. Each plugin has a directory and a .jpi file.

For eg: datadog plugin

mkdir datadog_backup

mv datadog* /datadog_backup

  • Place the .hpi file in /var/lib/jenkins
  • Restart Jenkins

service jenkins restart

  • A directory with the plugin name and a .jpi file will be created upon restart
  • Check if the plugin with the correct version appears on Jenkins website Manage Jenkins->Plugin Manager
  • If not, check the log on the server in /var/log/jenkins 
  • Log is usually called jenkins.log
  • Log will show the plugin installation failure as SEVERE
  • Usually this is due to version dependencies of other plugins as named in the log.

For eg:

SEVERE: Failed Loading plugin sauce-ondemand
java.io.IOException: Dependency workflow-job (1.15), workflow-cps (1.15), workflow-basic-steps (1.15), workflow-step-api (1.15) doesn’t exist

  • Download and install the missing dependencies plugin via the same method as above
  • Upon successful installation, you will see the plugin with the right version appearing in Manage Jenkins->Plugin Manager

Jenkins – config.xml

If there is a need to debug Jenkins configuration and the when the interface set up looks correct, it’s better to find more detailed configuration information via the config.xml.

  1. In the server where Jenkins is installed, usually in path /var/lob/jenkins/jobs
  2. Browse to your job folder
  3. There will be a config.xml which will contain the job configuration details

Jenkins – How To Disable Jobs

How To Disable One Jenkins Job

  • Get breadcrumb

curl -u “<user>:<password>“ ‘https://<jenkins_url>/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,”:”,//crumb)’

 

  • curl -I -X POST https://<jenkins_url>/<job_path>/disable –user <user>:<password> -H “<jenkins bread crumb>”

How To Disable Jenkins jobs Under A Folder

  • Via Jenkins script

https://<jenkins_url>/script

  • Enter the following in the script and execute it

folderName=”Build/APP/test” //full name of the folder you want to disable all jobs in

Jenkins.instance.getItemByFullName(folderName).allJobs

.each {

      it.setDisabled(true)

      println(“Disabled job: [$it.fullName]”)

    }

null

Note: Look for file structure in server, ignore the “jobs” path in URL

Jenkinsfile – Credentials Binding Plugin – sshUserPrivateKey

Example below shows how to use the sshUserPrivateKey of a server to do Git clone in Jenkinsfile:

node(‘jenkins_node’){

stage(‘checkout code’){

//Private key for server is stored in Jenkins with id ‘private_key_for_server’ and can be accessible via variable ‘private_key’

withCredentials([sshUserPrivateKey(credentialsId: ‘private_key_for _server’, keyFileVariable: ‘private_key’, passphraseVariable: ”, usernameVariable: ”)]){

// start ssh-agent
sh ‘ssh-agent /bin/bash’

// add private key to ssh-agent, check if private key is successfully added and git clone using the private key

sh ‘eval $(ssh-agent) && ssh-add ${private_key} && ssh-add -l && git clone git@git.test.com:test.git’

}

}

}

 

How To Trigger Downstream Job And Pass In Parameters

This is a Jenkinsfile that:

  • Takes in parameter “test_branch”
  • Runs Job-1 and pass in parameter “test_branch” to it
  • Upon successful run of Job-1, it will trigger Job-2 and pass in the “test_branch” parameter
  • Note that Job-1 and Job-2 are 2 separate Jenkins jobs, and the Jenkinsfile below belongs to the Jenkins job that triggers both Job-1 and Job-2

node(DOCKER_IMAGE){

deleteDir()

checkout([$class: ‘GitSCM’, branches: [[name: “${test_branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs:              [[credentialsId: ‘abc’, url: GIT_URL]]])

stage(‘job 1’){

build job: ‘Job-1’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}
stage(‘job 2’){

build job: ‘Job-2’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}

Jenkins 2 – How To Use Credentials Binding Plugin

This post will be demonstrated using Secret File.

  • Select the Credentials “Kind” – “Secret File”

Screen Shot 2017-12-27 at 10.39.56 AM

 

  • Enter the password in a plain text file
  • Enter a ID, Description and upload the plain text file
  • Using Credentials Binding Plugin, enter the following in the Jenkinsfile where credentialsId refers to ID field above

withCredentials([file(credentialsId: ‘ID’, variable: ‘FILE’)]) {
ansiblePlaybook(
playbook: ‘playbook.yml’,
sudo: true

)

}
}

 

 

Jenkinsfile – Build & Publish Docker In Docker

The Jenkinsfile below shows how build and publish a  Docker image to Docker registry on a Dockerized Jenkins node:

//Running on Docker node

node(DOCKER_IMAGE){

deleteDir()

checkout([$class: ‘GitSCM’, branches: [[name: “${git_branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs:     [[credentialsId: ‘abc’, url: GIT_URL]]])

stage(‘docker build, tag & push’){

//credentials for Docker registry

withCredentials([usernamePassword(credentialsId: ‘dockerpush’, passwordVariable: ‘pass’, usernameVariable: ‘user’)]) {

dir(“${dockerfile_path}”)
{

//Build the Docker image
def dockerImage=docker.build(“${docker_source_image_tag}”)

//Tag the image

sh ‘docker tag “${docker_source_image_tag}” “${docker_target_image_tag}”‘

docker.withRegistry(‘https://docker-test.com’, ‘dockerpush’) {

//Log into the Docker registry

sh “docker login -u ${user} -p ${pass} https://docker-test.com”

//Push the image

dockerImage.push(‘latest’)
}

}

}
}

}

How To Write Jenkinsfile

Jenkinsfile is another great feature from Jenkins2.

Below is an example of a Jenkinsfile:

properties(

[             

   //Parameters of a Jenkins build  
parameters(
[
text(defaultValue: ”, description: ‘URL’, name: ‘ARTIFACT’),
choice(choices: ‘qa’, description: ‘Deploy_Env’, name: ‘DEPLOY_ENV’),
string(defaultValue: ‘master’ , description: ‘ Branch’,name:’BRANCH’)
]
)
]
)

//Which node the job should run on

node(‘master’){

//Delete directory before job starts

deleteDir()

//Git checkout certain branch using defined Git credentials

checkout([$class: ‘GitSCM’, branches: [[name: “${branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: ‘abc’, url: GIT_URL]]])

//Name of which stage of task that is running
stage(‘deploy’){

//Credentials with secret file configured in Jenkins

withCredentials([file(credentialsId: ‘PASS’, variable: ‘FILE’)]) {

//Execute shell script

sh ‘ansible-galaxy install -r requirements.yml –force’

//Ansible command

ansiblePlaybook(
playbook: ‘deploy.yml’,
inventory: ‘inventory/qa.inventory’,
extraVars:[
artifact_url: “${ARTIFACT}”,
],
extras: ‘–diff –vault-password-file ${FILE} –tags ${ACTION}’,
colorized: true

)

}
}

}

Enter Jenkinsfile into Jenkins2 as below:

Screen Shot 2017-10-24 at 11.14.39 AM

References on Jenkinsfile

Screen Shot 2017-10-20 at 1.28.07 PM

Jenkins – Making Data Persistent In Docker Slave

As in Jenkins Docker Slave, every time a job runs, a new Docker container is started and terminated upon job completion. This will mean it will download all the dependencies every time. To avoid this, we will use the Volume function in Docker.

As we are using the Docker Plugin, there is a field to do this – Volumes.

Docker Jenkins plugin

 

In the example above, we map path /var/lib/jenkins/tools on the Slave machine to path /home/jenkins/tools in Docker container, and /home/jenkins/.m2 on Slave machine to /home/jenkins/.m2 on Docker container. We can specify the mode of Read/Write, Read-Only, etc, by default is Read/Write.

Therefore, all Jenkins tools and Maven dependencies will be stored on the host or Slave Machine every time Docker container runs and will not need to be downloaded upon next Docker container starts. So build time will be saved.

 

How To Set Up Jenkins 2.0 Master & Slaves On Docker

Jenkins 2.0 Master In Docker

We can just use the Jenkins Docker official image, or you can just install Jenkins 2.0 normally.

 Jenkins Slave In Docker

Setup

  • Install Docker
  • Open up TCP port by adding the following in /etc/default/docker
    DOCKER_OPTS="-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock"
  • On Jenkins Master, install Docker Plugin on Master->Manage Jenkins->Manage Plugins
  • On Master->Manage Jenkins->Cloud configure the communication with Jenkins slave node
  • Docker URL is the IP of the Slave machine in TCP
  • Click on Test Connection, if successful will show the Docker version and Docker API versionDocker Plugin

 

  • Have Docker Jenkins Slave images in the slave box
  • On Master->Manage Jenkins->Manage Plugins->Cloud->Add Docker Template
  • The SSH credentials to access the Docker Jenkins Slave container is the Jenkins user setup in Dockerfile and Jenkins MasterScreen Shot 2017-03-27 at 4.29.05 PM
Docker Slave Image

There are some Docker images that you can use like Docker Slave image , or you can write your own Dockerfile.

Docker image contains a minimum of:

  • Ubuntu
  • Java
  • Jenkins user and password
  • Git
  • OpenSSH for Jenkins Master to SSH to Slave machine
  • Maven, Ruby, etc depends on what your project needs
  • Version managers like NPM or RVM cannot be installed in Docker due to we cannot “source” files like .bashrc

It is a sshd service running on port 22, therefore in the Dockerfile, you will need:

EXPOSE 22

CMD [“/usr/sbin/sshd”, “-D”]

Running
  • We restrict Jenkins jobs to run based on the label we give in Docker template
  • When job runs, Jenkins Master will spin up the Docker image on the Slave machine
  • When job completes, the Docker image will be terminated
  • Therefore, we can have multiple Docker images for different types of jobs, Ruby, Maven, NodeJS on a single Slave machine
« Older posts

© 2019 Chuan Chuan Law

Theme by Anders NorenUp ↑