Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Page 2 of 12

Jenkins – config.xml

If there is a need to debug Jenkins configuration and the when the interface set up looks correct, it’s better to find more detailed configuration information via the config.xml.

  1. In the server where Jenkins is installed, usually in path /var/lob/jenkins/jobs
  2. Browse to your job folder
  3. There will be a config.xml which will contain the job configuration details

Jenkins – How To Disable Jobs

How To Disable One Jenkins Job

  • Get breadcrumb

curl -u “<user>:<password>“ ‘https://<jenkins_url>/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,”:”,//crumb)’

 

  • curl -I -X POST https://<jenkins_url>/<job_path>/disable –user <user>:<password> -H “<jenkins bread crumb>”

How To Disable Jenkins jobs Under A Folder

  • Via Jenkins script

https://<jenkins_url>/script

  • Enter the following in the script and execute it

folderName=”Build/APP/test” //full name of the folder you want to disable all jobs in

Jenkins.instance.getItemByFullName(folderName).allJobs

.each {

      it.setDisabled(true)

      println(“Disabled job: [$it.fullName]”)

    }

null

Note: Look for file structure in server, ignore the “jobs” path in URL

Docker – How To Source A File In Dockerfile

If you want to source /etc/profile.d/apache-maven.sh, do this in your Dockerfile

RUN echo ‘. /etc/profile.d/apache-maven.sh’ >> ~/.bashrc

Consul – Integrating Nagios Checks

We can integrate script checks into Consul.  To do so with Nagios:

  • Install nagios-plugins-basic. That will contain some basic checks such as disk util, and cpu load

apt-get install nagios-plugins-basic

  • There are also other custom Nagios checks on the internet for eg: checks for Open Connections

Put this script in:

/usr/lib/nagios/plugins

  • Create JSON config (connections.json) and put it under /etc/consul.d/client

{
“check”: {
“name”: “Open Connections”,
“interval”: “60s”,
“args”: [“/usr/lib/nagios/plugins/check_connections”, “-c”, “{{connection_limit_critical}}”, “-w”, “{{connection_limit_warn}}” ],
“status”: “passing”
}

}

  • Restart Consul service
  • You should see this on Consul UI

How To Publish Artifacts To Artifactory Using Curl

We can publish an artifact onto Artifactory using unconventional way such as Curl.

This is how we can do that:

  • Generate a pom file using a bash script, so the artifact will have versioning, etc.
#!/usr/bin/env bash
cat  << EOF
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>1.0.0</modelVersion>
  <groupId>com.org.test</groupId>
  <artifactId>apple</artifactId>
  <version>apple</version>
  <packaging>tar</packaging>
</project>
EOF
  • Post onto Artifactory using Curl

curl -i -X PUT -u username:password –data-binary @”apple.tar” “https://artifactory/snapshot/com/org/test/apple/apple.tar”

Starting Dropwizard Application Using Jar Packaging & SysV

By default Debian package will generate an Upstart and SysV script.

This blog will show how we can ship an application in Jar file and start it using SysV

  • Put your jar file somewhere. E.g: /opt
  • Copy the Debian package generated config in /etc into the new app config directory. You might need to reformat the YAML file a bit.

For example:

Some vars might look like this:

timeout: ${sys.TIMEOUT!”5000ms” }

We want to change it to look like:

timeout: 5000ms

  • Uninstall Debian packages

apt-get purge <package>

  • Delete the generated Upstart (/etc/init/<package>.conf) and SysV script (/etc/init.d/<package>)
  • Create a SysV script which looks like this:
#!/bin/bash
#
# dropwizard     This shell script takes care of starting and stopping Dropwizard applications
#
# chkconfig: - 80 20
#
### BEGIN INIT INFO
# Provides: dropwizard
# Required-Start: $network $syslog
# Required-Stop: $network $syslog
# Default-Start:
# Default-Stop:
# Short-Description: start and stop dropwizard
### END INIT INFO

#You just need to replace the {{ var }} below for your application
APPLICATION_NAME="{{ app_name }}"
APPLICATION_USER="{{ app_user }}"
APPLICATION_HOME="{{ app_home }}"
APPLICATION_JAR="{{ app_name }}.jar"
APPLICATION_CONFIG="{{ app_name }}.yml"
APPLICATION_CONFIG_DIR="{{ app_config_dir}}"
APPLICATION_CMD="java {{ additional_java_args }} -jar ${APPLICATION_HOME}/${APPLICATION_JAR} server ${APPLICATION_CONFIG_DIR}/${APPLICATION_CONFIG}"
APPLICATION_SHUTDOWN_WAIT=120

dropwizard_pid() {
    echo `ps aux | grep "${APPLICATION_CMD}" | grep -v grep | awk '{ print $2 }'`
}

start() {
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]
    then
        echo "${APPLICATION_NAME} is already running (pid: $pid)"
    else
        # Start dropwizard
        echo "Starting ${APPLICATION_NAME}"
        su ${APPLICATION_USER} -c "cd ${APPLICATION_HOME}; ${APPLICATION_CMD} > /dev/null &"
    fi
    return 0
}

stop() {
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]
    then
        echo "Stopping ${APPLICATION_NAME}"
        kill $pid

    let kwait=$APPLICATION_SHUTDOWN_WAIT
    count=0
    count_by=5
    until [ `ps -p $pid | grep -c $pid` = '0' ] || [ $count -gt $kwait ]
    do
        echo "Waiting for processes to exit. Timeout before we kill the pid: ${count}/${kwait}"
        sleep $count_by
        let count=$count+$count_by;
    done

    if [ $count -gt $kwait ]; then
        echo "Killing processes which didn't stop after ${APPLICATION_SHUTDOWN_WAIT} seconds"
        kill -9 $pid
    fi
    else
        echo "${APPLICATION_NAME} is not running"
    fi

    return 0
}

status(){
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]; then
        echo "${APPLICATION_NAME} is running with pid: $pid"
    else
        echo "${APPLICATION_NAME} is not running"
    exit 1
    fi
}


case $1 in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    status)
        status
        ;;
    *)
    echo "Usage: $0 {start|stop|restart|status}"
    exit 1
    ;;

esac

exit 0
  • Finally, we can start the application by:

service <package> start

Jenkinsfile – Credentials Binding Plugin – sshUserPrivateKey

Example below shows how to use the sshUserPrivateKey of a server to do Git clone in Jenkinsfile:

node(‘jenkins_node’){

stage(‘checkout code’){

//Private key for server is stored in Jenkins with id ‘private_key_for_server’ and can be accessible via variable ‘private_key’

withCredentials([sshUserPrivateKey(credentialsId: ‘private_key_for _server’, keyFileVariable: ‘private_key’, passphraseVariable: ”, usernameVariable: ”)]){

// start ssh-agent
sh ‘ssh-agent /bin/bash’

// add private key to ssh-agent, check if private key is successfully added and git clone using the private key

sh ‘eval $(ssh-agent) && ssh-add ${private_key} && ssh-add -l && git clone git@git.test.com:test.git’

}

}

}

 

How To Trigger Downstream Job And Pass In Parameters

This is a Jenkinsfile that:

  • Takes in parameter “test_branch”
  • Runs Job-1 and pass in parameter “test_branch” to it
  • Upon successful run of Job-1, it will trigger Job-2 and pass in the “test_branch” parameter
  • Note that Job-1 and Job-2 are 2 separate Jenkins jobs, and the Jenkinsfile below belongs to the Jenkins job that triggers both Job-1 and Job-2

node(DOCKER_IMAGE){

deleteDir()

checkout([$class: ‘GitSCM’, branches: [[name: “${test_branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs:              [[credentialsId: ‘abc’, url: GIT_URL]]])

stage(‘job 1’){

build job: ‘Job-1’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}
stage(‘job 2’){

build job: ‘Job-2’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}

Jenkins 2 – How To Use Credentials Binding Plugin

This post will be demonstrated using Secret File.

  • Select the Credentials “Kind” – “Secret File”

Screen Shot 2017-12-27 at 10.39.56 AM

 

  • Enter the password in a plain text file
  • Enter a ID, Description and upload the plain text file
  • Using Credentials Binding Plugin, enter the following in the Jenkinsfile where credentialsId refers to ID field above

withCredentials([file(credentialsId: ‘ID’, variable: ‘FILE’)]) {
ansiblePlaybook(
playbook: ‘playbook.yml’,
sudo: true

)

}
}

 

 

Jenkinsfile – Build & Publish Docker In Docker

The Jenkinsfile below shows how build and publish a  Docker image to Docker registry on a Dockerized Jenkins node:

//Running on Docker node

node(DOCKER_IMAGE){

deleteDir()

checkout([$class: ‘GitSCM’, branches: [[name: “${git_branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs:     [[credentialsId: ‘abc’, url: GIT_URL]]])

stage(‘docker build, tag & push’){

//credentials for Docker registry

withCredentials([usernamePassword(credentialsId: ‘dockerpush’, passwordVariable: ‘pass’, usernameVariable: ‘user’)]) {

dir(“${dockerfile_path}”)
{

//Build the Docker image
def dockerImage=docker.build(“${docker_source_image_tag}”)

//Tag the image

sh ‘docker tag “${docker_source_image_tag}” “${docker_target_image_tag}”‘

docker.withRegistry(‘https://docker-test.com’, ‘dockerpush’) {

//Log into the Docker registry

sh “docker login -u ${user} -p ${pass} https://docker-test.com”

//Push the image

dockerImage.push(‘latest’)
}

}

}
}

}

« Older posts Newer posts »

© 2019 Chuan Chuan Law

Theme by Anders NorenUp ↑