Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Page 2 of 12

Python: ImportError: /usr/local/lib/python2.7/lib-dynload/_io.so: undefined symbol: PyUnicodeUCS2_Replace

The above error might occur if you have compiled and installed Python from source on Ubuntu.

curl https://www.python.org/ftp/python/2.7.6/Python-2.7.6.tgz -o python.tgz

tar -xvzf python.tgz

cd Python-2.7.6/

./configure

make

make install

 

The error can be fixed by

./configure –enable-unicode=ucs4

Docker – How To Set Locale In Ubuntu

Add the following in your Dockerfile to set Locale in Ubuntu

RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

Jenkins – config.xml

If there is a need to debug Jenkins configuration and the when the interface set up looks correct, it’s better to find more detailed configuration information via the config.xml.

  1. In the server where Jenkins is installed, usually in path /var/lob/jenkins/jobs
  2. Browse to your job folder
  3. There will be a config.xml which will contain the job configuration details

Jenkins – How To Disable Jobs

How To Disable One Jenkins Job

  • Get breadcrumb

curl -u “<user>:<password>“ ‘https://<jenkins_url>/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,”:”,//crumb)’

 

  • curl -I -X POST https://<jenkins_url>/<job_path>/disable –user <user>:<password> -H “<jenkins bread crumb>”

How To Disable Jenkins jobs Under A Folder

  • Via Jenkins script

https://<jenkins_url>/script

  • Enter the following in the script and execute it

folderName=”Build/APP/test” //full name of the folder you want to disable all jobs in

Jenkins.instance.getItemByFullName(folderName).allJobs

.each {

      it.setDisabled(true)

      println(“Disabled job: [$it.fullName]”)

    }

null

Note: Look for file structure in server, ignore the “jobs” path in URL

Docker – How To Source A File In Dockerfile

If you want to source /etc/profile.d/apache-maven.sh, do this in your Dockerfile

RUN echo ‘. /etc/profile.d/apache-maven.sh’ >> ~/.bashrc

Consul – Integrating Nagios Checks

We can integrate script checks into Consul.  To do so with Nagios:

  • Install nagios-plugins-basic. That will contain some basic checks such as disk util, and cpu load

apt-get install nagios-plugins-basic

  • There are also other custom Nagios checks on the internet for eg: checks for Open Connections

Put this script in:

/usr/lib/nagios/plugins

  • Create JSON config (connections.json) and put it under /etc/consul.d/client

{
“check”: {
“name”: “Open Connections”,
“interval”: “60s”,
“args”: [“/usr/lib/nagios/plugins/check_connections”, “-c”, “{{connection_limit_critical}}”, “-w”, “{{connection_limit_warn}}” ],
“status”: “passing”
}

}

  • Restart Consul service
  • You should see this on Consul UI

How To Publish Artifacts To Artifactory Using Curl

We can publish an artifact onto Artifactory using unconventional way such as Curl.

This is how we can do that:

  • Generate a pom file using a bash script, so the artifact will have versioning, etc.
#!/usr/bin/env bash
cat  << EOF
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>1.0.0</modelVersion>
  <groupId>com.org.test</groupId>
  <artifactId>apple</artifactId>
  <version>apple</version>
  <packaging>tar</packaging>
</project>
EOF
  • Post onto Artifactory using Curl

curl -i -X PUT -u username:password –data-binary @”apple.tar” “https://artifactory/snapshot/com/org/test/apple/apple.tar”

Starting Dropwizard Application Using Jar Packaging & SysV

By default Debian package will generate an Upstart and SysV script.

This blog will show how we can ship an application in Jar file and start it using SysV

  • Put your jar file somewhere. E.g: /opt
  • Copy the Debian package generated config in /etc into the new app config directory. You might need to reformat the YAML file a bit.

For example:

Some vars might look like this:

timeout: ${sys.TIMEOUT!”5000ms” }

We want to change it to look like:

timeout: 5000ms

  • Uninstall Debian packages

apt-get purge <package>

  • Delete the generated Upstart (/etc/init/<package>.conf) and SysV script (/etc/init.d/<package>)
  • Create a SysV script which looks like this:
#!/bin/bash
#
# dropwizard     This shell script takes care of starting and stopping Dropwizard applications
#
# chkconfig: - 80 20
#
### BEGIN INIT INFO
# Provides: dropwizard
# Required-Start: $network $syslog
# Required-Stop: $network $syslog
# Default-Start:
# Default-Stop:
# Short-Description: start and stop dropwizard
### END INIT INFO

#You just need to replace the {{ var }} below for your application
APPLICATION_NAME="{{ app_name }}"
APPLICATION_USER="{{ app_user }}"
APPLICATION_HOME="{{ app_home }}"
APPLICATION_JAR="{{ app_name }}.jar"
APPLICATION_CONFIG="{{ app_name }}.yml"
APPLICATION_CONFIG_DIR="{{ app_config_dir}}"
APPLICATION_CMD="java {{ additional_java_args }} -jar ${APPLICATION_HOME}/${APPLICATION_JAR} server ${APPLICATION_CONFIG_DIR}/${APPLICATION_CONFIG}"
APPLICATION_SHUTDOWN_WAIT=120

dropwizard_pid() {
    echo `ps aux | grep "${APPLICATION_CMD}" | grep -v grep | awk '{ print $2 }'`
}

start() {
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]
    then
        echo "${APPLICATION_NAME} is already running (pid: $pid)"
    else
        # Start dropwizard
        echo "Starting ${APPLICATION_NAME}"
        su ${APPLICATION_USER} -c "cd ${APPLICATION_HOME}; ${APPLICATION_CMD} > /dev/null &"
    fi
    return 0
}

stop() {
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]
    then
        echo "Stopping ${APPLICATION_NAME}"
        kill $pid

    let kwait=$APPLICATION_SHUTDOWN_WAIT
    count=0
    count_by=5
    until [ `ps -p $pid | grep -c $pid` = '0' ] || [ $count -gt $kwait ]
    do
        echo "Waiting for processes to exit. Timeout before we kill the pid: ${count}/${kwait}"
        sleep $count_by
        let count=$count+$count_by;
    done

    if [ $count -gt $kwait ]; then
        echo "Killing processes which didn't stop after ${APPLICATION_SHUTDOWN_WAIT} seconds"
        kill -9 $pid
    fi
    else
        echo "${APPLICATION_NAME} is not running"
    fi

    return 0
}

status(){
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]; then
        echo "${APPLICATION_NAME} is running with pid: $pid"
    else
        echo "${APPLICATION_NAME} is not running"
    exit 1
    fi
}


case $1 in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    status)
        status
        ;;
    *)
    echo "Usage: $0 {start|stop|restart|status}"
    exit 1
    ;;

esac

exit 0
  • Finally, we can start the application by:

service <package> start

Jenkinsfile – Credentials Binding Plugin – sshUserPrivateKey

Example below shows how to use the sshUserPrivateKey of a server to do Git clone in Jenkinsfile:

node(‘jenkins_node’){

stage(‘checkout code’){

//Private key for server is stored in Jenkins with id ‘private_key_for_server’ and can be accessible via variable ‘private_key’

withCredentials([sshUserPrivateKey(credentialsId: ‘private_key_for _server’, keyFileVariable: ‘private_key’, passphraseVariable: ”, usernameVariable: ”)]){

// start ssh-agent
sh ‘ssh-agent /bin/bash’

// add private key to ssh-agent, check if private key is successfully added and git clone using the private key

sh ‘eval $(ssh-agent) && ssh-add ${private_key} && ssh-add -l && git clone git@git.test.com:test.git’

}

}

}

 

How To Trigger Downstream Job And Pass In Parameters

This is a Jenkinsfile that:

  • Takes in parameter “test_branch”
  • Runs Job-1 and pass in parameter “test_branch” to it
  • Upon successful run of Job-1, it will trigger Job-2 and pass in the “test_branch” parameter
  • Note that Job-1 and Job-2 are 2 separate Jenkins jobs, and the Jenkinsfile below belongs to the Jenkins job that triggers both Job-1 and Job-2

node(DOCKER_IMAGE){

deleteDir()

checkout([$class: ‘GitSCM’, branches: [[name: “${test_branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs:              [[credentialsId: ‘abc’, url: GIT_URL]]])

stage(‘job 1’){

build job: ‘Job-1’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}
stage(‘job 2’){

build job: ‘Job-2’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}

« Older posts Newer posts »

© 2019 Chuan Chuan Law

Theme by Anders NorenUp ↑