Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Category: Jenkins (page 1 of 2)

Docker – Jenkins – Get Sensitive Data From AWS SSM

Introduction

For Dockerized Jenkins, we can use the following method to store/get sensitive data from AWS SSM instead of hardcoding it in Docker file

Store secret in AWS SSM:

aws ssm put-parameter –name “artifactory_password” –value “thisisapassword” –type “String”

In Dockerfile:

COPY settings.xml /etc/maven/settings.xml

COPY jenkins-slave /home/jenkins
RUN chmod -R 755 /home/jenkins/jenkins-slave

In settings.xml

<settings xmlns=”http://maven.apache.org/SETTINGS/1.0.0″ xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd”>
  <servers>
    <server>
      <id>artifactory_password</id>
      <username>username</username>
      <password>blank</password>
    </server>

</servers>
  <pluginGroups> <pluginGroup>com.amashchenko.maven.plugin.ondeck</pluginGroup>
  </pluginGroups>
</settings>

In jenkins-slave


#!/bin/bash


# The MIT License
#
#  Copyright (c) 2015, CloudBees, Inc.
#
#  Permission is hereby granted, free of charge, to any person obtaining a copy
#  of this software and associated documentation files (the “Software”), to deal
#  in the Software without restriction, including without limitation the rights
#  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#  copies of the Software, and to permit persons to whom the Software is
#  furnished to do so, subject to the following conditions:
#
#  The above copyright notice and this permission notice shall be included in
#  all copies or substantial portions of the Software.
#
#  THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
#  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
#  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
#  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
#  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
#  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
#  THE SOFTWARE.


# take from https://github.com/cloudbees/jnlp-slave-with-java-build-tools-dockerfile/blob/master/jenkins-slave
# assumed that these env vars are present on the kubernetes slave


if [[ $# -eq 1 ]]; then


    # if `docker run` only has one arguments, we assume user is running alternate command like `bash` to inspect the image
    exec “$@”


else


    # if -tunnel is not provided try env vars
    if [[ “$@” != *”-tunnel “* ]]; then
        if [[ ! -z “$JENKINS_TUNNEL” ]]; then
            TUNNEL=”-tunnel $JENKINS_TUNNEL”        
        fi
    fi


    if [[ ! -z “$JENKINS_URL” ]]; then
        URL=”-url $JENKINS_URL”
    fi


    sed -i “s/blank/$(aws ssm get-parameters –region us-east-1 –with-decryption –names artifactory_password –query ‘Parameters[0].Value’ –output text )/” /etc/maven/settings.xml
   
    gosu jenkins java $JAVA_OPTS -Duser.home=/var/lib/jenkins -cp /home/jenkins/slave.jar hudson.remoting.jnlp.Main -headless $URL $JENKINS_SECRET $JENKINS_NAME “$@”
fi  

Configure home/jenkins/jenkins-slave as the entry point of the Jenkins container

Bug With Jenkins AWS Lambda Plugin

https://plugins.jenkins.io/aws-lambda/

The plugin has a bug where it will fail for Lambda that takes more than 1 minute to run.

Reference: https://github.com/XT-i/aws-lambda-jenkins-plugin/issues/95

Solution:

Explore Lambda alternatives such as AWS Step Functions

Jenkins – How To Automate Credentials Creation

Below is how to create a Jenkins new credentials of type Username & Password via Jenkins API using Ansible

– name: Automatically create Jenkins username & password credentials
uri:
body: |
json={
“”: “0”,
“credentials”: {
“scope”: “GLOBAL”,
“id”: “abcdefg”,
“username”: “testuser@jenkins.com”,
“password”: “{{ testuser_password }}”,
“description”: “test jenkins credentials”,
“$class”: “com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl”
}
}
follow_redirects: all
force_basic_auth: true
method: POST
password: “{{ jenkins_admin_password }}”
return_content: true
url: “{{ jenkins_url }}/credentials/store/system/domain/_/createCredentials”
user: “admin”
validate_certs: false

How To Fix “Cannot get CSRF” When Installing Jenkins-Plugin Using Ansible

Previous Ansible installation with jenkins_plugin will break with newer version of Jenkins. The workaround is as below before the jenkins_plugin step:

– name: disable csrf so we can install plugin
lineinfile:
path: /etc/default/jenkins
regexp: ‘JAVA_ARGS=”- Djava.awt.headless=true’
line: JAVA_ARGS=”- Djava.awt.headless=true – Dhudson.security.csrf.DefaultCrumbIssuer.EXCLUDE_SESSION_ID=true”


– name: restart jenkins
service:
name: jenkins
state: restarted


– pause:
minutes: 1

– name: Install plugins
jenkins_plugin:
name: “{{ item }}”
url_username: admin
url_password: “{{ jenkins_admin_password }}”
url: “{{ jenkins_url }}”
timeout: 90
with_items:
– artifactory

How To Fix Jenkins JaCoCo Plugin Does Not Take Exclusions From Maven Pom.xml File Issue

Issue: https://issues.jenkins-ci.org/browse/JENKINS-15570

In pom.xml, add the exclusion arguments as properties:

<coverage-exclusion-01>org/jooq/generated/**/*</coverage-exclusion-01>
<coverage-exclusion-02>**/Application.*</coverage-exclusion-02>

In Jenkinsfile add the following script to read the excluded properties. Eg: getMavenExclusionProps.groovy

def call(context) {
context.with {
def pom = readMavenPom file: ‘pom.xml’
props = readMavenPom().getProperties()
exclusionPattern = props.entrySet().findAll {entry -> entry.key.startsWith(‘coverage-exclusion-‘)}.collect{it.value}.join(‘,’)
return exclusionPattern
}
}

Pass the excluded properties variable when you call JaCoco test:

jacoco(execPattern: ‘**/target/jacoco.exec’, exclusionPattern: “${excludeProps}”)

Jenkins – How To Manually Upgrade Jenkins Plugin To Specific Versions

Introduction

Jenkins Plugin Manager will enable plugin installation of the latest version. Sometimes we might want to install plugin of a specific version, instead of the latest. To do that, we can do the following:

Steps

  • Older versions of plugins can be found here: https://updates.jenkins.io/download/plugins/
  • The plugins are in .hpi file extensions
  • Download the version you want and put it on the Jenkins server
  • Plugins are stored in /var/lib/jenkins
  • Backup – you might want to backup the current version of the plugins before upgrading. Each plugin has a directory and a .jpi file.

For eg: datadog plugin

mkdir datadog_backup

mv datadog* /datadog_backup

  • Place the .hpi file in /var/lib/jenkins
  • Restart Jenkins

service jenkins restart

  • A directory with the plugin name and a .jpi file will be created upon restart
  • Check if the plugin with the correct version appears on Jenkins website Manage Jenkins->Plugin Manager
  • If not, check the log on the server in /var/log/jenkins 
  • Log is usually called jenkins.log
  • Log will show the plugin installation failure as SEVERE
  • Usually this is due to version dependencies of other plugins as named in the log.

For eg:

SEVERE: Failed Loading plugin sauce-ondemand
java.io.IOException: Dependency workflow-job (1.15), workflow-cps (1.15), workflow-basic-steps (1.15), workflow-step-api (1.15) doesn’t exist

  • Download and install the missing dependencies plugin via the same method as above
  • Upon successful installation, you will see the plugin with the right version appearing in Manage Jenkins->Plugin Manager

Jenkins – config.xml

If there is a need to debug Jenkins configuration and the when the interface set up looks correct, it’s better to find more detailed configuration information via the config.xml.

  1. In the server where Jenkins is installed, usually in path /var/lob/jenkins/jobs
  2. Browse to your job folder
  3. There will be a config.xml which will contain the job configuration details

Jenkins – How To Disable Jobs

How To Disable One Jenkins Job

  • Get breadcrumb

curl -u “<user>:<password>“ ‘https://<jenkins_url>/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,”:”,//crumb)’

 

  • curl -I -X POST https://<jenkins_url>/<job_path>/disable –user <user>:<password> -H “<jenkins bread crumb>”

How To Disable Jenkins jobs Under A Folder

  • Via Jenkins script

https://<jenkins_url>/script

  • Enter the following in the script and execute it

folderName=”Build/APP/test” //full name of the folder you want to disable all jobs in

Jenkins.instance.getItemByFullName(folderName).allJobs

.each {

      it.setDisabled(true)

      println(“Disabled job: [$it.fullName]”)

    }

null

Note: Look for file structure in server, ignore the “jobs” path in URL

Jenkinsfile – Credentials Binding Plugin – sshUserPrivateKey

Example below shows how to use the sshUserPrivateKey of a server to do Git clone in Jenkinsfile:

node(‘jenkins_node’){

stage(‘checkout code’){

//Private key for server is stored in Jenkins with id ‘private_key_for_server’ and can be accessible via variable ‘private_key’

withCredentials([sshUserPrivateKey(credentialsId: ‘private_key_for _server’, keyFileVariable: ‘private_key’, passphraseVariable: ”, usernameVariable: ”)]){

// start ssh-agent
sh ‘ssh-agent /bin/bash’

// add private key to ssh-agent, check if private key is successfully added and git clone using the private key

sh ‘eval $(ssh-agent) && ssh-add ${private_key} && ssh-add -l && git clone git@git.test.com:test.git’

}

}

}

 

How To Trigger Downstream Job And Pass In Parameters

This is a Jenkinsfile that:

  • Takes in parameter “test_branch”
  • Runs Job-1 and pass in parameter “test_branch” to it
  • Upon successful run of Job-1, it will trigger Job-2 and pass in the “test_branch” parameter
  • Note that Job-1 and Job-2 are 2 separate Jenkins jobs, and the Jenkinsfile below belongs to the Jenkins job that triggers both Job-1 and Job-2

node(DOCKER_IMAGE){

deleteDir()

checkout([$class: ‘GitSCM’, branches: [[name: “${test_branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs:              [[credentialsId: ‘abc’, url: GIT_URL]]])

stage(‘job 1’){

build job: ‘Job-1’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}
stage(‘job 2’){

build job: ‘Job-2’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}

« Older posts

© 2020 Chuan Chuan Law

Theme by Anders NorenUp ↑