DevOps | Software Automation | Continuous Integration

Category: Jenkins (Page 1 of 2)

Jenkins: SonarQube Error 400 On projectKey

Error:

WARNING: Error fetching project information
org.sonarqube.ws.client.HttpException: Error 400 on https://sonar.system.tools/api/qualitygates/project_status?projectKey=
at org.sonarqube.ws.client.BaseResponse.failIfNotSuccessful(BaseResponse.java:34)
at hudson.plugins.sonar.client.HttpClient.getHttp(HttpClient.java:36)
at hudson.plugins.sonar.client.WsClient.getQualityGate54(WsClient.java:100)
at hudson.plugins.sonar.client.SQProjectResolver.getQualityGate(SQProjectResolver.java:104)
at hudson.plugins.sonar.client.SQProjectResolver.resolve(SQProjectResolver.java:66)

Solution:

Upgrade SonarQube Scanner Jenkins plugin. Versions 2.8+ will be able to fix the error above.

Jenkins – HTML Publisher Plugin – No CSS is displayed

Problem:

HTML is not displayed properly in the report.

Causes:

CSS is stripped out due to Content Security Policy.

https://wiki.jenkins-ci.org/display/JENKINS/Configuring+Content+Security+Policy

Solution:

Go to the following:

  • Manage Jenkins
  • Manage Nodes
  • Settings
  • Script Console

System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "")

  • Run

Drawback:

The above fix will be gone after a Jenkins restart. To make the fix permanent, add the following to JAVA_ARGS under /etc/default/jenkins:

-Dhudson.model.DirectoryBrowserSupport.CSP=""

Top 10 DevOps Tools Or Services Used

Below are the list of top 10 tools I use on a daily basis in my job

  1. Configuration management tool

Configuration management tool – Ansible takes up 80% to 90% of my daily life. All servers provisioning, software installation and management are automated using it. Automation with configuration management tool allows repetition on multiple servers and avoids human error.

2. Jenkins

All software compilation, build and deploys are automated on Jenkins. Includes, writing Jenkins pipeline, installing, upgrading Jenkins and its plugins.

3. AWS

This is where all the servers and resources are. EC2, DNS and other services like Elastic Search etc.

4. Terraform

This is used in to provision the services and resources in AWS. I view it as the configuration management tool of AWS that allows repetition and eliminates human error.

5. Elastic Search

This is where all the logs go to. Maintenance work such as automating snapshot, backup and curator clean up are part of the job.

6. Operating system

System administrating work on operating systems like Ubuntu. Diagnosing, troubleshooting issues, installing and upgrading packages.

7. Nginx

Load balancing for applications and services.

8. Docker

Containerization has become important these days due to cost savings, therefore most servers are shifted towards being provisioned in Docker.

9. Monitoring tools

Integrating monitoring software into the applications, services and databases using services such as New Relic, AppDynamics and Datadog.

10. Hashicorp Vault

Used to store all secrets and sensitive information of applications.

Docker – Jenkins – Get Sensitive Data From AWS SSM

Introduction

For Dockerized Jenkins, we can use the following method to store/get sensitive data from AWS SSM instead of hardcoding it in Docker file

Store secret in AWS SSM:

aws ssm put-parameter –name “artifactory_password” –value “thisisapassword” –type “String”

In Dockerfile:

COPY settings.xml /etc/maven/settings.xml

COPY jenkins-slave /home/jenkins
RUN chmod -R 755 /home/jenkins/jenkins-slave

In settings.xml

<settings xmlns=”http://maven.apache.org/SETTINGS/1.0.0″ xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd”>
  <servers>
    <server>
      <id>artifactory_password</id>
      <username>username</username>
      <password>blank</password>
    </server>

</servers>
  <pluginGroups> <pluginGroup>com.amashchenko.maven.plugin.ondeck</pluginGroup>
  </pluginGroups>
</settings>

In jenkins-slave


#!/bin/bash


# The MIT License
#
#  Copyright (c) 2015, CloudBees, Inc.
#
#  Permission is hereby granted, free of charge, to any person obtaining a copy
#  of this software and associated documentation files (the “Software”), to deal
#  in the Software without restriction, including without limitation the rights
#  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#  copies of the Software, and to permit persons to whom the Software is
#  furnished to do so, subject to the following conditions:
#
#  The above copyright notice and this permission notice shall be included in
#  all copies or substantial portions of the Software.
#
#  THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
#  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
#  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
#  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
#  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
#  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
#  THE SOFTWARE.


# take from https://github.com/cloudbees/jnlp-slave-with-java-build-tools-dockerfile/blob/master/jenkins-slave
# assumed that these env vars are present on the kubernetes slave


if [[ $# -eq 1 ]]; then


    # if `docker run` only has one arguments, we assume user is running alternate command like `bash` to inspect the image
    exec “$@”


else


    # if -tunnel is not provided try env vars
    if [[ “$@” != *”-tunnel “* ]]; then
        if [[ ! -z “$JENKINS_TUNNEL” ]]; then
            TUNNEL=”-tunnel $JENKINS_TUNNEL”        
        fi
    fi


    if [[ ! -z “$JENKINS_URL” ]]; then
        URL=”-url $JENKINS_URL”
    fi


    sed -i “s/blank/$(aws ssm get-parameters –region us-east-1 –with-decryption –names artifactory_password –query ‘Parameters[0].Value’ –output text )/” /etc/maven/settings.xml
   
    gosu jenkins java $JAVA_OPTS -Duser.home=/var/lib/jenkins -cp /home/jenkins/slave.jar hudson.remoting.jnlp.Main -headless $URL $JENKINS_SECRET $JENKINS_NAME “$@”
fi  

Configure home/jenkins/jenkins-slave as the entry point of the Jenkins container

Jenkins – How To Automate Credentials Creation

Below is how to create a Jenkins new credentials of type Username & Password via Jenkins API using Ansible

– name: Automatically create Jenkins username & password credentials
uri:
body: |
json={
“”: “0”,
“credentials”: {
“scope”: “GLOBAL”,
“id”: “abcdefg”,
“username”: “testuser@jenkins.com”,
“password”: “{{ testuser_password }}”,
“description”: “test jenkins credentials”,
“$class”: “com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl”
}
}
follow_redirects: all
force_basic_auth: true
method: POST
password: “{{ jenkins_admin_password }}”
return_content: true
url: “{{ jenkins_url }}/credentials/store/system/domain/_/createCredentials”
user: “admin”
validate_certs: false

How To Fix “Cannot get CSRF” When Installing Jenkins-Plugin Using Ansible

Previous Ansible installation with jenkins_plugin will break with newer version of Jenkins. The workaround is as below before the jenkins_plugin step:

– name: disable csrf so we can install plugin
lineinfile:
path: /etc/default/jenkins
regexp: ‘JAVA_ARGS=”- Djava.awt.headless=true’
line: JAVA_ARGS=”- Djava.awt.headless=true – Dhudson.security.csrf.DefaultCrumbIssuer.EXCLUDE_SESSION_ID=true”


– name: restart jenkins
service:
name: jenkins
state: restarted


– pause:
minutes: 1

– name: Install plugins
jenkins_plugin:
name: “{{ item }}”
url_username: admin
url_password: “{{ jenkins_admin_password }}”
url: “{{ jenkins_url }}”
timeout: 90
with_items:
– artifactory

How To Fix Jenkins JaCoCo Plugin Does Not Take Exclusions From Maven Pom.xml File Issue

Issue: https://issues.jenkins-ci.org/browse/JENKINS-15570

In pom.xml, add the exclusion arguments as properties:

<coverage-exclusion-01>org/jooq/generated/**/*</coverage-exclusion-01>
<coverage-exclusion-02>**/Application.*</coverage-exclusion-02>

In Jenkinsfile add the following script to read the excluded properties. Eg: getMavenExclusionProps.groovy

def call(context) {
context.with {
def pom = readMavenPom file: ‘pom.xml’
props = readMavenPom().getProperties()
exclusionPattern = props.entrySet().findAll {entry -> entry.key.startsWith(‘coverage-exclusion-‘)}.collect{it.value}.join(‘,’)
return exclusionPattern
}
}

Pass the excluded properties variable when you call JaCoco test:

jacoco(execPattern: ‘**/target/jacoco.exec’, exclusionPattern: “${excludeProps}”)

Jenkins – How To Manually Upgrade Jenkins Plugin To Specific Versions

Introduction

Jenkins Plugin Manager will enable plugin installation of the latest version. Sometimes we might want to install plugin of a specific version, instead of the latest. To do that, we can do the following:

Steps

  • Older versions of plugins can be found here: https://updates.jenkins.io/download/plugins/
  • The plugins are in .hpi file extensions
  • Download the version you want and put it on the Jenkins server
  • Plugins are stored in /var/lib/jenkins
  • Backup – you might want to backup the current version of the plugins before upgrading. Each plugin has a directory and a .jpi file.

For eg: datadog plugin

mkdir datadog_backup

mv datadog* /datadog_backup

  • Place the .hpi file in /var/lib/jenkins
  • Restart Jenkins

service jenkins restart

  • A directory with the plugin name and a .jpi file will be created upon restart
  • Check if the plugin with the correct version appears on Jenkins website Manage Jenkins->Plugin Manager
  • If not, check the log on the server in /var/log/jenkins 
  • Log is usually called jenkins.log
  • Log will show the plugin installation failure as SEVERE
  • Usually this is due to version dependencies of other plugins as named in the log.

For eg:

SEVERE: Failed Loading plugin sauce-ondemand
java.io.IOException: Dependency workflow-job (1.15), workflow-cps (1.15), workflow-basic-steps (1.15), workflow-step-api (1.15) doesn’t exist

  • Download and install the missing dependencies plugin via the same method as above
  • Upon successful installation, you will see the plugin with the right version appearing in Manage Jenkins->Plugin Manager

Jenkins – config.xml

If there is a need to debug Jenkins configuration and the when the interface set up looks correct, it’s better to find more detailed configuration information via the config.xml.

  1. In the server where Jenkins is installed, usually in path /var/lob/jenkins/jobs
  2. Browse to your job folder
  3. There will be a config.xml which will contain the job configuration details
« Older posts

© 2023 Chuan Chuan Law

Theme by Anders NorenUp ↑