Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Bug With Jenkins AWS Lambda Plugin

The plugin has a bug where it will fail for Lambda that takes more than 1 minute to run.



Explore Lambda alternatives such as AWS Step Functions

How To Fix GPG Signature Verification Failure During Installing RMV

Command: stable –ruby is script from


gpg: Signature made Wed Mar 25 21:58:42 2020 UTC using RSA key ID 39499BDB
gpg: Can’t check signature: public key not found
Warning, RVM 1.26.0 introduces signed releases and automated check of signatures when GPG software found. Assuming you trust Michal Papis import the mpapis public key (downloading the signatures).
GPG signature verification failed for ‘/home/jenkins/.rvm/archives/rvm-1.29.10.tgz’ – ‘’! Try to install GPG v2 and then fetch the public key:
gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
or if it fails:
command curl -sSL | gpg --import -
the key can be compared with:
NOTE: GPG version 2.1.17 have a bug which cause failures during fetching keys from remote server. Please downgrade or upgrade to newer version (if available) or use the second method described above.


gpg –keyserver hkp:// –recv-keys 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

Using Ansible To Display Message When Logged In

– name: Modify to include your message
    dest: /etc/
    group: root
    owner: root
    mode: ‘0644’

– name: Uncomment Banner on sshd_config
    path: /etc/ssh/sshd_config
    regexp: ‘#Banner’
    line: ‘Banner /etc/’
    owner: root
    group: root
    mode: ‘0600’

– name: restart ssh service
    name: ssh
    state: restarted

DevOpsDays NYC 2020


I am delighted to attend DevOpsDays NYC 2020 held on March 3 – 4, 2020 at the New York Academy of Medicine . I wanted to share about my experience for the 2 days, so we go.

Day #1

The location was sort of far in the upper town of Manhattan from where I live, Brooklyn. However, after getting off the subway, the walk to the venue through Central Park was nice.

I arrived around 10am and only attended the last bit of this part before coffee break.

Next was a series of Ignite Talks which comprised of multiple 5 minutes talks. The key takeaways from these are:

  • We should modularize notebooks for productionalizing data science models. Make them maintainable using modules and versions. Decouple and specialize child modules
  • Incorporate security tools into CI/CD pipeline
  • Resilience engineering is a community, and also means we should adapt to changes and learn from other industries such as medical, aviation, etc.

After the Ignite Talks was lunch. It was fine apart of the part that it is lacking of vegan options. I ended up having pasta with cheese. 😐

After lunch was Open Spaces where attendees get to suggest the topics that they want to talk about. There will be a subject matter expertise to facilitate each talk.

I selected these 3 topics and with the following takeways:


  • Bake security into process and tools
  • Automate as much as you can
  • Secure driven development – use tools to check flaws in security
  • Have someone as security champion in the team
  • Plug in security checks early, before pull request


  • Do not use multi-cloud, use namespace instead
  • kubetl weakness – async and does not know when it finishes
  • Use security tools to scan images


  • Use distributed tracing
  • Cloudwatch
  • Key in tarce id
  • Logs in json format
  • Incorporate tracing before going live
  • Use auto-instrumentation
  • Use open source tools
  • Use industry standard
  • Incorporate into pull requests
  • Tracing platforms (APM) are like DataDog, NewRelic, Elastic APM

Key takeaways from afternoon sessions on CI/CD Agility and Controlling Pipeline Sprawl by Angel Rivera.

  • Avoid clear text in CI/CD
  • Use tool like Hashicorp Vault to protect passwords
  • Use random password generator to change passwords often
  • Auto rotate the passwords
  • Pipeline in YAML format
  • 1 pipeline in 1 repo is not a good practice
  • Do not hardcode in pipeline, use scripts
  • Create vendor libraries for reusability
  • Minimize vendor lock in

Day #2

James Meickle – Cooperative Economics for Engineers; or, Why you have more in common with Pirate Fleets Than With Your Manager

Key takeaways from Ignite Talks

  • DevOps principle – has to have production mindset
  • Is K8 really necessary? Automate everything, test twice, change architecture instead
  • All tech is debt, people are gold – stop building new technology
  • When software incident happens, mitigate or rollback 1st, learn from it, and practice (drills)

Next were Open Spaces. I went to a salary negotiation, learning from software incidents and talk pay sessions.

Below are key takeaways from salary negotiation:

  • Do not give a number in the initial interview process
  • Focus on how you can give value to the company
  • Have multiple offers
  • Negotiate at the end of the interview process
  • Its hard to negotiate in the same company

Learning from software incidents:

  • Incident are operational surprises
  • When there is a problem, implement more metrics and have processes in place to prevent the problem
  • Test more
  • Think of different ways a problem could have happened
  • Learn from things that did not fail, how we did it right

Open space #3 was interesting as it had attendees to enter their base salaries based on dev, ops, or others (qa) regardless of experience levels. This session had the most attendees for obvious reason.

The ranges vary widely from 5 to high 6 digits.

Key takeaways from afternoon sessions:

  • Product management is customer focus, provides strategy + vision, allignment+leadership
  • Product = Customer * Business * Technology
  • Product managers gathers requirements, syhnthesize feedback, prioritize against business goals and broadcast value
  • Name your services and be specific, says what it does
  • Version your API, have clear documentations and examples
  • Update runbook regularly
  • Alerts for SLO level
  • After alert is triggered, tune it, see patterns and prune
  • All alerts should be actionable
  • Need to understand business impact of the alerts
  • DevOps should be low context, carefully constructing defaults, have ubiquitous documentation, document as much as you can

It was a very productive conference as it is relevant to what I do. Looking forward for another DevOpsDay!

Jenkins – How To Automate Credentials Creation

Below is how to create a Jenkins new credentials of type Username & Password via Jenkins API using Ansible

– name: Automatically create Jenkins username & password credentials
body: |
“”: “0”,
“credentials”: {
“scope”: “GLOBAL”,
“id”: “abcdefg”,
“username”: “”,
“password”: “{{ testuser_password }}”,
“description”: “test jenkins credentials”,
“$class”: “com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl”
follow_redirects: all
force_basic_auth: true
method: POST
password: “{{ jenkins_admin_password }}”
return_content: true
url: “{{ jenkins_url }}/credentials/store/system/domain/_/createCredentials”
user: “admin”
validate_certs: false

How To Fix “Cannot get CSRF” When Installing Jenkins-Plugin Using Ansible

Previous Ansible installation with jenkins_plugin will break with newer version of Jenkins. The workaround is as below before the jenkins_plugin step:

– name: disable csrf so we can install plugin
path: /etc/default/jenkins
regexp: ‘JAVA_ARGS=”- Djava.awt.headless=true’
line: JAVA_ARGS=”- Djava.awt.headless=true –”

– name: restart jenkins
name: jenkins
state: restarted

– pause:
minutes: 1

– name: Install plugins
name: “{{ item }}”
url_username: admin
url_password: “{{ jenkins_admin_password }}”
url: “{{ jenkins_url }}”
timeout: 90
– artifactory

How To Fix Jenkins JaCoCo Plugin Does Not Take Exclusions From Maven Pom.xml File Issue


In pom.xml, add the exclusion arguments as properties:


In Jenkinsfile add the following script to read the excluded properties. Eg: getMavenExclusionProps.groovy

def call(context) {
context.with {
def pom = readMavenPom file: ‘pom.xml’
props = readMavenPom().getProperties()
exclusionPattern = props.entrySet().findAll {entry -> entry.key.startsWith(‘coverage-exclusion-‘)}.collect{it.value}.join(‘,’)
return exclusionPattern

Pass the excluded properties variable when you call JaCoco test:

jacoco(execPattern: ‘**/target/jacoco.exec’, exclusionPattern: “${excludeProps}”)

How To Fix Passenger + Nginx Issue On Ubuntu 18


Nginx will not start due to the following error on Ubuntu 18:

nginx: [emerg] unknown directive "passenger_enabled" in /etc/nginx/sites-enabled/default:25

nginx: configuration file /etc/nginx/nginx.conf test failed


Install Passenger + Nginx module

apt-get install -y libnginx-mod-http-passenger

Add a line on top of /etc/nginx/nginx.conf

load_module /usr/lib/nginx/modules/;

Restart Nginx

service nginx restart

Terraform – External Data source

This can be used to pull data from external.


data “external” “download” {
program = [“${path.module}/”, “${var.filename_zip}”]


filename=$(curl -O $FILE)
echo -n “{\”Downloaded zip file\”:\”${filename}\”}”


variable “filename” {
description = “Filename for lambda zip file”
default = “”

Transitioning From SDET To DevOps

I started my career as a junior QA to Software Engineer In Test in Australia before a slight change in my IT career into a DevOps engineer in NYC.


I would like to share about the technical skills that could be transferred from SDET and what extra skills that need to be picked up in order to be a DevOps engineer.

Transferable skills

  • Programming/automation

The programming skills from writing automated tests will be helpful in DevOps as part of the job requires programming to automate processes.

SDET will use more programming language such as Java, Ruby, etc but DevOps will use more of shell and bash script.

  • CI/CD or Deployment

SDET’s involvement in deployment pipeline automation with tools like Jenkins is definitely a core part of a DevOps engineer. The only difference is that a SDET will usually use Jenkins to set up the automated test build or integrate it into the CI/CD pipeline.

DevOps’s involvement will be helping the development team with building the entire pipeline from compilation till deployment

  • Tools

SDET uses lots of Open Source tools such as Selenium, Cucumber to develop test automation frameworks.

DevOps’s job leverage a lot on tools as well, but different sets. We will discuss more about this in the “Skills to be picked up” section.

  • General computing/system knowledge

General computer knowledge of operation systems etc will be used but more in depth in a DevOps’s role.

Skills to be picked up

  • New automation tool – configuration management tools

Configuration management tools such as Ansible and Puppet are a key part for DevOps to automate deployment and server configuration.

  • New tools

A major portion of a DevOp’s tasks involves installation, configuration, setting up, upgrading and managing a bunch of tools used by the development tools. A list of it include:

  1. Jenkins
  2. Git
  3. Artifactory
  4. Docker
  5. Kubernetes
  6. Nginx
  7. Consul
  8. Hashicorp
  9. Elastic Search
  10. Sonarqube
  11. New Relic
  12. Datadog
  • More in-depth operating system

A lot of DevOps’s work involve in troubleshooting system issues. Therefore, knowledge of the operation system in use such as Linux, Ubuntu, RedHat, etc is very important.

  • Cloud

Working with data centers or more applicable nowadays are the cloud such as AWS and Google Cloud.

That will include usage of related tools such as Terraform.


Both SDET and DevOps are exciting jobs. I would not recommend one over the other. However for those who want to transition, it’s definitely not a difficult task. There are transferable skills which you could leverage but also new skills to learn. Learning new skills is unavoidable in the IT world anyway as technology keeps on evolving.

« Older posts

© 2020 Chuan Chuan Law

Theme by Anders NorenUp ↑