DevOps | Software Automation | Continuous Integration

Author: Chuan Chuan Law (Page 1 of 14)

Top 10 DevOps Tools Or Services Used

Below are the list of top 10 tools I use on a daily basis in my job

  1. Configuration management tool

Configuration management tool – Ansible takes up 80% to 90% of my daily life. All servers provisioning, software installation and management are automated using it. Automation with configuration management tool allows repetition on multiple servers and avoids human error.

2. Jenkins

All software compilation, build and deploys are automated on Jenkins. Includes, writing Jenkins pipeline, installing, upgrading Jenkins and its plugins.

3. AWS

This is where all the servers and resources are. EC2, DNS and other services like Elastic Search etc.

4. Terraform

This is used in to provision the services and resources in AWS. I view it as the configuration management tool of AWS that allows repetition and eliminates human error.

5. Elastic Search

This is where all the logs go to. Maintenance work such as automating snapshot, backup and curator clean up are part of the job.

6. Operating system

System administrating work on operating systems like Ubuntu. Diagnosing, troubleshooting issues, installing and upgrading packages.

7. Nginx

Load balancing for applications and services.

8. Docker

Containerization has become important these days due to cost savings, therefore most servers are shifted towards being provisioned in Docker.

9. Monitoring tools

Integrating monitoring software into the applications, services and databases using services such as New Relic, AppDynamics and Datadog.

10. Hashicorp Vault

Used to store all secrets and sensitive information of applications.

AWS Elastic Search Snapshot & Restore

Create s3 repo

  • Create a s3 repo in AWS
  • Create a policy and attach it to a role

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Action”: [
“s3:ListBucket”
],
“Effect”: “Allow”,
“Resource”: [
“arn:aws:s3:::snapshot_s3_repositorty”
]
},
{
“Action”: [
“s3:GetObject”,
“s3:PutObject”,
“s3:DeleteObject”
],
“Effect”: “Allow”,
“Resource”: [
“arn:aws:s3:::snapshot_s3_repository/*”
]
}
]
}

  • Establish a trust relationship for the snapshot role to allow access from Elastic Search service

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Sid”: “”,
“Effect”: “Allow”,
“Principal”: {
“Service”: “es.amazonaws.com”
},
“Action”: “sts:AssumeRole”
}
]
}

Create a AWS user

  • Allow the user to use the snapshot role to snapshot/restore onto s3 and bunch of ES actions

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: “iam:PassRole”,
“Resource”: “arn:aws:iam::123456789:role/elasticsearch_snapshot_role”
},
{
“Effect”: “Allow”,
“Action”: [
“es:ESHttpPut”,
“es:ESHttpGet”,
“es:ESHttpPost”,
“es:ESHttpDelete”
],
“Resource”: “arn:aws:es:us-east-1:123456789:domain/mydomain/*”
}
]
}

Register the snapshot repo

  • This is a shell command inside a Jenkins job
  • SECRET and ACCESS are the credentials for the AWS user created above and injected as passwords into the environment variables

!/bin/sh
pip install requests-aws4auth
pip install boto3
cat << EOF >> /home/jenkins/.aws/credentials
[default]
region=us-east-1
aws_access_key_id = $SECRET
aws_secret_access_key = $ACCESS
EOF
cat /home/jenkins/.aws/credentials
cat << ‘EOF’ >> register_es_snapshot.py
import boto3
import requests
from requests_aws4auth import AWS4Auth
host = ‘https://host.es.amazonaws.com/’
region = ‘us-east-1’
service = ‘es’
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)
Register repository
path = ‘_snapshot/my_es_snapshot’ # the Elasticsearch API endpoint
url = host + path
payload = {
“type”: “s3”,
“settings”: {
“bucket”: “snapshot_s3_repositorty”,
“region”: “us-east-1”,
“role_arn”: “arn:aws:iam::123456789:role/elasticsearch_snapshot_role”
}
}
headers = {“Content-Type”: “application/json”}
r = requests.put(url, auth=awsauth, json=payload, headers=headers)
print(r.status_code)
print(r.text)
EOF
chmod 755 register_es_snapshot.py
python register_es_snapshot.py

Creating a snapshot or backup

  • An error will occur if we use the same snapshot name, so $now is used to make snapshot name unique

!/bin/sh
pip install requests-aws4auth
pip install boto3
cat << EOF >> /home/jenkins/.aws/credentials
[default]
region=us-east-1
aws_access_key_id = $SECRET
aws_secret_access_key = $ACCESS
EOF
cat /home/jenkins/.aws/credentials
cat << ‘EOF’ >> backup_es_snapshot.py
import boto3
import requests
from requests_aws4auth import AWS4Auth
host = ‘https://host.us-east-1.es.amazonaws.com/’
region = ‘us-east-1’
service = ‘es’
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)
headers = {“Content-Type”: “application/json”}
Take snapshot
path = ‘_snapshot/my_es_snapshot/my_snapshot_$now’
url = host + path
r = requests.get(url, auth=awsauth)
print(r.status_code)
print(r.text)
EOF
chmod 755 backup_es_snapshot.py
python backup_es_snapshot.py

Restore single index

  • INDEX is a string parameter passed into the Jenkins job. For e.g: “logstash-2020.07.11”

!/bin/sh
pip install requests-aws4auth
pip install boto3
cat << EOF >> /home/jenkins/.aws/credentials
[default]
region=us-east-1
aws_access_key_id = $SECRET
aws_secret_access_key = $ACCESS
EOF
cat /home/jenkins/.aws/credentials
cat << EOF >> restore_single_index_es_snapshot.py
import boto3
import requests
from requests_aws4auth import AWS4Auth
host = ‘https://host.es.amazonaws.com/’
region = ‘us-east-1’
service = ‘es’
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)
headers = {“Content-Type”: “application/json”}
Restore snapshot (one index)
path = ‘_snapshot/my_es_snapshot/my_snapshot/_restore’
url = host + path
payload = {“indices”: “$INDEX”}
headers = {“Content-Type”: “application/json”}
r = requests.post(url, auth=awsauth, json=payload, headers=headers)
print(r.status_code)
print(r.text)
EOF
chmod 755 restore_single_index_es_snapshot.py
python restore_single_index_es_snapshot.py

Restore all indices

!/bin/sh
pip install requests-aws4auth
pip install boto3
cat << EOF >> /home/jenkins/.aws/credentials
[default]
region=us-east-1
aws_access_key_id = $SECRET
aws_secret_access_key = $ACCESS
EOF
cat /home/jenkins/.aws/credentials
cat << EOF >> restore_all_indices_es_snapshot.py
import boto3
import requests
from requests_aws4auth import AWS4Auth
host = ‘https://host.es.amazonaws.com/’
region = ‘us-east-1’
service = ‘es’
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)
headers = {“Content-Type”: “application/json”}
# Restore snapshot (all indices except Kibana and fine-grained access control)
path = ‘_snapshot/snapshot/my_es_snapshot/my_snapshot/_restore’
url = host + path
payload = {
“indices”: “-.kibana*,-.opendistro_security”,
“include_global_state”: False
}
headers = {“Content-Type”: “application/json”}
r = requests.post(url, auth=awsauth, json=payload, headers=headers)
print(r.status_code)
print(r.text)
EOF
chmod 755 restore_all_indices_es_snapshot.py
python restore_all_indices_es_snapshot.py

Docker – Jenkins – Get Sensitive Data From AWS SSM

Introduction

For Dockerized Jenkins, we can use the following method to store/get sensitive data from AWS SSM instead of hardcoding it in Docker file

Store secret in AWS SSM:

aws ssm put-parameter –name “artifactory_password” –value “thisisapassword” –type “String”

In Dockerfile:

COPY settings.xml /etc/maven/settings.xml

COPY jenkins-slave /home/jenkins
RUN chmod -R 755 /home/jenkins/jenkins-slave

In settings.xml

<settings xmlns=”http://maven.apache.org/SETTINGS/1.0.0″ xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd”>
  <servers>
    <server>
      <id>artifactory_password</id>
      <username>username</username>
      <password>blank</password>
    </server>

</servers>
  <pluginGroups> <pluginGroup>com.amashchenko.maven.plugin.ondeck</pluginGroup>
  </pluginGroups>
</settings>

In jenkins-slave


#!/bin/bash


# The MIT License
#
#  Copyright (c) 2015, CloudBees, Inc.
#
#  Permission is hereby granted, free of charge, to any person obtaining a copy
#  of this software and associated documentation files (the “Software”), to deal
#  in the Software without restriction, including without limitation the rights
#  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
#  copies of the Software, and to permit persons to whom the Software is
#  furnished to do so, subject to the following conditions:
#
#  The above copyright notice and this permission notice shall be included in
#  all copies or substantial portions of the Software.
#
#  THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
#  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
#  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
#  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
#  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
#  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
#  THE SOFTWARE.


# take from https://github.com/cloudbees/jnlp-slave-with-java-build-tools-dockerfile/blob/master/jenkins-slave
# assumed that these env vars are present on the kubernetes slave


if [[ $# -eq 1 ]]; then


    # if `docker run` only has one arguments, we assume user is running alternate command like `bash` to inspect the image
    exec “$@”


else


    # if -tunnel is not provided try env vars
    if [[ “$@” != *”-tunnel “* ]]; then
        if [[ ! -z “$JENKINS_TUNNEL” ]]; then
            TUNNEL=”-tunnel $JENKINS_TUNNEL”        
        fi
    fi


    if [[ ! -z “$JENKINS_URL” ]]; then
        URL=”-url $JENKINS_URL”
    fi


    sed -i “s/blank/$(aws ssm get-parameters –region us-east-1 –with-decryption –names artifactory_password –query ‘Parameters[0].Value’ –output text )/” /etc/maven/settings.xml
   
    gosu jenkins java $JAVA_OPTS -Duser.home=/var/lib/jenkins -cp /home/jenkins/slave.jar hudson.remoting.jnlp.Main -headless $URL $JENKINS_SECRET $JENKINS_NAME “$@”
fi  

Configure home/jenkins/jenkins-slave as the entry point of the Jenkins container

Installing Tomcat From Zip In Debian Format

Introduction

If you need to install a Tomcat that version that is not available in the Debian format, usually the newer versions, and would like to mimic the Debian installation, we can do the following steps either manually or automated via tools like Ansible.

---

- name: create tomcat7 group
  group: 
    name: tomcat7
    gid: 3013
    state: present


- name: create tomcat7 user
  user: 
    name: tomcat7
    group: tomcat7
    uid: 3013
    shell: /bin/bash
 


- name: reconstruct tomcat7 directories
  file: 
    path: "{{ item.dir }}"
    state: directory
    owner: "{{ item.owner }}"
    group: "{{ item.group }}"
    mode: "{{ item.mode }}"
  with_items:
    - { dir:/usr/share/tomcat7/bin,owner:root,group: root,mode: '0755' }
    - { dir:/usr/share/tomcat7/lib,owner:root,group: root,mode:'0755'  }
    - { dir:/usr/share/tomcat7-admin,owner:root,group:root,mode:'0755' }
    - { dir:/var/lib/tomcat7, owner: root, group: root, mode: '0755'   }
    - { dir:/etc/tomcat7, owner:root, group:tomcat7, mode:'0755'       }
    - { dir:/etc/tomcat7/policy.d,owner:root,group:tomcat7,mode:'0755' }
    - { dir:/var/cache/tomcat7,owner:tomcat7,group:adm,mode: '0750'    }
    - { dir:/var/log/tomcat7,owner:tomcat7,group:tomcat7,mode: '0644'  }


- name: Install prerequisites for Trusty
  apt:
    name: "{{ item }}"
    state: present
  with_items:
    - libpostgresql-jdbc-java
    - tomcat7-common


- name: download tomcat7 zip
  get_url:
    url: http://mirrors.advancedhosters.com/apache/tomcat/tomcat-7/{{tomcat7.pkgversion}}/bin/apache-tomcat-{{tomcat7.pkgversion}}.zip
    dest: /tmp


- name: unarchive tomcat7 zip file
  unarchive:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}.zip
    dest: /tmp
    remote_src: yes


- name: copy tomcat7 library files to /usr/share/java
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/lib/{{ item.src }}.jar
    dest: /usr/share/java/{{ item.dest }}-{{tomcat7.pkgversion}}.jar
    owner: root
    group: root
    mode: '0644'
    remote_src: yes
  with_items:
    - { src: annotations-api, dest: tomcat-annotations-api }
    - { src: catalina, dest: tomcat-catalina               }
    - { src: catalina-ant, dest: catalina-ant              }
    - { src: catalina-ha, dest: tomcat-catalina-ha         }
    - { src: catalina-tribes, dest: catalina-tribes        }
    - { src: ecj-4.4.2, dest: ecj-4.4.2                    }
    - { src: el-api, dest: el-api-2.2                      }
    - { src: jasper-el, dest: tomcat-jasper-el             }
    - { src: jasper, dest: tomcat-jasper                   }
    - { src: jsp-api, dest: jsp-api-2.2                    }
    - { src: servlet-api, dest: servlet-api-3.0            }
    - { src: tomcat-api, dest: tomcat-api                  }
    - { src: tomcat-coyote, dest: tomcat-coyote            }
    - { src: tomcat-i18n-de, dest: tomcat-i18n-de          }
    - { src: tomcat-i18n-es, dest: tomcat-i18n-es          }
    - { src: tomcat-i18n-fr, dest: tomcat-i18n-fr          }
    - { src: tomcat-i18n-ja, dest: tomcat-i18n-ja          }
    - { src: tomcat-i18n-ko, dest: tomcat-i18n-ko          }
    - { src: tomcat-i18n-ru, dest: tomcat-i18n-ru          }
    - { src: tomcat-jdbc, dest: tomcat-jdbc                }
    - { src: tomcat-util, dest: tomcat-util                }


- name: copy tomcat-juli bin files to /usr/share/java
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/bin/tomcat-juli.jar
    dest: /usr/share/java/tomcat-juli-{{tomcat7.pkgversion}}.jar
    owner: root
    group: root
    mode: '0644'
    remote_src: yes


- name: create symlink for tomcat7 library files in /usr/share/java
  file:
    src: /usr/share/java/{{ item.src }}-{{tomcat7.pkgversion}}.jar
    dest: "/usr/share/java/{{ item.dest }}.jar"
    owner: root
    group: root
    state: link
  with_items:
    - { src: tomcat-annotations-api, dest: tomcat-annotations-api }
    - { src: tomcat-catalina, dest: tomcat-catalina               }
    - { src: catalina-ant, dest: catalina-ant                     }
    - { src: tomcat-catalina-ha, dest: tomcat-catalina-ha         }
    - { src: catalina-tribes, dest: catalina-tribes               }
    - { src: el-api-2.2, dest: tomcat-el-api-2.2                  }
    - { src: ecj-4.4.2, dest: ecj                                 }
    - { src: ecj-4.4.2, dest: eclipse-ecj                         }
    - { src: tomcat-jasper-el, dest: tomcat-jasper-el             }
    - { src: tomcat-jasper, dest: tomcat-jasper                   }
    - { src: jsp-api-2.2, dest: tomcat-jsp-api-2.2                }
    - { src: servlet-api-3.0, dest: tomcat-servlet-api-3.0        }
    - { src: tomcat-api, dest: tomcat-api                         }
    - { src: tomcat-coyote, dest: tomcat-coyote                   }
    - { src: tomcat-i18n-de, dest: tomcat-i18n-de                 }
    - { src: tomcat-i18n-es, dest: tomcat-i18n-es                 }
    - { src: tomcat-i18n-fr, dest: tomcat-i18n-fr                 }
    - { src: tomcat-i18n-ja, dest: tomcat-i18n-ja                 }
    - { src: tomcat-i18n-ko, dest: tomcat-i18n-ko                 }
    - { src: tomcat-i18n-ru, dest: tomcat-i18n-ru                 }
    - { src: tomcat-jdbc , dest: tomcat-jdbc                      }
    - { src: tomcat-util, dest: tomcat-util                       }
    - { src: tomcat-juli, dest: tomcat-juli                       }


- name: copy tomcat7 library files to /usr/share/tomcat7/lib
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/lib/{{ item.src }}.jar
    dest: /usr/share/tomcat7/lib/{{ item.dest }}.jar
    owner: root
    group: root
    mode: '0644'
    remote_src: yes
  with_items:
    - { src: tomcat7-websocket, dest: tomcat7-websocket }
    - { src: websocket-api, dest: websocket-api         }
    - { src: tomcat-dbcp, dest: tomcat-dbcp             }


- name: create symlink for tomcat7 library files in /usr/share/tomcat7/lib
  file:
    src: /usr/share/java/{{ item.src }}-{{tomcat7.pkgversion}}.jar
    dest: /usr/share/tomcat7/lib/{{ item.dest }}.jar
    owner: root
    group: root
    state: link
  with_items:
    - { src: tomcat-annotations-api, dest: annotations-api         }
    - { src: catalina-ant, dest: catalina-ant                      }
    - { src: tomcat-catalina-ha, dest: catalina-ha                 }
    - { src: tomcat-catalina, dest: catalina                       }
    - { src: tomcat-catalina-jmx-remote, dest: catalina-jmx-remote }
    - { src: catalina-tribes, dest: catalina-tribes                }
    - { src: tomcat-jasper-el, dest: jasper-el                     }
    - { src: tomcat-jasper, dest: jasper                           }
    - { src: tomcat-jdbc, dest: java-tomcat-jdbc                   }
    - { src: tomcat-api, dest: tomcat-api                          }
    - { src: tomcat-coyote, dest: tomcat-coyote                    }
    - { src: tomcat-i18n-de, dest: tomcat-i18n-de                  }
    - { src: tomcat-i18n-es, dest: tomcat-i18n-es                  }
    - { src: tomcat-i18n-fr, dest: tomcat-i18n-fr                  }
    - { src: tomcat-i18n-ja, dest: tomcat-i18n-ja                  }
    - { src: tomcat-i18n-ko, dest: tomcat-i18n-ko                  }
    - { src: tomcat-i18n-ru, dest: tomcat-i18n-ru                  }
    - { src: tomcat-jdbc, dest: tomcat-jdbc                        }
    - { src: tomcat-util, dest: tomcat-util                        }


- name: create symlink for java library files in /usr/share/tomcat7/lib
  file:
    src: /usr/share/java/{{ item.src }}.jar
    dest: /usr/share/tomcat7/lib/{{ item.dest }}.jar
    owner: root
    group: root
    state: link
  with_items:
    - { src: commons-dbcp, dest: commons-dbcp           }
    - { src: commons-pool, dest: commons-pool           }
    - { src: tomcat-servlet-api-3.0, dest: servlet-api  }
    - { src: tomcat-el-api-2.2, dest: el-api            }
    - { src: tomcat-jsp-api-2.2, dest: jsp-api          }



- name: copy tomcat7 bin files to /usr/share/tomcat7/bin
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/bin/{{ item.src }}
    dest: /usr/share/tomcat7/bin/{{ item.dest }}
    owner: root
    group: root
    mode: "{{ item.mode }}"
    remote_src: yes
  with_items:
    - { src: bootstrap.jar, dest: bootstrap.jar, mode: '0644'          }
    - { src: catalina.sh, dest: catalina.sh, mode: '0755'              }
    - { src: catalina-tasks.xml, dest: catalina-tasks.xml, mode: '0644'}
    - { src: configtest.sh, dest: configtest.sh, mode: '0755'          }
    - { src: daemon.sh, dest: daemon.sh,  mode: '0755'                 }
    - { src: digest.sh, dest: digest.sh, mode: '0755'                  }
    - { src: setclasspath.sh, dest: setclasspath.sh, mode: '0755'      }
    - { src: shutdown.sh, dest: shutdown.sh, mode: '0755'              }
    - { src: startup.sh, dest: startup.sh, mode: '0755'                }
    - { src: tool-wrapper.sh, dest: tool-wrapper.sh, mode: '0755'      }
    - { src: version.sh, dest: version.sh, mode: '0755'                }



- name: create symlink for java library files in /usr/share/tomcat7/bin
  file:
    src: /usr/share/java/tomcat-juli.jar
    dest: /usr/share/tomcat7/bin/tomcat-juli.jar
    owner: root
    group: root
    state: link


- name: copy tomcat7 conf files to /etc/tomcat7
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/conf/{{ item.src }}
    dest: /etc/tomcat7/{{ item.dest }}
    owner: root
    group: tomcat7
    mode: "{{ item.mode }}"
    remote_src: yes
  with_items:
    - { src: catalina.properties, dest: catalina.properties, mode:'0640'}
    - { src: context.xml, dest: context.xml, mode: '0664'              }  
    - { src: logging.properties, dest: logging.properties, mode: '0640'}
    - { src: server.xml, dest: server.xml, mode: '0664'                }
    - { src: tomcat-users.xml, dest: tomcat-users.xml, mode: '0640'    }
    - { src: tomcat-users.xsd, dest: tomcat-users.xsd, mode: '0640'    }
    - { src: web.xml, dest: web.xml, mode: '0640'                      }


- name: copy tomcat7 catalina policy to /var/cache
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/conf/catalina.policy
    dest: /var/cache/tomcat7/catalina.policy
    owner: root
    group: root
    mode: '0644'
    remote_src: yes


- name: create symlink for tomcat files in /var/lib/tomcat7
  file:
    src: "{{ item.src }}"
    dest: "{{ item.dest }}"
    owner: root
    group: root
    state: link
  with_items:
    - { src: /etc/tomcat7, dest: /var/lib/tomcat7/conf       }
    - { src: /var/log/tomcat7, dest: /var/lib/tomcat7/logs   }
    - { src: /var/cache/tomcat7, dest: /var/lib/tomcat7/work }


- name: copy over tomcat7 policies
  copy:
    src: "{{ item }}"
    dest: /var/lib/tomcat7/conf/policy.d/{{ item }}
    owner: root
    group: root
    mode: '0640'
  with_items:
    - 01system.policy
    - 02debian.policy
    - 03catalina.policy
    - 04webapps.policy
    - 50local.policy


- name: copy tomcat7 start script
  copy:
    src: tomcat7_init
    dest: /etc/init.d/tomcat7
    owner: root
    group: root
    mode: '0755'


- name: copy tomcat7 admin files to /usr/share/tomcat7-admin
  shell: cp -r /tmp/apache-tomcat-{{tomcat7.pkgversion}}/webapps/* /usr/share/tomcat7-admin/
  

How To Fix GPG Signature Verification Failure During Installing RMV

Command:

rvminstall.sh stable –ruby

rvminstall.sh is script from https://raw.githubusercontent.com/rvm/rvm/master/binscripts/rvm-installer

Error:

Downloading https://github.com/rvm/rvm/archive/1.29.10.tar.gz
Downloading https://github.com/rvm/rvm/releases/download/1.29.10/1.29.10.tar.gz.asc
gpg: Signature made Wed Mar 25 21:58:42 2020 UTC using RSA key ID 39499BDB
gpg: Can’t check signature: public key not found
Warning, RVM 1.26.0 introduces signed releases and automated check of signatures when GPG software found. Assuming you trust Michal Papis import the mpapis public key (downloading the signatures).
GPG signature verification failed for ‘/home/jenkins/.rvm/archives/rvm-1.29.10.tgz’ – ‘https://github.com/rvm/rvm/releases/download/1.29.10/1.29.10.tar.gz.asc’! Try to install GPG v2 and then fetch the public key:
gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
or if it fails:
command curl -sSL https://rvm.io/mpapis.asc | gpg --import -
the key can be compared with:
https://rvm.io/mpapis.asc https://keybase.io/mpapis
NOTE: GPG version 2.1.17 have a bug which cause failures during fetching keys from remote server. Please downgrade or upgrade to newer version (if available) or use the second method described above.

Fix:

gpg –keyserver hkp://keys.gnupg.net –recv-keys 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

Using Ansible To Display Message When Logged In


– name: Modify issue.net to include your message
  copy:
    src: issue.net
    dest: /etc/issue.net
    group: root
    owner: root
    mode: ‘0644’

– name: Uncomment Banner on sshd_config
  lineinfile:
    path: /etc/ssh/sshd_config
    regexp: ‘#Banner’
    line: ‘Banner /etc/issue.net’
    owner: root
    group: root
    mode: ‘0600’

– name: restart ssh service
  service:
    name: ssh
    state: restarted

DevOpsDays NYC 2020

Introduction

I am delighted to attend DevOpsDays NYC 2020 held on March 3 – 4, 2020 at the New York Academy of Medicine . I wanted to share about my experience for the 2 days, so we go.

Day #1

The location was sort of far in the upper town of Manhattan from where I live, Brooklyn. However, after getting off the subway, the walk to the venue through Central Park was nice.

I arrived around 10am and only attended the last bit of this part before coffee break.

Next was a series of Ignite Talks which comprised of multiple 5 minutes talks. The key takeaways from these are:

  • We should modularize notebooks for productionalizing data science models. Make them maintainable using modules and versions. Decouple and specialize child modules
  • Incorporate security tools into CI/CD pipeline
  • Resilience engineering is a community, and also means we should adapt to changes and learn from other industries such as medical, aviation, etc.

After the Ignite Talks was lunch. It was fine apart of the part that it is lacking of vegan options. I ended up having pasta with cheese. 😐

After lunch was Open Spaces where attendees get to suggest the topics that they want to talk about. There will be a subject matter expertise to facilitate each talk.

I selected these 3 topics and with the following takeways:

DevSecOps

  • Bake security into process and tools
  • Automate as much as you can
  • Secure driven development – use tools to check flaws in security
  • Have someone as security champion in the team
  • Plug in security checks early, before pull request

Kubernetes

  • Do not use multi-cloud, use namespace instead
  • kubetl weakness – async and does not know when it finishes
  • Use security tools to scan images

Observability/Logs

  • Use distributed tracing
  • Cloudwatch
  • Key in tarce id
  • Logs in json format
  • Incorporate tracing before going live
  • Use auto-instrumentation
  • Use open source tools
  • Use industry standard
  • Incorporate into pull requests
  • Tracing platforms (APM) are like DataDog, NewRelic, Elastic APM

Key takeaways from afternoon sessions on CI/CD Agility and Controlling Pipeline Sprawl by Angel Rivera.

  • Avoid clear text in CI/CD
  • Use tool like Hashicorp Vault to protect passwords
  • Use random password generator to change passwords often
  • Auto rotate the passwords
  • Pipeline in YAML format
  • 1 pipeline in 1 repo is not a good practice
  • Do not hardcode in pipeline, use scripts
  • Create vendor libraries for reusability
  • Minimize vendor lock in

Day #2

James Meickle – Cooperative Economics for Engineers; or, Why you have more in common with Pirate Fleets Than With Your Manager

Key takeaways from Ignite Talks

  • DevOps principle – has to have production mindset
  • Is K8 really necessary? Automate everything, test twice, change architecture instead
  • All tech is debt, people are gold – stop building new technology
  • When software incident happens, mitigate or rollback 1st, learn from it, and practice (drills)

Next were Open Spaces. I went to a salary negotiation, learning from software incidents and talk pay sessions.

Below are key takeaways from salary negotiation:

  • Do not give a number in the initial interview process
  • Focus on how you can give value to the company
  • Have multiple offers
  • Negotiate at the end of the interview process
  • Its hard to negotiate in the same company

Learning from software incidents:

  • Incident are operational surprises
  • When there is a problem, implement more metrics and have processes in place to prevent the problem
  • Test more
  • Think of different ways a problem could have happened
  • Learn from things that did not fail, how we did it right

Open space #3 was interesting as it had attendees to enter their base salaries based on dev, ops, or others (qa) regardless of experience levels. This session had the most attendees for obvious reason.

The ranges vary widely from 5 to high 6 digits.

Key takeaways from afternoon sessions:

  • Product management is customer focus, provides strategy + vision, allignment+leadership
  • Product = Customer * Business * Technology
  • Product managers gathers requirements, syhnthesize feedback, prioritize against business goals and broadcast value
  • Name your services and be specific, says what it does
  • Version your API, have clear documentations and examples
  • Update runbook regularly
  • Alerts for SLO level
  • After alert is triggered, tune it, see patterns and prune
  • All alerts should be actionable
  • Need to understand business impact of the alerts
  • DevOps should be low context, carefully constructing defaults, have ubiquitous documentation, document as much as you can

It was a very productive conference as it is relevant to what I do. Looking forward for another DevOpsDay!

Jenkins – How To Automate Credentials Creation

Below is how to create a Jenkins new credentials of type Username & Password via Jenkins API using Ansible

– name: Automatically create Jenkins username & password credentials
uri:
body: |
json={
“”: “0”,
“credentials”: {
“scope”: “GLOBAL”,
“id”: “abcdefg”,
“username”: “testuser@jenkins.com”,
“password”: “{{ testuser_password }}”,
“description”: “test jenkins credentials”,
“$class”: “com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl”
}
}
follow_redirects: all
force_basic_auth: true
method: POST
password: “{{ jenkins_admin_password }}”
return_content: true
url: “{{ jenkins_url }}/credentials/store/system/domain/_/createCredentials”
user: “admin”
validate_certs: false

« Older posts

© 2020 Chuan Chuan Law

Theme by Anders NorenUp ↑