Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Category: Ansible

Top 10 DevOps Tools Or Services Used

November 7, 2020 / / 0 Comments

Below are the list of top 10 tools I use on a daily basis in my job

  1. Configuration management tool

Configuration management tool – Ansible takes up 80% to 90% of my daily life. All servers provisioning, software installation and management are automated using it. Automation with configuration management tool allows repetition on multiple servers and avoids human error.

2. Jenkins

All software compilation, build and deploys are automated on Jenkins. Includes, writing Jenkins pipeline, installing, upgrading Jenkins and its plugins.

3. AWS

This is where all the servers and resources are. EC2, DNS and other services like Elastic Search etc.

4. Terraform

This is used in to provision the services and resources in AWS. I view it as the configuration management tool of AWS that allows repetition and eliminates human error.

5. Elastic Search

This is where all the logs go to. Maintenance work such as automating snapshot, backup and curator clean up are part of the job.

6. Operating system

System administrating work on operating systems like Ubuntu. Diagnosing, troubleshooting issues, installing and upgrading packages.

7. Nginx

Load balancing for applications and services.

8. Docker

Containerization has become important these days due to cost savings, therefore most servers are shifted towards being provisioned in Docker.

9. Monitoring tools

Integrating monitoring software into the applications, services and databases using services such as New Relic, AppDynamics and Datadog.

10. Hashicorp Vault

Used to store all secrets and sensitive information of applications.

Installing Tomcat From Zip In Debian Format

July 17, 2020 / / 0 Comments

Introduction

If you need to install a Tomcat that version that is not available in the Debian format, usually the newer versions, and would like to mimic the Debian installation, we can do the following steps either manually or automated via tools like Ansible.

---

- name: create tomcat7 group
  group: 
    name: tomcat7
    gid: 3013
    state: present


- name: create tomcat7 user
  user: 
    name: tomcat7
    group: tomcat7
    uid: 3013
    shell: /bin/bash
 


- name: reconstruct tomcat7 directories
  file: 
    path: "{{ item.dir }}"
    state: directory
    owner: "{{ item.owner }}"
    group: "{{ item.group }}"
    mode: "{{ item.mode }}"
  with_items:
    - { dir:/usr/share/tomcat7/bin,owner:root,group: root,mode: '0755' }
    - { dir:/usr/share/tomcat7/lib,owner:root,group: root,mode:'0755'  }
    - { dir:/usr/share/tomcat7-admin,owner:root,group:root,mode:'0755' }
    - { dir:/var/lib/tomcat7, owner: root, group: root, mode: '0755'   }
    - { dir:/etc/tomcat7, owner:root, group:tomcat7, mode:'0755'       }
    - { dir:/etc/tomcat7/policy.d,owner:root,group:tomcat7,mode:'0755' }
    - { dir:/var/cache/tomcat7,owner:tomcat7,group:adm,mode: '0750'    }
    - { dir:/var/log/tomcat7,owner:tomcat7,group:tomcat7,mode: '0644'  }


- name: Install prerequisites for Trusty
  apt:
    name: "{{ item }}"
    state: present
  with_items:
    - libpostgresql-jdbc-java
    - tomcat7-common


- name: download tomcat7 zip
  get_url:
    url: http://mirrors.advancedhosters.com/apache/tomcat/tomcat-7/{{tomcat7.pkgversion}}/bin/apache-tomcat-{{tomcat7.pkgversion}}.zip
    dest: /tmp


- name: unarchive tomcat7 zip file
  unarchive:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}.zip
    dest: /tmp
    remote_src: yes


- name: copy tomcat7 library files to /usr/share/java
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/lib/{{ item.src }}.jar
    dest: /usr/share/java/{{ item.dest }}-{{tomcat7.pkgversion}}.jar
    owner: root
    group: root
    mode: '0644'
    remote_src: yes
  with_items:
    - { src: annotations-api, dest: tomcat-annotations-api }
    - { src: catalina, dest: tomcat-catalina               }
    - { src: catalina-ant, dest: catalina-ant              }
    - { src: catalina-ha, dest: tomcat-catalina-ha         }
    - { src: catalina-tribes, dest: catalina-tribes        }
    - { src: ecj-4.4.2, dest: ecj-4.4.2                    }
    - { src: el-api, dest: el-api-2.2                      }
    - { src: jasper-el, dest: tomcat-jasper-el             }
    - { src: jasper, dest: tomcat-jasper                   }
    - { src: jsp-api, dest: jsp-api-2.2                    }
    - { src: servlet-api, dest: servlet-api-3.0            }
    - { src: tomcat-api, dest: tomcat-api                  }
    - { src: tomcat-coyote, dest: tomcat-coyote            }
    - { src: tomcat-i18n-de, dest: tomcat-i18n-de          }
    - { src: tomcat-i18n-es, dest: tomcat-i18n-es          }
    - { src: tomcat-i18n-fr, dest: tomcat-i18n-fr          }
    - { src: tomcat-i18n-ja, dest: tomcat-i18n-ja          }
    - { src: tomcat-i18n-ko, dest: tomcat-i18n-ko          }
    - { src: tomcat-i18n-ru, dest: tomcat-i18n-ru          }
    - { src: tomcat-jdbc, dest: tomcat-jdbc                }
    - { src: tomcat-util, dest: tomcat-util                }


- name: copy tomcat-juli bin files to /usr/share/java
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/bin/tomcat-juli.jar
    dest: /usr/share/java/tomcat-juli-{{tomcat7.pkgversion}}.jar
    owner: root
    group: root
    mode: '0644'
    remote_src: yes


- name: create symlink for tomcat7 library files in /usr/share/java
  file:
    src: /usr/share/java/{{ item.src }}-{{tomcat7.pkgversion}}.jar
    dest: "/usr/share/java/{{ item.dest }}.jar"
    owner: root
    group: root
    state: link
  with_items:
    - { src: tomcat-annotations-api, dest: tomcat-annotations-api }
    - { src: tomcat-catalina, dest: tomcat-catalina               }
    - { src: catalina-ant, dest: catalina-ant                     }
    - { src: tomcat-catalina-ha, dest: tomcat-catalina-ha         }
    - { src: catalina-tribes, dest: catalina-tribes               }
    - { src: el-api-2.2, dest: tomcat-el-api-2.2                  }
    - { src: ecj-4.4.2, dest: ecj                                 }
    - { src: ecj-4.4.2, dest: eclipse-ecj                         }
    - { src: tomcat-jasper-el, dest: tomcat-jasper-el             }
    - { src: tomcat-jasper, dest: tomcat-jasper                   }
    - { src: jsp-api-2.2, dest: tomcat-jsp-api-2.2                }
    - { src: servlet-api-3.0, dest: tomcat-servlet-api-3.0        }
    - { src: tomcat-api, dest: tomcat-api                         }
    - { src: tomcat-coyote, dest: tomcat-coyote                   }
    - { src: tomcat-i18n-de, dest: tomcat-i18n-de                 }
    - { src: tomcat-i18n-es, dest: tomcat-i18n-es                 }
    - { src: tomcat-i18n-fr, dest: tomcat-i18n-fr                 }
    - { src: tomcat-i18n-ja, dest: tomcat-i18n-ja                 }
    - { src: tomcat-i18n-ko, dest: tomcat-i18n-ko                 }
    - { src: tomcat-i18n-ru, dest: tomcat-i18n-ru                 }
    - { src: tomcat-jdbc , dest: tomcat-jdbc                      }
    - { src: tomcat-util, dest: tomcat-util                       }
    - { src: tomcat-juli, dest: tomcat-juli                       }


- name: copy tomcat7 library files to /usr/share/tomcat7/lib
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/lib/{{ item.src }}.jar
    dest: /usr/share/tomcat7/lib/{{ item.dest }}.jar
    owner: root
    group: root
    mode: '0644'
    remote_src: yes
  with_items:
    - { src: tomcat7-websocket, dest: tomcat7-websocket }
    - { src: websocket-api, dest: websocket-api         }
    - { src: tomcat-dbcp, dest: tomcat-dbcp             }


- name: create symlink for tomcat7 library files in /usr/share/tomcat7/lib
  file:
    src: /usr/share/java/{{ item.src }}-{{tomcat7.pkgversion}}.jar
    dest: /usr/share/tomcat7/lib/{{ item.dest }}.jar
    owner: root
    group: root
    state: link
  with_items:
    - { src: tomcat-annotations-api, dest: annotations-api         }
    - { src: catalina-ant, dest: catalina-ant                      }
    - { src: tomcat-catalina-ha, dest: catalina-ha                 }
    - { src: tomcat-catalina, dest: catalina                       }
    - { src: tomcat-catalina-jmx-remote, dest: catalina-jmx-remote }
    - { src: catalina-tribes, dest: catalina-tribes                }
    - { src: tomcat-jasper-el, dest: jasper-el                     }
    - { src: tomcat-jasper, dest: jasper                           }
    - { src: tomcat-jdbc, dest: java-tomcat-jdbc                   }
    - { src: tomcat-api, dest: tomcat-api                          }
    - { src: tomcat-coyote, dest: tomcat-coyote                    }
    - { src: tomcat-i18n-de, dest: tomcat-i18n-de                  }
    - { src: tomcat-i18n-es, dest: tomcat-i18n-es                  }
    - { src: tomcat-i18n-fr, dest: tomcat-i18n-fr                  }
    - { src: tomcat-i18n-ja, dest: tomcat-i18n-ja                  }
    - { src: tomcat-i18n-ko, dest: tomcat-i18n-ko                  }
    - { src: tomcat-i18n-ru, dest: tomcat-i18n-ru                  }
    - { src: tomcat-jdbc, dest: tomcat-jdbc                        }
    - { src: tomcat-util, dest: tomcat-util                        }


- name: create symlink for java library files in /usr/share/tomcat7/lib
  file:
    src: /usr/share/java/{{ item.src }}.jar
    dest: /usr/share/tomcat7/lib/{{ item.dest }}.jar
    owner: root
    group: root
    state: link
  with_items:
    - { src: commons-dbcp, dest: commons-dbcp           }
    - { src: commons-pool, dest: commons-pool           }
    - { src: tomcat-servlet-api-3.0, dest: servlet-api  }
    - { src: tomcat-el-api-2.2, dest: el-api            }
    - { src: tomcat-jsp-api-2.2, dest: jsp-api          }



- name: copy tomcat7 bin files to /usr/share/tomcat7/bin
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/bin/{{ item.src }}
    dest: /usr/share/tomcat7/bin/{{ item.dest }}
    owner: root
    group: root
    mode: "{{ item.mode }}"
    remote_src: yes
  with_items:
    - { src: bootstrap.jar, dest: bootstrap.jar, mode: '0644'          }
    - { src: catalina.sh, dest: catalina.sh, mode: '0755'              }
    - { src: catalina-tasks.xml, dest: catalina-tasks.xml, mode: '0644'}
    - { src: configtest.sh, dest: configtest.sh, mode: '0755'          }
    - { src: daemon.sh, dest: daemon.sh,  mode: '0755'                 }
    - { src: digest.sh, dest: digest.sh, mode: '0755'                  }
    - { src: setclasspath.sh, dest: setclasspath.sh, mode: '0755'      }
    - { src: shutdown.sh, dest: shutdown.sh, mode: '0755'              }
    - { src: startup.sh, dest: startup.sh, mode: '0755'                }
    - { src: tool-wrapper.sh, dest: tool-wrapper.sh, mode: '0755'      }
    - { src: version.sh, dest: version.sh, mode: '0755'                }



- name: create symlink for java library files in /usr/share/tomcat7/bin
  file:
    src: /usr/share/java/tomcat-juli.jar
    dest: /usr/share/tomcat7/bin/tomcat-juli.jar
    owner: root
    group: root
    state: link


- name: copy tomcat7 conf files to /etc/tomcat7
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/conf/{{ item.src }}
    dest: /etc/tomcat7/{{ item.dest }}
    owner: root
    group: tomcat7
    mode: "{{ item.mode }}"
    remote_src: yes
  with_items:
    - { src: catalina.properties, dest: catalina.properties, mode:'0640'}
    - { src: context.xml, dest: context.xml, mode: '0664'              }  
    - { src: logging.properties, dest: logging.properties, mode: '0640'}
    - { src: server.xml, dest: server.xml, mode: '0664'                }
    - { src: tomcat-users.xml, dest: tomcat-users.xml, mode: '0640'    }
    - { src: tomcat-users.xsd, dest: tomcat-users.xsd, mode: '0640'    }
    - { src: web.xml, dest: web.xml, mode: '0640'                      }


- name: copy tomcat7 catalina policy to /var/cache
  copy:
    src: /tmp/apache-tomcat-{{tomcat7.pkgversion}}/conf/catalina.policy
    dest: /var/cache/tomcat7/catalina.policy
    owner: root
    group: root
    mode: '0644'
    remote_src: yes


- name: create symlink for tomcat files in /var/lib/tomcat7
  file:
    src: "{{ item.src }}"
    dest: "{{ item.dest }}"
    owner: root
    group: root
    state: link
  with_items:
    - { src: /etc/tomcat7, dest: /var/lib/tomcat7/conf       }
    - { src: /var/log/tomcat7, dest: /var/lib/tomcat7/logs   }
    - { src: /var/cache/tomcat7, dest: /var/lib/tomcat7/work }


- name: copy over tomcat7 policies
  copy:
    src: "{{ item }}"
    dest: /var/lib/tomcat7/conf/policy.d/{{ item }}
    owner: root
    group: root
    mode: '0640'
  with_items:
    - 01system.policy
    - 02debian.policy
    - 03catalina.policy
    - 04webapps.policy
    - 50local.policy


- name: copy tomcat7 start script
  copy:
    src: tomcat7_init
    dest: /etc/init.d/tomcat7
    owner: root
    group: root
    mode: '0755'


- name: copy tomcat7 admin files to /usr/share/tomcat7-admin
  shell: cp -r /tmp/apache-tomcat-{{tomcat7.pkgversion}}/webapps/* /usr/share/tomcat7-admin/

Using Ansible To Display Message When Logged In

April 9, 2020 / / 0 Comments

—
– name: Modify issue.net to include your message
  copy:
    src: issue.net
    dest: /etc/issue.net
    group: root
    owner: root
    mode: ‘0644’

– name: Uncomment Banner on sshd_config
  lineinfile:
    path: /etc/ssh/sshd_config
    regexp: ‘#Banner’
    line: ‘Banner /etc/issue.net’
    owner: root
    group: root
    mode: ‘0600’

– name: restart ssh service
  service:
    name: ssh
    state: restarted

Jenkins – How To Automate Credentials Creation

February 7, 2020 / / 0 Comments

Below is how to create a Jenkins new credentials of type Username & Password via Jenkins API using Ansible

– name: Automatically create Jenkins username & password credentials
uri:
body: |
json={
“”: “0”,
“credentials”: {
“scope”: “GLOBAL”,
“id”: “abcdefg”,
“username”: “testuser@jenkins.com”,
“password”: “{{ testuser_password }}”,
“description”: “test jenkins credentials”,
“$class”: “com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl”
}
}
follow_redirects: all
force_basic_auth: true
method: POST
password: “{{ jenkins_admin_password }}”
return_content: true
url: “{{ jenkins_url }}/credentials/store/system/domain/_/createCredentials”
user: “admin”
validate_certs: false

How To Write Jenkinsfile

November 1, 2017 / / 0 Comments

Jenkinsfile is another great feature from Jenkins2.

Below is an example of a Jenkinsfile:

properties(

[             

   //Parameters of a Jenkins build  
parameters(
[
text(defaultValue: ”, description: ‘URL’, name: ‘ARTIFACT’),
choice(choices: ‘qa’, description: ‘Deploy_Env’, name: ‘DEPLOY_ENV’),
string(defaultValue: ‘master’ , description: ‘ Branch’,name:’BRANCH’)
]
)
]
)

//Which node the job should run on

node(‘master’){

//Delete directory before job starts

deleteDir()

//Git checkout certain branch using defined Git credentials

checkout([$class: ‘GitSCM’, branches: [[name: “${branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: ‘abc’, url: GIT_URL]]])

//Name of which stage of task that is running
stage(‘deploy’){

//Credentials with secret file configured in Jenkins

withCredentials([file(credentialsId: ‘PASS’, variable: ‘FILE’)]) {

//Execute shell script

sh ‘ansible-galaxy install -r requirements.yml –force’

//Ansible command

ansiblePlaybook(
playbook: ‘deploy.yml’,
inventory: ‘inventory/qa.inventory’,
extraVars:[
artifact_url: “${ARTIFACT}”,
],
extras: ‘–diff –vault-password-file ${FILE} –tags ${ACTION}’,
colorized: true

)

}
}

}

Enter Jenkinsfile into Jenkins2 as below:

Screen Shot 2017-10-24 at 11.14.39 AM

References on Jenkinsfile

Screen Shot 2017-10-20 at 1.28.07 PM

© 2023 Chuan Chuan Law

Theme by Anders Noren — Up ↑