Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Category: DevOps

General Guideline To Take While Doing System Upgrade

Upgrading system and applications are important and major part of a DevOps role. Below are the general guideline to take while performing an upgrade.

Take a snapshot

Take a snapshot, create a backup, or image before an upgrade is very important in case we need to rollback.

Practice the upgrade process

Work out the process on a development environment, staging or test box before doing the actual upgrade, especially if it is for production environment.

Automate as much as you can

Try to automate the upgrade as much as you can, so we can repeat the process on other boxes or environment.

Make notes

Take notes especially on things that we cannot automate .

Have a runbook

List all the automation and manual steps to run in a runbook, which we can follow during the upgrade process

Prepare a rollback strategy

Ensure we know the steps to rollback if needed

Have peer reviews

Get peers or seniors to review the automated steps and runbook.

Have a helpline or support ready

Have a helpline or support ready during upgrade in case things go south during upgrade

Starting Dropwizard Application Using Jar Packaging & SysV

By default Debian package will generate an Upstart and SysV script.

This blog will show how we can ship an application in Jar file and start it using SysV

  • Put your jar file somewhere. E.g: /opt
  • Copy the Debian package generated config in /etc into the new app config directory. You might need to reformat the YAML file a bit.

For example:

Some vars might look like this:

timeout: ${sys.TIMEOUT!”5000ms” }

We want to change it to look like:

timeout: 5000ms

  • Uninstall Debian packages

apt-get purge <package>

  • Delete the generated Upstart (/etc/init/<package>.conf) and SysV script (/etc/init.d/<package>)
  • Create a SysV script which looks like this:
#!/bin/bash
#
# dropwizard     This shell script takes care of starting and stopping Dropwizard applications
#
# chkconfig: - 80 20
#
### BEGIN INIT INFO
# Provides: dropwizard
# Required-Start: $network $syslog
# Required-Stop: $network $syslog
# Default-Start:
# Default-Stop:
# Short-Description: start and stop dropwizard
### END INIT INFO

#You just need to replace the {{ var }} below for your application
APPLICATION_NAME="{{ app_name }}"
APPLICATION_USER="{{ app_user }}"
APPLICATION_HOME="{{ app_home }}"
APPLICATION_JAR="{{ app_name }}.jar"
APPLICATION_CONFIG="{{ app_name }}.yml"
APPLICATION_CONFIG_DIR="{{ app_config_dir}}"
APPLICATION_CMD="java {{ additional_java_args }} -jar ${APPLICATION_HOME}/${APPLICATION_JAR} server ${APPLICATION_CONFIG_DIR}/${APPLICATION_CONFIG}"
APPLICATION_SHUTDOWN_WAIT=120

dropwizard_pid() {
    echo `ps aux | grep "${APPLICATION_CMD}" | grep -v grep | awk '{ print $2 }'`
}

start() {
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]
    then
        echo "${APPLICATION_NAME} is already running (pid: $pid)"
    else
        # Start dropwizard
        echo "Starting ${APPLICATION_NAME}"
        su ${APPLICATION_USER} -c "cd ${APPLICATION_HOME}; ${APPLICATION_CMD} > /dev/null &"
    fi
    return 0
}

stop() {
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]
    then
        echo "Stopping ${APPLICATION_NAME}"
        kill $pid

    let kwait=$APPLICATION_SHUTDOWN_WAIT
    count=0
    count_by=5
    until [ `ps -p $pid | grep -c $pid` = '0' ] || [ $count -gt $kwait ]
    do
        echo "Waiting for processes to exit. Timeout before we kill the pid: ${count}/${kwait}"
        sleep $count_by
        let count=$count+$count_by;
    done

    if [ $count -gt $kwait ]; then
        echo "Killing processes which didn't stop after ${APPLICATION_SHUTDOWN_WAIT} seconds"
        kill -9 $pid
    fi
    else
        echo "${APPLICATION_NAME} is not running"
    fi

    return 0
}

status(){
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]; then
        echo "${APPLICATION_NAME} is running with pid: $pid"
    else
        echo "${APPLICATION_NAME} is not running"
    exit 1
    fi
}


case $1 in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    status)
        status
        ;;
    *)
    echo "Usage: $0 {start|stop|restart|status}"
    exit 1
    ;;

esac

exit 0
  • Finally, we can start the application by:

service <package> start

© 2019 Chuan Chuan Law

Theme by Anders NorenUp ↑