DevOps | Software Automation | Continuous Integration

Category: General (Page 1 of 3)

DevOpsDays NYC 2020


I am delighted to attend DevOpsDays NYC 2020 held on March 3 – 4, 2020 at the New York Academy of Medicine . I wanted to share about my experience for the 2 days, so we go.

Day #1

The location was sort of far in the upper town of Manhattan from where I live, Brooklyn. However, after getting off the subway, the walk to the venue through Central Park was nice.

I arrived around 10am and only attended the last bit of this part before coffee break.

Next was a series of Ignite Talks which comprised of multiple 5 minutes talks. The key takeaways from these are:

  • We should modularize notebooks for productionalizing data science models. Make them maintainable using modules and versions. Decouple and specialize child modules
  • Incorporate security tools into CI/CD pipeline
  • Resilience engineering is a community, and also means we should adapt to changes and learn from other industries such as medical, aviation, etc.

After the Ignite Talks was lunch. It was fine apart of the part that it is lacking of vegan options. I ended up having pasta with cheese. 😐

After lunch was Open Spaces where attendees get to suggest the topics that they want to talk about. There will be a subject matter expertise to facilitate each talk.

I selected these 3 topics and with the following takeways:


  • Bake security into process and tools
  • Automate as much as you can
  • Secure driven development – use tools to check flaws in security
  • Have someone as security champion in the team
  • Plug in security checks early, before pull request


  • Do not use multi-cloud, use namespace instead
  • kubetl weakness – async and does not know when it finishes
  • Use security tools to scan images


  • Use distributed tracing
  • Cloudwatch
  • Key in tarce id
  • Logs in json format
  • Incorporate tracing before going live
  • Use auto-instrumentation
  • Use open source tools
  • Use industry standard
  • Incorporate into pull requests
  • Tracing platforms (APM) are like DataDog, NewRelic, Elastic APM

Key takeaways from afternoon sessions on CI/CD Agility and Controlling Pipeline Sprawl by Angel Rivera.

  • Avoid clear text in CI/CD
  • Use tool like Hashicorp Vault to protect passwords
  • Use random password generator to change passwords often
  • Auto rotate the passwords
  • Pipeline in YAML format
  • 1 pipeline in 1 repo is not a good practice
  • Do not hardcode in pipeline, use scripts
  • Create vendor libraries for reusability
  • Minimize vendor lock in

Day #2

James Meickle – Cooperative Economics for Engineers; or, Why you have more in common with Pirate Fleets Than With Your Manager

Key takeaways from Ignite Talks

  • DevOps principle – has to have production mindset
  • Is K8 really necessary? Automate everything, test twice, change architecture instead
  • All tech is debt, people are gold – stop building new technology
  • When software incident happens, mitigate or rollback 1st, learn from it, and practice (drills)

Next were Open Spaces. I went to a salary negotiation, learning from software incidents and talk pay sessions.

Below are key takeaways from salary negotiation:

  • Do not give a number in the initial interview process
  • Focus on how you can give value to the company
  • Have multiple offers
  • Negotiate at the end of the interview process
  • Its hard to negotiate in the same company

Learning from software incidents:

  • Incident are operational surprises
  • When there is a problem, implement more metrics and have processes in place to prevent the problem
  • Test more
  • Think of different ways a problem could have happened
  • Learn from things that did not fail, how we did it right

Open space #3 was interesting as it had attendees to enter their base salaries based on dev, ops, or others (qa) regardless of experience levels. This session had the most attendees for obvious reason.

The ranges vary widely from 5 to high 6 digits.

Key takeaways from afternoon sessions:

  • Product management is customer focus, provides strategy + vision, allignment+leadership
  • Product = Customer * Business * Technology
  • Product managers gathers requirements, syhnthesize feedback, prioritize against business goals and broadcast value
  • Name your services and be specific, says what it does
  • Version your API, have clear documentations and examples
  • Update runbook regularly
  • Alerts for SLO level
  • After alert is triggered, tune it, see patterns and prune
  • All alerts should be actionable
  • Need to understand business impact of the alerts
  • DevOps should be low context, carefully constructing defaults, have ubiquitous documentation, document as much as you can

It was a very productive conference as it is relevant to what I do. Looking forward for another DevOpsDay!

Terraform – External Data source

This can be used to pull data from external.


data “external” “download” {
program = [“${path.module}/”, “${var.filename_zip}”]


filename=$(curl -O $FILE)
echo -n “{\”Downloaded zip file\”:\”${filename}\”}”


variable “filename” {
description = “Filename for lambda zip file”
default = “”

Transitioning From SDET To DevOps

I started my career as a junior QA to Software Engineer In Test in Australia before a slight change in my IT career into a DevOps engineer in NYC.


I would like to share about the technical skills that could be transferred from SDET and what extra skills that need to be picked up in order to be a DevOps engineer.

Transferable skills

  • Programming/automation

The programming skills from writing automated tests will be helpful in DevOps as part of the job requires programming to automate processes.

SDET will use more programming language such as Java, Ruby, etc but DevOps will use more of shell and bash script.

  • CI/CD or Deployment

SDET’s involvement in deployment pipeline automation with tools like Jenkins is definitely a core part of a DevOps engineer. The only difference is that a SDET will usually use Jenkins to set up the automated test build or integrate it into the CI/CD pipeline.

DevOps’s involvement will be helping the development team with building the entire pipeline from compilation till deployment

  • Tools

SDET uses lots of Open Source tools such as Selenium, Cucumber to develop test automation frameworks.

DevOps’s job leverage a lot on tools as well, but different sets. We will discuss more about this in the “Skills to be picked up” section.

  • General computing/system knowledge

General computer knowledge of operation systems etc will be used but more in depth in a DevOps’s role.

Skills to be picked up

  • New automation tool – configuration management tools

Configuration management tools such as Ansible and Puppet are a key part for DevOps to automate deployment and server configuration.

  • New tools

A major portion of a DevOp’s tasks involves installation, configuration, setting up, upgrading and managing a bunch of tools used by the development tools. A list of it include:

  1. Jenkins
  2. Git
  3. Artifactory
  4. Docker
  5. Kubernetes
  6. Nginx
  7. Consul
  8. Hashicorp
  9. Elastic Search
  10. Sonarqube
  11. New Relic
  12. Datadog
  • More in-depth operating system

A lot of DevOps’s work involve in troubleshooting system issues. Therefore, knowledge of the operation system in use such as Linux, Ubuntu, RedHat, etc is very important.

  • Cloud

Working with data centers or more applicable nowadays are the cloud such as AWS and Google Cloud.

That will include usage of related tools such as Terraform.


Both SDET and DevOps are exciting jobs. I would not recommend one over the other. However for those who want to transition, it’s definitely not a difficult task. There are transferable skills which you could leverage but also new skills to learn. Learning new skills is unavoidable in the IT world anyway as technology keeps on evolving.

General Guideline To Take While Doing System Upgrade

Upgrading system and applications are important and major part of a DevOps role. Below are the general guideline to take while performing an upgrade.

Take a snapshot

Take a snapshot, create a backup, or image before an upgrade is very important in case we need to rollback.

Practice the upgrade process

Work out the process on a development environment, staging or test box before doing the actual upgrade, especially if it is for production environment.

Automate as much as you can

Try to automate the upgrade as much as you can, so we can repeat the process on other boxes or environment.

Make notes

Take notes especially on things that we cannot automate .

Have a runbook

List all the automation and manual steps to run in a runbook, which we can follow during the upgrade process

Prepare a rollback strategy

Ensure we know the steps to rollback if needed

Have peer reviews

Get peers or seniors to review the automated steps and runbook.

Have a helpline or support ready

Have a helpline or support ready during upgrade in case things go south during upgrade

Hello from New York!


I haven’t been blogging for a while. This is because I have been busy settling in the Big Apple!

It was a big move and I have been very blessed to have the opportunity to explore the other side of the world.

I look forward to continue sharing what I have learned here. 🙂


Getting Selenium Test Running On Gradle Framework

Gradle + Selenium is probably not a popular framework yet when I was developing it as I find it difficult to look for relevant resources online. There were bits and pieces of information here and there which I then glue then up together.

My build.gradle contains the following implementations:

(1) Add cucumberRuntime under Configurations

configurations {
cucumberRuntime {
extendsFrom testRuntime


(2) Source sets

srcDirs ‘src/test’


(3) Compile test output

task testJar(type: Jar){
include ‘**/*.class’
exclude ‘**/*StepDef*.class’


(4) Glue everything together

task cucumber() {
    dependsOn assemble, compileTestJava
    doLast {
        javaexec {
            main = "cucumber.api.cli.Main"
            classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
            args = ['-f', 'json:target/cucumber-report.json','--plugin','pretty', '--glue', 'src/test/groovy/au/com/story/steps', 'src/test/groovy/au/com/story/features']

Evolution Of Software Testing

I was lucky enough to be able to present in the Melbourne Selenium Meetup which was held at on the 11th of December, 2014. 
Below is the script of my speech with the title Evolution Of Software Testing.


Thanks to Ray for inviting me to the MeetUp. I am definitely not the most experienced here, just to share with everybody the changes in testing that I have observed in the past 10 years, which you may be experiencing too and my personal experiences in the journey.


I would like to share my experience in terms of these 4 areas.
(1) Transition from traditional to Agile
(2) Importance of automation 
(3) Battling between time and quality in a world when we need to deploy fast 
(4) What do we need to do to keep up in a rapid changing industry

Waterfall to Agile

Most of our changes started after the adoption of Agile SDLC over Waterfall

Testing is part of software engineering

Testing no longer is a phase in SDLC. It is part of software engineering. What does that mean?

Bug catching at every phase

Bug catching is not done at the final phase before deployment. Testing starts at early phase, during planning or design, before development starts.

Real Life Scenario

My experience with the transition from Traditional to Agile is that it helps to deliver better quality results and on time as it eliminates the back and forth bug catching and fixing which will cost more towards the end of a development phase. Testing bit by bit is gives a much better result than testing in a big chunk. This also helps to ensure that the product is what the stakeholder wants.
I heard about a real life scenario where Company X invested in a 4 year traditional software project. When the stakeholder 1st time sees it, it was a shock as it was not the product they expected.
During Waterfall, testing is the final phase before production and testers are always being pressured to test fast in order to deliver on time. My worst experience is to receive a project that would need a few days of testing at 3pm and needs to be deployed the next day. As a result of that, we have to work overtime, or take out features that have show stopper bugs. PMs tend to give a lot of pressure to the testers back in those days.

Get rid of the documentations

We no longer need to write piles of test cases or test plans.

Automation is the key

Automation is our documentation. Yes, we need to learn to code. Test cases or test plans are converted into automated tests.
No technology is perfect without a testing framework. Every new technology or language comes with a way to test it.
Web applications: Selenium
Mobile: Calabash, Appium, Robotium
.NET: Nunit, Xunit
Java: Junit
Angular JS: Protractor

Choosing the right tool

How to choose the right automation tool? These are the key criteria to consider. 
Most popular used
Large community support
Open source vs Commercial
Skills of the team
Personally I prefer Open Source tools as they are more fun to work on. 🙂 Or even custom built. For example, we use .NET library Web Client to test API and Web services

Real Life Scenario

So how do we start from doing manual testing to automation? Take baby steps. Start writing BDD scenarios instead of test cases. Then learn how to use Selenium to drive web browser. Then try more complex code manipulation before moving on to the framework level in making sure the test framework is reusable, scalable, maintainable. 
As you expand your Selenium usage, you’ll realize that we will face some challenges in certain areas such as pages with lots of AJAX calls. In that case we will need to have workaround these scenarios, like checking that all jQuery has been loaded.
Another Selenium problem is Element No Longer Attached To DOM. In this case, we will need to have a while loop and try to find the element with a catch for StaleElementException.

Test = User + Design

We need to use the product as a user and also understand the design to be able to think of the possible loop holes 
In the older days there is less emphasize on tester to be know the software design architecture. But in Agile, as we are involved since the earliest phase of SDLC, we will are expected to know to pick up not merely software bugs, but also requirements and design issues.
Knowing the software design helps to pick up edge cases. For example:
If we know that the database field has a size of 50 varchar, we can try to input >50 chars into it and see what happens. Will it crash or truncated?
If we know that a CSS file is used in multiple location, we will need to test multiple locations if it is changed

Time vs Quality

Nowadays we need to deliver software fast. Deliver, get feedback, or fail fast is the best practice. So how can we do that without sacrificing quality? I used to fight with our PM back in he old days in order to get the best quality product out the door. But nowadays that does not work anymore because time = money for business and as a QA, we need to work towards the same goal too (management will like this).
We can deliver software with known bugs. We do this by understanding the critical path of the system usage and ensure that most red route are working correctly, while some bugs can be left in some feature that has minimal usage. We can use data or user analytics to understand which are the most visited pages, most used features, etc to help us make the decision.
My personal experience is when I started at my current job where we do not automation at all and testing happens after deployment. Our product is complex and I am having time pressure to put some automation in place.
How we go about to solve this problem is to find the top 20 pages which equals to 90% of usage. We then analyze the data and pages and put them into a logical user flow. With that we then translate that into BDD language and automate the scenario based on it.

How To Stay Alive

So software testing has changed so much and it is constantly changing just like everything is. How can we keep up with this?
We need to constantly learn. Don’t worry if there isn’t enough training budget (management will like me once more), as learning doesn’t always cost $.
There are many different channels that we can learn which are free. These are some of my personal likes. 
My personal experience with learning are: Examples:
New tool: Appium by Jonathan Lipps at GTAC 2013
New ideas: Take flaky tests from CI. Put is back when they are stable. Critical tests should never be taken out by Ankit Mehta at GTAC 2014
New ideas: Turn off a feature that is buggy by Ankit Mehta at GTAC 2014
I then I like to Blog about what I have learnt. Its not just serves to share with people what I’ve learned, but also to keep a note on what I’ve learnt.
We also need to get our hands dirty and get some hands on experience with what we learned. We can only really learn something when we work on it and gain experience from it. When I face problems I will post questions onto forums like StackOverflow, or Google forums. 
Lastly, share and give back to the community what we learnt. Try to help people out there that are learning just like us.
And most important of all is to love what do! As what Steve Jobs says.Thank you.

Future Of Software Testing

For the past few years there have been a lot of demand for automated testers. Companies are replacing manual regression test checklists with automated scripts, especially the ones written with Open Source tools such as Selenium.

However, I started to see job advertisements with graduate automation testers and developers with knowledge of testing tools.
Therefore, as time goes, software testing job or even automated testing job might be fading away as well, just like manual testers today.
So where do I see the future for software testing? I will see software testing as part of software development. Software testers will be focusing on the impact of the changes to other areas of the systems or edge cases finding. They will find them and fix them as well. They will also be focusing on performance and security aspects. They will also step in as a business analyst, product owner, or even iteration manager when needed.
Basically, what I see the future of software testing is to be a multi-skilled member of a software development team. I urge software testers to start improve their technical, communication, and business skills. Performance testing and security testing knowledge will also be considered as a basic skill of software tester instead of a specialist’s skill.
In this modern day, especially in IT where everything is changing dramatically, changes in a process or role is unavoidable. Therefore, keep learning and keep improving!

Automatically Runs Tests And Email Report From Your PC

I have set up my own PC as a build machine that runs regression tests automatically at night and then emails me the test result upon completion so that I can check it when I get in the morning.

The mechanism I use to do is:

  • Windows Task Sceduler to run the test automatically at certain time


  • Bat script to run the tests, generate Specflow report, and email the report. Note that I have included %date% and %time% variables to automatically include the date and time in the email subject

“C:Program Files (x86)NUnit 2.6.2binnunit-console.exe” /labels /out=TestResult.txt /xml=TestResult.xml C:gitPmasterbinDebugTestsRegressionRegression.dll
“C:Program Files (x86)TechTalkSpecFlowspecflow.exe” nunitexecutionreport C:gitPmasterTestsRegressionRegressionRegression.csproj /out:TestResult.html
“c:blat” c:gitPmasterTestsRegressionTestResult.html -s “TestResult at %time% on %date%” -to


  • Blat

As you can see in the Bat script, I have used a 3rd party open source tool Blat to send email. This blog gives comprehensive information on how to set up Blat.

Unable To Build After Upgrade To MVC4 Targeting 4.0 and VS2010

If you have upgraded your MVC3 project to MVC4 running in VS2010, and still targeting the 4.0 framework, you will have problem while running your build as below:

To fix this problem, you need to do the following:

  1. Unload the project in Visual Studio
    2.  Edit the .csproj file

3. Comment out the lines as below:


  4. Reload the project
  5. Re-build and you should see a success build!


« Older posts

© 2023 Chuan Chuan Law

Theme by Anders NorenUp ↑