Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Year: 2012 (page 1 of 2)

The Art of Test Automation

There is no one or absolute way for building test automation. I use the following approach to choose the technology or tool I use to build my test automation framework:

  • Open source
  • Most popular in the community
  • Regular maintenance by community
  • Able to solve my problem
Below is an example of the approach or tools I use to write a regression test suite for a customised exporter script in my company.
The behaviour of the exporter script is as below:
  • Log into a web interface
  • Fill in details as SQL statement to be run into a web form
  • Kicking off the exporter script via web form
  • Exporter script will execute the SQL statement and generate the output as text file
  • The text file will be sent to a FTP account
Below is how I build the test automation suite:
  • SpecFlow

I use this BDD tool for user readability purpose to explain the flow of the test scenarios

  • Selenium web driver in .NET

This is used to do the web interface interactions such as log in, filling in SQL details, and kicking off the tests

  • WinSCP

To log into the FTP server

  • Batch script

To run WinSCP.exe in order to delete and download files from FTP server

  • C# code

Using the Process class to run the batch script. It is also used to verify and display the output.

  • NUnit

To verify output

Below is the example of going down into each level of 1 step in the scenario:
The SpecFlow scenario looks like this:
Scenario: Kicking off PageUp Exporter
    Given I delete the output file
When I log into Feature Manager
And I kick off an Exporter process with query “SELECT * from dbo.applicant where semail like ‘%test%’ and linstid = ‘543’”
Then I verify that the file is exported
And we verify that the file has “60” rows
And we verify that the file has “100” delimiters
And we verify that the email address contains “test”
And we verify that the file size is “19484”

The step definitions for the 1st step consists of C# code that calls the batch file:

 [Given(@”I delete the output file”)]
        public void GivenIDeleteTheOutputFile()
            string fileName = @”c:PageUpOutexporterTest..txt”;
            Process p = new Process();
            p.StartInfo.UseShellExecute = false;
            p.StartInfo.RedirectStandardOutput = true;
            p.StartInfo.FileName = “C:\exporter_delete.bat”;

The batch file calls a text file which WinSCP needs:

cd c:Program Files (x86)WinSCP
start WinSCP.exe /console /script=c:exporter_delete.txt

The text file consists of more MS DOS commands to perform the operation on the FTP server:

# Automatically abort script on errors
option batch abort
# Disable overwrite confirmations that conflict with the previous
option confirm off
# Connect using a password
# open s -hostkey=”ssh-rsa 1024 xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx”
# Connect
open s -hostkey=”ssh-dss 1024 9e:bd:6f:c8:f9:68:18:81:f0:50:ce:71:dd:d6:53:2e”
cd /
# Download file to the local directory d:
rm *.txt
# Disconnect
# Exit WinSCP

Taking Screen Shots & Image comparison

Selenium offers the ability to take a screen shot of the state of the running test. It enables you to compare the screen of what you have just ran with the one previously ran.

The code that captures the screen shot looks like this:

WebDriver augmentedDriver = new Augmenter().augment(driver);
File screenshot1 = ((TakesScreenshot)augmentedDriver).getScreenshotAs(OutputType.FILE);

Image comparison compares 2 image’s:

  • Number of bands, width, and height
  • Pixels by pixels
In my iPhone application tests, I have used this mechanism. The code can be found in code


Mobile Automation For Android

If you have a native Android application, I would recommend you to use Robotium. Due to the fact that the project that I am working on is not, I have chosen Selenium.

Below are the details about the framework that I am using:

  • Follow the instructions as mentioned in
  • The method that I chose is Android WebDriver using Android Test Framework
  • Therefore, the main thing you need is Android SDK, Eclipse, Android phone, and a USB cable
Below are the stuff that I have learnt in the process of setting up this framework:
  • Would be best to test on actual device instead of emulator because Android emulator is too slow and consumes a lot of computer resources
  • The test only works for Android version 4.0 or above

Selenium-Ruby: How To Attach Document On Computer

I encounter a scenario where the browser opens up a window on the computer to attach a file.  I am delighted to find the “attach_file” function.

Below is the code:

Then /^I attach file “([^”]*)” into “([^”]*)”$/ do |filename, id|
sleep 5

Mobile Automation For iPhone & iPad Using Selenium WebDriver

If your mobile application is not a native mobile application, you might need to use some language independent testing framework such as Selenium.

Below shows how to set up this framework:

          – Right click on the test folder you created
          – Go to Build Path -> Configure Build Path
          – Click on the Libraries tab and click on Add External JARs
          – Select the selenium-server-standalone.jar file
  • The code for my project can be found in
  • Run the iPhone web driver and run the test via clicking on the Run button in Eclipse
  • Select iPad emulator if you want to run tests in iPad

Open Source Mobile Test Automation Framework

The popular ones in the market are as below:


  • Developed by Thoughtworks
  • Can only be used on iOS applications


  • Can only be used by Android applications
  • Easiest to use among all because no scripting is needed
  • It works based on screen capture
  • Need to have separate tests for iOS and Android applications because of the different UI
  • Most flexible. Can be used on both iOS and Android applications
  • Hardest to set up among all

Performance testing – JMeter

JMeter is a performance testing tool which I used recently for testing a new data centre. Apache JMeter provides a good guide on how to install and build a simple web test plan.

To performance test the new data centre, I created a simple test plan as below:

  • Go to the website
  • Login into the website
  • Perform some actions within the website
Therefore, I am trying measure the time it takes for a workflow instead of hitting a web page. To do this, I need to use JMeter Proxy. Basically, it is a browser recording function which translates into JMeter test plans automatically.
There is another popular option that does similar function, called Bad Boy Bad Boy. However, Bad Boy does not work for website with JavaScript functions which I need.
The JMeter Proxy guideline is quite comprehensive apart from the following points which need to be taken extra measure:
  • You need to include and exclude the file pattern of your test target. For e.g., in my case the files are of type “.asp” and “.aspx”
  • The test plans are created underneath the Recording Controller which needs to be expanded as you record the browser actions

IE Does Not Recognise Complex XPath

You may need different code for FireFox and IE. This is because IE does not recognise complex XPath expressions.

For eg:

This will work in FF


In IE it needs to be simplified to:



Selenium Grid 2

How To Set Up Selenium Grid 2 For Cross-Browser Testing

This page shows you how to to set up Selenium Grid 2 running in 2 machines, a WINDOWS and MAC, in parallel.
java -jar selenium-server-standalone-2.23.0.jar -role hub
  • Start a node for Firefox in MAC. is the IP address of the Hub machine. I need to specify the host IP, which is the IP of the MAC because I am in a LAN.
java -jar selenium-server-standalone-2.23.0.jar -role node -hub
:4444/grid/register -browser browserName=firefox,version=3,platform=MAC -hubHost -host “”
  • Start a node for Internet Explorer in WINDOWS (this is where we start up the Hub)
java -jar selenium-server-standalone-2.23.0.jar -role node -hub http://localhost:4444/grid/register -browser browserName=”internet explorer,version=3,platform=WINDOWS”
  • If you go to http://localhost:4444/grid/console you should see the following:
  • You need to have similar 2 programs running in different browsers
program1.cs runs the test in FireFox
private DesiredCapabilities capability = DesiredCapabilities.Firefox();
driver = new RemoteWebDriver(new Uri(“http://localhost:4444/wd/hub”), capability);
program2.cs runs the test in Internet Explorer
private DesiredCapabilities capability = DesiredCapabilities.InternetExplorer();
driver = new RemoteWebDriver(new Uri(“http://localhost:4444/wd/hub”), capability);
  • You need to use a test framework that is able to run tests in parallel. I am using MbUnit.
In AssemblyInfo.cs, add the following code to run program1.cs and program2.cs in parallel:
[assembly: DegreeOfParallelism(2)]
[assembly: Parallelizable(TestScope.All)]

  • Build the DLL for your tests. I am using Ms Visual Studio
  • Load the DLL via Icarus GUI Test Runner to start running the tests
  • You will have both program1.cs running in MAC using FireFox and program2.cs running in WINDOWS using IE8 in parallel

Getting Selenium To Work In Internet Explorer 8

Below are the 3 key things which you need to configure in order for Selenium to work in Internet Explorer 8

  • Tools->Internet Options->Security tab
Set the security level for all zones to high. This is due to all zones have to be on the same level and Restricted Sites can only be set to high.
  • Zoom level to be 100%
  • Tools->Internet Options->Security tab->Custom level->Scripting
Enable Active Scripting. This is required in order for JavaScript to work.

  • Install the correct version of InternetExplorerDriver – depending on how many bits is your machine
« Older posts

© 2020 Chuan Chuan Law

Theme by Anders NorenUp ↑