DevOps | Software Automation | Continuous Integration

Tag: Groovy

Building A Selenium Web Driver Test Framework From Scratch

Selenium UI test is the most used high level test these days for web applications. Regardless of which language you use, the framework is pretty much the same. I would like to share with everybody the Selenium web driver testing framework I worked on recently using Selenium in Java on a Groovy Gradle framework.

I will provide the step-by-step guideline on how to create a skeleton framework from scratch:

  1. Import libraries you need

dependencies {

    compile 'org.codehaus.groovy:groovy-all:2.4.3'
    testCompile 'junit:junit:4.12'
    testCompile 'org.seleniumhq.selenium:selenium-java:2.46.0'
    testCompile 'info.cukes:cucumber-groovy:1.2.3'
}

You will need the Selenium Java from Maven repository, Cucumber in Groovy as a BDD layer (although this is not necessary), JUnit to assert test values.

2. Create a webdriver class

import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebElement
import org.openqa.selenium.chrome.ChromeDriver

public class CsWebDriver {

 private static WebDriver _driver

 public static WebDriver get(String browserType)
 {
 if (browserType.equals("chrome"))
 {
 System.setProperty("webdriver.chrome.driver","srcmainresourceschromedriver.exe")
 ChromeOptions options = new ChromeOptions();
 options.setBinary("c:Chrome.V41GoogleChromePortable.exe");

 _driver=new ChromeDriver(options)

 }

 return _driver

 }

This is where we declare a new web driver object from the library and initialize it with a copy of driver downloaded from  ChromeDriver. You could create or extend all browser related functionalities in this class, such as Open a URL, Quit, Wait, Move To Click, etc.

For example: Move to an element to click using Action class

public static WebElement WaitTillElementExist(WebElement ele, int seconds)
{

   WebDriverWait wait = new WebDriverWait(_driver,seconds)
   return wait.until(ExpectedConditions.elementToBeClickable(ele))
}

3. Create Page Factory to host Page Objects

public class PageFactory {

    private static WebDriver _driver

    public static LoginPage LoginPage

   public static void InitialisePages(WebDriver driver)
   {
     _driver=driver
     LoginPage= new LoginPage(driver)
   }
   public static void ResetPages()
   {
     LoginPage=null
   }

 public static GetDriver() 
 { return _driver 
 }

Page Factory above is where we declare, initialize, and reset our Page Objects. An instance of web driver will be passed into each Page Object.

import org.openqa.selenium.support.FindBy 
import org.openqa.selenium.support.How

@InheritConstructors
class LoginPage extends BasePage {

    @FindBy(how=How.ID, using="Username")
    private WebElement userName

    @FindBy(how=How.ID, using="Password")
    private WebElement password
    @FindBy(how=How.CSS, using="button[class='btn']")
    private WebElement loginBtn

    public void Login(String username, String pwd)
    {
      userName.sendKeys(username)
      password.sendKeys(pwd)
      loginBtn.click()
    }
}

Every webpage that we are testing is treated as an object. By importing the Selenium library FindBy and How we can select the element that we want using ID, CSS, XPATH, etc. The page object will also contain functions specific to the page, for eg. Login function. As we can see, the page object is extended from the BasePage. A BasePage looks like this:

import org.openqa.selenium.support.PageFactory

public class BasePage {

    public WebDriver driver

    public BasePage(WebDriver webdriver)
    {
        this.driver=webdriver
        PageFactory.initElements(webdriver,this)

    }
}

We use the Selenium PageFactory library to easily implement this.

    4. Binding everything together

So we now have the web driver class, page factory, and page objects, how do we glue this all together into an executable test? To make test easier to understand and read, we usually like to use a BDD tool to describe a test Scenario. I will not go into depth on the BDD tool in this post, but I would like to touch base on the Before Scenario Hook of a BDD tool that is used to glue all pieces together.

this.metaClass.mixin(cucumber.api.groovy.Hooks) this.metaClass.mixin(cucumber.api.groovy.EN) Before("@web") 
{ 
  scenario -> PageFactory.InitialisePages(WebDriver.get("chrome")) 
  WebDriver.Open(oagUrl) 
}

Before every scenario, we initialize all Page Objects with a Chrome driver and then  open the URL.

After("@web") { scenario ->

    if (scenario.isFailed()) {
        String fileName=scenario.getName()
        WebDriver.PrintPageSource()
        WebDriver.TakeScreenShot(fileName, scenario)
    }

    WebDriver.Quit()
    PageFactory.ResetPages()
}

After every scenario, we check if the scenario is failed, if so, we print out the scenario name and take a screen shot. We then quit the driver and reset all Page Objects to null.

  5. Run the test via command line

To enable the test to be run on a CI box, we have to make the test run via command line. The example below is using Gradle to run the Cucumber Groovy test:

configurations {
    cucumberRuntime {
        extendsFrom testRuntime
    }
}
sourceSets{
    test{
        groovy{
            srcDirs 'src/test'
        }
    }
}
task cucumber() {
    dependsOn assemble, compileTestJava
    doLast {
        javaexec {
            main = "cucumber.api.cli.Main"
            classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
            args = ['-f', 'json:target/cucumber-report.json','--plugin','pretty', '--glue', 'src/test/groovy/au/com/test/story/steps', 'src/test/groovy/au/com/test/story/features']
        }
    }
}

We created a Gradle task called “cucumber” and execute the test using “javaexec”.  We need to define in the Source Sets the directory of the test. In “args –glue”, we define where the Feature  files and Steps Definitions are.

  6. Finally, run your test!

PACT: How To Write Provider Test

Writing PACT provider test is easy because you do not need to write them, you only need to call them!

Example below is in Groovy using the Gradle framework.

(1) Import the library and include it in the build script

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath ‘au.com.dius:pact-jvm-provider-gradle_2.11:3.0.1’
    }
}

(2) In build.gradle, call the API, which is the Provider and define the location of the PACT JSON file generated by the Consumer test. These are done under a “pact” segment.

pact {

    dataProviders {

         DataProvider {
            protocol = ‘http’
            host = ‘test.api.com.au’
            port = 80
            path = ‘/consumer/search’

             hasPactWith(‘Consumer’) {
                 pactFile = file(‘..\ApiTests\target\pacts\Consumer-Provider.json’)

            }

        }

}
 
The API that we want to call is http://test.api.com.au:80/consumer/search. Consumer-Provider.json is the PACT file generated by the Consumer test in my previous Consumer test blog. The file name “Consumer-Provider.json” is based on the “serviceConsumer” and “hasPactWith” definitions in the Consumer test. “APITests” is the root directory of the project, while “target” is the default directory where PACT files are going to be generated. However, this maybe overwritten (refer to the original GitHub wiki on how to do this). “hasPactWith(‘Consumer’)” in Provider test is merely a name. This is useful so we can run each test individually.
You may define as many tests as you want inside the “pact” segment.

    @Test
void “Consumer test for Consumer and Provider”() {
def consumer_service = new PactBuilder()
consumer_service {
serviceConsumer “Consumer”
hasPactWith “Provider”



(3) Execute the command gradle pactVerify from the root directory of the project. Or to call the test individually, we may specify gradle pactVerify -Ppact.filter.consumer=[hasPactWith value]Example of output:

PACT Provider will call the API that you defined in gradle.build and compare the actual output against the output generated by Consumer test. It compares for the response status, content type, and body as how you defined them.

This way, we know that the API contract that the Consumer expects is similar as what the API Provider produces.

Example of a PACT Provider failure:

 

In the example above, value Type is returned as “Car” but it is defined as an integer in the Consumer test.

PACT: How To Write Consumer Test

Below is an example of a PACT Consumer tests with example in Groovy.

  1. Import the library
Ensure that we downloaded the PACT consumer library from Maven repository import them in your consumer test file. We also need to include REST Client to make API calls and JUnit for assertion.
import au.com.dius.pact.consumer.PactVerified$
import au.com.dius.pact.consumer.groovy.PactBuilder
import groovyx.net.http.RESTClient
import org.junit.Test

    2. Define the consumer service that will consume the API provider


Example below is a consumer test class that defines a consumer service “Consumer” that is requesting data from API provider “Provider”. The API request is a GET method with parameter “q” with some value. Upon success, it returns a response code of 200 with Content-Type of “application/json;charset=UTF-8”. 

class ConsumerTests {

    @Test
    void “Consumer test for Consumer and Provider”() {
        def consumer_service = new PactBuilder()
        consumer_service {
            serviceConsumer “Consumer”
            hasPactWith “Provider”
            port 1237

            given(‘Scenario of Consumer is retrieving data from Provider’)
            uponReceiving(‘a retrieve a data request’)
            withAttributes(method: ‘get’, path: ‘/consumer/search’, query: [q: ‘refKey:4e6b75c2-b527-4af0-a098-b2ba81706bdc’], headers: [‘Accept’: ‘text/plain’, ‘Content-Type’: ‘application/json’])
            willRespondWith(
                    status: 200,
                    headers: [‘Content-Type’: ‘application/json; charset=UTF-8’]
            )

            withBody {

 }

3. Define the response body

There are 2 ways to define the response body.

(a) Response body with exact or fixed data.

Use this method if you need to test that the data returned by the API is exactly the same as what you defined in the consumer test.

For example:

  withBody{
                “Id” (“87992”)
                “Seller” (“Michael”)
                “CompanyName” (“ABC123”)
}

The example above will generate a PACT JSON file that looks like:

  “response” : {
      “status” : 200,
      “headers” : {
        “Content-Type” : “application/json;charset=utf-8”
      },
      “body” : {
        “Id” : “87992”,
        “Seller” : “Michael”,
        “CompanyName” : “ABC123”
      }
    }

So the PACT provider test will check for a response with body with exactly the same structure and data as the PACT JSON file.

(b) Response body with flexible data

Use this method if you only want to test that the contract or structure of the API remains the same. This is useful in the scenario when data changes frequently and the effort of test data maintenance is high.

For example:

  withBody{
                “Id” regexp(~/w.+/,”87992″)
                “SellerGuid” regexp(~/w.+/,”Michael”)
                “CompanyName” regexp(~/w.+/, “ABC123”)
}

The example above will generate PACT JSON file that looks like:

 “response” : {
      “status” : 200,
      “headers” : {
        “Content-Type” : “application/json;charset=utf-8”
      },
      “body” : {
        “Id” : “87992”,
        “SellerGuid” : “Michael”,
        “CompanyName” : “ABC123”
      },
      “matchingRules” : {
        “$.body.Id” : {
          “regex” : “\w.+”
        },
        “$.body.SellerGuid” : {
          “regex” : “\w.+”
        },
        “$.body.CompanyName” : {
          “regex” : “\w.+”
        }
      }

Instead of passing the exact data to match, we pass in a regular expression instead. Therefore, a set of matching rules will be generated in the PACT JSON file. PACT Provider test will look at the matching rules and ignore the data. So, if Company Name is changed from “ABC123” to “ABC678”, the test will still pass as the data still matches the regular expression defined.

This is a good example of a consumer with body that uses flexible matching of many different data types such as string, integer, IP, time stamp, etc.

Another key features of the flexible matching is the flexible array matching.

For example:

    hits minLike(1) {
                        _index(~/w.+/, “12345”)
                        _type(~/w.+/, “Dog”)
                        _id(~/d+/, “263287”)
                        _score real()

The above is useful if the JSON output that you want to test has array structures and the number of arrays in the structure changes.

More information can be found here

(4) Generate the PACT file

The last part of the consumer test is basically generating the PACT JSON output.


For example:

   def result = consumer_service.run() {
                def client = new RESTClient(‘http://localhost:1237/’)

                def response = client.get(path: ‘/consumer/search’, query: [q: ‘refKey:4e6b75c2-b527-4af0-a098-b2ba81706bdc’], headers: [‘Accept’: ‘text/plain’, ‘Content-Type’: ‘application/json’])

                assert response.status == 200
   
                consumer_service.buildInteractions()
            }


If everything is defined correctly, a JSON output will be generated in the file structure under the “target” directory.

The JSON output is contract of the API that you have defined in the consumer test, which is the API that the consumer consumes.

Introduction: API Contract Testing With PACT

PACT is a tool introduced to enable to testing of API contract between the consumer (the system that uses the API) and the provider (the system that produces the API). It is available in a couple of different languages. The version that I have been using is the JVM version in Groovy. This tool is very useful to detect API structure changes. It is good in the sense that we can test the structure  of the API irrespective of the data (we can just test the data type). It could be easily integrated into CI box as well.

 

Consumer Test

Consumer test defines the contract or structure of the API that the system uses. It can be used to enable TDD and in a situation when the system that consumes the API is built before the API exist. Upon successful consumer test run, a JSON file that mimic the actual API output will be generated under the “target” folder.

Provider Test

Provider test basically is a call to the actual API which then compares the actual API result with the structure which we expect (as defined in the Consumer test). It is built in the build script depending on which framework that we use, in my case it is called in the build.grade file.
I will include more specific examples with descriptions on the Consumer and Provider tests in my upcoming posts.

© 2023 Chuan Chuan Law

Theme by Anders NorenUp ↑