DevOps | Software Automation | Continuous Integration

Category: API Automation (Page 1 of 2)

Pact – Asserting The Function To Test from App Level

In your consumer test class, instead of calling the “subject.methodToTest()”,

For example:

describe MethodToTest{

before do

        AppleServiceClient.base_uri ‘localhost:1234’

    end

  subject { AppleServiceClient.new }

  describe “get_an_apple” do

    before do

        apple_service.given(“an apple exists”).

        upon_receiving(“a request for an apple”).

        with(method: :get, path: ‘/apple’, query: ”).

        will_respond_with(

          status: 200,

          headers: {‘Content-Type’ => ‘application/json’},

          body: {name: ‘Red Apple’} )

    end

    it “returns a an apple” do

      expect(subject.get_apple).to eq(Apple.new(‘Red Apple’))

    end

}

 

We could change our app to point to the mock localhost.

For example, in Ruby on Rails, we could change config->environments->environment.rb for the url for AppleServiceclient to point to the mock service at http://localhost:1234. Or better way is to create a new environment config for the Pact test.

# AppleServiceAPI
config.appleServiceApi_url = “http://localhost:1234”

Therefore, the new code in the assertion section will look something like:

  it “returns an apple” do

      expect(GetApple.get_apple).to eq(Apple.new(‘Red Apple’))

    end

where GetApple is the function in the application that makes the actual call to the AppleServiceClient.

How To Setup Pact Broker

Setting up Dockerized Pact Broker

The easiest and fastest way to do this is via Dockerized Pact Broker.

By following Step 1 to Step 4 in this document, you will be able to have a Dockerized Pact Broker linking to a Dockerized PSQL in your localhost within minutes!

All you then need is to publish the JSON file generated by your consumer test via Curl command:

curl -v -XPUT -H “Content-Type: application/json” -d @<path to JSON file> <http://localhost/pacts/provider/<Provider Name>I/consumer/<Consumer Name>/version/<App Version No.>>

  • Provider Name and Consumer Name can be found in the beginning of a Pact file
  • App version number is found in your consumer project’s code base

Older Version of Mac

A few things to share with my own experience is that (if you’re using Mac)

Pact Broker will be running on http://localhost

  • If you’re using an earlier Mac, therefore will be on Docker Toolbox, then will need to do the following to view the Pact Broker app:

docker-machine ip default

The command will enable the ip to be returned where you can view Pact Broker on port 80

Using Physical Database

If you do not wish yo use Dockerized PSQL, create your own database and set up the tables and users as explained. To link Pact Broker Docker to the physical database, do:

docker run –name pactbroker -e PACT_BROKER_DATABASE_USERNAME='{{user}}’ -e PACT_BROKER_DATABASE_PASSWORD='{{password}}’ -e PACT_BROKER_DATABASE_HOST='{{host}}’ -e PACT_BROKER_DATABASE_NAME='{{name}}’ -d -p 80:80 dius/pact_broker

 

Extending PACT Framework – Global Data Setup & Data Cleanup

In every testing framework we do data setup and clean up. Below is how I do it with Gradle framework:

Global Data Setup

Pact framework provides very good random test data generation with PactBodyBuilder. However, in some cases where our API involves data creation, we will need to remember the data we created, so we can use it across multiple tests, and then deleting it afterwards.

We can do this by using System Property in Gradle. Under test section in build.gradle, include this:

test{
    systemProperty 'RandomNum', new Random(1000)
}

In the Pact Consumer test where you need to create the data, do this:

  • Define the random number data that you want to generate and remember
class InsertCFCertConsumerTests {

    def ran=System.getProperty("RandomNum")

    @Test
    void "Insert CF Certificate" (){
  • In the Pact body, include your random number
withBody{
    requestId ran
}
  •   Of course, include this random number in the 2nd part of the Consumer test (service run)
VerificationResult result = CF_service.run() {

   def client = new RESTClient('http://localhost:1237/') 
   def body = new JsonBuilder([requestId: ran]) 
   def cf_response = client.post(path: '/v1/carfacts/',requestContentType: 'application/json',headers:['Content-Type': 'application/json'],body:body.toPrettyString())
  • The same random number ran can be called across multiple tests

Cleanup

To clean up data from database, create a task in build.gradle.

Clean up via database

  • Connect to the database
task cleanUp<<{
    gradle.class.classLoader.addURL(new File('src/main/resources/ojdbc6.jar').toURI().toURL())
    def sql = Sql.newInstance('jdbc:oracle:thin:@testDatabase.oracle.csuat.com.au:1521/test.test.office', 'USERNAME', 'PASSWORD', 'oracle.jdbc.driver.OracleDriver')

}

ojdbc6.jar is the Oracle database driver downloaded from the Oracle site . External file can be included during compilation by adding the following line under dependencies:

dependencies {
     compile files('src/main/resources/ojdbc6.jar')
}
  • Write the SQL to delete the data you want
String sqlDelete = 'delete from Table where id='VN000000000000001''
  • Execute the query
sql.execute(sqlDelete)

 

Clean up via API call

Sometimes, we could call a DELETE via API to do data clean up.

  • Get Id via database and delete via API call
task cleanUp<<{

   String sqlGetId= 'select id from Table where id='VN000000000000001''
   def delElas=new RESTClient('http://test.domain.com.au:9200') 
   def cfId=0 sql.eachRow(sqlGetId) { 
     println it.id cfId=it.id delElas.delete(path:"/id_checker/${cfId}") 
   }
}

However,  please note that the above will cause build task to fail as the data that you want to delete will not exist. Workaround for this is to exclude this task while building via command gradle build -x <name of clean up task>.

Some Workarounds In PACT Test

Writing PACT test might be quite a challenge in the beginning, as it is a new tool, and we might not be able to find much information in the web yet.

Personally, I find the PACT Google Support Group is very helpful when I encounter issues. Otherwise, sometimes I do some trial and error to solve my obstacles. Below are some quick solutions to some consumer/provider test issues that I have encountered.

Consumer Tests

Below are some scenarios when it comes to writing Body DSL for some JSON bodies.

Time stamp with certain format

Eg:  2015-05-13T06:18:06Z

Body DSL: yyyy-MM-dd’T’HH:mm:ssX

Negative integer

Eg: negative integer: -280

Body DSL: integer(-250)

Field with empty []

Eg: factory[]

Body DSL: factory([])

JSON body inside arrays

Eg: factory[{

field1 123

field2 12.5

}]

Body DSL:

factory([{

field1 integer()

field2 real()

}])

A field which might return an empty value

Eg:

“fruit”, a String which can be an empty String sometimes

Body DSL:

“fruit” (~/\w.+|^$/,”banana or apple”)

Note that we could use an or operator inside the regex. However, it is probably better to separate this into 2 test cases.

Provider Test

Unable to call API in Provider test via load balancer

Eg:

API connection fails when we call an API via a URL that goes through load balancer (not sure why)

Workaround:

Call the API directly via the server, bypassing load balancer

PACT: How To Write Provider Test

Writing PACT provider test is easy because you do not need to write them, you only need to call them!

Example below is in Groovy using the Gradle framework.

(1) Import the library and include it in the build script

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath ‘au.com.dius:pact-jvm-provider-gradle_2.11:3.0.1’
    }
}

(2) In build.gradle, call the API, which is the Provider and define the location of the PACT JSON file generated by the Consumer test. These are done under a “pact” segment.

pact {

    dataProviders {

         DataProvider {
            protocol = ‘http’
            host = ‘test.api.com.au’
            port = 80
            path = ‘/consumer/search’

             hasPactWith(‘Consumer’) {
                 pactFile = file(‘..\ApiTests\target\pacts\Consumer-Provider.json’)

            }

        }

}
 
The API that we want to call is http://test.api.com.au:80/consumer/search. Consumer-Provider.json is the PACT file generated by the Consumer test in my previous Consumer test blog. The file name “Consumer-Provider.json” is based on the “serviceConsumer” and “hasPactWith” definitions in the Consumer test. “APITests” is the root directory of the project, while “target” is the default directory where PACT files are going to be generated. However, this maybe overwritten (refer to the original GitHub wiki on how to do this). “hasPactWith(‘Consumer’)” in Provider test is merely a name. This is useful so we can run each test individually.
You may define as many tests as you want inside the “pact” segment.

    @Test
void “Consumer test for Consumer and Provider”() {
def consumer_service = new PactBuilder()
consumer_service {
serviceConsumer “Consumer”
hasPactWith “Provider”



(3) Execute the command gradle pactVerify from the root directory of the project. Or to call the test individually, we may specify gradle pactVerify -Ppact.filter.consumer=[hasPactWith value]Example of output:

PACT Provider will call the API that you defined in gradle.build and compare the actual output against the output generated by Consumer test. It compares for the response status, content type, and body as how you defined them.

This way, we know that the API contract that the Consumer expects is similar as what the API Provider produces.

Example of a PACT Provider failure:

 

In the example above, value Type is returned as “Car” but it is defined as an integer in the Consumer test.

PACT: How To Write Consumer Test

Below is an example of a PACT Consumer tests with example in Groovy.

  1. Import the library
Ensure that we downloaded the PACT consumer library from Maven repository import them in your consumer test file. We also need to include REST Client to make API calls and JUnit for assertion.
import au.com.dius.pact.consumer.PactVerified$
import au.com.dius.pact.consumer.groovy.PactBuilder
import groovyx.net.http.RESTClient
import org.junit.Test

    2. Define the consumer service that will consume the API provider


Example below is a consumer test class that defines a consumer service “Consumer” that is requesting data from API provider “Provider”. The API request is a GET method with parameter “q” with some value. Upon success, it returns a response code of 200 with Content-Type of “application/json;charset=UTF-8”. 

class ConsumerTests {

    @Test
    void “Consumer test for Consumer and Provider”() {
        def consumer_service = new PactBuilder()
        consumer_service {
            serviceConsumer “Consumer”
            hasPactWith “Provider”
            port 1237

            given(‘Scenario of Consumer is retrieving data from Provider’)
            uponReceiving(‘a retrieve a data request’)
            withAttributes(method: ‘get’, path: ‘/consumer/search’, query: [q: ‘refKey:4e6b75c2-b527-4af0-a098-b2ba81706bdc’], headers: [‘Accept’: ‘text/plain’, ‘Content-Type’: ‘application/json’])
            willRespondWith(
                    status: 200,
                    headers: [‘Content-Type’: ‘application/json; charset=UTF-8’]
            )

            withBody {

 }

3. Define the response body

There are 2 ways to define the response body.

(a) Response body with exact or fixed data.

Use this method if you need to test that the data returned by the API is exactly the same as what you defined in the consumer test.

For example:

  withBody{
                “Id” (“87992”)
                “Seller” (“Michael”)
                “CompanyName” (“ABC123”)
}

The example above will generate a PACT JSON file that looks like:

  “response” : {
      “status” : 200,
      “headers” : {
        “Content-Type” : “application/json;charset=utf-8”
      },
      “body” : {
        “Id” : “87992”,
        “Seller” : “Michael”,
        “CompanyName” : “ABC123”
      }
    }

So the PACT provider test will check for a response with body with exactly the same structure and data as the PACT JSON file.

(b) Response body with flexible data

Use this method if you only want to test that the contract or structure of the API remains the same. This is useful in the scenario when data changes frequently and the effort of test data maintenance is high.

For example:

  withBody{
                “Id” regexp(~/w.+/,”87992″)
                “SellerGuid” regexp(~/w.+/,”Michael”)
                “CompanyName” regexp(~/w.+/, “ABC123”)
}

The example above will generate PACT JSON file that looks like:

 “response” : {
      “status” : 200,
      “headers” : {
        “Content-Type” : “application/json;charset=utf-8”
      },
      “body” : {
        “Id” : “87992”,
        “SellerGuid” : “Michael”,
        “CompanyName” : “ABC123”
      },
      “matchingRules” : {
        “$.body.Id” : {
          “regex” : “\w.+”
        },
        “$.body.SellerGuid” : {
          “regex” : “\w.+”
        },
        “$.body.CompanyName” : {
          “regex” : “\w.+”
        }
      }

Instead of passing the exact data to match, we pass in a regular expression instead. Therefore, a set of matching rules will be generated in the PACT JSON file. PACT Provider test will look at the matching rules and ignore the data. So, if Company Name is changed from “ABC123” to “ABC678”, the test will still pass as the data still matches the regular expression defined.

This is a good example of a consumer with body that uses flexible matching of many different data types such as string, integer, IP, time stamp, etc.

Another key features of the flexible matching is the flexible array matching.

For example:

    hits minLike(1) {
                        _index(~/w.+/, “12345”)
                        _type(~/w.+/, “Dog”)
                        _id(~/d+/, “263287”)
                        _score real()

The above is useful if the JSON output that you want to test has array structures and the number of arrays in the structure changes.

More information can be found here

(4) Generate the PACT file

The last part of the consumer test is basically generating the PACT JSON output.


For example:

   def result = consumer_service.run() {
                def client = new RESTClient(‘http://localhost:1237/’)

                def response = client.get(path: ‘/consumer/search’, query: [q: ‘refKey:4e6b75c2-b527-4af0-a098-b2ba81706bdc’], headers: [‘Accept’: ‘text/plain’, ‘Content-Type’: ‘application/json’])

                assert response.status == 200
   
                consumer_service.buildInteractions()
            }


If everything is defined correctly, a JSON output will be generated in the file structure under the “target” directory.

The JSON output is contract of the API that you have defined in the consumer test, which is the API that the consumer consumes.

Introduction: API Contract Testing With PACT

PACT is a tool introduced to enable to testing of API contract between the consumer (the system that uses the API) and the provider (the system that produces the API). It is available in a couple of different languages. The version that I have been using is the JVM version in Groovy. This tool is very useful to detect API structure changes. It is good in the sense that we can test the structure  of the API irrespective of the data (we can just test the data type). It could be easily integrated into CI box as well.

 

Consumer Test

Consumer test defines the contract or structure of the API that the system uses. It can be used to enable TDD and in a situation when the system that consumes the API is built before the API exist. Upon successful consumer test run, a JSON file that mimic the actual API output will be generated under the “target” folder.

Provider Test

Provider test basically is a call to the actual API which then compares the actual API result with the structure which we expect (as defined in the Consumer test). It is built in the build script depending on which framework that we use, in my case it is called in the build.grade file.
I will include more specific examples with descriptions on the Consumer and Provider tests in my upcoming posts.

API Testing In General (PUT & DELETE cont.)

I have putting a few blogs about API testing in the past few months. I have been doing some research about how to test API, best tool to test API, etc and I guess there is no exact answer to this. However, SoapUI seems to appear in quite a number of searches. But I still do not choose SoapUI as it is not a pure Open Source tool.

So, I still think it all depends on the language of your API system, and there is a lot of built-in library to make a web request. All you need to do is to incorporate a unit testing framework to assert the result returned.

As I am working in a .NET environment, I have tried HttpClient and WebClient. I have used Curl for Ruby.

Example of POST and GET have been posted in the previous blogs.

Example of PUT is as below:

 

And DELETE:

 

« Older posts

© 2023 Chuan Chuan Law

Theme by Anders NorenUp ↑