REST API or SOAP Testing Automation with ZeroCode

JSON-Based Open Source Test Framework

Sidd Gautama

8 Sep 2019 Apache

Declarative, Configurable and Super Easy API testing lib using YAML/JSON steps

Browse the source code in GitHub repo

Browse the source code of HelloWorld demo

High Level Focus Points

Introduction
Why It's Useful
How the framework works in the background (for framework writers)
Usage example(HelloWorld) with invoking a simple GitHub API
Useful JUnit reports with Search and Filter
Point of interest (Chaining or weaving previous request and response)
See also - When you are behind firewall proxy or need a SSL client
History and Motivation for Zerocode
Contributing to Zerocode
Who uses Zerocode

 
It helps in clearly dividing your scenarios into tiny YAML/JSON steps which helps in identifying what exactly failed or passed during
an application end-to-end integration testing and consumer contract testing.

Introduction
Zerocode brings the simplicity in testing and validating APIs by eliminating repetitive code for test assertions, http calls and payload
parsing. See an example how. It's powerful response/payload comparison and assertions make the testing cycle a lot easy and clean.

E.g. Your below User-Journey or ACs (Acceptance Criterias) or a scenario,

AC1:
GIVEN- the POST api end point '/api/v1/users' to create an user,
WHEN- I invoke the API,
THEN- I will receive the 201 status with the a user ID and headers
AND- I will validate the response

AC2:
GIVEN- the REST api GET end point '/api/v1/users/${created-User-Id}',
WHEN- I invoke the API,
THEN- I will receive the 200(Ok) status with body(user details) and headers
AND- I will assert the response

will translate to the below executable JSON in Zerocode - Simple and clean !

Keep in mind: It's simple JSON.

No feature files, no extra plugins, no statements or grammar syntax overhead.
It also helps in mocking/stubbing interfacing APIs during the testing cycle. Its approach to IDE based performance testing to
generate load/stress on the target application is quite simple, flexible and efficient - It goes a step further enabling you to simply
reuse the test(s) from your regression pack.Why It's Useful, What Problem It Solves!

You do not need to write POJOs or builders to represent or create your test inputs/results

You just need the YAML or JSON equivalent for your payload

You do not have to write code for serializing or deserializing payload/response

The framework takes care of it for you

All assertion mismatches for a response are displayed at once (see picture below)

It makes the cycle easy, either to correct the assertions or fix the application code

Response validation is seamless however hierarchal they may be. You can verify the entire response as it is -or- a part of it -
or- a specific field by using Jayway JSON path

The framework handles it for you. See examples here (Observe the verify section in the steps).

The framework enables you to override the behaviour by using your own custom client (optional)

You can simply use @UseHttpClient (YourOwnCustomClient.class). See an example here.

 Also explained in the bottom sections how to override.

Jenkins CI Integration is seamless needed for various envs

Just pass the env value in -D param of maven. See example here.

JUnit test result reports enable fuzzy text matching with search and filter

Filter by author or filter by a scenario name -or- filter by PASS/FAIL status, etc. See examples here.

Suite runner is seamless, you do not have to copy/paste series of test-class names

Just point to the root of the test package. See an example here.
Optionally, copy/paste series of class names can be used as Junit Suite runners

Zerocode readme file has examples of all the above scenarios and much more usages with examples:

Link to the Zerocode README

When assertion fails, i.e., the actual response doesn't match with "verify" block, it displays all non-matching fields by their JSON
path (enlarge the pic for clarity) like below:
Background
In today's world of automation, API testing sounds easy enough, but are we doing it in a way where these (tests) are easy to share,
easy to maintain and easy to change? Probably not, because too much time gets wasted in preparing the test code in order to
bring the final request fired against the target system, then too much code goes into assert, assertThat, assertNull,
assertTrue, etc. and the cycle goes on, if the objects are hierarchal in nature, the complexity simply increases in
serializing/deserializing/writing builders/pojo, etc. Then even if we take that approach, are we in a state where we could easily share
the pojos/builders/serializers/desers, etc. and explain to the interfacing team what's going on inside them? Does the debugging
become easy when the tests fail?

The consumer-contract tests should be readable and maintainable to both parties (service provider and service consumer), they
should look simple enough and must contain the request, response in a readable structure to Business, as we share the JSON
contracts with interfacing teams. Same goes with End to End integration tests too.

Zerocode goes extra miles and makes these aspects of testing life cycle so simple that you will do BDD and/or TDD without much
effort, but with full clarity and focusing on business scenarios and ACs (Acceptance Criterias) with much more efficiency and
accuracy.

Using the Code

Here is an example of a scenario with a single step, run it as JUnit by using @Test annotation.

//
// See the "verify" section below, if the response matches this JSON block, then the test will
PASS
//
{
"scenarioName": "Testing the GitHub REST end point",
"steps": [
{
"name": "get_user_details",
"url": "/users/octocat",
"method": "GET",
"request": {
"queryParams":{
//key-value pair
}
},
"retry": {
"max": 3,
"delay": 1000,
},
"verify": {
"status": 200,
"body": {
"login" : "octocat",
"id" : 583231,
"name" : "The Octocat",
"type" : "User"
}
}
}
]
}

How to Run the Above Test?

Run it as JUnit by using @Test annotation from the IDE or using maven command as below:

$ mvn test -Dtest=org.jsmart.zerocode.testhelp.tests.helloworld#testGet

 //
// In the JUnit test below point to the above file,
// i.e., "helloworld/hello_world_status_ok_assertions.yml"
//
package org.jsmart.zerocode.testhelp.tests.helloworld;

import org.jsmart.zerocode.core.domain.JsonTestCase;
import org.jsmart.zerocode.core.domain.TargetEnv;
import org.jsmart.zerocode.core.runner.ZeroCodeUnitRunner;
import org.junit.Test;
import org.junit.runner.RunWith;

@TargetEnv("github_host.properties")
@RunWith(ZeroCodeUnitRunner.class)
public class JustHelloWorldTest {

@Test
@JsonTestCase("helloworld/hello_world_status_ok_assertions.yml")
public void testGetApi() throws Exception {
}
}

How Does the Framework Work?

Let's have a deeper look inside the framework to see what's going on.

In the above example, there are the below annotations:

@RunWith(ZeroCodeUnitRunner.class)
@JsonTestCase("helloworld/hello_world_status_ok_assertions.yml")
@TargetEnv("github_host.properties")
Let's discuss them below.

@RunWith(ZeroCodeUnitRunner.class)
This unit runner extends and builds on the JUnit core runner, i.e., BlockJUnit4ClassRunner. The framework source code
goes as follows. See the code snippet below:

public class ZeroCodeUnitRunner extends BlockJUnit4ClassRunner {

...
...
@Override
protected void runChild(FrameworkMethod method, RunNotifier notifier) {
...
}
...
...

What is Going On Inside the Above Code Snippets?

The runChild() method of "BlockJUnit4ClassRunner" has been overridden inside the framework to pick the test JSON
file from the resources folder via @JsonTestCase.

After it picks the test case, it decomposes the test scenario into one or more steps, each step having:

url - The FQDN (Fully Qualified Domain Name) of the REST application or a SOAP server, e.g., https://api.github.com
method - An Http method, e.g., GET/PUT/POST/DELETE, etc. all supported methods by Apache Http Client.
request - JSON request to the target system/url under test.
retry - Max retry with certain delay between the retries
 queryParams - Key-Value pairs to filter the REST api output
verify - Expected response from server, i.e. the system under test.
Optionally, if a Java class is used in the URL, then the below breakdown comes into play, where:

url - Fully qualified Java class name

method - A public method in the above Java class
request - A JSON equivalent of an object passed into the method/operation
verify - Expected return value, YAML/JSON equivalent of Java public method output

Note: The "request", "verify" field above supports XML input too which helps in testing SOAP end points. Along with
optionally converting an XML to JSON equivalent (see MIME converters in the README) for easily readable and more granular
assertions field by field.

...
...

@Override
public void run(RunNotifier notifier) {
notifier.addListener(new ZeroCodeTestReportListener(...);
super.run(notifier);
}

...
...

The "run()" method of the "BlockJUnit4ClassRunner" has been overridden here to aggregate the test result, once all
tests have been completed running.

Once the test run starts (via JUnit runner), the payLoad is picked from the "request" field, and gets invoked against the FQDN
with full path.

The framework source code goes as follows:

...
executionResult = serviceExecutor.executeRESTService
(serviceName, operationName, resolvedRequestJson)
...
final javax.ws.rs.core.Response serverResponse =
httpClient.execute(httpUrl, methodName, headers, queryParams, bodyContent);

@JsonTestCase("helloworld/hello_world_status_ok_assertions.yml")
This annotation tells the "ZeroCodeUnitRunner" to pick the test case from a specified folder in the resources.

...
JsonTestCase annotation = method.getMethod().getAnnotation(JsonTestCase.class);

if (annotation != null) {
currentTestCase = annotation.value();
} else {
currentTestCase = method.getName();
}
...

@TargetEnv("github_host.properties")
This annotation holds the host/port/context properties of the target application under test.

restful.application.endpoint.host=https://api.github.com
restful.application.endpoint.port=443
restful.application.endpoint.context=
Inside the framework, these properties get appended to the relative path mentioned in the "url" and then get invoked with the
payLoad. In the above example, it is resolved to https://api.github.com:443/users/octocat.
The properties are bound and injected via "Google Guice" as below code inside the framework:

// serverEnv below is "github_host.properties" as annotated in the example

public class ApplicationMainModule extends AbstractModule {

...

@Override
public void configure() {
/*
* Install other guice modules
*/
...
install(new HttpClientModule());

...
/*
* Bind properties for localhost, CI, SIT, PRE-PROD etc
*/
Names.bindProperties(binder(), getProperties(serverEnv));
}

In case of a Java method execution, the framework executes the method via reflection and returns an equivalent JSON as the
response. (See the framework source code for >>
org.jsmart.zerocode.core.engine.executor.JsonServiceExecutor, the method:
executeJavaService()).

...
//guice
@Inject
private JavaExecutor javaExecutor;

public String executeJavaService(String serviceName, String methodName, String requestJson)

throws JsonProcessingException {

if( javaExecutor == null) {

throw new RuntimeException
("Can not proceed as the framework could not load the executors. ");
}

List<Class<?>> argumentTypes = javaExecutor.argumentTypes(serviceName, methodName);

try {
Object request = objectMapper.readValue(requestJson, argumentTypes.get(0));
Object result = javaExecutor.execute(serviceName, methodName, request);

final String resultJson = objectMapper.writeValueAsString(result);

return prettyPrintJson(resultJson);

} catch (Exception e) {

e.printStackTrace();

throw new RuntimeException(e);

}
}
...

Points of Interest
In the "steps" section above, you can add multiple steps to make up a scenario using ${JSON Path to earlier steps}
e.g., See below:

//
// See the "verify" section below, if the response matches this section, then the test will
PASS
//
{
"scenarioName": "GIVEN- the REST end points, WHEN- I invoke POST and GET,
THEN- I will create and receive the new emp details",
"steps": [
{
"name": "create_emp",
"url": "/api/v1/google-uk/employees",
"method": "POST",
"request": {
"body": {
"id": 1000,
"name": "Larry Pg",
"addresses": [
{
"gpsLocation": "x9000-y9000z-9000-home"
},
{
"gpsLocation": "x9000-y9000z-9000-home-off"
}
]
}
},
"verify": {
"status": 201,
"body": {
"id": 1000
}
// You can have more assertions here, but for now I am interested in
// whether the POST has created the "id" or not.
// If the assertion fails here, then it will error out until
// you fix the target application.
}
},
{
"name": "get_user_details",
"url": "/api/v1/google-uk/employees/${$.create_emp.response.body.id}", //reuse
value
"method": "GET",
"request": {
},
"verify": {
"status": 200,
"body": {
"id": 1000,
"name": "Larry Pg",
"addresses": [
{
"gpsLocation":
"${$.create_emp.response.body.addresses[0].gpsLocation}"
},
{
"gpsLocation": "x9000-y9000z-9000-home-off"
}
]
}
}
}
]
}

// The above example is available in the same HelloWorld demo project

See also:

Using a SSL HttpClient

Usage of an Http Client behind a Corporate Proxy firewall
Testing SOAP end points

Ok, But What If I Want to Use My Custom HttpClient?

Yes, in fact, there will be many times as a developer or automation tester, you need to have your own httpclient. You can
simply override the framework provided HttpClient with your own HttpClient. It's simple and easy. See the below code
snippet:

The full source code can be found here on GitHub.

import javax.ws.rs.core.Response;

public class SslTrustHttpClient implements BasicHttpClient {

private static final Logger LOGGER = LoggerFactory.getLogger(SslTrustHttpClient.class);
...
@Override
public Response execute(String httpUrl, String methodName, Map<String, Object> headers,
Map<String, Object> queryParams, Object body) throws Exception {
...
}
}

What Does the Above Code Snippets Do ?

It overrides the frameworks BasicHttpClient's "public Response execute(...)" method and returns a
"javax.ws.rs.core.Response".

Inside the "execute(...)" method, you simply use your own "custom HttpClient" to invoke the REST/SOAP endpoint,
then return the "Response" object after mapping your actual response.

History
You can find release history and more about the framework in the GitHub Zerocode README file.

Current release is 1.3.x, please visit zerocode in maven central for latest releases.

Motivation for this Open Source Library

Motivation for inception of Zerocode has roots in the country's one of the largest Digital Transformation Project where a simple
and well maintainable testing approach was needed due to:

Large number of micro services being used to achieve various workflows

Large number of data pipe lines to ingest various legacy system's data into project’s Big Data store.
Micro Services to be tested in isolation as well as integrated

Over several brain storming sessions, automation testers and developers came to a common conclusion to write tests:

That will allow anyone to change, share and maintain them easily
Avoid writing boiler plate sticky code.
Large numbers of tests to be well organized and business-readable
Local Laptop to be configured for multiple environments (localhost, ci/dev/dit, SIT, UAT, PRE-PROD, etc.) to fire the tests
easily like JUnit tests.

Outcome of the sessions was to consider a steps based scenario build-up approach where the underlying test and test data is
picked from a referenced JSON file.

YAML/JSON is widely supported by most popular IDEs and hence proved to be very efficient approach.
Looking Towards Contributing to this Open Source Library?
It's easy! Please start with raising an issue and follow the guidelines in CONTRIBUTING.md.

Who uses Zerocode?

Visit who uses section in the README file.

License
This article, along with any associated source code and files, is licensed under The Apache License, Version 2.0

About the Author

Sidd GautamaNo Biography provided
United Kingdom

Comments and Discussions

2 messages have been posted for this article Visit https://www.codeproject.com/Articles/1242569/REST-API-or-SOAP-
Testing-Automation-with-ZeroCode to post and view comments on this article, or click here to get a print view with messages.

Permalink Article Copyright 2018 by Sidd Gautama

Advertise Everything else Copyright © CodeProject, 1999-
Privacy 2019
Cookies
Terms of Use Web01 2.8.190921.1