Postman Testrunner Framework

Overview

The Postman Testrunner Framework was born out of a need to test the API serving our mobile banking apps in a large, integrated test environment, over which we had little control.

Our API platform acts as an aggregator of several core online banking systems (OLB's), each serving multiple financial institutions (FI's), and each with its own API data/interface contract.

The integrated environment was used by hundreds of staff, across the company, and the data setup constantly change for the various testing needs. Testing here is never deterministic, it has to be opportunistic!

We needed a simple check of API functionality, capable of exercising all integration paths across the range of different OLB data interfaces, and FI/user configurations. It had to be flexible to run for a range of users, FI's, and OLB's, as well as for different deploy instances of our platform.

We captured the API calls from the mobile app (Using Fiddler, Burp Suite, and MITM Proxy) and designed a Postman solution that emulates the app's sequence of API calls.

For a test user to be able to start using the application they need to select the test environment that has an instance of the platform, a code to select the FI, and the user's credentials (as well as selecting the type of device). Thereafter, the information required must be obtained in preceding calls. For example, if you want to test a transfer scenario, you need to first obtain a list of their accounts.

The development teams had been using Postman for over a year and had built up a collection with 100+ endpoints and requests, many with test scripts that extracted data from the responses and saved them to the globals. The task, then, meant we needed to simply orchestrate the correct sequence of API requests to fulfillx the feature test scenarios. Finally, we need to handle different responses. We may receive different success or failure codes, but also we may receive information that makes the feature incapable of being executed, eg, if the feature is switched off for the FI, or a user isn't configured to permit it.

Postman Object Model

classes in Code lib

Settings & Variables

orthogonal syntax cascading over riding values

TFS

powershell script variables builds

- base
- Fi's
- time schedule

build agents

PTFWeb

Dashboards of results