End to End API Testing Framework: Visual Workflows and Workflow Doctor

This is lesson 9.8 of Shift-Left API in Depth, the product track of the learning center. In lesson 9.7, one test ran once for every row of a sheet. Now you chain tests together into one journey. Visual workflows and Workflow Doctor are in every edition of Shift-Left API, and the demo runs against the public sandbox store API.

How do you build an end to end API testing framework?

An end to end API testing framework does one job a single test can't: it follows a whole journey through your API. Create a customer, place their order, read it back, and check it all adds up. The questions are how you chain those calls, and how you know the chain really ran.

What is end to end API testing?

It tests a journey, not a single call: create a user, log in, create an order, read the order back, and clean up. Each step takes something an earlier step returned: the user's id, a token, the order's id. Those values go into a shared context, and later steps reuse them. At the end, a report shows how every step did.

So an e2e API testing framework needs five things:

• Steps that run in a fixed order.
• Values carried from one step to the next, so step three reads the order step two created.
• Checks at each step, on the real responses.
• A report that shows every step.
• A check on the wiring itself, because a chain can break without any call failing.

Passed is not the same as proven

A run can say passed and still not prove the journey. A success rate of 100% only counts the steps that ran, and no failed step doesn't prove it either. What does: every step ran, its checks passed, and the last step read back what the first steps wrote.

Open a workflow in Shift-Left API

The demo project, Store API Workflows, is built from the sandbox store's spec and has three tests: Create a user, Place an order, and Read the order back.

1. Click Workflows in the top bar. The project has one workflow, Order journey.
2. Click its Edit button to open it on the canvas.

Each block is one test. The green connectors set the order they run in: Start, Create a user, Place an order, Read the order back, Stop. The blue dotted lines carry data:

• Create a user publishes userId from its response, and a blue connector feeds it into Place an order's user_id.
• Place an order publishes orderId, which feeds the id in Read the order back's path.
• Read the order back also asks for expand=user,product, so the order comes back with its user and its product.

Steps come from the Test Library tab: drag a test onto the canvas, or click it. Node Types adds blocks in between: variables, data operations, control flow, assertions and more.

Run it, and read the node list

The workflow in the demo has one fault, left in on purpose.

1. Click Launch Workflow. It runs on the Shift-Left Engine, against the live sandbox.
2. Read the report. The success rate is 100%, but the node list reads Start, Create a user, Place an order. Read the order back never ran, so its checks never ran either, and the run still says passed.
3. Close the report.

Let Workflow Doctor find and fix it

1. Click Check workflow. Workflow Doctor reads the canvas, the steps and the runs, and reports four findings that stop it saving or running.
2. Read them top to bottom. Three say the same thing from three places: following the green connectors, nothing reaches Stop. The fourth names the cause: Read the order back never runs, because there's no path of green connectors from the Start block to it.
3. Click Why this matters: a block with no execution path from Start never runs, and the editor refuses to save a flow in this state.
4. Click the fix, which says exactly what it will do: Connect "Place an order" to "Read the order back". The Doctor checks again by itself, and has nothing to report.
5. Close the panel. The new green connector is there, but a fix stays on the canvas until you save it, so click Save.

Run it again, and check the last step

Click Launch Workflow again. Five nodes this time, all passed, and Read the order back is among them.

Click View Details, and open Read the order back, the journey's last step. The request URL carries the id of the order that Place an order just created, passed along by the blue connector. The response has the order with its user, Ada Byrne, and its product, a Wireless Mouse. All four checks passed: the status, the user's name, the product's name and the quantity. The last step read back what the first steps wrote: that is end to end testing for an API journey.

Try it yourself

On the sandbox store, make three tests: Create a user, Place an order, and Read the order back, with expand=user,product. Chain them on a canvas, and wire userId and orderId with blue data connectors. Run it, and count the steps that ran. Then delete one green connector, click Check workflow, and apply the fix.

Recap

• An end to end API testing framework follows one journey through your API: steps in order, values carried forward, checks on real responses.
• In Shift-Left API, a workflow chains your tests on a canvas: green connectors for the order, blue ones for the data. A 100% run only counts the steps that ran, so read the node list.
• When the wiring is wrong, Workflow Doctor names the step, says why it matters, and offers a fix you save yourself.

The free edition takes up to 50 workflows, of 10 steps each. Next, in lesson 9.9, you spin up mock servers from a spec, so a test can run before the real service is ready. For journey tests across services, read end-to-end API testing for microservices.