API Testing Requirements: From Documents to Traceable Tests

This is lesson 9.2 of Shift-Left API in Depth, the product track of the learning center. For the demo you need the Store API project from lesson 9.1, the free edition of Shift-Left API with an AI provider set up, and the sandbox store's requirements document.

How do you turn API testing requirements into tests?

In lesson 9.1 you generated tests from the API spec alone. A spec says what is valid. It doesn't say what the business needs, so an API can pass every one of those tests and still break a business rule. API testing requirements close that gap: they are the rules your API must follow, and each one tells you what to test.

The purpose of API testing

The purpose of API testing is to prove that an API does what the business asked for. What the business asked for lives in requirements. The sandbox store's requirements document is a good example of where they hide. In order, it holds:

• its purpose and scope
• system-wide behaviour, the rules every request follows
• the functional requirements, one for each feature
• use cases, the main journeys through the store
• field constraints and business rules
• test data, request by request
• end-to-end scenarios that tie it all together

On real projects these are spread across specs, spreadsheets, user stories and backlog exports.

What to test in API testing, rule by rule

Take one business rule from that document, RULE O-01: order quantity is an optional whole number of at least one, and it defaults to one. That one sentence tells you what to test in API testing:

• Positive tests: a quantity of 1 or 3 is accepted.
• Negative tests: 0, -1 or 1.5 are rejected with a validation error.
• A default to check: what happens when quantity is left out.

That is positive and negative API testing, straight from one rule. Now multiply it by every rule. The store's document runs to about 1,800 lines. Written by hand, some rules get tested twice and some never, and when a test fails, nobody can say which requirement it was protecting.

Upload a requirements document

Reading requirement documents uses AI, so set up a provider first: open Settings, then AI, and add your own API key or point it at a self-hosted model.

1. Open the Store API project from lesson 9.1. Users has tests generated from the spec; Orders has none.
2. Click Requirements in the top bar.
3. Leave Extract automatically ticked, click Choose requirement files, and pick the store's requirements document.

You can add Word, PDF, Excel, CSV, Markdown, plain text or Gherkin feature files, and backlog exports from Jira and Azure DevOps. Shift-Left API reads the document and extracts every requirement. Long documents are read in parts, and when you add more documents, each new one is checked against the ones before it, so a rule stated twice is counted once.

Review what it extracts

Click Review all requirements. Requirements interpreted from the document's prose wait for your review. The ones that come from structured parts, like the spec, the test-data tables and the status-code list, are approved automatically, because there was nothing to misread.

1. Type quantity in the search and open the approved group. RULE O-01 is there as approved requirements. Each card shows, from the top, its title, what it says (a quantity of 1.5 must be rejected), its type, where it came from in the document, the status code it expects, how confident the extraction is, and the endpoints it applies to. From your API spec, it adds that quantity must be one or more and defaults to one.
2. Clear the search and check the filters, left to right: requirements that would generate no test, conflicting values, low fidelity, and salvaged extractions. Look at any that need your attention.
3. Click Select all, then Approve selected. Every requirement is now approved and used in generation.

Generate tests from approved requirements

Open Orders in the project tree, click Create a new order, then Generate tests. In lesson 9.1 the summary said requirements were not used. Now it reports how many approved requirements matched this endpoint and how many tests they added. It also lists what it couldn't cover, like the quantity default, so you know what still needs a manual test.

Find the gaps with Traceability and Coverage

• Click Requirements, then Traceability. Each requirement is listed with its operation, its status, the document it came from and the tests that verify it. At a glance you can see which requirements no test protects yet.
• Click the Coverage tab. It maps every endpoint against every test dimension: functional status, schema contract, HTTP semantics, auth depth and more. Rows that say no tests are your gaps; rows with tests show their counts. Generate next where the gaps are.

Try it yourself

Download the store's requirements document, upload it to your Store API project, and search the review list for quantity. Approve the requirements, then generate tests for Create a new order. In the summary, how many requirements were matched?

Recap

• API testing requirements are the rules your API must follow, and the purpose of API testing is to prove it follows them.
• Each rule tells you what to test in API testing: positive and negative tests for every limit, plus any default.
• Traceability, from requirement to endpoint to test, shows exactly where the gaps are.

Specs don't stand still. Next, in lesson 9.3, you see how Shift-Left API spots changes when your spec moves on, and what to do about breaking ones. For the bigger picture of what API testing covers and why, read the API testing guide.