Generate API Test Cases Using AI: From a Requirement to a Reviewed Suite

This is lesson 8.5 of AI and Shift-Left Testing Strategy. Lesson 8.4 set the rules for AI in a pipeline; here you apply them to the first step, writing the cases.

How to generate API test cases using AI

To generate API test cases using AI you can trust, give the model three things: the contract (the spec for the endpoint), the rule (the requirement, with its ID and exact wording) and the format you want back. Ask for test cases as data, not test code, and require every expected outcome to quote its source. Then review the cases against those sources before any of them runs. The model is drafting; the reviewer is still the oracle.

The mechanics of how generators turn a schema into cases are in the AI API testing guide and in how AI generates API tests from OpenAPI. This lesson is the working method.

What to give the model

Leave out anything the model shouldn't use as an oracle, such as responses recorded from the current implementation. If the expected result comes from what the code does today, the case can never find a defect.

A prompt that returns reviewable cases

You are drafting API test cases for a human reviewer.
Use only the sources below. Do not invent fields, limits or status codes.

Endpoint: POST /api/v1/orders   (OpenAPI excerpt attached)
Requirement O-02 (quoted): "status — optional enum: pending, processing,
completed, cancelled. Default "pending". Invalid → 400: "shipped", "PENDING", ""."

Return JSON: an array of cases, each with
  id, requirement_id, title, request_body, expected_status,
  expected_fields, source_quote
Rules:
- One invalid input per negative case.
- Every expected value must be supported by source_quote.
- If the sources don't say what should happen, put the case in
  "open_questions" instead of guessing.

A good answer looks like this (shortened):

{
  "cases": [
    { "id": "O02-01", "requirement_id": "O-02", "title": "Accepts processing",
      "request_body": { "status": "processing" }, "expected_status": 201,
      "expected_fields": { "status": "processing" },
      "source_quote": "status — optional enum: pending, processing, completed, cancelled" },
    { "id": "O02-02", "requirement_id": "O-02", "title": "Defaults to pending",
      "request_body": {}, "expected_status": 201,
      "expected_fields": { "status": "pending" },
      "source_quote": "Default \"pending\"" },
    { "id": "O02-03", "requirement_id": "O-02", "title": "Rejects wrong case",
      "request_body": { "status": "PENDING" }, "expected_status": 400,
      "expected_fields": { "error": "VALIDATION_ERROR" },
      "source_quote": "Invalid → 400: \"shipped\", \"PENDING\", \"\"" }
  ],
  "open_questions": [ "Is status: null rejected or treated as the default?" ]
}

The open question is the most valuable line in the answer. It is a gap in the requirement, found before anyone codes against it.

The review checklist

Before a generated case joins the suite, check that:

1. It cites a real source. The quote exists in the spec or the requirement, word for word.
2. It invents nothing. No field, status code or limit that isn't in the sources. "Quantity at most 100" is a common invention: the store's rule sets no maximum.
3. The expectation comes from the source, not the server. An expected 500 almost always means the model mistook a bug for behaviour.
4. Each negative case breaks one thing. A request that is wrong in two ways can fail for the wrong reason and still pass.
5. The data is safe. Seeded or generated records, a non-production environment, and nothing that collides with another test.
6. It isn't a duplicate. Several cases often exercise the same rule with different values; keep the ones at the edges.

What AI adds, and what you shouldn't use it for

Boundaries, required fields, enum values and declared status codes are enumerable. A deterministic generator produces them the same way every time, so spending model calls on them adds variance and cost. Use the model where judgement is needed: reading prose rules that the schema can't express, choosing realistic values, spotting cross-field implications ("a cancelled order can't be set back to processing", if your rules say so), and turning a use case into an ordered journey. Typical mistakes to watch for are invented fields, invented limits and expectations copied from the current implementation.

Playwright + AI + API testing

If your team writes API tests in Playwright, keep the AI's output as data and the test as a short loop. The review happens on the JSON file, in the same pull request:

import { test, expect } from '@playwright/test';
import { readFileSync } from 'node:fs';

const { cases } = JSON.parse(readFileSync('tests/orders-o02.cases.json', 'utf8'));

for (const c of cases) {
  test(`${c.id} ${c.title}`, async ({ request }) => {
    const res = await request.post('https://demo.totalshiftleft.ai/api/v1/orders', {
      data: c.request_body,
    });
    expect(res.status()).toBe(c.expected_status);
    const body = await res.json();
    for (const [field, value] of Object.entries(c.expected_fields)) {
      const actual = field === 'error' ? body.error : body.data?.[field];
      expect(actual).toBe(value);
    }
  });
}

The same pattern works with pytest, REST Assured or any runner that can loop over data.

In Shift-Left API: fill coverage gaps from requirements

Requirements Intelligence in Shift-Left API applies this method at project scale. Lesson 9.2 covers uploading and reviewing the requirements document; this is the step after it. Everything below is in every edition, Free included, and needs an AI provider. On a trial licence, AI gap fill and Fix with AI share a default cap of 10 calls a day.

1. Find the gaps. With the store's requirements approved, open Requirements → Coverage. The per-endpoint coverage matrix shows tests per endpoint and dimension; an empty cell names the techniques that would cover it.
2. Ask for candidates. Click Fill coverage gaps. The model proposes a few candidate requirements for each open gap. Each candidate passes the same quality checks as an extracted requirement, and a message reports how many were created.
3. Review them. Open Review all requirements. The new candidates are AI-extracted and wait for a person: approve, edit or reject each one. Only approved requirements drive generation.
4. Generate for the gaps. On an endpoint, open its coverage panel and click Generate tests for gaps. The coverage engine plans the missing tests from the spec and the approved requirements; with AI set up, it adds a few more per endpoint, and the summary shows which came from where.

The order matters: AI drafts requirements, a person approves them, and the tests come from what was approved.

Try it yourself

With the store's requirements approved in your project, open Requirements → Coverage and click Fill coverage gaps. What does the message say? Then open Review all requirements and find the new candidates: what would you approve, and what would you reject?

Recap

• To generate API test cases using AI, give the model the contract, the quoted rule and an output format, and review every case against its source.
• An API testing with AI course should teach the review as much as the prompt: no invented fields, no expectations copied from the server.
• With Playwright + AI + API testing, keep the cases as reviewed data and the test as a loop.

Next, in lesson 8.6, you test the other direction: applications that have a language model inside them.