API Testing Using AI Agents: Permissions First, Autonomy Second
An agent that can import specs, generate tests and run them is useful in proportion to what it can't do. The loop, the MCP connection, a guardrail checklist, and Shift-Left API's MCP server.
This is lesson 8.7 of AI and Shift-Left Testing Strategy, the last in the track. Lesson 8.6 tested systems with a model inside; here a model drives the testing.
What API testing using AI agents means
API testing using AI agents means giving a language model tools (read the spec, list what is untested, generate tests, run them, read the results) and a goal such as "cover the new orders endpoint and tell me what fails". The agent then chooses its next step from what it observes, in a loop, until it reaches the goal or stops. The value is that loop: it keeps a developer in their editor and does the tedious lookups. The risk is the same loop running with more permission than anyone intended. Design the permissions first.
For the vendor landscape and the questions to ask before buying, read agentic QA tools. This lesson is about operating one safely.
How an API testing AI agent works
Every agent runs the same cycle:
- Observe. List projects, endpoints, environments and current coverage.
- Plan. Decide what's missing and what to do first.
- Act. Import a spec, generate tests, start a run.
- Check. Read the results and decide whether the goal is met.
- Repeat or stop, and report back.
In a sound setup, the agent doesn't send requests to the system under test itself; it asks a test platform to run tests, and the platform's runner does the sending. That separation is what keeps runs reproducible and auditable.
How agents connect: tools over MCP
The Model Context Protocol (MCP) is an open standard for connecting AI clients to external systems. An MCP server offers tools (actions the agent can call), resources (data it can read) and prompts (ready-made tasks). An MCP client, such as Claude Desktop or Cursor, discovers them and lets the model call them. Agent frameworks that make plain HTTP calls can use a REST API instead; MCP just saves everyone writing the same tool definitions.
A worked example: an agent covers a new endpoint
A developer has just added notes to orders and asks the agent, in their editor: "What's untested on Orders in the Store API project? Fill the gaps, run them on Development, and summarise failures." A well-behaved agent makes calls like these:
| Step | Tool call | Why |
|---|---|---|
| 1 | List projects, then list endpoints for Store API | Find the target |
| 2 | Explain coverage for the Orders endpoints | See what's missing, with reasons |
| 3 | Close coverage gaps | Generate exactly the missing tests |
| 4 | Run tests on Development | A named, non-production environment |
| 5 | Get run results | Read pass, fail and causes |
| 6 | Reply with a summary | Two failures on the notes length limit, with the test names |
And what it should not do, even if asked loosely: run on production, change expected results until tests pass, or start a load test.
Guardrails: a checklist for an AI agent for API testing
| Risk | Guardrail |
|---|---|
| The agent runs tests against production | Refuse production by default; allowing it takes an explicit, separate setting |
| The agent has more access than its user | A scoped key whose permissions can't exceed its creator's, with an expiry |
| A loop runs away | Rate limits per key and a cap on result size |
| The agent "fixes" tests until they pass | No permission to edit assertions, or proposals that a person applies |
| Results can't be reproduced | The agent triggers deterministic runs; generation without a model by default |
| Instructions hidden in an API response | Treat every tool result as data; use read-only mode for exploration |
| Prompts leave your network | Run the model, or at least AI-assisted generation, on a model you host |
| Nobody knows what the agent did | Every call goes through the platform's normal permissions and audit trail |
If a tool can't answer most of these rows, keep it to read-only work.
Where agents help today, and where they don't
They help with exploring an unfamiliar API, drafting coverage for a new endpoint, summarising a red run, and keeping a developer in flow. They don't replace review, they shouldn't be the merge gate, and they can't judge whether behaviour is right for the business. That judgement still comes from requirements and the people who own them.
In Shift-Left API: the MCP server for Claude and Cursor
Shift-Left API's MCP server lets Claude Desktop, Cursor or another MCP client work with your projects. It is in Professional, Enterprise and the 15-day trial; the free Citizen Developer edition doesn't include it.
How it's built. Your Shift-Left API server answers MCP tool calls over HTTPS. A small local client, started by your MCP client over stdio, calls it with a platform API key. The client isn't published on npm yet (an npm package is on the roadmap), so for now you run it from a local copy; contact support if you don't have one. There is no hosted remote MCP endpoint.
What the agent gets. 17 tools, 3 resources and 4 prompts: list projects, endpoints and environments; import a spec; generate tests; explain coverage; close coverage gaps; validate and run tests; read results; read load runs; and, with the API security testing add-on or a trial, read and start security checks.
The guardrails, as built:
- A run against an environment marked as production is refused, unless the tool call asks for it and the client was started with
SLS_ALLOW_PRODUCTION_RUNS=true. SLS_READ_ONLY=truegives the agent read access only.- Test generation is deterministic by default, with no model involved;
MCP_REQUIRE_LOCAL_AIkeeps any AI-assisted generation on a self-hosted model. - The key's permissions are its scopes intersected with its creator's, each key is limited to 300 requests every 5 minutes, and results are capped at 25 KB per call.
- An agent can read load runs but can't start a load test. Starting a security run needs its own permission, and production is refused.
Setting it up:
- As an administrator, open Settings → Integrations → AI Agents (MCP). Name the key, set when it expires, tick Read-only agent if the agent should only look, and click Create agent key. The key carries only the permissions the MCP tools need, never more than yours, and it is shown once.
- Under Run the client from, choose Local checkout and give the path to your copy of the client. The configuration for your AI client, Claude Desktop or Cursor for example, is filled in with
SLS_BASE_URL, your server's address, andSLS_API_KEY, the key. Paste it into the client's configuration. - Restart the MCP client and ask it to list your projects.
- Ask what is untested on a project built from the sandbox spec, ask it to fill the gaps, and to run the tests on a non-production environment.
Revoking the key in Access & Identity → API Keys disconnects the agent on its next call. One thing not to confuse: JSON-RPC 2.0 / MCP testing (Trial and Enterprise) is the opposite direction, where Shift-Left API tests your own MCP server; how to test MCP servers covers that.
Try it yourself
On a trial, as an administrator, open Settings → Integrations → AI Agents (MCP), name a key and click Create agent key. Read the configuration it fills in: which two settings does it set? Then find the key under Access & Identity → API Keys. If you have a local copy of the client, connect it and ask your agent to list your projects.
Recap
- API testing using AI agents is a loop of observe, plan, act and check; its value depends on what the agent is not allowed to do.
- An API testing AI agent connects through tools, over MCP or REST, and the platform's runner, not the agent, should send the test traffic.
- An AI agent for API testing needs a scoped key, a production lock, read-only mode for exploration, and a deterministic gate that stays in CI.
You've finished the track. Next, put it into practice from the first step: create a project and import a spec in Shift-Left API.
Try it: On a trial, as an administrator, open Settings → Integrations → AI Agents (MCP), name a key, click Create agent key, and read the client configuration the screen fills in.
Frequently asked questions
What can an API testing AI agent do that a script can't?+
It decides the next step from what it observes. Asked to cover a new endpoint, it can read the spec, see what is untested, generate tests for the gaps, run them and summarise the failures, adjusting as it goes. A script does the same steps in a fixed order. The flexibility is the value and the risk, which is why the agent's permissions matter more than its prompts.
Is an AI agent for API testing safe to point at production?+
Not by default. Give it a key scoped to the actions it needs, refuse production runs unless a person explicitly allows them, and keep data-changing and load tests away from production entirely. Read-only mode is a good start for exploration.
Does API testing using AI agents replace the CI gate?+
No. An agent is useful for exploring, drafting coverage and explaining failures inside a developer's editor. The gate that blocks a merge should stay a deterministic run with a written threshold, so the same commit always gets the same verdict.
Related lessons
Testing checks what must always be true; evaluation scores what can vary. A QA team's split of the work, a worked example, the open-source tools, and the same standard for AI-generated tests.
Shift-Left API doesn't ship a model of its own; you bring one. Pick a hosted provider with your own key or a self-hosted model, test the connection, and see which model reviewed your tests.
Every build runs the API tests, and a build that breaks them stops. See what a pipeline step does with a Test Run Pack, create a scoped API key, gate on a pass rate with JUnit results, and find the runs in Reporting.