Quality Agent: A Review of Your Project, in Order of Importance

Overview

A project can hold hundreds of tests and APIs. The Quality Agent looks at the APIs, tests and latest results together, puts what it finds in order of importance, and says what to do about each item.

It is a reviewer, not an operator:

• It reads results. It does not run a test.
• It does not change, rewrite or delete a test.
• At most, it can ask the failure-repair service to prepare fixes. They wait for you to review them, and nothing is applied by itself.

Before you begin

• A project with endpoints and at least one analysed test run. Failures are explained from the latest analysed run. If no run was analysed in the last two weeks, the review says so instead of guessing.
• To prepare fixes, an AI provider under AI Settings.

Step 1 — Find the panel

1. Open Project settings.
2. Find Quality agent under Failing tests.

If you cannot find it, your plan does not include it, or it is switched off for your installation. Ask your administrator. If you move to a plan with less, the agent stops at what that plan allows. Your settings are kept, so moving back brings them back.

Step 2 — Start a review

• Select Review this project now to run one straight away.
• Or tick Review this project on a schedule and choose every week or every day.

A review that is already running is not started twice. Earlier reviews are kept, so you can see how the project changed.

Step 3 — Choose how far it goes

On Professional, the panel offers only the first choice and says why.

Step 4 — Read the review

A review lists what it found, most important first. Every item says what is wrong and what to do next.

• APIs with failing tests, and which ones.
• Causes of failures from the latest analysed run. A cause that looks like a bug in the API is marked high. It is reported to be raised with the API's owners and is never "fixed" by changing the test, because a test that fails when the API is wrong is doing its job.
• Tests that pass and fail without any change. The agent points at them and never deletes them.
• Tests with no result lately, using the number of days you choose.
• APIs with no tests.
• Mocks that may be out of date, because the API's specification changed after the mock was made.
• Fixes already prepared and waiting for you.

It also says what it looked at: how many APIs and tests, and whether there was a recent analysed run. A project with nothing in it is reported as having nothing to review. It is never reported as clean.

Step 5 — Act on the result

Fixes the agent prepared are suggestions. Review each one, then apply the ones you accept. Use the usual run triage screens to understand a cause.

On an API's own page, a card shows the findings of the latest review that name the API. See The API page.

Where it fits with the release verdict

The review tells you what to fix first. The release verdict tells you whether to ship. Use the review to get to a ready verdict.

What it does not do

• It does not run tests. A project nobody has run lately is reported as having no recent run, and the next step is to run the tests yourself.
• It does not apply fixes by itself, set tests aside, write new tests, or run anything when a specification change is approved.
• It does not investigate root causes in databases, deployments or telemetry.
• It has no score.

Troubleshooting

• The panel is missing. Your plan does not include it, or your administrator turned it off.
• Only the first choice is offered. Preparing fixes needs Trial or Enterprise.
• The review says no run was analysed. Run a pack, open the run's analysis, and review again.

Related articles

• Release verdict
• Run triage: understand why a run failed
• Fix with AI: grounded repair
• Project settings