# Read a Performance Report: Percentiles, Verdicts and SLOs

> When a performance run finishes, Shift-Left Studio builds one report for it. The same report is what you see on screen, what you export as HTML, JSON or JUnit, what a share link shows, and what the project assistant reads.

Source: https://totalshiftleft.ai/help-center/product-documentation/performance-testing-reports-and-slos

> **Applies to:** Professional, Trial and Enterprise editions · Web app and Desktop app · Anyone who can read test reports can read performance reports; creating performance targets needs permission to edit the project

## Overview

When a performance run finishes, Shift-Left Studio builds one report for it. The same report is what you see on screen, what you export as HTML, JSON or JUnit, what a share link shows, and what the project assistant reads. So every place tells the same story about a run.

The report is designed to be handed to a release board. It leads with anything that limits what the run proves, then the verdict in words, then what went wrong, then the figures. Every figure says where it came from, and every chart can be shown as a table.

This article also covers **performance targets** (SLOs): project-level objectives that Studio checks continuously against your ordinary functional runs, and that load tests can judge and report into the requirements traceability matrix.

## Key concepts

| Term | Meaning |
|---|---|
| **p50 / p95 / p99** | The response time that 50, 95 or 99 of every 100 requests were faster than. p95 and p99 describe the slow tail your users notice. |
| **Merged histogram** | Every response time is recorded in a histogram. Histograms from every interval (and every load generator) are merged, so a whole-run p95 is the true p95 of every request. |
| **Judged window** | The part of the run the verdict is based on: the hold stages, or everything except warm-up and ramp-down. |
| **Verdict** | **Pass**, **Fail**, **Inconclusive** or **No verdict**, with the reason in a sentence. |
| **Baseline** | The run later runs are compared with. You can pin one; otherwise each run is compared with the previous one. |
| **Performance target (SLO)** | A project objective such as "p95 under 800 ms and errors under 1%", optionally for one endpoint and one environment. |

## Before you begin

- At least one finished performance run. See [Create and Run a Performance Test](/help-center/product-documentation/performance-testing-create-and-run).
- To create performance targets: permission to edit the project.

## Step 1 — Open a report

Do one of the following:

- On the Performance home page, under **Recent runs**, click a run.
- Click **All runs and why they failed**, then **Open** beside a run.
- In **Reporting**, open the **Performance runs** tab and click a run.
- When a run you are watching finishes, the live monitor turns into its report.

## Step 2 — Read the top of the report

The top of the report is always in the same order.

### 1. Notices that limit what the run proves

Anything that weakens the figures is shown first. The most important is an overloaded load generator: if the machine sending the load ran out of CPU, its measurements describe that machine, not your API, and the verdict becomes **Inconclusive**. Other notices include:

- The data set ran out.
- Some scheduled requests were not sent on time.
- Some requests were still in flight when the run ended.
- A variable could not be filled in, or a value could not be taken from a response.
- A load agent stopped reporting.
- Percentiles rest on fewer than 1,000 requests, so they are less reliable.

### 2. The verdict, in words

| Verdict | Meaning |
|---|---|
| **Pass** | Every target held during the judged window. |
| **Fail** | At least one target was missed, or the run stopped itself because a target it was told to stop on was breached. |
| **Inconclusive** | The run cannot prove a result: for example the generator was overloaded, a load agent was lost, or a target could not be measured (no request fell in its window). An unmeasured target never counts as a pass. |
| **No verdict** | The scenario had no targets, so the run reports figures only. |

The verdict comes with a sentence, for example: "All 2 targets met during the hold at 212 requests/s; error rate 0.1%; p95 184 ms."

### 3. What went wrong

A run can fail for several reasons at once, and each needs a different fix. The **What went wrong** card lists every cause, grouped by where to look:

| Group | Examples of causes |
|---|---|
| **The run itself** | The check before load failed; the run broke before it finished; it was stopped early. |
| **The load generator** | The load generator was overloaded; a load agent stopped reporting. |
| **Network / environment** | The API could not be reached; the host name did not resolve; the TLS or certificate handshake failed; requests were rate-limited (429). |
| **The API under test** | Requests timed out; the API refused or dropped connections under load; it said it was overloaded (503); a gateway or proxy failed (502/504); server errors (5xx); a JSON-RPC or GraphQL error inside a successful reply. |
| **The test or its data** | Requests refused as unauthorised (401/403); credentials stopped working part-way through; not found (404); collisions with existing data (409); requests rejected as invalid; responses did not pass the test's checks. |
| **Targets missed** | Responses were slower than the target; a target could not be measured. |

Each cause shows how many requests it affected, when it began and the load at that moment (for example, "Began at 1:30, with about 180 virtual users"), what the server said, and **What to do**. Click **See the failed requests** to open the **Errors** tab filtered to that cause.

> **Note:** Studio tells "credentials were refused from the start" apart from "credentials stopped working part-way through", and "the API could not be reached" apart from "connections dropped under load". These point to very different fixes.

### 4. Key figures

Cards show requests, throughput, p50/p95/p99, errors, check failures and more, with a comparison to the baseline where there is one. Hover over a figure to see where it came from:

- **Exact — from the merged latency histogram of every request**
- **Per interval — from the per-second samples**
- **Derived** — calculated by a stated formula.

If an AI provider is configured, a short written summary may appear, labeled **Summary generated from the figures above**. It is written only from the report's own figures; a sentence containing a number that is not in the report is dropped.

## Step 3 — Explore the report tabs

| Tab | What it shows |
|---|---|
| **Overview** | Verdict, causes, key figures, and the main charts: response time over time, throughput and load, and the latency distribution. |
| **Latency** | **Response times by step**: count, throughput, mean, std dev, p50 to p99.9, max, errors and check failures for the whole run and each step. Switch between **Whole run** and **Judged window**. Also the percentile curve, which shows the slow tail. |
| **Errors** | Errors grouped by step, kind and status, each with a count, first and last time seen, **What the server said**, and redacted examples (**Show examples**). **Ask about this** opens the assistant on that group. |
| **Run log** | What happened, in order: queued, the check before load, each stage, the first time each kind of failure appeared (with the load at that moment), warnings, any automatic stop, and the end. |
| **Where the time goes** | Each step's time split into **Waiting**, **DNS**, **Connect**, **TLS**, **First byte** and **Download**, plus the **Slowest requests**. This is where "the API is slow" becomes "TLS handshakes are slow because connections are not reused". Phases the platform cannot measure show as not measured. |
| **Capacity & analysis** | The capacity curve and the analysis for the kind of test: spike recovery, soak drift, concurrency outcomes, rate-limit result, and the comparison with an earlier run. |
| **Generator** | The load generator's health: CPU, event-loop delay, overloaded intervals, and for a split run each generator's outcome and how late it started. Also links to your monitoring dashboards for the same time window, if the environment has any. |
| **What ran** | The scenario exactly as it was when this run started: load profile, workloads and steps. |

Every chart has **Show as table** and **Copy CSV** buttons. Shaded areas on time charts are warm-up and ramp-down: measured, but not judged.

### How the figures stay honest

- **Percentiles are never averaged.** The p95 of the whole run is the true p95 of every request, computed from merged histograms. It is never an average of per-second p95 values, which would hide peaks.
- **The judged window excludes warm-up and ramp-down.** A load test is judged on its hold, never on its ramp.
- **One error definition everywhere.** An error is a request that got no response, or a status the test did not expect. A JSON-RPC or GraphQL error inside a successful HTTP reply also counts.
- **What the server said splits a status.** A 500 can mean a database pool ran out or a defect in a new release, so each error group lists the messages the server sent back, read from the first 200 failed responses of that group.

## Step 4 — Compare runs and track trends

1. Open **More** in the report header.
2. Choose one:
   - **Pin as baseline** — later runs are compared with this one. (**Clear baseline** removes it.)
   - **Compare with another run** — pick a run under **Compare with…** and see headline figures, each step and percentile curves side by side.
   - **Trend of this scenario** — each point is one run, judged on its steady stage. ▲ marks a run significantly slower than the one before; ◆ is the pinned baseline; amber means the generator was overloaded.

A change is called significant only when a statistical test over every interval's p95 says so and the difference is large enough to matter. A change within the baseline's normal variation is not called a regression. Without a pinned baseline, a run is only compared with a previous run that used the same environment, kind and load profile.

## Step 5 — Export, share and annotate

- **Export** offers:

| Option | Use it for |
|---|---|
| **HTML report** | Self-contained; open or print to PDF anywhere. Charts are embedded with their tables. |
| **Report data (JSON)** | The full report document for your own tools. |
| **JUnit XML for CI** | One test case per target, plus a run-verdict case, so a pipeline fails when the run fails. It includes why the run failed. |
| **Percentiles (CSV)** | The percentile table. |
| **Per-second samples (CSV)** | Every interval's figures. |

- **Share** creates a read-only link. Choose how long it lasts under **Expires after** (1, 7, 14, 30 or 90 days) and click **Create link**. Anyone with the link sees the figures, charts, causes and their counts. Request and response examples, what the server said, who ran the test, configuration and monitoring links are left out. Use **Revoke link** to stop it working.
- **Add note** marks a moment on every chart of the run, such as "deployed 2.3.1" or "cache flushed here". Enter the time as **At (m:ss into the run)** and the **Note**.
- **Run again** starts the same scenario.

## Step 6 — Ask the assistant

At the bottom of the report, click **Ask the assistant**. The project assistant reads this report and can explain it in plain words, starting with any warning about the load generator. On the **Errors** tab, **Ask about this** opens it on one error group. The assistant can also offer to pin a baseline, create a target or start a run, but nothing happens until you accept its offer.

## Step 7 — Set up performance targets (SLOs)

Performance targets give you a performance signal without running a load test.

1. Open the project's settings and choose the **Performance targets** tab.
2. Click **Add target**.
3. Fill in the form:

| Field | Meaning |
|---|---|
| **Name** | For example "Checkout on staging". |
| **Environment** | Empty means every environment. |
| **Endpoint** | Empty means every endpoint in the project. |
| **p95 under (ms)** | 95 of every 100 responses must be faster than this. |
| **p99 under (ms)** | The slow tail: 99 of every 100 responses. |
| **Errors under (%)** | The share of requests allowed to fail. |
| **At least (req/s)** | Minimum throughput. Only load tests measure this; ordinary runs leave it unmeasured. |
| **Judge over the last (runs)** | How many recent functional runs the target is judged on. |

4. Click **Save target**. Set at least one target value.

> **Tip:** Under **Suggested from requirements**, Studio lists requirements that state a response-time limit and have no target yet. Click **Add as SLO** to create one linked to that requirement.

### How targets are judged

- **Continuously, against ordinary test runs (passive).** Studio checks the response times of functional runs of the endpoint over the recent-runs window. Click **Evaluate now** to refresh. Each target shows a status:

| Status | Meaning |
|---|---|
| **Met** | Every target value holds. |
| **At risk** | It holds, but is within 10% of the limit. |
| **Breached** | At least one value is over the limit. |
| **Not enough data** | Too few runs to judge. |
| **Not evaluated yet** | It has not been judged yet. |

  The Performance home page shows these statuses in the **Performance targets (SLOs)** panel.
- **Inside a load test.** On the designer's **Targets & SLOs** step, tick the targets under **Use project SLOs**. The report's **Targets** section then lists them as **Met**, **Missed** or **Not measured**, under **Service-level objectives**.

### The "Under load" column in the traceability matrix

When a load test judges a target that is linked to a requirement, the requirements traceability matrix shows the result in its own **Under load** column, as **met** or **missed**. Hover over it to see which load test, which environment and the observed figures.

That column never replaces the functional verdict. A requirement can pass all its functional tests and still miss its target under load, and you need to see both. Only runs that completed with a real verdict count; stopped, broken or inconclusive runs are not used as evidence. The report also lists them under **Requirements verified under load**.

## Troubleshooting

| Symptom | Why it happens | What to do |
|---|---|---|
| The verdict is **Inconclusive** and the first notice is about the load generator. | The machine sending load was the bottleneck, so latency describes it, not the API. | Lower the load, or split the run across load agents. |
| A target shows **Not measured**. | No request fell in the window it is judged on, for example a "during the hold" target in a profile with no hold stage. | Add a hold stage, or change the target's scope. |
| The spike analysis says recovery could not be measured. | The profile after the spike is too short. | Lengthen the last stage and run again. |
| A percentile notice says there were few requests. | Fewer than 1,000 requests were judged. | Run longer, or with more load, before relying on p99. |
| A comparison says "within noise" though the numbers differ. | The difference is inside the baseline's normal variation. | Treat it as no change, or run more times to see a trend. |
| An SLO stays at **Not enough data**. | Too few functional runs of that endpoint in the window. | Run the tests more often, or lower **Judge over the last (runs)**. |
| A shared link shows no request examples. | Share links leave out examples and server messages by design, to avoid exposing data. | Export the HTML report for a colleague who needs full detail. |

## Best practices

- Read the notices before the verdict. A pass from an overloaded generator proves nothing.
- Pin a baseline from a clean run in a stable environment; re-pin after a deliberate change.
- Add a note when something happens during a run (a deployment, a failover) so the charts explain themselves later.
- Use the **JUnit XML for CI** export to gate a pipeline on performance targets.
- Link targets to requirements so the traceability matrix shows performance evidence beside functional evidence.
- Prefer p95 and p99 targets over mean latency; averages hide the slow requests users notice.

## FAQ

**Why are performance runs not in my functional pass-rate charts?**
A load run is thousands of requests judged against targets, not one pass or fail per test. Counting it as a test would distort every pass-rate chart. Performance runs are listed on their own, in the **Performance runs** tab of **Reporting**.

**Is the written summary an AI root-cause analysis?**
No. It is a short summary written only from the figures in the report. The causes in **What went wrong** are worked out by fixed rules, without a model.

**Can I get a PDF?**
Export the **HTML report** and print it to PDF from your browser.

**Why does my report's p95 differ from the average of the per-second p95 values on the chart?**
Because the report's p95 is the true p95 of every request, from merged histograms. Averaging per-second percentiles gives a wrong number.

## Related articles

- [Performance Testing: Load, Stress, Spike, Soak and More](/help-center/product-documentation/performance-testing-overview)
- [Create and Run a Performance Test](/help-center/product-documentation/performance-testing-create-and-run)
- [Requirements Traceability Matrix](/help-center/product-documentation/requirements-traceability-matrix)
- [Reporting](/help-center/product-documentation/test-reporting)
- [Project Assistant](/help-center/product-documentation/project-assistant-overview)