Understanding Analytics in Shift-Left API
How Analytics differs from Reporting: trends over time rather than one run. Covers project metrics, time ranges, and what each view answers.
Overview
Analytics & Dashboards turns your execution history into trends. It's reached from the top navigation and requires a Pro, Trial, or Enterprise plan.
The distinction that matters:
| Area | Question it answers | Scope |
|---|---|---|
| Reporting | "What happened in this run?" | One execution |
| Analytics | "What's been happening over time?" | Many executions |
If you're diagnosing a failure that just occurred, you want Test reporting, not Analytics. If you're asking whether quality is improving, whether a test is flaky, or which endpoint keeps breaking, you want Analytics.
Analytics is project-scoped — you'll be prompted to Select a Project first.
Time ranges
Every view is bounded by a range: Last 7 days, Last 30 days, Last 90 days, or Last year.
Pick the range to match the question. A regression introduced yesterday is invisible at Last year — the trend line barely moves. A gradual performance decline is invisible at Last 7 days — it looks like noise. Most "the dashboard doesn't show the problem" confusion is a range mismatch.
What's measured
| Metric | What it tells you |
|---|---|
| Total Tests | How many tests ran in the range. |
| Success Rate | The proportion that passed. |
| Coverage % | How much of your API surface is exercised. |
| Avg Response Time | Typical response time across runs. |
| Failed Tests | The count of failures. |
| Test Status Breakdown | Passed / Failed / Error / Not run over time. |
| Response Time Distribution | The spread of response times, not just the average. |
| Top Failing Tests | Which tests fail most often. |
Top Failing Tests is the most actionable panel in Analytics. A test that fails intermittently across many runs is either flaky or catching a real intermittent bug — and either way it's costing you trust in the whole suite. Neither is visible from a single report; you can only see it across time.
Response Time Distribution is worth more attention than Avg Response Time. An average hides the tail, and the tail is where users notice. A stable average with a widening distribution means some requests are getting much slower while most stay fine.
Reading Success Rate honestly
Success rate is the number most often quoted and most easily misread. Two cautions:
- Check it against Test Status Breakdown. A rate dragged down by Error means runs didn't complete — that's an environment problem, not an API quality problem, and "fixing tests" won't move it.
- Check it against Coverage %. A high success rate on low coverage means you're reliably testing a small slice. It is not evidence the API is healthy.
The pairing to watch is coverage rising while success rate dips slightly — that usually means you started testing things that were always broken, which is progress, not regression.
The views
| View | What it's for |
|---|---|
| Project Overview | The project's headline numbers. |
| Execution Summary | Run outcomes across the range. |
| Performance Summary / Performance Metrics | Response times and their distribution. |
| Endpoint Analytics | Which endpoints are healthy and which aren't. |
| Environment Comparison | The same tests across environments. |
| Trends Analysis | Movement over time. |
Environment Comparison answers a specific and common question: is this failure real, or is staging just broken? The same tests passing in one environment and failing in another points at the environment, not the contract.
For building views around a role or an audience, see Custom dashboard personas and Analytics dashboards. For one project in depth, see Project analytics.
Troubleshooting
- "No analytics data available" — no runs in the selected range. Widen the range, or check the project has actually run anything.
- "No endpoint analytics data available" — no per-endpoint data in range.
- "No environment comparison data available" — the tests haven't run in more than one environment, so there's nothing to compare.
- Analytics isn't in the navigation — it's gated on Pro, Trial, or Enterprise.
- Numbers disagree with a report you're looking at — check the range and project. Analytics aggregates many runs; a report is one.
- Success rate dropped with no code change — check Test Status Breakdown for a spike in Error. That's usually environment or connectivity, not quality.
Related articles
Related articles
- Analytics Dashboards: Panels and Views · Product documentation
- Project Analytics · Product documentation
- Regression Analysis · Product documentation
- Custom Dashboard Personas · Product documentation
- Sharing and Embedding Dashboards · Product documentation
- Scheduled Reports and Alerts · Product documentation
Next steps
- Getting started · Install + connect your spec
- Configuration fundamentals · Stabilize runs
- Initial configuration · Users, licensing, projects
- Release notes · Updates and fixes
Still stuck?
Tell us what you’re trying to accomplish and we’ll point you to the right setup—installation, auth, or CI/CD wiring.