Product documentation
Updated July 6, 2026

Continuous Learning Insights

The Learning tab shows what your workspace has learned from reviewer edits and test runs—acceptance rate, edit distance, time to green, flake rate, and a generation-quality score.

Overview

The Learning tab of Rule Intelligence is a read-only dashboard headed What your workspace learned. It shows how the platform improves over time by observing your reviewer edits, approvals, deletions, and test-run outcomes — and how those signals change what the generator produces next. Explore the capability at continuous learning.

Before you begin

  • Open the workspace (project editor → Requirements tab, or the standalone Rule Intelligence hub → select a project), then click the Learning tab.
  • The subtitle reads "From your team's edits, deletions and test runs over the last N days. Learning changes what the generator produces next." (the window is typically 30 days).
  • Nulls are normal on day one. Signals accumulate over time, so metrics without data yet show collecting data… rather than a number.
  • An AI provider (Settings → AI) is optional here — learning runs either way, but an AI judge refines ambiguous signals when one is configured.

Step 1 — Read the KPI cards

Four cards summarize how well generated tests land:

CardWhat it measures
Acceptance rateGenerated tests kept (not deleted or heavily rewritten). Turns red below 50%.
Edit distanceAverage structural change when users edit a generated test.
Time to greenMedian time from generation to the first passing run.
Flake rateTests with both passing and failing runs in the window. Turns red above 10%.

Step 2 — Check the generation-quality badge

Next to the header, a Generation quality NN/100 badge shows the latest golden-set structural evaluation for the current prompt version. It's green when the run passed and red when it didn't. This score is install-wide (not project-scoped), so its absence on a fresh install is expected.

Step 3 — Understand curation quality

The Curation quality card tells you how your learned labels were produced:

  • Heuristic + AI-refined"N ambiguous signal(s) judged by <model>" when an AI judge has refined signals.
  • Heuristic-only — shown when no AI provider is configured, with a pointer to Settings → AI. Learning still works; the judge just isn't disambiguating suspected product bugs vs. bad tests, or corrections vs. preferences.
  • Heuristic (default) — the judge refines ambiguous signals during nightly curation when it has something to decide.

Step 4 — Read "Learned so far"

The Learned so far card shows count pills for what curation has produced:

  • endpoints curated
  • field-name corrections
  • suppressed test categories
  • flaky tests identified
  • objective corrections
  • team preferences
  • suspected product bugs (only when any exist)

Before any curation has run it reads: "Nothing curated yet. Edit, delete, and run generated tests — the nightly curation job turns those signals into generation improvements automatically."

Step 5 — Read the per-endpoint breakdown

When signals exist, a table lists up to 50 endpoints with columns Endpoint, Signals, Corrections (shown as wrong→correct), Suppressed categories, Flaky tests, and Last curated. If there are more than 50, a note reads "Showing first 50 of N endpoints."

Step 6 — Run curation on demand

Curation runs automatically on a nightly schedule, but you can force it now:

  1. Click Run curation now (the button shows Curating… while it runs).
  2. On success you get a summary such as "Curated N new signal(s) across N endpoint(s)," and — when the AI judge participated — "…N refined by AI judge."
  3. The dashboard refreshes with the new aggregates.

How it improves generation

The signals shown here — field-name corrections, suppressed categories, flaky-test flags, and reviewer preferences — feed back into future generation so it drifts toward what your team actually accepts. Applied and dismissed suggestions from Fix tests with AI are learning signals too.

Notes

  • This tab never changes tests directly — it's an observation dashboard plus the manual Run curation now trigger.
  • Empty metrics mean "not enough data yet," not an error; use the workspace normally and revisit after a few review-and-run cycles.

Related articles

Next steps

Still stuck?

Tell us what you’re trying to accomplish and we’ll point you to the right setup—installation, auth, or CI/CD wiring.