Continuous Learning Insights
The Learning tab shows what your workspace has learned from reviewer edits and test runs—acceptance rate, edit distance, time to green, flake rate, and a generation-quality score.
Overview
The Learning tab of Rule Intelligence is a read-only dashboard headed What your workspace learned. It shows how the platform improves over time by observing your reviewer edits, approvals, deletions, and test-run outcomes — and how those signals change what the generator produces next. Explore the capability at continuous learning.
Before you begin
- Open the workspace (project editor → Requirements tab, or the standalone Rule Intelligence hub → select a project), then click the Learning tab.
- The subtitle reads "From your team's edits, deletions and test runs over the last N days. Learning changes what the generator produces next." (the window is typically 30 days).
- Nulls are normal on day one. Signals accumulate over time, so metrics without data yet show collecting data… rather than a number.
- An AI provider (Settings → AI) is optional here — learning runs either way, but an AI judge refines ambiguous signals when one is configured.
Step 1 — Read the KPI cards
Four cards summarize how well generated tests land:
| Card | What it measures |
|---|---|
| Acceptance rate | Generated tests kept (not deleted or heavily rewritten). Turns red below 50%. |
| Edit distance | Average structural change when users edit a generated test. |
| Time to green | Median time from generation to the first passing run. |
| Flake rate | Tests with both passing and failing runs in the window. Turns red above 10%. |
Step 2 — Check the generation-quality badge
Next to the header, a Generation quality NN/100 badge shows the latest golden-set structural evaluation for the current prompt version. It's green when the run passed and red when it didn't. This score is install-wide (not project-scoped), so its absence on a fresh install is expected.
Step 3 — Understand curation quality
The Curation quality card tells you how your learned labels were produced:
- Heuristic + AI-refined — "N ambiguous signal(s) judged by <model>" when an AI judge has refined signals.
- Heuristic-only — shown when no AI provider is configured, with a pointer to Settings → AI. Learning still works; the judge just isn't disambiguating suspected product bugs vs. bad tests, or corrections vs. preferences.
- Heuristic (default) — the judge refines ambiguous signals during nightly curation when it has something to decide.
Step 4 — Read "Learned so far"
The Learned so far card shows count pills for what curation has produced:
- endpoints curated
- field-name corrections
- suppressed test categories
- flaky tests identified
- objective corrections
- team preferences
- suspected product bugs (only when any exist)
Before any curation has run it reads: "Nothing curated yet. Edit, delete, and run generated tests — the nightly curation job turns those signals into generation improvements automatically."
Step 5 — Read the per-endpoint breakdown
When signals exist, a table lists up to 50 endpoints with columns Endpoint, Signals, Corrections (shown as wrong→correct), Suppressed categories, Flaky tests, and Last curated. If there are more than 50, a note reads "Showing first 50 of N endpoints."
Step 6 — Run curation on demand
Curation runs automatically on a nightly schedule, but you can force it now:
- Click Run curation now (the button shows Curating… while it runs).
- On success you get a summary such as "Curated N new signal(s) across N endpoint(s)," and — when the AI judge participated — "…N refined by AI judge."
- The dashboard refreshes with the new aggregates.
How it improves generation
The signals shown here — field-name corrections, suppressed categories, flaky-test flags, and reviewer preferences — feed back into future generation so it drifts toward what your team actually accepts. Applied and dismissed suggestions from Fix tests with AI are learning signals too.
Notes
- This tab never changes tests directly — it's an observation dashboard plus the manual Run curation now trigger.
- Empty metrics mean "not enough data yet," not an error; use the workspace normally and revisit after a few review-and-run cycles.
Related articles
Related articles
- Understanding Rule Intelligence · Product documentation
- Importing and Reviewing Requirements · Product documentation
- Test Data and Generation Setup · Product documentation
- Coverage and Extraction Fidelity · Product documentation
- Requirements Traceability Matrix · Product documentation
Next steps
- Getting started · Install + connect your spec
- Configuration fundamentals · Stabilize runs
- Initial configuration · Users, licensing, projects
- Release notes · Updates and fixes
Still stuck?
Tell us what you’re trying to accomplish and we’ll point you to the right setup—installation, auth, or CI/CD wiring.