Teaching the Assistant: Feedback and Learned Notes

Overview

The Project Assistant in Shift-Left Studio learns from you, one project at a time. When an answer misses something about your project, such as "our 401s come from the API gateway, not the API", you can tell it once. The assistant writes that down as a learned note and reads it at the start of every later conversation about that project.

You teach it in two ways: rate an answer and say what it should have said, or simply say it in the conversation. Every note is visible in one list, and you can stop using any of them at any time. Nothing that shapes future answers is hidden.

Notes are preferences, not permissions. A note can change how the assistant explains things or what it pays attention to. It cannot let the assistant skip checking your data, change things without your approval, or accept a failure as a pass. Notes that try are refused, and you are told why.

Key concepts

Before you begin

• You can open the assistant (see Project Assistant: Ask Questions About Your Project).
• Learning is per project. Open the project the note is about.
• Anyone who can view test cases in the project can teach notes and stop using them. Notes affect everyone's conversations in that project, so agree on team conventions.

Step 1 — Rate an answer

Every answer has two small thumb buttons under it: This answer was helpful and This answer was not helpful.

1. Click a thumb. Your rating is recorded immediately.
2. A prompt appears beside the thumbs:
   • After thumbs up: What should I keep doing?
   • After thumbs down: What should I have said?
3. Optionally, type a note in the box, for example "always tell me which environment a run went to" or "the /orders 500s are a known sandbox outage". Notes can be up to 600 characters.
4. Click Remember or press Enter.
5. The line reads Noted — I'll remember that on this project.

Why a thumbs-down alone teaches nothing. "That was wrong" tells the assistant an answer missed, but not what would have been right. If Studio turned a bare thumbs-down into a note, it would be writing a sentence you never said into every future conversation. So the rating is counted, and it appears in the rating summary, but only the words you type become a note. If you have a minute, answer "What should I have said?"; that is the part that changes future answers.

Step 2 — Or say it in the conversation

You do not have to use the thumbs. Corrections and instructions typed into the conversation are picked up as you go:

• "No, that's wrong — the /orders 500s are a known sandbox outage."
• "Remember that our 401s come from the gateway, not the API."
• "Always tell me which environment a run went to."

When the assistant takes a note from your message, a line appears under its reply: "Noted for this project: “…”". The note applies straight away: the reply to that same message is written already knowing it.

A few things to know:

• A question is not a correction. "Is that wrong?" or "Why is this incorrect?" is treated as a question and answered. "That's wrong. Why did you say that?" counts as a correction, because you stated a verdict first.
• A reaction alone is not a note. "That's wrong" with nothing after it teaches nothing, for the same reason as a bare thumbs-down.
• Your message is always answered. Feedback never replaces the reply. "Thanks, that's right" still gets a response.

Step 3 — Review what the assistant has learned

1. Open the assistant from the launcher (the project-wide panel).
2. At the bottom of the panel, click What I've learned about this project. The number beside it is the count of notes in use.
3. The list opens and shows:
   • A rating summary, for example "12 helpful · 3 not helpful, in the last 30 days."
   • Each note, with "said N times" when it has been reinforced.

If there are no notes yet, the list says so and explains how to add one.

Note: The list is shown only in the project-wide panel, not inside the drawers for failure causes or workflows. Notes taught in a drawer still appear here.

Step 4 — Stop using a note

1. In What I've learned about this project, find the note.
2. Click the trash icon beside it (tooltip: Stop using this).
3. The note disappears from the list and is no longer read by the assistant.

The note is retired, not erased. Studio keeps a record of who retired it and when, so "why did it stop saying that?" always has an answer. If you, or a teammate, teach the same thing again later, the note comes back into use.

Understanding the results

Notes the assistant refuses

A learned note is read on every later conversation, so it could be used to switch off the assistant's safeguards. Studio refuses, at the moment of writing, any note that would:

A refused note always comes with a sentence saying why. If you saw no confirmation and no reason, the note was not stored. Try saying it again as one plain instruction.

Limits

Recording feedback never interferes with your question. If a note cannot be saved for any reason, you still get your answer.

Troubleshooting

Best practices

• Teach facts about your environment that the data cannot show: known outages, gateway behavior, which environment is authoritative.
• Teach preferences about answers: "lead with the environment", "summarize in three bullets".
• Keep each note to one instruction.
• Review the list when a project changes (a new gateway, a retired environment) and retire notes that no longer hold.
• Never put credentials in a note.

FAQ

Who can see our notes? Anyone who can open the project's assistant panel. Treat notes as shared team knowledge.

Do notes change generated tests? No. Notes affect the assistant's answers. Test generation learns separately from how you edit and run tests.

Can a note make the assistant do something it normally would not? No. Notes rank below the assistant's rules, and notes that ask it to skip checks, act without approval or accept failures are refused when you write them.

Can I get a retired note back? Yes. Teach the same thing again; it returns with its history.

Related articles

• Project Assistant: Ask Questions About Your Project
• Project Assistant: Proposals You Approve
• Project Assistant: Create Projects, Endpoints, Tests and Workflows

For administrators (self-hosted installations)