Demo: rolling back a prompt change that broke refund answers
On This Lesson 3 topics
This lesson is planned. The outline below lists the topics it will cover. The full lesson has not been published yet.
In this lesson
detect and roll back a regressing prompt; prepares SRC-022-LAB1.
Draft outline details
Step by step
- Show config v42 and the change in v43: a shorter prompt that drops refund-window wording.
- Run the eval gate: refund cases fail; compare with v42 scores.
- Planned mistake: ship v43 to a canary without the gate because the average score looked fine. Fix: add per-category thresholds so refund cases gate alone.
- Watch the canary's escalation rate rise; abort the rollout.
- Roll back to v42 by changing the config pointer; confirm via logged bundle IDs.
- Add the failing cases to the eval set and write up the change.
Check your understanding
How was the rollback performed?
Show answer
By pointing back to the previous versioned configuration.
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.