Evaluation-gated and progressive releases
On This Lesson 4 topics
This lesson is planned. The outline below lists the topics it will cover. The full lesson has not been published yet.
In this lesson
block regressions before users see them and limit exposure when releasing.
Planned topics
- An evaluation gate runs the candidate configuration over a fixed case set and compares scores to the current version; failing blocks the release.
- Include safety cases (leaking data, refusing unsafe requests) as hard gates, not averages.
- Then release progressively: shadow mode, small canary share, widen, with online signals compared to baseline.
- Online signals: thumbs-down rate, escalation to human, refund-policy violations found by sampling.
Draft outline details
Demonstration
Pipeline: change PR | offline eval gate | shadow run | 5% canary | 50% | 100%, each with promote and abort rules.
Check your understanding
Why make safety cases hard gates rather than part of an average?
Show answer
A single leak must not be hidden by high scores elsewhere.
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.