Evaluations in CI
On This Lesson 4 topics
This lesson is planned. The outline below lists the topics it will cover. The full lesson has not been published yet.
In this lesson
make quality a pipeline gate.
Planned topics
- Run the evaluation set on every change to prompts, model version, retrieval config, or tools.
- Fail the pipeline when a metric falls below its threshold or drops more than an allowed margin from the baseline.
- Control cost and flakiness: cache retrieval results, fix the temperature, run a fast subset on every PR and the full set nightly.
- Store results with the prompt and model version for comparison.
Draft outline details
Demonstration
pipeline stage: eval job → report artifact → threshold check → pass or fail.
Check your understanding
What triggers an evaluation run?
Show answer
A change to prompt, model, retrieval, or tools.
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.