Regression detection in CI
On This Lesson 4 topics
This lesson is planned. The outline below lists the topics it will cover. The full lesson has not been published yet.
In this lesson
catch slowdowns before they reach production.
Planned topics
- Run a small stable set of benchmarks on every change or nightly, on dedicated, consistent hardware.
- Compare against a baseline with a tolerance based on measured noise; use statistics, not a single run.
- Fail or warn with a report showing the difference and confidence.
- Keep the suite small and fast enough that people trust it; quarantine flaky benchmarks.
Draft outline details
Demonstration
CI job description: run JMH subset, store JSON results as artifacts, compare to main-branch baseline, fail when slower by more than the tolerance with significance.
Check your understanding
Why use a noise-based tolerance?
Show answer
Run-to-run variation would otherwise cause false failures.
Practice
Apply this lesson with the module's lab, exercises and quiz.
Marking a lesson complete is optional and you can undo it any time. Your progress stays in this browser.