Measuring impact
On this page 9 topics
Overview
Topics
- Adoption
- reliability
- cost
- delivery
- developer experience
Learn first
3.5 h of practice (estimate)
Learning outcomes
- Define success measures for organization-wide work before starting it: adoption, reliability, cost, delivery, developer experience.
- Separate leading from lagging indicators and attribute impact honestly.
- Build a simple impact dashboard leadership trusts.
- Report impact without overstating it.
Lessons
Lab and exercises
Lab
STF-019-LAB1 · Success metrics
Outcomes: 1–4 · Time: ~3.5 h
- Take the STF-009 baseline and choose metrics for adoption, lead time, incidents, cost, and developer satisfaction.
- For each, write the definition, data source, owner, target, and whether it is leading or lagging.
- Build a one-page dashboard design with trend, target, definitions, and freshness.
- Write the attribution method (control group or prior trend) and a mock impact report with honest caveats.
- Review with the assessor.
Acceptance: becomes the capstone's success metrics; each metric has a data source.
Exercises (case-based; six exercises)
| ID | Outcome | Task |
|---|---|---|
| STF-019-EX01 | 1 | Choose five success measures for the golden path with definitions and baselines, and drop two vanity metrics from a proposed list. |
| STF-019-EX02 | 1 | A sponsor wants one number. Propose it, with its limits, and say what you will also show. |
| STF-019-EX03 | 2 | Pair three leading indicators with the lagging indicators they should predict, and say how you will test the link. |
| STF-019-EX04 | 2 | Incidents were already falling 5% a quarter before the program; they fell 12% after. State how much you claim and how you say it. |
| STF-019-EX05 | 3 | Design a one-page dashboard: metrics, layout, data sources, and what you leave out. |
| STF-019-EX06 | 4 | Write the "what did not work" section of an impact report where satisfaction stayed flat and cost rose. |
Show solutions
- EX01 Measures: teams on golden path, lead time for changes, incident count and duration, cost per service, developer satisfaction. Drop: lines of platform code, number of meetings.
- EX02 Single number: lead time to production, with the limits (varies by team, does not show reliability); also show incidents and adoption.
- EX03 Examples: golden-path adoption predicts lead time; support questions predict satisfaction; test by comparing adopters and non-adopters over time.
- EX04 Claim about 7 percentage points beyond the trend, with confidence "plausible, not proven" and a note on other changes.
- EX05 Five tiles with baseline, target, trend, definition, last updated; data from CI, incident tool, billing, and surveys; leave out activity counts.
- EX06 State facts, probable reasons (migration burden, support gaps), costs, and the planned changes with owners; do not hide or soften.
Outcomes 2 and 3 are also covered by the lab; outcome 4 by lab mock report.
Workplace evidence: an impact report you produced for work you led.
Quiz
Outcomes: O1 success measures up front · O2 leading and lagging indicators, attribution · O3 impact dashboards · O4 honest reporting.
A self-check for you: nothing is graded, saved or sent anywhere. Choose an answer, then check it. You can try again as often as you like.
Practical part
Six case exercises STF-019-EX01 to EX06 plus the lab, assessed in a metrics review. Pass: at least five of six exercises correct, quiz at least 8/10, and a metrics set where every metric has a data source, a baseline from STF-009, and a target.
Common mistakes
Vanity metrics; claiming credit for trends that started earlier.
Assessment
Metrics review.
Workplace evidence
An impact report you produced for work you led.
Career relevance
Promotion evidence (STF-020); architecture assurance (ARC-042).