Performance modeling
On this page 9 topics
Overview
Topics
- Latency budgets
- Little's Law
- bottleneck analysis
- queuing
Learn first
4 h of practice (estimate)
Learning outcomes
- Allocate an end-to-end latency budget across components and calls.
- Apply Little's Law to relate throughput, latency, and concurrency (threads, connections, consumers).
- Explain queueing behavior near saturation and why latency explodes at high utilization.
- Identify the bottleneck resource from a model and confirm it experimentally.
Lessons
Lab and exercises
Lab
SRC-005-LAB1 · Checkout latency budget
Outcomes: 1–4 · Time: ~2.5 h
- Allocate a p99 budget for checkout across the services and calls on the critical path.
- Model required HTTP threads, DB connections, and Kafka consumers with Little's Law at the 10× load.
- Compute utilization per resource on paper and name the predicted bottleneck.
- Verify with a load test; compare model and measurement.
Acceptance: model within 20% of measurement or discrepancy explained.
Exercises (modeling problem set; six exercises)
| ID | Outcome | Task |
|---|---|---|
| SRC-005-EX01 | 1 | Allocate a 900 ms p99 budget over a path with gateway, order, two parallel calls (inventory, pricing), then payment, then DB write. |
| SRC-005-EX02 | 2 | 800 requests/s at 150 ms: how many in flight? A pool has 60 connections: what is the maximum throughput at 30 ms per query? |
| SRC-005-EX03 | 3 | Explain why a service at 90% utilization shows much higher p99 than at 60%, and propose two protections. |
| SRC-005-EX04 | 3 | Latency of a downstream call rises from 50 ms to 500 ms at constant arrival rate. What happens to threads in flight? |
| SRC-005-EX05 | 4 | From a table of resource demand and capacity, identify the bottleneck and design an experiment to confirm it. |
| SRC-005-EX06 | 1 | A page calls 20 services in parallel, each with p99 100 ms and p50 10 ms. Estimate how often the page sees a slow call and suggest a fix. |
Show solutions
- EX01 Parallel calls count as the maximum: gateway 20, order 100, parallel max(150, 80) = 150, payment 350, DB 100, margin 70 ms = 790 ms plus reserve; state timeouts per call.
- EX02 800 x 0.15 = 120 in flight. Pool throughput = 60 / 0.03 = 2,000 queries/s.
- EX03 Queueing delay grows steeply near saturation; protections: bounded queues with load shedding, autoscaling or capacity headroom, timeouts.
- EX04 In flight = rate x latency, so concurrency grows ten times; pools exhaust and requests queue unless timeouts and limits cap it.
- EX05 Highest utilization resource (for example DB pool at 92%) is the candidate; raise load stepwise, watch pool wait time and CPU, then change pool size and see whether the knee moves.
- EX06 1 - 0.99^20 is about 18% of pages see a slow call; fix with fewer calls, caching, timeouts with degraded fallback, hedged requests.
Workplace evidence: a performance target you set or met using a model rather than trial and error.
Quiz
Outcomes: O1 latency budgets · O2 Little's Law · O3 queueing near saturation · O4 finding the bottleneck.
A self-check for you: nothing is graded, saved or sent anywhere. Choose an answer, then check it. You can try again as often as you like.
Practical part
Six modeling problems SRC-005-EX01 to EX06 plus the lab. Pass: quiz at least 8/10, at least five problems correct, and a model within 20% of the load-test measurement or an explained discrepancy.
Common mistakes
Budgets based on averages; ignoring fan-out amplification of tail latency.
Assessment
Modeling problem set.
Workplace evidence
A performance target you set or met using a model rather than trial and error.
Career relevance
Performance engineering (SRC-016).