Performance & Capacity
Redesign under load, not in the demo
Tune the system end-to-end: latency budgets per tier, caching and query/index strategy, queue-depth and connection-pool sizing, and capacity planning with headroom that survives a forecast, not just an average.
What you will learn
- Build a tier-by-tier latency budget
- Apply the big levers (cache, queries, batching) and size pools
- Do capacity planning with headroom
Performance is an end-to-end budget: break the p99 into tier budgets (LB→app→cache→DB→egress) and only optimize where the budget breaks. Levers: cache hot reads, fix the expensive queries (index/count-less pagination), batch small calls, size connection pools and queues with arithmetic, not guesses. Capacity planning projects peak, not average, and sizes with headroom because forecasts are lies with timestamps.
What is this?
Latency budgeting: define the journey p99, then carve tier budgets (e.g. 1.5s total: edge 100ms, app 500ms, cache 40ms, DB 300ms, egress/3rd-party 400ms, slack 160ms). Optimize the tier over budget; ignore the green ones. Request-path tuning: connect + optimize on the hot path (N+1, JSON bloat, uncompressed egress), cache at the right layer (page/edge, service, query), asynchronize the slow-but-not-sync-needed work. Pool/queue sizing: connection-pool threads bounded to DB capacity (pool ≤ DB max_con − admin/other pools), queue depth sized to burst + consumer concurrency with backpressure, not unbounded. Caching correctness: keys include version/schema, invalidation by event (not TTL laziness), and consistency modes chosen per read. Capacity planning: instrument peak shapes (traffic, data growth), model with headroom (say 40–60% above forecast), autoscale per metric, and load-test quarterly with the latest peak replay.
Why do you need it?
- Tier budgets stop 'optimize random panel' — you only touch the tier that's over budget.
- The big levers pay for themselves before any box gets bigger.
- Pool/queue arithmetic prevents the classic outage: pool threads pile up faster than the DB can drain.
- Headroom is cheap insurance against the predictable lie that is a growth forecast.
Mental model
Buying a bigger plane because boarding is slow
Finding the bottleneck is at the gate scanner, not the plane
Tier budgets show you which 'gate' is slow; the box size is a last lever.
How it works
- Measure the real journey p99; carve tier budgets from it.
- Fix the tier over budget with the biggest lever (cache/query/async/pool).
- Size pools + queues against measured concurrency, not defaults.
- Project peak with headroom; autoscale and load-test quarterly.
Performance: budget, lever, capacity.
In production
- A checkout p99 of 1.2s after adding an edge cache for static payloads and an indexed read model — no bigger DB.
- Connection pool set to 40 app threads when the DB comfortably drains 60 — queueing gone.
- Quarterly capacity review replays Black-Friday shape with +50% headroom, surfaces the DB write tier.
Best practices
- Measure before optimizing; the budget tells you where.
- Cache near consumers, invalidate by event, version the keys.
- Pool/queue sizes are arithmetic over measured concurrency.
- Capacity = forecast peak + explicit headroom, revisited quarterly.
Common mistakes
- Optimizing a tier that's already under budget.
- Pool sized to 'be safe' in both directions — too many threads queue, too few starve.
- Capacity plan sized to average traffic, surprised by the spike.
Debugging tips
- p99 spiking but p50 flat → there's a tail cause (GC pause, a slow query, pool-reuse war): trace the p99 requests, don't touch the p50 path.
- Cache hit-rate high but slow → verify invalidation events aren't being missed and keys aren't per-request unique.
Interview checkpoint
Quick quiz
Too many app threads against one DB → symptom + fix?