Cache Metrics
Does the cache earn its keep?
Hit rate, miss latency, key churn and stale-hit rate tell you whether your cache is earning its keep. Metrics decide TTLs, thresholds and storage sizes — a cache measured badly is a cache tuned blindly.
What you will learn
- Track hit rate and breakdown by tier
- Detect churn and stale-hit drift
- Tune from metrics, not vibes
What is this?
Cache health is four numbers: hit rate (fraction of requests served from cache), miss latency (the cost of a rebuild), key churn (how fast keys are created/expire) and stale hits (answers that later proved wrong). Together they tell you if shrinking TTL, raising thresholds or resizing storage is the fix.
Why do you need it?
- A 90% hit rate hides a 10% tail that may be your worst customer.
- High churn means keys didn't repeat — the cache is burning memory for nothing.
- Stale-hit rate is the quality tax you pay for caching; you must know it.
Mental model
Cache metrics
A restaurant's dish-return rate
High table-turn with low returned-plates is success; high returns (stale) plus empty kitchen fires (misses) tells you where to fix the kitchen.
How it works
- Count lookups, hits, and stale detections per key namespace per tier.
- Latency histograms: hit latency vs miss latency vs model call time.
- Track key churn = set ops / get ops; near-1 means nothing repeated.
- Sample stale: periodically re-evaluate a cached answer vs a fresh call.
Dashboards the cache team reads.
In production
- A team notices hit rate slides after midnight → TTL was too tight for the batch jobs.
- Stale-hit alarms fire after a data refresh → purge events weren't wired.
- Churn spikes after a deploy → keys include a deploy-stamp; version keying needed.
Best practices
- Tag every request with cache tier and hit/miss for tracing.
- Chart hit rate per namespace, not globally.
- Set a stale budget (e.g. <0.5% wrong-hit) and alert on it.
- A/B TTLs: compare hit rate and stale rate before committing.
Common mistakes
- Global hit-rate only — a hot namespace masks broken ones.
- No stale tracking — nobody notices wrong answers until tickets.
- Chasing 100% hit rate by raising TTL and swallowing staleness.
Debugging tips
- Hit rate fine but costs flat → you're caching the cheap requests, not the expensive ones.
- Miss latency spike with good hit rate → the 10% miss is the problem; cache that namespace.
- Stale rising with TTL extension → revert, or add version-key invalidation.
Interview checkpoint
Quick quiz
Key churn near 1 suggests what?
A 100% hit rate is always the goal.