Anomaly Detection
Catch the day the metric breaks
Anomaly detection flags values that diverge from expected patterns — a crash in revenue, a spike in errors, a hung checkout. It replaces manual monitoring thresholds with statistically grounded alerting.
What you will learn
- Distinguish outliers, anomalies, and expected variation
- Detect daily anomalies against trend and seasonality baselines
- Design alerting that triggers on signal, not noise
What is this?
Anomaly detection identifies observations that depart from the expected pattern of a series. Because metrics carry trend and weekly seasonality, a value that looks fine against the overall series can be a severe anomaly relative to its day-of-week expectation. Good detectors model the baseline and flag deviations beyond a threshold.
Why do you need it?
- Manual thresholds miss slow creep and fire constantly on normal cycles.
- Early detection of metric crashes often times product incidents.
- Automated baselines scale alerting across hundreds of metrics.
Mental model
Noticing the one quiet stretch of road in a noisy city
An anomaly in a noisy metric
The signal you care about is not the level — it is the deviation from what that particular place and time usually sounds like.
How it works
- Estimate the expected value and spread for each point using history and calendaring.
- Compare actual to expected with the day-of-week (and holiday) context.
- Flag points beyond a z-score or percentile threshold of the residual.
- Deduplicate into incidents and route alerts by severity.
Expected band from history reveals the points that break the pattern.
In production
- Revenue monitoring pages alert when daily revenue drops below its day-of-week band.
- SRE watches error-rate anomalies separate from planned releases.
- Fraud teams score transaction deviations from a user's own history.
Best practices
- Detect against the seasonality-aware expectation, never the flat global mean.
- Threshold by the cost of false alarms vs missed incidents, and review alert rates.
- Keep a changelog of known events so the detector can be tuned, not hated.
Common mistakes
- Flagging every noisy point instead of grouping them into one incident.
- Using a threshold tuned on the past that drowns after a newer regime starts.
- Ignoring holidays — they look like anomalies every single year.
Debugging tips
- If alerts fire every Monday, the detector is missing the weekly pattern in its baseline.
- If a genuine crash produces no alert, lower the per-point threshold or shorten the baseline window.
Interview checkpoint
Hands-on lab
- Build a seasonality-aware expected baseline
- Compute residual z-scores and flag anomalies
- Group flagged days into incident-like alerts
Daily Revenue Anomaly Detection
40 min · advanced
Quick quiz
Anomaly detection should compare today's value against
Alerts that fire every Monday on a weekly-seasonal metric indicate