Concept vs Data Drift
Which one moved — the data or the meaning?
Data drift moves the inputs; concept drift moves the meaning. Telling them apart matters because the fix differs — re-encode or re-baseline vs retrain on new labels — and many real incidents involve both at once.
What you will learn
- Distinguish the two drift families by their failure signatures
- Map each to its right remediation
- Handle incidents where both drift together
What is this?
Data drift and concept drift are different failure families: data drift changes P(x), the inputs; concept drift changes P(y|x), the meaning of inputs. Remediation diverges — data drift usually wants re-baselining or window-shifting, concept drift wants retraining on newly-labeled reality — and incidents commonly mix both.
Why do you need it?
- The fix differs: re-baselining won't fix a moved relationship, and retraining won't cure a poisoned reference.
- Misdiagnosis wastes a retrain on the wrong problem and delays the real one.
- Real incidents blend both; clean separation is the diagnostic habit.
- Evidence (which signals moved) tells you which family you're in.
Mental model
A restaurant's fridge vs its recipes
Data drift vs concept drift
The fridge being stocked differently (data drift) means the menu ingredients changed. The recipes being wrong for the same ingredients (concept drift) means the cooking no longer matches taste. Fixing the fridge won't fix the recipes and vice versa.
How it works
- Check input distributions: moved? That's data-drift evidence.
- Check outcome error trends with labels/proxies: moved while inputs stayed? Concept-drift evidence.
- Classify the incident: data-only, concept-only, or both.
- Choose remediation per family: for data-only, re-baseline/feature handling; for concept-only, retrain on new labels.
- Document and verify the fix against post-change metrics.
Diagnosing drift family by signal.
In production
- A fraud model: region mix shifted (data) while fraud semantics drifted after a policy change (concept) — both handled.
- A forecast model: seasonality drift (data) plus changed demand elasticity (concept) needed different remedies.
- A credit model rips between 'new applicant mix' and 'default meaning changed for the same applicant mix'.
Best practices
- Keep both input-distribution and outcome-reconciliation detectors live.
- Document a diagnostic step into the drift playbook that classifies family first.
- Fix each family separately and verify the remedy moved the right metric.
- Record co-occurring cases — they're the rule in production, not the exception.
Common mistakes
- Auto-retraining on any drift signal, regardless of family.
- Re-baselining inputs when the real problem is the relationship — cosmetic fix.
- Retraining into data-drift-only problems with stale labels, teaching wrong lessons.
- Diagnosing by vibe ('must be data') instead of by measured signals.
Debugging tips
- Retrain didn't fix errors — check whether the training labels captured the NEW concept, not the old.
- Inputs moved but everyone says 'the world changed' — measure it before assuming concept drift.
- Both signals moving — treat as two fixes, don't let one mask the other's evidence.
Interview checkpoint
Hands-on lab
- Classify signals into data vs concept drift
- Prove the two families with a toy model
- Choose the remediation per family
Diagnose the Drift Family
35 min · intermediate
Quick quiz
Outcomes degrade while inputs look identical — that's evidence of…
Real incidents often involve both data and concept drift at once.