Detecting Concept Drift

• Detect drift on reconciled error over adaptive windows
• Use ADWIN-style tests to flag significant error change
• Fall back to proxies and outcome metrics when labels lag

• The relationship itself is unobservable directly — correctness is its readout.
• Adaptive windows track concept changes instead of averaging them flat.
• Statistical tests separate real error movement from noise.
• Proxies extend detection to the label-light tier of production.

Shots, not rims, are watched: when make% (error inverse) drops over an adaptive minutes window despite unchanged opponents, the coach adjusts — the relationship 'shot → made' changed mid-game. Averages over the whole night would hide it.

• Reconciled labels arrive; per-window error rate is computed.
• An adaptive-window test (ADWIN-like) finds the shortest interval where error changed significantly.
• A drift flag fires when the change is both significant and sustained.
• Without labels, tracked proxies (conversion, accept rate, feedback) play the role of outcome.
• The detector writes evidence — windows, deltas, p-worthy stats — for the response playbook.

• A fraud model's windowed AUC dips twice consecutively; ADWIN flags concept drift before recall collapses.
• A recommender uses CTR proxy as label-light drift readout when explicit labels lag days.
• A lending model's default-outcome trend declines steadily — the detector triggers a redevelopment cycle.

• Compute drift on error/outcome trends, not predictions alone.
• Pair adaptive windows with a minimum sustained window requirement.
• Use the best available proxy when labels lag, and reconcile to labels later.
• Emit actionable evidence: which error changed, over which window, by how much.

• Detecting on prediction distributions — that's data-drift territory, not concept.
• Whole-history averages that smooth away regime change.
• Reacting to a single noisy window without confirmation.
• Ignoring label lag: evaluating 'now' against labels from an older concept.

• False flag — check label quality/coverage first; a label-definition change mimics drift.
• Missed drift — window too wide or baseline variance swamping the change.
• Proxy artifact — a business metric shift from an ops change, not the model; triage it out.