Lifecycle Stages in Production

• Map a model through development, staging, production and retirement
• Define what evidence is required to promote a model between stages
• Design retirement so old models are removed safely

• Stages force evidence: a model can't wander into production without an audit trail.
• Staging catches the gap between offline metrics and real-world behavior before full blast radius.
• Defined retirement prevents zombie models from silently serving stale predictions.
• Governance and audits (SOC2, internal AI risk) need an answer to 'what is serving, and why?'

Keel laid (development), trials in harbor (staging under shadow traffic), full sea trials (canary), commissioned for service (production), decommissioned safely (retirement). Nobody sails without trial data, and nobody abandons a decommissioned ship in the harbor.

• Development: experiments tracked, candidates compared, one model wins on held-out evidence.
• Registration: the winner is stored in the model registry with metrics, data version and code ref.
• Staging: the model serves shadow or canary traffic while side-by-side behavior is compared to the incumbent.
• Production: after staging evidence passes thresholds, traffic scales to full.
• Monitoring follows the model: drift, performance and business metrics are attributed to this version.
• Retirement: when the successor proves out, the old version is drained, archived and blocked from serving.

• A payments model stays in staging for two weeks while offline recall (0.91) is checked against live agreement with the incumbent (0.88) before promotion.
• An ad-platform stages every candidate on 5% of traffic; only candidates with no revenue regression are promoted.
• After a successor ships, a lending model is drained over a week, archived with its full lineage, then unregistered from serving.

• Write the promotion gate as a checklist: offline metrics, data version, code hash, shadow agreement, owner sign-off.
• Freeze promotions when monitoring is not yet reporting live metrics for the current version.
• Auto-archive on successor promotion; require an explicit exception to re-serve an archived model.
• Keep one current and one (former) champion per decision so rollback is a registry pointer change.

• Skipping staging and going straight from notebook to production because 'it works locally'.
• Two models serving production simultaneously without an owner remembering why.
• Retiring a model by deleting it — losing the audit trail instead of draining and archiving it.
• Promoting when monitoring isn't wired, so the new version is governance-blind.

• When a promotion seems stuck, check the gate evidence: a missing metric or stale data version blocks everything.
• Diff staging agreement versus offline metrics to find where assumptions diverge.
• List what is serving right now; production surprise is usually a model that was never officially promoted.