Deployment Strategies: Canary and Shadow

• Distinguish shadow from canary traffic handling
• Design a canary with a promotion/rollback rule
• Use shadow mode to evaluate without affecting users

• Offline evals can't see production data — shadow and canary see it live.
• Shadow is zero-risk for users; canary is low-risk and measurable.
• Promotion becomes 'metrics held', not a vibe check.
• Both produce the evidence your approval gates want.

First she watches and preps the exact same plates without a ticket (shadow — no one eats them). Then she plates for real but only for one table in ten (canary). Promotion happens when the tables don't complain and plates match spec.

• Shadow: a mirror of production traffic is duplicated to the candidate; its outputs are logged, not returned.
• Canary: a fraction of real traffic (e.g. 5%) is routed to the candidate and its live outcomes are compared.
• Both pipeline predictions through the same monitoring and metric pipeline as production.
• A decision rule (metric holds over a window) either stretches the canary or rolls back.
• Full rollout: traffic shifts to the candidate; the old model becomes the rollback target.

• A team shadows a new churn version for 48h, compares lift, then canaries it at 10%.
• A risk platform canaries 1% of live traffic — metrics hold, then auto-promotes to 100%.
• A shadow run reveals a feature-skew bug between train and serving before any user sees it.

• Shadow first, canary second — risk-free evidence before any live exposure.
• Define the promotion rule before the canary starts: metric, window, threshold.
• Log outcomes, not just predictions — you compare effects, not scores.
• Keep rollback one step: the previous production version, fully warmed.

• Canarying on score distributions when the business metric is what matters.
• Shadow used forever, never promoted, while the champion rots.
• A 'canary' that's really 100% because someone forgot the split.
• Promotion on a 10-minute window — noise, not evidence.

• Canary 'wins' but prod hurts — the canary traffic slice isn't representative or outcomes weren't logged.
• Rollback slow — you didn't pre-warm the old version; make rollback a pinned, ready revision.
• Shadow shows skew — compare train vs shadow input distributions; that's the bug report.