Environments & Deployments
Promote builds, not hope
Turn shipping into a boring routine: environment topology (dev/staging/prod), deployment strategies (rolling, blue/green, canary, feature flags), and the gates that promote a build from safe to live with evidence.
What you will learn
- Design a dev→staging→prod environment topology that isolates risk
- Choose a deployment strategy per risk profile
- Add evidence gates (tests, metrics) to promotion
Environments form a risk ladder: dev (fast, throwaway), staging (prod-like, integration/QA), prod (real). Deployments move the same immutable artifact up the ladder via strategies fit to risk — rolling for steady rollout, blue/green for instant switch + rollback, canary for risk-subset exposure, feature flags to decouple shipping from releasing. Gates are the evidence between rungs.
What is this?
Environment topology: dev (per engineer or shared, cheap, dirty data), staging (prod-shaped: same sizes, real dependencies, synthetic/replay traffic), prod with prod safeguards. Promotion: CI builds one artifact + signs it; every env deploys that same artifact so 'staging validated' means 'prod will run this exact image'. Strategies: rolling (replace pods a few at a time — fine for homogeneous, blast-radius small), blue/green (two full stacks, flip DNS/LB — instant rollback by flipping back; 2x cost), canary (send a small % of traffic to the new version, compare p99/error rate, grow or abort), feature flags (both versions behind a flag, kill-switch independent of deploy). Gates: lint, unit/integration, contract/consumer tests, image scan, staging smoke (DB migrations run first), performance in staging, then canary metrics in prod (error rate, p99, saturation). Rollback hygiene: migrations must be backward-compatible for one release (expand-migrate-contract), DB changes gate deploys. Database migrations run ahead of the code they need.
Why do you need it?
- Same-artifact promotion makes staging evidence actually mean something for prod.
- The right strategy matches blast radius to risk — canary a risky change, not a trivial one.
- Feature flags let you ship and release independently; an incident becomes a flag flip, not a redeploy.
- Backward-compatible migrations keep rollback a real option instead of a fantasy.
Mental model
Copying files to prod and praying
A cargo plane with a checklist and a parachute
Promotion with gates is the parachute; the checklist is the environment topology.
How it works
- CI builds + scans + signs one immutable artifact per commit.
- Staging deploys the artifact, runs migrations, smoke-tests, load-tests.
- Promotion to prod picks the deployment strategy for the risk.
- Canary/blue-green measures; gates compare; the run either finishes or rolls back.
Shipping: one artifact promoted with evidence.
In production
- A release train: Monday builds, Tuesday staging evidence, Wednesday canary, Thursday full — boring on purpose.
- DB expand-contract released one version ahead of the app that uses it.
- Death-star feature shipped dark behind a flag, released to 1% then 10% by toggling config.
Best practices
- One artifact, every environment.
- Exactly one opinionated deployment strategy per service (configurable).
- Rollback path rehearsed, not assumed.
- Migrations are backward-compatible for at least one release.
Common mistakes
- Staging with different sizes/datasets, so load tests mean nothing.
- Deploying schema and code together — rollback becomes impossible.
- Canary for one second then ramping to 100% on 'looks fine'.
Debugging tips
- 'Works staging, not prod' → compare env variables, secrets, data skew, not app code.
- Weird post-canary errors → check whether the old and new share state (queues, tables) and the flag kill-switch.
Interview checkpoint
Hands-on lab
- Map the environment ladder for a service
- Lay out the gates between envs
- Choose + justify the prod deployment strategy
Design a Promotion Pipeline with Gates
25 min · intermediate
Quick quiz
Rollback-safe migration pattern for DB?