Advanced 16 min read Architecture

Cloud-Native Patterns

Cloud-native patterns are the recurring solutions to the hard problems of distributed systems on cloud platforms: retries with backoff, circuit breakers, health checks, idempotency, configuration injection and observability. They are the difference between an app that runs in the cloud and one that is resilient in it.

12-factorcircuit breakerretry backoffhealth checksidempotencygraceful shutdownobservabilityservice mesh

Prerequisites: Serverless Architecture Event-Driven Architecture

Introduction

What is this?

Cloud-native patterns are battle-tested recipes for building applications that treat the infrastructure as a platform rather than a machine: stateless processes configured by environment, resilient to partial failure, auto-scaled, and observable by default. They cover the 12-Factor principles plus the distributed-system defenses (retries, circuit breakers, bulkheads) that keep a cloud app alive when dependencies degrade.

Motivation

Why do you need it?

Cloud infrastructure fails regularly and silently: instances die, networks drop, regions degrade — resilience must be designed in.
Stateless, horizontally scaled processes make the autoscaler's job trivial and turn capacity into a solved problem.
Configuration via environment keeps the same artifact safe in dev, staging and prod with zero code changes.
Health checks and readiness probes let orchestrators route traffic away from sick instances before users feel it.
Observability (metrics, logs, traces) is the only way to debug a fleet of instances you never SSH into.

Think about it

Mental model

Imagine

A single employee in an office

Think of

A cloud-native service on a platform

One employee works until they burn out and the office notices too late. A cloud-native service publishes a heartbeat, takes on only what it can carry, steps aside for replacements when unhealthy, and reports its own vital signs — so the platform can act automatically.

Mechanics

How it works

Stateless instances are started from an immutable artifact; all config is injected via environment variables at boot.
The service exposes /health (liveness) and /ready (readiness) endpoints the platform polls to route traffic and restart instances.
Dependencies are guarded: calls get timeouts, bounded retries with jittered exponential backoff, and circuit breakers that fail fast when a dependency is down.
Writes are made idempotent with client-supplied keys so retried requests cannot double-apply.
The service logs structured events, emits metrics and exports traces, and shuts down gracefully on SIGTERM, draining in-flight work.

A request's journey through the resilience stack around a cloud-native service.

Load balancerRouts only to ready instances via /ready

InstanceStateless, env-configured, immutable artifact

Health probes/health and /ready determine traffic + restart

Dependency callTimeout + retries + circuit breaker

WriteIdempotent via request id / idempotency key

ObservabilityLogs, metrics, traces shipped to the platform

Reference

Key commands

kubectl get pods -o wide

See which instances exist and their ready state.

kubectl rollout restart deployment/api

Roll new instances in a stateless fleet without downtime.

curl -s localhost:8080/ready

Probe the readiness endpoint to see if the instance accepts traffic.

{"status":"ready"}

curl -s localhost:8080/health

Probe liveness; the platform restarts instances that fail this.

{"status":"alive"}

kubectl get hpa

Check the horizontal autoscaler's current and target replicas.

NAME   REFERENCE          TARGETS   MINPODS   MAXPODS

kubectl logs deploy/api --tail=100

Stream the last 100 structured log lines from the deployment.

Hands-on

Examples

Retry with jittered exponential backoff and a circuit breaker

// resilience/retry.ts
export async function withRetry<T>(fn: () => Promise<T>, opts: {
  attempts?: number
  baseMs?: number
  maxMs?: number
} = {}): Promise<T> {
  const { attempts = 4, baseMs = 100, maxMs = 2000 } = opts
  let lastErr: unknown
  for (let attempt = 1; attempt <= attempts; attempt++) {
    try {
      return await fn()
    } catch (err) {
      if (attempt === attempts) throw err
      lastErr = err
      const backoff = Math.min(maxMs, baseMs * 2 ** (attempt - 1))
      const jitter = Math.random() * backoff * 0.5
      await sleep(backoff + jitter)   // jitter prevents thundering herd
    }
  }
  throw lastErr
}
 
export function circuitBreaker(fn: () => Promise<void>, { threshold = 5, resetMs = 10_000 } = {}) {
  let failures = 0
  let openUntil = 0
  return async () => {
    if (Date.now() < openUntil) throw new Error("circuit open")
    try {
      await fn()
      failures = 0
    } catch (err) {
      failures += 1
      if (failures >= threshold) openUntil = Date.now() + resetMs
      throw err
    }
  }
}

withRetry retries transient failures with exponential backoff plus random jitter so simultaneous retries do not stampede the recovering dependency. circuitBreaker trips open after five consecutive failures and refuses calls for ten seconds, giving the dependency room to recover while the caller fails fast.

Real world

In production

A payments API uses idempotency keys so a retried debit cannot charge twice.
Order ingestion breaks failures by tenant with bulkheads so one abusive tenant cannot take down everyone.
A video platform scales from 1 to 200 replicas on a view spike, then back to 1 at 3 a.m., paying only for what runs.
Health and readiness probes let a mesh canary 5% of traffic to a new version and roll back automatically on failure.

Guidelines

Best practices

Separate /health (liveness, always cheap) from /ready (readiness, checks dependencies) — they drive different orchestrator actions.
Use client-generated idempotency keys for every mutating API call and store them with a TTL.
Cap retries and always use exponential backoff with jitter; never retry on 4xx client errors, only on 5xx and network errors.
Put all config in environment variables and fail fast at boot if a required variable is missing.
Design for graceful shutdown: handle SIGTERM, stop accepting traffic, finish in-flight requests, then exit.

Watch out

Common mistakes

Retrying on non-idempotent endpoints without a key, creating duplicate side effects.
Health checks that call the database — liveness must never depend on dependencies, or the whole fleet restarts in a DB outage.
Fixed backoff (or no jitter) that thunders a recovering service with synchronized retries.
Stateless processes holding state in memory, so scaling and restarts silently lose data.

Root cause analysis

Common issues

Error

502 Bad Gateway from the load balancer after a deploy

Root cause

Instances are starting but failing readiness, or the artifact crashed on boot.

Fix

Watch kubectl get pods for CrashLoopBackOff or readiness 0/1, then read startup logs for config or dependency failures.

Error

All retries fail at the same instant after every deploy

Root cause

Synchronized clients retrying together without jitter, stampeding the dependency.

Fix

Add random jitter to backoff and stagger retry starts across clients.

Error

Duplicate payment records after a client retry

Root cause

The endpoint is not idempotent and the retried request applied twice.

Fix

Require an idempotency key header, dedupe on the key with a unique constraint, and return the stored result on replay.

When things break

Debugging tips

Start with the trace: a p95 spike is a dependency slowdown, while an error-rate spike with healthy deps is usually code.
Check circuit state — an open circuit turns all calls into fast errors that look like outages.
Confirm the readiness endpoint flips false before a dependency actually fails; if not, add dependency checks to readiness.
Look for thundering-herd patterns: synchronized spikes in retry logs usually mean missing jitter.

Interviews

Interview checkpoint

Practice

Hands-on lab

Build a Resilient Gateway with Retries and a Circuit Breaker

45 min · advanced

In progress

Objectives

Build a simulated flaky dependency
Implement retry with jittered exponential backoff
Add a circuit breaker that fails fast
Observe how each layer changes failure behavior

Build a Resilient Gateway with Retries and a Circuit Breaker

45 min · advanced

0/5 steps

sandbox · Linux sandbox

Try these commands

Test yourself

Quick quiz

0/3

What does a readiness probe tell the platform?

A liveness probe should check that the service can reach its database.

Your client retried a failed order request and two orders were created. What pattern was missing?

Quick reference

Cheat sheet

The resilience stack

TimeoutEvery external call gets one

Retry + jitterExponential backoff, randomized

Circuit breakerFail fast when dependency is down

BulkheadIsolate failures per tenant/type

Idempotency keySafe retries for writes

Platform contracts

/healthLiveness; restart on failure

/readyTraffic routing; checks deps

SIGTERM handlingDrain in-flight work, then exit

Env configSame artifact everywhere

Go deeper

Official documentation

The Twelve-Factor App

spec

Azure Cloud Design Patterns

docs

Patterns of Distributed Systems

docs

AWS Well-Architected Framework

docs

Cloud-Native Patterns

What is this?

Why do you need it?

Mental model

How it works

Key commands

Examples

Retry with jittered exponential backoff and a circuit breaker

In production

Best practices

Common mistakes

Common issues

Debugging tips

Interview checkpoint

Hands-on lab

Build a Resilient Gateway with Retries and a Circuit Breaker

Quick quiz

Cheat sheet

Related topics

Official documentation