High Availability
Beginner 13 min read High Availability

High Availability Fundamentals

Design so any single part can die quietly

High availability is design, not luck: 9s (99.9%, 99.99%), redundancy across AZs, statelessness, health checks, and graceful degradation. The goal is to survive ANY single failure without user-visible impact.

availabilitynine ninesredundancyfailoverstatelesshealth check
Prerequisites: Availability Zones
Before we start

What you will learn

  • Read availability percentages (nines) as downtime budgets
  • Apply the redundancy + statelessness playbook
  • Understand what HA does (and does NOT) solve vs DR
In 60 seconds

HA means redundant components plus quick detection (health checks) and quick switchover (failover), targeting availability like 99.99% (~52min/yr). Stateless compute + AZ-redundant data + health-checked traffic = the recipe. Counterpoint: HA keep services up during failures; DR recovers the whole region after unrecoverable loss.

Introduction

What is this?

Definitions: availability=nines (99.9% ≈ 8.7h/yr, 99.99% ≈ 52min/yr, 99.999% ≈ 5min/yr), MTBF/MTTR = up-time equation. Redundancy: N+1 hot-standby forms, active-active vs active-passive. Stateless tiers (anything can be recreated quickly), stateful tiers (DB/queues) are the HA hardest part. Mechanisms: health checks (TCP/HTTP), auto-failover (LB/health-lists, replication failover), quorum (avoid split-brain in active/active). Examples per provider: multi-AZ DB (RDS/Azure SQL/GCP managed), target groups + ALB, instance groups regional MIGs/Auto Scaling across AZs, CloudFront/Global load balancer. HA tools don't cost only money — complexity, ops, blast-radius. Never forget Failover tests (the golden rule).

Motivation

Why do you need it?

  • You can't promise customers 99.99% with a single server and hope.
  • The cloud gives you AZs so you can build the same service twice — cheaply vs a whole second DC.
  • HA prevents outage windows; each avoided 8h outage matters when one costs your company a contract.
  • It changes the mindset from 'protect every part' to 'any part can die at any time'.
Think about it

Mental model

Imagine

A single-engine plane vs. a twin-engine one flying over water

Think of

HA = redundant engines

The twin fails gracefully on one dead engine (redundant unit, health check, pilot switchover); the single fails with no flyable path. Same for AZs: two live engines beats one orphaned spare that nobody knows how to engage.

Mechanics

How it works

  • Deploy at least 2 replicas of every role into ≥2 AZs.
  • Health checks mark instances dead when they don't respond.
  • Traffic routers (ALB/LB) stop sending to dead instances; autoscalers replace them.
  • Data layer replicates synchronously/asynchronously across AZs with failover.
  • Clients/backends reconnect; sessions state lives outside the dying instance (Redis/cookies).

AZ-redundant service.

Clients
LB (across 3 AZs)
Compute fleet
DB
Queues
Hands-on

Examples

99.9% vs 99.99% budget

99.9% vs 99.99% budget
# monthly for 30 days (44_640 min)
# 99.9%  → 44_595 min allowed, 44.6 min downtime
# 99.99% → 44_635 min allowed, 4.5 min downtime
# 99.999%→ 44_639 min allowed, 27 sec downtime

Each extra 9 costs a factor ~10 in ops rigor and architecture — spend only where the SLA demands.

Real world

In production

  • ALB + target-group health checks × ASG across 3 AZs for the web tier.
  • Managed multi-AZ DB (RDS Multi-AZ / Azure SQL business tiers / Cloud SQL high availability).
  • S3 + Cloud Front redundant static; any instance can die without a phone call.
Guidelines

Best practices

  • Health check what matters (readiness = has sidecar warmed, not just TCP).
  • Test failover on a schedule — the drill uncovers what sim-curves missed.
  • Keep state out of instances: sessions in Redis/cookies, uploads in object storage.
  • Quorum-aware actives: never 2 masters writing the same backend.
  • Design blast radius: a failed deploy shouldn't cross every AZ at once (lane deploys).
Watch out

Common mistakes

  • HA only for compute; the DB is a single RDS and everyone prays.
  • Health check passes but service unusable (TCP-only checks with 500 responses).
  • Failover never rehearsed: 'we restored from backup–ish' is a DR phrase, not HA.
  • State stored on instance: the restart kills sessions mid-flight.
When things break

Debugging tips

  • Instance failing unhealthy periodically — check regression edges of the health endpoint itself.
  • Failover takes long when it happens — trace the health/failover timings and RBAC gaps.
  • One AZ black-holes requests subtly — vary your client test traffic zš and watch LB zš split.
  • Auto-Failover split-brain scare — confirm baseline web tier is stateless and DB votes quorum.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

99.99% availability allows roughly how much downtime a year?

2

The HA trick that matters most for compute tiers is:

Go deeper

Official documentation

AWS high availability docs
docs
AWS multi-AZ
docs
Azure availability zones
docs