Networking
Advanced 15 min read Networking

Load Balancers & Proxies

Load balancers distribute traffic across backend servers for scale and availability. Understand L4 versus L7, balancing algorithms, health checks, sticky sessions, SSL termination and how nginx and HAProxy implement them.

load balancerL4L7reverse proxyhealth checkround robinsticky sessionsnginx
Prerequisites: TCP/IP & the OSI Model HTTP & HTTPS
Introduction

What is this?

A load balancer sits in front of a pool of backend servers and distributes requests so no single server is overwhelmed. L4 balancers forward at the transport layer using IP and port; L7 balancers understand the HTTP request (method, path, headers) and can route, rewrite and terminate TLS. A reverse proxy is an L7 balancer that also fronts and hides backend servers.

Motivation

Why do you need it?

  • Distributing load horizontally is how you scale past one server and survive server failures.
  • Health checks let a balancer remove sick backends automatically — no manual intervention.
  • L7 routing enables path-based and canary deployments with zero client changes.
  • SSL termination centralizes certificate management at the edge.
Think about it

Mental model

Imagine

The front desk at a busy office

Think of

A load balancer

Guests arrive and the desk sends each one to an available person. If someone is on leave, the desk stops sending people to them. If the desk itself knows details about the request, it can send billing questions to the billing team — that's L7.

Mechanics

How it works

  • The balancer owns the public IP or DNS name and maintains a list of backend servers (the upstream pool).
  • For each new connection (L4) or request (L7), it picks a backend using an algorithm such as round robin or least connections.
  • Active health checks probe backends on a timer (TCP connect or HTTP GET /healthz) and mark them up or down.
  • L7 balancers terminate TLS, read the HTTP request, and may rewrite paths or headers before forwarding.
  • Clients keep talking to the balancer, so a backend can be added, removed or scaled without changing client configuration.

L7 load balancing with health checks and session stickiness.

Client request
L7 decision
Health check loop
Healthy backend
Unhealthy backend
Response
Reference

Key commands

nginx -t

Validate the nginx configuration before reloading.

systemctl reload nginx

Apply a new config without dropping in-flight requests.

curl -sI http://localhost -H 'Host: example.com'

Test an L7 rule by sending a custom Host header.

haproxy -c -f /etc/haproxy/haproxy.cfg

Validate an HAProxy configuration file.

ss -tulpn | grep ':80 '

Confirm the balancer is listening on port 80.

Hands-on

Examples

nginx reverse proxy with a least-connections pool

nginx reverse proxy with a least-connections pool
http {
  upstream backend {
    least_conn;
    server 10.0.1.10:8080 weight=2;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080 down;
  }
 
  server {
    listen 80;
    location / {
      proxy_pass http://backend;
      proxy_set_header Host $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
    }
  }
}

nginx routes every request to the upstream pool using least connections. Host and X-Forwarded-* headers let backends see the original client and protocol. The down server is excluded; weight=2 sends twice the traffic to 10.0.1.10.

Real world

In production

  • Canary deployments: route 5% of requests to the new version via an L7 path or weight.
  • SSL termination at the edge means backends serve plain HTTP on private networks.
  • Blue-green cutovers flip the upstream pool from blue to green with one config reload.
  • WebSocket and gRPC traffic needs L4 or protocol-aware L7 balancing, not plain HTTP routing.
Guidelines

Best practices

  • Always define health checks; a balancer without them will keep sending to dead servers.
  • Terminate TLS at the edge, but keep the certificate renewal automated (Let's Encrypt, ACME).
  • Prefer least-connections or weighted algorithms over plain round robin when backend capacity differs.
  • Reserve sticky sessions for stateful apps; stateless services should distribute freely.
  • Track active connections and request latency on the balancer as its own SLO.
Watch out

Common mistakes

  • Misconfiguring health checks to check the balancer instead of each backend.
  • Using sticky sessions everywhere, which destroys the balancing benefit and hides scaling problems.
  • Forgetting X-Forwarded-For, so the app logs the balancer IP instead of real clients.
  • Applying config changes without nginx -t / haproxy -c validation, risking a full outage on reload.
Root cause analysis

Common issues

Error
502 Bad Gateway from the balancer
Root cause

No healthy upstreams or a misconfigured proxy_pass target.

Fix

Check backend health via the status page, confirm backends listen (ss -tulpn), then reload the validated config.

Error
Requests always hit the same backend
Root cause

Sticky sessions (cookie or IP hash) plus a few clients.

Fix

Confirm stickiness is intentional; for stateless apps remove the session cookie and use least_conn.

Error
Timeouts on uploads or WebSockets
Root cause

proxy_read_timeout / proxy_send_timeout too low for long-lived requests.

Fix

Raise the timeouts and tune proxy_buffering for streaming endpoints.

When things break

Debugging tips

  • curl the balancer with -H 'Host: ...' to test virtual-host routing directly.
  • Read the balancer's access logs — the upstream chosen and the status tell you which backend failed.
  • Check upstream health from the balancer's admin/stats page (HAProxy stats, nginx status module).
  • Test each backend directly (curl backend-ip:port) to rule the pool out before the balancer.
Interviews

Interview checkpoint

Practice

Hands-on lab

Build an nginx Reverse Proxy Load Balancer
35 min · advanced
In progress
Objectives
  • Run two distinct backend servers
  • Configure an nginx upstream pool with health checks
  • Validate and reload the configuration
  • Observe round-robin distribution

Build an nginx Reverse Proxy Load Balancer

35 min · advanced

0/5 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/3
1

An L4 load balancer makes forwarding decisions based on...

2

Which algorithm sends each new request to the server with the fewest active connections?

3

A reverse proxy can hide backend details and terminate TLS.

Quick reference

Cheat sheet

Balancing algorithms
round_robinCycles through backends in order
least_connFewest active connections wins
weightSend proportionally more traffic (e.g. weight=2)
ip_hashSame client IP → same backend
randomUniform random pick, often with weights
Reverse proxy headers
X-Forwarded-ForOriginal client IP chain
X-Forwarded-Protohttp or https seen by the client
X-Forwarded-HostOriginal Host header
Go deeper

Official documentation

nginx upstream module
docs
HAProxy documentation
docs
HTTP Semantics — RFC 9110
rfc