Statistics
Intermediate 12 min read Statistics

Central Limit Theorem

Nature's favor to statisticians

The CLT says means of large samples are approximately normal no matter the population's shape — the single theorem that makes most of statistics and many ML assumptions work.

central limit theoremsample meannormalityasymptoticsstandard error
Prerequisites: Sampling Distributions
Before we start

What you will learn

  • State what the CLT claims and doesn't claim
  • Demonstrate it on skewed distributions
  • Know when approximations are good enough
Introduction

What is this?

The Central Limit Theorem states that the distribution of the sample mean of n independent, identically distributed draws, scaled by √n, converges to a normal distribution as n grows — whatever the population looks like. Fat tail, uniform, skewed: enough averaging, and means look Gaussian. That is why z-tests, t-tests and confidence intervals work so broadly.

Motivation

Why do you need it?

  • It justifies 'normal-based' inference on non-normal underlying data.
  • It explains why ensemble averages and model 'mean predictions' behave more stably than raw values.
  • It's the mathematical license to use z-scores on means.
  • Knowing its limits prevents trusting it on tiny samples or heavy-tailed data.
Think about it

Mental model

Imagine

Averages of dice rolls across many throws

Think of

Sample means under the CLT

One 6 can be a 1; roll ten dice and the average rarely deviates far from 3.5. Average many independent draws and the extreme cases wash out toward the bell.

Mechanics

How it works

  • Let X̄ = (1/n)ΣXᵢ with population mean μ and variance σ².
  • Then √n (X̄ − μ) → Normal(0, σ²) as n → ∞.
  • Concretely, X̄ ≈ Normal(μ, σ/√n) for large n.
  • Independence is required; correlation slows or breaks convergence.
  • n ≈ 30 is a rule of thumb for mild skew; heavy tails need far more.

Means of skewed draws become bell-shaped.

Skewed population
Draw n samples
Averaging
Distribution of means
Z/t inference
Hands-on

Examples

Watch skew become a bell

Watch skew become a bell
import numpy as np
rng = np.random.default_rng(3)
for n in (1, 30, 100):
    means = [rng.exponential(scale=2, size=n).mean() for _ in range(5000)]
    print(n, round(means.mean(), 2), round(means.std(), 2))

n=1 reproduces the skewed exponential distribution (many small, few large means). By n=100 the means cluster tightly around the true mean 2 with a near-normal symmetric spread — standard error σ/√n ≈ 2/10 = 0.2.

Real world

In production

  • A/B platforms compute z-tests on session means knowing CLT validates them at ≥ hundreds of users.
  • A monitoring team averages latency in 1-minute windows; the windowed average is normal enough for ±3σ alerts even though raw latency is skewed.
  • An actuarial team sizes risk reserves from mean claim windows, relying on CLT convergence.
Guidelines

Best practices

  • Check n vs skew/tails before assuming normality of means.
  • Remember the CLT is about means/averages, not individual observations.
  • Independence matters — autocorrelated streams (time series) converge differently.
Watch out

Common mistakes

  • Applying the CLT to a single individual's value — it's about averages.
  • Expecting normality of sample means at n=5 on extremely skewed data.
  • Ignoring dependence; correlated draws inflate variance and break the σ/√n formula.
When things break

Debugging tips

  • Histogram the means over many resamples — that's the empirical check for CLT applicability.
  • If your 'mean' metric is a ratio of two randoms (e.g. revenue per session), CLT still helps but convergence can be slower.
  • Time-series data rarely meets the independence clause; use block-style or robust methods.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

The CLT says that for large n, the sample mean is approximately:

2

The CLT requires the underlying population to be normal.

Go deeper

Official documentation

Central Limit Theorem — Wikipedia
docs