Statistics
Intermediate 14 min read Statistics

Normal Distribution

The bell curve that powers statistics

The bell curve is the default error model of ML. Its 68-95-99.7 rule, z-scores and tail probabilities power outlier detection, confidence intervals, and almost every parametric test.

normal distributionz-scoreempirical rulecentral limittailsoutliers
Prerequisites: Common Probability Distributions
Before we start

What you will learn

  • Interpret μ and σ of a normal
  • Apply the 68-95-99.7 rule
  • Convert values to z-scores and back
Introduction

What is this?

The Normal (Gaussian) distribution N(μ, σ) is the symmetric bell-shaped curve fully described by its mean and standard deviation. About 68% of values fall within ±1σ, 95% within ±2σ and 99.7% within ±3σ. A z-score (x − μ)/σ expresses a value in standard deviations from the mean, letting you compare across scales and translate to tail probabilities.

Motivation

Why do you need it?

  • Residuals of most models are treated as approximately normal, so many metrics and tests depend on it.
  • ±3σ is the classic outlier threshold used in anomaly detection and monitoring.
  • z-scores are how you compare a 200ms latency to a 2s latency meaningfully.
  • The CLT converts means of large samples to normality even when raw data isn't.
Think about it

Mental model

Imagine

A bell-curve shooter's target

Think of

Normal distribution of measurement error

Most shots land near the bullseye (μ), fewer scatter wider, and the ring at ±3σ catches essentially everything. Deviations beyond 3σ mean something other than luck.

Mechanics

How it works

  • N(μ, σ) has density shaped like a symmetric bell centered at μ; σ controls width.
  • Empirical rule: P(μ−σ ≤ x ≤ μ+σ) ≈ 68%, ±2σ ≈ 95%, ±3σ ≈ 99.7%.
  • z = (x − μ)/σ; P(Z < z) comes from the standard normal CDF (tables or scipy.stats.norm.cdf).
  • Quantile functions invert the CDF: scipy.stats.norm.ppf(0.975) ≈ 1.96, the z critical value for a 95% interval.
  • A value more extreme than roughly 3σ is unusual in a genuine normal — the basis of one classic outlier rule.

From raw value to z-score to probability.

Raw x
z = (x−μ)/σ
CDF / tails
Decision
Hands-on

Examples

z-scores, tails and quantiles in scipy

z-scores, tails and quantiles in scipy
from scipy import stats
mu, sigma = 200.0, 5.0
x = 210.0
z = (x - mu) / sigma
print(z)                                # 2.0
print(stats.norm.cdf(z))                # ~0.9772 → 97.7th percentile
print(stats.norm.ppf(0.975))            # 1.96  → z for 95% two-sided
print(1 - stats.norm.cdf(z))            # right-tail ≈ 2.3%

210ms sits 2σ above the mean → 97.7th percentile. The two-sided tail beyond ±2σ is ~4.6%, which is why '>2σ' is already rare. This exact arithmetic backs confidence intervals and z-tests.

Real world

In production

  • A monitoring tool alerts when latency exceeds rolling μ + 3σ, catching true regressions while tolerating realistic noise.
  • A fraud team z-scores each transaction feature per user so a 'high spend' means the same thing for a student and a millionaire.
  • An experiment platform computes a z-test (or t-test) on normalized differences to decide a launch.
Guidelines

Best practices

  • Verify approximate normality before leaning on ±Nσ rules — plot a histogram or Q-Q plot first.
  • For skewed metrics, transform (log) then use normal-population rules on the transformed values.
  • Use scipy.stats.norm.ppf rather than hardcoding critical values when setting thresholds.
Watch out

Common mistakes

  • Applying ±3σ outlier rules to right-skewed data where legitimately large values exceed it routinely.
  • Using σ of the sample as the scale for z-scores without adjusting for n. (use standard error for means).
  • Assuming 1.96 only for means — it's a z value, but t-tests need small-sample corrections.
When things break

Debugging tips

  • If nearly everything flags as an 'outlier', check you standardized a skewed column (log first).
  • A flat-topped histogram means the data may be uniform, not normal — don't use σ rules.
  • Check tails: heavy-tailed data (kurtosis ≫ 3) breaks ±3σ rules; use IQR-based limits instead.
Interviews

Interview checkpoint

Practice

Hands-on lab

z-scores and the 3-sigma Rule
30 min · intermediate
In progress
Objectives
  • Compute empirical rule percentages on real samples
  • Transform data to z-scores
  • See how skew breaks 3-sigma outlier detection

z-scores and the 3-sigma Rule

30 min · intermediate

0/5 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

For N(100, 10), what range holds ~95% of the data?

2

The z-score of a value exactly at the mean is always 0.

Go deeper

Official documentation

scipy.stats.norm
docs
Normal distribution — Wikipedia
docs