Descriptive Statistics
One number each for where, how wide and how lopsided
Descriptive statistics summarize a dataset with a few numbers on location (mean, median), spread (variance, IQR) and shape (skew), turning raw columns into a mental picture.
What you will learn
- Choose the right measure of central tendency for a distribution
- Interpret standard deviation and IQR as spread
- Spot skew from mean-vs-median comparison
What is this?
Descriptive statistics compress a column of numbers into a few meaningful values. Location measures (mean, median, mode) say where the data sits. Spread measures (variance, standard deviation, IQR, range) say how far values scatter from there. Shape (skew, kurtosis) tells you whether the distribution leans or has heavy tails.
Why do you need it?
- You can't eyeball 100k rows; summary stats are the first check before any modeling.
- Mean vs median discrepancy is the fastest skew detector.
- Distribution choice, outlier flags and sanity checks all start from descriptive stats.
- They define what 'unusual' means for every later statistical test.
Mental model
Describing a crowd by 'average height' and 'how varied heights are'
Location and spread of a dataset
'The crowd averages 1.7m' says little on its own — add 'with a standard deviation of 8cm' or 'half between 1.55 and 1.85' and you've painted the picture.
How it works
- Mean sums all values and divides by count; median is the middle value when sorted; mode is the most frequent value.
- Variance is the average squared distance from the mean; the standard deviation is its square root, back in data units.
- The IQR = Q3 − Q1 (25th to 75th percentile) is a robust spread measure that ignores extreme outliers.
- Skew = 3(mean − median)/std approximately: positive skew → mean pulled right by a long upper tail.
- Fifty-one percent of real-world columns (revenue, latency, spend) are right-skewed — a fact that drives log transforms everywhere.
Describing one column.
Examples
Descriptive stats in pandas
import pandas as pds = pd.Series([100, 95, 70, 3, 99, 98, 97, 96])print(s.describe()) # count, mean, std, min, quartiles, maxprint('skew:', s.skew()) # large positive → right heavy tailprint('median:', s.median()).describe() dumps the default summary in one call. The single 3 pulls the mean below the median — the tell-tale signature of a right-skewed tail in a defensible, library-provided form.
In production
- An anomaly-detection team sets alert thresholds from rolling mean ± 3·std on latency each hour.
- A growth team compares median revenue per user across cohorts — medians resist the few whales that would wreck means.
- A data-quality job fails a pipeline when a column's mean or std drifts beyond defined bounds ('stats drift') before serving features.
Best practices
- Always pair location with spread: 'median 45, IQR 10-95' beats 'mean 60'.
- Prefer median + IQR for skewed or outlier-prone business metrics.
- Describe each column before modeling — it catches encoding and unit errors early.
Common mistakes
- Using the mean on skewed data and being shocked when the 'average' exceeds 90% of customers.
- Quoting variance (squared units) as if it were in the data's units.
- Deleting outliers before checking whether they represent real events (fraud, whales).
Debugging tips
- If mean ~= median and std is huge, look for clustered bimodal data, not a tail.
- Compare describe() against known domain bounds; a negative revenue is a parse or unit bug.
- Use df.describe(percentiles=[.05,.5,.95]) for a fuller picture of tails.
Interview checkpoint
Hands-on lab
- Compute location, spread and shape statistics in pandas
- Interpret mean-vs-median difference as skew
- Choose the right summary for a skewed metric
Summarize a Column Like a Pro
25 min · beginner
Quick quiz
For revenue data with a few billionaire customers, which location statistic should you report?
Standard deviation is the square root of the variance.