Statistics
Beginner 13 min read Statistics

Probability Basics

The language of uncertainty, made rigorous

Probability quantifies uncertainty: P(A), conditioning, independence, and Bayes' rule. It is the machinery behind sampling, p-values, forecasts and every 'likelihood' claim in analytics.

probabilityBayesconditionalindependencerandom variableexpected value
Prerequisites: Descriptive Statistics
Before we start

What you will learn

  • Compute basic, conditional and joint probabilities
  • Detect independence and dependence between events
  • Apply Bayes' rule to update beliefs with evidence
Introduction

What is this?

Probability assigns numbers in [0,1] to uncertain events. Core rules: P(A or B), P(A and B) with conditioning P(A|B) = P(A,B)/P(B), independence when P(A|B) = P(A), and Bayes' rule P(A|B) = P(B|A)P(A)/P(B). Analysts use it to say how likely a result is, to update prior beliefs, and to design sampling.

Motivation

Why do you need it?

  • 'There's a 15% chance of fraud' is a probability — and needs rules to be coherent.
  • Every p-value, confidence interval and forecast is probability in disguise.
  • Bayesian thinking formalizes how to update a belief when new evidence arrives.
  • Expected value = probability × payoff; product decisions run on it.
Think about it

Mental model

Imagine

Odds at a sportsbook

Think of

Probability

Bookies quote P(team wins); sharp bettors combine conditioning ('against a left-handed pitcher'), independence checks ('streak and matchup') and updating (Bayes) — the same structure as data analysis decisions.

Mechanics

How it works

  • P(A) is the long-run share of outcomes where A occurs among all outcomes.
  • P(A|B) = P(A and B)/P(B) counts the A-share within B-world only.
  • Independence means P(A|B) = P(A); dependence means B's occurrence changes A's odds.
  • Bayes: P(A|B) = P(B|A)·P(A)/P(B) flips conditional direction — posterior = likelihood × prior / evidence.
  • Expected value E[X] = Σ value·probability over all outcomes.

Bayes' rule as a belief update.

Prior P(A)
Likelihood P(B|A)
Evidence P(B)
Posterior P(A|B)
Hands-on

Examples

Bayes on a test

Bayes on a test
# Disease prevalence 1%, test sensitivity 99%, specificity 95%
# P(disease | positive) = ?
prior      = 0.01
sensitivity= 0.99          # P(positive | disease)
fp         = 0.05          # 1 - specificity
p_pos      = prior*0.99 + (1-prior)*0.05
posterior  = prior*0.99 / p_pos
print(round(posterior, 3)) # ~0.167 — most positives are false

Even a great test yields a low posterior on a rare condition — the classic base-rate lesson that Bayes formalizes.

Real world

In production

  • Fraud teams set thresholds from P(fraud | rule hits), not raw rule lift.
  • Product pricing computes expected value across plan price × adoption probability.
  • Experiment analysis updates a prior conversion belief with the observed sample.
Guidelines

Best practices

  • Always define the sample space before computing any probability.
  • State independence assumptions explicitly — the most common silent math error.
  • Convert odds ↔ probabilities carefully when reading sports/risk reports.
  • Use Bayes to keep base rates visible in rare-event analyses.
Watch out

Common mistakes

  • Treating dependent events as independent (sessions and purchases).
  • Ignoring base rates, generating a false-positive cascade in anomaly flagging.
  • Confusing P(fraud | rule) with P(rule | fraud).
  • Adding probabilities of non-exclusive events (double counting overlap).
When things break

Debugging tips

  • Posterior higher than prior by a factor > 1 is fine, but recheck you divided by the right P(B).
  • Probabilities outside [0,1] mean you mis-modeled an event space.
  • If 'independent' events clearly co-occur, your assumption, not the data, is wrong.
  • Expected value negative? Check sign and the payoff weights per outcome.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

Events A and B are independent when:

2

A test with 99% sensitivity will always be a reliable positive signal.

Go deeper

Official documentation

Khan Academy — probability
docs