Probability Basics
The grammar of uncertainty
Probability assigns numbers to uncertainty: the rules of independence, conditional probability and Bayes' theorem underpin every classifier, model calibration and A/B decision.
What you will learn
- Apply the sum and product rules
- Use conditional probability and Bayes' theorem
- Recognize independence vs correlation traps
What is this?
Probability is the formal language for uncertainty. A random variable maps outcomes to numbers; its distribution describes how likely each value is. The sum rule marginalizes over hidden variables, the product rule factors joint probabilities, and Bayes' theorem P(A|B) = P(B|A)P(A)/P(B) reverses conditioning — the mental engine of spam filters, medical tests and Bayesian models.
Why do you need it?
- Every ML classifier outputs a probability, and interpreting it requires probability literacy.
- Bayes' theorem explains the base-rate fallacy that sinks many real products.
- Independence assumptions simplify models (naive Bayes) but mislead when violated.
- A/B testing conclusions are probability statements about differences.
Mental model
Updating a rumor as evidence arrives
Bayesian updating
You start with a prior belief, see evidence, and revise. Bayes' theorem is the exact arithmetic of 'revise belief given evidence' — P(belief | evidence) = P(evidence | belief)·P(belief) / P(evidence).
How it works
- P(A or B) = P(A) + P(B) − P(A∩B); for disjoint events, just the sum.
- P(A and B) = P(A)·P(B) if independent; otherwise it's P(A)·P(B|A).
- Conditional probability P(B|A) = P(A∩B)/P(A) — the probability of B inside the A universe.
- Bayes' theorem rearranges conditioning: P(A|B) = P(B|A)·P(A) / P(B).
- A random variable has a distribution: discrete (e.g., Bernoulli) or continuous (e.g., normal).
Bayes' theorem in a spam filter.
In production
- Spam filters and fraud scores are posterior probabilities computed with Bayes' factors.
- A medical-diagnosis product reports P(disease | test) — not P(test | disease) — because word-of-mouth gets this backwards dangerously.
- Recommender systems use probabilistic graphical models for click-through predictions.
Best practices
- Always ask: is that probability conditional, and on what?
- State priors explicitly before quoting a posterior.
- Test independence claims with data before letting them simplify your model.
Common mistakes
- Confusing P(A|B) with P(B|A) — the 'prosecutor's fallacy' that generates false positives.
- Multiplying probabilities of dependent events as if independent.
- Ignoring base rates, so rare-but-real classes get over-penalized.
Debugging tips
- Sanity check any conditional with the base rate: if P(bug) is 1%, a 99% accurate test is still mostly false alarms.
- Verify sum rules numerically: the full set of outcomes must sum to 1 (or 100%).
- Simulate small cases in numpy to check hand-derived formulas.
Interview checkpoint
Quick quiz
You draw one card from a full 52-card deck. P(Ace) and P(Ace | it's a heart) are:
P(A|B) always equals P(B|A).