Statistics
Advanced 14 min read Statistics

Bayesian Thinking

Uncertainty you can talk about directly

Bayesian analysis updates a prior into a posterior with data, producing direct probability statements about parameters — an intuitive, flexible alternative to classical significance.

Bayesianpriorposteriorcredible intervalMCMCconjugateupdating
Prerequisites: Probability Basics Confidence Intervals
Before we start

What you will learn

  • Formulate prior, likelihood and posterior
  • Interpret credible intervals
  • Decide when Bayesian over frequentist adds value
Introduction

What is this?

Bayesian statistics treats parameters as random and updates beliefs with Bayes' rule: posterior ∝ likelihood × prior. The output is a probability distribution over the parameter — a posterior — from which you read credible intervals ('95% chance the conversion rate is in here'). Conjugate models make this analytical; MCMC (e.g., PyMC) extends it to complex models.

Motivation

Why do you need it?

  • Credible intervals say directly what people wrongly assume p-values say.
  • Priors encode domain knowledge and shrink unstable small-sample estimates.
  • Sequential updating matches how businesses learn (data arrives, beliefs refine).
  • Flexible modeling: hierarchical and multi-source models fit naturally.
Think about it

Mental model

Imagine

A detective updating suspect lists as clues arrive

Think of

Bayesian updating

You start with a prior hunch about the suspect (base-rate-weighted), then each clue re-weights the list. The detector's final degrees of belief over suspects are the posterior — a whole distribution, not a binary verdict.

Mechanics

How it works

  • Prior p(θ) encodes belief before data.
  • Likelihood p(data|θ) describes how plausible the data is at each θ.
  • Posterior p(θ|data) ∝ p(data|θ)·p(θ) — the updated belief.
  • Draws from the posterior (MCMC) estimate means, intervals and probabilities of any quantity.
  • A 95% credible interval is the central 95% of the posterior — direct probability, unlike a CI.

Prior × likelihood = posterior.

Prior
Likelihood
Multiply
Posterior
Decisions
Hands-on

Examples

A conjugate update in pure Python

A conjugate update in pure Python
# Beta-binomial: prior Beta(3, 10), observe 8 successes / 25 trials
a0, b0 = 3, 10
s, f = 8, 25 - 8
a_post, b_post = a0 + s, b0 + f
mean = a_post / (a_post + b_post)
 
# crude 95% interval via quantiles of the Beta posterior
from math import lgamma, exp
import statistics, random
random.seed(10)
samples = [random.betavariate(a_post, b_post) for _ in range(50000)]
samples.sort()
lo, hi = samples[2500], samples[47500]
print(f"posterior p ≈ {mean:.3f}  95% credible [{lo:.3f}, {hi:.3f}]")

After data, the prior Beta(3,10) becomes Beta(11,27); the posterior's interval is a direct probability statement.

Real world

In production

  • Conversion-rate dashboards showing credible intervals instead of significance flags.
  • Fraud scoring with priors tuned to base rates, updated daily as cases resolve.
  • Multi-region models that share strength (hierarchical) instead of analyzing slices in isolation.
Guidelines

Best practices

  • State your prior and its rationale on the same slide as the posterior.
  • Use weakly informative priors by default to avoid over-influence.
  • Summarize posterior draws (mean, interval, P(θ>0)) — not just a point estimate.
  • Validate with prior predictive and posterior predictive checks.
Watch out

Common mistakes

  • Using complex priors when data is abundant — the prior becomes a footnote.
  • Equating a credible interval with a frequentist CI (they answer different questions).
  • Skipping convergence checks on MCMC chains and trusting the draw.
  • A strongly informative prior that swamps all but enormous evidence.
When things break

Debugging tips

  • Posterior barely changed from prior — your likelihood is weak or the prior is strong; check both.
  • MCMC divergences/ESS warnings — reparameterize or thin; inspect trace plots.
  • Credible interval wider than expected — sample size under the model's variance.
  • P(θ>0) ≈ 0.5 means no signal — don't over-parse near-50 probabilities.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

The Bayesian posterior is proportional to:

2

A 95% credible interval is a probability statement about the parameter.

Go deeper

Official documentation

PyMC docs
docs
Bayesian Methods for Hackers (open book)
docs