Random Forest
Beginner 10 min read Random Forest

Random Forest Overview

Many wobbly trees, one steady voice

Random Forest fits many de-correlated decision trees on bootstrapped samples and votes them together — trading a single brittle tree for a smooth, stable, high-accuracy ensemble.

random forestensemblebaggingvotingbootstrapde-correlation
Prerequisites: Decision Trees Overview
Before we start

What you will learn

  • Explain the ensemble voting intuition
  • Know the two randomness sources (samples + features)
  • See why the forest is more stable than one tree
Introduction

What is this?

Random Forest trains N trees, each on a bootstrapped re-sample of the rows, and at every split only a random subset of features can be considered. Individual trees are wobbly, but their average — mode for classification, mean for regression — cancels variance into a smooth, high-accuracy model with built-in estimates of feature importance and out-of-bag error.

Motivation

Why do you need it?

  • Accuracy: averaging de-correlated trees beats any one tree on most tabular problems.
  • Stability: no single slit path controls the answer, so small data wobbles rarely flip output.
  • Effort: two hyperparameters (n_estimators, max_features) get you 90% of the way.
  • Built-ins: OOB score and importances arrive free with no extra split.
Think about it

Mental model

Imagine

One nervous referee vs a jury

Think of

Single tree vs Random Forest

One referee leans on every whistle; a jury of dozens averages the biases into a fair verdict nobody can individually flip.

Mechanics

How it works

  • Bootstrap: draw n bootstrap samples of the rows (with replacement), one per tree.
  • Random features: at each split, only a random subset (typically sqrt(num_features)) is considered.
  • Grow each tree deep with its own sample; no pruning needed — variance is averaged away.
  • Aggregate: classification by majority vote of class probabilities; regression by mean.
  • OOB rows (never sampled) give per-tree holdout predictions -> out-of-bag error and per-tree importance free.

Forest pipeline at a glance.

Data
Bootstrap
Trees
Vote
Hands-on

Examples

Forest in five lines

Forest in five lines
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
from sklearn.datasets import make_classification
 
X, y = make_classification(n_samples=1200, n_features=20, n_informative=8,
                           flip_y=0.08, random_state=0)
rfc = RandomForestClassifier(n_estimators=200, random_state=0,
                             n_jobs=-1)
print("forest F1:", round(cross_val_score(rfc, X, y, cv=5).mean(), 4))
print("OOB acc :", round(RandomForestClassifier(n_estimators=200, oob_score=True,
      random_state=0).fit(X, y).oob_score_, 4))

Fit, cross-validate, and read an honest OOB estimate — three production-grade signals from a handful of lines.

Real world

In production

  • Baselined tabular model: works on mixed dtypes, no scaling, seeds reproducible — the default 'let's see where we stand' model.
  • Feature screening: importances + OOB rank candidates before anything fancy.
  • Data glitch tolerance: averaging swallows a few bad rows that a single tree would trace into a misleading path.
Guidelines

Best practices

  • Set n_estimators high enough that the OOB curve flattens; keep n_jobs=-1; fix random_state for reproducible marketing; always compare against a pruned single tree to quantify the ensemble payoff.
Watch out

Common mistakes

  • Thinking 10 trees is enough ('more always better' stops early); ignoring OOB while trusting a test set; letting high-cardinality features dominate importance readings; forgetting bootstrap default True means rare-class rows vanish from some trees.
When things break

Debugging tips

  • OOB >> test score => unlikely by design (same data), check the split leakage.
  • Importance dominated by one id/index column => drop cardinality noise before fit.
  • Slow training on 10k rows => reduce trees, cap depth; forest cost grows with size.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

Random Forest prediction is built by:

2

Feature-subset randomness exists mainly to:

Go deeper

Official documentation

Random Forest (sklearn)
docs
Random forest | Wikipedia
docs
Random forests for classification (Breiman paper)
docs