Classification
Intermediate 13 min read Classification

Choosing Among Classifiers

The sensible classifier map

Logistic (linear, auditable), naive Bayes (fast on sparse), SVM/kernel (small separable sets), trees and forests (interactions, robustness) — pick by data shape, audit needs and latency.

classifier comparisonlineartreesSVMnaive bayeschoose modeltrade-offs
Prerequisites: Multiclass Classification
Before we start

What you will learn

  • Map dataset traits to a short classifier menu
  • Design the mandatory baseline-vs-candidate comparison
  • Price interpretability, latency and calibration into the choice
Introduction

What is this?

Choosing a classifier is a trade of boundary flexibility, sample efficiency, interpretability, calibration, and latency. Rough map: logistic = linear + auditable baseline; naive Bayes = sparse/high-dim + instant; SVM-RBF = small + separated + needs tuning; trees = interpretable thresholds; forests/GBM = interactions + robustness with tuning cost; neural = big unstructured data. Decide on validation + constraints, not reputation.

Motivation

Why do you need it?

  • A tuned tree often beats a default deep net on clean tabular data — the map prevents wasted sprints.
  • Audit requirements (regulatory) can force the linear/interpretable lane regardless of scores.
  • Latency budgets in serving decide the envelope before model quality is even discussed.
  • Every comparison needs the shared CV scaffold — otherwise you're comparing apples to axes.
Think about it

Mental model

Imagine

Picking a vehicle per road and budget

Think of

Choosing the classifier

A sedan (logistic) for the highway, a bike (naive Bayes) to weave traffic, a rally car (SVM/tuned) on tight hills, an SUV+parts (forest) for mud. No universal winner — match the machine to the road and the cargo rules.

Mechanics

How it works

  • Catalog constraints: interpretability (audit/legal), latency targets, data size and dimension.
  • Fit the baseline: regularized logistic (+ the constant-predictor bar).
  • Fit the contenders under identical CV/seed/preprocessing.
  • Compare on the business metric, then re-check with latency/calibration in the room.
  • Escalate to the winner only if it clears the baseline by a decision-worthy margin.

Roadmap for classifier choice.

Linear fits?
Sparse/hi-dim
Non-linear small
Interactions+robust
Hands-on

Examples

The constraint-first comparison

The constraint-first comparison
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import GaussianNB
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
 
X, y = make_classification(n_samples=4000, n_features=12, n_informative=6,
                           class_sep=1.0, random_state=0)
cv = StratifiedKFold(5)
 
menu = {
    "logistic": make_pipeline(StandardScaler(), LogisticRegression(max_iter=3000)),
    "gaussianNB": GaussianNB(),
    "rbf_svm": make_pipeline(StandardScaler(), SVC(C=1, gamma="scale")),
    "forest": RandomForestClassifier(n_estimators=200, random_state=0),
}
for name, m in menu.items():
    s = cross_val_score(m, X, y, cv=cv, scoring="f1_macro")
    print(f"{name:>10}: F1 {s.mean():.4f} ± {s.std():.4f}")

Four candidate families face identical CV, so the score table is honest and the format invites constraints (latency, audit) to veto before latency decides.

Real world

In production

  • Regulated banking: logistic wins despite lower raw AUC because coefficients must be explained.
  • Text spam New build: MultinomialNB ships first at 0.1% of the compute of the eventual GBM.
  • Real-time ad scoring: shallow forest <2ms latency replaces a gaudier model that couldn't serve.
Guidelines

Best practices

  • Always include the constant baseline and the linear baseline; hold preprocessing identical; quote mean AND std; run the latency/audit veto BEFORE burning a sprint on the winner.
Watch out

Common mistakes

  • Comparing models built with different feature sets; omitting the baseline; favoring the 'most advanced' by default; ignoring calibration/latency until deployment hits a wall.
When things break

Debugging tips

  • If every contender is within noise of logistic, your problem is basically linear — stop shopping.
  • A winner that only wins in recall but craters in precision needs both numbers on the table.
  • Re-run the 2-way tie under a different seed before believing a 0.002 lead.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

Which abstraction fits 'auditability required'?

2

The mandatory baseline in every comparison is:

Go deeper

Official documentation

Choosing the right estimator (sklearn map)
docs
Classifier comparison (sklearn example)
video
F1 score | Wikipedia
docs