Classification
Beginner 10 min read Classification

Classification Overview

Deciding, not guessing, the bucket

Classification assigns rows to discrete classes via learned decision boundaries — binary vs multiclass, hard labels vs probabilities, and the metrics that measure each.

classificationbinarymulticlassdecision boundaryprobabilitieslabelling
Prerequisites: Supervised Learning Overview
Before we start

What you will learn

  • Contrast binary, multiclass and multilabel problems
  • Explain the decision-boundary picture of classifiers
  • Use probabilities over hard labels for downstream decisions
Introduction

What is this?

Classification predicts which class a row belongs to. Binary = two classes (fraud/tot), multiclass = many unordered classes (0-9 digits), multilabel = many classes can co-occur (tags). Classifiers learn decision boundaries in feature space and (for the useful ones) output class probabilities — which support ranking and threshold decisions that hard labels can't.

Motivation

Why do you need it?

  • Most business calls are classification-style: approve/reject, detect/ignore, bin/prioritize.
  • Boundary geometry explains why sklearns like linear vs trees differ in the same problem.
  • Probabilities enable ranking and cost-aware thresholds — the difference between a model and a product.
  • Metrics (precision/recall/F1/AUC) all presuppose the classification framing.
Think about it

Mental model

Imagine

A customs officer sorting travelers into 'allow/check/search'

Think of

Binary/multiclass classifiers

The officer learns from years of labeled travelers where the risky-feature line runs; errors cost money either way — that's precision/recall.

Mechanics

How it works

  • Define classes + label quality; check distribution before anything else.
  • Split stratifying by class so rare classes survive in validation.
  • Train a family: linear (linear boundary), trees (rectangular), kernel/neural (flexible).
  • Read probabilities, not just argmax; rank by predicted class-1 probability.
  • Choose the operating threshold from the business cost (not 0.5 by default).

Boundary vocabulary.

Binary
Multiclass
Multilabel
Probabilities
Hands-on

Examples

Binary, multiclass and multilabel in one line each

Binary, multiclass and multilabel in one line each
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.multiclass import OneVsRestClassifier
from sklearn.metrics import roc_auc_score
 
X = np.random.RandomState(0).normal(size=(300, 4))
y_binary = (X[:, 0] + X[:, 1] > 0).astype(int)
y_multi = ((X[:, 0] > 0).astype(int) + (X[:, 1] > 0).astype(int)).astype(int)
 
bin_model = LogisticRegression().fit(X[:, :2], y_binary)
print("binary AUC:", round(roc_auc_score(y_binary, bin_model.predict_proba(X[:, :2])[:, 1]), 3))
 
multi = OneVsRestClassifier(LogisticRegression()).fit(X[:, :2], y_multi)
print("multiclass classes seen:", np.unique(y_multi))

The binary model scores with a probability-based AUC; the multiclass model predicts from a one-vs-rest voting family — two flavors of the same boundary idea.

Real world

In production

  • Fraud: binary classifier + threshold tuned to alert cost; labels later feed a small review queue.
  • Triage: multiclass incident classifier sends tickets to the right team.
  • Content: multilabel tagger predicts all tags that apply per row.
Guidelines

Best practices

  • Audit the label distribution and label errors early; stratify every split; work with probabilities throughout the pipeline; keep class dictionaries versioned so predict labels can't silently reorder.
Watch out

Common mistakes

  • Multiclass when a binary high/low framing fits the decision better; confusing multiclass with multilabel; thresholding at 0.5 without a cost model; letting label names reorder between training and serving.
When things break

Debugging tips

  • Print predicted probability histograms per class — a one-sided model is a data problem.
  • Check version of label encoder maps between train and prod.
  • Confusion matrix per class reveals which pairs the boundary mixes.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

Predicting the digit {0..9} for a photo is:

2

A movie tagged [action, comedy, romance] is:

Go deeper

Official documentation

Classification module (sklearn)
docs
Classification crash course (Google)
video
Statistical classification | Wikipedia
docs