Intermediate 14 min read Classification

Class Imbalance

Don't let the majority drown the signal

When one class is rare, accuracy lies: fix with class weights, resampling (oversample/undersample/SMOTE), and metric switches — but leakage hides behind every resampling too.

class imbalanceclass weightsSMOTEoversamplingundersamplingPR-AUCrare class

Prerequisites: Logistic Regression

Before we start

What you will learn

Diagnose imbalance's trap: 99% accuracy on a 1% problem
Apply class weighting, oversampling, undersampling and SMOTE appropriately
Evaluate with PR-AUC/recall-curves instead of accuracy

Introduction

What is this?

Class imbalance means the minority class is rare (fraud 1%, defect 0.2%). Naive accuracy then rewards the majority blindly. Remediation: reweight the loss (class_weight), resample (oversample minority, undersample majority, SMOTE synthetic), and — the most important — evaluate with precision/recall/PR-AUC rather than accuracy. Weights/resampling are not interchangeable: correlate to the model and cost profile.

Motivation

Why do you need it?

A 99% accurate constant-predictor is operationally useless for catching rare events.
Every real fraud/anomaly/failure asset is imbalance-shaped.
Asymmetric error costs (a missed fraud $$ vs an annoying false alert) need explicit encoding.
Careless resampling (on full data, before split) is a classic discretely-bad leak.

Think about it

Mental model

Imagine

A goalkeeper whose practice had 99% shots at the empty corner

Think of

Class imbalance in training

Trainers rebalance drills to force saves (weighting/resampling) because practicing to 99% on one corner makes the real match a slaughter.

Mechanics

How it works

Measure the imbalance ratio and the cost of FN vs FP first.
Weights: class_weight='balanced' penalizes the majority class relative to size — zero sampling artifacts.
Resampling: oversample minority (and/or SMOTE for synthetic neighbors) or undersample majority (careful: loses data).
Split FIRST, then resample INSIDE the training fold — never the validation fold.
Evaluate precision/recall/PR-AUC and choose the operating threshold by cost.

Imbalance response options.

Diagnoseratio + error costs

Weightsloss reweighting

Resampleover / under / SMOTE

EvaluatePR-AUC + threshold by cost

Hands-on

Examples

Weighted LR on a 1-in-100 fraud set

import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split, cross_val_score, StratifiedKFold
from sklearn.metrics import roc_auc_score, average_precision_score
 
X, y = make_classification(n_samples=5000, n_features=10, weights=[0.99, 0.01],
                           random_state=0)
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, stratify=y, random_state=1)
 
plain = LogisticRegression(max_iter=3000).fit(Xtr, ytr)
w = LogisticRegression(max_iter=3000, class_weight="balanced").fit(Xtr, ytr)
 
for name, m in [("plain", plain), ("balanced", w)]:
    p = m.predict_proba(Xte)[:, 1]
    print(f"{name:>8}: AUC {roc_auc_score(yte, p):.3f}"
          f"  PR-AUC {average_precision_score(yte, p):.3f}"
          f"  #pred positives { (p > 0.5).sum()}")

Balanced weights trade a worse prior-matching threshold for real detection: PR-AUC rises and the model finally emits positive predictions instead of a monotone 0.

Real world

In production

Fraud: balanced-lightGBM + cost-threshold produces an alert list reviewers can actually clear.
Defects: SMOTE-oversampled tabular QC data lifts recall of rare defect codes.
Churn: weights handle 5% churn so the retention crew isn't fed 19,000 false alarms a day.

Guidelines

Best practices

Choose weights/resampling before the split is fabled; always split FIRST; evaluate PR-AUC plus recall@top-k for search/alert use-cases; tune the threshold with the cost model — imbalance remediation is upstream of that consideration.

Watch out

Common mistakes

SMOTE the full dataset (leak); measuring success with accuracy after remediation; oversampling to balance exactly (rarely the optimum); resampling the validation fold; confusing class weights with having 'fixed the problem'.

When things break

Debugging tips

Precision collapses after you raise recall — see the PR curve before committing.
Cross-tag check: did SMOTE rows appear in validation? If so, redo the split.
A model that returns 'predict all negative' is your first honest baseline — improvement is measured AGAINST it.

Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2

The correct split/resample order is:

Best metric on a 99/1 fraud problem?

Go deeper

Official documentation

Balancing classes (sklearn)

docs

imbalanced-learn docs