Supervised Learning
Beginner 12 min read Supervised Learning

Classification vs Regression

One pipeline, two target worlds

Same pipeline, different targets, losses and metrics: classification optimizes a mislabeling or ranking loss, regression optimizes an error against a continuous target.

classificationregressionlosscross-entropyMSEtarget typesmetrics
Prerequisites: Supervised Learning Overview
Before we start

What you will learn

  • Contrast the target types, losses and metrics of both families
  • Explain why regression metrics fail for rare-class classification
  • Pick metric sets appropriate to each family
Introduction

What is this?

Classification and regression run the same splitting, scaling and search machinery but differ in: target (discrete vs continuous), loss (cross-entropy vs squared error), and metrics (accuracy/PR-AUC vs MAE/R2). Some models exist only on one side (logistic = classifier, ridge = regressor), while trees and forests wear both hats.

Motivation

Why do you need it?

  • Mixing up losses/metics produces nonsense comparisons — a 99% accuracy 'regression' is meaningless.
  • The loss choice says what kind of mistake you're minimizing, and that should match the business.
  • Rare-event metrics (precision/recall, PR-AUC) matter for classification but have no regression analog.
  • Interviewers love the sharp distinction — it separates solid ML thinking from pattern-matching.
Think about it

Mental model

Imagine

A yes/no jury verdict vs a sentencing scale

Think of

Classification vs regression

A jury never hands out '3.6 guilty'; a judge never votes 'guilty/no'. Different outputs, different goals, different rules of evidence.

Mechanics

How it works

  • Classification: minimize cross-entropy / log loss; predict probabilities then thresholds; metrics accuracy, precision, recall, F1, PR/ROC AUC.
  • Regression: minimize squared / absolute error (or variants); metrics MAE, RMSE, R2, MAPE.
  • Shared: feature engineering, scaling for linear models, splits, CV, regularization, ensemble averaging.
  • Some estimators migrate: trees/forests/GBM do both; linear can't — logistic is inherently a classifier.

Two world maps.

Classification
Regression
Shared
Pick
Hands-on

Examples

Metrics that look right but mislead

Metrics that look right but mislead
import numpy as np
from sklearn.metrics import accuracy_score, mean_absolute_error
 
# classification: 99/1 split, constant model is 99% accurate
y = np.array([0]*99 + [1])
print("constant classifier accuracy:", round(accuracy_score(y, np.zeros_like(y)), 3))
 
# regression: tiny MAE, but the important tail is wrong
y_true = np.array([10, 10, 10, 1000])
y_pred = np.array([11, 9, 10, 300])
print("regression MAE:", round(mean_absolute_error(y_true, y_pred), 2))

Accuracy hides the 1% miss entirely; MAE can look low while a big-money tail is badly wrong — metric choice is a model-selection decision, not a formality.

Real world

In production

  • Fraud classification: PR-AUC + threshold-cost analysis; a regression MAE would be worse than useless.
  • Pricing regression: RMSE in log-space; a classification 'high/low price' framing would shred nuance.
  • Hybrid: classify 'will churn' then regress 'expected LTV loss' per churner.
Guidelines

Best practices

  • Write the metric into the framing doc before training; for classification report both a ranking metric and a threshold metric; for regression report at least RMSE + MAE to understand tail behavior.
Watch out

Common mistakes

  • Accuracy on imbalanced data; MSE on dollar targets where a log transform is needed; a threshold decision with no cost model; evaluating regression with correlation alone.
When things break

Debugging tips

  • Histogram predicted probabilities for a classifier — is the model using the whole [0,1]?
  • Scatter residuals vs fitted for a regressor — fan shape => log target.
  • If classification metrics plateau, review thresholds; if regression metrics plateau, review scaling of target.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

The loss of choice for classification is typically:

2

A valid regression metric pair is:

Go deeper

Official documentation

sklearn metric scoring docs
docs
Classification metrics guide | Google
video
Mean squared error | Wikipedia
docs