Model Explainability
Advanced 15 min read Model Explainability

Counterfactual Explanations

Not why — what would change the answer

A counterfactual says what minimal change to the inputs would flip a prediction. Unlike attributions, it's actionable: 'raise revenue by 8% and keep inventory under 1.2M and the loan is approved'.

counterfactualwhat-ifminimal changeactionabilityflipadversarial-ish
Prerequisites: SHAP Values Explainability Overview
Before we start

What you will learn

  • Formulate counterfactual search as minimal perturbation
  • Compare counterfactuals vs SHAP (attribution vs action)
  • Apply constraints: feasibility, actionability, sparsity
Introduction

What is this?

A counterfactual explanation answers: 'what is the smallest change to THIS input that would flip its prediction?' — e.g. from reject to approve. You search the input space for a nearby instance whose predicted class follows the desired rule, minimizing a distance (ideally weighted by feature actionability and feasibility). Results read as 'change A to A′ (and nothing else) to get X'. It's the natural bridge from explainability to decision support.

Motivation

Why do you need it?

  • Actionability: attribution says 'income pushed you down'; counterfactual says what to ACTUALLY change.
  • Justification: 'we can't approve today; here's exactly what would change that' is defensible.
  • Sparsity: models surface a one-line, low-dim instruction rather than a full explanation.
  • Natural AI-as-advisor: used directly in catalogs, portfolio and what-if tools.
Think about it

Mental model

Imagine

A driving instructor saying exactly which move changes the route

Think of

A counterfactual

Instead of 'your route was heavy on the highway' (attribution), the instructor says: 'turn right at the next light instead of the highway and the trip fits the time limit' — a minimal, actionable change that flips the outcome from late to on time.

Mechanics

How it works

  • Define desired outcome (e.g. y ≥ threshold) and feature distance (weighted L1/L2 in scaled space).
  • Search: optimize (or enumerate) neighboring inputs that achieve the flip while minimizing distance.
  • Sparsity: penalize nonzero changes so few features move — explanations stay legible.
  • Feasibility: hard-constrain immutable features (age, zip, contract type) to remain fixed.
  • Validity check: the flipped instance must be a plausible real-world point, not adversarial noise.

Nearest instance on the other side of the boundary.

Prediction f(x)=no
Feature changes
Constraints
f(x′)=yes
Readable CF
Real world

In production

  • Lending: auto-generate 'how to get approved' letters with concrete knobs.
  • Churn: alert the CS team with the one or two plan changes predicted to keep the customer.
  • Ops: model flips into diagnostic suggestions for failing services.
Guidelines

Best practices

  • Weight distance by actionability: immutable features cost infinite; hard-to-change cost high.
  • Verify plausibility — reject CF instances that fall outside the training manifold.
  • Offer 3-5 diverse alternatives (different features) so users have choices.
  • Pair CFs with a validity number (model confidence at the flipped point) for honesty.
Watch out

Common mistakes

  • Suggesting changes to immutable features ('change your age').
  • Finding a mathematically-near but physically-impossible point and shipping it.
  • Treating CFs as upper-bound guarantees — the model is what it is.
  • Retraining drift: a CF computed on a stale model version misleads decisions.
Root cause analysis

Common issues

Error
CF suggests changing immutable field
Root cause

No actionability weights in the objective

Fix

Set immutable-feature cost to ∞ (hard constraint) in the search.

Error
All users get the same CF
Root cause

Search collapsed to one global boundary point or over-tight sparsity

Fix

Diversify: multiple restarts, different intializations, and a diversity term across returned CFs.

Error
CF flips prediction but looks impossible in reality
Root cause

Objective minimized raw distance off-manifold

Fix

Constrain features to plausible ranges; clamp to observed percentiles; validate with a density check.

When things break

Debugging tips

  • Sanity: flip a known-easy boundary case first; the CF should be a small, sensible move.
  • Run sensitivity: does the minimal CF jump when you tweak weights? Unstable CF = weak search config.
  • Log the instance's true position: nearby adversarial CFs (with microscopically small deltas) should be excluded.
  • Compare a CF against a hand-built feasible alternative for realism.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

A counterfactual explanation is primarily:

2

Counterfactual search should always allow changing age or contract type freely.

Go deeper

Official documentation

Counterfactual explanations survey
docs