EDA in Service of Modeling
Explore with the model in mind
The EDA that matters for machine learning: target shape, leakage spotting, redundant and missing-heavy features, class balance and drift checks — decisions made during exploration that models inherit.
What you will learn
- Profile the target before any feature
- Spot leakage and redundant feature patterns
- Check class balance and drift implications
What is this?
Modeling EDA focuses the exploration lens on what training will inherit: the target's distribution and balance, feature missingness and redundancy, degree-versus-hours leakage (fields that encode the answer), and train/test drift. Getting these right in EDA is cheaper than rebuilding a model that silently learned the leakage.
Why do you need it?
- Leakage is the silent model killer — features that only exist after the outcome happened.
- Class imbalance discovered in EDA preps weighting/sampling decisions.
- Redundant features were already flagged by correlation matrices; modeling EDA formalizes them.
- Drift between train and live data is an EDA-detectable bankruptcy you otherwise find in production.
Mental model
A contractor inspecting the site before pouring foundations
Modeling EDA
The ground survey — soil, water table, load-bearing features — decides the foundation type. Pouring blind and 'adjusting later' is how buildings (and models) tilt. Modeling EDA is the site survey for training ML.
How it works
- Target pass: distribution, class counts, imbalance ratio, nulls in the target.
- Leakage scan: features that are only defined by the outcome (refund=0 implies no charge) or future-timestamped.
- Feature hygiene: redundancy pairs, null budgets over 50%, zero-variance columns.
- Balance: check class distribution and sample-size-per-class before any sampler.
- Drift: compare train snapshot stats to live/recent stats per key feature.
- Record decisions (drop/sample/weight) in the EDA notes for the modeling phase.
EDA gates a model relies on.
Examples
Catch the leak in five lines
import pandas as pddf = pd.DataFrame({ "months_active": [12, 6, 1], "churned": [0, 1, 1], "post_churn_support": [2, 5, 1], # only exists after churn event})print(df.corr().round(2))print("leak check: post_churn_support is defined ONLY post-outcome -> drop")The correlation column flags a perfect post-churn relationship; the EDA note says drop the leaked feature before training.
In production
- A churn pipeline EDA deletes 'last_support_after_churn' before any model sees it.
- A credit model's EDA checks class balance (98/2 default) and adds weighting to the plan.
- A feature review compares train and live distributions monthly and pages on drift.
Best practices
- Profile the target first — everything else serves it.
- Leakage scan every feature against the outcome timeline.
- Record model decisions in EDA notes so the pipeline is explainable.
- Pin train-snapshot statistics for the drift checkpoint.
Common mistakes
- Leakage through future timestamps, refund/status fields, or post-event tags.
- Training on a 98:2 imbalance without a plan and praising the 99% 'accuracy'.
- Dropping null-heavy features instead of flagging them as informative absence.
- Static train hash reused so drift between train and live never gets caught.
Debugging tips
- A feisty 0.9 feature-target correlation → ask 'could this only be known after the event?'.
- Perfect train accuracy with weak live scores → classic leak; re-run the timeline audit.
- A class with no rows in live data → drift or sampling bug; EDA catches both.
- When model metrics look great but the business doesn't, return to the EDA notes.
Interview checkpoint
Quick quiz
The signature of a leaked feature is:
99% accuracy on an imbalanced churn set can be meaningless.