The EDA Framework
Ask the data who it is before you ask it anything else
EDA is a disciplined conversation with the data before the analysis answers a question: shape, cleanliness, distributions, relationships, gaps — each inspected, each documented.
What you will learn
- Run the standard EDA pass: shape, dtypes, nulls, distributions, relationships
- Document findings rather than letting charts accumulate
- Finish EDA with concrete questions for the analysis that follows
What is this?
EDA is the structured first look: dataset shape and grain, column types and cardinals, missingness, distributions of every variable, pairwise relationships, and odd corners. Its output is not a chart dump but a short report of facts and hypotheses that both guides and de-risks the formal analysis which follows.
Why do you need it?
- Decisions made from unexamined data are bets; EDA turns bets into expectations.
- Revealing data quirks early prevents embarrassing late 'why is it like this?' moments.
- Hypotheses generated during EDA get tested rigorously instead of assumed.
- Analysts who explore systematically produce analyses that survive review.
Mental model
A doctor's initial workup before differential diagnosis
EDA
The doctor checks vitals, history, labs — systematically, before any treatment. EDA is the same: inspect the table's vitals (shape/types), history (distributions, missing data), and relationships (correlations) before committing to any analytic conclusion.
How it works
- Skeleton: shape, columns, dtypes, head/tail, info().
- Univariate: value counts, distributions, percentiles, no-NaN check per column.
- Bivariate: cross-tabs, correlations, grouped summaries by key columns.
- Gaps: missingness map, duplicates, outliers triaged.
- Collect: write down facts, suspicions and hypotheses; leave the notebook with questions, not just views.
The EDA loop.
In production
- A growth analyst's EDA pass uncovers that 'signups' double-count emails before any revenue analysis starts.
- A data-quality review runs a standardized EDA template across 40 tables monthly.
- A modeling handoff ships its EDA report to explain feature choices to stakeholders.
Best practices
- Always start with the skeleton — inspect before infer.
- Write findings as you go; three bullet facts beat thirty unlabeled charts.
- Use consistent summaries (describe, value_counts, missingness) so comparisons are fair.
- End every EDA with the questions to answer next.
Common mistakes
- Jumping to charts before checking dtype and nulls.
- Drowning in uninterpreted plots with no written takeaways.
- Treating EDA as one-time; data changes, so exploration should too.
- Editing data during exploration — explore, then clean deliberately.
Debugging tips
- A surprising histogram → re-check the underlying dtype and null handling.
- Two tables disagree → grain difference; confirm row identity for each.
- Missing sheer volume of charts = missing QUESTIONS — go back to facts.
- Keep raw frames untouched during EDA so cleanup decisions stay clean.
Interview checkpoint
Hands-on lab
- Execute the skeleton + univariate + bivariate pass
- Record findings as a list of facts
- Finish with at least two hypotheses
Run a Structured EDA Pass
30 min · beginner
Quick quiz
The first thing to check about any new dataset is:
A good EDA ends with a list of questions to test next.