Bivariate Analysis
The relationships that become features
Pairing variables reveals relationships: scatter/hexbins for two numerics, box/violin for numeric-by-category, correlation and crosstabs for categorical pairs — the raw material of feature ideas.
What you will learn
- Choose the right pair plot per variable types
- Read correlation with a scatter's eyes
- Build crosstabs for categorical relations
What is this?
Bivariate analysis pairs two columns. Numeric×numeric: scatter, hexbin for density, and the Pearson correlation. Numeric×categorical: box/violin per group, group means + error bars. Categorical×categorical: crosstabs and stacked bars. Each pairing produces signals that become modeling hypotheses.
Why do you need it?
- Scatter reveals non-linearity correlation scores will hide (r=0 for a U-shape).
- Group differences by a categorical prompt encoding/grouping decisions.
- Crosstabs expose interactions you'd never see univariately.
- EDA here is where the 'interesting feature' lightbulb usually goes on.
Mental model
A dance partner pairing audit
Bivariate analysis
You dance with one partner to feel the chemistry. A numeric pair's chemistry is correlation + scatter; the categorical-box tells you how each group moves; the crosstab shows who shows up together in clusters.
How it works
- Two numerics: sns.scatterplot(x, y, alpha=...) or sns.hexbin; compute df[['a','b']].corr().
- Numeric × categorical: sns.boxplot(x='group', y='value') + group means.
- Categorical × categorical: pd.crosstab(a, b, normalize='index') and % row shares.
- sns.pairplot(df) renders the whole correlation grid at once for exploration.
- Beware: a nonlinear U-relationship hides at correlation 0 — always eyeball the scatter.
Pair chart selection.
Examples
The three pairings
import pandas as pd, numpy as nprng = np.random.default_rng(3)x = rng.normal(size=2000)y = x ** 2 + 0.1 * rng.normal(size=2000) # U-shapeprint(pd.Series(x).corr(pd.Series(y))) # ~0 — hides the curvedf = pd.DataFrame({'group': rng.choice(['a','b'], 2000), 'v': rng.normal(size=2000)})print(df.groupby('group')['v'].mean())print(pd.crosstab(df['group'], rng.choice(['x','y'], 2000), normalize='index'))The U-shape returns correlation ≈ 0 — a scatter would have shown it instantly. Box-per-group and normalized crosstab round out the picture for the other pair types.
In production
- A pricing analyst scatters price vs purchases, spotting a quadratic sweet spot that r≈0 hid.
- A churn team box-plots tenure by churn class, prompting a 'early-tenure' binary feature.
- A recommender's crosstab of view×purchase by category reveals a cross-sell feature.
Best practices
- Always pair corr() with a scatter — correlation without shape is blind.
- Normalize crosstabs ('index') before comparing categories of different sizes.
- Add alpha to scatters; a sea of black dots hides density.
Common mistakes
- Trusting a single correlation number for non-linear relations.
- Imputing causation because a crosstab showed association.
- Zero-variance hue columns producing empty groups in plots.
Debugging tips
- correlation 0 in a plot that looks strongly curved? Compute x² as a feature and re-measure.
- Group means look equal but distributions differ wildly → check the box plot, not the mean.
- A crosstab with tiny cells → collapse rare categories or pooled estimates.
Interview checkpoint
Hands-on lab
- Scatter + correlation on nonlinear data
- Box-plot numeric by categorical groups
- Normalized crosstabs for categorical pairs
Pair Everything
30 min · intermediate
Quick quiz
Pearson correlation of a perfect U-shaped relationship is typically:
pd.crosstab(a, b, normalize='index') reports row proportions, enabling group comparison.