Feature Engineering
Intermediate 14 min read Feature Engineering

Feature Construction

Invent signal the raw data only implies

Feature construction is the domain-informed creation of new columns — ratios, aggregates, lags, interactions, differences — that expose relationships models can't infer from raw fields.

feature constructionfeature interactionratiosaggregateslagsdomain knowledge
Prerequisites: Scaling and Normalization
Before we start

What you will learn

  • Build ratio, interaction, aggregate and lagged features from domain facts
  • Test each constructed feature honestly against a baseline on validation
  • Keep a feature manifest for production parity
Introduction

What is this?

Feature construction proposes new columns that encode domain or temporal structure: per-unit ratios (amount per item), interactions (age x income groups), aggregates (customer lifetime total spend, rolling means), and temporal shifts (lag 1 demand, day-of-week). Each candidate must justify itself by improving validation score — the baseline is the model without it.

Motivation

Why do you need it?

  • Raw columns hide the relationships that matter: a $1M order is huge for a cafe, trivial for Amazon.
  • Interactions express 'context matters': the sign of a feature may flip within another feature's group.
  • Time-series models can't learn 'last week's demand' unless you hand it over as a lag feature.
  • Good construction lets shallow (fast, interpretable) models match deep ones.
Think about it

Mental model

Imagine

A recipe that knows acidity balances sweetness

Think of

Domain-aware feature construction

The recipe didn't discover balance by staring at ingredients; a cook's knowledge proposed it. Same for features: know the business, propose the ratio, then the model confirms.

Mechanics

How it works

  • Brainstorm from domain: ratios, margins, densities, per-unit economics.
  • Interactions: multiply/cross numeric groups; for trees, combined columns can expose conditional splits.
  • Aggregates: group by key (customer, product, region) -> count, mean, std, min, max.
  • Temporal: lag_1, lag_7 rolling mean/std, difference from last period, holiday flags.
  • Validate each candidate: holdout or CV score with vs without; drop silent features.

Feature construction sources.

Domain
Interactions
Aggregates
Time
Validate
Hands-on

Examples

Ratios, interactions and lags

Ratios, interactions and lags
import pandas as pd
import numpy as np
 
df = pd.DataFrame({
    "customer": ["a", "a", "b", "b", "c", "c"],
    "order_qty": [10, 20, 5, 15, 8, 12],
    "order_value": [100, 260, 30, 120, 60, 140],
    "date": pd.to_datetime([
        "2026-01-01", "2026-01-15", "2026-01-02",
        "2026-01-16", "2026-01-03", "2026-01-17"]),
})
 
df["price_per_unit"] = df["order_value"] / df["order_qty"]
df["is_bulk"] = (df["order_qty"] >= 10).astype(int)
df["qty_x_bulk"] = df["order_qty"] * df["is_bulk"]
 
df = df.sort_values(["customer", "date"])
df["lag1_value"] = df.groupby("customer")["order_value"].shift(1)
df["rolling_value_mean"] = (
    df.groupby("customer")["order_value"]
    .transform(lambda s: s.rolling(2, min_periods=1).mean())
)
 
print(df.round(2))

price_per_unit normalizes across order sizes, the interaction flags bulk-with-value, and the lag/rolling features hand temporal memory to the model instead of making it rediscover last order.

Real world

In production

  • E-commerce demand: lag-7 + rolling-28 demand plus promo flag cut MAPE by ~40% vs raw timestamps.
  • Churn: 30/60/90-day spend velocity and tenure-based ratios beat raw monthly spend.
  • Pricing: per-SKU margins and competitor-price ratios drive regression lift.
  • Fraud: 1-hour transaction velocity and amount/vs-total ratio features are top ranked.
Guidelines

Best practices

  • Source each feature from an audited business fact; derive test/prod features through one shared function so train and live get identical math; keep lag windows small (persistential state grows with window).
Watch out

Common mistakes

  • Building features from future rows (leakage in lag misuse); using validation rows in aggregates; overfitting to one leading edge feature that decays (holiday flags on old data); inventing features with no business rationale and hoping.
When things break

Debugging tips

  • Plot each new feature vs target — visual signal check before submission.
  • Watch for NaN from shift()/rolling on the first rows; decide fill vs drop deliberately.
  • Compare feature importance ranks before/after construction to confirm new features earn weight.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

Which of these risks target/date leakage?

2

A feature helps train but not validation. Likeliest cause?

Go deeper

Official documentation

sklearn.preprocessing.PolynomialFeatures
docs
Feature Engineering for ML (book walkthrough)
video
Time-based features engineering guide | Influx
docs