Python for Analytics
Intermediate 13 min read Python for Analytics

Functional Python

Data as transformations, not bookkeeping

map, filter, reduce, lambdas and itertools process data as transformations rather than step-by-step state — leading to cleaner, more composable analytics code.

mapfilterreducelambdazipitertoolscompositionimmutability
Prerequisites: Comprehensions and Generators
Before we start

What you will learn

  • Use map, filter and functools.reduce fluently
  • Compose small pure functions into pipelines
  • Leverage itertools for combinatorial and grouping logic
Introduction

What is this?

Functional style treats data processing as composing pure functions — no hidden state, predictable output. Python supports it through first-class functions, lambdas, map/filter/reduce, zip, and itertools' combinatorial generators. Analysts get declarative pipelines: 'take rows, filter, map to a metric, reduce to totals'.

Motivation

Why do you need it?

  • Pure transformations are easy to test, since output depends only on input.
  • Chained map/filter reads like the business rule instead of like loop plumbing.
  • Immutability prevents the shared-state bugs that corrupt shared notebooks.
  • itertools solves tricky grouping/combining problems in one line.
Think about it

Mental model

Imagine

Conveyor-belt stations in a factory

Think of

Functional pipelines

Each station takes an item, does one transformation, and passes a new item onward — nothing is altered in place. map/filter/reduce are exactly those stations: transform, keep-or-drop, and combine into one final output.

Mechanics

How it works

  • map(f, seq) applies f to every element lazily; list(map(...)) materializes.
  • filter(pred, seq) keeps only elements where pred(x) is truthy.
  • functools.reduce(f, seq) folds two-at-a-time into a single accumulator (sum, max, combine dicts).
  • lambda x: expr defines a small anonymous function on the spot.
  • zip(*seqs) pairs parallel sequences; itertools.groupby, chain, product, accumulate handle grouping and combinations.
  • Prefer comprehensions when they read equally well; use map/filter when they read better.

A functional pipeline.

Source
filter
map
group/zip
reduce
Hands-on

Examples

Pipeline vs loop

Pipeline vs loop
from functools import reduce
import itertools
 
rows = [("austin", 120), ("austin", 80), ("houston", 200), ("houston", 300), ("austin", 40)]
valid = filter(lambda r: r[1] >= 50, rows)
cities = map(lambda r: r[0], valid)
counts = {c: sum(1 for _ in g) for c, g in itertools.groupby(sorted(cities))}
total = reduce(lambda a, b: a + b, (s for c, s in rows if s >= 50))
print(counts, total)

filter removes low rows, map extracts cities, groupby counts them, and reduce sums the qualifying sales — each step a pure transform.

Real world

In production

  • A support analytics job pipelines ticket rows through filter(open) → map(duration) → reduce(avg_daily) without mutation.
  • An ETL uses itertools.groupby to bucket event streams by session key after sorting.
  • A feature pipeline composes 6 small pure functions that are unit-tested independently.
Guidelines

Best practices

  • Keep functions pure: no global mutation, no printing inside transformations.
  • Use lambdas only for tiny logic; name anything with a real rule.
  • Compose with functools.partial for parameterized transforms.
  • Prefer the comprehension form when it's equally readable — both are idiomatic.
Watch out

Common mistakes

  • Mutating the input list inside a map/filter — you've broken the purity contract.
  • Confusing filter (keeps truthy) with rejecting truthy.
  • Forgoring that map/filter are lazy; a bare call returns an iterator, not a result.
  • Using reduce where sum/count/functools already exists — a readability tax.
When things break

Debugging tips

  • Print(list(iterator)) to see what a lazy map/filter actually yields.
  • A wrong filter drops everything — test the predicate on three known rows.
  • groupby requires sorted input; forgetting that silently splits groups.
  • When a reduce misbehaves, run the same fold manually on 3 elements to find the failing step.
Interviews

Interview checkpoint

Practice

Hands-on lab

Refactor a Loop into a Functional Pipeline
30 min · intermediate
In progress
Objectives
  • Transform a for-loop aggregation into map/filter/reduce
  • Use itertools.groupby for clean bucketing
  • Verify the refactor produces identical results

Refactor a Loop into a Functional Pipeline

30 min · intermediate

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

filter(lambda x: x>0, data) returns:

2

groupby works correctly regardless of input order.

Go deeper

Official documentation

Python docs: itertools
docs
Python docs: functools
docs