CNN
Intermediate 15 min read CNN

Convolution & Pooling in Depth

The exact math of sliding windows

Convolution slides a small kernel over an image to detect patterns, while pooling downsamples the map. Padding, stride and kernel size govern the output geometry — and most shape bugs live here.

convolutionstridepaddingmax poolingaverage poolingreceptive fieldfeature map size
Prerequisites: CNN Architecture
Before we start

What you will learn

  • Compute output sizes from input, kernel, stride and padding
  • Contrast max vs average pooling behavior
  • Grow the receptive field interpretably with depth
Introduction

What is this?

Convolution computes a weighted sum between a tiny kernel and each image window; pooling downsamples each channel by taking the max (or mean) over a window. Output size follows: out = (H − k + 2p)/s + 1. Padding preserves borders, stride trades resolution for cost, and stacking layers grows the receptive field so deep units 'see' more of the image.

Motivation

Why do you need it?

  • Every CNN shape bug traces to this one formula.
  • Pooling choice changes what's kept: max preserves the strongest activation, average preserves overall level.
  • Receptive field growth is why deep layers encode objects while shallow layers encode edges.
  • Kernel geometry directly sets compute: FLOPs scale with (H/s)·(W/s)·k²·C.
Think about it

Mental model

Imagine

Magnifying glass vs wide-angle lens staged in steps

Think of

Convolution layers growing receptive field

Early lenses see a dozen pixels (edges); mid lenses see a face region; late lenses see the whole scene context. The sequence of small lenses composes into a giant, but structured, view.

Mechanics

How it works

  • Kernel zips over positions: for window w out[i,j] = Σₖ,ₗ kernel[k,l]·input[i·s+k, j·s+l] + bias.
  • Stride 1 keeps spatial size (with padding); stride 2 halves it — the work-horse downsampler.
  • Max pooling: window's maximum, translating the strongest local feature. Average pooling: smooth average.
  • Padding 'valid' (none) shrinks the map; 'same' preserves size (pad = (k−1)/2).
  • Receptive field after stacking k×k kernels of depth D is approximately k + (D−1)·(k−1).

One kernel sliding over a 5×5 image with stride 1 and its output.

5×5 input
3×3 kernel
Dot-product map
2×2 max pool
Downsampled
Real world

In production

  • Downsampling pipelines (stride-2 convs) implement the 'halving' budget in ResNet-style spines.
  • Max pooling dominates early/mid stacks; average pooling appears in the final global pooling head.
  • Transposed/strided variants power upsampling in segmentation and GAN generators.
Guidelines

Best practices

  • Use 'same' padding to keep shape reasoning simple; explicitly compute sizes when you don't.
  • Prefer stride-2 conv over max-pool where you want learned downsampling.
  • Keep kernel sizes small (3×3/5×5) and stack — a 3×3 stack beats a single 7×7 with fewer params.
  • Add a tiny test: torch/tf auto-checks shapes at build time, but a hand table catches intent drift.
Watch out

Common mistakes

  • Mixing stride and kernel size in the output formula (out = (H−k+2p)/s + 1, not ×)
  • Too-aggressive early pooling discarding fine detail for later stages.
  • Forgetting padding entirely and watching the map shrink to nothing in 4 layers.
  • Believing max-pool always beats average — it's data-dependent (textures favor max, smooth maps favor avg).
Root cause analysis

Common issues

Error
Feature map becomes 0×0 after 5 convs
Root cause

Valid padding + large kernels + stride >1 multiply the shrink

Fix

Use 'same' padding/introduce pooling instead of consecutive strides.

Error
Large receptive field but low accuracy
Root cause

You shrank the field with early pooling — model sees too little context

Fix

Move downsampling later or reduce stride in early blocks.

Error
Dense layer dimension mismatch at runtime
Root cause

Conv output size changed but the flattened size wasn't updated

Fix

Use global average pooling, or calculate flat = C×H'/×W' exactly.

When things break

Debugging tips

  • Compute expected sizes on paper for a 32×32 input before running: mismatches reveal formula bugs.
  • Print one channel of a feature map as ASCII art to be sure the filter actually moved.
  • Compare FC layers' expectation: with strided pooling the first dense layer must see the right vector length.
  • If 'same' padding doubles your memory, double check you needed same vs valid.
Interviews

Interview checkpoint

Practice

Hands-on lab

Compute Conv and Pool Sizes in NumPy
25 min · intermediate
In progress
Objectives
  • Implement naive convolution and max pool in numpy
  • Verify the output-size formula on several configs
  • Compare max vs average pool outputs on a synthetic image

Compute Conv and Pool Sizes in NumPy

25 min · intermediate

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Input 32×32, kernel 5×5, padding 2, stride 1. Output spatial size?

2

Max pooling keeps the location-perfect 'where' detail at each downsampling step.

Go deeper

Official documentation

Compute Transposed Conv guide
docs
CS231n — pooling
docs