CNN
Intermediate 16 min read CNN

CNN Architecture

Built for spatial structure

CNNs use local connectivity, shared weights and pooling to exploit spatial structure in images. A stack of convolution stages followed by a classifier head is how most image models are built.

CNNfeature mapsconvolutionpoolingshared weightsreceptive fieldimage classification
Prerequisites: Deep Learning Basics MLP
Before we start

What you will learn

  • Explain the three key CNN ingredients: local connectivity, weight sharing, pooling
  • Trace the tensor shapes through a typical conv stack
  • Explain what a feature map actually encodes
Introduction

What is this?

A Convolutional Neural Network processes images as grids of pixels using small, weight-shared filters (kernels) that slide over the input producing feature maps. Stacked conv layers build features from edges and textures to parts and objects, while pooling downsamples to keep the computation tractable. A final classifier (usually a couple of dense layers over the flattened/global-pooled features) outputs predictions.

Motivation

Why do you need it?

  • Images have translation invariance — a cat is a cat wherever it is; weight sharing encodes that.
  • Local filters keep parameters tiny: a 3x3 kernel has 9 weights versus a dense layer's n².
  • Convolutions compose: deep stacks learn edges → shapes → parts → objects hierarchically.
  • CNNs beat MLPs on images because locality and translation are baked in, not learned.
Think about it

Mental model

Imagine

A detective squad sweeping a city grid with the same checklist

Think of

Convolution with shared filters

Each detective uses the same question-list (a kernel) at every window of the grid (weight sharing), building a map of 'where is this pattern present'. Deeper teams integrate 'chalk outline found then nearby ID found' into 'solved case' — features of features.

Mechanics

How it works

  • A kernel of size k×k slides across the input with stride s, computing a dot product at each position.
  • Each kernel yields one feature map channel; C kernels give C channels (e.g. H×W×C).
  • Pooling (max/avg over a window) reduces spatial size, adds slight translation robustness, cuts FLOPs.
  • Conv blocks alternate conv → activation (ReLU) → pooling; the final global pooling flattens to a vector.
  • A dense head maps that vector to class logits; the whole thing trains end-to-end with backprop.

The conv-pool-classify pipeline.

Input image
Conv block
More blocks
Global pooling
Dense head
Real world

In production

  • ImageNet-scale classification, product-tagging, medical scan screening — all CNN stacks with dense heads.
  • Feature extractors for fine-tuning: freeze the conv spine, swap the head for your task.
  • Embedding featurization: CNN feature vectors feed image-search and clustering systems.
Guidelines

Best practices

  • Prefer pretrained spines (ImageNet) whenever your dataset isn't millions-thick — usually wins.
  • Keep spatial dims sensibly halved per pooling; watch memory for H×W×C blowups.
  • Use global average pooling before the dense head to keep the classifier translation-robust.
  • Match input resolution to the problem: don't pay 512×512 when 224×224 moves the needle.
Watch out

Common mistakes

  • Making every filter huge (7×7 everywhere) — wastes parameters and compute.
  • Skipping normalization before convs, destabilizing deep stacks.
  • Reading 'conv' as magic and never checking whether a pretrained baseline solves it.
  • Pooling too aggressively early, discarding detail the later layers need.
Root cause analysis

Common issues

Error
Out of memory very early in training
Root cause

Feature maps huge from large input + no pooling

Fix

Add pooling or stride-2 convs and reduce input resolution.

Error
Training is stable but accuracy is low
Root cause

Input scaling or color-space mismatch vs what the kernel expects

Fix

Normalize like the reference pipeline (e.g. /255 then per-channel stats).

Error
Validation jumps when input size differs from train
Root cause

A resolution-dependent head (dense after flatten) hard-sizes

Fix

Use global average pooling so the classifier accepts variable spatial sizes.

When things break

Debugging tips

  • Print tensor shapes after every conv/pool — a single stride bug corrupts the whole stack.
  • Visualize a learned edge detector early in training — gradient/test failures show here first.
  • Overfit one image: if loss won't collapse to ~0 on a single sample, the conv pipeline is broken.
  • Check channel counts at the pooling boundary: H×W×C transitions are the classic mismatch.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

Which CNN property encodes translation invariance?

2

A 3×3 convolution followed by activation produces a single-channel output.

Go deeper

Official documentation

CS231n CNN notes
docs