CNN
Advanced 16 min read CNN

Object Detection

Where things are, not just what they are

Object detection locates every instance of known classes inside an image — boxes plus labels. Two families dominate: two-stage (R-CNN) and one-stage (SSD/YOLO), trading accuracy for latency.

object detectionbounding boxYOLOFaster R-CNNanchorIoUNMS
Prerequisites: CNN Architecture
Before we start

What you will learn

  • Explain the detection output: boxes + classes, IoU, NMS
  • Compare one-stage vs two-stage detector trade-offs
  • Read NMS and anchor side-effects when boxes duplicate
Introduction

What is this?

Object detection outputs a set of bounding boxes, each with a class and confidence. Two-stage detectors (Faster R-CNN) first propose candidate regions then classify each; one-stage detectors (SSD, YOLO) predict boxes and classes directly from the feature grid — faster but historically less accurate at small objects. Evaluation uses IoU against ground truth and mean Average Precision (mAP).

Motivation

Why do you need it?

  • Classification answers 'what'; detection answers 'what AND where' — the core of surveillance, X-rays, satellites and self-driving.
  • The speed/accuracy split is a real deployment decision: one-stage for real-time, two-stage for accuracy-on-small.
  • Chasing detection quality means debugging losses, anchors and NMS — three separate systems.
  • It's the foundation that segmentation and tracking build on.
Think about it

Mental model

Imagine

A zookeeper scanning an enclosure

Think of

Object detection

The keeper must count and locate every animal (boxes) while judging each species (class). Hasty keepers (one-stage) miss tiny monkeys in the trees; meticulous keepers (two-stage) inspect candidate regions more carefully at the cost of time.

Mechanics

How it works

  • Anchors: predefined boxes at each feature-grid cell (aspect ratios/sizes) — the 'candidate boxes' the head refines.
  • Two-stage: RPN proposes ~2k regions, crops them, RegionCNN head classifies each — slow but precise.
  • One-stage: the head regresses deltas to anchors and class probabilities directly on the dense grid — fast.
  • IoU: overlap between predicted and ground-truth box; matches ordered by confidence.
  • NMS: suppresses lower-confidence boxes that heavily overlap a kept one (duplicates).

From features to final boxes.

Backbone
Candidate boxes
Head
NMS
Final boxes
Real world

In production

  • YOLO-family serves real-time video analytics (traffic, retail, security) on cheap GPUs/edge.
  • Faster R-CNN models audit documents/medical images where small-object recall beats latency.
  • Custom heads on detector frameworks (Detectron2, MMDetection) dominate industry adapts.
Guidelines

Best practices

  • Start with a pretrained detector and fine-tune; training from scratch needs enormous data.
  • Match your two metrics (mAP@.5 and mAP@.5:.95) to the deployment's tolerance to localization error.
  • Tune anchor sizes to YOUR typical object scale — the #1 small-object win.
  • Set NMS IoU threshold deliberately: too high = duplicates, too low = missed instances.
Watch out

Common mistakes

  • Reusing classification augmentations (flips can break 'left/right' detector semantics in traffic).
  • Evaluating with raw confidence instead of AP thresholds — mAP tells a different story.
  • Feeding un-resized images to a fixed-input detector and silently losing aspect ratio.
  • Expecting one-stage accuracy without its anchor/scale tuning.
Root cause analysis

Common issues

Error
Two boxes on one object at inference
Root cause

NMS IoU threshold too strict (no suppression)

Fix

Lower NMS IoU threshold (e.g. 0.45→0.5 is common; tune negative) or add score cutoff.

Error
Small objects never detected
Root cause

Anchors too large or backbone stride dropping them to one feature cell

Fix

Add small anchors/P-2 features or a separate small-object head.

Error
High false positives on background
Root cause

Score threshold too low or class imbalance in data

Fix

Raise score threshold, add hard negatives, and balance the class weights.

When things break

Debugging tips

  • Qualitative-first: print grid of detections (boxes,NMS artefacts) — easier than reading mAP.
  • Diagnose by failure category: missed = recall/anchors; duplicated = NMS; misplaced = localization loss weight.
  • Check per-class AP, not just mAP — a top-2 class can hide the bottom-10.
  • Visualize anchors on a few images to confirm they respect the object scale distribution.
Interviews

Interview checkpoint

Practice

Hands-on lab

Compute IoU and NMS by Hand
30 min · advanced
In progress
Objectives
  • Implement IoU from two bounding boxes in numpy
  • Implement a minimal NMS routine and verify on crafted cases
  • Understand threshold effects on duplicates and misses

Compute IoU and NMS by Hand

30 min · advanced

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

What does NMS (non-maximum suppression) do?

2

mAP is computed only at a single IoU threshold.

Go deeper

Official documentation

Faster R-CNN paper (v1)
docs
Detectron2 docs
docs