Cosine Similarity & Distance
Measuring direction, not size
Cosine similarity measures orientation, not magnitude: identical direction = 1 regardless of length. Choosing the right metric — cosine vs Euclidean vs dot — decides whether your thresholds mean anything.
What you will learn
- Compute cosine, dot and Euclidean by hand
- Explain why normalization matters
- Choose a metric per use case
What is this?
Cosine similarity measures the angle between vectors, ignoring magnitude: perfectly aligned = 1, orthogonal = 0, opposite = −1 (for standardized embeddings, near 0..1). Euclidean uses actual distance; dot product mixes magnitude and direction. L2-normalized vectors make all three family-friendly — but they are only comparable within the same normalization convention.
Why do you need it?
- Cosine dominates semantic similarity because magnitude (word length/complexity) is noise.
- Thresholds ('similarity > 0.78') mean nothing without the metric+normalization defined.
- Vector DBs may index with one metric — choose before ingest, not after.
- Clustering/DEDUP algorithms assume a consistent distance everywhere.
Mental model
Comparing two arrows by direction only
Cosine similarity
A long arrow pointing NE and a short arrow pointing NE describe the same trend; their lengths are noise. Cosine throws away the shaft lengths and reports how aligned the directions are. Your application then decides whether 'aligned' is 'similar'.
How it works
- Cosine(a, b) = (a·b) / (|a||b|) — normalized dot of directions.
- Euclidean distance = |a − b|; direct geometric distance in the space.
- Dot = Σ a_i b_i — no normalization; magnitude bleeds in.
- L2 normalization: v = v / |v| makes cosine = dot product and Euclidean rank-equivalent to 2−2·cos.
- Which to pick follows your storage + train conventions and threshold semantics.
Metric decision tree.
Examples
Metric comparison
import numpy as np a = np.array([2.0, 0.0])b = np.array([1.0, 0.0]) # same direction, half length cos = np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))eucl = np.linalg.norm(a - b)print("cosine:", round(float(cos), 3), "(same direction = 1)")print("euclid:", round(float(eucl), 3), "(magnitude differs)")Cosine ignores length; Euclidean does not. That difference is why cosine suits semantically-similar-but-differently-sized text.
In production
- Set the vector index metric to cosine for semantics-first search.
- Clustering with Euclidean on L2-normalized vectors ≈ cosine clustering.
- Threshold tuning experiments log the exact metric per run.
Best practices
- Normalize when you embed; keep the pipeline idempotent.
- Document the metric in the index config and code review.
- Tune thresholds on labeled pairs, not vibes.
- Verify index metric == your query metric after migrations.
Common mistakes
- Cosine thresholds applied to non-normalized vectors.
- Mixing Euclidean during clustering and cosine at query.
- Assuming cosine ∈ [0,1] for raw embeddings (often negative for unrelated texts).
Common issues
Index and query disagree on nearnessMetric mismatch between index build and query
Standardize on one metric + normalization; rebuild index.
Thresholds behave oddly after model swapNew embedding model—different score distribution
Re-tune thresholds on a labeled sample after embedding model changes.
Debugging tips
- Scores suddenly ~1 for everything → non-normalized long vectors or index metric mismatch.
- Threshold '0.8' fails → check whether you normalized and which metric the index actually used.
- Clustering islands look wrong → same root cause: metric inconsistency.
Interview checkpoint
Hands-on lab
- Normalize a vector to unit length
- Prove dot-product-of-units equals cosine
- Show length-invariance with a scaled example
Cosine Is Direction, Not Length
20 min · beginner
Quick quiz
Cosine similarity focuses on…
After L2 normalization, cosine equals the dot product.