Similarity Search
Finding real neighbors, not souvenirs
Similarity search finds the nearest vectors to a query under a metric. Correcting the knobs — k, threshold, index metric — is where 'search works' separates real products from demos.
What you will learn
- Run a top-k similarity query
- Explain the meaning of k and thresholds
- Design a search UI that handles empty/weak results
What is this?
Similarity search takes an embedded query and returns the vectors nearest under the configured metric — usually respecting a top-k limit and optional score threshold. It is the read-side of every semantic search engine, and its quality story (metric, k, threshold, blank-slate handling) is what users actually feel.
Why do you need it?
- Top-k alone without a threshold returns 'the least bad result' instead of nothing — a UX bug.
- K is a WHAT answer; threshold is a WHETHER answer.
- Score distributions shift per corpus: static thresholds drift silent.
- Embedded queries must match the index's metric and normalization to be meaningful.
Mental model
'Closest three restaurants' house rules
Similarity search
You don't ask for the nearest three restaurants and let a 20 km diner count as 'near'. Rules: within X km (threshold) and best-rated top 3 (k). Without the radius, you'd always present some restaurant — even a bad one. Same for vector search.
How it works
- Query text → the SAME embedding model → vector.
- Metric (cosine/L2) matches index convention — always.
- k controls returned count; score returned per neighbor.
- Threshold filters candidates after the ANN walk.
- Exact fallback (brute force) optional for small corpora or high-recall needs.
Search lifecycle.
In production
- RAG: top-k passes to the LLM with scores for downstream reprocessing.
- Product search: k=5 with a conservative threshold dials 'no matches' honesty.
- Dedup jobs search neighbors then threshold for true duplicates.
Best practices
- Return scores and let product logic decide 'none'.
- Re-calibrate thresholds on real query distributions, not launch demos.
- Log query/scores with sampled relevance to spot distribution drift.
- Engage UX: empty results with suggestions beat a random neighbor every time.
Common mistakes
- No threshold: always answering top-k, even when irrelevant.
- Querying with a different model/metric than the index.
- Comparing scores across different k or threshold settings.
- Assuming scores are calibrated confidences (they aren't).
Common issues
All queries return low-similarity itemsK too large with no threshold
Add threshold or reduce k; make 'no match' a first-class outcome.
Scaled score shift after code releaseNormalization or index metric changed
Pin metric+normalization in config; re-tune threshold after any change.
Debugging tips
- Irrelevant-but-nearest results → threshold too loose or model mismatch.
- Empty results on plausible queries → threshold too strict for expected similarity.
- Score histogram bimodal → two embedding populations; re-check normalization.
Interview checkpoint
Hands-on lab
- Score a small corpus against a query with cosine
- Rank documents and return the top-k
- Filter results below a similarity threshold
Brute-Force Top-k in Pure Python
25 min · beginner
Quick quiz
Threshold in similarity search answers the question…
Scores are calibrated probabilities of relevance.