Embedding Models
Picking the right geometry factory
Embedding models differ by dimension, training data, multilingual coverage and cost. Choosing the right one (for your text and task) matters more than any post-processing.
What you will learn
- Compare embedding-model properties to evaluate
- Match model choice to your text/task
- Re-run QA when swapping embedding models
What is this?
Embedding models are trained — usually contrastively — on pairs and hard negatives, and come in API and open variants with different dimensions, languages and quality footprints. The 'right' model is whatever scores best on YOUR similarity/dedup/classification task; model comparison is a benchmark you run, not a spec page.
Why do you need it?
- Dimension = storage + latency vs expressiveness trade-off.
- Multilingual coverage varies hugely — English-tuned models fail on mixed corpora.
- Open models let you self-host and avoid lock-in; APIs handle ops.
- Quality headroom between models can be the difference between search you trust and search you apologize for.
Mental model
Hiring a translator for your corpus
Embedding models
A translator fluent in your dialect and field cost more but renders meaning precisely; a cheaper generalist fumbles jargon and dialects. You interview (benchmark) candidates against your actual documents, not just read their résumés (spec sheets).
How it works
- Training: contrastive pairs — same-meaning texts pulled together, hard negatives pushed apart.
- Scores: retrieve-then-eval (e.g. retrieval hit rate) on your task.
- Dimension: small (256) vs large (1536+) trade-offs govern storage and latency.
- Multilingual: eval per-language; don't assume 'multilingual' in a name.
- Swap = new space; every downstream threshold/embedding must be recomputed.
Choosing an embedding model.
In production
- RAG teams hold an A/B of two embedding models on their retrieval dev set before committing.
- Multi-lingual support desks choose models evaluated on each supported language.
- High-volume ingest chooses dimension to fit memory/cost budgets.
Best practices
- Benchmark on YOUR task before choosing; spec sheets are the menu, not the tasting.
- Lock the embedding model in config and migrate deliberately.
- Consider multilingual + code coverage for mixed corpora.
- Keep a reusable eval snippet so the swap decision is rerunnable.
Common mistakes
- Choosing the biggest-number dimension 'because better'.
- Assuming one embedding model serves all languages well.
- Quietly upgrading embedding models without recomputing stored vectors and thresholds.
Common issues
Mixed embedding spaces after model swapExisting vectors not recomputed
Re-run the whole corpus through the new model before enabling queries.
Multilingual recall is lopsidedModel trained on skewed languages
Benchmark per language and pick a properly multilingual model.
Debugging tips
- Search collapses after 'minor' model bump — you mixed old and new vectors.
- Poor hits on one language but not others → monolingual skew in the chosen model.
- Index rebuild long → dimension too high for your ingest budget; consider a smaller model.
Interview checkpoint
Hands-on lab
- Mean-pool token vectors into one sentence vector
- Normalize the pooled vector
- See why mixing dimensions across models breaks comparability
Mean-Pooling and the One-Model Rule
25 min · intermediate
Quick quiz
The best way to pick an embedding model is…
You can swap embedding models without recomputing existing vectors.