Embedding Model Selection for Domain-Specific Corpora

A retrieval system built on the top-ranked embedding model from a public leaderboard can still return the wrong documents for most queries a domain corpus generates. The number a team sees on that leaderboard is not the number that matters: it is not a measurement of how well that model will retrieve legal filings, clinical notes, or internal codebases, because the leaderboard was never built to test that.

Public leaderboards measure the wrong thing for specialized corpora

Teams choosing an embedding model typically start with a single number: a model's rank or average score on MTEB, the Massive Text Embedding Benchmark. That number is aggregated across dozens of datasets and tasks, and that aggregation makes it unreliable for a retrieval-augmented generation system built on a specialized corpus. MTEB combines results across classification, clustering, and semantic textual similarity, among other task types, and a model's overall position can be driven substantially by strength in classification even when its retrieval performance, the capability RAG systems actually depend on, is mediocre. A model can rank near the top of the leaderboard and still be a poor retriever.

This is a structural feature that industry guidance has already flagged directly: one 2026 ranking guide notes that a model dominating classification tasks can underperform on retrieval, and recommends that teams evaluating models for retrieval use task-specific NDCG scores rather than the overall MTEB average. Averaging across task types treats retrieval as one input among many, when for a RAG pipeline it is the only input that matters.

The second structural problem sits in the composition of the underlying datasets. MTEB's benchmark data is drawn overwhelmingly from English, web-scale sources: Wikipedia, news corpora, general-purpose question-answering sets. None of that resembles the text most enterprise retrieval systems actually run against, whether that is internal product documentation, medical notes, legal filings, or a company's own codebase. A model's leaderboard rank reflects how well it represents general-purpose sources, not how well it represents a patent claim, a discharge summary, or a ticket filed by a support engineer.

The leaderboard is also a moving target. Rankings shift monthly as new models are submitted, and any snapshot reflects only the state of things in early 2026, likely to change again before a team finishes running its own evaluation cycle. A model selection process anchored to this week's leaderboard position is anchored to a number that will look different by the time the system ships.

None of this means MTEB is worthless. It means the leaderboard measures something real, just not the thing most teams assume it measures when they pick a model for a specialized retrieval task. The diagnosis here is structural. The mechanism behind the failure must be clear before the fix can follow.

How vocabulary fragmentation and semantic conflation degrade domain retrieval

Two distinct failure modes explain why a model with a strong general-purpose reputation can perform poorly once it meets domain-specific text, and understanding both gives a team a concrete signal to look for rather than a vague warning to distrust the leaderboard.

The first is vocabulary fragmentation. Embedding models tokenize text into subword units before they ever produce a vector, and the vocabulary size used during training sets a hard limit on how many terms get their own dedicated token. A roughly 30,000-word vocabulary causes a technical term absent from that vocabulary to be broken apart into smaller, generic subword pieces, and that fragmentation degrades semantic accuracy in legal, medical, and scientific search. A legal citation format, a medical abbreviation, or an internal product code does not show up in the web-scale text most models train on, so it gets shredded into fragments that individually carry no domain meaning. The resulting vector for that term is an average of noise rather than a representation of what the term actually denotes. A better prompt or a smarter chunking strategy cannot patch over this flaw. It is baked into which tokens the model learned to represent well during training, and no amount of downstream engineering recovers information the tokenizer never captured.

The second failure mode is semantic conflation, and it operates in the opposite direction from fragmentation. Rather than atomizing a term into meaningless pieces, a general-purpose model can collapse genuinely distinct meanings into the same vector. A word like "discharge" carries entirely different meanings in a hospital record, an electrical engineering spec, and a legal settlement, but a model trained on broad web text has no strong incentive to carve those senses apart, so it can embed the word nearly identically across all three contexts. The vector space that model learned reflects the statistical regularities of general text, and those regularities are simply the wrong geometry for a domain where a single surface form needs several distinct semantic identities. This appears concretely in entity resolution: general-purpose models optimized for broad semantic relatedness tend to score different entities with overlapping names as highly similar, while scoring genuine duplicate records that use abbreviation variants as dissimilar, the reverse of what accurate domain retrieval requires. The model is not wrong in any general sense. It has simply learned a similarity structure suited to a different problem.

A third dynamic compounds both of these failures once a system is live: the asymmetry between how a query is phrased and how a target document is written. In domain settings, queries tend to be short and colloquial, a clinical shorthand, a legal lookup term, a phrase lifted from an internal support ticket, while the documents being retrieved are long, formal, and structured differently. Public benchmarks, built largely on synthetic or symmetric query-document pairs, rarely expose that mismatch, so a model can score well on a benchmark's clean pairs while struggling against the ragged, asymmetric queries a production system actually receives.

What empirical results from domain benchmarks show

These are not theoretical concerns. Domain-specific evaluations across legal, financial, engineering, and code corpora have repeatedly produced rankings that contradict MTEB, and the size of the gap is large enough to change which model a team should actually deploy.

In code retrieval, one internal evaluation found that GTE-Qwen2-7B, the top-ranked model on MTEB at the time, reached a Recall@5 of only 0.61 on a real code corpus. Voyage-3, ranked third on MTEB, reached 0.74 on the same task, and a fine-tuned version of BGE-en-v2.0, a model that did not appear in MTEB's rankings at all, reached 0.87. The MTEB rank-1 model finished last among the three.

Financial text produces the same pattern. Testing embedding models against SEC filings, one evaluation from Tigerdata found that Voyage finance-2 achieved higher overall accuracy than OpenAI's text-embedding-3-small, with the largest gap appearing on direct, fact-seeking financial queries, the exact query type a financial retrieval system needs to handle well. A more comprehensive test of this pattern comes from FinMTEB, a benchmark built by Tang and Yang and published at EMNLP 2025, covering 64 financial domain datasets across seven task types, including classification, clustering, retrieval, pair classification, reranking, summarization, and semantic textual similarity, drawn from annual reports, ESG disclosures, regulatory filings, and earnings call transcripts. Performance on general-purpose benchmarks showed limited correlation with performance on these financial domain tasks, and FinBERT, a finance-specific model, substantially outperformed general BERT on average FinMTEB score.

FinMTEB also produced a result that complicates any simple prescription to just pick a better dense embedding model: on financial semantic textual similarity, a plain Bag-of-Words model outperformed dense embeddings entirely. Financial text is often formulaic and disclaimer-heavy, repeating boilerplate language across filings and reports, and that repetition may not be the terrain where dense, learned representations hold their usual edge over a much simpler lexical approach. A domain's structure, not just its vocabulary, can determine which class of model is even the right tool.

Engineering documentation shows the value of direct adaptation. An ACM study from 2025, working with tens of thousands of engineering documents, found that fine-tuning for domain-adapted retrieval produced a substantial increase in Recall@10 on domain-specific retrieval tasks, while the fine-tuned model still generalized well on MTEB. Adapting to a domain did not come at the cost of general competence in that case.

Across these domains, the magnitude of the gap between the best- and worst-performing models on domain-specific benchmarks can reach double-digit percentage points in retrieval accuracy. That gap separates a retrieval system that reliably surfaces the right document from one that fails on a meaningful share of real queries.

Running an evaluation on your own corpus before choosing a model

The rankings above are a case for building a small evaluation harness against an actual corpus before any model gets selected or any fine-tuning budget gets spent.

The scale required is modest. A few hundred representative query-document pairs are enough to expose ranking inversions between models. Most teams can achieve this inside a single sprint.

The metrics used matter as much as the data. Retrieval-specific metrics, Recall@5, Recall@10, and NDCG@10, measure whether the correct document appears near the top of a ranked list; similarity scores do not predict that. A cosine similarity score between a query and a document says nothing about where that document lands relative to every other candidate in the corpus. Ranking metrics answer the question a production retrieval system actually needs answered.

Query selection shapes whether the evaluation reflects reality. Queries should come from actual user logs or realistic domain questions, not synthetic paraphrases generated for convenience, because synthetic queries tend to mirror the document's own phrasing and hide the query-to-document asymmetry that breaks retrieval in production.

Pipeline consistency is a separate discipline from model selection but just as consequential. Mixing embedding models between indexing time and query time is one of the most common and costly mistakes teams make, the MLflow embeddings guide notes, because a vector space is only internally consistent if the same model produced every vector in it. An evaluation needs to test the full pipeline end to end, index, query embedding, and retrieval logic together, rather than the embedding model in isolation on a held-out test set.

Dimensionality belongs in this evaluation as a cost variable, not an afterthought addressed after deployment. Embedding dimension multiplies directly into storage, memory footprint, and approximate nearest-neighbor search latency, and one cost analysis singles this out as a primary driver of retrieval infrastructure cost. A single 3,072-dimension vector stored as 32-bit floats consumes roughly 12KB, and that figure compounds quickly once a corpus reaches millions of documents. A model that scores marginally higher on an evaluation set but carries twice the dimensionality may not be the more practical choice once storage and latency are priced in.

Candidate models by domain and deployment constraint

Once an evaluation harness exists, the choice of which models to run through it should be driven by domain fit and deployment constraint, not by leaderboard rank. As of mid-2026, the landscape offers distinct, well-defined options across three tiers.

For teams that need the broadest general baseline without standing up their own infrastructure, several managed API models cover different constraint profiles. Gemini Embedding 001 held the top position on the MTEB Multilingual leaderboard as of its mid-2025 general availability, with its context window acting as the binding limit for long-form legal or research documents, and pricing available through Vertex AI. Voyage-3-large scores approximately 67+ on the overall MTEB metric, handles long documents at four times the capacity of OpenAI's comparable model, and outperforms text-embedding-3-large by 10.58% at matched dimensions across a broad set of datasets spanning law, finance, code, and multilingual content, premai.io found. Cohere's embed-v4 offers a very long context window and is available for VPC and on-premises deployment, a relevant option for regulated industries where data cannot leave controlled infrastructure, with pricing available directly from Cohere.

For teams with data sovereignty requirements, cost constraints, or plans to fine-tune, open-source self-hosted models form a second tier. NV-Embed-v2 from NVIDIA also supports a long context window, is released under a CC-BY-NC-4.0 license, and underlies the Cisco and NVIDIA NeMo Retriever fine-tuning pipeline, which is built on a large-parameter NV-EmbedQA variant.

For corpora that are predominantly legal, financial, or code-based, a third tier of domain-pre-adapted hosted models already exists and should be evaluated before any general-purpose fallback. Voyage AI publishes voyage-law-2, voyage-finance-2, voyage-code-3, and voyage-multilingual-2, each of which outperforms Voyage's own general-purpose model on corpora matching its specialty, with voyage-code-3 scoring 71.2 on code retrieval specifically. A related but distinct category is domain-native models trained from the ground up on domain text rather than adapted afterward: BioMedLM for biomedicine, SaulLM for legal text, and BloombergGPT for finance represent that from-scratch strategy, while specialized embedding variants such as BioWordVec, BioSentVec, and FinBERT follow the alternative path of fine-tuning a general architecture on domain data. Both strategies appear in the empirical results already discussed, FinBERT's strength on FinMTEB is a fine-tuning result, and both belong in a candidate set for a domain where they exist.

Choosing between a domain-specialized or pre-adapted model and fine-tuning

The evaluation results from a team's own corpus should determine which of three paths to take, and the threshold for each path is visible directly in the numbers the evaluation produces.

If a pre-adapted model, such as voyage-law-2 for a legal corpus or voyage-finance-2 for financial filings, already clears the retrieval targets a team needs on Recall@10 and NDCG@10 against its own evaluation set, no further investment is justified. The SEC filings result showing Voyage finance-2 ahead of a general-purpose OpenAI model demonstrates that a pre-adapted model can close most of the domain gap without any custom training, and the same logic applies to voyage-code-3's strength on code retrieval specifically. Reaching for fine-tuning when an existing domain-adapted model already performs well spends engineering time on a problem that is already solved.

Fine-tuning becomes warranted when the evaluation set exposes a gap that no available pre-adapted model closes, and the corpus is large and distinctive enough to justify the cost of training. The code retrieval case makes the ceiling on this approach clear: a fine-tuned BGE-en-v2.0 reached 0.87 Recall@5 against a real code corpus, ahead of both the MTEB rank-1 model at 0.61 and voyage-3 at 0.74. The ACM 2025 study on engineering documentation shows the same pattern, with fine-tuning producing a substantial jump in Recall@10 on domain-specific retrieval while the resulting model still generalized well on MTEB, meaning fine-tuning does not have to trade general competence for domain accuracy when done against a well-constructed dataset.

On financial semantic textual similarity, a simple Bag-of-Words model outperformed dense embeddings, suggesting that formulaic, disclaimer-heavy financial text may not be the terrain where dense models hold their usual advantage. The evaluation built on a team's own corpus is what reveals which of these three outcomes applies, whether a pre-adapted model already suffices, whether fine-tuning is worth the investment, or whether the corpus calls for a fundamentally different retrieval approach than dense embeddings altogether.