--- title: "Best Open-Source Embedding Models for RAG in 2026" published: 2024-12-19T09:00:44.000-05:00 updated: 2026-10-23T08:16:01.000-04:00 excerpt: "Eight open-source embedding models across three deployment tiers, with MTEB scores, VRAM needs, context limits, and pgvector storage sizing." tags: AI, RAG, Open Source authors: Hervé Ishimwe --- > **TimescaleDB is now Tiger Data.** _Updated October 2026_ - Picking an embedding model felt easy back in late 2024. Three names showed up in every comparison post: nomic-embed-text, mxbai-embed-large, and BGE-M3. You picked one, wired it into LangChain, and shipped your RAG pipeline. That playbook doesn't hold up in 2026. The [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard), the main benchmark researchers use to rank embedding models, has shifted a lot since then. The Qwen3-Embedding family from Alibaba took the top spot when it launched in June 2025. Its 8B variant [scores 70.58 on the MTEB multilingual benchmark](https://arxiv.org/html/2506.05176v1#:~:text=Qwen3-8B-Embedding%20attains%20a%20score%20of%2070.58), compared to [BGE-M3's 59.56](https://arxiv.org/html/2506.05176v1#:~:text=BGE-M3) on the same benchmark.  That roughly 11-point gap can translate into retrieval quality your users will notice. Two newer ideas have also entered the embedding world: Matryoshka Representation Learning, which lets you shrink an embedding without retraining the model, and Mixture-of-Experts, where only part of the model activates for a given query. Both change how much you pay in storage and compute, so it's worth understanding them before you lock in an index schema. This post walks through eight current models across three deployment tiers: GPU-required quality leaders, production workhorses, and CPU-friendly options for local or edge deployment. Every model here is free to use, and all but EmbeddingGemma ship under an OSI-approved open-source license. If you're weighing proprietary embedding APIs instead, check our [OpenAI embedding model comparison](https://www.tigerdata.com/blog/which-openai-embedding-model-is-best) and our broader guide on [evaluating open-source vs. OpenAI embeddings for RAG](https://www.tigerdata.com/blog/open-source-vs-openai-embeddings-for-rag). One disclosure worth stating. Tiger Data builds pgvectorscale, pg\_textsearch, and the new pg\_vectorsearch extension, and runs pgvectorscale and pg\_textsearch alongside pgvector on Tiger Cloud. We approach this from a Postgres-native RAG angle, and that shapes the deployment examples later in this post. The model comparisons and benchmarks below come from published third-party data, not our own internal testing, except where we cite Tiger Data's own pg\_vectorsearch benchmark.  ## Why embedding model choice has compounding consequences Picking an embedding model shapes three things: retrieval quality, operational cost, and pipeline architecture. Because these effects stack over the life of your RAG build, this is one of the highest-impact decisions you'll make. **Retrieval quality comes first.** An nDCG@10 score of 56.4 versus 51.8, Qwen3-Embedding-8B against BGE-M3 in Contra Collective's June 2026 BEIR test, sounds like a lab metric, but it plays out in real terms. In a corpus of technical support documents, a gap like that can decide whether the right chunk lands in position 1 or slips to position 4 or 5. When your LLM generates an answer from context sitting in position 4, quality drops in ways that are hard to trace, since the pipeline still technically works. **Operational cost comes second.** According to a [June 2026 benchmark by Contra Collective on Apple Silicon](https://contracollective.com/blog/local-embeddings-apple-silicon-nomic-bge-qwen3-m5-max-2026) (M5 Max, MLX 0.21, 512-token passages), Qwen3-Embedding-8B processes roughly 580 passages per second at batch size 32, while BGE-M3 handles about 4,800 passages per second at batch size 64. For a 10-million-passage corpus, that translates to about 4.8 hours of ingestion time versus about 35 minutes. At scale, that's a real infrastructure constraint. **Architecture comes third.** BGE-M3 produces dense vectors, sparse vectors (BM25-style term weights), and multi-vector representations from a single inference call. Most other models only produce dense vectors. If your RAG pipeline needs [hybrid search](https://www.tigerdata.com/blog/postgresql-hybrid-search-using-pgvector-and-cohere), your model choice decides whether one model can produce both the dense and sparse sides, or whether you pair dense embeddings with keyword search such as BM25. The word _"best"_ in this post's title is deliberately context-dependent. The tier framework below explains why. ## The 2026 open-source embedding model field The 2026 field has grown well beyond the three models that dominated late-2024 comparisons. It now includes new quality leaders, a novel MoE architecture, and stronger options for edge deployment. In late 2024, the field was narrow. nomic-embed-text was the Ollama default, mxbai-embed-large was the go-to English-only CPU workhorse, and BGE-M3 handled multilingual and hybrid search use cases. All three remain valid choices in 2026, and developers still actively compare them (queries like "mxbai-embed-large vs nomic-embed-text" are still common), so the guidance below still holds where it applied before. What's changed is the quality ceiling. The Qwen3-Embedding family (Apache 2.0, released by Alibaba's Qwen team in 2025) topped the [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard) at release, with the 8B variant scoring 70.58 on the multilingual benchmark.  Nomic Embed Text v2 introduced the first Mixture-of-Experts architecture built for text embeddings, landing in roughly the same quality range as BGE-M3 while activating far fewer parameters per query. EmbeddingGemma, Google's 308M-parameter model, targets on-device and edge deployment, with embeddings generated in under 15ms for 256 input tokens on EdgeTPU hardware.  Three dimensions that barely registered in 2024 comparisons now drive real production decisions. Context window length shapes your [chunking strategy](https://www.tigerdata.com/blog/which-rag-chunking-and-formatting-strategy-is-best): mxbai-embed-large's 512-token ceiling is a real constraint for long-document chunking, while BGE-M3 and Qwen3 handle 8K to 32K tokens.  Matryoshka Representation Learning (MRL) allows you to store compressed embeddings without retraining the model, which changes the storage math significantly once you're indexing millions of documents. And BGE-M3's built-in hybrid search support removes the need for a separate sparse embedding model. ### How to read the MTEB leaderboard (and when not to trust it) MTEB (Massive Text Embedding Benchmark) is an open-source benchmark whose leaderboard is hosted on Hugging Face at [huggingface.co/spaces/mteb/leaderboard](http://huggingface.co/spaces/mteb/leaderboard). It groups tasks into separate benchmarks: the original English MTEB averages 56 datasets, while MTEB (Multilingual) averages 131 tasks covering retrieval, clustering, classification, pair classification, reranking, semantic textual similarity, and more. The 70.58 and 59.56 scores are averages across that 131-task multilingual suite.  On the MTEB Multilingual benchmark, Qwen3-Embedding-8B scores 70.58 and BGE-M3 scores 59.56. nomic-embed-text-v1.5 scores 62.28 on the separate MTEB English benchmark, its intended use case. Scores from the two benchmarks aren't directly comparable.  Those numbers are useful as a starting filter, not as a final verdict. MTEB is predominantly English unless you're looking at the multilingual leaderboard specifically, and a model's rank can shift sharply between the two. A model scoring 70 on the multilingual leaderboard may still underperform a lower-scoring model on your specific corpus, especially if your documents are technical, short-form, domain-specific, or multilingual. The roughly 11-point MTEB advantage Qwen3-Embedding-8B holds over BGE-M3 reflects aggregate performance across diverse public benchmarks. On a corpus of internal engineering runbooks in English, that difference could be smaller, larger, or reversed. The custom evaluation section below covers how to measure this on your own data. Use MTEB as a starting filter to narrow the field, not as the final word. ## The top open-source embedding models for RAG compared The table below covers all eight models across the three deployment tiers. | Model | Params | Dims | Context | License | MTEB Avg | Best for | | Qwen3-Embedding-8B | 8B | 4,096 (MRL) | 32K | Apache 2.0 | 70.58* | Max retrieval quality, GPU available | | Qwen3-Embedding-0.6B | 0.6B | 1,024 (MRL) | 32K | Apache 2.0 | 64.33* | GPU-constrained but want Qwen3 architecture | | BGE-M3 | 568M | 1,024 | 8,192 | MIT | 59.56* | Hybrid search, multilingual, long docs | | Nomic Embed Text v2 | 475M | 768-256 | 512 | Apache 2.0 | Not published | Multilingual, efficient MoE architecture | | mxbai-embed-large | 335M | 1,024 | 512 | Apache 2.0 | 64.68† | English-benchmark, CPU-friendly, widely tested | | nomic-embed-text v1.5 | 137M | 768 | 8,192 | Apache 2.0 | 62.28† | CPU default, Ollama ecosystem, sub-300MB | | EmbeddingGemma-300M | 308M | 768 | 2,048 | Gemma ToS | 61.15* | On-device, mobile, edge deployments | | E5-small / all-MiniLM | 22-33M | 384 | 512 / 256 | MIT / Apache 2.0 | ~48-59† | Rapid prototyping only | \*MTEB Multilingual benchmark. †MTEB English benchmark. Scores from the two benchmarks aren't directly comparable.  MTEB scores shift as models get re-evaluated and new ones enter the rankings. Check the [live leaderboard](https://huggingface.co/spaces/mteb/leaderboard) for current numbers before you commit to a model. The tier framing below is a compute environment classification. - Tier 1 needs significant GPU memory. - Tier 2 is GPU-optional at that parameter scale but runs better with one. - Tier 3 runs comfortably on CPU. Your tier depends on the hardware you have available. ### Tier 1: Quality ceiling models that require GPU These models deliver some of the highest retrieval quality available in open-source. They need a GPU for practical throughput and don't belong on a CPU-only deployment. #### Qwen3-Embedding-8B Qwen3-Embedding-8B is one of the strongest open-source embedding models available. It topped the MTEB multilingual leaderboard at release with a score of 70.58. Released by Alibaba's Qwen team in 2025 under Apache 2.0 (commercially permissive), it supports 100-plus languages and a 32K-token context window. Two architectural features set it apart from earlier-generation models. First, it's instruction-aware. You can prepend task-specific instructions to shape the embedding for your retrieval task, a different mechanism from the prefix injection pattern nomic-embed-text uses. Second, it implements Matryoshka Representation Learning (MRL). Its 4,096-dimension output vectors can be truncated to 256 or 512 dimensions with minimal retrieval quality loss, since training packs the most discriminative information into the earlier dimensions. For pgvector deployments, MRL changes the storage math significantly. Storing 1 million rows at 4,096 dimensions takes roughly 16.4 GB of base storage. At 256 dimensions, that same corpus drops to around 1.0 GB, about a 16x reduction, while keeping most of the retrieval quality. This makes Qwen3-Embedding-8B practical at scale in ways its raw dimension count wouldn't suggest. The practical constraint is GPU memory. Quantized to Q4 precision, Qwen3-Embedding-8B needs somewhere in the range of 5 to 8 GB of VRAM depending on the exact quantization scheme and context length, which puts it beyond a typical laptop GPU and into dedicated inference card or cloud GPU territory. Throughput on Apple M5 Max (GPU inference, batch size 32, 512 tokens) runs roughly 580 passages per second, according to a [June 2026 Contra Collective benchmark](https://contracollective.com/blog/local-embeddings-apple-silicon-nomic-bge-qwen3-m5-max-2026). For a 10-million-document corpus, plan for roughly 4.8 hours of initial ingestion at that throughput. Choose this model when retrieval quality is the primary constraint, you have a GPU with at least 8 GB VRAM available, and your corpus is multilingual or needs the highest-quality cross-lingual retrieval. #### Qwen3-Embedding-0.6B Qwen3-Embedding-0.6B is the hidden value play in the 2026 model field. At 0.6 billion parameters, it keeps the core Qwen3 architectural advantages (MRL support, 32K context window, 1,024 dimensions, Apache 2.0 license) in a model under 1 GB. Its MTEB multilingual score of 64.33 puts it ahead of BGE-M3 (59.56) on the same benchmark, all while running at a fraction of the 8B variant's compute cost. The Qwen3 architecture's advantages carry down to this scale, keeping the 32K context window, MRL truncation, and multilingual support intact. This is the right model for developers who want Qwen3's long-context handling and MRL storage flexibility but can't run the 8B variant due to GPU memory limits. It also makes a solid candidate for local development environments where you want to prototype with a Qwen3 model before committing to the 8B version in production. ### Tier 2: Production workhorses Tier 2 models are the 2026 defaults for most production RAG pipelines, where GPU is available but Tier 1's memory needs are too high, or where a specific capability like hybrid search or multilingual retrieval matters more than raw MTEB score. #### BGE-M3, the multilingual hybrid search default BGE-M3 comes from BAAI (Beijing Academy of AI) under the MIT license. At roughly 568M parameters and 1,024 dimensions, it runs on consumer GPU hardware and reaches practical throughput of about 4,800 passages per second on M5 Max at batch size 64, according to a [June 2026 Contra Collective benchmark](https://contracollective.com/blog/local-embeddings-apple-silicon-nomic-bge-qwen3-m5-max-2026), using roughly 1.4 GB of memory. BGE-M3's most important capability is producing three types of vectors from a single inference call. Dense vectors are the standard cosine similarity representation used for semantic search. Sparse vectors apply BM25-style term weights, catching exact technical vocabulary that pure semantic search can miss. Multi-vector representations follow the ColBERT late-interaction approach, adding re-ranking quality through token-level similarity scoring. For teams building [hybrid search in PostgreSQL](https://www.tigerdata.com/blog/postgresql-hybrid-search-using-pgvector-and-cohere), BGE-M3 means one model handles what would otherwise need two separate models: it produces dense vectors for a pgvector index and sparse weights you can store in pgvector's sparsevec type, each with its own index. That's a real architectural simplification. Served through Ollama, BGE-M3 returns dense vectors only, so the sparse output needs a runtime that exposes it, such as BAAI's FlagEmbedding library. Its 8,192-token context window handles full-page documents without truncation. mxbai-embed-large's 512-token ceiling, by contrast, means a 400-word technical document chunk is already near capacity before you add any surrounding context. BGE-M3 doesn't run into that limit. BGE-M3 also covers 100-plus languages, making it the production default for any multilingual corpus. One limitation worth naming: Qwen3-Embedding-8B holds a roughly 11-point MTEB multilingual advantage over BGE-M3 (70.58 versus 59.56). If retrieval quality is your only constraint and you have GPU headroom, Tier 1 is worth the extra infrastructure cost. BGE-M3's advantage is breadth: hybrid search, long context, and multilingual support in one model, at a fraction of the 8B Qwen3 variant's GPU memory footprint. #### Nomic Embed Text v2, the MoE lightweight option Nomic Embed Text v2 is the first embedding model to apply a Mixture-of-Experts (MoE) architecture to the embedding task. Total parameter count is 475M, but only 305M are active per inference call. That active-parameter efficiency means compute cost lands closer to a 300M-parameter model, while overall model capacity sits closer to a 500M-parameter model. In practice, that makes each embedding cheaper to compute than with a dense model of the same total size. Benchmark it on your own hardware before you plan ingest capacity around it.  It supports around 100 languages and carries an Apache 2.0 license. Its context window tops out at 512 tokens, so it's better suited to shorter passages than to full-page documents; BGE-M3 and Qwen3-Embedding handle much longer chunks. Training on 1.6 billion text pairs gives it broad coverage despite its efficient size. Where it falls short relative to BGE-M3: it doesn't produce sparse vectors, so you don't get native hybrid search from a single model. If your pipeline needs keyword matching alongside semantic search, BGE-M3 remains the cleaner option. For cost-sensitive deployments that need multilingual support and lower compute per embedding but can live with dense-only retrieval and shorter chunks, Nomic Embed Text v2 is the 2026 production workhorse. ### Tier 3: CPU-friendly and fast Tier 3 is a legitimate production tier. nomic-embed-text v1.5 has racked up more than 90 million Ollama downloads. mxbai-embed-large is among the most thoroughly community-tested embedding models available. Both are real production choices for the right deployment context. #### mxbai-embed-large mxbai-embed-large runs at 335M parameters, 1,024 dimensions, under an Apache 2.0 license. It's CPU-compatible, widely supported across RAG frameworks, and backed by two years of community documentation from production deployment. The context limit is worth taking seriously. At 512 tokens maximum, mxbai-embed-large processes roughly 350 to 400 words per chunk before quality degrades. Most developers building document RAG pipelines don't chunk that aggressively. If your chunking strategy produces 500-token chunks, common for paragraph-level splitting, you're already past mxbai-embed-large's window. The model doesn't error on longer inputs. It silently truncates, so retrieval quality drops without any warning in your logs. BGE-M3 and nomic-embed-text v1.5 both handle 8,192 tokens. In 2024, mxbai-embed-large's context limit was a known trade-off. In 2026, it's a reason to default to nomic-embed-text v1.5 or BGE-M3 for most new projects. Where mxbai-embed-large still makes sense: English-only corpora with chunks reliably sized to 300 words or fewer, CPU-only deployments, and teams that have already built tooling around it with a chunking pipeline tuned to its constraints. #### nomic-embed-text v1.5 nomic-embed-text v1.5 runs at 137M parameters, 768 dimensions, under Apache 2.0, with an 8,192-token context window. Its default Ollama (fp16) build weighs around 274 MB, making it one of the smallest practical models with a full 8K context window available today (the full-precision FP32 weights run closer to 550 MB, if you need those instead). With more than 90 million Ollama downloads, it's the de facto default for Ollama-based local RAG deployments.  It scores 62.28 on MTEB English, a competitive result for a model with less than one-quarter of BGE-M3's parameters and a fraction of its memory footprint. BGE-M3's 59.56 comes from the multilingual benchmark, so the two scores don't compare directly. If your corpus is English-only, test both on your own data.  One configuration requirement trips up a lot of teams. nomic-embed-text v1.5 is trained with task-specific prefixes, and omitting them measurably reduces retrieval quality. The prefix injection section below covers this in detail. Your document chunks need the search\_document: prefix and your queries need the search\_query: prefix. Many integrations, such as LangChain's Ollama integration, don't inject these automatically.  Choose nomic-embed-text v1.5 for CPU-only deployment, Ollama-based local RAG, or any environment where a small model footprint is a meaningful constraint. Its 8K context window handles document variety well without requiring the careful chunking mxbai-embed-large demands. #### EmbeddingGemma-300M EmbeddingGemma-300M runs at 308M parameters, 768 dimensions, 2,048-token context, with under 15ms inference latency for 256 input tokens on EdgeTPU, per Google. It's built for EdgeTPU support, mobile deployment, and on-device inference where model footprint and latency are the primary constraints. One licensing note. EmbeddingGemma uses the Gemma Terms of Service, which differs from Apache 2.0 or MIT. The Gemma ToS is generally permissive for most commercial applications, but if your deployment requires a fully OSI-approved license, verify Gemma ToS compatibility with your legal team before committing. Choose EmbeddingGemma-300M for edge, mobile, or embedded scenarios where low on-device latency is the primary requirement rather than maximum retrieval quality. For teams starting from scratch with no clear requirement driving the decision, nomic-embed-text v1.5 is the right default for CPU and local deployment. BGE-M3 is the right default if you have a GPU and aren't sure what your corpus demands yet. Both have the 8K context window that forgives early chunking strategy mistakes. ## Deployment considerations for VRAM, throughput, and context windows Model selection connects directly to infrastructure planning. The table below gives the practical picture across the model tiers. | Model | VRAM / RAM | Throughput (pass/sec) | Context window | Ollama available | | Qwen3-Embedding-8B | ~5-8 GB (Q4 GPU) | ~580 | 32K | Yes (ollama pull qwen3-embedding:8b) | | Qwen3-Embedding-0.6B | ~1 GB | Higher than 8B | 32K | Yes (ollama pull qwen3-embedding:0.6b) | | BGE-M3 | 1.4 GB | ~4,800 | 8,192 | Yes (ollama pull bge-m3) | | Nomic Embed Text v2 | ~1 GB | Not benchmarked | 512 | Yes (ollama pull nomic-embed-text-v2-moe) | | mxbai-embed-large | ~700 MB CPU | High (CPU) | 512 | Yes (ollama pull mxbai-embed-large) | | nomic-embed-text v1.5 | ~300 MB CPU | High (CPU) | 8,192 | Yes (ollama pull nomic-embed-text) | | EmbeddingGemma-300M | ~700 MB | <15ms latency (256 tokens, EdgeTPU) | 2,048 | Yes (ollama pull embeddinggemma) | Throughput numbers for BGE-M3 and Qwen3-Embedding-8B come from a [June 2026 Contra Collective benchmark](https://contracollective.com/blog/local-embeddings-apple-silicon-nomic-bge-qwen3-m5-max-2026) (M5 Max, MLX, 512 tokens), at batch size 64 for BGE-M3 and 32 for Qwen3-Embedding-8B. CPU-tier model throughput varies by hardware and isn't directly comparable to GPU-accelerated numbers. The context window column drives your [chunking strategy](https://www.tigerdata.com/blog/which-rag-chunking-and-formatting-strategy-is-best). For models with 512-token context (mxbai-embed-large, Nomic Embed Text v2), keep chunks at 300 words or fewer and test at that boundary. For models with 8K+ context (BGE-M3, nomic-embed-text v1.5, Qwen3-Embedding), paragraph-level or section-level chunking works without careful token counting. ### The prefix injection problem and how to fix it [nomic-embed-text v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5) and [nomic-embed-text-v2](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe) are trained with task-specific prefixes as part of their representation learning. For retrieval tasks, document chunks should be prefixed with search\_document: and queries should be prefixed with search\_query:. These prefixes are part of the training signal the models were optimized against, so omitting them changes what the model produces. The failure mode is silent. Many integrations, including LangChain's Ollama integration, don't inject these prefixes automatically. When you call nomic-embed-text through one of them without prefix configuration, retrieval quality drops measurably. The pipeline runs, the embeddings generate, and the vector search returns results that are noticeably worse than what the model card benchmarks show, with no error to debug. Fix it by manually prepending the appropriate prefix before each call: ```Python import ollama # Missing prefix -- common default in tutorials embedding = ollama.embeddings( model="nomic-embed-text", prompt=chunk_text ) # Correct: task-aware prefix on document chunks embedding = ollama.embeddings( model="nomic-embed-text", prompt=f"search_document: {chunk_text}" ) # Correct: different prefix for query embeddings query_embedding = ollama.embeddings( model="nomic-embed-text", prompt=f"search_query: {user_query}" ) ``` [mxbai-embed-large](https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1) uses a different mechanism, worth calling out separately since it's easy to conflate with nomic's scheme. Only queries need a prefix, and document chunks are embedded as-is with no prefix at all: ```Python # mxbai-embed-large: documents need no prefix doc_embedding = ollama.embeddings( model="mxbai-embed-large", prompt=chunk_text ) # mxbai-embed-large: queries need this specific prefix query_embedding = ollama.embeddings( model="mxbai-embed-large", prompt=f"Represent this sentence for searching relevant passages: {user_query}" ) ``` For Qwen3-Embedding models, the mechanism differs again. Qwen3 uses an instruction parameter for task-aware embedding rather than raw string prefixes. Check the Qwen3-Embedding-8B model card on Hugging Face for the current API signature before building your ingestion pipeline. Of the Tier 2 options, only BGE-M3 skips task-specific prefixes entirely, which removes one configuration step. Elsewhere in this post, all-MiniLM also needs no prefix, while EmbeddingGemma and E5 use their own prompts or prefixes.  ## How to evaluate on your own data (beyond MTEB) MTEB tells you how a model performs across a diverse set of public benchmarks, but your corpus probably isn't a diverse set of public benchmarks. A 3-nDCG-point MTEB advantage can reverse on domain-specific data, especially for technical corpora (API documentation, engineering runbooks, medical records) where specialized vocabulary and abbreviations behave differently from general text. Start by sampling 500 to 2,000 representative chunks from your actual corpus. Use a random sample of the documents your production system will actually retrieve from, including edge cases like short documents, long documents, and documents with heavy technical terminology. Generate evaluation questions against those chunks. You can write 50 to 100 by hand and use an LLM to generate the rest. The question set should include short factual questions, longer contextual questions, and questions that imply a concept without naming it directly, what practitioners call "implied" or "indirect" queries. That last category is where model quality differences tend to surface most clearly. Run top-K retrieval with each candidate model and score the results. For binary scoring, check whether the source chunk appears in the top-5 or top-10 results. For graded scoring, compute nDCG@10, which accounts for rank position. nDCG weights position heavily, so a chunk at position 1 contributes more than the same chunk at position 8. The ranking itself comes from your similarity metric, typically [cosine similarity](https://www.tigerdata.com/learn/understanding-cosine-similarity).  Compare performance across question categories separately. A model that outperforms on short factual questions may underperform on long contextual questions, and that difference matters more than the overall average for picking a model that fits your specific query distribution. For teams building on PostgreSQL, the evaluation loop fits in the database you already run. Embed the same sample chunks with each candidate model into its own table, run the same question set against each, and compare the scores in SQL. Adding another candidate means one more table, not a new pipeline.  This evaluation methodology won't match the rigor of a published academic benchmark, but it doesn't need to. Its job is to tell you which model performs better on your specific retrieval task, the only measurement that directly reflects your production retrieval quality. ## Storing and querying embeddings in PostgreSQL with pgvector Once you've chosen a model, the next problem is storage and retrieval infrastructure. For teams already on PostgreSQL, [pgvector as a vector database](https://www.tigerdata.com/blog/postgresql-as-a-vector-database-using-pgvector) is the natural choice, and the model you pick has direct consequences for storage sizing and index configuration. The base storage formula is straightforward: each vector takes 4 bytes per dimension (float32) plus 8 bytes, so dimensions × 4 bytes × row count gives base vector storage. An HNSW index stores its own copy of every vector plus links between them, so budget at least that much again for the index. For 1 million documents at 1,024 dimensions, that's about 4.1 GB of vectors and at least another 4.1 GB for the HNSW index. Check real sizes with pg\_relation\_size once your data is loaded.  pgvector’s HNSW index caps standard vector types at 2,000 dimensions and halfvec types at 4,000 dimensions. For a 4,096-dimensional model like Qwen3-Embedding-8B, you can truncate the vectors, index a truncated subvector or a binary-quantized copy (pgvector indexes bit vectors up to 64,000 dimensions), or use pgvectorscale’s StreamingDiskANN index, which supports vectors up to 16,000 dimensions. MRL truncation to 256 or 512 dimensions avoids this constraint entirely while reducing storage dramatically. Tiger Data's newer [pg\_vectorsearch](https://www.tigerdata.com/blog/announcing-pg_vectorsearch-scalable-open-source-vector-search-postgresql) extension takes a different approach to index size. Its PRISM index stores compressed vectors in partitions instead of a graph. In Tiger Data's 1-million-vector benchmark, the index was about 7x smaller than HNSW's and built in about 3 minutes instead of almost a day on the same 4 GB machine. pg\_vectorsearch is in alpha, requires PostgreSQL 18, and isn't yet recommended for production, but it works on existing pgvector columns and is worth testing on your own data.  ### pgvector storage sizing by model These estimates use float32 (4 bytes per dimension). pgvector also supports halfvec (2 bytes per dimension), which cuts all of these figures roughly in half with minimal retrieval quality impact, an option worth evaluating for large deployments. | Model | Dims | 1M rows (no index) | 1M rows + HNSW index (at least) | | Qwen3-Embedding-8B (full) | 4,096 | ~16.4 GB | No HNSW index (above the 2,000-dim cap) | | Qwen3-Embedding-8B (MRL 256 dims) | 256 | ~1.0 GB | ~2.1 GB | | BGE-M3 | 1,024 | ~4.1 GB | ~8.2 GB | | Nomic Embed Text v2 | 768 | ~3.1 GB | ~6.2 GB | | mxbai-embed-large | 1,024 | ~4.1 GB | ~8.2 GB | | nomic-embed-text v1.5 | 768 | ~3.1 GB | ~6.2 GB | | EmbeddingGemma-300M | 768 | ~3.1 GB | ~6.2 GB | The MRL story is the most important insight in this table. At 4,096 dimensions, Qwen3-Embedding-8B's vectors take about 16.4 GB per million rows and exceed pgvector's HNSW dimension cap, so they can't take an HNSW index without truncation or a different index type.  With MRL truncation to 256 dimensions, the same model's embeddings fit in roughly 1.0 GB of vectors (about 2 GB or more with an HNSW index), a 16x reduction in vector storage, while retrieval quality stays competitive with BGE-M3 on most corpora. That trade-off is what makes Qwen3-Embedding-8B a realistic production option. For complete setup and indexing guidance, see the [pgvector setup and indexing guide](https://www.tigerdata.com/learn/postgresql-extensions-pgvector). If you're uncertain about the distinction between vector stores and databases, the [vector store vs. vector database](https://www.tigerdata.com/learn/vector-store-vs-vector-database) explainer covers the relevant architecture decisions. ### Switching models without downtime  Different embedding models produce incompatible vector spaces, so switching from mxbai-embed-large to BGE-M3 requires re-embedding your entire corpus. You can't convert existing mxbai embeddings into BGE-M3 embeddings. The operational question is how disruptive that re-embedding process needs to be. It doesn't have to be disruptive. Build the new model's embeddings in their own table next to the existing ones, index that table, and switch your queries over only when it's ready:  ```SQL -- Existing mxbai-embed-large embeddings stay live in -- documents_embedding_mxbai while the new table fills -- 1. Create a table for the new model's embeddings CREATE TABLE documents_embedding_bgem3 ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, document_id bigint NOT NULL REFERENCES documents (id), chunk text NOT NULL, embedding vector(1024) NOT NULL ); ``` ```Python # 2. Backfill it from your application or a batch job import numpy as np import ollama import psycopg from pgvector.psycopg import register_vector with psycopg.connect(DATABASE_URL) as conn: register_vector(conn) docs = conn.execute("SELECT id, content FROM documents").fetchall() for doc_id, content in docs: chunks = chunk_text(content) # your chunker result = ollama.embed(model="bge-m3", input=chunks) conn.cursor().executemany( "INSERT INTO documents_embedding_bgem3" " (document_id, chunk, embedding) VALUES (%s, %s, %s)", [(doc_id, c, np.array(e)) for c, e in zip(chunks, result["embeddings"])], ) ``` ```SQL -- 3. Index it once the backfill finishes CREATE INDEX CONCURRENTLY ON documents_embedding_bgem3 USING hnsw (embedding vector_cosine_ops); -- Or with pgvectorscale's StreamingDiskANN: -- CREATE INDEX ON documents_embedding_bgem3 -- USING diskann (embedding vector_cosine_ops); -- Or, to test pg_vectorsearch (alpha, PostgreSQL 18+): -- CREATE INDEX ON documents_embedding_bgem3 -- USING prism (embedding vec32_cosine_ops); -- 4. Point queries at the new table SELECT document_id, chunk FROM documents_embedding_bgem3 ORDER BY embedding <=> $1 -- query embedded with BGE-M3 LIMIT 10; ``` The backfill writes to a separate table, so your existing embeddings stay available for retrieval the whole time. Once you've evaluated the new model and are satisfied with it, switch your application's queries to the new table and drop the old one when you're ready. From then on, embed new or changed documents as your application writes them, or from a queue worker. [Tiger Cloud](https://www.tigerdata.com/cloud) runs pgvector and pgvectorscale as a managed database, so the embedding code is the only part you run yourself.  ## Choosing the right model for your RAG pipeline Model selection has compounding consequences across storage sizing, index configuration, chunking constraints, and throughput requirements. The right choice depends on your compute environment, corpus characteristics, and operational constraints, not on which model a tutorial used in 2024. For most teams starting fresh, nomic-embed-text v1.5 is the CPU default and BGE-M3 is the GPU default. Both have an 8K context window that forgives early chunking mistakes. When retrieval quality becomes the binding constraint and a GPU is available, Qwen3-Embedding-8B with MRL truncation to 256 or 512 dimensions sits at or near the current quality ceiling at a manageable storage cost. If you're building on PostgreSQL, keep each model's embeddings in their own table, so evaluating or switching models never takes your current index offline. Tiger Cloud provides the managed database for pgvector, pgvectorscale, and pg\_textsearch, so dense, sparse, and keyword search all run next to your application data. To test where Postgres vector search is heading, try Tiger Data's new [pg\_vectorsearch](https://www.tigerdata.com/blog/announcing-pg_vectorsearch-scalable-open-source-vector-search-postgresql) extension, now in alpha. The [pgvector setup and indexing guide](https://www.tigerdata.com/learn/postgresql-extensions-pgvector) covers the setup details.  * * * ## Frequently asked questions **What is the best free open-source embedding model for RAG in 2026?** Qwen3-Embedding-8B topped the MTEB multilingual leaderboard at release with a score of 70.58 (Apache 2.0, free) and remains one of the strongest open-source options for RAG. For CPU-only or Ollama-based local RAG, nomic-embed-text v1.5 is the most widely deployed option, with more than 90 million Ollama downloads, Apache 2.0 license, and sub-300MB footprint. GPU availability points to Qwen3 or BGE-M3, while CPU-only constraints point to nomic-embed-text v1.5. **Is BGE-M3 better than nomic-embed-text for RAG?** For multilingual and hybrid search use cases, BGE-M3 may offer stronger retrieval than nomic-embed-text v1.5, primarily due to its 8,192-token context window, sparse retrieval mode for keyword matching, and 100+ language support. For English-only, CPU-constrained deployments, nomic-embed-text v1.5 is faster and smaller, and scores 62.28 on MTEB English. BGE-M3's 59.56 comes from the multilingual benchmark, so the two scores don't compare directly. BGE-M3 is the better default when you need hybrid search or multilingual coverage, and nomic-embed-text v1.5 is better when compute is the primary constraint. **What does Matryoshka mean for embedding models?** Matryoshka Representation Learning (MRL) allows a single embedding model to produce vectors that can be truncated to smaller dimensions with minimal retrieval quality loss. A Qwen3-Embedding-8B vector can be truncated from 4,096 to 256 dimensions, reducing pgvector storage by roughly 16x while retaining most retrieval quality. The name comes from Russian nesting dolls, where smaller, complete versions are nested inside the larger representation. **Can I run Qwen3-Embedding-8B locally?** Yes, with a GPU. Quantized to Q4, Qwen3-Embedding-8B needs somewhere in the range of 5 to 8 GB of VRAM, depending on the exact quantization scheme. Throughput on GPU inference runs roughly 580 passages per second, according to a Contra Collective benchmark (M5 Max, batch size 32, June 2026). CPU-only inference is impractical for production ingestion at that throughput. If your GPU can't fit the 8B model, Qwen3-Embedding-0.6B (roughly 1 GB model size, MRL-capable, 32K context, MTEB 64.33) is the more practical option. **How much storage does each embedding model need in pgvector?** Storage scales with vector dimensions: each vector takes 4 bytes per dimension plus 8 bytes. At 1 million rows, BGE-M3 (1,024 dims) needs about 4.1 GB of vectors and nomic-embed-text v1.5 (768 dims) about 3.1 GB. An HNSW index stores its own copy of every vector, so plan for at least double those figures with an index. Qwen3-Embedding-8B at full 4,096 dimensions needs about 16.4 GB and is above pgvector's 2,000-dimension HNSW limit; MRL truncation to 256 dims brings it to about 1.0 GB. Switching to halfvec (half-precision) roughly halves the vector storage. **What is MTEB and is it a reliable guide for RAG performance?** MTEB (Massive Text Embedding Benchmark) is an open-source benchmark with a leaderboard hosted on Hugging Face. Its English benchmark averages 56 datasets and its multilingual benchmark 131 tasks, across retrieval, classification, clustering, and semantic similarity. It's the best available proxy for comparing embedding models at a glance, but scores aggregate across diverse public corpora. A model with a higher MTEB score may underperform on your specific corpus if your documents are technical, short-form, non-English, or domain-specific. Use MTEB as a starting filter to narrow the field, then evaluate candidates on a representative sample of your own data. **What embedding models support hybrid (dense plus sparse) search?** BGE-M3 is the primary open-source option that produces both dense vectors and sparse vectors (BM25-style term weights) from a single inference call, so one model covers both sides of hybrid search. You still store and index the sparse output, for example with pgvector's sparsevec type. Most other open-source embedding models produce dense vectors only. For hybrid search with a different model, you'd typically combine dense embeddings in pgvector with BM25 keyword search, such as [pg\_textsearch](https://www.tigerdata.com/docs/learn/search/using-pg-textsearch) on Tiger Cloud, or PostgreSQL's built-in full-text search.  **What embedding model should I use with Ollama for local RAG?** nomic-embed-text v1.5 is the Ollama community default with more than 90 million downloads. It runs on CPU, has an 8,192-token context window, and is available via ollama pull nomic-embed-text. mxbai-embed-large is a strong alternative for English corpora with well-controlled chunk sizes (keep chunks under 300 words to stay within its 512-token limit). BGE-M3 is also available via Ollama for multilingual use cases, though Ollama returns its dense vectors only. **What is the prefix injection problem with nomic-embed-text and mxbai-embed-large?** nomic-embed-text is trained with task-specific prefixes, where document chunks should use the search\_document: prefix and queries should use the search\_query: prefix. mxbai-embed-large uses a different, single-sided scheme: only queries need a prefix (Represent this sentence for searching relevant passages:), and document chunks are embedded as-is. Many integrations, including LangChain's Ollama integration, don't inject either of these automatically. Omitting them can measurably reduce retrieval quality with no visible error. Fix it by manually prepending the correct prefix for your model before calling the embedding API, as shown in the code examples above. **How do I evaluate an embedding model on my own data rather than MTEB?** Sample 500 to 2,000 representative chunks from your corpus. Generate evaluation questions manually or with an LLM, covering short questions, long contextual questions, and implied questions. Run top-K retrieval with each candidate model and score results using binary scoring (source chunk in top-K) or graded scoring (nDCG@10). Compare across question types separately. This evaluation reveals domain-specific behavior that aggregate MTEB scores miss, and it's the only measurement that directly reflects your production retrieval quality. **Can I switch embedding models in pgvector without re-embedding everything?** No. Different embedding models produce incompatible vector spaces, so switching models requires re-embedding your corpus. You can avoid downtime by building the new embeddings in a separate table, indexing it, and switching queries over once you've evaluated the new model. Your existing embeddings stay live and queryable the whole time, and you drop the old table when you're done.