Overview
BGE-M3 is a multilingual text embedding model from BAAI, introduced for its versatility in what its authors call Multi-Functionality, Multi-Linguality, and Multi-Granularity. It supports more than 100 working languages and processes inputs of different granularities, from short sentences to documents of up to 8192 tokens, producing 1024-dimensional embedding vectors. Unlike earlier BGE models, BGE-M3 no longer requires adding instruction prefixes to queries.
This template serves BGE-M3's dense embeddings with vLLM through an OpenAI-compatible embeddings API. The model also carries sparse (lexical weight) and ColBERT-style multi-vector heads from its unified fine-tuning; those retrieval modes use a different serving path and are not exposed by this template.
Key Features
- Multilingual retrieval across more than 100 languages with a single model
- Long inputs: sequences up to 8192 tokens, suited to whole-document embedding
- 1024-dimensional dense embeddings for semantic search and similarity
- Trained for all three common retrieval functionalities (dense, sparse, and multi-vector), with dense retrieval served here
- No query-side instruction prefixes needed, simplifying drop-in use in existing pipelines
Architecture
BGE-M3 builds on the XLM-RoBERTa architecture, extended to an 8192-token maximum length via the RetroMAE pretraining recipe (BAAI/bge-m3-retromae), then trained with contrastive learning (BAAI/bge-m3-unsupervised) and unified fine-tuning across dense, sparse, and ColBERT objectives to produce the final BAAI/bge-m3 checkpoint.
Use Cases
- Semantic search and retrieval-augmented generation (RAG) over multilingual corpora
- Long-document retrieval where passages exceed the 512-token limit of most embedding models
- Cross-lingual matching, where queries and documents are in different languages
- Clustering, deduplication, and recommendation over text collections
The model card recommends a hybrid retrieval plus re-ranking pipeline for RAG: combine embedding retrieval with a lexical method such as BM25, then re-rank candidates with a cross-encoder re-ranker.
Benchmarks
On the MIRACL multilingual retrieval benchmark, BAAI reports updated evaluation results showing BGE-M3's overall performance higher than its originally published numbers after a scoring correction. The team also released MLDR, a long-document retrieval dataset covering 13 languages (Shitao/MLDR), on which BGE-M3's 8192-token input length is exercised. Third-party experiments cited on the model card report BGE-M3 achieving top performance in both English and other languages among the compared open and commercial embedding models.
