Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for
  Multilingual and Cross-Lingual Tasks
We introduce llama-embed-nemotron-8b, an open-weights text embedding model
that achieves state-of-the-art performance on the Multilingual Massive Text
Embedding Benchmark (MMTEB) leaderboard as of October 21, 2025. While recent
models show strong performance, their training data or methodologies are often
not fully disclosed. We aim to address this by developing a fully open-source
model, publicly releasing its weights and detailed ablation studies, and
planning to share the curated training datasets. Our model demonstrates
superior performance across all major embedding tasks -- including retrieval,
classification and semantic textual similarity (STS) -- and excels in
challenging multilingual scenarios, such as low-resource languages and
cross-lingual setups. This state-of-the-art performance is driven by a novel
data mix of 16.1 million query-document pairs, split between 7.7 million
samples from public datasets and 8.4 million synthetically generated examples
from various open-weight LLMs. One of our key contributions is a detailed
ablation study analyzing core design choices, including a comparison of
contrastive loss implementations, an evaluation of synthetic data generation
(SDG) strategies, and the impact of model merging. The llama-embed-nemotron-8b
is an instruction-aware model, supporting user-defined instructions to enhance
performance for specific use-cases. This combination of top-tier performance,
broad applicability, and user-driven flexibility enables it to serve as a
universal text embedding solution.
69131915a644ba07c499c8af
Yauhen Babakhin
claimed_verified
2025-11-11T19:41:30.752Z
643f884da649e1a499446c11
https://cdn-avatars.huggingface.co/v1/production/uploads/643f884da649e1a499446c11/4lmRszXeD1S62sfwUHg2D.jpeg
Yauhen Babakhin
ybabakhin
user
ybabakhin
69131915a644ba07c499c8b0
Radek Osmulski
69131915a644ba07c499c8b1
Ronay Ak
claimed_verified
2025-11-11T19:41:27.473Z
6814af6d59c3d8d6f9b09f2b
/avatars/d9fa8404f258c96df1c500cffd10752f.svg
Ronay Ak
ronay-nv
user
ronay-nv
69131915a644ba07c499c8b2
Gabriel Moreira
69131915a644ba07c499c8b3
Mengyao Xu
69131915a644ba07c499c8b4
Benedikt Schifferer
69131915a644ba07c499c8b5
Bo Liu
claimed_verified
2025-11-12T12:20:59.269Z
6172d79a6c2ff4e51d0ea593
/avatars/ad474c8048b2899191fdbc05125b784c.svg
Bo Liu
BoLiu
user
BoLiu
69131915a644ba07c499c8b6
Even Oldridge
Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for
  Multilingual and Cross-Lingual Tasks
We introduce llama-embed-nemotron-8b, an open-weights text embedding model
that achieves state-of-the-art performance on the Multilingual Massive Text
Embedding Benchmark (MMTEB) leaderboard as of October 21, 2025. While recent
models show strong performance, their training data or methodologies are often
not fully disclosed. We aim to address this by developing a fully open-source
model, publicly releasing its weights and detailed ablation studies, and
planning to share the curated training datasets. Our model demonstrates
superior performance across all major embedding tasks -- including retrieval,
classification and semantic textual similarity (STS) -- and excels in
challenging multilingual scenarios, such as low-resource languages and
cross-lingual setups. This state-of-the-art performance is driven by a novel
data mix of 16.1 million query-document pairs, split between 7.7 million
samples from public datasets and 8.4 million synthetically generated examples
from various open-weight LLMs. One of our key contributions is a detailed
ablation study analyzing core design choices, including a comparison of
contrastive loss implementations, an evaluation of synthetic data generation
(SDG) strategies, and the impact of model merging. The llama-embed-nemotron-8b
is an instruction-aware model, supporting user-defined instructions to enhance
performance for specific use-cases. This combination of top-tier performance,
broad applicability, and user-driven flexibility enables it to serve as a
universal text embedding solution.
A fully open-source text embedding model achieves state-of-the-art performance across embedding tasks, particularly in multilingual scenarios, through a novel data mix and detailed ablation studies.
text embedding model Multilingual Massive Text Embedding Benchmark (MMTEB) open-source ablation studies retrieval classification semantic textual similarity (STS) low-resource languages cross-lingual setups contrastive loss synthetic data generation (SDG) model merging instruction-aware model