Embedding models decide which passages an agent ever sees. NVIDIA released Nemotron 3 Embed model：https://huggingface.co/collections/nvidia/nemotron-3-embed to work on that layer. It targets production-scale RAG, agentic retrieval, code retrieval, and agent memory.
The model collection includes three open checkpoints. Nemotron-3-Embed-8B-BF16 is the accuracy-first option. Nemotron-3-Embed-1B-BF16 carries the same design into a smaller footprint. Nemotron-3-Embed-1B-NVFP4 is the Blackwell-optimized 4-bit path.
All three are transformer encoders trained with bidirectional attention masking . The final embedding comes from average pooling over token-level representations. Maximum sequence length is 32,768 tokens on every checkpoint.
Each model was evaluated across 34 languages. All three carry the OpenMDW License Agreement, version 1.1 (OpenMDW-1.1) . Notably, the bases are Mistral models. The 8B is built with Ministral-3-8B-Instruct-2512 . Both 1B variants use Ministral-3-3B-Instruct-2512 .
Nemotron-3-Embed-8B-BF16 ranks #1 overall on RTEB：https://mteb-leaderboard.hf.space/benchmark/RTEB%28beta%29 (as of July 17 2026), the Retrieval Embedding Benchmark. Evaluation covers its 16 public tasks. Every figure below is average NDCG@10, at model sequence length 4096.
Two gaps are worth noting. The 1B gains 10.4 RTEB points over llama-nemotron-embed-vl-1b-v2 , the prior-generation baseline. Separately, NVFP4 costs 0.38 RTEB points against its BF16 parent, or 99.5% retention.
Those 1B scores come from a compression pipeline, not a smaller training run. The parent was nemotron-3-embed-3b , pruned and distilled across two iterative rounds.
First, the 3B parent was pruned to 2B using NVIDIA ModelOpt mcore_minitron Neural Architecture Search (NAS) . The search covers hidden width, FFN size, attention heads, and depth. It then picks the best candidate from the top-10 Pareto front. A 50k in-domain calibration corpus scored those candidates.
Next, the 2B model was distilled from the fine-tuned 8B embedding teacher. Distillation combined cosine distance loss (COS) and mean squared error (MSE) loss. The data blend was multilingual and in-domain. Finally, the same procedure repeated to produce the 1.14B checkpoint.
Compression then continues into the serving format. Quantization hit weights and activations of linear layers only, targeting the NVFP4 data type. The research team used nvidia-modelopt v0.45.0 . Quantization-Aware Distillation (QAD) followed, primarily to recover accuracy on long inputs.
Calibration used 512 samples: 256 queries and 256 passages from abisee/cnn_dailymail . QAD training used 20k samples.
The rsesearch team reports NVFP4 on Blackwell delivers up to 2x higher throughput than BF16 . It retains 99%+ of BF16 retrieval accuracy . The NVFP4 card also documents dynamic embedding sizes. You can slice the 2048-d vector from the start to 1024 or 512 dimensions. Re-normalize afterward.
Before touching code, watch the path run. It animates prefixing, bidirectional encoding, average pooling, L2 normalization, and dot-product scoring. Scores come from each card’s published expected output.
As that walkthrough implies, the checkpoints do not share runtime paths.
Alongside the checkpoints, NVIDIA research team released an optimized NIM microservice for the 1B model. The Rust-based NIM matches or outperforms the vLLM checkpoint on GB200 and RTX PRO 6000. NVIDIA tested input sequence lengths of 256 and 1024. Separately, NVIDIA NeMo AutoModel recipes cover fine-tuning and distillation.
With those paths in mind, prefixes come first. Queries take query: and documents take passage: . Embeddings are L2-normalized, so dot product equals cosine similarity.
encode_query and encode_document read the saved prompts. So you never add prefixes by hand. For serving, /v2/embed applies them from input_type instead:
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us ：https://forms.gle/wbash1wF6efRj8G58
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.
Scan to open this article