NVIDIA Nemotron 3.5 Lightning 30B A3B NVFP4

NVIDIA Nemotron 3.5 Lightning 30B A3B NVFP4 is the quantized, inference-ready release of Nemotron 3.5 Lightning, a hybrid Mixture-of-Experts language model trained by NVIDIA. It has 30 billion total parameters with only 3 billion active per token, supports context lengths up to 1 million tokens, and was released on August 11, 2026.

Nemotron is NVIDIA's family of open models — open weights, open training data, and open recipes — aimed at building specialized AI agents efficiently. This is the checkpoint NVIDIA recommends for serving. The weights are quantized to NVFP4, a 4-bit floating-point format, which cuts the checkpoint to roughly a third the size of the full-precision release and lets the model run on a single GPU.

Architecture

The model interleaves Mamba-2 state-space layers with Mixture-of-Experts layers and a small number of Attention layers. Of its 52 layers, 23 are Mamba-2, 23 are MoE, and 6 are Attention. That hybrid design is what keeps the active parameter count at 3 billion and lets the model hold very long contexts without the memory growth a pure attention stack would incur.

The model routes to 6 of 128 experts per token, alongside a shared expert, and ships with multi-token prediction layers plus separate draft models for speculative decoding.

Key features

• Hybrid Mamba-2, MoE and Attention architecture with 3B active parameters out of 30B total
• NVFP4 weights, quantized by NVIDIA with its own tooling rather than by a third party
• 1 million token context served by default
• Configurable reasoning: thinking can be switched on or off through the chat template, and a runtime thinking budget can be set per request
• Native tool calling, with reasoning and tool-call output parsed into separate response fields
• Multilingual: English and coding languages as primary, plus Spanish, French, German, Italian and Japanese
• Pre-trained on more than 20 trillion tokens

Accuracy after quantization

NVIDIA published side-by-side scores for the full-precision and NVFP4 checkpoints under a consistent internal harness, so the cost of quantization is measurable rather than assumed. The two are close across the board, and on several tasks the quantized checkpoint scores higher.

On general knowledge it scores 81.62 on MMLU Pro against 81.94 for the full-precision weights. On reasoning it reaches 75.57 on GPQA Diamond without tools, slightly ahead of full precision at 75.44. On coding and agentic work it scores 52.80 on SWE-bench Verified against 51.56, 23.46 on Terminal-Bench 2.1 against 24.58, 83.43 on PinchBench against 85.37, and 36.81 on BrowseComp against 36.97. On instruction following it scores 72.88 on IFBench (loose) against 71.88. The largest drop is on long-context retrieval, where AA-LCR falls from 52.00 to 49.19.

These numbers were measured by NVIDIA and may differ from other vendors' self-reported results.

Use cases

• Long-running autonomous agents and sub-agent workhorse deployments
• Agentic applications that need tool calling and controllable reasoning
• Long-document analysis, retrieval-augmented generation and summarization
• High-throughput batch serving where the smaller checkpoint leaves more room for concurrency
• Efficient inference on a single GPU

Sampling

NVIDIA recommends a temperature of 1.0 and top-p of 0.95.

Related releases

The full-precision reference weights, intended for fine-tuning and further quantization, are published separately as NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, and the base (pre-instruction-tuning) weights as NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16.