NVIDIA Nemotron 3.5 Super VL 120B A12B: Long-Context Vision-Language Reasoning
NVIDIA Nemotron 3.5 Super VL is a vision-language model from NVIDIA built for reasoning, coding, tool use, document understanding and video understanding. It reads text, images and video and answers in text. The model has 120 billion total parameters with 12 billion active per token, supports contexts of up to 1 million tokens, and was released on Hugging Face on October 15, 2026.
Nemotron is NVIDIA's family of open models, with open weights, open training data and open recipes, aimed at building specialized AI agents efficiently. NVIDIA publishes a full-precision BF16 release and an NVFP4 release refined with quantization-aware distillation from the BF16 weights.
Architecture
The language backbone interleaves Mamba-2 state-space layers, Mixture-of-Experts layers and a small number of attention layers. Of its 88 layers, 40 are Mamba-2, 40 are MoE and 8 are attention, which keeps the active parameter count low and lets the model hold very long contexts without the memory growth of a pure attention stack. Each MoE layer has 512 routed experts plus a shared expert.
A C-RADIO vision encoder and a vision-language connector turn images, document pages and sampled video frames into representations the language backbone processes. A shared Multi-Token Prediction head predicts future tokens for native speculative decoding.
Key Features
- Image and video understanding on top of a strong text reasoning model, with documents supplied as page images alongside text
- Video input as MP4 clips of up to two minutes, sampled into frames
- Configurable reasoning: off, on (medium) and on, selected through the chat template; medium effort produces fewer reasoning tokens at some cost in accuracy
- Native tool calling, with reasoning and tool-call output parsed into separate response fields
- Context window of up to 1 million tokens, shared by the prompt, visual inputs, conversation history, tool results and the response
- Multilingual training data covering English, 19 other spoken languages and 43 programming languages
Training
The language backbone starts from Nemotron 3 Super Base. NVIDIA first trained the shared MTP head with the backbone frozen, then distilled from Nemotron 3 Ultra Base, and ran continued pre-training that includes an extension to 1 million tokens of context and software-engineering mid-training on repositories, issue and pull-request pairs, build logs and agent trajectories. Vision-language training then added the C-RADIO encoder and connector, followed by supervised fine-tuning on packed sequences of up to 524,288 tokens, reinforcement learning with verifiable rewards and human feedback, and multi-teacher on-policy distillation. Pre-training data runs through December 2025.
Benchmarks
NVIDIA collected all of its results with NeMo Gym under consistent settings.
On reasoning and knowledge the model scores 89.52 on GPQA-Diamond, 31.79 on text-only Humanity's Last Exam and 88.36 on the APEX shortlist. On coding it reaches 88.22 on LiveCodeBench v6, 71.10 on SWE-bench Verified, 51.78 on SWE-bench Pro and 60.67 on Terminal-Bench 2.1. On agentic and long-context work it scores 78 on BrowseComp, 76.50 on tau2-bench airline, 89.71 on GraphWalks and 68.62 on AA-LCR. Its hallucination rate on AA-Omniscience is 30.0 percent.
On vision it scores 85.60 on V*, 80.82 on CharXiv reasoning, 78.16 on MathVision, 65.08 on OCR-Reasoning, 62.82 on MMLongBench-Doc and 51.14 on OSWorld-Verified.
Use Cases
- Agents that read screenshots, documents and charts and act through tool calls
- Coding assistants and software-engineering agents
- Document question answering over scanned pages, forms and long reports
- Video summarization and question answering on short clips
- Research and search workflows that combine reasoning with tools
Sampling
NVIDIA recommends a temperature of 1.0 and top-p of 0.95 for text, and a temperature of 0.6 and top-p of 0.95 for vision tasks. Reasoning tokens and the final answer share the max_tokens budget, so leave room for both.
