Laguna S 2.1 NVFP4

Laguna S 2.1 is a 117.6B total-parameter Mixture-of-Experts language model from poolside, with 8.5B parameters activated per token. It is built for agentic coding and long-horizon work, and this NVFP4 checkpoint packages the model in native 4-bit weights so it can run efficiently on modern hardware while preserving quality.

Overview

The model pairs a sparse Mixture-of-Experts feed-forward design (256 experts plus one shared expert) with a hybrid attention layout tuned for fast inference and low memory pressure. Of its 48 layers, 36 use Sliding Window Attention with per-head gating and 12 use global attention, in a 3:1 ratio. This mix keeps most layers operating over a short sliding window while a smaller set of global layers preserves long-range context, which reduces the KV cache footprint at long sequence lengths. The attention stack uses softplus gating with per-layer rotary scales. The KV cache is quantized to FP8 to further reduce memory per token.

Key features

• Mixed sliding-window and global attention for fast inference with a small KV cache
• Native reasoning support with interleaved thinking between tool calls, which can be enabled or disabled per request
• Sparse MoE routing that activates only 8.5B of 117.6B parameters per token
• A 256K-token context window, suitable for large codebases and long agent trajectories
• Native NVFP4 weights, with FP8 KV cache detected automatically at load time

Architecture

Laguna S 2.1 was trained with pre-training, post-training, and reinforcement-learning stages using the Muon optimizer. It has 48 layers, 256 routed experts with one shared expert, and a 512-token sliding window on its local-attention layers. The checkpoint ships configured for a 262,144-token context window, which poolside recommends for best output quality; the weights are calibrated from native 1M-token checkpoints and the context can be extended by adjusting the model configuration, with some possible quality trade-off.

Benchmarks

poolside reports that Laguna S 2.1, at 118B total and 8.5B active parameters, is competitive with much larger models on agentic coding and tool-use evaluations. Reported scores include 70.2% on Terminal-Bench 2.1, 78.5% on SWE-bench Multilingual, 59.4% on SWE-Bench Pro (public dataset), 40.4% on DeepSWE, 46.2% on SWE Atlas codebase question answering, and 49.7% on Toolathlon Verified. These results place it ahead of several models with many times its parameter count on the same tasks.

Use cases

• Autonomous and assisted software engineering agents
• Long-horizon coding tasks across large repositories
• Tool-calling and function-calling workflows with interleaved reasoning
• Codebase question answering and multilingual code understanding

For full model details and evaluation methodology, see the model card on Hugging Face.