Kimi K3

Kimi K3 is an open-weight, native multimodal agentic model from Moonshot AI and their most capable model to date. It is a 2.8T-parameter Mixture-of-Experts model built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), with native vision capabilities and a 1-million-token context window. It is the first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.

Key Features

• New Architecture — Built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), scaling MoE sparsity with a Stable LatentMoE framework that activates 16 of 896 experts, yielding roughly a 2.5x improvement in overall scaling efficiency over Kimi K2.
• Long-Horizon Coding — Operating with minimal human oversight, Kimi K3 sustains long engineering sessions, navigates massive repositories, and orchestrates terminal tools, from GPU kernel optimization and compiler development to vision-in-the-loop game dev, CAD, and chip design.
• Agentic Knowledge Work — Advances end-to-end knowledge work, producing deep research with interactive visualizations, widgets and dashboards, and motion design and video editing, powered by its native multimodal architecture.
• Native Multimodality and Long Context — Understands text, images, and video within the same model, and supports a 1-million-token context window.
• Always-On Thinking — Thinking is always enabled and returned as reasoning_content; thinking effort is selectable between low, high, and max. The model was trained in preserved thinking history mode, so prior assistant reasoning is passed back across multi-turn conversations and tool calls.
• Open Frontier Weights — The full model weights are openly released, making frontier-class multimodal agentic capability available to run on your own infrastructure.

Architecture

Kimi K3 is a Mixture-of-Experts model with 2.8T total parameters and 104B activated per token, spread across 93 layers (1 dense). Its attention stack composes 69 KDA layers with 24 Gated MLA layers over 96 attention heads and a 7168-dimensional attention hidden size. The Stable LatentMoE framework uses a 3584-dimensional latent MoE space with 3072 hidden dimensions per expert, selecting 16 of 896 experts per token alongside 2 shared experts. The model uses SiTU-GLU activations, a 160K vocabulary, and a 1,048,576-token context length. Quantization-aware training is applied from the SFT stage onward, giving native MXFP4 weights with MXFP8 activations for broad hardware compatibility.

On Vast, Kimi K3 is available as a full-precision vLLM flagship served from the original weights at moonshotai/Kimi-K3, and as single-node Unsloth dynamic 1-bit and 2-bit GGUF quantizations from unsloth/Kimi-K3-GGUF.

Benchmarks

On reasoning and knowledge, Kimi K3 scores 93.5 on GPQA Diamond, 74.7 on AA-LCR, 43.5 on HLE-Full (56.0 with tools), and 23.4 on CritPt.

On coding, it reaches 88.3 on Terminal-Bench 2.1, 81.2 on FrontierSWE, 77.8 on ProgramBench, 72.9 on Kimi Code Bench 2.0, 67.5 on DeepSWE, 58.7 on SciCode, 48.3 on MLS-Bench-Lite, 42.0 on SWE-Marathon, and 36.6 on PostTrainBench.

On agentic tasks, it scores 95.0 on DeepSearchQA (F1), 94.6 on Harvey Lab-AA, 94.5 on MCPMark-Verified, 91.2 on BrowseComp, 84.8 on OSWorld-Verified, 84.2 on MCP-Atlas, 76.5 on Toolathlon-Verified, 76.2 on ResearchRubrics, 71.6 on CorpFin v2, 63.3 on OfficeQA Pro, 60.1 on SaaS-Bench, 58.3 on OSWorld 2.0, 54.4 on Finance Agent v2, 54.3 on JobBench, 44.2 on Legal Research Bench, 41.0 on APEX-Agents, 34.8 on SpreadsheetBench 2, 33.4 on tau-cubed Banking, 30.8 on AutomationBench, and 28.3 on Agents' Last Exam. It posts Elo ratings of 1686 on GDPval-AA v2 and 1548 on AA-Briefcase.

On vision, it scores 94.3 on MathVision (97.8 with python), 91.1 on OmniDocBench, 90.0 on Video-MME with subtitles, 85.7 on BabyVision with python, 84.8 on CharXiv reasoning questions (91.3 with python), 82.1 on MMVU, 81.6 on MMMU-Pro (83.4 with python), 58.5 on PerceptionBench, 51.0 on WorldVQA ForceAnswer, and 23.0 on ZeroBench pass@5 (41.0 with python).

Use Cases

• Long-horizon autonomous software engineering across large repositories
• Terminal and tool orchestration for build, test, and optimization loops
• Deep research producing interactive visualizations, dashboards, and widgets
• Document, chart, and screenshot understanding at scale
• Video understanding and motion design workflows
• Computer-use and browser agents operating over extended sessions
• Million-token context tasks spanning entire codebases or document sets