GLM 5.3-Flash: Natively Multimodal 1M-Context MoE Model

GLM 5.3-Flash is the first natively multimodal model in the GLM-5 series from Z.ai. It has 320B total parameters with 18B active per token, accepts text, image, and video input, and serves a 1M-token context window. Z.ai reports that it outperforms GLM 5.2 across benchmarks and real-world workloads. It supports English and Chinese natively.

Key Features

• Native Multimodality - Text, image, and video inputs handled by the model itself rather than a bolted-on adapter, the first such model in the GLM-5 series
• 1M-Token Context - Long-document analysis, large-repository navigation, and extended agentic trajectories in a single context
• Hybrid Sparse and Linear Attention - Combines sparse and linear attention to cut long-context serving cost while preserving long-context accuracy
• Sparse Mixture-of-Experts - 288 routed experts with 8 active per token, so only 18B of the 320B parameters run on any given token
• Adjustable Thinking Budget - A reasoning effort setting with low, high, and max levels; max is the default
• Tool Use and Agentic Work - Evaluated on agentic suites including Terminal-Bench 2.1, Toolathlon Verified, AutomationBench, and DeepSWE

Use Cases

• Visual question answering, chart and document understanding, and video comprehension
• Repository-level software engineering and code generation across very long contexts
• Multi-step agentic workflows with tool calling
• Long-document analysis, synthesis, and generation
• Terminal-based development and operations work
• Bilingual English and Chinese assistants

Architecture and Design

The language model has 45 layers that combine sparse attention with linear attention, alongside a 24-layer vision encoder for image and video input. Z.ai introduced Manifold-Constrained Hyper-Connections in this release to improve scaling efficiency, and the checkpoint includes a native draft layer for speculative decoding. Routing sends each token through 8 of 288 experts plus one shared expert.

Training and Inference

GLM 5.3-Flash starts from a newly trained base model rather than a refresh of GLM 5.2, with its architecture and training recipe redesigned around capability and efficiency, and it was pre-trained on a 30T-token multimodal corpus. Thinking is on by default. Z.ai evaluates the model at temperature 1.0 with top-p 0.95 and a maximum generation length of 163,840 tokens, and reports long-context results at context lengths up to 1M tokens.

Deploy GLM 5.3-Flash on Vast.ai for multimodal reasoning, agentic coding, and million-token context work on flexible GPU infrastructure.