ERNIE 4.5 300B A47B

ERNIE 4.5 300B A47B is Baidu's flagship large language model, built as a Mixture-of-Experts (MoE) network with 300 billion total parameters and 47 billion activated per token. By routing each token to a small subset of specialized experts, the model delivers the quality of a very large network while keeping the compute cost of each forward pass close to that of a much smaller dense model. It is released under the Apache 2.0 license for open commercial use.

Architecture

The model is organized into 54 transformer layers with 64 experts, of which 8 are activated for any given token. It supports a native context length of 131,072 tokens, making it well suited to long documents, extended conversations, and large retrieval contexts. ERNIE 4.5 was developed with a scaling-efficient training stack that combines hierarchical load balancing and FP8 mixed-precision training, and its post-training pipeline applies Supervised Fine-Tuning, Direct Preference Optimization, and Unified Preference Optimization to align the model for instruction following and dialogue.

Key Features

• Mixture-of-Experts design that activates only 47B of 300B parameters per token for efficient inference
• Long 131,072-token context window for long-form and retrieval-heavy workloads
• Supports both standard and reasoning-oriented inference styles
• Apache 2.0 licensed for commercial deployment

Benchmarks

ERNIE 4.5 300B A47B posts strong results on broad knowledge and reasoning evaluations, including a score of 78.4 on MMLU-Pro, reflecting competitive performance among leading open-weight models on graduate-level, multi-domain reasoning tasks.

Use Cases

• Conversational assistants and chatbots
• Long-document summarization, analysis, and question answering
• Content creation and writing assistance
• Knowledge retrieval and reference-grounded generation
• Research, reasoning, and general-purpose text generation

More detail is available on the model card.