Pangu Large Model 5.5 - Huawei's New Generation AI Large Model

What is Pangu Large Model 5.5?

Pangu Large Model 5.5 is the latest generation of artificial intelligence large model released by Huawei at the Developer Conference (HDC 2025). The model emphasizes "not writing poetry, only doing things," focusing on solving practical industry problems and driving the intelligent upgrade of thousands of industries. Pangu Large Model 5.5 includes five foundational models, respectively oriented toward the fields of natural language processing (NLP), multimodality, prediction, scientific computing, and computer vision (CV).

Pangu Ultra MoE is a 718-billion-parameter deep thinking model, built based on full-stack Ascend software-hardware collaboration, with core capabilities such as efficient long-sequence processing, low hallucination, and deep research. Pangu Pro MoE is a 72B A16B model, performing excellently on agent tasks and comparable to the 671-billion-parameter DeepSeek-R1. Pangu Large Model 5.5 introduces adaptive fast-and-slow thinking integration technology, improving reasoning efficiency, and provides support for intelligent driving and embodied intelligent robot training through a multimodal world model.

Main Functions of Pangu Large Model 5.5

• Natural Language Processing (NLP)
  • Efficient Long-Sequence Processing: Through Adaptive SWA and ESA technologies, it can easily handle contexts with a length of 1 million tokens.
  • Low Hallucination: Adopts innovative solutions such as knowledge boundary determination and structured thinking verification to improve the accuracy of model reasoning.
  • Fast-and-Slow Thinking Fusion: Adaptive fast-and-slow thinking integration technology automatically switches thinking modes according to the difficulty of the problem, quickly responding to simple problems and deeply thinking about complex problems, improving reasoning efficiency by 8 times.
  • Deep Research Capability: Pangu DeepDiver, through long-chain difficult problem synthesis and progressive reward mechanisms, performs excellently in applications such as web search and commonsense question answering, can complete complex question answering with more than 10 hops within 5 minutes, and generate professional research reports of more than 10,000 words.
• Multimodality
  • World Model: Constructs a digital physical space for intelligent driving and embodied intelligent robot training, enabling continuous optimization and iteration. For example, in the field of intelligent driving, it can generate large amounts of training data without relying on high-cost road data collection.
• Prediction
  • Triplet Transformer Architecture: Uniformly performs triplet encoding and pretraining on data from different industries, improving prediction accuracy and generalization across industries and scenarios.
• Scientific Computing
  • AI Ensemble Forecasting: For example, the "Zhiji" large model upgraded by the Shenzhen Meteorological Bureau based on the Pangu Large Model is the first to achieve AI ensemble forecasting, and can more intuitively reflect the possible evolution of weather systems.
• Computer Vision (CV)
  • 30-Billion-Parameter Vision Large Model: Supports multi-dimensional pan-visual perception, analysis, and decision-making, builds a pan-visual fault sample library that is scarce in industrial scenarios, and improves the recognizable types and accuracy in business scenarios.

Introduction to the Models of Pangu Large Model 5.5

• Pangu Ultra MoE
  • Ultra-Large Scale and Sparse Activation: It has 718 billion parameters, adopts 256 routed experts, activates 8 experts per token, with an activation volume of 3.9 billion, and has the characteristics of ultra-large scale and a high sparsity ratio.
  • Advanced Architecture Design: Introduces the MLA (Multi-head Latent Attention) attention mechanism, effectively compressing the KV Cache space and alleviating the memory bandwidth bottleneck in the inference stage. At the same time, MTP (Multi-Token Parallelism) multi-head extension is adopted, expanding from single-head MTP training to a multi-head structure, enabling multi-token speculative inference and accelerating the overall inference process.
  • Stable Training Techniques: Proposes the Depth-Scaled Sandwich-Norm (DSSN) stable architecture and the TinyInit small initialization method, solving the stability challenges during the training of ultra-large-scale MoE models and achieving long-term stable training on more than 18TB of data.
  • Efficient Load Optimization: Adopts the EP group loss load optimization method, ensuring good load balancing among experts while enhancing the domain specialization capability of experts.
  • Training Strategy Optimization: Uses a Dropless training strategy to avoid the inconsistency between training and inference caused by Drop&Pad and improve training data efficiency. In addition, it adopts iterative hard example mining and a reward function that balances multiple capability items, referencing the GRPO algorithm, to improve the model's training efficiency and final inference performance.
• Pangu Pro MoE
  • Grouped Mixture-of-Experts Architecture: Innovatively proposes the Grouped Mixture-of-Experts model (MoGE), which groups experts during the expert selection stage and constrains tokens to activate an equal number of experts within each group, achieving expert load balancing and significantly improving the model's deployment efficiency on the Ascend platform.
  • Efficient Inference Performance: On the Ascend 300I Duo platform, single-card throughput can reach 201 tokens/s, and can be further increased to 321 tokens/s by introducing MTP decoding and multi-token optimization. On the Ascend 800I A2 platform, millisecond-level response can be achieved in low-concurrency scenarios, and single-card throughput can reach 1148 tokens/s under high-concurrency conditions, which can be increased to 1528 tokens/s after optimization, greatly outperforming dense models of the same scale.
  • Outstanding Reasoning Capability: Demonstrates excellent performance in cross-lingual multi-domain benchmark tests, covering English general reasoning, reading comprehension, commonsense reasoning, code generation in logical reasoning and Chinese-English bilingual mathematical problems, as well as Chinese knowledge question answering and reading comprehension, comprehensively validating the model's generality and domain adaptability on complex cognitive tasks.
  • Hardware-Affinity Optimization: Performs systematic optimization for the Ascend 300I Duo and 800I A2 platforms, deeply integrating the parallel computing characteristics of the Ascend hardware acceleration architecture with operator-level compilation optimization technology, achieving full-stack innovation from algorithm design to system implementation.
• Pangu Embedding
  • Fast and Slow Thinking Fusion: Adopts a dual-system cognitive architecture, integrating both "fast thinking" and "slow thinking" reasoning modes. Through a two-stage training framework, the first stage builds a basic reasoner through iterative distillation and the Multi-source Adaptive Reward System (MARS); the second stage endows the model with fast and slow thinking capabilities, allowing it to automatically switch modes according to task difficulty, achieving a dynamic balance between reasoning efficiency and depth.
  • Efficient Training Strategy: Proposes an SFT scheme based on Model-aware Iterative Distillation, dynamically selecting data samples that match the model's current capabilities for training, and preserving early knowledge through a model merging strategy during training to continuously improve performance.
  • Industry Vertical Domain Capability Expansion: By continuing training with the introduction of domain-specific long-chain-of-thought data, the model's capability level on professional tasks can be significantly improved. For example, in the legal domain, after training on legal corpora, the average accuracy on the LawBench benchmark reached 54.59%.
  • Adaptive Mode Switching: The model can automatically adjust the depth of reasoning according to the complexity of the task. It quickly outputs answers for simple questions and performs in-depth analysis for complex questions to ensure the accuracy of the output.
• Pangu DeepDiver
  • Complex Task Handling: For deep research scenarios, such as scientific assistants, personalized education, and complex industry report research, it can complete complex question answering with more than 10 hops and generate professional research reports of more than 10,000 words.
  • Efficient Information Acquisition: By constructing a large amount of synthetic interaction data and using a progressive reward strategy for reinforcement learning training, it performs excellently in open-domain information acquisition, can complete complex tasks within 5 minutes, and generate high-quality research reports.
  • Advanced Capability Enhancement: It significantly improves the Pangu large model's advanced capabilities such as autonomous planning, exploration, and reflection, enabling it to perform better in complex task handling.
• Pangu Prediction Large Model: It adopts the industry's first triplet transformer unified pre-training architecture, uniformly encodes data from different industries into triplets, and efficiently processes and pre-trains them within the same framework, improving the accuracy of the prediction large model and its generalization across industries and scenarios.
• Pangu Scientific Computing Large Model: Huawei Cloud continues to expand the integration of the Pangu Scientific Computing Large Model with more scientific application domains. For example, the Shenzhen Meteorological Bureau further upgraded the "Zhiji" large model based on Pangu, achieving AI ensemble forecasting for the first time, which can more intuitively reflect the possible evolution of weather systems and reduce the error of a single forecast model.
• Pangu Computer Vision CV Large Model: Huawei Cloud released a new 30-billion-parameter vision large model with MoE architecture, currently the industry's largest vision model, fully supporting multi-dimensional and pan-visual perception, analysis, and decision-making for images, infrared, laser point clouds, spectra, radar, and more. Through cross-dimensional generative models, it builds scarce pan-visual fault sample libraries for industrial scenarios such as oil and gas, transportation, and coal mines, improving the recognizable types and accuracy in business scenarios.
• Pangu Multimodal Large Model: A world model based on the Pangu Multimodal Large Model has been newly released, which can construct the required digital physical space for the training of intelligent driving and embodied intelligent robots, enabling continuous optimization and iteration.

Project Address of Pangu Large Model 5.5

• Pangu Ultra MoE Technical Paper:https://arxiv.org/pdf/2505.04519

  Pangu Pro MoE Project Address:https://gitcode.com/ascend-tribe/pangu-pro-moe

  Pangu Embedding Technical Paper:https://arxiv.org/pdf/2505.22375

  Pangu DeepDiver Technical Paper:https://arxiv.org/pdf/2505.24332

Application Scenarios of Pangu Large Model 5.5

• Intelligent Driving: The Pangu Multimodal Large Model can generate large amounts of training data for intelligent driving without relying on high-cost road data collection.
• Embodied Intelligent Robots: The Pangu World Model can construct the required digital physical space for the training of embodied intelligent robots, enabling continuous optimization and iteration.
• Weather Forecasting: The "Zhiji" large model, upgraded by the Shenzhen Meteorological Bureau based on the Pangu scientific computing large model, has achieved AI ensemble forecasting for the first time, which can more intuitively reflect the possible evolution of weather systems and reduce the errors of a single forecast model.
• Industrial Scenarios: The Pangu CV large model uses cross-dimensional generative models to build a scarce general visual fault sample library for industrial scenarios such as oil and gas, transportation, and coal mines, greatly improving the identifiable types and accuracy in business scenarios.