What is openPangu 2.0
openPangu 2.0 is a major upgrade of Huawei's open-source Pangu large model, offering a Pro version with up to 505B parameters and a Flash version with 92B, with all models uniformly equipped with a 512K ultra-long context and natively deeply tuned for Ascend and Kirin chips. On June 30, the model structure, weights, technical report, inference code, pre-training code, post-training code, and training operators will be gradually open-sourced across the full pipeline.
Main features of openPangu 2.0
-
Dual-version matrix: The Pro version targets extreme performance, while the Flash version targets lightweight deployment.
-
512K ultra-long context: Uniformly supports ultra-long text understanding, complex code analysis, and cross-application Agent interaction.
-
Ascend native tuning: Single-card inference throughput reaches 2 times that of mainstream open-source models in the industry, with lower latency.
-
HarmonyOS Agent adaptation: Faster task completion and higher accuracy, with significantly reduced Token consumption and resource usage.
-
On-device model: The model is a Kirin-friendly model with 30B total parameters / 2B activated, supporting local inference on phones and IoT devices.
Technical principles of openPangu 2.0
- DSA+SWA independent layered hybrid architecture: The industry's first to adopt an independent layered hybrid architecture, separately deploying Dynamic Sparse Attention (DSA) and Sliding Window Attention (SWA) to achieve an ultra-sparse attention mechanism. While maintaining 512K long-context understanding capability, the architecture greatly reduces redundant attention overhead through layered computation, causing the compute consumption of ultra-long sequence inference to decline non-linearly.
- Extreme sparse activation and expert routing: It adopts a 28:1 sparsity ratio; the Pro version has 505B total parameters but only activates 18B to complete inference. Through dedicated optimization of expert routing and precise compute allocation, the model's consistency between training and inference stages exceeds 99%, avoiding the training-inference distribution shift problem common in traditional MoE models and significantly reducing inference cost and VRAM usage.
- Ascend native full-stack optimization: Full adaptation to Ascend chips from algorithm architecture to training and inference. On the training side, it achieves super-node affinity optimization, improving training efficiency by 30%, and 512K long-sequence training throughput by 50%; on the inference side, it integrates native operators such as high-precision mHC, Muon, and ModAttn, achieving single-card throughput 2 times that of mainstream open-source models in the industry and balancing high throughput with low latency.
- On-device quantization pruning and expert reuse: The on-device model performs native quantization pruning for Kirin chips, speeding up inference by 50% and reducing memory usage by 20%. It introduces an expert reuse Loss mechanism, reducing expert switching frequency by 50%; combined with activated expert prediction technology, on-device model throughput increases 5 times, enabling local high-performance inference on phones and IoT devices.
How to use openPangu 2.0
- Project official website：https://gitcode.com/ascend-tribe
Core advantages of openPangu 2.0
-
Native compute affinity: Deeply adapted to Ascend and Kirin chips, releasing peak hardware performance without additional conversion.
-
Extreme sparsity ratio: The 28:1 sparsity ratio greatly reduces inference load, achieving large-model capability with fewer activated parameters.
-
Unified ultra-long context: The 512K context window covers long documents, multi-turn conversations, and complex Agent tasks.
-
Fully open training pipeline: A rare complete open-source pretraining and post-training codebase in the industry, lowering the barrier for secondary development.
-
Device-cloud collaboration coverage: Dual architecture with Ascend on the cloud side + Kirin on the device side, enabling full-scenario deployment from data centers to mobile devices.
Comparison of openPangu 2.0 with similar competitors
| Dimension | openPangu 2.0 Pro | DeepSeek-V3 |
|---|---|---|
| Total parameters | 505B | 671B |
| Activated parameters | 18B (sparsity ratio 28:1) | 37B (sparsity ratio approximately 18:1) |
| Context length | 512K ultra-long context | 128K context |
| Architectural innovation | The industry's first DSA+SWA independent layered hybrid architecture, ultra-sparse attention | Traditional MoE architecture, FP8 mixed-precision training |
| Hardware affinity | Native deep tuning for Ascend/Kirin chips, training-inference consistency >99% | Optimized for general-purpose NVIDIA GPUs, domestic compute power requires adaptation |
| Degree of open source | Full-chain open source of 7 major components starting June 30 (including pretraining, post-training code, and operators) | Open-source model weights, inference code, and technical report; training code not fully open-sourced |
Application scenarios of openPangu 2.0
-
Enterprise knowledge base: Use 512K context to handle in-depth analysis of ultra-long legal contracts, scientific research papers, and financial reports.
-
HarmonyOS intelligent agent: Build cross-application scheduling agents in the HarmonyOS ecosystem to enable device automation control and multimodal interaction.
-
Ascend cloud inference: Deploy cost-effective large model API services based on Ascend clusters, reducing the inference cost of domestic compute power.
-
On-device privacy computing: Through the 30B on-device model, implement local document processing and offline intelligent assistants on mobile phones.
-
Vertical industry customization: With open-source training code, quickly fine-tune dedicated models for fields such as manufacturing, healthcare, and government affairs.