BitCPM-CANN - ModelBest's Open-Source On-Device Large Model with Tsinghua University

What is BitCPM-CANN

BitCPM-CANN is China's first 1.58-bit ternary large model fully trained end-to-end on Huawei Ascend's domestic computing power platform, jointly launched by ModelBest, Tsinghua University, and the OpenBMB open-source community. The model adopts a quantization-aware training approach and includes four sizes from 0.5B to 8B. During inference, it unlocks approximately 6x memory savings, with model capability retention reaching 90.1%–97.2%, marking the first time a domestic NPU has a complete low-bit training stack.

Main Features of BitCPM-CANN

• 1.58-bit ternary large model inference: Available in four parameter specifications: 0.5B, 1B, 3B, and 8B, enabling efficient operation on on-device devices; the 8B model can be easily deployed on current mainstream flagship smartphones.
• Full-chain training loop on domestic Ascend: The complete pipeline from training to inference is natively completed on Huawei's Ascend platform, without relying on foreign GPUs for pre-training.
• Ultimate memory savings: Compared with traditional BF16 precision, inference unlocks approximately 6x memory savings, allowing the same memory to support far greater model capability than before.
• Full-precision comparative evaluation and validation: Completed a 1:1 performance comparison with same-size MiniCPM4 full-precision models on 11 tasks including commonsense, reading comprehension, subject knowledge, and mathematics and reasoning.
• Open-source models and training foundation: The full series of model weights and the low-bit training foundation based on MindSpeed × Megatron-LM are openly available, supporting developers in reproduction and secondary innovation.

Technical Principles of BitCPM-CANN

• Quantization-aware training (QAT): The model actively learns to use ternary weights (-1, 0, +1) for knowledge representation from the initial stage of training, rather than being passively compressed after training is complete, fundamentally achieving native growth at extremely low bit widths.
• Maximizing the information density of ternary weights: The 1.58-bit width constraint forces each bit to achieve maximum knowledge-carrying efficiency, making weight precision depend on the amount of knowledge carried per bit rather than the bit width.
• MindSpeed × Megatron-LM training foundation: Based on the backbone framework, a complete low-bit training infrastructure is built on the Ascend platform, including environment adaptation, 32K long-sequence support, parallel strategies, and a fused operator system.
• End-to-end native adaptation to domestic computing power: From the lowest-level quantization operators and QAT algorithms to complete parallel strategies and training frameworks, everything is natively developed and optimized for Huawei Ascend, achieving integrated training and inference.
• Systematic scalable low-bit paradigm: Through engineering validation of the full-size series from 0.5B to 8B, it proves that this low-bit training route has cross-scale systematicness, scalability, and engineering reproducibility.

How to Use BitCPM-CANN

• Obtain open-source model weights: Visit HuggingFace to download the full series of model weights for 0.5B, 1B, 3B, and 8B.
• Deploy the training foundation: Based on ModelBest's open-source MindSpeed × Megatron-LM low-bit training foundation, complete environment adaptation, dependency installation, and framework deployment on the Huawei Ascend platform.
• Run on-device inference: Use the model's 6x memory savings advantage to directly run inference with models of 8B and below on current mainstream flagship smartphones or PC-side on-device devices.
• Secondary training innovation: Based on the QAT training infrastructure and open-source code, perform secondary fine-tuning, domain adaptation, or larger-scale low-bit training innovation on the Ascend platform.

Core advantages of BitCPM-CANN

• Breakthrough in a closed loop for domestic compute: For the first time, end-to-end 1.58-bit training is achieved on Ascend with full-precision comparative evaluation up to the 8B level, breaking the stereotype that domestic chips can only run inference.
• Industry-leading performance retention: The 3B model reaches a 97.2% capability retention rate, the highest across all tiers; 8B reaches 95.7%, and 0.5B also reaches 90.1%. It is systematically scalable and engineering-reproducible.
• Training efficiency significantly better than comparable models: Previously, the GPU version of BitCPM 1B achieved comprehensive superiority using only 1/20 of the training compute of Microsoft BitNet 2B, and the efficiency advantage has been validated.
• The on-device ceiling is greatly raised: A 6x memory dividend combined with MoE architecture is expected to fit ultra-large models with 50B or even 100B parameters into terminals, bringing on-device AI capabilities to a new height.

Project address of BitCPM-CANN

• HuggingFace model repository：https://huggingface.co/collections/openbmb/bitcpm-cann

Comparison of BitCPM-CANN with similar competing products

Application scenarios of BitCPM-CANN

• Smartphone on-device AI: Run larger-parameter models under limited memory; in the future, it is expected to run 60B large models on phones, directly improving terminal intelligence and user experience.
• Completion of the domestic chip ecosystem: Provide high-quality model-side supply for on-device chips such as Qualcomm Snapdragon 8 Gen 4 that already support 2-bit inference, enabling software-hardware collaboration and a closed ecosystem loop.
• Enterprise on-device deployment: Suitable for low-memory-cost large-model inference on PCs, cars, IoT, and other devices, hedging against supply chain pressure brought by soaring global HBM memory prices.
• Low-bit training research innovation: Provide Ascend platform developers with reusable and scalable QAT public infrastructure, supporting secondary training and fine-tuning innovation based on domestic compute.