MiniCPM 4.0 - ModelBest's Open-Source Extremely Efficient On-Device Large Model

What is MiniCPM 4.0

MiniCPM 4.0 is an extremely efficient on-device large model launched by ModelBest. The models come in two parameter scales: 8B and 0.5B. The 8B Lightning Sparse version adopts an innovative sparse architecture and can efficiently handle long-text tasks; the 0.5B version is known for its low computational resource consumption and high performance. The self-developed CPM.cu inference framework can achieve up to a 220x speedup in extreme scenarios, and can also achieve a 5x speed improvement under normal circumstances. MiniCPM 4.0 supports deployment on open-source frameworks such as vLLM, SGLang, and LlamaFactory, and has been adapted to mainstream chips such as Intel, Qualcomm, MTK, and Huawei Ascend.

MiniCPM 4.0 Open-Source Model Collection

MiniCPM4-8B: The flagship model of MiniCPM4, with 8 billion parameters, trained on 8T tokens.

MiniCPM4-0.5B: The small version of MiniCPM4, with 0.5 billion parameters, trained on 1T tokens.

MiniCPM4-8B-Eagle-FRSpec: The Eagle head for FRSpec, accelerating speculative inference of MiniCPM4-8B.

MiniCPM4-8B-Eagle-FRSpec-QAT-cpmcu: The Eagle head of FRSpec trained with QAT, efficiently combining speculation and quantization to achieve super acceleration of MiniCPM4-8B.

MiniCPM4-8B-Eagle-vLLM: The Eagle head in vLLM format, accelerating speculative inference of MiniCPM4-8B.

MiniCPM4-8B-marlin-Eagle-vLLM: The quantized Eagle head in vLLM format, accelerating speculative inference of MiniCPM4-8B.

BitCPM4-0.5B: Applied ExtremeTernary QuantizationOn MiniCPM4-0.5B, compressing model parameters into ternary values, achieving a 90% reduction in bit width.

BitCPM4-1B: Applying extreme ternary quantization to MiniCPM3-1B, compressing model parameters into ternary values, achieving a 90% reduction in bit width.

MiniCPM4-Survey: Based on MiniCPM4-8B, it accepts user queries as input and automatically generates trustworthy long-form survey papers.

MiniCPM4-MCP: Based on MiniCPM4-8B, it accepts user queries and available MCP tools as input, and automatically calls the relevant MCP tools to meet user needs.

Main Features of MiniCPM 4.0

• Efficient Dual-Frequency Gear-Shifting Mechanism: The MiniCPM 4.0-8B version can automatically switch attention modes based on task characteristics. It enables sparse attention when processing long texts to reduce computational complexity; it switches to dense attention when processing short texts to ensure accuracy.
• Extreme Inference Speed Improvement: Through the self-developed CPM.cu inference framework, it achieves up to a 220x speedup in extreme scenarios, and can also achieve a 5x speed improvement in normal scenarios.
• Model Slimming and Efficient Deployment: It adopts an innovative sparse architecture and extreme low-bit-width quantization technology to reduce model size by 90% while maintaining excellent performance. It supports deployment on open-source frameworks such as vLLM, SGLang, and LlamaFactory.
• On-Device Inference Optimization: It has a built-in self-developed CPM.cu ultra-fast on-device inference framework, bringing 90% model slimming and speed improvements through innovations in speculative sampling, model compression and quantization, and on-device deployment frameworks.
• Multi-Platform Adaptation: It has been adapted to mainstream chips such as Intel, Qualcomm, MTK, and Huawei Ascend, and can run smoothly on a variety of on-device devices.
• Multiple Version Choices: It provides versions with two parameter scales, 8B and 0.5B, to meet the needs of different scenarios.

Technical Principles of MiniCPM 4.0

• Sparse Attention Mechanism: MiniCPM 4.0-8B adopts an innovative trainable sparse attention mechanism (InfLLM v2). When processing long texts, each token only needs to perform relevance computations with less than 5% of other tokens, greatly reducing the computational overhead of long-text processing.
• Efficient Dual-Frequency Shifting: Automatically switches attention modes based on task characteristics. When processing long texts, sparse attention is enabled to reduce computational complexity, and when processing short texts, it switches to dense attention to ensure accuracy.
• Model Wind Tunnel 2.0: Introduces an advanced downstream task Scaling prediction method, enabling more accurate search and determination of the optimal model training configuration.
• BitCPM: Implements an extreme ternary quantization technique, compressing the bit width of model parameters by more than 90%, while maintaining excellent performance during substantial "slimming."
• Efficient Training Engineering: Fully adopts FP8 low-precision computing technology, combined with a Multi-token Prediction training strategy, to further improve training efficiency.
• Self-Developed Inference Framework: MiniCPM 4.0 has a built-in self-developed CPM.cu ultra-fast on-device inference framework. Through innovations in speculative sampling, model compression and quantization, and on-device deployment frameworks, it delivers 90% model slimming and speed improvement.

MiniCPM 4.0 Project Address

• Github Repository：https://github.com/OpenBMB/MiniCPM
• HuggingFace Model Library：https://huggingface.co/collections/openbmb/minicpm4
• Technical Report：https://github.com/OpenBMB/MiniCPM/blob/main/report/MiniCPM_4_Technical_Report.pdf

Application Scenarios of MiniCPM 4.0

• Intelligent Tutoring System: MiniCPM 4.0 can serve as the core of an intelligent tutoring system. Through natural language processing technology, it understands students' questions and provides detailed answers and explanations, helping students better understand and master knowledge.
• Case Analysis and Diagnostic Assistance: MiniCPM 4.0 can assist doctors in case analysis. Through natural language processing technology, it understands the content of medical records, provides doctors with diagnostic suggestions and reference information, and improves the accuracy and efficiency of diagnosis.
• Medical Literature Retrieval: Helps doctors and researchers quickly retrieve relevant medical literature and research results, provides accurate literature recommendations and summaries, and saves time and effort.
• Intelligent Customer Service: As the core of financial customer service, it quickly and accurately answers customers' consulting questions, provides personalized services and solutions, and improves customer satisfaction.
• Game Plot Generation: MiniCPM 4.0 can automatically generate rich and diverse game plots and tasks based on game settings and player behavior, increasing the fun and playability of the game.