What is MiniCPM-SALA
MiniCPM-SALA is a 9B on-device large model open-sourced by ModelBest. It adopts the innovative SALA (Sparse-Linear hybrid Attention) architecture, in which 75% linear attention is responsible for efficient global modeling, while 25% sparse attention precisely captures key information. It introduces the hybrid position encoding HyPE to achieve seamless switching between long and short texts. The model is the first to achieve million-level context inference on consumer-grade GPUs (such as the RTX 5090), with low VRAM usage and fast inference speed, providing a feasible path for deploying long-text capabilities in on-device Agents.
Main Features of MiniCPM-SALA
-
Million-level long-text processing: Supports 1M+ tokens of context, and is the first to fully run million-length inference on the RTX 5090 consumer-grade GPU.
-
Efficient inference acceleration: Achieves a 3.5x inference speed improvement on 256K sequences compared with dense models of the same size.
-
Low VRAM usage: By optimizing the KV Cache, it still runs stably without OOM at lengths from 512K to 1M.
-
Compatibility with long and short texts: The model maintains general capabilities comparable to Qwen3-8B on short texts, and has significant advantages on long texts.
-
On-device deployment optimization: Designed specifically for terminal devices such as phones, cars, and robots, enabling localized long-context Agents.
Technical Principles of MiniCPM-SALA
- SALA hybrid attention architecture: Combines 75% linear attention (Lightning Attention) with 25% sparse attention (InfLLM v2) in a layered manner. The former handles global modeling with O(N) complexity, while the latter computes key local information on demand, achieving a balance between efficiency and accuracy.
- HyPE hybrid position encoding: The linear layers retain RoPE to maintain short-text performance, while the sparse layers use NoPE to decouple the KV-Cache from position, avoiding long-distance decay and supporting efficient retrieval over million-level context.
- HALO low-cost transfer: Through four steps-parameter conversion, hidden-state alignment, layer selection, and knowledge distillation-it transfers a full-attention model to a hybrid architecture, reducing the training budget to 25% of pretraining from scratch.
Project Address of MiniCPM-SALA
- GitHub repository：https://github.com/openbmb/minicpm
- HuggingFace model library：https://huggingface.co/openbmb/MiniCPM-SALA
Application Scenarios of MiniCPM-SALA
-
Personal intelligent assistant: Continuously retains million-level chat records and personal preferences on a local device, realizing a truly "understanding" privacy-preserving personal assistant.
-
On-device knowledge base: Conducts professional Q&A based on an offline million-word document library, meeting the localization needs of sensitive enterprise data.
-
Code development assistant: Understands the context of an entire code repository at once, assisting with cross-file debugging and complex refactoring tasks.
-
In-vehicle intelligent system: Integrates vehicle manuals, navigation history, and user habits to provide personalized driving services in an offline in-car environment.
-
Scientific literature analysis: The model quickly processes massive academic papers and extracts cross-document关联信息 to assist with review writing.