What is Xiaomi MiMo
Xiaomi MiMo is Xiaomi's first open-source reasoning large model, designed to enhance model performance in complex reasoning tasks. The model is based on linked pre-training and post-training, mines a large amount of reasoning-rich corpus and adopts innovative reinforcement learning algorithms, significantly improving mathematical reasoning and code generation capabilities. With only a 7B parameter scale, MiMo surpasses OpenAI's o1-mini and Alibaba Qwen's QwQ-32B-Preview and other larger-scale models on public benchmark sets. Xiaomi MiMo includes 4 model versions. The pre-trained model MiMo-7B-Base, the supervised fine-tuned model MiMo-7B-SFT, the reinforcement learning models MiMo-7B-RL and MiMo-7B-RL-Zero have been open-sourced on HuggingFace, providing developers with powerful reasoning tools.
Xiaomi's latest released open-source large language modelMiMo-V2-Flash, adopts a 5:1 ratio of sliding window attention (window size 128) alternating with global attention, reducing KV cache by nearly 6 times, balancing long-text performance and efficiency. It supports parallel prediction of multiple tokens, with an average acceptance length of 2.8-3.6, and inference speed improved by 2-2.6 times, especially optimizing coding task efficiency. It is comparable to top open-source models such as DeepSeek-V3.2 and Kimi-K2 in multiple benchmark tests.
Main features of Xiaomi MiMo
- Powerful mathematical reasoning ability: Solves complex mathematical problems and provides accurate reasoning paths and answers.
- Efficient code generation ability: Generates high-quality code, suitable for a variety of programming tasks.
- Optimized reasoning performance: Enhances reasoning ability based on linked pre-training and post-training, surpassing larger-scale models with a 7B parameter scale, demonstrating efficient reasoning performance.
Technical principles of Xiaomi MiMo
- Pre-training stage: Focuses on mining reasoning-rich corpus, synthesizing approximately 200B tokens of reasoning data, ensuring the model has seen more reasoning patterns. Based on three-stage training, gradually increasing training difficulty, with a total training volume of 25T tokens, the model gradually improves its capabilities in tasks of different difficulty levels.
- Post-training stage：
- Reinforcement learning algorithm: Introduces the Test Difficulty Driven Reward algorithm, alleviating the reward sparsity problem in difficult algorithmic problems and improving the model's performance in complex tasks.
- Data re-sampling strategy: Introduces the Easy Data Re-Sampling strategy to stabilize the reinforcement learning (RL) training process.
- Efficient training framework: Designs the Seamless Rollout system, accelerating RL training (2.29 times) and validation (1.96 times), improving training efficiency.
- Model architecture optimization: Optimizes the model architecture for reasoning tasks, ensuring efficient reasoning capability under a limited parameter scale.
Project address of Xiaomi MiMo
- GitHub repository：https://github.com/XiaomiMiMo
- HuggingFace model library：https://huggingface.co/XiaomiMiMo
- Technical paper：https://github.com/XiaomiMiMo/MiMo/blob/main/MiMo-7B-Technical-Report.pdf
Application scenarios of Xiaomi MiMo
- Education field: Assists with mathematical problem-solving and programming learning, providing solution steps and code examples.
- Scientific research and academia: Assists with logical reasoning and algorithm development, helping verify hypotheses and design experiments.
- Software development: Generates and optimizes code, assisting with code debugging and problem-solving.
- Intelligent customer service: Answers complex questions, improving the efficiency of question-answering systems.
- Gaming and entertainment: Provide strategy suggestions and answers to intellectual puzzles to increase the game's fun.