MiMo is a series of language models open-sourced by Xiaomi, with the full name MiMo: Unlocking the Reasoning Potential of Language Model – From Pretraining to Posttraining. The project is hosted on GitHub (https://github.com/XiaomiMiMo/MiMo）。
Purpose MiMo is aimed at reasoning tasks, especially mathematical and code reasoning. The project believes that the effectiveness of training reasoning models with reinforcement learning depends on the inherent reasoning potential of the base model, so it not only focuses on post-training, but also optimizes pretraining strategies for reasoning, in order to fully unlock the reasoning potential of language models.
Main Capabilities MiMo-7B is a model series trained from scratch and born for reasoning tasks. Its RL experiments show that the model has outstanding reasoning potential, even surpassing larger 32B models. MiMo-7B-RL, obtained by RL training starting from the cold-start SFT model, performs excellently on mathematical and code reasoning tasks and can match the performance of OpenAI o1-mini. The project has open-sourced multiple checkpoints, including the base model, the SFT model, the RL model trained from the base, and the RL model trained from SFT.
In terms of pretraining, the project optimized the data preprocessing pipeline, enhanced the text extraction toolkit, and applied multi-dimensional data filtering to increase the density of reasoning patterns in the pretraining data; adopted multiple strategies to generate a large amount of diverse synthetic reasoning data; and adopted a three-stage data mixing strategy for pretraining. MiMo-7B-Base was overall pretrained on about 25 trillion tokens, and introduced multi-token prediction (MTP) as an additional training objective to improve model performance and accelerate inference. The MTP layer was adjusted during the pretraining and SFT stages and frozen during the RL stage; when using one MTP layer for speculative decoding, the acceptance rate is about 90%.
Regarding the post-training recipe, the project curated 130,000 math and code problems as RL training data. These problems can be verified by rule-based verifiers, and each problem has undergone careful cleaning and difficulty assessment. Only rule-based accuracy rewards are used to avoid potential reward hacking; for the sparse reward problem of difficult code problems, code rewards driven by test difficulty are introduced, using dense reward signals to optimize the policy more effectively by assigning fine-grained scores to test cases of different difficulty levels; a data resampling strategy is implemented for easy problems to improve rollout sampling efficiency and stabilize policy updates, especially in the later stages of RL training.
In terms of RL infrastructure, the project developed the Seamless Rollout Engine to accelerate RL training and validation. Its design integrates continuous rollout, asynchronous reward computation, and early termination to reduce GPU idle time, achieving a 2.29x speedup in training and a 1.96x speedup in validation; it supports MTP in vLLM and enhances the robustness of the inference engine in the RL system.inference engine robustness.
May 30, 2025 update: The SFT dataset was expanded from about 500,000 instances to 6 million instances, and the RL training window size was continuously extended from 32K to 48K. The performance of MiMo-7B-RL-0530 on AIME24 can continue to improve, ultimately surpassing DeepSeek R1 (79.8).
Usage Models are available at https://huggingface.co/XiaomiMiMo and https://www.modelscope.cn/organization/XiaomiMiMo Retrieval. In terms of inference support, the SGLang team provides MiMo model support and MTP. MiMo is already supported in mainstream SGLang, and detailed usage can be found in the SGLang documentation. The project officially recommends using its vLLM branch (developed based on vLLM 0.7.3) for MiMo-MTP inference; alternatively, register a vLLM loader for MiMo without loading MTP parameters—you can copy registry/register_mimo_in_vllm.py to your own directory and import it. The official recommendation is to use an empty system prompt. The performance of MiMo on other inference engines has not yet been verified, and contributions based on the model definitions in the Hugging Face repository are welcome. If you have any questions, you can contact mimo@xiaomi.com or submit an issue.
Applicable scenarios Suitable for tasks requiring strong reasoning abilities such as mathematical reasoning and code reasoning. It can be used for further training, evaluation, or deployment based on open-source checkpoints, and can also be used to study reasoning enhancement methods from pre-training to post-training.
Limitations Evaluation was conducted at temperature=0.6; AIME24 and AIME25 use the average score of 32 repetitions, LiveCodeBench v5 (20240801-20250201), LiveCodeBench v6 (20250201-20250501), GPQA-Diamond, and IF-Eval use the average score of 8 repetitions, and MATH500 and SuperGPQA are single runs. The official team has not yet verified other inference engines; other features, charges, prices, platforms, effects, or usage steps not described in the materials are not yet provided.
