MiMo-Embodied - Xiaomi's Cross-Domain Embodied Large Model

What is MiMo-Embodied

MiMo-Embodied is the world's first open-source cross-domain embodied large model released by Xiaomi. It integrates tasks from two major domains, autonomous driving and embodied intelligence, achieving outstanding performance in environmental perception, task planning, spatial understanding, and more. The model is based on a vision-language model (VLM) architecture and, through a four-stage training strategy, including embodied intelligence supervised fine-tuning, autonomous driving supervised fine-tuning, chain-of-thought fine-tuning, and reinforcement learning fine-tuning, significantly improves cross-domain generalization ability. In the autonomous driving domain, MiMo-Embodied can accurately perceive traffic scenes, predict the behavior of dynamic targets, and generate safe and efficient driving plans; in the embodied intelligence domain, it can understand natural language instructions and complete complex task planning and spatial reasoning. MiMo-Embodied surpasses existing open-source and specialized models in multiple benchmark tests, demonstrating strong multimodal interaction capabilities.

Main Features of MiMo-Embodied

• Cross-Domain Integration Capability: MiMo-Embodied is the first model to successfully integrate autonomous driving and embodied intelligence tasks, covering core capabilities such as environmental perception, task planning, and spatial understanding, and is suitable for multimodal interaction in complex dynamic environments.
• Environmental Perception: In autonomous driving scenarios, the model can accurately understand traffic scenes, including recognizing key elements such as traffic signs, vehicles, and pedestrians, and predict their dynamic behavior to support safe driving.
• Task Planning and Execution: In the embodied intelligence domain, MiMo-Embodied can generate executable action sequences based on natural language instructions and complete complex task planning, such as robot navigation and manipulation.
• Spatial Understanding and Reasoning: The model has strong spatial reasoning capabilities, can understand the spatial relationships between objects, and supports tasks such as navigation, interaction, and scene understanding, making it suitable for robot manipulation and path planning in autonomous driving.
• Multimodal Interaction: Through deep fusion of vision and language, MiMo-Embodied can process image, video, and text inputs and supports multimodal tasks such as visual question answering, instruction following, and scene description.
• Reinforcement Learning Optimization: It uses reinforcement learning fine-tuning to improve the model's decision-making ability in complex scenarios and the reliability of task execution, ensuring efficient deployment in real environments.
• Open Source and Generality: MiMo-Embodied is fully open source, and the code and model can be obtained on Hugging Face, providing researchers and developers with powerful tools and promoting innovation in the fields of embodied intelligence and autonomous driving.

Technical Principles of MiMo-Embodied

• Cross-Domain Fusion Architecture: MiMo-Embodied adopts a unified vision-language model (VLM) architecture, integrating autonomous driving and embodied intelligence tasks into one model, and achieving deep fusion of visual input and text understanding through a visual encoder, projector, and large language model (LLM).
• Multi-Stage Training Strategy: The model gradually improves performance through four-stage training, including embodied intelligence supervised fine-tuning, autonomous driving supervised fine-tuning, chain-of-thought fine-tuning, and reinforcement learning fine-tuning, ensuring generalization ability across different tasks and scenarios.
• Visual Input Processing: It uses Vision Transformer (ViT) to encode single images, multiple images, and videos, extracts visual features, and maps them through a multilayer perceptron (MLP) into a latent space aligned with the LLM, achieving seamless fusion of vision and language.
• Data-Driven Cross-Domain Learning: A diverse dataset covering general visual-language understanding, embodied intelligence, and autonomous driving scenarios was constructed, providing rich multimodal supervision signals for the model and supporting learning from basic perception to complex reasoning.
• Reinforcement Learning Optimization: In the final stage of training, the Group Relative Policy Optimization (GRPO) algorithm is used for reinforcement learning fine-tuning, optimizing the model's decision quality and reliability for complex tasks and edge scenarios.
• Reasoning and Output Generation: Through the reasoning capabilities of the LLM, visual input is combined with language instructions to generate task-related responses and decisions, supporting path planning in autonomous driving, task execution in embodied intelligence, and more.

MiMo-Embodied Project Address

• Github Repository：https://github.com/XiaomiMiMo/MiMo-Embodied
• HuggingFace Model Library：https://huggingface.co/XiaomiMiMo/MiMo-Embodied-7B
• arXiv Technical Paper：https://arxiv.org/pdf/2511.16518

Application Scenarios of MiMo-Embodied

• Autonomous Driving: MiMo-Embodied can handle complex traffic scenarios, perform environmental perception, state prediction, and driving planning, and is suitable for various autonomous driving scenarios such as urban roads and highways, providing decision support for intelligent driving systems.
• Robot Navigation and Manipulation: In the field of embodied intelligence, the model can complete tasks such as indoor navigation and object manipulation based on natural language instructions, supporting autonomous action by robots in environments such as homes and industrial settings.
• Visual Question Answering and Interaction: Suitable for Visual Question Answering (VQA) tasks, it can understand image or video content and answer related questions, supporting information retrieval and explanation in human-computer interaction.
• Scene Understanding and Description: The model can perform semantic understanding of complex scenes and generate descriptions, and is suitable for scenario analysis in fields such as security monitoring and intelligent transportation.
• Multimodal Task Execution: It supports multimodal input such as images, videos, and text, and can handle cross-modal tasks such as instruction following and image captioning, making it suitable for intelligent assistants and automated systems.
• Task Planning in Complex Environments: In complex environments, MiMo-Embodied can generate multi-step task plans based on instructions, supporting robots in completing complex tasks such as cleaning and cooking.