MiniCPM-Robot - ModelBest's Open-Source Embodied Intelligence VLA Model Series

What is MiniCPM-Robot

MiniCPM-Robot is ModelBest's open-source lightweight embodied intelligence VLA model series, including the 1.5B-parameter MiniCPM-RobotManip general manipulation model and the 0.9B-parameter MiniCPM-RobotTrack tracking and navigation model, accompanied by the PhyAI inference framework. The models employ visual token compression and streaming context memory technology, support long-horizon fine manipulation by robotic arms and indoor/outdoor target tracking by robot dogs, and can run locally on-device in weak-network environments with inference latency as low as 120ms, ranking among the top tier in mainstream benchmarks.

Main Features of MiniCPM-Robot

• MiniCPM-RobotManip: Controls robotic arms to complete long-horizon manipulation tasks (such as making a sandwich), supports multi-camera input, and has context memory capability, able to remember task progress and historical actions.
• MiniCPM-RobotTrack: Enables a robot dog to continuously track a specified target according to natural language instructions, supports single-target tracking, multi-person interference tracking, and ambiguous target tracking, and can run locally in weak-network/no-network environments.

Technical Principles of MiniCPM-Robot

• Visual Token Compression: Filters out redundant visual information irrelevant to the task, reducing the computation per inference.
• Streaming Context Memory: Reuses already-processed historical information, so new frames only require incremental computation, avoiding explosive growth in computation for long tasks.
• Self-Evolving Data Pipeline: Proactively exposes the model's weak points in simulation and real environments, collecting error scenarios for expert correction and closed-loop training.

Follow WeChat and reply "Open Source" to join theAI Open Source Project Communication Group

How to Use MiniCPM-Robot

• Environment Preparation：
  • Clone the GitHub repository and install dependencies.
  • Prepare hardware: a robotic arm (RobotManip) or a Unitree Go2 Edu robot dog (RobotTrack).
• Model Loading: Download the corresponding model weights from Hugging Face. Load the model through the PhyAI framework to complete quantization and operator optimization.
• Camera Connection: Configure the wrist camera and main-view camera (RobotManip) or the robot dog's original camera (RobotTrack).
• Instruction Input: Input natural language instructions (such as "follow the person in black ahead" or "make a sandwich").
• One-Click Launch: Run the open-source script; the model receives visual and language input, outputs robot action instructions, and executes them.

Core Advantages of MiniCPM-Robot

• Extremely Small Parameters, Top-Tier Performance: With 1.5B parameters, it rivals larger models such as π0.5 in benchmarks including LIBERO, Calvin, and RoboTwin2; its RMBench context memory score is about 53 points, far exceeding π0.5's about 10 points.
• Halved Inference Latency: Single-frame decision latency on H100 is about 120ms, nearly half of π0.5 (234ms).
• Pure On-Device Operation: The robot dog's local compute alone can stably track at over 5Hz, and it can continue working even when disconnected from the network.