What is ACE-Ego
ACE-Ego is an open-source one-brain-multi-body embodied manipulation VLA model developed by Daxiao Robotics in collaboration with CUHK MMLab. The model is pre-trained primarily on 6.0K+ hours of first-person human videos, and through camera-space action and morphology-conditioned encoding, the same model can adapt to different robot bodies. It refreshes SOTA on the RoboCasa GR1 TableTop benchmark with 72.8%, and can stably complete complex retail operations such as plastic bag packaging and placing shoes into shoeboxes, greatly reducing multi-body deployment costs.
Main Features of ACE-Ego
- One-Brain-Multi-Body Adaptation: The same VLA model adapts to different robot bodies through morphology-conditioned encoding, with no need to train separately for each robot.
- Complex Manipulation Execution: Supports long-horizon, high-contact retail operations, such as plastic bag packaging and placing shoes into shoeboxes and other fine-grained tasks.
- Dual-Arm Coordinated Control: Achieves a 90.62% success rate on the highly challenging dual-arm manipulation benchmark RoboTwin 2.0, supporting complex two-handed collaboration.
- Open Open-Source Ecosystem: The model and related resources are open-sourced to the industry, lowering the barrier to embodied intelligence R&D.
Technical Principles of ACE-Ego
- Human-Centric Pre-training: Integrates 6.0K+ hours of first-person human videos, multi-body robot demonstrations, and simulation data for pre-training, uses videos of everyday human operations to build general manipulation representations, and achieves skill transfer through fine-tuning with a small amount of robot data.
- Camera-Space Actions: Unifies action prediction into the camera coordinate system, eliminating the action space inconsistency caused by differences in robot bodies and camera viewpoints, enabling the model to output unified action commands across morphologies.
- Morphology-Conditioned Encoding: By encoding robot morphology information, such as joint structure and degrees-of-freedom configuration, the model understands the physical constraints of different bodies and adapts to multiple robot hardware within the same model.
- Reliability-Aware Training: Introduces a reliability assessment mechanism to distinguish high-confidence and low-confidence action predictions during training, improving the model's stability and safety in complex contact operations.
Follow WeChat and reply "Open Source" to jointhe AI Open-Source Project Exchange Group
How to Use ACE-Ego
- Environment Preparation: Clone the official GitHub repository and install the Python dependency environment, ensuring the system supports PyTorch and CUDA.
- Model Download: Obtain the ACE-Ego pre-trained weight files from open-source channels, and load the human-centric pre-trained VLA foundation model.
- Body Configuration: Enter the target robot's URDF structure, joint degrees of freedom, and camera parameters through the morphology-conditioned encoding interface to complete one-brain-multi-body adaptation.
- Camera Calibration: Calibrate the intrinsic and extrinsic parameters of the RGB-D camera mounted on the robot to establish a unified camera-space action coordinate system.
- Data Fine-Tuning (Optional): Collect a small amount of robot demonstration data for a specific task, and perform lightweight fine-tuning combined with the human video pre-trained weights.
- Visual Input: Input the robot's real-time camera footage and language instructions, such as "put the shoes into the shoebox," into the model.
- Action Inference: The model outputs an end-effector action sequence in camera space, including position, orientation, and gripper opening/closing degree.
- Mapping Execution: Converts camera-space action commands into joint control signals for the target robot body, driving the hardware to complete operations.
- Reliability Verification: Enables the reliability-aware module to filter out low-confidence actions, ensuring safe and stable execution of long-horizon complex operations.
Core Advantages of ACE-Ego
- Leading Performance: RoboCasa GR1 TableTop 72.8%, setting a new SOTA, greatly surpassing mainstream models such as π0.5 (37.0%) and GR00T-N1.6 (47.6%).
- Cross-Embodiment Generalization: The one-brain-multi-body architecture supports the same model adapting to different robots, reducing multi-embodiment deployment costs.
- Complex Task Capability: Performs stably in strong-contact, long-horizon retail operations, covering the full pipeline from product organization to packaging and fulfillment.
- High Data Efficiency: Pretrained on human videos, reducing reliance on expensive robot demonstration data.
ACE-Ego Project Links
- Project Website：https://acerobotics-vla.github.io/ACE-Ego/
- GitHub Repository：https://github.com/ACERobotics-VLA/ACE-Ego-0
- Technical Paper：https://acerobotics-vla.github.io/ACE-Ego-0/paper.html
Comparison of ACE-Ego with Similar Competitors
| Dimension | ACE-Ego | π0.5 |
|---|---|---|
| Publisher | Daxiao Robotics × CUHK MMLab | Physical Intelligence |
| Positioning | One-brain-multi-body embodied manipulation VLA | General flow-matching VLA model |
| RoboCasa GR1 | 72.8%（SOTA） | 37.0% |
| Pretraining Data | 6.0K+ hours of human videos + robot data | Large-scale multi-embodiment robot data |
| Core Features | Human-centric, camera-space actions, embodiment-conditioned encoding | Flow-matching action generation, large-scale pretraining |
| Open-source Status | Open source | Closed source (API/model weights restricted) |
| Embodiment Adaptation | Native support for one-brain-multi-body | Requires separate fine-tuning for each embodiment |
Application Scenarios of ACE-Ego
- Retail Fulfillment: Automation of offline retail operations such as product organization, plastic bag packaging, and shoe boxing in supermarkets, convenience stores, and other scenarios.
- Logistics and Warehousing: Intelligentization of logistics processes requiring strong-contact operations such as item sorting, boxing, and palletizing in warehouses.
- Home Services: Assistance with daily household chores such as item storage, organization, and simple cleaning in home environments.
- Commercial Services: Item handling, display maintenance, and countertop organization in scenarios such as shopping malls, hotels, and restaurants.
- Industrial Manufacturing: Fine industrial operations such as component assembly, tool picking and placing, and material transfer on production lines.