What is Being-M0
Being-M0 is the first large-scale general motion generation model for humanoid robots jointly launched by Peking University, Renmin University of China, and other institutions. Being-M0 is based on MotionLib, the industry's first million-level motion dataset, and uses the innovative MotionBook encoding technology to transform motion sequences into two-dimensional images for efficient representation and generation. Being-M0 validates the technical feasibility of big data + large models in the field of motion generation, significantly improves the diversity of motion generation and the accuracy of semantic alignment, and achieves efficient transfer from human motion to multiple humanoid robots, laying the foundation for general motion intelligence.
Main features of Being-M0
- Text-driven motion generation: Generates corresponding human motion sequences based on input natural language descriptions.
- Motion transfer: The model transfers generated human motions to multiple humanoid robot platforms, such as Unitree H1, H1-2, G1, etc., enabling the robots to perform similar human motions.
- Multimodal data support: Being-M0 supports multiple modalities of data, including RGB video, depth information, etc., and can handle multi-person interaction scenarios, providing support for motion generation in complex scenarios.
Technical principles of Being-M0
- Large-scale dataset MotionLib: Based on MotionLib, the industry's first million-level motion generation dataset, the dataset contains more than 1.2 million motion sequences, 15 times that of the largest existing public dataset. The dataset is collected from public datasets and online platforms based on an automated processing pipeline and undergoes fine-grained annotation.
- MotionBook encoding technology: Based on MotionBook encoding technology, motion sequences are modeled as single-channel two-dimensional motion images, with independent encoding spaces constructed along the time axis and joint axis respectively, fully preserving the multidimensional structural characteristics of motion. Dimensionality reduction projection eliminates the traditional codebook lookup step, increasing the capacity of the motion vocabulary by two orders of magnitude.
- Two-stage optimization + learning solution: In terms of motion transfer, a two-stage optimization and learning solution is used. The optimization stage generates motion sequences that satisfy the robot's kinematic constraints based on a multi-objective optimization method; the learning stage uses a lightweight MLP network to learn the mapping relationship from human motion to humanoid robot motion, achieving efficient motion transfer.
Project address of Being-M0
- Project official website：https://beingbeyond.github.io/Being-M0/
- arXiv technical paper：https://arxiv.org/pdf/2410.03311
Application scenarios of Being-M0
- Humanoid robot control: Enables robots to complete motions according to text instructions.
- Animation production: Quickly generates high-quality motion sequences, improving production efficiency.
- Virtual reality and augmented reality: Generates interactive motions in real time, enhancing immersion.
- Human-robot interaction: Enables robots to better understand human instructions and improve collaboration efficiency.
- Motion analysis and rehabilitation: Assists athlete training and patient rehabilitation.