What is UniAct
UniAct is a novel embodied foundation model framework that addresses the issue of behavioral heterogeneity among different robots. By learning universal behaviors, it captures the atomic behavior features shared by different robots and eliminates the behavioral heterogeneity caused by differences in physical morphology and control interfaces. UniAct's architecture includes a universal action extractor, a universal action space, and heterogeneous decoders. The universal action extractor is based on a vision-language model and extracts universal actions by observing and task goals; the universal action space is implemented in the form of a vector-quantized codebook, where each vector represents an atomic behavior; the heterogeneous decoders translate universal actions into control signals specific to a particular robot.
UniAct's main features
- Universal action encoding: UniAct uses vector quantization to form a universal codebook from the atomic behaviors of different robots (such as "move to the target position" or "avoid obstacles"), where each token represents a universal skill that can be shared across platforms.
- Lightweight architecture and efficient performance: The UniAct-0.5B model requires only 50 million parameters, and in task tests in real and simulated environments, its performance has already surpassed the OpenVLA model with 1.4 billion parameters.
- Rapid adaptation to new environments and robots: UniAct requires only 50 specialized demonstration data samples to complete fine-tuning of the model in a new environment, and can quickly adapt to new robots and control interfaces. By adding new lightweight decoders, it can be easily extended to new robot platforms.
- Cross-domain data utilization: Through the universal action space, UniAct can better utilize cross-domain data for training and achieve more efficient generalization across different robots and environments.
- Consistent behavior patterns: Across different deployment scenarios and robot types, the same universal action can exhibit consistent behavior patterns. This provides a new and convenient way to control embodied agents: operators only need to select the corresponding universal action from the codebook to command different types of robots to complete tasks.
UniAct's technical principles
- Universal Action Space: UniAct constructs a discrete universal action space through vector quantization. This space is designed as a vectorized codebook, where each vector embedding represents a universal atomic behavior. These atomic behaviors are basic behavior patterns shared by different robots in different situations, such as "move to the target position" or "avoid obstacles." In this way, UniAct can uniformly represent the actions of different robots and eliminate the heterogeneity of the action space.
- Universal Action Extractor: UniAct uses a universal action extractor based on a vision-language model (VLM) to identify and extract universal actions. Given observations and task goals, the extractor outputs the probability of selecting a universal action. In this way, UniAct can extract from complex visual and language inputs the universal actions directly related to task progress, rather than merely identifying observed changes. This extraction method avoids interference from external factors (such as environmental changes or human intervention) and ensures the purity and consistency of the action space.
- Heterogeneous Decoders: To translate universal actions into executable commands for specific robots, UniAct introduces heterogeneous decoders. These decoders are designed for different robot platforms and can convert universal actions into specific control signals based on the robot's specific characteristics (such as joint torques or camera viewpoints). In this way, UniAct can flexibly adapt to different types of robots and achieve efficient cross-platform control.
- Lightweight Architecture and Efficient Training: UniAct adopts a lightweight model architecture, maintaining good performance even under low-parameter settings. UniAct is trained through behavior cloning loss and selects an appropriate loss function based on the nature of the action labels (such as cross-entropy for discrete actions and mean squared error for continuous actions). This efficient training approach enables UniAct to quickly adapt to new environments and new robots.
UniAct Project Address
- Project Website：https://2toinf.github.io/UniAct/
- Github Repository：https://github.com/2toinf/UniAct
- arXiv Technical Paper：https://arxiv.org/pdf/2501.10105
UniAct Application Scenarios
- Autonomous Driving and Intelligent Transportation: UniAct's technical principles and architecture are also applicable to the field of autonomous driving. By learning universal driving behavior patterns, UniAct can provide more efficient action planning and control for autonomous driving systems.
- Medical Robots: UniAct can be applied in the field of medical robots, such as assistive rehabilitation robots or surgical robots. Through the universal action space, medical robots can more flexibly adapt to different patient needs and surgical scenarios.
- Industrial Automation: In the field of industrial automation, UniAct can be used to control various industrial robots and achieve efficient production process optimization. By quickly adapting to different robot platforms and task requirements, UniAct can significantly improve the flexibility and efficiency of industrial production.
- Smart Homes and Service Robots: UniAct can be applied in the fields of smart homes and home service robots. Through the universal action space, service robots can interact with humans more naturally and complete various household tasks.