What is UnifoLM-VLA-0
UnifoLM-VLA-0 is a general-purpose vision-language-action (VLA) large model open-sourced by Unitree Robotics, built on the Qwen2.5-VL-7B architecture through continual pre-training. By integrating multi-dimensional supervision signals such as 2D/3D spatial perception and trajectory prediction, the model achieves evolution from "visual-language understanding" to "embodied intelligent agent". With a single policy, the model can complete 12 categories of complex humanoid robot manipulation tasks, including tidying desktops, folding towels, and sorting fruits, achieving an average accuracy of 98.7% on the LIBERO simulation benchmark, demonstrating strong spatial reasoning capabilities and cross-task generalization performance.
Main Features of UnifoLM-VLA-0
- General-Purpose Robotic Manipulation: As an end-to-end vision-language-action model, UnifoLM-VLA-0 can control humanoid robots via natural language instructions to complete complex manipulation tasks, including 12 categories of multi-step long-horizon tasks such as tidying items, folding towels, sorting fruits, returning tools to their places, and wiping desktops.
- Spatial Perception and Reasoning: The model possesses strong 2D/3D spatial understanding capabilities, including object detection and segmentation, 3D bounding box prediction, spatial relationship reasoning (e.g., "the pencil on the left"), affordance reasoning (graspable position determination), and trajectory planning.
- Single-Policy Multi-Task: Only a single model weight is needed to generalize to different scenarios and tasks, without the need for separate training for each task, demonstrating excellent cross-task transfer capability.
Technical Principles of UnifoLM-VLA-0
- Model Architecture: Built on the open-source Qwen2.5-VL-7B vision-language large model, with an added Action Head for action prediction, constructing an end-to-end vision-language-action architecture that directly outputs robot control actions from visual perception and natural language understanding.
- Continual Pre-Training: The model undergoes continual pre-training on multi-task datasets covering robotics and general scenarios, integrating diverse supervision signals such as 2D detection and segmentation, hierarchical task decomposition, 3D object detection, spatial reasoning, and trajectory prediction, strengthening the model's multimodal perception and physical understanding capabilities.
- Action Modeling: The model introduces an action chunk prediction mechanism, simultaneously applying forward and inverse dynamics constraints to uniformly model action sequences, enabling the model to deeply understand the physical interaction dynamics between the robot and objects, and supporting long-horizon action planning and decision-making.
- Spatial Enhancement: Through deep fusion of text instructions with 2D/3D spatial details, it establishes precise alignment between semantic logic and geometric space, significantly improving spatial perception and geometric understanding capabilities, meeting the requirements of manipulation tasks for instruction understanding and spatial reasoning.
Project Address of UnifoLM-VLA-0
- Project Official Website：https://unigen-x.github.io/unifolm-vla.github.io/
- GitHub Repository：https://github.com/unitreerobotics/unifolm-vla
Application Scenarios of UnifoLM-VLA-0
- Home Services: The model can perform daily household chores such as tidying desktops, folding towels, and wiping stains.
- Office Assistance: The model can complete office environment maintenance tasks such as organizing stationery, packing backpacks, and returning tools to their places.
- Healthcare: Supports medical assistance scenarios requiring fine manipulation, such as opening medicine bottles and dispensing medications.
- Education and Training: Can be used for structured cognitive teaching demonstrations such as sorting items by color and stacking building blocks.
- Industrial Sorting: The model supports automated sorting by placing items such as fruits and parts into designated areas according to rules.