What is HOVER
HOVER is a 1.5M small model released by NVIDIA, with the full name "Humanoid Versatile Controller," a versatile whole-body neural general controller for humanoid robots. The model uses 1.5 million parameters to control complex robot movements and can adapt to diverse tasks such as navigation, tabletop manipulation, and mobile manipulation, each requiring a unique control mode. HOVER's core innovation lies in using whole-body motion imitation as a general abstraction for various tasks, providing general motor skills for learning multiple whole-body control modes. Based on a multi-mode policy distillation framework, it integrates different control modes into a unified policy, enabling seamless switching between modes while preserving the strengths of each mode, improving the efficiency and flexibility of future humanoid robot applications.
HOVER's main features
- Multi-mode control: HOVER can seamlessly switch between different control modes, such as navigation, tabletop manipulation, and mobile manipulation, each with specific control requirements.
- Kinematic position tracking: HOVER supports tracking the 3D positions of key robot parts to achieve precise motion replication.
- Joint angle tracking: HOVER can track the target joint angles of each robot motor to achieve complex limb movements.
- Root tracking: HOVER can track the robot's root velocity, height, and orientation, which is crucial for maintaining balance and performing dynamic movements.
- Unified command space: HOVER designs a unified command space that adapts to various control devices, such as joysticks, motion capture systems, exoskeletons, and virtual reality headsets.
- Policy distillation: HOVER extracts and distills from Oracle policies, integrating multiple control skills into a single "general policy," improving efficiency and flexibility.
HOVER's technical principles
- Proprioception and command masking: HOVER determines task commands based on proprioception (the robot's own state perception) and command masking (selectively activating different command spaces), supporting independent control of the robot's upper body and lower body.
- Multi-mode policy distillation framework: HOVER uses policy distillation technology to transform an Oracle policy learned from large-scale human motion data into a single policy capable of handling multiple control modes.
- DAgger framework: HOVER adopts the DAgger (Dataset Aggregation) framework to align the actions of the student policy and the Oracle policy, optimizing the student policy based on supervised learning.
- Simulation training: HOVER's training is conducted in the NVIDIA Isaac simulation suite, which is GPU-accelerated and can quickly simulate long training durations, transferring the model to the real world without additional fine-tuning.
- Unified command space design: HOVER's command space design is both general and atomic, covering most existing control configurations and supporting arbitrary combinations for various modes.
- Motion retargeting: HOVER can retarget human motion datasets into robot motion datasets, achieved by matching keypoints and optimizing parameters.
HOVER's project addresses
- Project website：hover-versatile-humanoid.github.io
- arXiv technical paper：https://arxiv.org/pdf/2410.21229
HOVER's application scenarios
- Navigation and mobility: HOVER can control humanoid robots to navigate and move in complex environments, such as walking on uneven ground or moving through narrow spaces, making it suitable for scenarios such as search and rescue and guided tours.
- Desktop manipulation: In desktop tasks that require fine manipulation, such as assembling electronic components or conducting laboratory work, HOVER can precisely control the robot's upper limbs and hand joints to achieve complex hand-eye coordination operations.
- Loco-manipulation: For tasks that combine movement and manipulation, such as grasping and carrying objects while moving, HOVER can coordinate the robot's whole-body movements to achieve smooth carrying and manipulation tasks.
- Human-robot interaction: HOVER supports various input devices (such as VR headsets, motion capture systems, and exoskeletons) for human-robot interaction, enabling robots to imitate human actions and behaviors, making it suitable for fields such as entertainment, education, and training.
- Teleoperation and simulation: In teleoperation scenarios, HOVER can respond to the operator's commands in real time and perform precise remote tasks, such as remote surgical assistance or operations in hazardous environments.