What is VLAC
VLAC is an embodied reward large model released by Shanghai Artificial Intelligence Laboratory. Built on the InternVL multimodal large model, it integrates internet video data and robot manipulation data to provide process rewards and task completion estimates for reinforcement learning of robots in the real world. VLAC can effectively distinguish between normal progress and abnormal/stalled behavior, and supports few-shot rapid generalization through in-context learning. It has local smoothness and a negative reward mechanism, ensuring the stability and effectiveness of reinforcement learning. VLAC not only outputs reward signals but also outputs robot action instructions, helping robots learn autonomously in the real world and quickly adapt to new scenarios. VLAC supports human-machine collaboration modes, further improving training efficiency.
Main Features of VLAC
-
Provides process rewards and completion estimates: Provides continuous, reliable supervision signals for reinforcement learning of robots in the real world, determines whether a task is completed and estimates completion progress.
-
Distinguishes normal and abnormal behavior: Effectively identifies normal progress, abnormal, or stalled behavior in robot manipulation, avoiding ineffective exploration.
-
Supports few-shot rapid generalization: Through in-context learning, achieves few-shot rapid generalization, improving the model's adaptability in new scenarios.
-
Outputs robot action instructions: While providing reward signals, it can also output action instructions for the robot to execute, helping the robot learn autonomously and adjust behavior.
-
Builds a reinforcement learning framework: The VLA reinforcement learning framework built around VLAC enables robots to quickly adapt to new scenarios during real-world interaction, improving task success rates.
-
Supports human-machine collaboration modes: Through multiple human-machine collaboration paradigms, further improves training flexibility and reinforcement learning efficiency.
Technical Principles of VLAC
-
Multimodal fusion: Based on the InternVL multimodal large model, it integrates visual, language, and other multimodal data, improving comprehensive understanding of tasks and environments.
-
Data-driven reward generation: Utilizes internet video data and robot manipulation data to generate dense reward signals through learning, providing stable feedback for reinforcement learning.
-
Task progress estimation: Through the model's real-time understanding of tasks, estimates task completion progress, providing process rewards for reinforcement learning.
-
Abnormal behavior detection: By analyzing robot manipulation data, identifies abnormal or stalled behavior, avoiding ineffective exploration and improving learning efficiency.
-
In-context learning mechanism: Supports in-context learning, quickly adapting to new tasks through a small number of samples, improving the model's generalization ability.
-
Action instruction generation: While providing reward signals, generates robot action instructions, realizing closed-loop control from perception to action.
-
Reinforcement learning framework integration: Builds a VLA reinforcement learning framework, combining process rewards and task completion status to improve robots' learning and adaptability in the real world.
-
Human-machine collaboration enhancement: Through human-machine collaboration modes, such as expert data replay and manual assistance in exploration, further optimizes the model training process.
VLAC Project Address
- Project Website：https://vlac.intern-ai.org.cn
- Github Repository：https://github.com/InternRobotics/VLAC
- HuggingFace Model Library：https://huggingface.co/InternRobotics/VLAC
VLAC Application Scenarios
-
Robot Reinforcement Learning: Provides process rewards and task completion estimation for robot reinforcement learning in the real world, helping robots quickly adapt to new tasks and environments.
-
Human-Robot Collaboration Tasks: Supports human-robot collaboration modes, improving the flexibility and efficiency of robot training through expert data replay, manual assistance in exploration, and other methods.
-
Multi-Robot Collaborative Learning: In multi-robot environments, through the VLA reinforcement learning framework, enables multiple robots to interact and learn in the real world simultaneously, improving task success rates.
-
Complex Task Decomposition and Learning: Decomposes complex tasks into multiple subtasks, provides reward signals for each subtask, and helps robots gradually complete complex tasks.
-
Rapid Adaptation to New Scenarios: Through few-shot rapid generalization capability, enables robots to quickly learn and adapt in new scenarios, improving task completion rates.