Qwen-VLA - Alibaba Tongyi's General Vision-Language-Action Model

What is Qwen-VLA

Qwen-VLA is a general vision-language-action model launched by the Tongyi Lab, using Qwen3.5-4B as the vision-language backbone, paired with a 1.15B-parameter DiT action decoder. Through a unified action trajectory prediction framework, the model unifies the three major tasks of manipulation, navigation, and trajectory prediction into the same model; with proprioceptive prompt conditioning, it can adapt to 11 robot platforms by only modifying the text description. The model surpasses specialized models on multiple benchmarks, achieving a 26.6% zero-shot dynamic manipulation success rate, driving embodied intelligence from skill specialists toward general actors.

Main Features of Qwen-VLA

• Cross-task unified control: A single model simultaneously handles the three major tasks of manipulation, navigation, and trajectory prediction, breaking the fragmented situation of traditional specialized models.
• Cross-embodiment plug-and-play: Supports 11 robot platforms including WidowX, Franka Panda, Mobile ALOHA, and Galaxea R1, covering configurations such as single-arm/dual-arm/mobile base, and can be switched by only modifying the text prompt.
• Zero-shot open-world generalization: Maintains high success rates across five out-of-distribution dimensions: unseen colors, instances, positions, backgrounds, and instructions, without the need for retraining for new environments.
• Dynamic object manipulation: Without dynamic training data, it zero-shot manipulates moving objects, achieving a 26.6% success rate on the DOMINO benchmark, surpassing specialized fine-tuned models.
• Long-horizon navigation: With a task-adaptive token allocation mechanism, it preserves richer episode history for long instructions, achieving 57.5% R2R and 59.6% RxR on the VLN-CE benchmark, surpassing specialized navigation models.

Technical Principles of Qwen-VLA

• Unified action trajectory prediction framework: Traditional embodied intelligence splits manipulation, navigation, and trajectory prediction into independent models, leading to limited cross-task transfer. Qwen-VLA found that the three share an isomorphic computational structure—all are "observe scene + understand instruction → predict future action sequence"—and therefore models them uniformly. The model combines the Qwen3.5-4B vision-language backbone with a 1.15B DiT action decoder. Data from different tasks is jointly supervised in the same training, and visual grounding and spatial reasoning capabilities transfer across tasks.
• Proprioceptive prompt conditioning: Faced with differences in robot hardware, traditional solutions customize independent branches for each embodiment. Qwen-VLA encodes hardware differences as structured text prompts as the only platform interface. The prompts include information such as robot model, number of arms, joint configuration, and control frequency, and are processed by the VLM. The backbone network hidden states are concatenated with noisy action chunks and fed into the DiT decoder. At inference time, switching platforms only requires replacing the prompt.
• Text-to-action DiT pretraining (T2A): Training faces the problem that the VLM is already pretrained while the DiT is initialized from scratch. Solution: freeze the VLM, train the DiT separately, and provide only text input without images. Let the DiT first learn the action distribution, text-action alignment, and proprioceptive conditioning. The computational cost of this stage is only 1/10 of multimodal training. Ablation experiments verify: 20% synthetic + 80% real data is optimal (71.1%), no images is actually better than with images, and it peaks at 2,000 steps.

How to Use Qwen-VLA

• Environment preparation: Clone the official repository from GitHub and install dependencies, then download the pretrained weights.
• Model inference: Generate a proprioceptive prompt according to the robot hardware configuration, feed images, instructions, and the prompt into the model, obtain the action sequence and send it to the robot for execution, iterating in a loop to achieve closed-loop control.
• Model Training (Advanced): Train following the four-stage process of T2A → CPT → SFT → RL, progressively improving action generation, visual perception, task adaptation, and closed-loop optimization capabilities.
• Cross-Platform Deployment: When switching robots, only the hardware description field in the prompt needs to be modified; when adapting to a new embodiment, fine-tuning on collected data during the SFT stage is sufficient.

Core Advantages of Qwen-VLA

• Generality Surpasses Specialization: A single general model surpasses the best specialized models on 3 of 5 simulation benchmarks.
• Lightweight Cross-Embodiment Adaptation: Only the text prompt needs to be modified; no need to retrain the model for each type of robot.
• Efficient Pretraining Strategy: The T2A stage costs only 1/10 of multimodal training in computation, avoiding interference with VLM pretraining results.
• Strong OOD Generalization Capability: Real-world average OOD success rate is 76.9%, surpassing π₀.₅ by +35.4pp and the no-pretraining variant by +40.7pp.
• Zero-Shot Breakthrough in Dynamic Scenarios: DOMINO benchmark zero-shot 26.6%, surpassing the specialized fine-tuned model PUMA 17.2%.

Qwen-VLA Project Address

• Project Website：https://qwen.ai/blog?id=qwenvla
• GitHub Repository：https://github.com/QwenLM/Qwen-VLA
• arXiv Technical Paper：https://arxiv.org/pdf/2605.30280

Comparison of Qwen-VLA with Similar Competitors

Application Scenarios of Qwen-VLA

• Multi-Task Industrial Robots: The same model completes heterogeneous tasks on an assembly line such as grasping, transporting, and navigating to different workstations, without deploying separate models for each task.
• Cross-Scenario Deployment of Service Robots: A home robot performs manipulation in the kitchen, autonomously navigates in the living room, and plans trajectories in the hallway, seamlessly switching task types.
• Research and Education Platforms: Researchers do not need to retrain models for each type of robot; they can quickly validate algorithms on new hardware platforms by only modifying the text prompt.
• Dynamic Environment Manipulation: Grasping moving packages on a conveyor belt in logistics warehousing, without the need to recollect training data for dynamic scenarios.
• Long-horizon complex instruction following: A museum guide robot executes multi-step long instruction tasks such as "first go to Exhibition Hall A to take photos, then bypass the crowd to go to Exhibition Hall B."