Wall-OSS-0.5 - A domestic embodied intelligence model open-sourced by X自变量机器人

What is Wall-OSS-0.5

Wall-OSS-0.5 is a domestic embodied intelligence model open-sourced by X自变量机器人, belonging to the vision-language-action (VLA) large model category, with "zero-shot" capability to be deployed on real robots without task-specific fine-tuning. Through the innovative Gradient-Bridge collaborative training method, the model integrates action capabilities into the foundation model backbone, enabling robots to directly execute 17 real tasks such as transport and sorting, among which 4 tasks achieve a completion rate exceeding 80%.

Main Features of Wall-OSS-0.5

• Zero-shot real robot operation: Pretrained checkpoints can be directly deployed to physical hardware, executing operations such as grasping, sorting, and organizing without task-specific fine-tuning, achieving "deploy upon pretraining".
• Unified adaptation across multiple embodiments: A single pretrained checkpoint can adapt to over 20 robot embodiments, including desktop dual-arm and mobile manipulation, breaking the limitation of traditional models that require separate training for each hardware type.
• Three-objective collaborative training: Through the complementary optimization of discrete action prediction, multimodal prediction, and continuous flow matching, a "gradient bridge" is formed, allowing action learning and vision-language understanding to not interfere with each other and to improve collaboratively.
• Mixture of Experts architecture (MoT): Vision-language tokens are routed and processed by the VL Expert, while continuous action computation is handled by the Action Expert, with a joint attention mechanism enabling end-to-end gradient flow, balancing language understanding and action generation.
• Preservation of vision-language capabilities: Action training does not erode the foundation VLM capabilities; embodied grounding ability improves by 21.8 percentage points, and general vision-language understanding and reasoning capabilities are fully preserved.
• Efficient fine-tuning adaptation: After fine-tuning on the LeRobot dataset, the average progress on 15 real robot tasks reaches 60.5%, leading the comparable model π0.5 by 17.5 percentage points.
• Strong generalization and long-horizon execution: It maintains high progress on 3 unseen tasks (such as deformable object manipulation) and supports multi-stage continuous operations (such as long-horizon tasks like rope tightening, cloth folding, and fruit basket organizing).

Technical Principles of Wall-OSS-0.5

• 4B-parameter VLA architecture: Built on the 3-billion-parameter Qwen2.5-VL vision-language model as the backbone, with an action generation component added, totaling 4 billion parameters, achieving end-to-end unification of visual perception, language understanding, and robot action generation.
• Mixture of Experts routing (MoT): It adopts a Mixture-of-Experts architecture, where vision-language tokens are routed and processed by the VL Expert, and continuous action computation is processed by an independent Action Expert; the two achieve end-to-end gradient flow through a joint attention mechanism, preventing action training from interfering with language understanding.
• Gradient-Bridge collaborative training: It innovatively uses discrete action prediction, multimodal prediction, and continuous flow matching as complementary optimization objectives for collaborative training, forming a "gradient bridge" that allows action policy learning and vision-language pretraining knowledge to mutually reinforce rather than conflict.
• Continuous flow matching action generation: It uses Flow Matching technology to model the continuous action space; compared with traditional discretization or regression methods, it can more precisely generate smooth, high-dimensional robot control signals.
• Design of deployment upon pretraining: Through large-scale heterogeneous data pretraining, the model acquires general physical manipulation priors during the pretraining stage, and the checkpoint can directly perform zero-shot inference on real robot hardware without task-specific post-fine-tuning processing.
• Large-scale heterogeneous training data: Covering more than 20 robot embodiments, with over 1 million real robot trajectories per training round; simultaneously integrating 90 million multimodal corpus samples (including 12 million entity bridging samples), achieving task-balanced sampling of approximately 60% self-collected data and 40% open-source data.
• Joint attention and end-to-end optimization: The VL Expert and Action Expert share the attention computation graph, ensuring deep coupling of language instructions, visual scenes, and action outputs at the feature level, with gradients propagating bidirectionally across modalities.
• Capability decoupling preservation mechanism: Through three-objective collaboration and expert separation design, action training does not erode the foundational VLM capabilities, entity grounding capability improves by 21.8 percentage points, while fully preserving general visual-language understanding and reasoning capabilities.

How to use Wall-OSS-0.5

• Environment preparation: Create a conda environment with Python 3.10, install basic dependencies such as PyTorch, and ensure Flash Attention 2.7.4 or above is installed to accelerate inference.
• Install the LeRobot library: Clone the Hugging Face LeRobot repository, checkout to the specified compatible version, then execute pip install -e . to complete installation.
• Install the Wall-X toolchain: Clone the wall-x open-source repository, run git submodule update --init --recursive to pull submodules, then execute the installation command to complete toolchain deployment.
• Download pretrained weights: From Hugging Face (x-square-robot/wall-oss-0.5) download the officially released pretrained model checkpoint.
• Configure robot parameters: According to the target robot type (such as desktop dual-arm, mobile manipulation, etc.), set the degrees of freedom (DOF) configuration, model path, data path, and training hyperparameters.
• Execute fine-tuning (optional): If optimization for a specific task is needed, run the fine-tuning script (such as bash ./workspace/lerobot_example/run.sh) on a dataset in LeRobot format to improve task performance.
• Deploy to real hardware: Load the pretrained or fine-tuned checkpoint, perform zero-shot inference or post-fine-tuning inference on real robot hardware, and directly output an executable robot control policy.

Core advantages of Wall-OSS-0.5

• Ready to deploy: Unlike traditional VLA models that must be fine-tuned before use, the pretrained checkpoint directly produces an executable robot policy.
• Efficient adaptation: After fine-tuning on 15 real robot tasks, the average progress is 60.5%, which is 17.5 percentage points higher than π0.5's 43.0%.
• Strong generalization ability: It still maintains high progress on 3 unseen tasks (such as deformable object manipulation), reaching 82% on the rope tightening task.
• Preservation of visual-language capabilities: Action training does not erode the foundational VLM capabilities, entity grounding capability improves by 21.8 percentage points, and general VL capabilities are fully preserved.
• Leading data scale: Covering 20+ robot embodiments, 1 million+ trajectories per round, and 90 million+ multimodal samples.

Project address of Wall-OSS-0.5

• Project address：https://x2robot.com/oss#resources
• GitHub repository：https://github.com/X-Square-Robot/wall-x
• Paper address：https://x2robot.com/api/files/file/wall_oss_05.pdf

Comparison of Wall-OSS-0.5 with similar competitors

Application scenarios of Wall-OSS-0.5

• Home services: Daily household operations such as block sorting, fruit classification, drawer organization, and ring stacking.
• Deformable object handling: Deformable object manipulation tasks such as rope tightening, cloth folding, and flower arranging.
• Industrial assembly: Repetitive manufacturing processes such as precision part grasping, color classification, and tool returning.
• Dual-arm collaboration: Complex assembly, organization, and long-horizon combined actions that require both hands working together.
• Long-horizon task execution: Multi-stage continuous operations, such as basket organizing and other tasks that require sustained attention.