LingBot-Video - Ant Lingbo's open-source video model for embodied intelligence

What is LingBot-Video

LingBot-Video is the world's first MoE video generationfoundation model for embodied intelligence, open-sourced by Ant Lingbo Technology. The model is based on a DiT + MoE architecture, with 30B total parameters but only about 3B activated, balancing large capacity with efficient inference. The model integrates 70,000 hours of internet and robotics data, and through multi-dimensional reinforcement learning reward alignment for physical plausibility and task completion, it ranks first in both RBench and Physics-IQ evaluations, and can generate robot manipulation and interaction videos that conform to real physical laws.

LingBot-Video's main features

• Text/image-to-video generation: Supports unified T2V, TI2V, and T2I generation, covering open-world scenarios and embodied tasks.
• Robot motion prediction: Generates motion sequences and contact states for robotic arms, humanoid robots, quadruped robots, etc.
• Physical simulation video generation: Simulates real physical phenomena such as material deformation, fluid motion, and light reflection.
• Simulation data synthesis: Provides large-scale, low-cost, physically plausible video data for robot policy training.
• World model research: Serves as a physical world simulator, supporting policy evaluation and action planning.

LingBot-Video's technical principles

• DiT + Sparse MoE architecture: The model uses a Single-Stream Diffusion Transformer to uniformly process visual latent codes and condition tokens, replacing Dense FFN with MoE, with 30B total parameters but only about 3B activated, achieving about 3x inference acceleration. 128 fine-grained routed experts work with shared experts, capturing different physical modes through top-K routing and avoiding subtask interference.
• Data profiling engine and 70,000 hours of embodied data: On top of massive internet video, it introduces robot data such as VLA, VLN, and Ego, covering dexterous manipulation, navigation, and first-person interaction, enabling the model to learn the intrinsic relationship between actions and environmental changes.
• Multi-dimensional reinforcement learning reward alignment: In addition to aesthetics, prompt following, and motion consistency, the training phase additionally introduces physical plausibility and task completion rewards, and uses real-world videos as preference signals, so that the generated results conform to real physical laws and robot task requirements.
• Unified input and 3D RoPE: T2I/T2V/TI2V tasks are unified into a single sequence of tokens, and through multimodal 3D MM-RoPE, condition tokens and visual tokens are mapped to non-overlapping spatiotemporal coordinates, eliminating the need for task-specific architectures while preserving spatial locality and temporal order.
• Cascaded generation and stable training: The model adopts a base generator + refiner cascaded design, where the base generates compact latent codes and the refiner enhances details. Combined with QK-Norm, AdaLN-Single modulation, and an auxiliary-loss-free sequence-level load balancing strategy, it ensures stable training for long sequences and high resolutions.

Follow WeChat and reply "open source" to join theAI open-source project exchange group

How to use LingBot-Video

• Environment preparation: Ensure that Python 3.10+ and a CUDA environment are configured locally or on the server, and reserve sufficient GPU memory.
• Clone the repository: Pull the official LingBot-Video code repository from GitHub to the local working directory.
• Install dependencies: Install the required deep learning and inference libraries according to the project's requirements.txt.
• Download weights: Obtain the model checkpoint files of the corresponding scale from HuggingFace or ModelScope.
• Prepare prompts: Organize text descriptions, reference images, or structured control signals into an input format supported by the model.
• Run inference: Load the model and execute the generation script to output embodied intelligence or open-world videos.

Core advantages of LingBot-Video

• The world's first open-source embodied video MoE model: Fills the gap in open-source video foundation models for robots.
• Decoupling of capacity and efficiency: With a 30B parameter scale, only 3B are activated, and inference speed is approximately 3 times that of a Dense architecture of the same scale, suitable for real-time interaction needs.
• Leading physical plausibility: Ranked first on both RBench (0.620) and Physics-IQ Verified, surpassing Wan2.6, Seedance 1.5 Pro, Cosmos 3, etc.
• Data specialization: 70,000 hours of robot data injection enables the model to understand action-environment causal relationships, rather than just visual style.
• Unified task framework: T2I/T2V/TI2V are handled in a unified manner by a single model, reducing multi-task deployment costs.

Project address of LingBot-Video

• Project official website：https://technology.robbyant.com/lingbot-video
• GitHub repository：https://github.com/Robbyant/lingbot-video
• HuggingFace model library：https://huggingface.co/collections/robbyant/lingbot-video
• arXiv technical paper：https://arxiv.org/pdf/2607.07675

Comparison of LingBot-Video with similar competing products

Application scenarios of LingBot-Video

• Robotic simulation training: Generate physically plausible manipulation videos for robotic arms and humanoid robots, replacing expensive real-world data collection.
• Simulation data synthesis: Batch-generate diverse scenarios and motion trajectories for policy pretraining and data augmentation.
• World model research: Serve as a physical world predictor to support imagination-based robot planning and decision-making.
• Autonomous driving simulation: Generate physically consistent road interactions and ego-view videos for perception algorithm validation.
• Service and manufacturing: Simulate human-robot interaction workflows in warehousing, healthcare, and home service scenarios to optimize task strategies.