Xiaomi Auto World Model - An Assisted Driving World Model Launched by Xiaomi

What Is Xiaomi Auto World Model

Xiaomi Auto World Model is an assisted driving world model launched by Xiaomi Auto, and is the first to deeply couple 3D reconstruction (WorldRec) with video generation (WorldGen). WorldRec uses sparse 3D anchors to achieve rapid reconstruction of a 10-second video in 10 seconds, while WorldGen achieves 4-step denoising and 0.19 seconds/frame generation through two-stage training and ODE distillation, supporting videos up to 1 minute long. It achieves SOTA on both Waymo reconstruction accuracy (PSNR 28.48) and nuScenes generation quality (FVD 64.97), and has been deployed in three major business scenarios: synthetic data, simulation testing, and the assisted driving academy.

Main Features of Xiaomi Auto World Model

• WorldRec 3D Reconstruction Module: Uses sparse 3D query anchors instead of traditional dense Gaussians to achieve multi-view multi-timestamp feature aggregation and visibility-weighted fusion. Reconstructing a 10-second video takes only 10 seconds, eliminating multi-view conflicts and ghosting issues.
• WorldGen Video Generation Module: Based on two-stage training with fully bidirectional temporal attention pretraining plus causal fine-tuning, it compresses denoising steps from 50 to 4 through ODE distillation, generating future frames, unobserved viewpoints, and occluded region content at 0.19 seconds/frame, supporting continuous generation of up to 81 frames (about 1 minute).
• Deep Reconstruction-Generation Coupling: The reconstruction side provides 3D geometric priors to constrain generation stability, while the generation side extends the reconstruction boundary to unobserved spatiotemporal regions. The two correct each other and suppress long-sequence drift.
• Extreme Scenario Generation: Supports high-quality synthesis of extreme weather such as heavy rain, heavy snow, and dense fog, as well as long-tail dangerous scenarios such as animal intrusions, providing scarce data for perception model training.
• Three Major Business Deployments: Synthetic data generation (over 100,000 clips delivered), closed-loop simulation testing (reproducing real accidents for targeted optimization), and the assisted driving academy (dynamically generating first-person driving instructional videos).

Technical Principles of Xiaomi Auto World Model

• WorldRec Sparse Anchor Representation: Abandons the per-pixel Gaussian prediction paradigm and instead uses sparse 3D query points as scene representation. Each anchor actively aggregates multi-camera multi-timestamp features and automatically selects reliable observations through visibility weighting, ensuring cross-view consistency from the source.
• 4D Gaussian Global Representation: Maintains a 4D Gaussian scene representation that expands incrementally with observations. After projection to the ego-vehicle viewpoint, it serves as a rendering prior input to the generative model, providing deterministic geometric constraints for generation.
• WorldGen Two-Stage Training: The first stage uses fully bidirectional temporal attention pretraining to establish a global spatiotemporal understanding of driving scenes; the second stage uses causal attention fine-tuning plus teacher forcing, combined with ODE distillation for a 12x speedup, while distribution matching distillation addresses exposure bias.
• Reconstruction-Generation Closed-Loop Constraints: The deterministic geometric priors from reconstruction continuously correct the generation process, while generation completes the spatiotemporal regions that reconstruction cannot cover. The objective functions of the two structurally constrain each other, achieving high stability, high consistency, and high realism.

How to Use Xiaomi Auto World Model

• Launch Platform: It has been launched in the 'Assisted Driving Academy - Real-Scene Simulated Scenarios' across all Xiaomi vehicle models.
• Usage Requirements: Requires owning a Xiaomi vehicle (such as SU7) and entering the Assisted Driving Academy module in the in-vehicle system.

Core Advantages of Xiaomi Auto World Model

• SOTA Performance: Waymo reconstruction PSNR 28.48 (about 1 point higher than DGGT), and zero-shot generalization on nuScenes also leads with PSNR 26.54; on the generation side, FVD 64.97 surpasses all bidirectional and autoregressive baseline models.
• Ultra-fast inference: Single-view generation 0.19 seconds/frame, three-view 0.46 seconds/frame, 5.6 times faster than the comparable autoregressive method Epona (1.06 seconds/frame).
• Ultra-long temporal sequence: Supports 81 frames of continuous generation (10Hz/30Hz, up to 1 minute), far exceeding the 8-16 frame limit of public baseline models.
• Zero-shot generalization: Still maintains a lead in zero-shot testing on nuScenes, demonstrating strong adaptability to new scenes.
• Already deployed in production: Has already completed the business loop in three core scenarios of Xiaomi Auto: synthetic data, simulation testing, and smart cockpit.

Project address of Xiaomi Auto World Model

• Project official website：https://JointWM.github.io/
• arXiv technical paper：https://arxiv.org/pdf/2605.18137

Comparison of Xiaomi Auto World Model with similar competitors

Application scenarios of Xiaomi Auto World Model

• Synthetic data generation: provides high-quality, highly diverse, and high-risk long-tail scenario synthetic data for assisted driving perception model training, addressing the problem of insufficient samples of rare scenarios in the real world.
• Simulation testing: builds a closed-loop simulation environment to reproduce real accident scenarios for targeted optimization, improving testing efficiency and completeness, and reducing the cost and risk of real-vehicle testing.
• Assisted driving academy: dynamically generates first-person driving tutorial videos in the smart cockpit, showing users the correct operations for complex road conditions, improving the safety of human-machine co-driving and the user experience.