What is MirageLSD
MirageLSD is the world's first Live-Stream Diffusion AI video model launched by the Decart AI team. It can achieve real-time video generation of unlimited duration, with latency as low as under 40 milliseconds, and supports smooth output at 24 frames per second. Through Diffusion Forcing technology and history-enhanced training, it solves the error accumulation problem of traditional autoregressive models in long-duration generation and achieves unlimited video generation. Based on technologies such as Hopper-optimized Mega Kernels, architecture-aware pruning, and Shortcut Distillation, MirageLSD greatly improves generation speed while maintaining high image quality, achieving true real-time interaction.
Main features of MirageLSD
-
Unlimited-duration real-time video generation: MirageLSD can generate video streams of unlimited duration, with latency as low as under 40 milliseconds, and supports real-time generation speed at 24 frames per second, solving the error accumulation problem of traditional video generation models in long-duration generation.
-
Real-time interactivity: Users can prompt, transform, and edit in real time during video generation, enabling a continuous interactive experience.
-
Low-latency processing: Through optimization technologies such as Hopper-optimized Mega Kernels and architecture-aware pruning, the model achieves ultra-low-latency processing of 40 milliseconds and supports real-time video generation.
-
Wide range of application scenarios: It can be used in many fields such as live streaming, video calls, game development, and animation production, and can convert ordinary scenes in real time into user-specified styles or scenes.
Technical principles of MirageLSD
-
Diffusion Forcing technology: Through frame-by-frame denoising, it allows the model to generate a single-frame image without complete video context, thereby achieving frame-level generation.
-
History-enhanced training: By introducing noisy data from historical frames during training, the model can predict and correct errors in the input, thereby achieving unlimited generation.
-
Optimization strategies：
-
Hopper-optimized Mega Kernels: Optimized for the NVIDIA Hopper GPU architecture, reducing latency for each model layer.
-
Architecture-aware pruning: Reducing computation by adjusting model parameter sizes to fit the GPU architecture.
-
Shortcut Distillation: Reducing the diffusion steps required for generation by training a smaller model to match the denoising trajectory of a larger model.
-
Project address of MirageLSD
- Technical paper：https://about.decart.ai/publications/mirage
Application scenarios of MirageLSD
-
Live streaming and video calls: Convert ordinary video calls or live stream content in real time into user-specified scenes, such as turning real-world scenes into a sci-fi world.
-
Game development: It can convert game visuals in real time into different visual styles, such as turning an ordinary combat scene into a lightsaber duel.
-
Animation production and virtual dress-up: Providing real-time visual effects support for animation production and virtual dress-up.