What is Lucy 2
Lucy 2.0 is a real-time world transformation model launched by Decart AI, supporting the leap of high-fidelity video editing from offline rendering to an interactive real-time experience. As a pure diffusion model, the model does not rely on 3D geometry or depth maps. Its physical behavior is entirely learned autonomously by observing video evolution, and it can continuously generate frames at 1080p resolution at 30fps with near-zero latency. Based on Smart History Augmentation technology, the model can self-correct quality drift during long-term operation and supports hours of uninterrupted coherent generation. The model is deeply optimized for AWS Trainium3 and can be used for visual effects such as real-time character replacement and virtual try-on, and can provide real-time data augmentation and physically consistent simulated environments for robot training.
Main features of Lucy 2
- Real-time visual transformation: The model continuously generates frames at 30fps and 1080p resolution with near-zero latency, supporting instant editing in live streams without pre-rendering.
- Multi-dimensional editing: Enables character replacement, clothing changes, product placement, motion control, and full environment transformation through text prompts and reference images.
- Persistent operation capability: With Smart History Augmentation technology, it can run continuously for hours without identity collapse, geometric distortion, or texture degradation.
- Robot data augmentation: As a real-time simulation engine, it changes materials, lighting, and environments in real time while preserving physical consistency, expanding a single real demonstration into thousands of training variants.
Technical principles of Lucy 2
- Pure diffusion architecture: It does not rely on depth maps, 3D meshes, or explicit physics engines. All visual dynamics are learned entirely autonomously by observing video evolution.
- Emergent physical understanding: The model implicitly learns world structure from data (such as finger geometry, cloth folds, and object separation) without the need for manually defined topology rules.
- Smart History Augmentation: During training, the model is exposed to its own imperfect outputs and penalized, enabling it to learn to identify drift and actively correct it rather than blindly following the previous frame.
- Hardware co-optimization: Deeply customized for AWS Trainium3, it uses mega-kernels to reduce startup overhead, on-chip SRAM to avoid high-bandwidth memory latency, and a custom WebRTC pipeline to achieve end-to-end real-time transmission.
Project address of Lucy 2
- Project official website：https://decart.ai/publications/lucy-2-introducing-sota-video-generation-in-realtime
- Online experience Demo：https://lucy.decart.ai/
Application scenarios of Lucy 2
- Real-time live interaction: Streamers can instantly complete character replacement, virtual try-on, or dynamic product placement during a live stream, presenting diverse visual effects in real time without post-production.
- On-site content creation: Creators can adjust the visual style and environmental atmosphere in real time through text prompts on the shooting site, moving the post-production stage earlier to achieve what-you-see-is-what-you-get instant visual previews.
- Robot training augmentation: R&D teams can change object materials, lighting, and background environments in real time, expanding a single real operation recording into thousands of physically consistent diverse training samples, solving the problem of data scarcity.
- Virtual production simulation: Production teams can leverage the model's ability to run stably for long periods to generate interactive dynamic backgrounds and environment simulations in real time, directly replacing traditional offline rendering production workflows.