SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

LTX-2.3 - Lightricks' latest open-source video generation model

Author: AI小集 · Original source: AI工具集

What is LTX-2.3

LTX-2.3 is the latest generation open-sourced by the Israeli AI company Lightricksvideo generationmodel. It adopts a Diffusion Transformer architecture and has 22 billion parameters. The model supports three input methods—text, image, and audio—to generate videos, can output up to 4K resolution, and natively supports 9:16 vertical format and 24/48FPS frame rate options. Compared with the previous generation, LTX-2.3 significantly improves image detail sharpness through a newly trained VAE architecture, solving the problem of texture blurring at high resolutions, while also adding native audio generation to achieve synchronized audio-video output. The model provides 7 generation endpoints, can generate videos up to 20 seconds long in a single pass, and supports LoRA fine-tuning.

LTX-2.3

Main features of LTX-2.3

  • Multimodal video generation: Supports three core generation methods—text-to-video, image-to-video, and audio-to-video—to meet different creative needs.
  • Native vertical screen support: Adds 9:16 vertical format (up to 1080×1920), directly adapting to short video platforms and social media content creation.
  • Synchronized audio-video generation: Adds native audio generation, which can synchronously output ambient sound, sound effects, and dialogue, and can also use audio input to drive video frame generation.
  • Flexible frame rate options: Supports two frame rate modes: 24FPS cinematic feel and 48FPS smooth motion.
  • Video extension and retake: Provides extend-video and retake-video features, generating up to 20 seconds at a time, and can be extended further through extension.
  • Fast generation mode: Provides accelerated versions text-to-video fast and image-to-video fast for efficiency scenarios.
  • High-resolution output: Supports up to 4K resolution, achieving sharper image details and texture expression through the new VAE architecture.
  • LoRA fine-tuning support: Creators can train LoRA adapters locally, completing customized model fine-tuning in less than an hour.
  • Supporting super-resolution tools: Provides post-processing models for 2x/1.5x spatial super-resolution and 2x frame rate improvement, optimizing final output quality.
  • Local desktop editor: Simultaneously launches LTX Desktop, an open-source video editor based on the LTX-2.3 engine, running entirely locally without the cloud.

Technical principles of LTX-2.3

  • DiT Diffusion Transformer architecture: Based on the Diffusion Transformer architecture, combining diffusion models with Transformers to generate high-quality videos through an iterative denoising process, with a scale of about 22 billion parameters.
  • New VAE Variational Autoencoder: The retrained Variational Autoencoder greatly improves encoding-decoding quality, significantly enhancing image sharpness, texture details, and facial feature clarity, solving the problem of blurred details at high resolutions in the previous generation.
  • Joint spatiotemporal modeling: Uses a spatiotemporally separated attention mechanism to process video data, simultaneously modeling the image content in the spatial dimension and the motion changes in the temporal dimension, ensuring the temporal coherence of generated videos.
  • Native audio generation module: Integrates an audio generation subnetwork to enable end-to-end synchronized audio-video generation, supports driving visual content generation from audio input, and ensures audio-visual synchronization.
  • Multimodal Condition Injection: Uses different condition encoders to uniformly map text, image, and audio modality inputs into a latent space, enabling flexible multimodal control.
  • Distilled Acceleration Version: Provides a distilled model, using knowledge distillation technology to compress model size and improve inference speed while maintaining quality.
  • LoRA Low-Rank Adaptation: Supports Low-Rank Adaptation technology, allowing users to quickly inject specific styles or concepts on top of a pretrained model, enabling low-cost customization.
  • Super-Resolution Post-Processing: Comes with an independent super-resolution model, using spatial upscaling (2x/1.5x) and frame rate interpolation (2x) technology to perform secondary optimization on generated videos.

LTX-2.3 Project Address

  • Project Website：https://ltx.io/model/ltx-2-3
  • Hugging Face：https://huggingface.co/Lightricks/LTX-2.3
  • arXiv Technical Paper：https://arxiv.org/pdf/2601.03233

LTX-2.3 Application Scenarios

  • Short Video and Social Media Content: Native 9:16 vertical screen support and 24/48FPS frame rate options, perfectly suited to the content creation needs of platforms such as Douyin, TikTok, and Instagram Reels.
  • Advertising and Marketing Videos: Quickly generate product showcases and brand promotional videos, supports image-to-video conversion, and can transform static product images into dynamic advertising materials.
  • Film and Television Previsualization and Concept Design: Directors and producers can quickly generate shot previews with text, validate creative concepts before committing to formal shooting, and reduce upfront costs.
  • Game and Animation Production: Generate game cutscenes and character motion references, or use them as initial materials for animation production, accelerating the content production process.
  • Music MV and Audio-Video Creation: The audio-to-video feature supports using audio to drive image generation, suitable for musicians to quickly create lyric MVs or visualize audio tracks.
  • Education and Training Content: Generate instructional demonstration videos and visualize operational steps, transforming static teaching materials into dynamic explanatory content.