TurboDiffusion

TurboDiffusion is a video generation acceleration framework released by the thu-ml team, aimed at significantly improving the end-to-end generation speed of diffusion models. According to the official description, it can achieve 100 to 200 times acceleration on a single RTX 5090 while maintaining video quality. Its core technologies include SageAttention, SLA (Sparse-Linear Attention) for attention acceleration, and rCM for timestep distillation. The related paper is "TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times."

Purpose and Applicable Scenarios TurboDiffusion is primarily aimed at accelerated inference and training of video diffusion models. It supports text-to-video (T2V) and image-to-video (I2V) tasks for the Wan2.1 and Wan2.2 series of models. All checkpoints support generating 480p or 720p videos. The framework is suitable for researchers and developers who want to quickly generate videos under limited hardware resources, and it also provides training code to support custom training.

Main Capabilities

• Provides 100-200 times end-to-end diffusion generation acceleration, based on SageAttention and SLA attention acceleration as well as rCM timestep distillation.
• Supports models such as Wan2.1-T2V-1.3B, Wan2.1-T2V-14B, and Wan2.2-I2V-A14B, and provides quantized and non-quantized checkpoints.
• Provides inference scripts for T2V and I2V models.
• Supports interactive inference through the terminal (turbodiffusion/serve/), allowing multiple rounds of video generation without reloading the model.
• Provides training code based on Wan2.1 and synthetic data; the training is built on the rCM codebase and supports FSDP2, Ulysses CP, and selective activation checkpointing (SAC).
• Provides SLA white-box training, reducing distribution shift by aligning predictions of the SLA-enabled model with a full-attention pretrained model.
• Provides the ComfyUI community integration Comfyui_turbodiffusion.
• The turbot2va directory provides LTX-2 text-to-audio-video distillation and inference, using an independent environment without modifying the main runtime.

Usage and Installation Basic environment requirements: python>=9, torch>=2.7.0, torch==2.8.0 is recommended (higher versions may cause OOM). TurboDiffusion can be installed via pip or compiled from source. To enable SageSLA (fast SLA forward pass based on SageAttention), SpargeAttn must be installed first. For GPUs with more than 40GB of VRAM (such as H100), please use non-quantized checkpoints (without -quant) and remove --quant_linear from the command. For GPUs such as RTX 5090 and RTX 4090, please use quantized checkpoints (with -quant) and add --quant_linear to the command. You need to download the VAE (applicable to Wan2.1 and Wan2.2) and umT5 text encoder checkpoints. Example download commands: mkdir checkpoints cd checkpoints wget https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B/resolve/main/Wan2.1_VAE.pth wget https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B/resolve/main/models_t5_umt5-xxl-enc-bf16.pth Example download of quantized model checkpoints:

For Wan2.1-T2V-1.3B

wget https://huggingface.co/TurboDiffusion/TurboWan2.1-T2V-1.3B-480P/resolve/main/TurboWan2.1-T2V-1.3B-480P-quant.pth

For Wan2.2-I2V-14B

wget https://huggingface.co/TurboDiffusion/TurboWan2.2-I2V-A14B-720P/resolve/main/TurboWan2.2-I2V-A14B-high-720P-quant.pth wget https://huggingface.co/TurboDiffusion/TurboWan2.2-I2V-A14B-720P/resolve/main/TurboWan2.2-I2V-A14B-low-720P-quant.pth Example download of non-quantized model checkpoints:

For Wan2.1-T2V-1.3B

wget https://huggingface.co/TurboDiffusion/TurboWan2.1-T2V-1.3B-480P/resolve/main/TurboWan2.1-T2V-1.3B-480P.pth

For Wan2.2-I2V-14B

wget https://huggingface.co/TurboDiffusion/TurboWan2.2-I2V-A14B-720P/resolve/main/TurboWan2.2-I2V-A14B-high-720P.pth wget https://huggingface.co/TurboDiffusion/TurboWan2.2-I2V-A14B-720P/resolve/main/TurboWan2.2-I2V-A14B-low-720P.pth Use the inference script for the T2V model or the script for the I2V model to generate.

Training The training code is based on Wan2.1 and synthetic data, built on the rCM codebase (https://github.com/NVlabs/rcm）之上。对于 rCM/SLA training requires running additional related steps. You need to download the Wan2.1 pretrained checkpoints (.pth format) and the VAE/text encoder to assets/checkpoints. FSDP2 relies on distributed checkpointing (DCP) for loading and saving; before training, the .pth teacher checkpoint needs to be converted to .dcp. After training, the saved .dcp checkpoint can be converted to .pth using scripts/dcp_to_pth.py. A Wan2.1-14B synthetic dataset is provided; download it to assets/datasets. SLA training adopts a white-box approach, aligning the SLA model predictions with the full-attention pretrained model. For a single-node training example, refer to turbodiffusion/rcm/configs/experiments/sla/wan2pt1_t2v.py (14B configuration) or modify it as needed. The parameter updates from SLA training can be merged into the rCM checkpoint using turbodiffusion/scripts/merge_models.py, specifying --base as the rCM model, --diff_base as the pretrained model, and --diff_target as the SLA fine-tuned model.

Limitations and Notes The current model is trained only with long English prompts. If you use other types of prompts, please enhance them to achieve better performance. The checkpoints and paper are not yet finalized and will be updated in the future to improve quality. The base environment requires python>=3.9, torch>=2.7.0, with torch==2.8.0 recommended; higher versions may cause OOM.

Future Plans The team is actively developing the following features: organizing and releasing the training code; optimizing the infrastructure to support flattened context parallelism; providing acceleration for MiniMax-H3; vLLM-Omni integration; and supporting autoregressive video generation models.