VDN-MiniMax-H3 - OpenVDN's Open-Source Video Generation Acceleration Solution

What is VDN-MiniMax-H3

VDN-MiniMax-H3 is an open-source project by the OpenVDN team forvideo generationacceleration, built upon MiniMax-H3 . The model adopts a hybrid attention architecture: local sliding-window Softmax preserves fine details, while long-range linear attention handles long-range context, and then 8-step distillation greatly compresses the number of denoising steps. On 8 NVIDIA B200s, the model takes only 11.23 seconds to generate approximately 14.4 seconds of 768p video, with visual quality nearly lossless compared to the original 50-step Dense H3.

Main Features of VDN-MiniMax-H3

Hybrid Attention Architecture: Splits video attention into a local Softmax branch and a long-range linear branch, balancing fine details with long-range context.
8-Step Ultra-Fast Generation: Through DMD2 distillation, the number of denoising steps is compressed from 50 to 8, greatly shortening generation time.
Nearly Lossless Visual Quality: While generating 14.4 seconds of video in 11.23 seconds, visual quality is close to the original 50-step Dense H3 baseline.
Three-Stage Training Adaptation: Uses layer-by-layer alignment, end-to-end branch adaptation, and LoRA joint adaptation to avoid disturbing the pretrained model with random initialization.
Branch-Specific Parallelism: Allocates the Softmax and linear branches to different GPUs, reducing latency by an additional 13.3% compared to standard Ulysses parallelism.

Technical Principles of VDN-MiniMax-H3

Hybrid Attention Architecture: VDN splits video attention into two complementary branches, local Softmax and long-range linear; the former preserves fine details, while the latter handles long-range context with linear complexity, thereby replacing full quadratic attention.
Local Softmax Branch: Uses a bidirectional sliding window, with each group of 5 frames attending to neighboring blocks, and introduces the first and last frames as boundary anchors, maintaining global temporal consistency while increasing attention density by only 3.57%.
Video Delta Attention: Upgrades the traditional token-by-token Delta Rule to joint solving per frame, using normal equations to let all spatial tokens jointly determine the new state; it is naturally non-expansive and can adapt to token relevance without hard frame-size scaling.
Text Dual-Branch Injection: In the Softmax branch, text is globally visible to each frame; the linear branch writes text into the bidirectional state in one pass at initialization, avoiding repeated computation.
Gated Fusion Mechanism: The two branches have different output scales, so each is equipped with content-related Sigmoid gating and independent output projection before merging; the Softmax gate remains frozen during the first two training stages to prevent the optimizer from reducing loss by suppressing the original branch.
Three-Stage Training Adaptation: Aligns the linear branch layer by layer, then jointly optimizes all hybrid modules end-to-end, and finally fine-tunes together via QKVO LoRA and the linear branch, freezing the pretrained backbone throughout to ensure generation quality does not collapse.

Follow WeChat and reply "open source" to join theAI Open-Source Project Communication Group

How to Use VDN-MiniMax-H3

Environment Preparation: After cloning the GitHub repository, create a Python 3.12 environment and install PyTorch 2.13.0 and project dependencies.
Download Model: Download via ModelScope OpenVDN/vdn-minimax-h3 weights to the local ckpts directory.
Run the official example: Execute scripts/inference/8nfe_tuned_fp8.sh to generate a sample video with a single GPU.
Multi-GPU inference: Depending on the hardware, choose 8nfe_tuned_fp8_ulysses_h200.sh or b200.sh script to run on 8 GPUs.
Custom prompts: Use encode_prompt.py to call Qwen3-VL-32B to encode text into .pt feature files, then pass them to infer.py to generate videos.

Core advantages of VDN-MiniMax-H3

Ultra-fast generation: 8-step distillation combined with hybrid attention generates approximately a 14.4-second 768p video in only 11.23 seconds on 8 B200s, a 74.5x speedup over the original model.
Near-lossless image quality: Visual details, subject consistency, and instruction-following ability are close to the 50-step Dense H3 baseline and significantly better than 4-step FastH3.
Hybrid attention architecture: Local Softmax preserves fine temporal details, while long-range linear attention handles long-range context with linear complexity, balancing quality and efficiency.
Frame-level joint solving (VDA): Upgrades token-by-token updates to frame-by-frame joint optimization, which is naturally non-expansive and adaptively accounts for token relevance without hard scaling.
Staged smooth adaptation: Through three-stage training with layer-by-layer alignment, end-to-end branch adaptation, and LoRA joint fine-tuning, it avoids disturbing the pretrained model with random initialization.
Branch-specific parallelism: Assigns the Softmax and VDA branches to different GPUs for execution, reducing latency by an additional 13.3% compared with standard Ulysses parallelism.

Project address of VDN-MiniMax-H3

Project official website：https://openvdn.github.io/
GitHub repository：https://github.com/OpenVDN/vdn-minimax-h3
HuggingFace model library：https://huggingface.co/OpenVDN/vdn-minimax-h3

Comparison of VDN-MiniMax-H3 with similar competitors

Comparison dimension	VDN-MiniMax-H3	Sparse VideoGen2 (SVG2)
Core idea	Replace full quadratic attention with hybrid attention (local Softmax + long-range linear VDA), combined with few-step distillation	A training-free sparse attention framework that identifies key tokens through semantic-aware permutation and reorders them to improve GPU computational efficiency
Technical route	Architecture modification + three-stage training adaptation (A1/A2/B) + 8-step DMD2 distillation	Semantic k-means clustering + top-p dynamic budget control + custom sparse attention kernel, no retraining required
Speedup ratio	74.5×(compared with single-GPU Dense H3 baseline),10.7×(compared with 8-GPU baseline)	2.30×（HunyuanVideo），1.89×（Wan 2.1）
Generation speed	Generates a 14.4-second 768p video in 11.23 seconds (8×B200, 8 NFE)	Generates a 5-second video in about 13 minutes (H100, after sparsification)
Image quality performance	Near-lossless, with visual details and instruction following close to 50-step Dense H3	PSNR up to 30 (HunyuanVideo) / 26 (Wan 2.1), outperforming other sparse methods at the same sparsity rate
Applicable models	Specifically for MiniMax H3 modification	General framework, plug-and-play for various DiT models such as HunyuanVideo and Wan 2.1
Training cost	Requires three-stage training (layer-by-layer alignment → end-to-end adaptation → LoRA joint fine-tuning)	Zero training cost, directly applied to pretrained model inference

Application Scenarios of VDN-MiniMax-H3

Real-time interactive video creation: 8-step ultra-fast generation enables creators to adjust prompts in real time and preview results instantly, greatly shortening the creative iteration cycle.

Mass production of advertising and marketing materials: E-commerce and brands can quickly generate multi-version product showcase videos and dynamic posters, meeting the high-frequency material needs of holiday promotions and similar events.
Film and television previsualization and dynamic storyboarding: Directors and production teams can quickly generate storyboard animations before formal shooting to validate camera language, scene staging, and narrative rhythm.
Dynamic game content generation: Generate NPC animations, environmental cutscenes, and personalized player story segments in real time for open-world games or virtual social platforms.
Industrialized production of short social media videos: MCN agencies and self-media creators can produce short video content such as narrative shorts and virtual character vlogs at low cost and with high concurrency.