What is GPDiT
GPDiT (Generative Pre-trained Autoregressive Diffusion Transformer) is a new type ofvideo generationmodel launched by Peking University, Tsinghua University, StepFun, and the University of Science and Technology of China. The model combines the advantages of diffusion models and autoregressive models, predicts future latent frames in an autoregressive manner, and naturally models motion dynamics and semantic consistency. GPDiT introduces a lightweight causal attention mechanism to reduce computational cost, and proposes a parameter-free rotation-based time conditioning strategy that effectively encodes temporal information. GPDiT performs excellently in video generation, video representation, and few-shot learning tasks, demonstrating its versatility and adaptability across a variety of video modeling tasks.
GPDiT's main features
- High-quality video generation: Generate long-sequence videos with high temporal consistency and motion coherence.
- Video representation learning: Based on autoregressive modeling and the diffusion process, learn semantic and dynamic representations of videos for use in downstream tasks.
- Few-shot learning: Can quickly adapt to a variety of video processing tasks, such as style transfer, edge detection, etc.
- Multi-task learning: Supports a variety of video processing tasks, such as grayscale conversion, depth estimation, person detection, etc.
GPDiT's technical principles
- Autoregressive diffusion framework: Predicts future latent frames in an autoregressive manner, naturally modeling motion dynamics and semantic consistency.
- Lightweight causal attention: Introduces a lightweight causal attention mechanism that eliminates attention computation between clean frames during training, reducing computational cost without degrading generation performance.
- Rotation-based time conditioning mechanism: Proposes a parameter-free rotation-based time conditioning strategy that reinterprets the noise injection process as rotation on a complex plane defined by data and noise components, removes adaLN-Zero and related parameters, and effectively encodes temporal information.
- Continuous latent space: Performs modeling in a continuous latent space, enhancing generation quality and representation capability.
GPDiT's project address
- arXiv technical paper：https://arxiv.org/pdf/2505.07344
GPDiT's application scenarios
- Video creation: Generate high-quality videos for use in advertising, film and television, animation, etc.
- Video editing: Achieve style transfer, color adjustment, resolution enhancement, etc.
- Few-shot learning: Quickly adapt to tasks such as person detection and edge detection.
- Content understanding: Automatically annotate, classify, and retrieve video content.
- Creative generation: Inspire the creativity of artists and designers, and generate artistic-style videos.