What is the Wan First-Last Frame Model
The Wan First-Last Frame Model (Wan2.1-FLF2V-14B) is an open-source 14B-parameter first-last frame video generation model. Based on the first-frame and last-frame images provided by the user, the model automatically generates smooth high-definition video transitions, supporting a variety of styles and special effect transformations. The Wan First-Last Frame Model is based on an advanced DiT architecture, combined with an efficient video compression VAE model and a cross-attention mechanism, ensuring that the generated video is highly consistent in space and time. Users can try it for free on theTongyi Wanxiangofficial website.
Main Features of the Wan First-Last Frame Model
- First-Last Frame Video Generation: Based on the first-frame and last-frame images provided by the user, generate a natural and smooth video with a duration of 5 seconds and 720p resolution.
- Supports Multiple Styles: Supports generating videos in styles such as realistic, cartoon, comic, and fantasy.
- Detail Replication and Realistic Motion: Accurately replicates the details of the input images and generates vivid and natural motion transitions.
- Instruction Following: Controls video content based on prompts, such as camera movement, subject motion, special effect changes, etc.
Technical Principles of the Wan First-Last Frame Model
- DiT Architecture: The core architecture is based on the DiT (Diffusion in Time) architecture, specifically used for video generation. Based on the Full Attention mechanism, it accurately captures the long-range spatiotemporal dependencies of the video, ensuring high consistency of the generated video in time and space.
- Video Compression VAE Model: Introduces an efficient video compression VAE (Variational Autoencoder) model, significantly reducing computational cost while maintaining the high quality of the generated video. This makes high-definition video generation more economical and efficient, supporting large-scale video generation tasks.
- Conditional Control Branch: The first frame and last frame provided by the user serve as control conditions, and based on an additional conditional control branch, smooth and accurate first-last frame transformation is achieved. The first frame and last frame are concatenated with several zero-filled intermediate frames to form a control video sequence. The sequence is further concatenated with noise and a mask as the input to the diffusion transformer model (DiT).
- Cross-Attention Mechanism: Extracts the CLIP semantic features of the first frame and last frame, and injects them into the DiT generation process through the Cross-Attention Mechanism. Image stability control ensures that the generated video remains highly consistent in semantics and vision with the input first and last frames.
- Training and Inference: The training strategy is based on a distributed strategy combining data parallelism (DP) and fully sharded data parallelism (FSDP), supporting 720p, 5-second video clip training. Model performance is gradually improved in three stages:
- First Stage: Mixed training, learning the mask mechanism.
- Second Stage: Specialized training, optimizing first-last frame generation capability.
- Third Stage: High-precision training, improving detail replication and motion smoothness.
Project Address of the Wan First-Last Frame Model
- GitHub Repository：https://github.com/Wan-Video/Wan2.1
- HuggingFace Model Library：https://huggingface.co/Wan-AI/Wan2.1-FLF2V-14B-720P
Application Scenarios of the Wan First-Last Frame Model
- Creative Video Production: Quickly generate creative videos with scene transitions or special effect changes.
- Advertising and Marketing: Produce attractive video advertisements and enhance visual effects.
- Film and Television Special Effects: Generate special effect shots such as the alternation of the four seasons and day-night changes.
- Education and Presentation: Create vivid animation effects to assist teaching or presentations.
- Social Media: Generate personalized videos to attract followers and increase engagement.