What is SkyReels-V2
SkyReels-V2 is an infinite-length film generation model launched by Kunlun Wanwei's SkyReels team. Based on the Diffusion-forcing framework, it combines multimodal large language models (MLLM), multi-stage pretraining, reinforcement learning, and other technologies to generate high-quality, infinite-length video content. SkyReels-V2 addresses the challenges of existing technologies in prompt adherence, visual quality, motion dynamics, and video duration coordination, and supports multiple application scenarios such as story generation, image-to-video synthesis, camera directing, and multi-subject consistent video generation. The model and related code have been open-sourced, providing a powerful tool for creative content production and virtual simulation.
Main Features of SkyReels-V2
- Infinite-Length Video Generation: Supports generating theoretically infinite-length video content, breaking through the duration limitations of traditional video generation models.
- Story Generation: Arranges complex multi-action sequences according to narrative text prompts to achieve dynamic storytelling.
- Image-to-Video Synthesis: Provides two methods, including fine-tuning a full-sequence text-to-video diffusion model (SkyReels-V2-I2V) and combining a diffusion forcing model with frame conditioning (SkyReels-V2-DF), to transform static images into coherent videos.
- Camera Directing Function: Supports generating smooth and diverse camera movement effects, enhancing the cinematic feel of videos.
- Element-to-Video Generation: Combines arbitrary visual elements (such as characters, objects, and backgrounds) into coherent videos guided by text prompts, suitable for applications such as short dramas, music videos, and virtual e-commerce content creation.
Technical Principles of SkyReels-V2
- Multimodal Large Language Model (MLLM): Generates initial descriptions of videos based on a multimodal large language model, and combines sub-expert models (such as shot type, shot angle, shot position, expression, and camera movement, etc.) to provide more detailed shot language descriptions. Based on manual annotation and model training, it further improves the understanding of shot language and significantly enhances the prompt adherence of generated videos.
- Multi-Stage Pretraining：
- Progressive Resolution Pretraining: Gradually improves from low resolution (256p) to high resolution (720p), progressively enhancing the model's generation capability.
- Multi-Stage Post-Training Optimization: Includes initial concept-balanced supervised fine-tuning (SFT), motion-specific reinforcement learning (RL) training, diffusion forcing framework (DF) training, and high-quality SFT, ensuring optimal performance of the model across different aspects.
- Reinforcement Learning (RL): Optimizes motion quality based on reinforcement learning, addressing the shortcomings of existing models in motion dynamics, smoothness, and physical plausibility. A semi-automatic data collection pipeline is used to generate preference comparison data pairs, train a reward model, and perform direct preference optimization (DPO) to improve motion quality.
- Diffusion Forcing Framework: Assigns an independent noise level to each frame, enabling infinite scalability in video generation. Using a non-decreasing noise schedule, it reduces the search space of denoising schedules for consecutive frames from _O_(1 e 48) to _O_(1 e 32), significantly improving generation efficiency.
- Efficient Data Processing and Optimization: Integrating general datasets, self-collected media, and art resource libraries, based on multi-stage filtering and annotation, to ensure the quality of training data. Using techniques such as FP8 quantization, multi-GPU parallelism, and model distillation, it significantly reduces inference time and computational cost, improving the practicality of the model.
SkyReels-V2 Project Address
- GitHub Repository：https://github.com/SkyworkAI/SkyReels-V2
- HuggingFace Model Library：https://huggingface.co/collections/Skywork/skyreels-v2
- arXiv Technical Paper：https://arxiv.org/pdf/2504.13074
SkyReels-V2 Application Scenarios
- Film Production: Generate coherent videos of unlimited duration for complex narratives and long-take creation.
- Advertising Creation: Transform static images into dynamic videos, enhancing the appeal and expressiveness of advertisements.
- Video Shooting Assistance: Generate smooth camera movement effects, helping to design and implement complex shooting shots.
- Short Dramas and Music Videos: Rapidly generate high-quality videos, reducing shooting costs and time.
- Virtual Reality and Game Development: Generate realistic virtual scenes and character animations, enhancing user experience and immersion.