Live Portrait is a video-driven portrait animation framework focused on improving generalization ability, controllability, and efficiency for real-world usage scenarios. It can synthesize realistic videos from a single source image, with motion information (facial expressions and head pose) coming from a driving video, audio, text, or generated content.
In terms of use, Live Portrait is mainly used to transform static portrait photos into dynamic videos. Users upload a clear facial portrait photo, the system extracts facial features, then they select or upload a driving video showing facial movement, and click the generate button to create the animation. During the process, configuration parameters can be adjusted or default values can be used.
Main capabilities include:
- Multi-style portrait animation: Using stitching technology, static images can be animated across different styles and sizes.
- Portrait video editing and stitching: Provides dynamic video-to-video face animation and editing capabilities based on stitching technology.
- Precise eye animation control: Dynamically adjusts the degree of eye opening and closing through scalar input to achieve detailed expression control.
- Accurate lip movement control: Fine-tunes lip position through scalar-based input to achieve precise speech and expression animation.
- Advanced self-reenactment: Generates high-quality animation from a single source frame, outperforming existing methods in reconstructing dynamic video sequences.
- Powerful cross-reenactment: Accurately transfers motion between different portraits, handling scenarios ranging from large pose changes to subtle expressions and multi-person input.
Core technical principles: Based on the vid2vid model, trained using a large number of high-quality image and video datasets, it extracts key features from the source image and driving video. It adopts an innovative implicit keypoint framework to map motion from the driving video onto the portrait, ensuring efficient and controllable animation. The optimized decoder quickly generates each frame, efficiently creating smooth animation while preserving the identity of the source image. The stitching and retargeting modules optimize the output, achieving fine-grained control of facial expressions and motion with minimal computational overhead.
How to use: Three-step operation. Step one, upload a portrait photo with a clear face, and Live Portrait will extract facial features for animation; step two, select or upload a driving video showing facial movement, and this video will guide the generation of facial movement for the portrait image; step three, click the "Animate" button, adjust configuration parameters as needed or keep the default values, then click generate and wait for completion.
Applicable scenarios: Including self-reenactment, cross-reenactment, static image portrait animation, portrait video editing, eye and lip retargeting control, and animation between different styles. It can also be used for animal animation (cats, dogs, pandas) through fine-tuning.
Limitations and notes: Official materials do not provide specific pricing information or prices, platform limitations, or detailed usage steps. The known developers are researchers from Kuaishou Technology, the University of Science and Technology of China, and Fudan University. In terms of performance, it achieves a generation speed of 12.8ms on an RTX 4090 GPU using PyTorch, and training used approximately 69 million high-quality frames. The open-source project inference code and model can be obtained at https://github.com/KwaiVGI/LivePortrait , and a free trial can be accessed at https://live-portrait.org/free-playground .
