SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI videoEasy-Wav2Lip

Easy-Wav2Lip

Easy-Wav2Lip is a lipsync tool based on Wav2Lip, designed to simplify the use of Wav2Lip and improve output quality. It offers two methods: Google Colab and local installation, with no need to manually download and install prerequisites. The Colab version can be run with just two cells; Windows users only need one batch file to complete installation, updating, and running. The project improves visual quality and provides three quality options: basic, improved (with feathered mask), and enhanced (combined with GFPGAN upscaling). According to official tests, processing a 9-second 720p 60fps video is reduced from nearly 7 minutes to less than 1 minute, and tracking data is saved across multiple generations of the same video to save time. Local installation requires an Nvidia graphics card supporting CUDA 12.2, or a MacOS device supporting MPS (Apple Silicon or AMD GPU). It supports setting input and output paths and parameters through the config.ini configuration file, and supports batch processing, debug masks, preview settings, and other features.

Open website

About this tool

Easy-Wav2Lip is an open-source project designed to make Wav2Lip's high-quality lipsync simple and easy to use. It offers two methods: Google Colab and local installation, simplifying the installation and running process.

Main capabilities

  • Simplified setup and execution: On Colab, only two cells need to be run; Windows users only need one batch file to install, update, and run.
  • Improved speed: Official tests show that processing a 9-second 720p 60fps video is reduced from nearly 7 minutes to less than 1 minute.
  • Improved visual effects: Fixes visual artifacts on the lips and provides three quality options: basic Wav2Lip, improved (uses a feathered mask to restore the original resolution of other areas of the face), and enhanced (Wav2Lip + mask + GFPGAN upscaling of the face).
  • Cached tracking data: Multiple generations of the same video save tracking data, further saving time.
  • Detailed documentation: Provides explanations of each setting in the configuration options.

How to use

Google Colab version

Visit the provided Colab link and run two cells. This is the simplest and most compatible method. Link:https://colab.research.google.com/github/anothermartz/Easy-Wav2Lip/blob/v8.3/Easy_Wav2Lip_v8.3.ipynb

Local installation

Requirements: An Nvidia graphics card supporting CUDA 12.2, or a MacOS device supporting MPS (Apple Silicon or AMD GPU).

Automatic installation (Windows 64-bit):

  1. Download Easy-Wav2Lip.bat
  2. Place it in a folder on your computer
  3. Run it and follow the instructions. It will create an Easy-Wav2Lip folder in the running directory.
  4. From then on, just run this file each time you use it.

Manual installation: Ensure the terminal can access the following components: Python 3.10 (only 3.10.11 has been tested), Git, CUDA for Windows & Linux (the latest Nvidia driver is sufficient, only 12.2 has been tested).

Windows manual installation:

  1. Open cmd and use cd to navigate to the installation directory.
  2. Copy and paste the following code (note: two folders will be created: Easy-Wav2Lip and Easy-Wav2Lip-venv):
git clone https://github.com/anothermartz/Easy-Wav2Lip.git
cd Easy-Wav2Lip
python -m venv ../Easy-Wav2Lip-venv
call ../Easy-Wav2Lip-venv/Scripts/activate
python -m pip install --upgrade pip
pip install ffmpeg-python
pip install -r requirements.txt
  1. Run: close and reopen cmd, cd to the same directory, paste:
call ../Easy-Wav2Lip-venv/Scripts/activate
python run.py

MacOS and Linux installation (not tested):

  1. Open the terminal and use cd to navigate to the installation directory.
  2. Copy and paste the following code:
git clone https://github.com/anothermartz/Easy-Wav2Lip.git
cd Easy-Wav2Lip
python3 -m venv ../Easy-Wav2Lip-venv
source ../Easy-Wav2Lip-venv/bin/activate
python -m pip install --upgrade pip
pip install ffmpeg-python
pip install -r requirements.txt
  1. Run: close and reopen the terminal, cd to the same directory, paste:
source ../Easy-Wav2Lip-venv/bin/activate
python run.py

After installation is complete, the config.ini file will pop up. Add the paths to the video and audio files here, and configure the settings. After saving and closing, Wav2Lip processing will begin, and the output file will be saved in the same directory as the video_path file.

Advanced Adjustments

  • wav2lip_version: It is recommended to try Wav2Lip first, and if the effect of large tooth gaps appears, then switch to the GAN version.
  • nosmooth: When enabled, wav2lip will crop the face of each frame independently, suitable for fast motion or video editing; when disabled, it will blend the detected face positions between 5 frames, suitable for slow motion.
  • pads: Controls the number of pixels to add or remove from the face crop in each direction. Usually add 10 pixels at the bottom, but experimentation is needed.
  • mask: Controls how the processed face is blended with the original face. size increases the mask coverage area, feathering determines the degree of blending, and mouth_tracking updates the mask position each frame (slower).
  • debug_mask: Turns the background into grayscale and displays the mask in color, making it easier to see the mask position.

Batch Processing

  • Name files ending with numbers (such as Video1.mp4, Video2.mp4) and place them in the same folder.
  • Process in numerical order. Numbered videos can be mixed with unnumbered audio, or unnumbered videos with numbered audio.
  • A suffix will be added to output files to avoid overwriting the original files.
  • include_settings_in_suffix can add the settings used, making comparison easier.

Best Practices

  • Before passing through wav2lip, it is best to align the speech with the speaker's movements and expressions.
  • There must be a face in all frames, otherwise it will fail.
  • Crop or mask out faces you do not want to sync, otherwise one will be selected randomly.
  • Use h264 .mp4 format.
  • Try with small files (<720p, <30 seconds, 30fps, etc.).
  • Save audio as .wav, with the same length as the input video. Note that about 80ms will be cut off after processing.

Applicable Scenarios

Suitable for scenarios requiring lip sync for videos, such as creating virtual streamers, video dubbing, animation production, etc.

Limitations

  • Local installation requires specific hardware and software environments.
  • Processing large files may fail.
  • The output format is h264 .mp4 (other formats may be supported but have not been tested).
  • About 80ms of audio will be cut off, requiring additionalReserved.

Acknowledgments

This project is based on Wav2Lip, the speed improvements and basic quality improvements come from cog-Wav2Lip, and the GFPGAN upscaling code mainly comes from wav2lip-hq-updated-ESRGAN.

Related tools

Browse category