GPT-SoVITS

GPT-SoVITS is an open-source text-to-speech (TTS) and voice cloning project, hosted on GitHub (https://github.com/RVC-Boss/GPT-SoVITS）。项目以低数据量训练为主要卖点，官方描述为「1 minutes of speech data can also train a decent TTS model (few-shot voice cloning)."

Main capabilities

• Zero-shot TTS (Zero-shot TTS): input a 5-second voice sample to perform text-to-speech instantly.
• Few-shot TTS (Few-shot TTS): fine-tune the model with only 1 minute of training data to improve timbre similarity and realism.
• Cross-language support: inference can be performed in a language different from the training dataset; currently supports English, Japanese, Korean, Cantonese, and Chinese.
• High inference speed: official RTF (inference speed) data shows that GPT-SoVITS v2 ProPlus is 0.028 on a 4060Ti, 0.014 on a 4090 (about 1400 words ≈ 4 minutes, inference time 3.36 seconds), and 0.526 on an M4 CPU.

WebUI tools The project provides GPT-SoVITS-WebUI, which integrates tools such as accompaniment separation, automatic training set segmentation, Chinese ASR, and text annotation, helping beginners create training datasets and GPT/SoVITS models.

Usage

• Windows users (win>=10 tested) can download the integrated package and double-click go-webui.bat to launch GPT-SoVITS-WebUI.
• Manual installation: download ffmpeg.exe and ffprobe.exe and place them in the GPT-SoVITS root directory, and install Visual Studio 2017.
• Docker method: check Docker Hub for the latest image tags and choose a suitable tag; the Lite version does not include ASR models and UVR5 models. Docker Compose will mount all files in the current directory; before use, switch to the project root directory and pull the latest code; you can also use the provided Dockerfile to build the image locally. is_half controls whether half precision (fp16) is enabled; when GPU support is available, it can be set to true to reduce VRAM usage. Windows (Docker Desktop) has a small default shared memory, which may cause abnormalities; adjust shm_size according to system memory (e.g., 16g). docker-compose.yaml defines the GPT-SoVITS-CU126, GPT-SoVITS-CU128 full versions and the corresponding Lite lightweight services.
• Command-line method: you canopen UVR5's WebUI via the command line, perform dataset audio segmentation, execute dataset ASR processing (Chinese), and perform non-Chinese ASR annotation via Faster_Whisper.

Pretrained models and dependencies

• Pretrained model download: download from GPT-SoVITS Models and place into GPT_SoVITS/pretrained_models.
• Chinese TTS requires downloading the G2PW model; after extraction, rename it to G2PWModel and place it into GPT_SoVITS/text.
• UVR5 (vocal/accompaniment separation and reverberation removal): download models from UVR5 Weights and place them into tools/uvr5/uvr5_weights; when using bs_roformer or mel_band_roformer models, you need to manually download the model and the corresponding configuration file, and the file name must contain roformer so it can be recognized; it is recommended to specify the model type directly in the model name and configuration file name (such as mel_mand_roformer, bs_roformer).
• Chinese ASR: download from Damo ASR Model, Damo VAD Model, Damo Punc Model and place into tools/asr/models.
• English or Japanese ASR: download from Faster Whisper Large V3 and place into tools/asr/models; other models may also have similar effects and use less disk space.

Fine-tuning and inference Double-click go-webui.bat to launch (if you need to switch to V1, you can use go-webui.ps1 or go-webui-v1.bat/go-webui-v1.ps1, or manually switch versions in the WebUI). The process is: fill in the audio path → segment the audio into small chunks → optional denoising → proofread the ASR transcribed text → proceed to the next tab to fine-tune the model → open the inference WebUI (you can double-click go-webui-v2.bat or use go-webui-v2.ps1, then open the inference WebUI in 1-GPT-SoVITS-TTS/1C-inference).

Version highlights

• V2: supports Korean and Cantonese; optimizes the text frontend; pretrained models expanded from 2k hours to 5k hours; improves the synthesis quality of low-quality reference audiovolume.
• V3: Higher timbre similarity; with less training data it can approach the target speaker (using the base model directly without fine-tuning can also significantly improve timbre similarity); the GPT model is more stable, with fewer repetitions and omissions, and it is easier to generate emotionally rich speech.
• V4: Fixes the metallic artifact problem in V3 caused by non-integer upsampling, and natively outputs 48k audio to avoid muffled sound (V3 natively outputs only 24k). The author considers V4 a direct replacement for V3, but further testing is still needed.
• V2Pro: Slightly higher VRAM usage than v2, performance exceeds v4, and it combines v2's hardware cost and speed. v1/v2 and v2Pro have similar characteristics, and v3/v4 have similar characteristics; for training sets with average audio quality, v1/v2/v2Pro can produce good results, while v3/v4 cannot; the timbre synthesized by v3/v4 leans more toward the reference audio rather than the overall training set.
• V5: Significantly improves voice similarity without SoVITS fine-tuning; updates the vocoder to significantly reduce high-frequency spectral mirroring and aliasing artifacts; supports cuda_graph and flash_attention inference acceleration.

Applicable scenarios Suitable for individual developers, researchers, and enthusiasts who need text-to-speech, voice cloning, cross-lingual speech synthesis, and speech dataset creation (segmentation, annotation, denoising, accompaniment separation).

Limitations and not yet provided In the official documentation, the user guide, Japanese and English localization, and Japanese and English dataset fine-tuning training are listed as high-priority to-dos or planned items. Inference speed, training results, and other factors may vary under different hardware and data conditions, and the official sources provide only partial test data. Regarding charging, pricing, platform support scope, and similar information, the materials do not yet provide it.