TTS

🐸TTS is a deep learning toolkit for advanced text-to-speech generation, open-sourced by Coqui AI and validated in research and production.

Purpose: It is mainly used to synthesize text into speech, and provides the ability to train new models, fine-tune existing models, and analyze speech datasets. It is suitable for researchers, developers, and production teams that need to build TTS applications.

Main capabilities:

• Provides pretrained models for more than 1,100 languages.
• Supports training new models and fine-tuning existing models in any language.
• Provides dataset analysis and organization tools (located in the dataset_analysis directory).
• Supports multi-speaker TTS, and speakers can be selected via speaker_id.
• Supports voice conversion: it can convert the timbre of source audio to target audio, and can also be used for voice cloning.
• Models cover Text2Spec (Tacotron, Tacotron2, Glow-TTS, SpeedySpeech, Align-TTS, FastPitch, FastSpeech, FastSpeech2, SC-GlowTTS, Capacitron, OverFlow, Neural HMM TTS, Delightful TTS, etc.), vocoders (MelGAN, Multiband-MelGAN, GAN-TTS, ParallelWaveGAN, WaveGrad, WaveRNN, etc.), end-to-end models (YourTTS, Tortoise, Bark), and speaker encoders.
• New features: XTTSv2 supports 16 languages with better overall performance; XTTS fine-tuning code has been released; XTTS supports streaming output with latency below 200 ms; the XTTS production model can speak 13 languages; Bark supports unconstrained voice cloning inference; about 1,100 Fairseq models can be used; Tortoise supports faster inference.
• Provides an efficient training pipeline, detailed terminal and TensorBoard training logs, and a flexible and lightweight Trainer API.

How to use:

• Runtime environment: tested on Ubuntu 18.04, requires Python >= 3.9 and < 3.12.
• If you only use released models to synthesize speech, you can install from PyPI; if you need to encode or train models, clone the repository and install locally. Official installation instructions are provided for Ubuntu/Debian and Windows, and it can also be used without installation via a Docker image.
• Command-line tool: use the tts command. For example, tts --text "Text for TTS" --out_path output/path/speech.wav can generate speech; you can use --model_name、--vocoder_name to select a model;tts --list_models lists available models;tts --model_info_by_name or --model_info_by_idx queries model information; multi-speaker models support specifying --speaker_idx; self-trained models can be used by specifying --model_path and --config_path .
• A Python API is also provided for speech synthesis and voice conversion.
• Fairseq model naming format:tts_models/<lang-iso_code>/fairseq/vits.

Applicable scenarios: Speech synthesis research and development, production TTS deployment, multilingual speech generation, multi-speaker speech systems, voice cloning and voice conversion, and custom TTS model training and fine-tuning.

Limitations: The materials do not provide specific price or charging information. The underlined “TTS*” and “Judy*” in the performance comparison chart are internal models that have not been open-sourced and are only used to demonstrate potential; models beginning with a dot (.Jofish .Abe .Janice) correspond to real human voices. Complete installation details (such as the specific commands for Windows and Docker) require reference to the official links.