IMS-Toucan (also known as ToucanTTS) is a Text-to-Speech (TTS) toolkit developed by the Institute for Natural Language Processing (IMS) at the University of Stuttgart, Germany, for training, using, and teaching advanced speech synthesis systems. Its core system, ToucanTTS, is fast and controllable, does not require extensive computational resources, and is officially claimed to support over 7,000 languages. The toolkit is completely free, with open-source code and models, and offers a free GPU online demo on Hugging Face as well as a large-scale multilingual TTS dataset.
Purpose: IMS-Toucan is mainly used for multilingual text-to-speech synthesis and can be applied to scenarios such as voice assistants, reading tools, and voice interaction interfaces; it can also serve as a research and teaching platform for training custom speech models or fine-tuning.
Main capabilities: Supports speech synthesis for over 7,000 languages; provides pretrained models that can be downloaded for inference; supports language embeddings and speaker embedding control; supports control of prosodic parameters such as speaking rate, pitch curve variance, and energy curve variance; supports multi-GPU training; provides an online interactive demo and dataset.
Usage: Python 3.10 is recommended. Installation requires cloning the repository. If training a model, the machine should have at least one CUDA-capable GPU; inference does not require a GPU. On Linux, specific dependencies need to be installed (usually preinstalled), and creating a virtual environment is recommended. On Windows, commands need to be adjusted according to the venv documentation. When using a GPU, no additional action is needed on Linux; on Windows, refer to the PyTorch official website to install a GPU-supported version. Model and cache storage paths can be customized by editing Utility/storage_config.py. Pretrained models are automatically downloaded from Hugging Face. eSpeak-NG is an optional dependency used to handle special cases in many languages. On Linux it is usually already installed; on Windows, download the .msi installer and set the PHONEMIZER_ESPEAK_LIBRARY environment variable; installation on Mac is more complex and can refer to the relevant guide, and after installation the same environment variable must be set to point to libespeak-ng.dylib.
During inference, load the model through InferenceInterfaces/ToucanTTSInterface.py, and language and speaker embeddings can be set. Two methods are provided: read_to_file (synthesizes a list of strings into an audio file) and read_aloud (directly plays the synthesized speech). For examples, see run_interactive_demo.py and run_text_to_file_reader.py. Prosodic parameters can be adjusted in the code or passed to the interface.
Training a custom model requires creating a Recipe: write a function in Utility/path_to_transcript_dicts.py that returns a dictionary of audio paths and transcript texts; copy finetuning_example_simple.py (single dataset) or finetuning_example_multilingual.py (multiple datasets) under TrainingInterfaces/Recipes and modify it, replacing path_to_transcript_dict, the cache directory name, and save_dir, and change lang to the corresponding ISO 639-3 language ID. If training from scratch, refer to the pipeline whose name contains ToucanTTS. Finally, in run_training_pipeline.py, import the run function and register it in pipeline_dict. During training, GPU IDs can be specified (for multiple GPUs, separate them with commas and use torchrun), and Weights & Biases logging is supported. Checkpoints are saved every epoch or after a specified number of steps, keeping only the five most recent, while also generating best.py for inference. If CUDA out of memory occurs, reduce batchsize.
Applicable scenarios: multilingual speech synthesis research, low-resource language TTS development, voice cloning and prosody control experiments, teaching demonstrations, etc.
Limitations: Although 7,000+ languages are supported, the synthesis quality for each specific language may be affected by the training data; training requires a GPU, and poor data quality may cause loss NaN; some warning messages (such as xFormers loading failure, torchaudio backend switching, etc.) can be ignored.
