What Is IndexTTS-2.5
IndexTTS-2.5 is Bilibili's open-source industrial-grade zero-shotvoice cloningmodel, with only 0.8B parameters, supporting cross-lingual transfer across five languages: Chinese, English, Japanese, Spanish, and Arabic. The model refactors its inference architecture, achieving a 2.28x speedup in inference, supporting speech rate adjustment from 0.5x to 2.0x, as well as fine-grained control over pronunciation, emotion, and more. The complete code, model weights, and paper have all been released, under the Bilibili Model License.
IndexTTS-2.5's Main Features
- Zero-Shot Voice Cloning: With just a 3 to 10 second reference audio, it can precisely replicate the timbre characteristics of any speaker, without additional training for the target speaker.
- Five-Language Cross-Lingual Support: Supports five languages: Chinese, English, Japanese, Spanish, and Arabic.
- Fine-Grained Emotion Control: Provides multiple emotion adjustment methods: conveying emotion through an independent emotion reference audio, using an eight-dimensional emotion vector to precisely specify emotion intensity, enabling text-driven automatic emotion inference, and adjusting the strength of emotion through the
emo_alphaparameter. - Timbre and Emotion Decoupling: The timbre prompt audio and emotion prompt audio can be specified completely independently. Users can separately control "who is speaking" and "how they speak."
- Stepless Speech Rate Adjustment: Through the
duration_factorparameter, the synthesis speech rate can be arbitrarily adjusted between 0.5x and 2.0x, meeting the pacing needs of different scenarios. - Precise Pronunciation Intervention: Supports pronunciation control at three granularities: Chinese pinyin, English CMU phonemes, and Japanese kana, effectively solving issues with polyphonic characters, variant pronunciations, and context-dependent pronunciation.
IndexTTS-2.5's Technical Principles
- Three-Stage Generation Pipeline: The model adopts a three-stage architecture of "text → semantic tokens → mel spectrogram → waveform." A Transformer-based text-to-semantic (T2S) language model generates semantic tokens, which are then processed by a non-autoregressive semantic-to-mel (S2M) flow matching module to generate mel spectrograms, and finally restored to the final audio through a neural vocoder.
- Semantic Codec Compression: Compresses the frame rate of semantic tokens from 50Hz to 25Hz, directly halving the sequence length and significantly reducing the computational and memory overhead of the T2S module during training and inference, which is one of the keys to inference acceleration.
- Zipformer Architecture Upgrade: The backbone network of the S2M module is replaced from U-DiT with the more efficient Zipformer architecture. Zipformer achieves stronger long-range dependency modeling capability with fewer parameters, making mel spectrogram generation faster and more stable, while maintaining synthesis audio quality.
- Cross-Lingual Modeling Strategy: To address the confusion caused by character overlap in multilingual scenarios, the team proposed three strategies: boundary-aware alignment (inserting language markers before and after text, such as
<ZH>), token-level concatenation (fusing language-specific embeddings for each input token), and instruction-guided generation (adding a natural language instruction prefix before the text, such as "Please read in English"). - GRPO Reinforcement Learning Optimization: Introduce GRPO (Group Relative Policy Optimization) in the post-training phase of the T2S module, freeze the word error rate (WER) of the ASR model as the reward signal, generate four candidate samples each time, and continuously improve pronunciation accuracy and speech naturalness by optimizing the relative likelihood of high-reward samples.
Follow WeChat and reply "open source" to join theAI Open Source Project Exchange Group
How to use IndexTTS-2.5
- Environment preparation: Users need to first install the Git and uv package managers, and use
git clone https://github.com/index-tts/index-tts.gitto download the official code repository to the local machine. - Dependency installation: After entering the project directory, execute the
uv sync --all-extrascommand to automatically create a virtual environment and install all required Python dependencies. - Model download: Use
huggingface-cliormodelscopetool from the official repositoryIndexTeam/IndexTTS-2.5to download the model weights to the localcheckpointsdirectory. - Launch the Web interface: Execute
uv run webui.pyto launch the Gradio interactive interface, and visithttp://127.0.0.1:7860in the browser for visual speech synthesis operations. - Load the model: In a Python script, import the
indextts.infer_v2_5fromIndexTTS2class, and pass in the configuration file path and model directory to complete model instantiation. - Basic voice cloning: Call the instance's
infermethod, and pass in the reference audio path, target text, and language code to generate and save a speech file with the cloned timbre. - Emotion control: In the
infermethod, additionally pass in theemo_audio_promptparameter to specify the emotion reference audio, or useemo_vectorto pass in an eight-dimensional emotion vector to control the emotional expression of the synthesized speech. - Speech rate adjustment: During inference, set the
duration_factorparameter to a value between 0.5 and 2.0 to proportionally speed up or slow down the speech rate of the synthesized speech. - Pronunciation intervention: Insert annotations such as
<行|XING2>、<minute|M IH1 . N AH0 T>or<上手|じょうず>into the input text to precisely control the pronunciation of polyphonic characters or heteronyms.
Core advantages of IndexTTS-2.5
- Small parameters, high efficiency: The model has only 0.8B parameters. Through semantic codec compression and Zipformer architecture upgrades, it achieves a 2.28x inference speedup, with the real-time factor (RTF) as low as 0.20 under BF16 precision, balancing lightweight design and high performance.
- Five-language cross-lingual transfer: It supports five languages: Chinese, English, Japanese, Spanish, and Arabic. Users can use reference audio in any one language to clone speech in another language, and can achieve cross-lingual emotion transfer without emotional training data in the target language.
- Fine-grained multi-dimensional controllability: The model supports stepless speech rate adjustment from 0.5x to 2.0x, eight-dimensional emotion vector control, text-driven emotion inference, and pronunciation intervention at three granularities: pinyin, CMU phonemes, and Japanese kana, meeting the fine-tuning needs of content creation for speech synthesis.
- Complete decoupling of timbre and emotion: Users can independently specify the timbre prompt audio and the emotion prompt audio, and the two can even come from different languages, flexibly injecting the desired emotional style while preserving the target speaker's timbre.
- Open-source support for Arabic: Among mainstream open-source TTS models, IndexTTS-2.5 is one of the few models that fully supports Arabic voice cloning and cross-lingual transfer, filling the gap for this language in the open-source speech synthesis field.
IndexTTS-2.5 Project Address
- GitHub Repository：https://github.com/index-tts/index-tts
- arXiv Technical Paper：https://arxiv.org/pdf/2601.03888
Comparison of IndexTTS-2.5 with Similar Competing Products
| Comparison Dimension | IndexTTS-2.5 | CosyVoice 3 |
|---|---|---|
| Publisher | Bilibili Index Speech Team | Alibaba |
| Parameter Count | 0.8B | 0.5B / 1.5B |
| Supported Languages | Chinese, English, Japanese, Spanish, Arabic (5 languages) | Public evaluations cover Chinese and English (3+ languages) |
| Chinese Speaker Similarity (SS) | 77.10 (RL version 77.92) | 80.01 (0.5B version) |
| English Speaker Similarity (SS) | 68.06 (RL version 67.79) | 74.16 (0.5B version) |
| Chinese Word Error Rate (WER) | 4.36% (RL version 3.93%) | 3.84% |
| English Word Error Rate (WER) | 5.12% (RL version 3.89%) | 4.88% |
| Cross-lingual Transfer | ✅ Supported (Chinese→Japanese/Spanish/Arabic, etc.) | No public evaluation data available |
| Timbre-Emotion Disentanglement | ✅ Fully disentangled, can be combined across languages | No public support for disentanglement at the same granularity |
| Speech Rate Control | ✅ Stepless adjustment from 0.5x to 2.0x | No public interface at the same level |
| Pronunciation Intervention (G2P) | ✅ Pinyin / CMU phonemes / Japanese kana | Partially supported |
Application Scenarios of IndexTTS-2.5
- AI Dubbing and Audio Content Generation: Content creators only need to provide a reference audio clip of a host or character to quickly generate audiobooks, radio dramas and podcast content in multiple languages and emotional styles, without repeated recording.
- Cross-lingual Video Localization: Film and short video teams can use Chinese reference audio to directly generate dubbing in English, Japanese, Spanish or Arabic while preserving the original actor's timbre, greatly reducing overseas distribution costs.
- Game and Virtual Character Voices: Game developers can clone specific timbres for NPCs or virtual idols and switch between joy, anger, sorrow and happiness in real time via emotion vectors, enabling dynamic plot voiceovers and livestream interaction.
- Real-time Voice Interaction and Digital Humans: With an ultra-low real-time factor of 0.20, IndexTTS-2.5 can be deployed in intelligent customer service, digital human livestreaming and real-time translation scenarios, supporting low-latency personalized voice replies.
- Education and Language Learning: Educational institutions can use the pronunciation control feature to generate demonstration audio in standard Pinyin, CMU phonemes or Japanese kana, and can clone a teacher's timbre to produce synchronized multilingual courses.