SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

Qwen3-TTS - Alibaba Tongyi's open-source series of speech generation models

Author: AI小集 · Original source: AI工具集

What is Qwen3-TTS

Qwen3-TTS is an open-source series from Qwenspeech generationmodels, with powerful voice cloning, creation, and speech control capabilities. The models are based on the innovative Qwen3-TTS-Tokenizer-12Hz multi-codebook speech codec, achieving efficient speech compression and high-fidelity restoration. The models adopt Dual-Track dual-track modeling and support low-latency streaming generation, with the first audio packet requiring waiting for only one character. The models cover 10 mainstream languages (Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian) and multiple dialects, and possess intelligent text understanding capabilities, able to adaptively adjust tone, rhythm, and emotion. The full multi-codebook Qwen3-TTS series models have all been open-sourced, including two sizes, 1.7B and 0.6B, to meet different performance and efficiency needs, providing developers and users with comprehensive speech generation capabilities.

Qwen3-TTS

Main features of Qwen3-TTS

  • Voice cloning: The models can clone the voice of a specific speaker from a small amount of reference audio, achieving highly similar speech synthesis.
  • Voice creation: Supports generating customized voice identities through natural language descriptions. Users can freely define acoustic attributes, persona, and background information to create unique voices.
  • Speech control: Supports users in flexibly regulating multi-dimensional acoustic attributes such as voice, emotion, and prosody through instructions, achieving precise speech expression.
  • Multilingual support: The models cover 10 mainstream languages (such as Chinese, English, Japanese, etc.) and multiple dialects, meeting the needs of global applications.
  • Low-latency streaming generation: Based on innovative Dual-Track dual-track modeling, it achieves extremely fast bidirectional streaming generation. The waiting time for the first audio packet is only one character, and the end-to-end synthesis latency is as low as 97ms.
  • Context understanding: The models possess powerful text semantic understanding capabilities and can automatically adjust tone, rhythm, and emotion according to the input text, adapting to different scenarios.
  • High-fidelity restoration: Relying on the self-developed Qwen3-TTS-Tokenizer-12Hz, it fully preserves paralinguistic information and acoustic environment features, achieving efficient, high-fidelity speech restoration.

Technical principles of Qwen3-TTS

  • Qwen3-TTS-Tokenizer-12Hz: Based on a multi-codebook speech codec, it can efficiently compress speech signals and perform high-dimensional semantic modeling. The codec fully preserves paralinguistic information (such as intonation, rhythm, and emotion) and acoustic environment features, and achieves high-speed, high-fidelity speech restoration through a lightweight non-DiT (Discrete Inverse Transform) architecture.
  • Dual-Track dual-track modeling: Combining streaming and non-streaming generation methods, a single model supports both modes simultaneously. It achieves extreme low-latency streaming generation, with the first audio packet output immediately after inputting a single character at the fastest, and end-to-end synthesis latency as low as 97ms, meeting the needs of real-time interaction scenarios.
  • Discrete multi-codebook LM architecture: The models adopt a discrete multi-codebook language model (LM) architecture to achieve end-to-end modeling of full speech information. It avoids the information bottleneck and cascading errors in traditional LM+DiT solutions, significantly improving the model's generality, generation efficiency, and performance ceiling.
  • Natural language instruction-driven: The model supports speech generation driven by natural language instructions, allowing users to control attributes such as timbre, emotion, and prosody through simple text descriptions. It deeply integrates text semantic understanding and adaptively adjusts tone, rhythm, and emotion to achieve anthropomorphic expression where "what you think is what you hear."

Qwen3-TTS project address

  • GitHub repository：https://github.com/QwenLM/Qwen3-TTS
  • HuggingFace model library：https://huggingface.co/collections/Qwen/qwen3-tts

Qwen3-TTS application scenarios

  • Intelligent voice assistant: Provides natural voice interaction for smart home devices and in-vehicle systems, supports multiple languages and dialects, and improves user experience.
  • Content creation: Quickly converts text into natural speech, supports multiple timbres and emotional expressions, and is suitable for audiobooks and video dubbing.
  • Education: Provides multilingual and multi-timbre speech output for language learning and online teaching, enhancing learning outcomes.
  • Games and entertainment: Generates personalized timbres for game characters, supports emotion and intonation adjustment, and enhances game immersion.
  • Customer service and support: Provides multilingual and multi-timbre support for intelligent customer service and public voice announcements, improving service efficiency.