Luna-TTS - A text-to-speech large model launched by VUI Labs

What is Luna-TTS

Luna-TTS is a self-developedtext-to-speechlarge model from VUI Labs, a startup affiliated with Shanghai Jiao Tong University. The model abandons the mainstream autoregressive approach and adopts a Masked Diffusion masked diffusion architecture, is trained based on Qwen3-0.6B, and is paired with the self-developed Luna-Codec; it is pretrained on 1 million hours of speech data in Chinese, English, Japanese, and Korean. The model topped the global rankings on the Hugging Face TTS Arena V2 blind listening leaderboard, and ranked third on the authoritative Artificial Analysis leaderboard, surpassing Google.

Main features of Luna-TTS

• Multilingual speech synthesis: Supports high-quality speech synthesis in four languages: Chinese, English, Japanese, and Korean, with natural timbre and smooth prosody, suitable for long-text scenarios such as audiobooks and podcasts.
• Emotion and paralinguistic control: Can precisely control emotions such as neutral, sad, and surprised, as well as paralinguistic details such as breathing, pauses, laughter, and sighs.
• Real-time streaming output: Luna-TTS Realtime supports chunk-level streaming generation, with first-chunk latency of only 41.6ms and end-to-end RTF as low as 0.024.
• Timbre consistency and cloning: Supports timbre cloning and preservation, with stable timbre and no drift during long-text generation.

Technical principles of Luna-TTS

• Masked Diffusion architecture: Abandons the mainstream autoregressive approach, treats the speech RVQ token grid as a whole, and predicts masked positions in parallel through multiple rounds of iteration; the generation order is dynamically determined by the model, avoiding error accumulation along the prefix.
• Self-developed Luna-Codec: Encodes 24kHz waveforms into a discrete token grid of 25Hz and 8 codebooks, with a bitrate of only 2.2kbps, and uses a causal codec to achieve streaming, frame-synchronous output.
• Qwen3 backbone model: Continues training based on Qwen3-0.6B; the standard version uses bidirectional attention, while the Realtime version switches to chunk-level causal attention and works with KV Cache to achieve streaming generation.
• Large-scale training and reinforcement learning: Conducts about 1 million hours of pretraining and 100,000 hours of high-quality annealing in four languages: Chinese, English, Japanese, and Korean, transfers GRPO reinforcement learning to the diffusion denoising trajectory, and optimizes content correctness and speaker consistency.

Project addresses of Luna-TTS

• Project official website：https://vuilabs-ai.github.io/luna-tts/
• arXiv technical paper：https://arxiv.org/pdf/2608.11593

Comparison of Luna-TTS with similar competitors

Application Scenarios of Luna-TTS

• Intelligent Customer Service and Telemarketing: Replaces traditional voice robots, providing a natural conversational experience with emotional variations, supporting real-time streaming responses, and reducing user hang-up rates.
• Audiobook Content and Podcasts: Generates high-quality long-form speech for audiobooks, news broadcasts, and knowledge-based podcasts, maintaining timbre consistency and narrative rhythm, and supporting switching between multiple emotional segments.
• Online Education and Training: Provides clear and standard multilingual pronunciation for course explanations and language learning, and supports emotional adjustment to match teaching content (such as encouragement, seriousness, and gentleness).
• In-Vehicle and Smart Hardware: Through the low-latency features of Luna-TTS Realtime, it provides instant and natural voice interaction entry points for in-vehicle navigation, smart homes, and headset assistants.
• Real-Time Simultaneous Interpretation and Meeting Assistants: Combined with speech understanding capabilities, it enables low-latency, highly natural speech translation output in cross-language meeting scenarios, improving communication efficiency.