What is Speech-02
Speech-02 is MiniMax's new generationtext-to-speech(TTS) model. The model is based on a regression Transformer architecture, enabling zero-shotvoice cloning, requiring only a few seconds of reference speech to generate highly similar target speech. The Flow-VAE architecture enhances the information representation capability of speech generation, improving the quality and similarity of synthesized speech. Speech-02 offers two versions: Speech-02-HD is designed for high-fidelity applications such as dubbing and audiobooks, eliminating rhythm inconsistency issues and maintaining clear audio quality; Speech-02-Turbo is optimized for real-time performance, balancing ultra-low latency with excellent audio quality, suitable for interactive applications. The Speech-02 model is now available onMiniMax Audioplatform and MiniMax API platform.
Main Features of Speech-02
- Zero-Shot Voice Cloning: Requires only a few seconds of reference speech to generate highly similar target speech.
- High-Quality Speech Synthesis: Generates natural and fluent speech, supporting multiple languages and dialects.
- Multilingual Support: Supports 32 languages, excels in Chinese, English, Cantonese and others, with cross-language switching capability.
- Personalized Voice Generation: Users provide sample audio, and the model learns to generate personalized speech.
- Emotion Control: Supports multiple emotions (such as happiness, sadness, etc.), guiding speech generation based on text descriptions.
Technical Principles of Speech-02
- Autoregressive Transformer Architecture: Based on autoregressive Transformer architecture, generating speech with better prosody, intonation, and overall naturalness. The autoregressive model generates speech features one by one during the generation process, ensuring the generated speech is more natural and coherent.
- Zero-Shot Voice Cloning: Introduces a learnable speaker encoder, which specifically learns the voice features most useful for synthesized speech, such as the speaker's unique pronunciation habits. The model requires only a few seconds of reference speech to generate highly similar target speech.
- Flow-VAE Architecture: Transforms the latent space based on invertible mapping, more accurately capturing complex patterns in data. The Flow-VAE architecture enhances the information representation capability during speech generation, improving the overall quality and similarity of synthesized speech.
- T2V Framework: The T2V framework combines open-ended natural language descriptions with structured tag information, achieving highly flexible and controllable timbre generation. Users guide the model to generate speech with specific timbres and emotions based on text descriptions.
Speech-02 Project Address
- Project Official Website：https://www.minimax.io/news/speech-02-series
- Technical Paper：https://huggingface.co/spaces/MiniMaxAI/MiniMax-Speech-Tech-Report
Application Scenarios of Speech-02
- Intelligent Voice Assistants: Provides natural and fluent human-computer interaction experiences for smart devices, improving user satisfaction.
- Audiobooks and Dubbing: Produces high-quality audiobooks, advertising dubbing, etc.
- Social Media and Entertainment: Provides personalized voice generation in social media, live streaming, singing chat and other scenarios, enhancing user interactivity and entertainment.
- Education and Children's Toys: Applied in educational learning machines, children's toys, etc., providing more vivid and interesting learning experiences.
- Smart Hardware Integration: Integrates with smart speakers, automotive smart cockpits and other hardware devices, improving the intelligence level of devices.