What is DragonV2.1
DragonV2.1 (DragonV2.1Neural) is Microsoft's latest zero-shot text-to-speech (TTS) model. The model is based on the Transformer architecture, supports multilingual and zero-shot voice cloning, and can generate natural, expressive speech with only 5-90 seconds of voice prompts. The model has made significant improvements in pronunciation accuracy, speech naturalness, and controllability. Compared with DragonV1, the model's word error rate (WER) is reduced by an average of 12.8%, supports SSML phoneme tags and custom dictionaries, and can precisely control pronunciation and accent. The model integrates watermarking technology to ensure the compliance and security of speech synthesis.
Main Features of DragonV2.1
- Multilingual Support: Supports more than 100 Azure TTS locales, supports synthesizing speech in multiple languages, and meets the needs of different users.
- Emotion and Accent Adaptation: Adjusts the emotion and accent of speech according to context, making the speech more expressive and personalized.
- Zero-Shot Voice Cloning: With only 5-90 seconds of voice prompts, it can quickly generate the user's own AI voice copy, greatly lowering the threshold for voice cloning.
- Fast Generation: Can generate high-quality speech synthesis results in a short time, with latency less than 300 milliseconds and real-time factor (RTF) less than 0.05, suitable for real-time application scenarios.
- Pronunciation Control: Supports the use of phoneme tags in SSML (Speech Synthesis Markup Language). Users can precisely control speech pronunciation through International Phonetic Alphabet (IPA) phoneme tags and custom dictionaries.
- Custom Dictionary: Users can create custom dictionaries to define the pronunciation of specific words, ensuring the accuracy of speech synthesis.
- Language and Accent Control: Supports generation in multiple languages and specific accents, such as British English (en-GB), American English (en-US), etc.
- Watermarking Technology: Automatically adds watermarks to generated speech output, effectively preventing the misuse of speech synthesis content.
Technical Principles of DragonV2.1
- Transformer Architecture: DragonV2.1 is based on the Transformer model architecture, a deep learning architecture widely used in natural language processing and speech synthesis. Transformer processes input data based on the self-attention mechanism, can capture long-distance dependencies, and generate more natural and coherent speech.
- Multi-Head Attention Mechanism: The multi-head attention mechanism in Transformer enables the model to focus on different parts of the input data from different perspectives, improving the model's ability to capture speech features.
- SSML Support: SSML is a markup language used to describe speech synthesis. DragonV2.1 supports phoneme tags and custom dictionaries in SSML. Users can precisely control pronunciation, intonation, rhythm, etc. through SSML, ensuring the accuracy and naturalness of speech synthesis.
Project Address of DragonV2.1
- Project Official Website：https://techcommunity.microsoft.com/blog/azure-ai-services-blog/personal-voice-upgraded-to-v2-1-in-azure-ai-speech-more-expressive-than-ever-bef/4435233
Application Scenarios of DragonV2.1
- Video Content Creation: Generates multilingual dubbing and real-time subtitles for videos, preserves the original actors' voice style, and improves the viewing experience for global audiences.
- Intelligent Customer Service and Chatbots: Generates natural and expressive voice replies, supports multiple languages, improves user experience, and reduces customer service costs.
- Education and Training: Generate speech in multiple languages to help language learners practice pronunciation and listening, and enhance the interactivity of online courses.
- Intelligent assistants: Provide natural voice interaction for smart home devices and in-vehicle systems, support multiple languages, and improve user convenience.
- Enterprises and brands: Create brand voices for advertising and marketing promotion, support multiple languages, and enhance brand recognition and global market coverage.