What is Gummy
Gummy is an end-to-end speech translation large model launched by the Tongyi Lab at the 2024 Yunqi Conference. The model can generate speech recognition and translation results in real-time streaming, and supports speech input in more than ten languages including Chinese, English, Cantonese, Japanese, Korean, French, German, Russian, Italian, Spanish, etc., translating them into the target language. Through an end-to-end approach, the Gummy model reduces translation latency and improves translation quality, achieving SOTA (State of the Art) results on multiple test sets. Gummy supports capabilities for commercial deployment such as multilingual mixed translation, terminology intervention, and domain prompting, and can smoothly translate languages from various countries into the target language in scenarios such as multinational conferences without specifying the source language.
Main Features of Gummy
- Multilingual support: Gummy can process speech input in more than ten languages including Chinese, English, Cantonese, Japanese, Korean, French, German, Russian, Italian, Spanish, etc., and translate it into the target language in real time.
- End-to-end translation: Unlike traditional cascaded systems, Gummy adopts an end-to-end design that directly translates speech into the target language without relying on an intermediate text stage.
- Low-latency translation: Gummy's translation latency has been reduced to within 0.5 seconds, faster than the simultaneous interpretation delay of human experts.
- High-quality translation: On multiple industry-recognized open-source test sets, Gummy achieves SOTA (State of the Art) translation quality results.
- Streaming translation: Gummy supports translating as you speak, that is, translating while listening, making it suitable for real-time communication scenarios.
Technical Principles of Gummy
- End-to-end design: Through an end-to-end architecture, the Gummy model maps speech input in the source language to text output in the target language, simplifying the development process and improving system performance.
- Deep neural networks: Based on deep learning technology, especially deep neural networks, it learns the complex mapping relationship from speech to text.
- Real-time streaming processing: Supports real-time speech recognition and translation, enabling translation while listening.
- wait & predict mechanism: The model internally adopts a special mechanism to automatically determine the timing of translation and optimize translation quality and latency.
Gummy Project Address
- Project official website：tongyi.aliyun.com, currently some functions of the speech translation large model Gummy have been launched on the Tongyi APP for download and experience.
Application Scenarios of Gummy
- Real-time speech translation: The Gummy model can translate speeches in meetings in real time, providing simultaneous interpretation services for international conferences, multilingual negotiations, etc.
- Education and training: In the field of education, Gummy assists language learning by providing real-time translation of multilingual teaching content, helping students and teachers overcome language barriers.
- Travel and navigation: Provides real-time speech translation for travelers, helping them communicate with local people who speak different languages, or providing multilingual guidance during navigation.
- Customer service: In the field of customer service, Gummy serves as a multilingual customer service assistant, providing fast and accurate language support and improving customer satisfaction.
- Medical consultation: In the medical field, Gummy provides multilingual medical consultation translation services, helping communication between doctors and patients.