SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

Universal-1 - AssemblyAI's Multilingual Speech Recognition and Transcription Model

Author: AI小集 · Original source: AI工具集

What is Universal-1

Universal-1 is a multilingual speech recognition and transcription model launched by AI speech startup AssemblyAI, trained on over 12.5 million hours of multilingual audio data, supporting English, Spanish, French, German, and more. The model provides highly accurate speech-to-text services in various environments, including noisy backgrounds, different accents, and natural conversations, and also features fast response times and improved timestamp accuracy. Universal-1 is designed to improve the accuracy of every aspect of speech recognition, able to meet customers' needs for nuances in speech data, and is a powerful tool for building next-generation AI products and services.

Universal-1

Key Features of Universal-1

  • Multilingual Support: Universal-1 can handle multiple languages including English, Spanish, French, and German, and has been optimized for these languages to improve speech recognition accuracy.
  • High Accuracy: Under different conditions, such as background noise, accent diversity, natural conversation, and language variations, Universal-1 maintains excellent speech-to-text conversion accuracy.
  • Reduced Hallucination Rate: Compared to Whisper Large-v3, Universal-1 reduces the hallucination rate on speech data by 30%, i.e., reduces instances where the model incorrectly generates text without audio input.
  • Fast Response: Universal-1 is designed with efficient parallel inference capabilities, able to quickly process long audio files and provide fast response times. Its batch processing capability is 5 times faster than Whisper Large-v3.
  • Accurate Timestamp Estimation: The model provides timestamps accurate to the word level, which is crucial for applications such as audio and video editing and meeting transcription. Universal-1's timestamp accuracy is 26% higher than Whisper Large-v3.
  • User Preference: In user preference tests, users preferred Universal-1's output 71% of the time, indicating that it better meets user needs in actual use.

Performance Comparison of Universal-1

  • English Speech-to-Text Accuracy：Universal-1 achieved the lowest word error rate (WER) in 5 out of 11 datasets, compared to models such as OpenAI's Whisper Large-v3, NVIDIA's Canary-1B, Microsoft Azure Batch v3.1, Deepgram Nova-2, Amazon, and Google Latest-long.
    Universal-1 English WER
  • Non-English Speech-to-Text Accuracy：In tests for Spanish, French, and German, Universal-1 had lower WER on 5 out of 15 datasets, demonstrating its competitiveness in these languages.
    Universal-1 WER by Language
  • Timestamp Accuracy：In terms of timestamp accuracy, Universal-1 increased the proportion of words with predicted timestamps within 100 milliseconds by 25.5%, from 67.2% to 84.3%, compared to Whisper Large-v3.
    Universal-1 Timestamp Accuracy
  • Inference Efficiency：On an NVIDIA Tesla T4 machine, Universal-1 is 3 times faster than the faster whisper backend without parallelization, and with 64 parallel inferences, it can transcribe 1 hour of audio in just 21 seconds.
  • Hallucination Reduction：Compared to Whisper Large-v3, Universal-1 reduces the hallucination rate by 30% when transcribing audio.
  • Human Preference Test：In human preference tests, evaluators preferred Universal-1's output 60% of the time, while Conformer-2 only 24%.
    Universal-1 Human Preference
  • Speaker Diarization：In terms of Speaker Diarization accuracy, Universal-1 achieves the following improvements compared to Conformer-2:
    • Diarization Error Rate (DER) is relatively reduced by 7.7%.
    • cpWER, which jointly measures WER and speaker labeling accuracy, is relatively reduced by 13.6%.
    • Speaker count estimation accuracy is relatively improved by 71.3%.

How to Use Universal-1

Currently, Universal-1 is available in English and Spanish, with German and French versions coming soon. In the future, AssemblyAI will also add additional language support to future universal models. Interested users can try it in the Playground or via the API.

  1. Try via the Playground：The easiest way to try Universal-1 is throughAssemblyAI's Playground。In the Playground, users can directly upload audio files or enter a YouTube link, and the model will quickly generate a transcription.
  2. Free API Trial: Userscansign up for freeand get an API token.After signing up, go to AssemblyAI's Docs or Welcome Colab, which are resources that help you quickly get started with the API.

For more information about Universal-1, see AssemblyAI's official technical report:https://www.assemblyai.com/discover/research/universal-1

Universal-1 Use Cases

  • Conversation Intelligence Platforms: Can quickly and accurately analyze large amounts of customer data to provide key voice-of-the-customer insights and analytics, regardless of accent, recording conditions, or number of speakers.
  • AI Notetakers: Generate highly accurate, hallucination-free meeting notes, providing a foundation for large language model-based summarization, action items, and other metadata generation, including accurate proper nouns, speakers, and timing information.
  • Creator Tools: Build AI-driven video editing workflows for end users, leveraging precise speech-to-text output in multiple languages, low error rates, and reliable word timing information.
  • Telehealth Platforms: Automate clinical note entry and claims submission processes, leveraging accurate and faithful speech-to-text output, including rare vocabulary such as prescription names and medical diagnoses, with high success rates even under adversarial and far-field recording conditions.