SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

Voxtral Transcribe 2 - A series of speech-to-text models launched by Mistral AI

Author: AI小集 · Original source: AI工具集

What is Voxtral Transcribe 2

Voxtral Transcribe 2 is a new-generation series launched by Mistral AIspeech-to-textmodels, including two versions: Voxtral Mini Transcribe V2 focuses on batch transcription, supporting 13 languages, speaker diarization, word-level timestamps, and contextual biasing; Voxtral Realtime is designed for real-time scenarios, adopting a streaming architecture with configurable latency as low as under 200 milliseconds, suitable for interactive applications such as voice assistants. Both models lead in accuracy on benchmarks such as FLEURS, and their cost-effectiveness is significantly better than GPT-4o mini、Gemini and other competitors.

Voxtral Transcribe 2

Main features of Voxtral Transcribe 2

  • Multilingual transcription: Supports high-accuracy speech-to-text in 13 languages: English, Chinese, Hindi, Spanish, Arabic, French, Portuguese, Russian, German, Japanese, Korean, Italian, and Dutch.
  • Word-level timestamps: Generates precise start and end times for each transcribed word, suitable for subtitle generation and content alignment.
  • Speaker diarization: Automatically identifies different speakers and labels their speaking periods, clearly distinguishing multi-party conversations.
  • Contextual biasing: Supports inputting up to 100 custom terms to improve recognition accuracy for proper nouns and industry terminology.
  • Ultra-low-latency real-time transcription: Voxtral Realtime adopts a streaming architecture, with latency configurable to under 200 milliseconds, enabling transcription while listening.
  • Noise robustness: Maintains high transcription accuracy even in noisy environments such as factories and call centers.
  • Long audio processing: Can process audio files up to 3 hours long in a single request.
  • Multi-format support: Compatible with .mp3, .wav, .m4a, .flac, and .ogg formats, with a maximum single file size of 1GB.

Technical principles of Voxtral Transcribe 2

  • Streaming architecture: Voxtral Realtime adopts a native streaming architecture, enabling configurable latency as low as under 200 milliseconds to meet real-time interaction needs.
  • Dynamic latency configuration: The Realtime model supports flexible latency settings. At 2.4 seconds of latency, accuracy matches the batch model, and at 480 milliseconds of latency, the word error rate is only 1-2% higher than the offline model. Users can balance speed and accuracy according to the scenario.
  • Unified multilingual modeling: Both models use a single architecture to handle 13 languages, achieving cross-language transfer through shared representation learning, so that non-English languages achieve recognition performance close to English.
  • Contextual biasing mechanism: The system supports injecting up to 100 custom terms as prior knowledge, increasing the recognition probability of specific terms during decoding and optimizing the spelling accuracy of proper nouns and industry terminology.
  • Edge-optimized design: Voxtral Realtime achieves efficient inference with a scale of 4 billion parameters and can run on consumer-grade hardware, balancing model capability and deployment cost while supporting privacy-first local processing.

Voxtral Transcribe 2 project address

  • Project official website：https://mistral.ai/news/voxtral-transcribe-2
  • HuggingFace model repository：https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602

Application scenarios of Voxtral Transcribe 2

  • Meeting intelligence: The model can transcribe multilingual meeting recordings and clearly label speaker identities through speaker diarization, processing large-scale meeting content at extremely low unit cost to achieve efficient meeting notes and knowledge management.
  • Voice assistants and virtual assistants: Build conversational AI with ultra-low latency under 200 milliseconds, connecting large language models with speech synthesis pipelines to create voice user interfaces with natural responses and smooth interaction.
  • Call center automation: Transcribe call content in real time, enabling AI systems to analyze customer sentiment, recommend response scripts, and automatically populate CRM fields during the call, while speaker diarization ensures clear distinction between agent and customer speech.
  • Media and broadcasting: Generate real-time multilingual subtitles with minimal latency; contextual biasing accurately handles personal names and technical terms that general transcription services struggle to recognize.