MAI-Transcribe-2-Streaming - A Speech Transcription Model Launched by Microsoft

What Is MAI-Transcribe-2-Streaming

MAI-Transcribe-2-Streaming is Microsoft's first real-time streamingspeech transcriptionmodel, capable of continuously outputting text while speech is in progress, covering 60 languages with automatic language-switch detection, with first-batch text latency as low as a few hundred milliseconds and text displayed as fast as 320 milliseconds after speech. MAI-Transcribe-2-Streaming tops the Artificial Analysis streaming transcription leaderboard with a 2.50% word error rate and 0.13-second latency, making it suitable for real-time subtitles, customer service agents, and other scenarios that require understanding while listening.

Main Features of MAI-Transcribe-2-Streaming

• Real-Time Streaming Transcription: Continuously outputs text while speech is in progress, without waiting for the sentence to end.
• 60-Language Coverage: A single model supports speech recognition in 60 languages.
• Automatic Language Detection: No need to preset a language; it can automatically recognize and follow language switches during a conversation.
• Ultra-Low-Latency Output: The first batch of text is generated within a few hundred milliseconds after receiving audio, with text displayed as fast as 320 milliseconds after speech.
• High-Precision Transcription: On the Artificial Analysis streaming leaderboard, it achieves a final word error rate of 2.50% and ranks first in both final and partial transcription accuracy.

Technical Principles of MAI-Transcribe-2-Streaming

• Incremental Streaming Processing Architecture: The model processes audio streams in small chunks while receiving them, eliminating at the architectural level the wait of traditional batch transcription where "results come only after speaking finishes," compressing latency from seconds to hundreds of milliseconds and laying the foundation for real-time conversation scenarios.
• Two-Stage Output Mechanism of Partial Hypotheses and Stable Commit: After receiving audio, the model first quickly generates partial hypotheses, i.e., preliminary text results; it continuously corrects the hypotheses using subsequent context that keeps arriving, and commits a stable final transcription once the sentence ends. This mechanism balances speed and accuracy: applications can immediately start understanding based on partials.
• Unified Multilingual Modeling and Continuous Language Detection: 60 languages are carried by a single model, with no need to specify a language before a session; the model determines the language based on the speech content itself and can continuously follow when the speaker switches languages. This avoids the overhead of multi-model routing and interruptions during language switching.
• "Listen While Understanding" Design for Agent Workflows: Applications can infer user intent based on partials before a sentence is finished, starting reasoning or tool calls early, changing transcription, understanding, and execution from serial to parallel and significantly shortening the end-to-end response time of voice agents.

How to Use MAI-Transcribe-2-Streaming

• Register an Azure Account: Create a Microsoft Foundry project and obtain an API key and endpoint.
• Choose an Access Method: Call via the Azure Speech SDK/REST API streaming interface, or call the model API directly on OpenRouter, Vercel, or LiveKit.
• Establish a Streaming Audio Channel: Continuously push microphone or real-time audio streams to the model in small chunks, rather than uploading a complete recording file.
• Receive and Process Partial Results: Listen for partial text returned by the model (arriving within a few hundred milliseconds) to display real-time subtitles or trigger intent understanding early.
• Submit stable transcription results: At the end of an utterance, obtain the final transcription text committed by the model, for storage, logging, or subsequent processing.
• Handle multilingual scenarios: No need to preset the language; the model automatically detects the spoken language and continuously follows when it switches.

Core advantages of MAI-Transcribe-2-Streaming

• Industry-leading accuracy: A 2.50% word error rate tops the Artificial Analysis streaming transcription leaderboard (28 models in total), ranking first in both partial and final transcription accuracy.
• Extremely low transcription latency: Final transcription latency is only 0.13 seconds, with text displayed as fast as 320 milliseconds after speech, while the closest competitor needs more than 500 milliseconds.
• Real-time architecture that understands while you speak: Streams partial results, allowing applications to start inference and tool calls without waiting for the user to finish speaking, shortening the end-to-end response of voice agents.
• 60 languages ready out of the box: A single model covers 60 languages, automatically and continuously detecting and following language switches, eliminating the hassle of multi-model routing and manual configuration.
• Flexible deployment options: Supports direct APIs for Microsoft Foundry and Azure Speech, as well as integration with multiple third-party platforms such as OpenRouter, Vercel, and LiveKit.

Comparison of MAI-Transcribe-2-Streaming with similar competitors

Application scenarios of MAI-Transcribe-2-Streaming

• Real-time captions and accessibility services: In livestreams, video conferences, and classrooms, text appears along with speech, with 320-millisecond-level display speed approaching "what you say is what you see," greatly improving the experience for people with hearing impairments.
• Customer service center agents: Before the caller finishes speaking, the system has already started recognizing the issue and retrieving information, shortening average handling time and increasing concurrent reception capacity during peak periods.
• Voice assistants and AI Agents: Based on high-accuracy partial results, inference and tool calls are started in advance, turning "listen-think-act" from serial to parallel, so conversations no longer have awkward pauses.
• Real-time translation for multilingual meetings: Automatic detection and switching among 60 languages; during multinational meetings, when a speaker changes language, no manual resetting is needed, and it can serve as the recognition front end for simultaneous interpretation systems.
• Real-time medical and legal documentation: In outpatient consultations and court records, drafts are produced while speaking; doctors/lawyers obtain a complete transcript immediately after finishing, significantly reducing documentation work time.