FLM-Audio - A Fully Duplex Audio Dialogue Model Open-Sourced by BAAI

What is FLM-Audio

FLM-Audio is a native fully duplex audio dialogue large model jointly released by the Beijing Academy of Artificial Intelligence (BAAI), Spin Matrix, and Nanyang Technological University, Singapore, supporting both Chinese and English. It adopts a native fully duplex architecture that can merge auditory, speaking, and monologue channels at each time step, avoiding the high latency issues of traditional time-division multiplexing schemes. Its unique natural monologue and dual training paradigm brings the model closer to natural human communication during dialogue, effectively solving the asynchronous alignment problem. FLM-Audio is trained with only 1 million hours of data, a significantly reduced data volume, yet delivers high-quality responses and agile, natural responses, with strong robustness against noise and user interruptions.

Main Features of FLM-Audio

Fully Duplex Voice Interaction: Enables "listening while speaking"; users can interrupt the model at any time, and the model can immediately pause output, understand the new question, and respond, resulting in natural, smooth interaction with low latency.
Multilingual Support: The model supports both Chinese and English, meeting the dialogue needs of users in different languages.
Natural Speech Modeling: Adopts a "natural monologue" approach to simulate human speech rhythm; through "dual training," it strengthens language and acoustic semantic alignment, balancing low latency and language modeling performance.
Low-Data Efficient Training: Trained a 7-billion-parameter model with only about 1 million hours of audio data, maintaining high robustness and naturalness in noisy environments and frequently interrupted scenarios.
Strong Robustness: Strong robustness against noise and user interruptions; it can quickly pause the current output, accurately understand new questions, and respond immediately, ensuring dialogue fluency and accuracy.
Fully Open-Source Support: The paper, model weights, and code are all publicly available, supporting local deployment and secondary development, facilitating research and application expansion.

Technical Principles of FLM-Audio

Native Fully Duplex Architecture: The model design supports simultaneous voice input and output, can process voice streams in real time, and enables a listening-while-speaking interaction mode.
Natural Monologue Training: Uses "natural monologue" composed of continuous sentence segments and pauses instead of word-by-word alignment, closer to real human speech and improving the naturalness of voice interaction.
Dual Training Strategy: By alternately placing monologues at the beginning and end of audio for training, it strengthens language and acoustic semantic alignment and improves the model's understanding and generation of speech content.
Small-Data Efficient Training: Uses a small amount of audio data (about 1 million hours) to train a high-parameter model, achieving low latency and high robustness through optimized training methods and architecture.

FLM-Audio Project Address

GitHub Repository：https://github.com/cofe-ai/flm-audio
HuggingFace Model Library：https://huggingface.co/CofeAI/FLM-Audio
arXiv Technical Paper：https://arxiv.org/pdf/2509.02521

Application Scenarios of FLM-Audio

Online Education: AI teaching assistants can answer students' questions in real time, providing a more natural and efficient interactive experience.
Games and Virtual Reality (VR): NPCs can achieve uninterrupted, interruptible natural voice interaction, enhancing the immersion of games.
Intelligent Customer Service: Low-latency dialogue reduces user waiting time, improving customer service efficiency and user experience.
Intelligent Companionship: Provides users with voice interaction closer to that of a real person, enhancing the sense of companionship.
Voice Assistant: In scenarios such as smart homes and smart offices, it provides a more natural voice interaction experience.
Meeting Assistance: In multi-person meetings, real-time translation, recording, and interaction improve meeting efficiency.