Qwen-Audio-Agent - Alibaba's Open-Source Real-Time Voice Agent Framework

What is Qwen-Audio-Agent

Qwen-Audio-Agent is an open-source real-time voice Agent framework launched by Alibaba's Voice AI team, built on the Qwen-Audio-3.0-Realtime model. As a unified real-time voice entry layer, the framework lets users converse in full-duplex with backend Agents such as OpenCode、OpenClaw、Codex through natural speech, supporting listening while speaking and interruption at any time, delegating complex tasks to the backend for execution and bringing back the results.

Main Features of Qwen-Audio-Agent

• Full-Duplex Real-Time Voice: Supports continuous speaking and natural interruption. While the Agent is executing a task, users can supplement or correct their requirements at any time without waiting for the previous turn to end.
• Agent Ecosystem Integration: Already compatible with mainstream Agents such as OpenCode, OpenClaw, Qoder, Hermes, CodeBuddy, and Codex, and supports the ACP stdio protocol to extend to more backends.
• Task Delegation and Context Continuity: Simple questions are answered immediately by the real-time voice frontend, while complex tasks automatically pass the context to the backend Agent Runtime for execution, with results broadcast back into the conversation in real time.
• Multiple Interaction Interfaces: Provides a TUI terminal interface, a WebUI web version, and a macOS desktop voice floating ball (supporting two appearances: the flowing light soundwave ball and the liquid gradient ball).
• Configuration-Based Management: Through qwenaudio config , quickly configure the DashScope API Key and the backend Agent protocol, and switch between different Agent backends with one click.

Technical Principles of Qwen-Audio-Agent

• Full-Duplex Real-Time Voice Interaction Architecture: The core voice layer of Qwen-Audio-Agent is based on the Qwen-Audio-3.0-Realtime model and uses a full-duplex communication mechanism to listen while speaking. The architecture supports users interrupting at any time while the Agent is broadcasting or executing. The system truncates the current output stream in real time and re-parses the new voice input, simulating the natural turn-taking rhythm of human conversation.
• Frontend-Backend Separated Task Delegation Model: The system adopts a two-layer architecture decoupling design of a real-time voice frontend + Agent Runtime backend. The frontend handles real-time transcription of voice signals, semantic understanding, and immediate responses; when deep tasks such as search, reasoning, or code modification are detected, the frontend delegates the current conversation context to the backend Agent for execution through a standardized interface. After completing the computation, the backend calls the results back to the frontend in text or voice form, and the frontend broadcasts them uniformly, avoiding complex tasks blocking the real-time interaction flow.
• Context Continuity and State Management: In multi-turn continuous conversations, the system maintains a unified dialogue state machine. User interruptions, supplements, or direction changes do not clear the existing context, but instead incrementally append to the current session state. When a task is delegated to the backend Agent, the relevant context is serialized and passed along; the results returned by the Agent are automatically merged into the current session, ensuring that users are always in a coherent semantic environment while chatting and working at the same time.

Follow WeChat and reply "open source" to join theAI Open Source Project Exchange Group

How to Use Qwen-Audio-Agent

• Environment Preparation: Ensure that the Node.js environment is installed and that the DashScope API Key is ready.
• Global Installation: Execute npm install -g qwen-audio-agent to complete the command-line tool installation.
• Create Configuration: Run qwenaudio config, fill in the DashScope API Key, and select the backend Agent protocol (e.g., opencode）。
• Launch TUI: Run qwenaudio tui to open the terminal voice interaction interface and start real-time conversation.
• Launch WebUI: Run qwenaudio webui to open the web-based voice interaction interface in your browser.
• Desktop experience: After cloning the source code, run npm install && npm run desktop to launch the macOS desktop floating ball, which stays in a corner of the screen for voice activation at any time.

Core advantages of Qwen-Audio-Agent

• Natural interaction experience: Full-duplex voice makes collaboration feel more like a real conversation, reducing input costs and improving communication efficiency and emotional value.
• Zero migration cost: It does not replace existing Agents; it directly reuses the user's existing toolchain, project context, and permission system.
• Clear architectural decoupling: The real-time voice frontend and Agent backend are separated, each fulfilling its own role, making independent iteration and expansion easier.
• Multi-platform coverage: TUI, WebUI, and desktop floating ball, three forms that adapt to different work scenarios and usage habits.
• Open source and extensible: Based on the ACP standard protocol, developers can freely integrate custom Agent backends and business logic.

Project address of Qwen-Audio-Agent

• GitHub repository：https://github.com/QwenAudio/qwen-audio-agent

Comparison of Qwen-Audio-Agent with similar competitors

Application scenarios of Qwen-Audio-Agent

• Programming collaboration: While writing code, developers can use voice to direct the Agent to modify files, run tests, and check errors, interrupting at any time to add requirements.
• Project management: Continuously query the progress of multiple projects, delegate tasks, and obtain execution results by voice, without switching chat windows.
• Document processing: Voice commands drive the Agent to generate, modify, and translate documents, confirming content in real time and iterating for optimization.
• Intelligent customer service: Enterprises deploy it as a private voice frontend, connecting to internal business Agents to provide natural and smooth customer interaction.
• Accessibility assistance: Provides an entry point for users who have difficulty typing to control the computer by voice, completing complex system operations through the Agent.