Qwen3.8-Omni-Flash - Alibaba Qwen's Natively Omnimodal Model

What is Qwen3.8-Omni-Flash

Qwen3.8-Omni-Flash is a natively omnimodal model launched by Alibaba Qwen. It supports text, image, audio, and video input as well as a 1M long context, and focuses on "from understanding to delivery" Agent capabilities, autonomously completingvideo editing、MV creation, short-drama translation, movie commentary, meeting-minutes execution, and other end-to-end workflows. Compared with the previous generation, it improves by over 25% on average, the API audio input price drops by over 98%, and it open-sources two supporting frameworks: Qwen-MM-Plugins and Qwen-Live Harness.

Main Features of Qwen3.8-Omni-Flash

• Controllable Audio-Video Captioning: Users can freely specify the description target, time range, information granularity, and output format, achieving "what the user wants to know" rather than "what the model saw."
• Agentic Long Audio-Video Understanding: Starting from the question, it autonomously decides where to look and what to listen to, locating key information through coarse-to-fine multi-round evidence gathering, improving accuracy while reducing token consumption by about 45.7%.
• Long Meeting Understanding and Execution: Natively supports one-hour audio-video input, end-to-end completing speaker segmentation, transcription, identity alignment, minutes generation, and to-do execution.
• In-Depth Audio-Video Research: Combining video content with user needs, it automatically retrieves multimodal materials and generates video-centered, richly illustrated research reports.
• Music2MV: Fine-grained understanding of song structure, rhythm, and emotion, outputting timestamped lyrics, running through the complete MV creation workflow from music understanding to final-cut quality inspection.
• Short-Drama Translation: Completes character dialogue recognition, colloquial translation, voice-cloned dubbing, audio track remixing, and quality inspection in one sentence, achieving automated delivery of dubbed translations.
• Long Movie Commentary: With a movie and a one-sentence request as input, it automatically completes full-film understanding, plot extraction, commentary scriptwriting, voiceover and music, editing and rendering, and quality inspection.
• Video2Note: Compresses hours of video into text-image-corresponding, timestamped PDF notes, and automatically reviews, iterates, and corrects them.
• Omni Skill Creator: Extracts SOPs from operation demonstrations or expert teaching, transforming them into validated, reusable Agent Skills.
• Real-Time Spoken Language Practice: Jointly models pronunciation and semantics, understands accent deviations, and generates standard pronunciation demonstrations in real time.

Technical Principles of Qwen3.8-Omni-Flash

• Native Omnimodal Unified Architecture: The model jointly models text, images, audio, and video within a single architecture, achieving cross-modal understanding through modality-aligned encoding and a unified representation space while maintaining text capabilities comparable to same-sized pure-text models; the 1M token long context enables it to natively accommodate hours of audio-video input, avoiding the information fragmentation caused by segmented processing.
• Agentic On-Demand Perception and Evidence Gathering: The model first scans at a coarse granularity to locate candidate segments, then retrieves relevant audio-visual content at a fine granularity for verification, forming a coarse-to-fine multi-round evidence chain and concentrating computation and token budget on truly relevant segments.
• Audio-Video and Agent Environment Collaboration: The bottleneck in making long audio-video agentic lies in the harness's lack of native audio-video support, so the official approach evolves the model and toolchain in tandem: Qwen-MM-Plugins provides on-demand perception, tool calling, and workflow execution capabilities, integrating with mainstream Agent frameworks such as Claude Code and OpenClaw to string material understanding, task planning, tool execution, and result delivery into a complete chain.
• Multi-speaker audio-visual collaborative recognition: The model jointly models the visual and audio streams, using visual information (the appearance and speaking state of people in the frame) to help resolve references and entity ambiguities in the audio, completing speaker segmentation, speech transcription, and identity alignment end-to-end, which greatly optimizes recognition metrics in meeting scenarios with multiple people speaking in turn and overlapping voices.
• Real-time streaming interaction architecture: Qwen3.8-Omni-Flash-Realtime receives audio and video streams via WebSocket/WebRTC, completing perception and response while input is ongoing, with a first-token latency of about 600-980ms and an audio generation RTF of about 0.153; spoken correction relies on joint modeling of pronunciation and semantics, and the model continuously updates its judgment as a new round of speech arrives, distinguishing errors that affect understanding from natural accents.

Follow WeChat and reply "open source" to join theAI Open Source Project Exchange Group

How to use Qwen3.8-Omni-Flash

• Use it directly on the web: Open the Qwen AI platformhttps://www.qianwenai.com/models/qwen3.8-omni-flash，搜索 qwen3.8-omni-flash, upload a video/audio/image and you can converse.
• API call: Apply for an Alibaba Cloud DashScope API Key, call it via an OpenAI-compatible interface, and pass in image, audio, and video links plus a text question.
• Install plugins to let it work: In Claude Code, Qwen Code and other tools, install Qwen-MM-Plugins, then just say like in a chat, "@video.mp4 help me make it into a commentary video," and the AI automatically completes the whole process.

Core advantages of Qwen3.8-Omni-Flash

• Unified omni-modality: Text, images, audio, and video are all handled by one model, with support for a 1M ultra-long context, and one-hour long videos can be natively input.
• From understanding to delivery: It does not just "understand" audio and video, but can also autonomously plan, call tools, and produce finished videos and documents, completing end-to-end tasks.
• Outstanding Agent capabilities: It leads by a wide margin on audio-visual Agent benchmarks, with WildClawBench-MM improving by 36.5 points over the previous generation, and long-horizon tasks reaching a high score of 69.6 on UniClawBench.
• Audio capabilities surpass Gemini: Its metrics for 60-language speech recognition, 39 Chinese dialects, and multi-speaker recognition are comprehensively better than Gemini 3.8 Flash.
• Higher efficiency and more savings: Agentic forensics reduces token consumption for long-video understanding by about 45.7%, while accuracy rises instead of falling.
• Significantly lower price: API pricing for hourly audio input drops by over 98%, and audio-video input drops by over 93%, greatly reducing the cost pressure of large-scale use.
• Complete open-source support: Qwen-MM-Plugins and Qwen-Live Harness are fully open source and compatible with mainstream Agent frameworks such as Claude Code and OpenClaw.

Project address of Qwen3.8-Omni-Flash

• GitHub repository：
  • Qwen-MM-Plugins: https://github.com/QwenLM/Qwen-MM-Plugins
  • Qwen-Live Harness: https://github.com/QwenLM/Qwen-Live-Harness

Comparison of Qwen3.8-Omni-Flash with similar competitors

Application scenarios of Qwen3.8-Omni-Flash

• Video content production: Enter a single request, and it automatically completes end-to-end delivery of movie commentary, MV production, and short-drama translation and dubbing.
• Meeting efficiency tools: Upload a one-hour meeting recording, and it automatically generates minutes and to-do items, and can even directly send emails and create tasks.
• Learning and knowledge retention: Automatically compress several hours of tutorial videos into PDF notes with screenshots and timestamps.
• In-depth video research: Automatically search the entire web for text and image materials related to video content, and generate in-depth research reports rich in both text and images.
• Real-time interactive services: Low-latency audio-video interaction scenarios such as real-time spoken language practice, intelligent customer service, and sound-based navigation.