Qwen-CUA - Native Computer Use Agent launched by Alibaba Qwen and others

What is Qwen-CUA

Qwen-CUA is a native Computer Use Agent jointly launched by the Qwen team and XLang Lab. Based on a 397B-A17B mixture-of-experts architecture, it perceives interface states only through screenshots, without relying on the DOM tree or accessibility metadata, and directly uses keyboard and mouse events to control browsers, desktop applications, and professional software. It achieved a score of 86.2 on the OSWorld-Verified benchmark, while the larger Qwen-CUA-Max version, with over one trillion parameters, reached 87.6.

Main Features of Qwen-CUA

• Pure visual perception: Understands interface states only through screenshots, without relying on HTML structure or accessibility metadata, providing stronger cross-platform generality.
• Native input simulation: Directly outputs keyboard and mouse events, and can perform operations in browsers, desktop applications, and professional software.
• MoE architecture foundation: Based on a 397B-A17B mixture-of-experts model, balancing performance and inference cost; the Max version has over one trillion parameters for further performance improvement.
• Open technical report: A technical report and reference Demo have been released on GitHub, facilitating reproduction and secondary development by researchers.

Technical Principles of Qwen-CUA

• Foundation architecture: Based on the Qwen3.5-397B-A17B mixture-of-experts architecture, with 397B total parameters and only 17B activated per forward pass, controlling inference cost while maintaining strong expressive capability.
• Hybrid attention mechanism: Uses GatedDeltaNet linear attention and Gated Full Attention arranged alternately at a 75:25 ratio, reducing the computational complexity of long sequences from O(n²) to near-linear, and increasing 256K context throughput by 19 times compared with traditional architectures.
• Expert routing: Each layer is configured with 512 fine-grained routed experts, activated through Top-10 selection + weight renormalization, and combined with 1 shared expert and sigmoid gating to adaptively regulate the contribution of general knowledge, avoiding information loss.
• Pure visual perception: Perceives interface states only through screen screenshots, completely independent of the HTML DOM tree, accessibility metadata, or operating system APIs, enabling cross-platform general perception of any GUI application.
• Visual encoding and localization: Inherits the ViT architecture of the Qwen-VL series to support dynamic-resolution input, and combines MRoPE multimodal rotary position embedding to simultaneously encode three-dimensional information of time, height, and width, accurately localizing interface element coordinates.

How to Use Qwen-CUA

• Visit the repository: Go to GitHub and search for xlang-ai/Qwen-CUA to enter the project homepage.
• Read the technical report: Download and read the technical report in the project to learn about the model architecture and training details.
• Run the reference Demo: Configure the environment according to the README, and run the official reference Demo to experience screenshot-driven automated operations.
• Connect to your own environment: Connect the model to a local or cloud desktop/browser environment, and execute tasks in a closed loop through screenshot input and keyboard/mouse output.

Core Advantages of Qwen-CUA

• High benchmark score: 86.2 on OSWorld-Verified, and 87.6 for the Max version, placing it at a leading level in the open-source CUA field.
• Truly native CUA: Unlike solutions that rely on APIs or multi-Agent assembly, Qwen-CUA is an end-to-end trained vision-action model with higher stability.
• Cross-platform compatibility: Not limited by operating system or application type; any software with a graphical interface can be controlled.
• Open-source ecosystem: Apache 2.0 license; technical report and demo are publicly available, and the community can freely extend it.

Qwen-CUA project address

• GitHub repository：https://github.com/xlang-ai/Qwen-CUA
• Technical paper：https://github.com/xlang-ai/Qwen-CUA/blob/main/paper/Qwen-CUA.pdf

Comparison of Qwen-CUA with similar competing products

Application scenarios of Qwen-CUA

• Automated software testing: Recognizes interface elements through screenshots and performs operations such as clicking, inputting, and scrolling, enabling cross-platform UI regression testing without writing specific scripts.
• RPA process automation: Replaces manual work to complete repetitive desktop office tasks such as data entry, report generation, and ERP operations, without needing to modify the enterprise's existing IT systems.
• Accessibility assistive tools: Automatically recognizes software interface elements for visually impaired users or the elderly and performs precise operations on their behalf, lowering the threshold for using professional software.
• Scientific research and professional software control: Automatically controls professional software without API interfaces, such as MATLAB and SPSS, to batch-execute long-cycle tasks such as experimental data processing and chart generation.
• Cloud Agent hosting service: Deployed in a cloud desktop environment to perform background tasks such as website inspections, data scraping, and scheduled report downloads 24/7.