Xuanwu CLI - Qingmao Intelligence's open-source deployment tool for domestic large models

What is Xuanwu CLI

Xuanwu CLI (xw-cli) is an open-source deployment tool for domestic large models by Qingmao Intelligence, deeply optimized for domestic chips such as Huawei Ascend, MetaX, and Enflame. Through automatic hardware detection and intelligent engine scheduling, users can launch model services with one click without complex configuration. The tool is compatible with Ollama command conventions and the OpenAI API, supports DeepSeek、Qwen3 and other mainstream models, completes deployment within 5 minutes, and thoroughly solves the ecosystem problem that domestic computing power is "usable but not easy to use."

Main features of Xuanwu CLI

• One-click deployment: Users do not need to install Python or configure complex dependencies; they only need to unzip and run it, and can start a production-grade model service in as little as 1 minute.
• Intelligent hardware recognition: The system automatically detects multiple domestic chip models such as Huawei Ascend, MetaX, and Enflame, with no need to manually specify the backend or adjust parameters.
• Model management: Provides command-line operations highly consistent with Ollama (such as xw pull to pull models,xw run to run conversations,xw ls to view local models), reducing the cost of migration and learning.
• Multi-engine scheduling: Built in with the self-developed MLGuider inference engine and compatible with third-party engines such as vLLM, it intelligently selects the optimal backend according to the scenario to balance performance and compatibility.
• API compatibility: Fully supports the OpenAI API format; LangChain, LlamaIndex, and various IDE plugins can be integrated seamlessly by simply changing the API address.
• Offline operation: All model management and inference tasks are completed locally without relying on cloud services, meeting the compliance requirements of high-privacy scenarios such as finance and healthcare.
• Ecosystem integration: As an underlying capability foundation, it works with local AI tools such as Clawdbot to provide model support for automated tasks and intelligent applications.

Technical principles of Xuanwu CLI

• Heterogeneous computing abstraction layer: Builds a unified hardware abstraction interface at the底层, converging differences among different chip architectures such as Huawei CANN and Moore Threads MUSA into internal system processing. When users execute commands, the program automatically identifies the current hardware type, completes driver version matching, environment variable injection, and runtime initialization, and provides a standardized model service interface to the upper layer, allowing users to achieve "zero-debug deployment" without understanding underlying architecture details.
• Intelligent engine routing: The system has a built-in self-developed MLGuider high-performance inference engine and is also compatible with open-source engines such as vLLM, forming a multi-engine coexistence architecture. During the model loading stage, Xuanwu CLI automatically selects the optimal execution path based on factors such as chip computing characteristics, model architecture type (such as Dense or MoE), and quantization precision.
• Subprocess isolation architecture: To ensure service stability, Xuanwu CLI adopts an architecture design of a main process plus independent subprocesses. Each model instance runs in an independent subprocess, so a crash or exception in a single task will not affect the normal operation of the main process and other models.
• Model-framework-operator joint optimization: Targeting the instruction set characteristics and memory bandwidth features of domestic chips, the team develops customized operators for new model architectures (such as FP8 quantization and MoE sparse structures), ensuring that popular models can be adapted on the day of release (Day0) and achieve ideal performance, fundamentally solving the pain point of domestic computing power being "unsuitable for new models".

Xuanwu CLI project address

• GitHub repository：https://github.com/TsingmaoAI/xw-cli

Xuanwu CLI application scenarios

• Local AI development environment setup: Developers can quickly deploy open-source models such as DeepSeek and Qwen3 on personal workstations or servers, without purchasing expensive NVIDIA graphics cards or Mac devices, directly activating the domestic computing power resources at hand for model debugging and application development.
• Enterprise private deployment: The tool can meet the needs of industries with extremely high data security requirements, such as finance, healthcare, and government affairs, enabling models to run completely offline, avoiding the upload of sensitive data to the cloud, while reducing long-term API call costs.
• Agent infrastructure: As the underlying model foundation for local AI tools such as Clawdbot, it provides stable, low-latency inference capability support for Agent applications such as automated code generation, bug fixing, and voice interaction.
• Domestic computing power ecosystem validation and promotion: Helps chip manufacturers and integrators quickly validate the large model running performance of domestic hardware, lowers the threshold for developers to try it out, promotes the transformation of domestic AI chips from "usable" to "easy to use", and accelerates ecosystem development.