Higress is a CNCF Sandbox open-source AI gateway built on Istio and Envoy, providing unified access, traffic governance, security protection, and observability capabilities for LLM APIs, AI Agents, and MCP Servers, positioned as production-grade infrastructure for Agent development and LLM API management. It runs in parallel through two channels: the open-source version provided on the official website higress.ai, and a commercial cloud product, allowing users to experience AI application scenarios with one click.
Main capabilities Unified access and protocol conversion: The AI gateway can forward requests that externally call different large models to the corresponding internal large models through a unified invocation method, making backend model scheduling more flexible; it supports unified protocol conversion for 100+ commonly used models and supports model-level Fallback. It can connect to OpenAI, Claude, and domestic large models. Data security and content filtering: It provides privacy protection for data input to external models and content filtering for data output to users. Related materials mention that it can "make AI content security part of the gateway's main link without changing a single line of business code." Inference result caching: It supports exact caching and semantic caching of inference result contexts, saving Tokens and reducing latency for common similar or repeated questions, improving the invocation experience. Token tracking and quotas: It can track the number of Tokens used by large models and impose limits when consumer usage exceeds the quota, thereby better managing user quotas for invoking AI applications and providing data support for Token usage analysis. MCP service hosting: It receives MCP requests and completes protocol offloading, providing unified identity authentication, traffic scheduling, parameter mapping, and security auditing capabilities, so that developers can quickly expose existing services as MCP Servers without needing to deeply understand the details of the MCP protocol. The Higress MCP marketplace provides comprehensive solutions and can quickly turn data APIs into MCP tools. It embraces MCP 2026-07-28 and is compatible with existing protocols. Plugins and multiple data sources: It encapsulates the open-source wasm client to facilitate plugin development by developers, such as controlling the scope in which plugins take effect; the community already has many plugin capabilities that can be used directly; it integrates multiple mainstream registries, and multi-data-source capabilities can be reused directly, reducing duplicate construction.
How to use Materials show that no registration is required, and it can be experienced with a one-click Docker startup; it also provides a fully managed, operations-free commercial cloud version that automatically scales elastically based on invocation volume and uniformly carries LLM / Agent / MCP core capabilities. An optional enterprise edition is available for production environments, providing a 99.95% SLA. The official website also provides version comparison, pricing and a TCO calculator, as well as related content from the new version of MCP to Gateway API inference scaling.
Applicable scenarios For issues such as multi-model access and the lack of unified traffic governance that arise when applying AI technology at scale, enterprises can build an AI gateway based on Higress to uniformly manage access to large model services and MCP services, enabling functions such as authentication and authorization, traffic control, and model mapping. Related materials mention that Ctrip chose to build an AI gateway based on Higress; the system runs stably and supports multiple scenarios, and in the future it will optimize the ability to convert existing HTTP APIs into MCP services. Junrun Human Resources uses Higress in the process of building digital employees (agents) that deliver work results for different positions. As of now, more than 1,000 digital employees work 24/7, saving a cumulative labor cost of more than tens of millions. The SOFA AI gateway is built on the open-source Higress core, focusing on core functions such as security protection, traffic management and control, semantic caching, and building a financial MCP marketplace. Through intelligent routing and unified access, it addresses challenges in AI scenarios such as high latency and high resource consumption. Today's Investment launched the Today's Investment financial data MCP service, converting financial data APIs into MCP tools through the Higress MCP marketplace. Users can call professional financial data through MCP to build financial large model applications.
Limitations and other notes The existing materials do not provide specific installation command details, detailed price figures, a list of functional differences among versions, or performance benchmark data. The relevant information is not yet available.
