Gateway is Portkey's open-source AI gateway, positioned as a fast, reliable, and secure routing layer for connecting to 1600+ language, vision, audio, and image models. It is officially described as a lightweight, open-source, and enterprise-ready solution that lets developers integrate with any language model in 2 minutes, features sub-1ms latency and a 122kb footprint, and processes over 10 billion tokens per day.
In terms of use, it primarily solves the problem of a unified entry point for multi-model access: through a unified API, it masks the differences between providers and, on that basis, provides reliability, security, cost management, and collaboration capabilities. It is officially described as an "extremely fast AI gateway with integrated guardrails," supporting routing to 1600+ LLMs and 50+ AI Guardrails.
Main capabilities include: reliable routing, supporting fallback to another provider or model on failure and allowing specification of the triggering error type; automatic retries up to 5 times with exponential backoff; load balancing across multiple API keys or providers by weight; fine-grained request timeouts; multimodal calls, allowing vision, audio (text-to-speech and speech-to-text), and image generation models to be called with OpenAI-style signatures; integration with OpenAI's Realtime API through server calls via WebSocket. In terms of security and accuracy, it provides 40+ preset Guardrails, also allows bringing your own guardrails or choosing partner solutions, and supports your own keys or instant generation of virtual keys, role-based access control, and SOC2, HIPAA, GDPR, and CCPA compliance. In terms of cost management, it supports simple caching and semantic caching, usage analytics (request volume, latency, cost, error rate), and provider optimization. In terms of collaboration and workflows, it can integrate with Autogen, CrewAI, LangChain, LlamaIndex, Phidata, Control Flow, and custom Agents, and provides prompt template management and versioning (some features are marked as provided by the hosted app). The project also includes an LLM pricing database for 40+ providers for cost tracking.
In terms of usage, the official quick start states: the gateway runs at http://localhost:8787/v1，网关控制台运行在 http://localhost:8787/public/。支持的库包括 JS, Python, REST, OpenAI SDK, LangChain, LlamaIndex, Autogen, CrewAI, etc. Configs can be used to create routing rules, increase reliability, and set guardrails. The enterprise edition supports private deployment, with platforms including AWS, Azure, GCP, OpenShift, and Kubernetes; the enterprise edition provides organization management, governance, security, and other capabilities. MCP Gateway provides a centralized control plane for MCP (Model Context Protocol) servers within an organization, with unified authentication, access control, observability (recording the context, parameters, responses, and latency of each tool call), and identity forwarding, and can be used with MCP-compatible clients such as Claude Desktop, Cursor, and VS Code.
Applicable scenarios include: AI applications that need to connect to multiple model providers and want a unified API and failover; teams that need caching, cost tracking, and usage analytics to control LLM expenses; enterprise deployments that need to add security and compliance validation at the input and output layers; and organizations that need centralized governance of MCP servers and agent workflows.
Limitations: the official materials do not provide the specific open-source license name, complete installation command details for the code repository (only the runtime address and supported library list are given), nor do they provide specific pricing for various features, SLA figures, or the complete test conditions for performance benchmarks; features such as semantic caching, provider optimization, and prompt template management are marked as provided on the hosted app. The above unspecified content is not yet provided.
