LLM Gateway is an open-source LLM API gateway whose goal is to unify calls to multiple large model providers with a single API. According to official information, it supports more than 40 providers and over 250 models, including OpenAI, Anthropic, and Google. It provides an OpenAI-compatible API interface, so users can continue to use their existing OpenAI SDK code and only need to change the base URL to integrate.
Main capabilities include:
- Unified API interface: Keep the existing OpenAI SDK code and use it by changing the base URL.
- Multi-provider support: Access more than 40 providers through a single integration, switch models without changing code, and avoid vendor lock-in.
- Performance monitoring: Compare the latency, cost, and quality of different models to help choose the model suitable for a specific use case.
- Secure key management: Centrally manage all provider keys in one dashboard, reducing the risk of scattered or leaked credentials.
- Self-hosted or cloud: Can run on your own infrastructure for full control, or use the official hosted service.
- Cost analysis: View the number of requests, number of tokens, total spend, and average cost per 1K tokens over 7 or 30 days; break down usage and spend by provider and model to quickly spot high-cost anomalies.
- Error and reliability monitoring: Monitor error rates, cache hit rates, and reliability trends in the dashboard.
- Project-level usage: View requests, models, errors, cache, and costs by project, with dedicated charts and tables.
- Enterprise audit logs: Track who did what and when, retaining a complete audit trail to support compliance.
- Smart guardrails: Prevent prompt injection, detect PII, and block malicious requests.
How to use it: Officially, it is said that you can start routing requests in about 30 seconds. Send one request in your application, and the gateway will automatically route it to OpenAI, Anthropic, Google, or one of more than 40 other providers. It supports using its AI SDK provider, any OpenAI SDK, cURL, or any language; you can run DevPass Code, Claude Code, Codex, OpenCode, or Empryo on any model, and switch models by changing one string. Examples of supported endpoints include: POST /v1/chat/completions, POST /v1/responses, POST /v1/images/generations, POST /v1/images/edits.
Production controls: Scoped keys, spend caps, failover, and guardrails all live in the gateway, not in application code. The Enterprise edition supports SSO (Okta, Entra ID, or another SAML 2.0 provider, including SCIM provisioning), roles by organization and project, and API keys scoped per application. It can detect email addresses, card numbers, secrets, prompt injection, and jailbreak attempts in chat requests before a provider sees sensitive data, and choose per organization or project to block, redact, or warn. Spend caps can be set for any API key or member; once reached, requests stop, with optional alerts before a key's quota runs out; the Enterprise edition supports team budgets. When a provider errors, times out, or rate-limits, the gateway retries within the same request with another provider for that model up to two times (requests pinned to a provider stay with that provider).
Deployment:
- Enterprise Cloud: operated by the official team, offering a 99.9% SLA and dedicated support.
- Self-hosted: deploy with Docker or Kubernetes (official Helm chart) in your own network; enterprise features require a license key. The core is open source under AGPLv3, and the enterprise code is source-available.
- Provider policies (Enterprise edition): route only to providers that comply with the organization's rules on headquarters country, data retention, and training policies.
Pricing and plans: The official offering includes a free tier with no credit card required; Bring Your Own Keys is supported for free use of your own provider API keys. There are also five product lines (API, analytics, AI coding plans, chat and creation, model listing), with routing, billing, and data analytics the same across product lines. Specific pricing materials mention only the chat product: fast models start at $9 per month, flagship models start at $19 per month; detailed pricing for other plans is not yet available.
Other information: The official team states that each provider averages about 94% uptime independently, and with multi-provider automatic failover, the probability of simultaneous outages is near zero, so effective uptime can reach 99.9999%. Differences from OpenRouter include: full self-hosting under the AGPLv3 license; deeper real-time cost and latency analytics; free Bring Your Own Keys; and flexible enterprise add-ons (dedicated shards, custom SLA).
