llmkit

LLMKit is an open-source AI gateway and SDK suite whose core goal is to make users clearly aware of the cost of AI agents and to block requests when they exceed the budget. It covers three main areas: cost attribution, budget admission, and request evidence.

1. Main uses LLMKit is used to measure and control the API spending of AI agents. Before dispatching a request to a model provider, the gateway estimates and reserves the cost; if the request cannot satisfy the currently active budget, it is rejected; once the response is complete, the reserved amount is settled according to actual usage.

2. Multiple ways to use it

3. Local Python tracing: the transport layer reads the provider's usage metadata and estimates cost based on the built-in pricing catalog. This method does not send tracing data to LLMKit.

4. Zero-code CLI tracing: use -v to get per-request output, and use --json to get machine-readable results.

5. Gateway mode (requires an existing key): the gateway example requires a ready-made LLMKit API key. Account creation and key management are currently unavailable while the authentication service is being restored; if you do not have a key, you can use the local tracing methods above instead.

6. Budget control path The control path is built around three boundaries:

• Atomic admission: Durable Object holds reservation state for each budget scope, so concurrent requests cannot spend the same remaining balance.
• Dispatch-aware idempotency: deterministic failures that occur before dispatch release the key; after provider dispatch may have occurred, failures remain in a terminal state to avoid duplicate spending.
• Bounded responses: non-streaming response bodies and individual SSE frames both have explicit byte limits, and when the limit is exceeded LLMKit cancels the upstream read. Request receipts bind admission decisions, provider attempts, settlement, and analytics handoff together through stable identifiers. Database writes use the outbox pattern, so a disruption in the analytics service does not silently erase budget evidence.

1. Tool set Five local tools can inspect supported Claude Code sessions and Cline task data without an LLMKit key. Six gateway tools can query spend, budgets, keys, sessions, and service health when LLMKIT_API_KEY contains a valid key. In total, 11 tools are provided.

2. Pricing catalog The fixed catalog is a built-in reference snapshot, not a live quote. A single source file, packages/shared/pricing.json, records the snapshot date and generates TypeScript, Python, and MCP tables. CI rejectsdrift between source files and generated files. The public site renders only provider tables that have data, and displays the source date. The public comparison endpoint can be used without an account; it prices only the exact model keys provided by the caller, and does not search for or recommend the cheapest model. Pricing is an estimate, not a provider invoice; provider billing rules, model modalities, and catalog freshness are all part of the margin of error.

3. Security and deployment Provider credentials are encrypted with AES-256-GCM, using a random IV and additional authenticated data binding the owner/provider. LLMKit API keys are hashed before storage. CI includes secret scanning, static analysis, dependency review, CodeQL, and package provenance checks. For local use, a development-only binding can be used to run the Worker. General deployment commands are intentionally omitted; staging and production use their own protected scripts and require explicit confirmation of the target.

4. Limitations Account creation and key management are temporarily unavailable while the authentication service is being restored. The pricing catalog is a reference snapshot rather than live pricing. Links to the project website, documentation, pricing API, architecture, and security guarantees are provided in the repository, but details such as specific charges and prices are not provided.