agentlens

AgentLens is an open-source observability and audit trail platform for AI agents, positioned as the "flight recorder" for AI agents. It captures every LLM call, tool call, approval decision, and error, and presents them through a queryable API and a real-time web dashboard.

Core Feature: Tamper-Proof Design

Unlike other observability tools, every event in AgentLens is linked to the previous record through a SHA-256 hash chain, similar to the way git commits and blockchains are linked. The audit log is append-only and cryptographically verifiable—modifying, deleting, or reordering any record after the fact will cause verification to fail and will pinpoint exactly which event broke the chain. This design was built specifically to satisfy the record-keeping obligations of Article 12 of the EU AI Act and the emerging IETF Agent Audit Trail work. Auditors can obtain a signed, verifiable JSON snapshot via GET /api/audit/verify/export.

Main Capabilities

• Tamper-proof audit trail: append-only event storage maintained with a SHA-256 hash chain per session.
• LLM call tracing: full prompt/completion visibility, token usage, cost aggregation, latency measurement, and privacy redaction.
• Real-time dashboard: session timeline, event browser, LLM analytics, cost tracking, and alerts.
• Cost tracking: track token usage and estimated costs by session, by agent, and by model, with alerts for cost spikes.
• Alerts: configurable rules covering error rates, cost thresholds, latency anomalies, and stalls.
• Health scoring: 5-dimension health scoring with trend tracking.
• Cost optimization: a complexity-aware model recommendation engine that provides estimated savings.
• Session replay: step-by-step replay of any historical session with full context reconstruction.
• A/B benchmarking: statistical comparison of agent variants using Welch's t-test and chi-square analysis.
• Guardrails: automated safety rules with a dry-run mode for safe testing.
• Framework plugins: LangChain, CrewAI, AutoGen, Semantic Kernel, with automatic detection, fail-safe behavior, and non-blocking operation.
• Tenant isolation: multi-tenant support, with per-tenant data scoping bound to API keys.
• Self-hosting: SQLite by default, no external dependencies, MIT license.

How to Use

Five integration methods are provided, which can be chosen according to your tech stack:

1. Start the server and dashboard with a single command (SQLite, zero-configdeployment, Docker required).
2. Run directly without Docker.
3. Full stack (Postgres + Redis, authentication, TLS), run from source.
4. OpenTelemetry: if the agent is already instrumented according to the OTel GenAI semantic conventions (via OpenLLMetry, OpenInference, or official OTel instrumentation), simply point the OTLP exporter at AgentLens, no AgentLens SDK required. Each OTel trace maps to one session (or use gen_ai.conversation.id), and events are likewise protected by the hash chain. OTel only reports tokens and rarely reports cost; AgentLens reconstructs costUsd based on per-million-token model pricing (fuzzy-matched by model id). In production, OTLP_AUTH_TOKEN can be set to require a bearer token.
5. Python auto-instrumentation: one line of code automatically captures every LLM call across 9 providers (OpenAI, Anthropic, LiteLLM, AWS Bedrock, Google Vertex AI, Google Gemini, Mistral AI, Cohere, Ollama), guaranteeing determinism, fail-safety, non-blocking behavior, and supporting init(redact=True) for privacy redaction.

It also provides MCP integration: for Claude Desktop, Cursor, or any MCP client, just add it to the configuration. AgentLens ships with 22 built-in MCP tools covering core observability, intelligence and analytics, and operations. A programmatic SDK (TypeScript / Node.js, npm) is also provided.

For a quick local trial, you can use AUTH_DISABLED=true (the hardened image still requires JWT_SECRET). Any shared environment should remove AUTH_DISABLED, set a real JWT_SECRET, and create an API key (als_... prefix, shown only once).

Dashboard features

The real-time web dashboard includes: Overview (real-time metrics such as sessions, events, errors, active agents, a 24-hour event timeline chart, recent sessions and error stream), Session list (sortable by agent name, status, starttime, duration, event count, error count, total cost), Session details (full event timeline and hash chain verification, filterable by event type, with cost breakdown), Event browser (a searchable, filterable view across all sessions), LLM analytics (total call count, cost, latency, token usage, and model comparison), LLM call pairing in the session timeline, Prompt details (chat-bubble-style viewer with a metadata panel), Health overview, Cost optimization suggestions, Session replay, Benchmarks, Guardrail management.

Use cases

Teams that need auditable, verifiable run records for AI agents, especially scenarios subject to the record-keeping obligations of Article 12 of the EU AI Act; developers who need to monitor and optimize cost, latency, and tokens for LLM calls; organizations that need multi-tenant isolation and self-hosted deployment.

Limitations and requirements

• Self-hosting uses SQLite by default; the full stack requires Postgres + Redis.
• Building from source requires Node.js ≥ 20.0.0 and pnpm ≥ 10.0.0.
• The quick demo requires Docker.
• Cloud hosting (AgentLens Cloud) is a managed SaaS offering managed Postgres (multi-tenant row-level security), team features such as organizations/RBAC/audit logs, with the dashboard at app.agentlens.ai; specific billing and pricing information is not yet available.
• Integrates with the AgentKit ecosystem, including AgentGate, FormBridge, Lore, AgentEval, and others.