SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI codingmcp-observatory

mcp-observatory

Regression testing for MCP servers: discovers configured servers, checks capabilities, invokes tools, detects schema drift and recommends new servers.

Open website

About this tool

Purpose

MCP Observatory is a CI-native security tool for teams building custom MCP (Model Context Protocol) servers. MCP servers are becoming production dependencies, and if agents rely on them, teams need to discover broken tools, insecure schemas, schema drift, slow responses, and security issues before failures reach users. Observatory is used to test during development, catch schema drift, simulate attacks, and generate compliance evidence before agents depend on your server.

Key Capabilities

  • Capability checks: Connect to the server and verify that tools, prompts, and resources respond correctly.
  • Tool invocation: Go beyond listing tools and actually invoke safe tools (no required parameters / readOnlyHint), and report which work and which crash.
  • Schema drift detection: Diff two runs and show added/removed fields, type changes, and breaking parameter changes.
  • Server recommendations: Scan the project for language, framework, database, and cloud provider, cross-reference the MCP registry, and recommend missing servers.
  • Security scanning: Analyze tool schemas for dangerous patterns, including shell injection surface, broad filesystem access, missing authentication, and credential leakage in responses.
  • Record / replay / verify: Capture live sessions, replay them offline in CI, and verify no changes, similar to VCR for MCP.
  • Regression monitoring: Re-run checks at intervals and alert when changes occur.
  • CI integration: Complete CI setup with one command (setup-ci --all); supports GitHub PR comments, GitHub Code Scanning SARIF, and GitLab SAST reports.
  • MCP receipts and risk graph: Package targets, evidence, verdicts, actions, and reproduction commands; group servers by capability boundary, receipt status, CI posture, and recommended action.
  • MCP server mode: After adding an AI tool as an MCP server, your AI agent can autonomously test, diagnose, and monitor other MCP servers, gaining 10 tools.

Usage

Installation:

npx -y @kryptosai/mcp-observatory@latest
brew tap kryptosai/mcp-observatory https://github.com/KryptosAI/mcp-observatory && brew install mcp-observatory
docker pull ghcr.io/kryptosai/mcp-observatory:latest
curl -fsSL https://raw.githubusercontent.com/KryptosAI/mcp-observatory/main/scripts/install.sh | sh

First scan: Run the above npx command in the project folder. It will download Observatory and check the first configured MCP server; if no server is configured, it will use the built-in example. To try only the example:

npx -y @kryptosai/mcp-observatory@latest demo --example

In-depth check (also invokes safe tools to verify they actually run):

npx -y @kryptosai/mcp-observatory@latest test --invoke-tools

Test a specified server:

npx -y @kryptosai/mcp-observatory@latest scan --target "npx -y my-mcp-server"

Generate a complete CI workflow:

npx -y @kryptosai/mcp-observatory@latest setup-ci --all --command "npx -y my-mcp-server" --sarif

Save a free hosted snapshot (optional): In the same project folder, run npx -y @kryptosai/mcp-observatory@latest cloud upload, open the verification link printed in the terminal, confirm the code, connect with GitHub, then return to the terminal and wait for Upload complete and open the dashboard link.

Add as an MCP server to Claude Code:

npx -y @kryptosai/mcp-observatory@latest serve

Deny connections by default:npx -y @kryptosai/mcp-observatory@latest protect rewrites every discovered client configuration (Cursor, Claude, VS Code, .mcp.json, etc.); to undo:protect --undo. HTTP/SSE servers are unaffected.

Deny by default at runtime:npx -y @kryptosai/mcp-observatory@latest enforce --start-proxy writes a Seatbelt policy based on the scan and starts the proxy.

During scanning, MCP configuration is looked for in the following locations:~/.claude.json（Claude Code）、~/Library/Application Support/Claude/claude_desktop_config.json（Claude Desktop，macOS）、%APPDATA%/Claude/claude_desktop_config.json(Claude Desktop, Windows), the current directory's .claude.json and .mcp.json.

Scanning servers on the host requires network access, and for local files also requires volume mounts (-v "$PWD:$PWD" -w "$PWD").

Applicable Scenarios

  • Teams developing their own MCP servers and needing regression testing and security checks in CI.
  • Security and platform teams: MCP servers are becoming part of the AI software supply chain, and reliable, testable, auditable tools are needed before critical workflows depend on them.
  • Maintainers who need to add public compatibility/security signal badges to MCP repositories.

Limitations

  • Servers requiring interactive OAuth (such as Google Drive) need to be pre-authenticated before Observatory can connect.
  • Custom WebSocket transports (such as BrowserTools MCP) are not supported.
  • A small number of servers time out or close before init; see the known issues and compatibility documentation for details.
  • In MCP server mode, to prevent prompt injection: the command allowlist only permits npx, node, python, python3, uvx, docker, deno, and bun as base executables; file reading tools are restricted to the runs/cassettes directory; arbitrary execution is not supported.

Pricing

Local scanning and local CI remain free. Hosted snapshots are free and include one target, the latest snapshot, seven-day retention, and ten uploads per month. Individual Pro is $29 per month for 10 targets, 90-day history, hosted CI ingestion, or hosted HTTP scanning. For production support, the MCP Release Gate Pilot has a fixed public introductory price of $15,000, covering 1-3 critical MCP servers over ten business days; broader work is scoped after a release decision.

Additional Notes

Observatory uses jurisdiction-aware product telemetry after displaying a collection notice. Available telemetry preview to view the exact fields, use telemetry status to view the current policy, or use telemetry disable、MCP_OBSERVATORY_TELEMETRY=0、DO_NOT_TRACK=1 to stop collection. The repository is open source under the MIT license.

Related tools

Browse category