Purpose
mcp-drill is used for fault injection and reliability testing of MCP (Model Context Protocol) servers and AI agents. Real MCP deployments exhibit failure modes that integration tests never cover: tool timeouts, servers returning syntactically valid but semantically wrong payloads, responses truncated mid-stream. Most agents have never been exercised against these paths, and mcp-drill makes these failure paths reproducible.
Key Capabilities
- Fault Injection: Deterministically inject timeouts, malformed/oversized/truncated responses, corrupted but schema-conformant payloads, missing tools, and delays.
- CI-First: Provides a CLI and GitHub Action, with no real model or API key required in the loop.
- Reliability Scorecard: A model-free scan that assesses how a server responds to erroneous input and how machine-verifiable its tool output contracts are.
What Is Measured (Model-Free)
The scan command is deterministic and does not involve a language model, so its values are properties of the server and protocol themselves, rather than depending on whichever agent happens to invoke it:
- Error Consistency: For invalid requests (unknown method, unknown tool, missing required parameters), does the server return a specification-compliant JSON-RPC error, a correct tool error, or does it hang/crash/respond as if nothing happened?
- Output Contract Coverage: The proportion of the server's tools that declare a machine-verifiable outputSchema. Tools without one leave downstream validators with nothing to check.
- Output Contract Enforceability: Among tools that declare an outputSchema, how many actually reject corrupted (type-correct but semantically wrong) responses? Many declared schemas only validate shape, so corrupted payloads still pass the server's own contract.
Usage
Get started by wrapping any MCP server with a single command. The project provides a CLI and GitHub Action. The proxy/injector core is pure standard library; schema scoring uses jsonschema (scan extra). Telemetry is off, and the tool does not send data back.
Suitable Scenarios
- Test the failure paths of an MCP server in CI without a real LLM.
- Assess whether an MCP server's responses to erroneous input are specification-compliant.
- Check the coverage and enforceability of tool output contracts.
Limitations and Notes
The project is in early development. The license is Apache-2.0. For the distinction between mcp-drill (contract compliance) and mcp-scan (security), see the relevant comparison note.
