crw (project name fastCRW) is a fast, lightweight web crawling and search API written in Rust, positioned as an alternative to Firecrawl / Tavily. It integrates search, scrape, site map, crawl, and extract into a single engine that can convert URLs into clean Markdown or structured JSON.
Main capabilities:
- Provides a Firecrawl-compatible interface, including /scrape, /crawl, /search, and can serve as a migration replacement for Firecrawl.
- Built-in MCP server for AI agents, along with a CRW skill that can be automatically installed in detected AI tools.
- Officially published performance data: on Firecrawl's public 1,000 URL dataset, fastCRW's recall is higher than Crawl4AI and Firecrawl, median latency is on par with the fastest, and idle memory is about 14 MB; in another 1K URL benchmark test, it is claimed to be 2.3 times faster than Tavily and 1.5 times faster than Firecrawl. On AA-Omniscience's 600 questions, the answer accuracy is 90.0%.
- Small size, single binary file, officially described as about 6 MB RAM.
How to use:
- Install with one command, run locally for free, no account required; if you want to use the cloud service, you can paste the API key in the same command, and it will install the binary, connect the key, and register the MCP server in the already configured AI programming tools (Claude Code, Cursor, Codex, Gemini CLI, OpenCode, Windsurf will be automatically recognized). The key is stored in ~/.config/crw/config.toml and will not be copied into each tool. You can use CRW_NO_AGENTS=1 to skip this step, or run crw setup separately to choose interactively.
- Supports macOS and Linux, compatible with Intel and ARM.
- To use the cloud service, you first need to obtain an API key and export it: export CRW_API_KEY=your_key_here
- Provides REST API, Python and TypeScript SDK examples; self-hosting requires a Rust 1.85 or higher workspace.
- The cloud service provides managed proxies, JS rendering, and search, with no need to maintain them yourself; the official statement says there is a 1000 free quota, no credit card required, and you can log in with GitHub to get it.
Applicable scenarios: Developers who need to provide web scraping, crawling, and search capabilities for AI agents; projects that want to self-host or use a managed cloud service and need a Firecrawl-compatible API; individuals and teams that use crawling and search capabilities through MCP in AI programming tools.
Limitations and notes:
- The capabilities and response structures of different deployment methods (local, self-hosted, cloud) may differ; refer to /v1/capabilities and the response structure documentation.
- crawl and map follow robots.txt by default; the official reminder is to respect website policies.
- Self-hosting requires Rust 1.85 or higher.
- The engine and MCP server use the AGPL-3.0 license; the Python and TypeScript SDKs use the MIT license.
- Regarding pricing, specific cloud service billing, platform support details, etc., some of the materials only provide the free quota and installation method; the remaining charging information is not yet provided.
