Always use the firecrawl CLI first. Fall back to the firecrawl-mcp MCP server only when the CLI genuinely cannot do the job. The CLI runs token-free via Bash and is faster; the MCP costs context on every call.
Step 0 — Verify prerequisites
Before running anything, confirm the CLI is installed and the API key is set:
firecrawl --version # should print a version number
echo $FIRECRAWL_API_KEY # should print fc-...
If the CLI is missing:
npm install -g firecrawl-cli
If the API key is empty, add it to .env and export it:
export FIRECRAWL_API_KEY=fc-your-key-here
# Get your key at: https://www.firecrawl.dev/app/api-keys
Step 1 — Choose the right command (CLI first)
Work through this decision in order. Stop at the first match.
Is the task a single URL scrape? → firecrawl <url>
firecrawl https://example.com # Markdown (default)
firecrawl https://example.com --format html,links # specific formats
firecrawl https://example.com -o .tmp/page.md # save to file
firecrawl https://example.com --json --pretty > .tmp/page.json
Is the task a web search? → firecrawl search
firecrawl search "your query"
firecrawl search "AI pricing" --limit 10 # cap results
firecrawl search "topic" --scrape --scrape-formats markdown # fetch full pages
Is the task a multi-page site crawl? → firecrawl crawl
firecrawl crawl https://example.com --wait --progress
firecrawl crawl https://example.com --limit 50 --max-depth 3 --wait > .tmp/crawl.json
Is the task URL discovery across a domain? → firecrawl map
firecrawl map https://example.com
firecrawl map https://example.com --search "blog" --json > .tmp/urls.json
Is the task AI-driven open-ended extraction? → firecrawl agent
firecrawl agent "Extract all pricing plans from this site" --wait
firecrawl agent "Extract contact info" \
--schema '{"type":"object","properties":{"email":{"type":"string"}}}' \
--wait
Save all output to .tmp/ — never to tracked paths.
Step 2 — Escalate to MCP only when CLI cannot do the job
The CLI covers ~90% of use cases. The MCP is the right choice only for:
| Scenario | MCP tool to use |
|---|---|
| Scraping a pre-built list of URLs in one call | firecrawl_batch_scrape |
| Schema-driven LLM extraction where structured output feeds directly into a downstream tool call | firecrawl_extract |
| Polling an async crawl job without keeping a terminal open | firecrawl_check_crawl_status, firecrawl_check_batch_status |
If none of those apply, use the CLI. The MCP has the same underlying API — it just adds protocol overhead.
Available MCP tools (firecrawl-mcp): firecrawl_scrape, firecrawl_batch_scrape, firecrawl_search, firecrawl_crawl, firecrawl_map, firecrawl_agent, firecrawl_extract, firecrawl_check_crawl_status, firecrawl_check_batch_status, firecrawl_agent_status
Step 3 — Handle errors and escalate when stuck
| Error | What to do |
|---|---|
firecrawl: command not found |
npm install -g firecrawl-cli |
FIRECRAWL_API_KEY not set / 401 Unauthorized |
Add key to .env + export FIRECRAWL_API_KEY=fc-...; regenerate at firecrawl.dev if expired |
429 Rate Limit |
Reduce --limit; wait and retry |
| Crawl timeout | Add --max-depth 2, reduce --limit 20 |
| Empty / blocked content | Try --format html; if still blocked, escalate to Apify MCP (handles bot detection) |
| Need interactive browser (login flows, JS forms) | Use node tools/playwright.js or playwright-mcp instead |
| 1 000+ pages | Use Apify MCP — it handles massive scale Firecrawl cannot |
Output handling
- Always write to
.tmp/(scratch space, not committed) - Read
.tmp/page.mdinline to inspect content in-session - Pass
.tmp/page.jsonto a Python script or the next pipeline step - Final deliverables go to cloud services: Google Drive, Supabase, Sheets, email, etc.