SuperAPI AI Directory

brightdata-mcp

Data & analytics

View repository

Updated 11-10 17:22

About this skill

Bright Data MCP (Anti-Bot Web Scraping & Data Extraction)

Based on brightdata/brightdata-mcp: an all-in-one Model Context Protocol (MCP) server providing AI agents with resilient, real-time public web access powered by Bright Data's Web Unlocker and proxy infrastructure.


🎯 When to Use

  • Anti-Bot & CAPTCHA Bypassing: When target sites block curl, standard fetch, or headless browsers with Cloudflare, DataDome, or Akamai anti-bot challenges.
  • SERP & Search Engine Scraping: Extracting Google, Bing, or DuckDuckGo search engine result pages (SERPs) with structured JSON output.
  • Large-Scale Public Web Extraction: Scraping product catalogs, reviews, public directory profiles, or news articles without rate-limiting IP bans.

🚀 Setup & Registration

MCP Configuration

Register Bright Data MCP in your agent's configuration (.mcp.json or IDE settings):

{
  "mcpServers": {
    "brightdata": {
      "command": "npx",
      "args": ["-y", "@brightdata/mcp"],
      "env": {
        "BRIGHTDATA_API_KEY": "YOUR_BRIGHTDATA_API_KEY"
      }
    }
  }
}

Note: Bright Data offers a free tier of 5,000 requests/month without credit card.


🛠️ Key Capabilities & Tools

  • scrape_url: Fetches clean HTML or Markdown content from any URL, automatically executing JavaScript and solving CAPTCHAs.
  • search_engine: Runs structured queries across Google and returns ranked organic, sponsored, and snippet items.
  • extract_structured_data: Converts target pages directly into validated JSON schemas.

⚡ Integration with REA & Security Audits

  • Auditing Blocked Endpoints: When an external target API or website endpoint returns 403 Forbidden due to IP reputation, route through Bright Data MCP to analyze real response headers and payloads.
  • Competitor & Pre-Release Research: Gather public security advisories and competitor feature implementations without triggering rate limits.