SuperAPI AI Directory

mcp-server-rag-web-browser

This project is an MCP server implementation of Apify's RAG Web Browser Actor, used to let large language models and AI agents access web pages and extract content. It supports scraping result pages via Google Search or directly scraping a specified URL, and returns cleaned content in formats such as Markdown. The repository is currently maintained for archival purposes only, and the official recommendation is to use mcp.apify.com or directly call the RAG Web Browser Actor's HTTP/SSE interface as alternatives.

Open website

About this tool

Purpose

This repository provides an MCP server implementation of the RAG Web Browser Actor. The RAG Web Browser Actor is equivalent to a tool that provides web browsing capabilities for large language models (LLMs) and RAG pipelines, similar to the web search feature in ChatGPT. This MCP server is specifically designed to provide fast responses for AI agents and LLMs, enabling them to interact with the web and extract information from web pages.

Main Capabilities

The server runs locally, communicates with the RAG Web Browser Actor in Standby mode, sends search queries, and receives extracted web page content. Specific capabilities include:

  • Web search: queries Google Search, scrapes the top N URLs, and returns the content cleaned as Markdown.
  • Single URL scraping: scrapes a specified URL and returns its content as Markdown.
  • Local MCP integration: communicates with AI clients via standard input/output (stdio).

Usage

The tool interface name is search, described as: query Google Search or scrape a direct URL and return cleaned page content. Its parameters include:

  • query(string, required): search keywords or a full URL, supporting advanced Google operators.
  • maxResults(number, optional, default 1): the maximum number of organic results to scrape, ignored when query is a URL.
  • scrapingTool(string, optional, default raw-http): possible values are browser-playwright or raw-http. raw-http is fast (does not execute JS) and suitable for static pages; browser-playwright can handle JS-heavy websites, is slower but more robust.
  • outputFormats(string array, optional, default [markdown]): can be one or more of text, markdown, html.
  • requestTimeoutSecs(number, optional, default 40, minimum 1 maximum 300): total server-side and client-side waiting budget, enforcing a local abort.

Environment requirements include: Node.js (v18 or higher), Apify API Token (APIFY_TOKEN). You can first clone and install dependencies. Because the MCP server runs via standard input/output (stdio), debugging can be challenging. For the best debugging experience, you can use MCP Inspector, launched via an npm command. After startup, it displays a URL; visit it in a browser to start debugging.

Applicable Scenarios

Suitable for scenarios that require AI agents or LLMs to access web pages in real time, perform searches, and extract page content for use in RAG pipelines.

Limitations and Migration

This MCP server has been deprecated, and the official recommendation is to use mcp.apify.com. This alternative requires no local setup, is always up to date, can access 6000+ Apify Actors (including RAG Web Browser), supports OAuth, and supports dynamic tool discovery. The usage steps are: visit https://mcp.apify.com,授权客户端(Claude、VS Code, etc.), copy the generated MCP server configuration (or use the OAuth flow), and you can immediately use browsing and other tools. Another alternative is to call directly through the RAG Web Browser Actor's HTTP/SSE interface, without mcp.apify.com, supporting real-time streaming via Server-Sent Events, with full control over the integration and no additional dependencies. You can also use the hosted endpoint https://mcp.apify.com(当客户端支持 For HTTP transport/remote MCP).

This repository is maintained for archival purposes only; please use the recommended alternatives above for active development.