amenbo

Purpose

amenbo (アメンボ / water strider) is an MCP server for programming agents such as Claude Code and Codex, used to collect information from the Web. Its positioning is "gliding over the water without making ripples": low load on the collection target, while obtaining web page content with fewer Tokens, and it is particularly optimized for Japanese websites.

Main Capabilities

Progressive Disclosure and Differential Responses

• With mode: outline, first return the heading tree and the Token count of each section, then use section to fetch only the needed sections, avoiding pouring the entire page into the context.
• When fetching a URL that has already been fetched again, return unchanged if there is no change, and return only the changed sections if there is a change.

Japanese Site Adaptation

• For pages with loose structure (nested divs, table layouts), infer the main content area based on the geometric layout of the rendered result.
• Automatically determine Shift_JIS / EUC-JP / ISO-2022-JP encoding.
• Remove <ruby> furigana to prevent duplication of the main text.
• For image-based information (such as price tables or pages dominated by banners), automatically switch to screenshots when text extraction is weak.
• Detect and restore high-link-density data tables (such as comparison tables) that tend to be lost entirely during main content extraction, restoring them to their original positions; normalize the colspan/rowspan of remaining tables and multi-row headers to prevent column misalignment.
• For pages where headings are wrapped in wrapper layers with edit links (such as Wiki-style sites), detect and restore the headings of the remaining sections in the main content, ensuring stable progressive disclosure of outline / section.
• Provide dedicated adapters for Qiita / Zenn / note / Hatena Blog / Yahoo! News / PR TIMES.

Main Content Processing and Budget

• CJK-compatible main content pruning: remove navigation, ads, and footers based on punctuation density, character type ratio, and link density.
• Token estimation by character category: Japanese, Korean, Cyrillic, emoji, etc. have a heavier Token unit price than English; use coefficients by character category (calibrated through actual measurement) to calculate the pagination budget.
• Automatic Markdown / image switching: use screenshots only for pages with low quality scores, avoiding repeated refetching after reading corrupted Markdown.

Fetching and Etiquette

• Two-stage fetching: first a normal HTTP GET, and only upgrade to [the next stage] for pages that require JS rendering.Headless Chromium; most fetches do not launch a browser.
• Respect robots.txt and Crawl-Delay; serial within the same domain, default 1 req/sec (fetching robots.txt also counts toward this 1 request). Prefer sitemap / RSS for link enumeration; do not crawl page by page.
• User-Agent that explicitly identifies the bot; do not implement anti-bot bypass.
• Cache: revalidate with ETag / If-Modified-Since; default TTL is 15 minutes (AMENBO_CACHE_TTL_MS); Cache-Control is adopted only in the extension direction (if max-age is longer than 15 minutes, it is used, with an upper limit of 24 hours; no-store does not store); do not shorten the TTL because of max-age=0 or no-cache.

Security

• SSRF protection: reject protocols other than http/https (file:, ftp:, etc.); if the connection target resolved by DNS is a private / loopback / link-local / reserved address, reject it; for DNS rebinding (TOCTOU), pin the actual connection to the validated IP.
• Upper limit on response body size to prevent OOM caused by huge responses.

Usage

MCP Server

Markdown fetching (ordinary fetch / links) only requires the following installation; only fetching that uses a browser (Chromium) (SPA upgrade, screenshots, etc.) requires running it once the first time (about 170MB download). The command forms given in the repository documentation include: Claude Code (--scope user means common to all projects; remove it for per-project), and global installation via npx ("command": "npx", "args": ["-y", "amenbo"]) or use of a local build ("command": "node", "args": ["/path/to/amenbo/dist/server.js"]). Other clients (Cursor / Cline, etc.) add the corresponding entry in their respective MCP configurations (Cursor: ~/.cursor/mcp.json; Cline: settings JSON on the MCP Servers screen).Under stdio, it responds to both MCP 2026-07-28 (stateless core) and the 2025-series versions, so clients can connect with the above configuration regardless of which version they use.

Prompts

Tool definitions alone cannot convey usage conventions such as "fetch using progressive disclosure." You can copy the official usage instructions into CLAUDE.md or AGENTS.md so the agent can use amenbo more efficiently; you can also load them only for the current session instead of writing them into CLAUDE.md. In clients that support MCP prompts, the server distributes the same conventions as the usage prompt (in Claude Code, /mcp__amenbo__usage).

CLI

amenbo can also run as a CLI that shares the same core (fetching, caching, politeness, extraction logic) with the MCP server. With no arguments, or with amenbo serve, it still starts as an MCP server ("command": "amenbo" in .mcp.json works as usual), without affecting existing MCP registrations. For details of each subcommand, see amenbo <fetch|links|screenshot> --help.

Division of labor between MCP and CLI: MCP is the agent's main path; the browser (Chromium) stays warm in-process, and images such as screenshots can be returned directly in the conversation, and it can also reach hosts like claude.ai that have no shell; the CLI is suitable for shell scripts, CI, debugging, saving output to disk and then grepping / reading part of it, or agents/toolchains that do not support MCP, because 1 command = 1 process, so the browser starts each time.

The state of caching, differential responses (unchanged / diff), and rate control (robots.txt / serial access per domain) is shared between the MCP server and the CLI (using the same ~/.cache/amenbo), but inter-process sharing of rate control is best-effort: serial access within the same domain is strictly guaranteed only within each process, and when the MCP server and multiple CLIs access the same domain at the same time, the minimum interval may be slightly violated.

Link enumeration order is sitemap -> RSS/Atom -> in-page links; screenshot is used for screenshots.

Development-related

After cloning, run once to enable the secret information check (gitleaks) pre-commit hook (the repository documentation provides the corresponding command).

Citation

When citing in an article or research, please use the Zenodo DOI. The 10.5281/zenodo.21553636 in the badge is the Concept DOI common to all versions, and always resolves to the latest version; to point to a specific version, obtain that version's DOI from the Zenodo record. Machine-readable citation information is in CITATION.cff (GitHub's "Cite this repository" can generate BibTeX / APA).

Limitations

• The body-text extraction heuristic is tuned for Japanese pages: progressive disclosure, caching, low load, and language-independent, so it also works on non-Japanese sites; however, if the body text is missing on a non-Japanese page, specify it with a selector or re-fetch with mode: "screenshot".
• HTTP proxies are not supported: environment variables such as HTTP_PROXY / HTTPS_PROXY are not respected. The reason is that SSRF protection pins the connection target to a verified IP (a countermeasure against DNS rebinding), which cannot coexist with delegating domain name resolution to a proxy. Networks that must use an upstream proxy are currently unusable.
• No anti-bot bypass is implemented: retrieval failures caused by robots.txt denial or bot protection are expected behavior, and failures are reported truthfully.
• For actual comparisons with similar tools (official fetch MCP / Jina Reader / Playwright MCP / PixelRAG pixelshot), see the repository article and the bench/ directory (which mentions a claim of a 5000-fold Token difference); this page does not elaborate.

Not Yet Provided

Apart from the above, materials such as pricing, charging model, list of supported platforms, and license agreement are not provided, and are not yet available.