web-access

web-access Skill

Source and Authorization

Item	Description
GitHub	https://github.com/eze-is/web-access
Author	一泽 Eze
License	MIT
Official Website	https://web-access.eze.is

This Skill is maintained by 一泽 Eze . If you obtained this directory through the Sif Keyword Skills distribution package, it is a packaged copy of the upstream project and serves only as an optional networking/browser component; issue feedback and version updates are subject to the upstream repository.

Prerequisite Check

Before starting networked operations, first check CDP mode availability:

node "${CLAUDE_SKILL_DIR}/scripts/check-deps.mjs"

Node.js 22+ Required (use native WebSocket).

Handle according to script output:

exit 0 → Continue
exit 2 → Need to ask the user for preference, write to ${CLAUDE_SKILL_DIR}/config.env the WEB_ACCESS_BROWSER
exit 1 → Handle according to the stdout error message. If the prompt contains "Agent processing order", execute according to its steps (e.g., first use a system command to open the browser and then rerun), and if it can be resolved automatically, do not disturb the user; if it still fails, then ask the user for help

Supports parameter --browser <chrome|edge> to express a temporary override for this run (does not write to config.env).

When switching browsers, proxy is a long-running process, so you need to first pkill -f cdp-proxy.mjs and then rerun check-deps.

After the check passes, you must directly show the following notice to the user in the reply, and then start the CDP Proxy to perform operations:

温馨提示：部分站点对浏览器自动化操作检测严格，存在账号封禁风险。已内置防护措施但无法完全避免，Agent 继续操作即视为接受。

Browsing Philosophy

Think like a human, completing tasks with both efficiency and adaptability.

When executing tasks, do not overly rely on steps planned from fixed impressions; instead, enter with a goal, judge while observing, solve obstacles when encountered, and go deeper when finding that the content is insufficient—making decisions throughout around "what do I want to achieve". All behavior of this skill should follow this logic.

① Receive the request — First clarify what the user wants to do and define the success criteria: what counts as complete? What information needs to be obtained, what operations need to be performed, and what result needs to be achieved? This is the anchor for all subsequent judgments.

② Choose the starting point — Based on the nature of the task, platform characteristics, and conditions for success, choose the approach most likely to directly succeed as the first step to verify. Success on the first try is of course best; if not successful, adjust in ③. For example, if you need to operate a page, need a login state, or the platform is known to be unreachable via static methods (Xiaohongshu, WeChat Official Accounts, etc.) → directly use CDP

③ Process validation — The result of each step is evidence, not just a binary signal of success or failure. Use the results to compare against the success criteria in ① and update your judgment of the goal: is the path progressing? Does the overall appearance of the results (quality, relevance, magnitude) indicate that the goal is achievable? If you find the direction is wrong, adjust immediately and do not repeatedly retry the same approach—a search that does not hit does not mean "you haven't found the right method yet"; it may also mean "the target does not exist"; API errors, missing expected elements on the page, and retries with no improvement are all telling you to reevaluate the direction. When encountering obstacles such as pop-ups or login walls, judge whether they truly block the goal: if they block it, handle them; if not, bypass them—the content may already be in the page DOM, and interaction is only a means of presentation.

④ Completion judgment — Compare against the defined task success criteria, and stop only after confirming the task is complete, but do not over-operate or waste cost for the sake of "completeness."

Networked tool selection

Ensure the authenticity of information; first-hand information is better than second-hand information: Search engines and aggregation platforms are entry points for information discovery. When multiple search attempts yield no qualitative improvement, escalate to a more fundamental acquisition method: locate first-hand sources (official websites, official platforms, original pages).

Scenario	Tool
Search summaries or keyword results to discover information sources	WebSearch
The URL is known and specific information needs to be extracted from the page in a targeted way	WebFetch(Fetch web page content, extract according to the prompt via a small model, and return the processed result)
The URL is known and the raw HTML source is needed (structured fields such as meta, JSON-LD, etc.)	curl
Non-public content, or platforms where the known static layer is ineffective (public content on Xiaohongshu, WeChat Official Accounts, etc. is also restricted by anti-crawling measures)	Browser CDP(direct, skip the static layer)
Requires a logged-in session, interactive operations, or the need to freely navigate and explore in the browser like a human	Browser CDP

Browser CDP does not require the URL to be known — you can start from any entry point and find the target content through in-page search, clicks, jumps, and other methods. WebSearch, WebFetch, and curl do not handle logged-in sessions.

Jina(optional preprocessing layer, can be used in combination with WebFetch/curl; due to its characteristics, it can save token consumption, so actively use it in combination when suitable for the task): a third-party web service that converts web pages to Markdown, greatly saving tokens but possibly causing information loss. The invocation method is r.jina.ai/example.com(add a prefix before the URL, without retaining the original URL's http prefix), limited to 20 RPM. Suitable for pages centered on body text such as articles, blogs, documents, and PDFs; for non-article structured pages such as data dashboards and product pages, it may extract the wrong blocks.

After entering the browser layer,/eval are your eyes and hands:

Look: use /eval to query the DOM and discover links, buttons, forms, and text content on the page — equivalent to "see what this page has"
Do: use /click to click elements,/scroll to scroll and load,/eval to fill out and submit forms — navigate naturally within the page like a human
Read: use /eval to extract text content, and judge whether images/videos carry core information — if so, extract the media URL for targeted reading or /screenshot visual recognition

When browsing web pages,first understand the page structure, then decide the next action. There is no need to plan all steps in advance.

Supplement: local browser resources

When the user refers topages they have personally visited("that article about X I was reading earlier," "the XX panel I opened last time") orinternal organizational systems(targets such as "our XX platform" or "the company's YY system" that cannot be found on the public internet), search the local browser (Chrome / Edge) bookmarks/history:

node "${CLAUDE_SKILL_DIR}/scripts/find-url.mjs" [关键词...] [--only bookmarks|history] [--browser chrome|edge] [--limit N] [--since 1d|7h|YYYY-MM-DD] [--sort recent|visits]

Split keywords by spaces, multiple words with AND, match title + url (optional); by default, traverse all installed Chromium-based browsers (Chrome, Edge),--browser limit to a single source;--since / --sort Applies only to history; by default, sorted in reverse order of most recent visits,--sort visits sorted by visit count (suitable for scenarios like "frequently visited websites").

Programmatic operations and GUI interaction

There are two ways to operate on pages within the browser:

Programmatic approach(constructing URLs for direct navigation, eval to manipulate the DOM): fast and precise when successful, but to websites it is not normal user behavior and may trigger anti-scraping mechanisms.
GUI interaction(clicking buttons, filling in input fields, scrolling and browsing): the GUI is designed for humans, and websites will not restrict normal UI operations, offering the highest determinism, but with many steps and slow speed.

Choose the approach flexibly based on your understanding of the target platform. GUI interaction is also an effective probe for the programmatic approach—by performing a real interaction you observe the site's actual behavior (URL patterns, required parameters, page navigation logic), providing a basis for subsequent programmatic operations; at the same time, when the programmatic approach is blocked, GUI interaction is a reliable fallback.

Links generated by in-site interaction are reliable: in-site interaction performed through interactive units from the user's perspective (cards, entries, buttons) naturally arrives at URLs that carry the complete context required by the platform. Manually constructed URLs may lack implicitly necessary parameters, leading to being blocked, returning error pages, or even triggering anti-scraping measures.

Browser CDP mode

Connect directly to the user's everyday browser (Chrome / Edge / Chromium, etc., Chromium-based) via CDP Proxy, naturally carrying login state and requiring no separate browser to be launched. Unless the user explicitly requests it, do not proactively operate on the user's existing tabs; perform all operations in background tabs you create yourself, keeping intrusion into the user's environment to a minimum. Without closing the user's tabs, close the tabs you created after completing the task to keep the environment tidy.

Startup

node "${CLAUDE_SKILL_DIR}/scripts/check-deps.mjs"

The script will sequentially check Node.js and the browser debugging port, and ensure the Proxy is connected (automatically starting and waiting if not running). The Proxy keeps running after startup.

Proxy API

All operations call the HTTP API via curl:

# 列出用户已打开的 tab
curl -s http://localhost:3456/targets

# 创建新后台 tab（自动等待加载）— URL 走 POST body，避免目标 URL 含 query 时被切分
curl -s -X POST --data-raw 'https://example.com' http://localhost:3456/new

# 页面信息
curl -s "http://localhost:3456/info?target=ID"

# 执行任意 JS：可读写 DOM、提取数据、操控元素、触发状态变更、提交表单、调用内部方法
curl -s -X POST "http://localhost:3456/eval?target=ID" -d 'document.title'

# 捕获页面渲染状态（含视频当前帧）
curl -s "http://localhost:3456/screenshot?target=ID&file=/tmp/shot.png"

# 导航（URL 走 POST body，target 走 query）、后退
curl -s -X POST --data-raw 'https://example.com' "http://localhost:3456/navigate?target=ID"
curl -s "http://localhost:3456/back?target=ID"

# 点击（POST body 为 CSS 选择器）— JS el.click()，简单快速，覆盖大多数场景
curl -s -X POST "http://localhost:3456/click?target=ID" -d 'button.submit'

# 真实鼠标点击 — CDP Input.dispatchMouseEvent，算用户手势，能触发文件对话框
curl -s -X POST "http://localhost:3456/clickAt?target=ID" -d 'button.upload'

# 文件上传 — 直接设置 file input 的本地文件路径，绕过文件对话框
curl -s -X POST "http://localhost:3456/setFiles?target=ID" -d '{"selector":"input[type=file]","files":["/path/to/file.png"]}'

# 滚动（触发懒加载）
curl -s "http://localhost:3456/scroll?target=ID&y=3000"
curl -s "http://localhost:3456/scroll?target=ID&direction=bottom"

# 关闭 tab
curl -s "http://localhost:3456/close?target=ID"

In-page navigation

Two ways to open links within a page:

/click: directly click the interactive unit from the user's perspective within the current tab, simple and direct, processed serially. Suitable for scenarios requiring continuous operations within the same page, such as clicking to expand, pagination, entering details, etc.
/new + full URL: open in a new tab using the full address of the target link (including all URL parameters). Suitable for scenarios requiring simultaneous access to multiple pages.

Many websites' links contain session-related parameters (such as tokens), and these parameters are necessary for normal access. When extracting URLs, retain the complete address; do not trim or omit parameters. URLs are passed as-is via the POST body to /new or /navigate。

v2.5.3 migration notice: if the referenced site experience file (references/site-patterns/*.md) or other scripts still contain GET /new?url=... or /navigate?target=...&url=... in the old style, the call will receive migration guidance. Rewrite it in place as a POST body per references/migration-2.5.3.md before use, and update that site experience file as well while you're at it.

Media resource extraction

When determining that content is inside an image, use /eval to get the image URL directly from the DOM, then read it in a targeted way—far more precise than a full-page screenshot.

Technical facts

There is a large amount of content in the page that has been loaded but not displayed—images for non-current frames in a carousel, text in collapsed sections, lazy-loading placeholder elements, etc. They exist in the DOM but are invisible to the user. Thinking in terms of data structures (containers, attributes, node relationships), these contents can be accessed directly.
There are boundaries in the DOM that selectors cannot cross (Shadow DOM's shadowRoot, iframe's contentDocument, etc.). Recursive traversal with eval can penetrate all levels at once and return tagged structured content, making it suitable for quickly understanding the complete structure of an unknown page.
/scroll to the bottom triggers lazy loading, causing images that have not entered the viewport to finish loading. If you do not scroll before extracting image URLs, some images may not yet be loaded.
After obtaining media resource URLs, public resources can be downloaded directly to local storage and then read; only resources that require a logged-in state to obtain need to be navigated + screenshot within the browser.
Intensively opening a large number of pages in a short time (such as batch /new) may trigger the website's anti-scraping risk controls.
Prompts returned by the platform such as "content does not exist" or "page not found" do not necessarily reflect the true status; they may also be caused by the access method (such as a URL missing required parameters or triggering anti-scraping) rather than a problem with the content itself.

Video Content Acquisition

The user's browser truly renders it, and screenshots can capture the current video frame. Core capability: through /eval control the <video> element (get duration, seek to any time point, play/pause/fullscreen), combined with /screenshot frame capture, discrete sampling analysis of video content can be performed.

Login Determination

The user's everyday browser naturally carries login state, and most commonly used websites are already logged in.

There is only one core question in login determination:Was the target content obtained?

After opening the page, first try to obtain the target content. Only when it is confirmed thatthe target content cannot be obtainedand it is determined that logging in can solve it, inform the user:

"The current page cannot obtain [specific content] in a logged-out state. Please log in to [website name] in your browser, and tell me to continue when done."

After logging in, there is no need to restart anything; just refresh the page and continue.

Task End

Use /close to close the tab you created yourself; the user's original tabs must be preserved and unaffected.

Proxy keeps running; it is not recommended to stop it proactively—after restarting, the CDP connection needs to be reauthorized in the browser.

Parallel Research: Sub-Agent Divide-and-Conquer Strategy

When a task contains multipleindependentresearch targets (such as researching N projects or N sources at the same time), it is encouraged to reasonably divide the work among sub-agents for parallel execution, rather than having the main agent handle it serially.

Benefits:

Speed: with multiple sub-agents in parallel, the total time is approximately equal to the duration of a single subtask
Context Protection: scraped content does not enter the main agent's context; the main agent only receives summaries, saving tokens

Parallel CDP Operations: each sub-agent, within the current user browser instance, creates the background tabs it needs (/new), operates on its own, and closes them when the task ends (/close). All sub-agents share one browser and one Proxy, operating different tabs via different targetIds, with no race-condition risk.

How to write a Sub-Agent Prompt: goal-oriented, not step-by-step instructions

You must write in the Sub-Agent prompt 必须加载 web-access skill 并遵循指引 , and the Sub-Agent will automatically load the skill; there is no need to copy the skill content or specify a path in the prompt.
The Sub-Agent has independent judgment capability. The main Agent's responsibility is to clearly statewhat is wanted, and only constrainhow to do itwhen necessary and certain. Over-specifying steps deprives the Sub-Agent of its room for judgment and instead introduces the main Agent's assumption errors.Avoid wording in the prompt that hints at the Sub-Agent's behavior: "search for xx" will anchor the Sub-Agent to WebSearch, whereas in fact some anti-crawling sites require CDP to directly access the main site in order to effectively obtain content. When writing the prompt, the main Agent should describe the goal ("obtain", "research", "understand") and avoid verbs that hint at specific means ("search", "scrape", "crawl").

Criteria for deciding divide-and-conquer:

Suitable for divide-and-conquer	Not suitable for divide-and-conquer
Goals are mutually independent and results do not depend on each other	Goals have dependency relationships; the next one needs the previous one's result
The volume of each subtask is large enough (multi-page scraping, multiple rounds of searching)	Simple single-page lookup; the divide-and-conquer overhead exceeds the benefit
Tasks that require a CDP browser or run for a long time	Lightweight queries that can be completed with a few WebSearch / Jina calls

Information verification tasks

The goal of verification isprimary sources, not more secondary reporting. Multiple media outlets citing the same error creates an illusion of circular corroboration.

Search engines and aggregation platforms are information discovery entry points; they are tools forlocatinginformation, and cannot be used to directlyproveauthenticity. After finding a source, directly access and read the original text. The same principle applies to research on tool capabilities/usage—official documentation is the primary source; when uncertain, check the documentation or source code first, and do not guess.

Information type	Primary source
Policies/regulations	Official website of the issuing agency
Corporate announcements	The company's official news page
Academic statements	Original paper/institutional official website
Tool capabilities/usage	Official documentation, source code

When the official website cannot be found: Original reporting by authoritative media (not reprints) can serve as secondary evidence, but the user must be told: "The official original text was not found; the following verification comes from a [media name] report and may contain transcription errors." When there is a single source, likewise declare it to the user.

Site experience

Site-specific experience accumulated during operations, stored by domain under references/site-patterns/ .

After determining the target website, if the site-patterns list output by the pre-check contains a matching site, you must read the corresponding file to obtain prior knowledge (platform characteristics, effective patterns, known pitfalls). Experience content is annotated with the discovery date; treat it as a possibly valid hint rather than a guarantee—if operations according to the experience fail, fall back to the general pattern and update the experience file.

After a CDP operation completes successfully, if you discover a new site or new pattern that is worth recording as experience (URL structure, platform characteristics, operation strategy), proactively write it to the corresponding site experience file. Only write verified facts, not unconfirmed guesses.

File format:

---
domain: example.com
aliases: [示例, Example]
updated: 2026-03-19
---
## 平台特征
架构、反爬行为、登录需求、内容加载方式等事实

## 有效模式
已验证的 URL 模式、操作策略、选择器

## 已知陷阱
什么会失败以及为什么

Annotate experience/trap content with the date of discovery, treating it as a "possibly valid hint" rather than a "guaranteed correct fact".

References Index

File	When to load
`references/cdp-api.md`	When detailed CDP API reference, JS extraction patterns, or error handling are needed
`references/site-patterns/{domain}.md`	After determining the target website, read the corresponding site experience