Tool selection
- For resource URLs such as GitHub repo, issue, PR, comment, release, workflow, etc., prefer using
github-cli。 - If
ghor the GitHub API can obtain the content required for the task, do not simultaneously callfetch-url。 - Only when
github-clicannot obtain the required page content, or the target is not a GitHub resource it supports, should this skill be used as a fallback.
URL only supports http / https。
By default, automatically detects the local Chromium-based browser path; if not detected, you need to install the Playwright browser:
uv run playwright install chromium
Parameters:
--output: Write the output to a file (default stdout).--timeout-ms: Playwright navigation timeout (milliseconds, default 60000).--browser-path: Specify the local Chromium-based browser path (auto-detected by default).--output-format: Output format (defaultmarkdown), supportscsv、html、json、markdown、raw-html、txt、xml、xmltei;raw-htmldirectly outputs the rendered HTML (without going through trafilatura).--fetch-strategy: Onlymarkdownavailable, supportsauto、agent、jina、browser. Defaultauto。
--fetch-strategy Common values:
auto: Default choice.agent: Prefer native-site Markdown negotiation.jina: Prefer Jina Reader.browser: Directly use local Playwright.
Environment variables:
- You can set
JINA_API_KEYto improve Jina Reader rate limiting:JINA_API_KEY=your-token ./scripts/fetch_url.py ...
Examples:
./scripts/fetch_url.py https://example.com --output ./page.md --timeout-ms 60000
./scripts/fetch_url.py https://example.com --fetch-strategy jina
JINA_API_KEY=your-token ./scripts/fetch_url.py https://example.com --fetch-strategy jina
./scripts/fetch_url.py https://example.com --fetch-strategy browser
./scripts/fetch_url.py https://x.com/jack/status/20 --output-format markdown
./scripts/fetch_url.py https://x.com/jack/status/20 --output-format markdown --fetch-strategy browser
Reference:scripts/fetch_url.py