qiaomu-markdown-proxy

Markdown Proxy - URL to Markdown

Convert any URL to clean Markdown. Supports pages requiring login, PDFs, and proprietary platforms.

Trigger Priority (trigger first, then route)

When the user provides a URL and asks to "read, fetch, extract, parse, or convert to Markdown," use this Skill first.
When the user asks to "read this link and then write a draft, summarize, translate, make a podcast, or analyze," first use this Skill to retrieve the original text, then hand the Markdown to the downstream Skill; do not skip fetching just because the final task is writing.
mp.weixin.qq.com and Feishu docs are dedicated routes. Do not first try opening them as ordinary web pages or with a general-purpose content parser.
Do not trigger when the user has already pasted the full text. YouTube goes to qiaomu-youtube-download, and ordinary search questions go to the search tool.

URL Routing (judge first, then execute)

After receiving a URL, first determine its type; different types go through different channels:

URL Pattern	Route To	Reason
`mp.weixin.qq.com`	`scripts/fetch_weixin.sh`	Proxy first, automatically fall back to Playwright for CAPTCHA pages
`feishu.cn/docx/` `feishu.cn/wiki/` `larksuite.com/docx/`	`scripts/fetch_feishu.py`	Requires Feishu API authentication
`youtube.com` `youtu.be`	`qiaomu-youtube-download` skill	YouTube has a dedicated toolchain
`huggingface.co/papers/`	Extract arXiv ID → `scripts/extract_tex.py`	HuggingFace paper pages are actually arXiv mirrors; first find the arXiv link, then go through LaTeX extraction
`arxiv.org/abs/` `arxiv.org/pdf/`	`scripts/extract_tex.py`	Extract structured content from LaTeX source (sections/figures/tables/formulas)
`.pdf` (URL or local path)	`scripts/extract_pdf.sh`	Dedicated PDF extraction
All other URLs	`scripts/fetch.sh`	Proxy cascade with automatic fallback

Workflow

Step 1: Route by URL Type

if URL contains "mp.weixin.qq.com":
    → bash ~/.agents/skills/qiaomu-markdown-proxy/scripts/fetch_weixin.sh "URL"
    → Done

if URL contains "feishu.cn/docx/" or "feishu.cn/wiki/" or "larksuite.com/docx/":
    → python3 ~/.agents/skills/qiaomu-markdown-proxy/scripts/fetch_feishu.py "URL"
    → Done

if URL contains "huggingface.co/papers/":
    → First fetch the page (WebFetch) to find the arXiv URL
    → Then python3 ~/.agents/skills/qiaomu-markdown-proxy/scripts/extract_tex.py "{arxiv_url}"
    → Done

if URL contains "arxiv.org/abs/" or "arxiv.org/pdf/":
    → python3 ~/.agents/skills/qiaomu-markdown-proxy/scripts/extract_tex.py "URL"
    → Done

if URL contains "youtube.com" or "youtu.be":
    → Call qiaomu-youtube-download skill
    → Done

if URL ends with ".pdf" or is local PDF path:
    if remote URL:
        → Try: curl -sL "https://r.jina.ai/{url}"
        → If fails: download + extract_pdf.sh
    if local path:
        → bash ~/.agents/skills/qiaomu-markdown-proxy/scripts/extract_pdf.sh "PATH"
    → Done

else:
    → bash ~/.agents/skills/qiaomu-markdown-proxy/scripts/fetch.sh "URL"
    → Done

Step 2: Display Content

After fetching, show to user:

Title:  {title}
Author: {author} (if available)
Source: {platform} (公众号 / 飞书文档 / 网页 / PDF)
URL:    {original_url}

Summary
{3-5 sentence summary}

Content
{full Markdown, truncated at 200 lines if long}

Step 3: Continue or Save

Composite request("read then write draft/summarize/analyze"): hand the extraction result directly to the downstream task and continue in the same turn; unless the user requests it, there is no need to additionally save the source file.
Extraction-only request("read only/convert to Markdown"): save to ~/Downloads/{title}.md, using YAML frontmatter.
Filename: use article title, remove special characters.
Format: YAML frontmatter (title, author, date, url, source) + Markdown body.
Tell the user the saved path.
Skip saving if the user says “just preview” or “don't save”.

Only stop after extraction when extraction was the complete request. If the user asked for a downstream deliverable, continue to that deliverable.

Examples

General URL

bash ~/.agents/skills/qiaomu-markdown-proxy/scripts/fetch.sh "https://example.com/article"

X/Twitter Post

bash ~/.agents/skills/qiaomu-markdown-proxy/scripts/fetch.sh "https://x.com/username/status/1234567890"

WeChat Article

bash ~/.agents/skills/qiaomu-markdown-proxy/scripts/fetch_weixin.sh "https://mp.weixin.qq.com/s/abc123"

Feishu Document

python3 ~/.agents/skills/qiaomu-markdown-proxy/scripts/fetch_feishu.py "https://xxx.feishu.cn/docx/xxxxxxxx"

arXiv LaTeX Source

python3 ~/.agents/skills/qiaomu-markdown-proxy/scripts/extract_tex.py "https://arxiv.org/abs/1706.03762"

PDF (Remote)

curl -sL "https://r.jina.ai/https://example.com/paper.pdf"

PDF (Local)

bash ~/.agents/skills/qiaomu-markdown-proxy/scripts/extract_pdf.sh "/path/to/paper.pdf"

With Custom Proxy

bash ~/.agents/skills/qiaomu-markdown-proxy/scripts/fetch.sh "https://example.com" "http://127.0.0.1:7890"

Notes

r.jina.ai and defuddle.md require no API key
fetch.sh handles proxy cascade with automatic fallback
Content validation: filters error, login-wall, and WeChat verification pages; requires >5 lines
WeChat wrapper prefers dependency-free proxies, then uses local Playwright; when Python packages are missing and uv is available, it runs them in an isolated environment
Playwright fallback requires a Chromium runtime; install once with python3 -m playwright install chromium if absent
Feishu script requires: FEISHU_APP_ID + FEISHU_APP_SECRET env vars
PDF extraction tries: marker-pdf → pdftotext → pypdf
For detailed method documentation, see references/methods.md