wechat-article-fetcher

WeChat Article Fetcher Skill

WeChat Official Account article scraping Skill, supports any WeChat Official Account article URL, automatically downloads images, exports JSON, and saves full screenshots.

Features

✅ Supports any WeChat Official Account article URL
✅ Automatically extracts title, author, and publish time
✅ Extracts the full article content
✅ Automatically downloads and saves article images
✅ Saves a full-page screenshot
✅ Exports results in JSON format
✅ Supports virtual environments (such as playwright-env)
✅ Headless mode (deployment on servers without a UI)
✅ Command-line tool support

Quick Start

1. Use the command-line tool

# 爬取单篇文章
python3 skills/wechat-article-fetcher/scripts/fetch.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw

# 指定输出目录
python3 skills/wechat-article-fetcher/scripts/fetch.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \
  --output ./my_articles

# 不保存图片
python3 skills/wechat-article-fetcher/scripts/fetch.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \
  --no-images

2. Run using a virtual environment

# 方式 1: 激活虚拟环境后运行
source playwright-env/bin/activate
python3 skills/wechat-article-fetcher/scripts/fetch.py <url>

# 方式 2: 使用提供的包装脚本
python3 skills/wechat-article-fetcher/scripts/run_in_venv.py \
  <url> --venv /path/to/playwright-env

3. Use in code

from wechat_article_fetcher import fetch_article

# 爬取文章
result = fetch_article("https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw")

print(f"标题: {result['title']}")
print(f"作者: {result['author']}")
print(f"内容长度: {result['length']}")
print(f"图片数量: {result['images_count']}")

Command-line Tools

fetch.py - Main scraping tool

python3 skills/wechat-article-fetcher/scripts/fetch.py [OPTIONS] URL

参数:
  URL                   微信公众号文章 URL（必需）

选项:
  -o, --output PATH     输出目录（默认: ./wechat_articles）
  --no-images           不保存图片
  --no-screenshot       不保存截图
  --headless BOOLEAN    无头模式（默认: true）
  --timeout INTEGER     超时时间（毫秒，默认: 60000）
  -h, --help            显示帮助信息

batch_fetch.py - Batch scraping tool

# 从文件中读取 URL 列表
python3 skills/wechat-article-fetcher/scripts/batch_fetch.py \
  --urls-file urls.txt

# 或者直接在命令行指定多个 URL
python3 skills/wechat-article-fetcher/scripts/batch_fetch.py \
  --urls "url1" "url2" "url3"

run_in_venv.py - Virtual environment runner tool

python3 skills/wechat-article-fetcher/scripts/run_in_venv.py \
  <url> --venv /path/to/playwright-env

Output Description

Each scrape creates a directory named with a timestamp:

wechat_articles/
└── 20260302_125500/
    ├── article.json          # JSON 格式结果
    ├── article.png           # 完整页面截图
    └── images/               # 文章图片目录
        ├── 001_image1.jpg
        ├── 002_image2.png
        └── ...

article.json structure

{
  "title": "文章标题",
  "author": "作者名称",
  "publish_date": "2026-03-02",
  "url": "https://mp.weixin.qq.com/s/...",
  "content": "完整文章内容...",
  "images": [
    {
      "index": 1,
      "url": "https://mmbiz.qpic.cn/...",
      "alt": "图片描述",
      "filename": "001_image.jpg",
      "success": true
    }
  ],
  "images_count": 5,
  "images_dir": "wechat_articles/20260302_125500/images",
  "fetch_time": "2026-03-02T12:55:00.000000",
  "length": 15000
}

Configuration File

You can create config.json to customize the default configuration:

{
  "headless": true,
  "timeout": 60000,
  "output_dir": "./wechat_articles",
  "save_images": true,
  "save_screenshot": true,
  "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36..."
}

Usage Examples

Example 1: Scrape a single article

python3 skills/wechat-article-fetcher/scripts/fetch.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw

Example 2: Batch scraping

Create urls.txt：

https://mp.weixin.qq.com/s/xxx
https://mp.weixin.qq.com/s/yyy
https://mp.weixin.qq.com/s/zzz

Then run:

python3 skills/wechat-article-fetcher/scripts/batch_fetch.py \
  --urls-file urls.txt

Example 3: Use in a virtual environment

python3 skills/wechat-article-fetcher/scripts/run_in_venv.py \
  https://mp.weixin.qq.com/s/HTGvy5C6SYyr5XQhTfTfzw \
  --venv /home/user/playwright-env

Best Practices

Use a virtual environment: Isolate dependencies and avoid conflicts
Set timeouts reasonably: Adjust according to network conditions
Add delays during batch scraping: Avoid putting pressure on the server
Check output regularly: Ensure images and content are complete
Comply with robots.txt: Respect the website's scraping rules

Troubleshooting

Issue: Module not found

Solution: Make sure you are running in the correct virtual environment, or install the dependencies:

pip install playwright
playwright install chromium

Issue: Browser fails to start

Solution: Install system dependencies:

sudo apt install -y libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 \
  libcups2 libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 \
  libxfixes3 libxrandr2 libgbm1 libasound2

Issue: Image download fails

Solution: Check the network connection, or use --no-images to skip image downloads.

Issue: Page load timeout

Solution: Increase the timeout:

python3 skills/wechat-article-fetcher/scripts/fetch.py \
  <url> --timeout 90000

Related Resources

Changelog

v1.0.0

Initial version
Supports scraping a single article
Supports image downloads
Supports JSON export
Supports full-page screenshots
Supports virtual environments
Provides a command-line tool
Provides a batch scraping tool