wechat-article-archive

WeChat official account article archive

Default behavior

• Only collect content that the user has the right to access; do not bypass CAPTCHAs or risk controls. Do not use Cookies for body scraping; when precisely obtaining the official account history list, the user may scan a QR code to log in to the WeChat official platform, and the login state is saved only in the user's local cache.
• Users can customize the number of articles to collect; if unspecified, the default target is the most recent 50 articles.
• By default, generate the article archive,<博主>-文章清单.csv and ZIP.
• Only when the user requests analysis of the author's methodology, call author-methodology-analysis。
• After entering the methodology analysis process, by default automatically generate an HTML dashboard and sync to Feishu.
• Only when the user explicitly requests "do not generate HTML" or "do not sync to Feishu" should the corresponding outputs be disabled.
• When public sources are insufficient, deliver the actual quantity; do not fabricate "the most recent N articles" or "complete history".

Sole output contract

First identify the official account name and biz, then determine:

author_root = <workspace>/output/<safe-author-name>/

If a directory with the same name already belongs to another biz, use <safe-author-name>-<biz-tail>. WeChat internal identifiers, time confidence, timeline completeness, collection sources, and internal status are used only for collection and validation, and are not written into the final article list.

Final structure:

output/<safe-author-name>/
  <safe-author-name>-文章清单.csv
  articles/
    01-文章标题/
      <文章标题>.md
      images/
  <safe-author-name>-方法论报告.md          # 仅按需分析
  <safe-author-name>-文案框架.md            # 仅按需分析
  <safe-author-name>-分析数据.json           # 分析时生成
  <safe-author-name>-文章特征.csv            # 分析时生成
  <safe-author-name>-方法论看板.html          # 分析时默认生成
  <safe-author-name>-飞书同步.json            # 成功同步飞书后生成
  <safe-author-name>-竞品标题样本.json        # 提供竞品样本时生成
  <safe-author-name>-文章归档.zip

The ZIP must preserve the <safe-author-name>/ top-level directory, and the internal structure is consistent with the above, but the ZIP itself must not be packed into the ZIP.

Each article directory may contain only one Markdown file with the same name and images/. File and directory names must remove /\\:*?"<>|, and directory names must retain a two-digit numeric sequence number.

Workflow

1. Parse the entry point and lock the identity

Extract from the entry article:

• Title:#activity-name、msg_title or <title>
• Official account name:#js_name or nickname
• __biz/biz、mid/appmsgid、idx、sn
• Publish time, collection information, and explicit article links on the page

Create or verify author_root. An existing directory can be reused incrementally only when biz is the same, or when biz is missing but the official account name and multiple article URLs are all consistent.

2. Lock the candidate list

Candidate fields include at least:

title,url,publish_time,source_type,accessible,biz,mid,idx,sn

Source priority:

1. WeChat official platform article history list after user QR code authorization
2. Public history entry points for official accounts that can be paginated without login
3. Public collections/albums
4. Explicit internal links on the current article page
5. At most two rounds of public search supplementation
6. Existing local archives verified by identity

Candidates are deduplicated by biz + mid + idx + sn and normalized URL. Articles with the same name but different article keys must be retained.

When the user requests "the most recent N articles", "complete collection", or only provides the official account name, prioritize the deterministic history list script.N is the user-specified quantity; if unspecified, take 50：

python3 scripts/discover_account_articles.py \
  --account "<公众号精确名称>" \
  --limit <N> \
  --output "<workspace>/tmp/<safe-author-name>-candidates.csv"

On first run, a QR code will be generated; after the user scans and confirms, the login state is saved by default to ~/.cache/wechat-article-archive/session.json, and the file permissions are set to read/write only by the current user. If search results have duplicate names, the script refuses to guess and lists the candidates; in this case use --fakeid <fakeid> Precise selection. When the session expires, the script deletes the cache; rerun and scan the QR code.

The history list script paginates in the order returned by the backend, extracts the title, link, explicit publish time, and stable account identity, and outputs a candidate CSV. Only when the script returns timeline_complete: true, and body collection has no failures and no unknown publish times, may the body collection command pass --timeline-complete。

When QR code scanning is not needed or not possible, use search, public history entry points, or collections to obtain URLs, and write candidates as a CSV containing at least url, optionally containing source_type、publish_time、time_confidence. Explicit internal links on the current article page can be automatically discovered by the collection script. Public search and history entry points requiring browser observation are still handled by the available search/browser tools, and access restrictions are not bypassed in the script.

Trusted sources for publish time, in priority order, are:

1. An explicit publish time in the page DOM
2. In page scripts, ct、publish_time、create_time and other Unix timestamps
3. A publish time field explicitly returned by a collection or history API
4. A traceable publish time in candidate sources

scene and similar access context parameters must not be used as the publish time. When it cannot be confirmed, write 未知. Only when the sources cover the timeline and the publish times are reliable may it be called "the latest N articles"; otherwise express it as "the N publicly collectible articles".

3. Scrape body text and images

For new archives, preferentially run the deterministic collection script:

python3 -c "import requests, lxml"
python3 scripts/collect_articles.py \
  --author-root "<author_root>" \
  --candidate-csv "<candidate_csv>" \
  --limit <N> \
  --workers 1 \
  --image-workers 2 \
  --article-delay 2 \
  --resume

When a complete candidate CSV already exists, do not use --discover-links, to avoid mixing recommended articles or other official account links into the complete timeline. Enable this parameter only when exploring public internal links from a small number of entry articles.

You may also pass one or more public WeChat article URLs directly at the end of the command. When the target directory already exists:

• Deny the operation by default.
• --resume Reuse existing successful articles and retry only missing or failed articles.
• --replace Do not reuse the existing archive; rebuild it.
• --no-cache Force skipping external body text and image caches, suitable when confirming that the article content has been updated.

All modes use a complete build in a temporary directory, validation, and side backups, and preserve the original archive on failure. Body text and image caches are located by default in ~/.cache/wechat-article-archive/content/, and do not enter the ZIP.

Only pass --timeline-completewhen the candidate sources have been confirmed to cover the complete timeline. If there are unknown publish times or scraping failures, the script automatically degrades to false。

The script performs the following operations:

• Only accept mp.weixin.qq.com public article URLs.
• The body container prefers #js_content, followed by .rich_media_content。
• Preserve titles, subheadings, paragraphs, lists, quotes, and body links.
• Download images in body order to images/image-01.<ext>, allow only known WeChat image domains, and limit a single image to 20 MB.
• Image downloads use the article URL as the Referer; the same URL is downloaded only once, with a default concurrency of 2 images, and on failure the remote reference is deleted and recorded image_failures。
• Articles are collected serially by default, with a 2-second wait between request starts; configurable via --workers、--image-workers and --article-delay . To reduce environment verification risk, it is not recommended to increase article concurrency for batch tasks.
• By default, body text, image, and backend list requests are retried 2 times on failure with exponential backoff, configurable via --retries .
• Markdown conversion preserves titles, paragraphs, lists, quotes, links, code blocks, and tables; videos, audio, and embedded content are preserved as source links or explicit placeholders.
• Automatically generate the article directory, Markdown, and <博主>-文章清单.csv。
• --limit accepts any positive integer; when omitted, defaults to 50。

The top of the Markdown must include:

# 文章标题

- 公众号：<名称>
- 公众号标识：<biz 或未知>
- 原文链接：<url>
- 发布时间：<YYYY-MM-DD HH:mm:ss 或未知>
- 采集来源：<source_type>

---

4. Generate article manifest

<博主>-文章清单.csv must use UTF-8 BOM, and at minimum include:

index,title,url,publish_time,error,author_name,article_dir,markdown_file

• identity_key、biz、mid、idx、sn、time_confidence、timeline_complete、source_type、status is only for candidate discovery and internal validation fields, and must not appear in the final delivery manifest.
• article_dir and markdown_file both having values indicates successful archiving; both being empty and error having a value indicates collection failure.
• markdown_file explicitly records the cleaned and truncated filename, to avoid the validator re-guessing the filename.
• Failure rows must fill in error, and must not reference the article directory.
• The ZIP contains only articles with a valid article directory and Markdown file.

5. Analysis orchestration

When the user requests methodological analysis, call author-methodology-analysis, passing in:

• input_dir = <author_root>/articles
• output_dir = <author_root>
• author_name
• article_list = <author_root>/<safe-author-name>-文章清单.csv
• generate_html = true, unless the user explicitly disables
• sync_lark = true, unless the user explicitly disables

The archive skill does not repeatedly generate analysis reports, nor does it directly depend on a nonexistent third-party copywriting skill.

6. Validation and packaging

First run:

python3 scripts/validate_archive.py "<author_root>"

The validator must pass the following checks:

• The CSV index is continuous, URLs and article keys are not duplicated, and identities are consistent.
• article_dir cannot be an absolute path, contain .. or escape articles/。
• Symbolic links are prohibited in the archive.
• Markdown metadata must be consistent with the CSV.
• Markdown, HTML, and reference-style images must not point to remote or out-of-bounds paths.
• The Markdown body must not be empty or contain markers of abnormal pages such as verification pages, rate-limit pages, or deletion pages; overly short bodies and abnormal conversion retention rates are counted in quality statistics.
• The root directory must not contain undeclared files.

After validation passes, use the companion script to recreate the ZIP. It is forbidden to call system zipthat may corrupt Chinese filenames, and it is also forbidden to incrementally overwrite the old ZIP:

python3 scripts/package_archive.py "<author_root>"

After packaging, run:

python3 scripts/validate_archive.py "<author_root>" --zip "<zip_path>"

The packager only includes archived articles from the CSV and the six declared analysis artifacts, and will not recursively package unknown files. The secondary ZIP validation must check the file whitelist, extra entries, missing entries, and CRC.

If entering the analysis flow, additionally validate that the methodology report, copywriting framework, analysis data, article features, and HTML all exist. Unless the user explicitly disables Feishu, validate that the Feishu document is readable and contains the key sections of the main report; after Feishu succeeds, there should also exist <博主>-飞书同步.json. Feishu failure must not block local archiving and ZIP, but must be explained in the final result.

Failure boundaries

• profile_ext returns no session, login page, or verification page: record once and immediately downgrade.
• WeChat Official Platform history list session expired: delete the local session cache and prompt to scan the code again; do not retry in a loop.
• Sogou or other redirects triggering a CAPTCHA: immediately stop that path.
• The entry article itself is not publicly accessible: explain the restriction and request an accessible link.
• Inaccessible search results must not enter the final package.
• Reposted content must not be marked as the original WeChat Official Account article.

Final reply

Brief report:

• Actual archived count and failure count
• Source scope, and whether it can strictly be called "the most recent N articles"
• Number of unknown publish times, number of failed images, number of too-short body/conversion quality warnings, and validation results
• Whether the methodology, HTML, and Feishu were completed; if disabled by the user or execution failed, explain the reason
• Return the ZIP using a normal clickable local absolute path link, without using local-file://

Resources

• scripts/validate_archive.py: Validate the directory, CSV, Markdown, image references, and ZIP.
• scripts/package_archive.py: Recreate the ZIP with UTF-8 filenames, and exclude the ZIP itself.
• scripts/collect_articles.py: Fetch article bodies from public WeChat URLs/candidate CSV, convert to Markdown, localize images, and generate a manifest.
• scripts/discover_account_articles.py: Log in to the WeChat Official Accounts Platform by scanning a QR code, accurately search for the official account, and generate a paginated list of candidates for the most recent N articles, defaulting to 50.
• scripts/archive_common.py: Shared CSV schema, safe paths, naming, and packaging whitelist rules.