SuperAPI AI Directory

soia-pkm-clip-rednote

General & other

View repository

Updated 10-16 08:46

About this skill

soia-pkm-clip-rednote

Preserve valuable single Xiaohongshu (rednote) notes into an Obsidian vault, building a searchable knowledge asset. Video/image binary files do not go into the vault; only a local path is recorded in the note.

Client-readable description

What this skill can do

Archive a single Xiaohongshu note (image-text or video) to an Obsidian vault: scrape the title, body text, author, publish time, engagement data, and topic tags, and download the video/images locally; generate a lightweight Markdown note inside the vault.

What the client wants What the skill does What the client sees
Archive a Xiaohongshu note Parse the share link → scrape the note detail page → extract metadata and body text → write to the vault A Markdown note with frontmatter + a terminal receipt
Keep the video/images in the note Download locally ~/Downloads/soia-pkm-clip-rednote/<note_id>/ Local file path (recorded in the note's media_local_path field; the vault itself does not store binaries)
First check whether the link can be parsed Add --metadata-only, only write metadata + body text, do not download media The note file will still be generated; the candidate direct media links are listed in the file for later manual or re-run downloads
Notes that have already been archived Deduplicate by note_id in url: , skip by default SKIP: <路径> prompt, do not overwrite

How the client uses it

  1. Open the target note from the Xiaohongshu App → "Share" → "Copy link" to get the full share link (must include the xsec_token parameter; hand-typed or truncated links cannot properly render the note content).
  2. Send the link to the Agent and say "Archive this Xiaohongshu post" or directly call python3 scripts/archive_rednote.py <URL>。
  3. The Agent runs the script in the vault root directory (or the specified --vault); the script first checks for duplicates, then scrapes, downloads media, and writes the Markdown.
  4. After execution, the Agent fills in the ## 摘要、topics/people double links;## 我的看法 is always left empty for the user.

Dependencies and installation

Installation (recommended: install the entire domain plugin to install all skills in this repository at once):

claude plugin marketplace add soia-team/soia-open-skills
claude plugin install soia-pkm-vault@soia

When you only want this one skill, you can use the npx route. Note that the skill will land in the shared source of truth ~/.agents/skills; if the plugin is also installed, the same skill will have two indexes and each will drift, so it is recommended to choose one of the two:

npx skills add soia-team/soia-open-pkm-vault-skills -g -a '*' -s soia-pkm-clip-rednote -y
Dependency Type Installation / Configuration What to do if missing
Python 3(stdlib:urllib、re、html、json) Hard dependency Built into the system Cannot run; prompt to install Python 3
Obsidian vault Hard dependency Any existing local vault Prompt --vault <path> or set OBSIDIAN_VAULT
XHS_COOKIE(optional) Optional enhancement Copy cookies from the browser login session and write them to a private config.yml or process environment Most public notes can be scraped without login; only needed when the page prompts that a login session is required

Configuration conventions:

~/.config/soia-skills/soia-pkm-clip-rednote/config.yml
SOIA_PKM_CLIP_REDNOTE_CONFIG_FILE=<custom-config-path>
  • If you don't need a private config (most public notes), you can skip creating config.yml。
  • XHS_COOKIE can only go in the private config.yml, process environment; it cannot be written into the repo, vault body, or logs; the script never prints its value. To prevent a configured XHS_COOKIE from being tricked by a malicious/forged link into sending it to a third-party host, the script only accepts links under the xiaohongshu.com/xhslink.com domain and rejects all others.
  • See config.example.yml。

WorkBuddy 's loading unit is role-based experts rather than plugins,npx skills add -a '*' does not cover it and it needs to be installed separately; see docs/install/workbuddy.md。

Logging and completion receipts

Archiving to disk only means captured; a single object by default continues through the lifecycle-managed organized → MOC/导航 → map → Base five-stage contract, with item-by-item receipts, and stages that don't apply are written as not_applicable。

Every execution must let the client see the process and results. Minimum receipt format:

完成:<一句话说明本次完成了什么>。

日志摘要:
- started: <检查到的 vault / articles-dir / 是否配置 XHS_COOKIE,不打印秘密值>
- processed: <笔记 ID / 类型>
- created/updated: <笔记文件路径>
- skipped/failed: <数量和原因>

文件变化:
- <vault 内 Markdown 笔记的绝对路径>
- <本地媒体目录路径,或"未下载">

验证:
- <脚本 stdout 回执 / 人工核对点>

问题与下一步:
- <缺配置 / 媒体下载失败 / 需要客户确认;没有则写“无”>

Locating

The native Xiaohongshu App is not conducive to long-term retrieval: content only exists in the feed, notes get deleted or set to private, and cross-device searching is difficult. This skill fully captures a note into Obsidian:

  • Title, body (including the original topic tag markers #tag[话题]#, with no secondary processing)
  • Author, publish time, engagement data (likes/saves/comments/shares)
  • Topic tag list
  • Videos/images are downloaded locally, and only the paths are recorded in the note
  • The frontmatter structure is consistent with other clip series in this repo, making unified MOC aggregation easier

Prerequisites

  • Obsidian vault (any structure)
  • Python 3, pure stdlib (urllib.request / re / html / json), no third-party dependencies
  • Vault location priority:
    1. Command line --vault <path>
    2. Environment variable OBSIDIAN_VAULT
    3. Automatically discover the vault upward from the current working directory (look for AGENTS.md / .obsidian)
  • Article directory location priority:
    1. Command line --articles-dir <subdir>
    2. Environment variable OBSIDIAN_ARTICLES
    3. Fall back to the generic default Articles; the script prints a warning to stderr and keeps the exit code at 0:WARN: 未找到 articles 目录配置(--articles-dir / OBSIDIAN_ARTICLES / config.yml),已落默认 Articles/——该目录不是本 vault 的正式归档位,请确认或配置

Note: tools such as Claude Code / Codex / Gemini CLI / opencode often execute commands in a non-login shell and may not inherit the exportfrom shell startup files. The script first loads the private config.yml, then automatically discovers the vault from cwd; if the agent is already in the vault directory, just run it directly.

  • Private config.yml(optional,do not put it in the vault or an open-source skill repo):

    Priority:$SOIA_PKM_CLIP_REDNOTE_CONFIG_FILE(or compatible alias $SOIA_PKM_CLIP_REDNOTE_ENV_FILE)> ~/.config/soia-skills/soia-pkm-clip-rednote/config.yml。

    Supported variable names:OBSIDIAN_VAULT、OBSIDIAN_ARTICLES、XHS_COOKIE(optional, only needed for notes gated behind login state). The real value is written only in the private config.yml and is not written into this file.

Fetch method: parse the note detail page's window.__INITIAL_STATE__

The Xiaohongshu note detail page is server-side rendered, and the body embeds a JS object literal of window.__INITIAL_STATE__ = {...}; . Script flow:

  1. Use urllib.request with a real desktop Chrome UA to directly GET the share link (the full query string must be preserved, including xsec_token; removing the token will most likely return an incomplete/intercepted page).
  2. Regex-locate window.__INITIAL_STATE__\s*=\s* to the text between it and the next </script> , and remove the trailing semicolon.
  3. html.unescape() Process HTML entities, then replace the bare undefined keyword with null(common in JS object literals, but not valid JSON).
  4. json.loads() obtain the note state tree, recursively search for dictionaries containing the noteDetailMap key, and retrieve by note_id extract {"note": {...}} using note_id {"note": {...}}。
  5. From note retrieve fields:title / desc / type(video/normal)/ user.nickname / time(millisecondstimestamp) / tagList / interactInfo(string typenumeric count fields, requiring int() conversion) / imageList / video.media.stream.h264[0].masterUrl(video direct link,signed URL, must not be truncated)。

On failure, do not write a partial file; report an error and exit directly (non-zero exit code): cannot find __INITIAL_STATE__ or cannot find the target note_id data, prompt "the note may require a valid login session or xsec_token has expired", and suggest setting XHS_COOKIE to retry.

Trigger words

User says invoke
"Archive this Xiaohongshu: URL" / "clip this Xiaohongshu note" / "archive this rednote: URL" Run in the vault root directory archive_rednote.py <URL>; when not inside the vault, add --vault <path>
"First see whether this link can be parsed" / "No need to download the video" add --metadata-only: write only metadata + body text, record media candidate direct links in the file, do not download
"Re-archive this one (content has updates)" add --force: overwrite the already archived same note_id, preserve existing topics/people/## 摘要

Workflow (default archive)

  1. Parse the share link → use regex to try in order /explore/([id])、/discovery/item/([id])、[?&]note_id=([id]) extract note_id
  2. call script archive_rednote.py <URL>
  3. The script checks whether the vault has already archived it (by whether the frontmatter url: field contains that note_id)
    • already exists and no --force → output SKIP: <路径>, exit code 0
    • otherwise → fetch the note detail page, download media (unless --metadata-only), write to the vault
  4. AI supplements later:
    • 1 sentence in Chinese ## 摘要
    • 1-3 topics double links + people double links
  5. The "My thoughts" section is always left blank for the user

Media storage (does not enter the vault)

Videos/images are downloaded to ~/Downloads/soia-pkm-clip-rednote/<note_id>/(videos are saved as video.mp4, images are saved to images/<序号>.<扩展名>),not written into the vault(the vault is a git-tracked Markdown knowledge base, not suitable for storing large binary files). The note frontmatter's media_local_path field records the absolute path of this local directory;--metadata-only or when download fails, this field is left blank, and the media's original direct links will be recorded in the body text ## 媒体候选(未下载) section, making it convenient to manually download later or rerun the script.

File naming

<YYYY-MM-DD>-rednote-<作者前 20 字>-<标题前 50 字>.md

Multiple entries with the same date, same author, and same title → append at the end -<note_id 后 6 位>。

Output frontmatter

---
tags: [视频摘抄]
source: 小红书
url: <完整分享链接,原样保留>
author: <昵称>
published_at: YYYY-MM-DD HH:mm
captured_at: YYYY-MM-DD HH:mm
language: zh
type: video / image
topics: []   # AI 填
people: []   # AI 填
media_local_path: "<本地绝对路径,未下载则为空字符串>"
content_complete: true
metrics:
  likes: N
  collects: N
  comments: N
  shares: N
---

Boundaries and exceptions

Scenario Handling
The link is not xiaohongshu.com/xhslink.com domain Refuse to fetch, report an error and exit directly——the script will put XHS_COOKIE(if configured) into the HTML request headers; without host validation it would send the login session to any domain; this validation is executed first before parsing the URL
Share link is missing xsec_token or is a hand-typed short link High probability of getting an incomplete/intercepted page; prompt the user to re-obtain it via the App's "Share" → "Copy Link"
Page not found window.__INITIAL_STATE__ Error exit, do not write the file; prompt that a login state may be required or the token may have expired, suggest setting XHS_COOKIE
Direct video link download 403 / timeout Direct link has a time-limited signature (t= parameter); candidates (masterUrl + backupUrls, h264→h265→av1) are tried one by one; if all fail, skip the download but still write the note body + candidate direct links, not counted as a script failure
interactInfo count fields all empty/0 frontmatter notes "interaction data not public", not presented as a real 0
--metadata-only Only write metadata + body, do not download media; candidate direct links recorded in the body ## 媒体候选 section
Same note_id already archived, without --force SKIP: <路径>, do not overwrite
--force overwrite Keep existing topics/people/## 摘要, only update the remaining fields and the body
The note itself is login-restricted content Set XHS_COOKIE(only added to HTML request headers, not added to media download requests)

Command-line reference

python3 scripts/archive_rednote.py <URL>
python3 scripts/archive_rednote.py <URL> --force                  # 覆盖已归档(保留人工填写字段)
python3 scripts/archive_rednote.py <URL> --vault /path/to/vault  # 覆盖环境变量
python3 scripts/archive_rednote.py <URL> --articles-dir <articles-subdir>
python3 scripts/archive_rednote.py <URL> --metadata-only          # 只写元数据+正文,不下载媒体

Verification and testing

Fixed after three-lens independent adversarial review (2026-07-21): ① (security, blocking level)fetch_note_html previously, for args.url made a request without host validation and attached XHS_COOKIE——forging one that can hit the note_id regex of a non-Xiaohongshu link would send the user's login state to any third-party host; added validate_host(), only allowing xiaohongshu.com/xhslink.com(including subdomains, to prevent bypass concatenation), executed first before parsing the URL. ② (correctness)clip_rednote_env.py when run independently would print XHS_COOKIE in plaintext to stdout, contradicting its own "never prints" claim; desensitization has been applied to COOKIE/TOKEN/SECRET/KEY-type fields. ③ (correctness)--force when rerun, if the video direct link fails to re-download due to an expired signature,media_local_path would be cleared, even if the previous successfully downloaded file is still on disk; changed so that on download failure it first checks whether a local file already exists before deciding whether to clear.

Real end-to-end verification (non-mock/fixture, 2026-07-21, real Xiaohongshu note):

  • Fully archive a real video note: video download 30MB,file confirmed as a legitimate MP4, frontmatter likes/favorites/comments/shares counts consistent with the real-time numbers manually verified in the browser
  • Rerun the same link (without adding --force): correctly SKIP, no duplicate download
  • CLI-level measured security fix: forged https://evil.example/explore/x1y2z3?note_id=x1y2z3 rejected by validate_host before any network request is sent (exit code 1), did not reach fetch_note_html
  • clip_rednote_env.py Run independently: after configuring a real XHS_COOKIE outputs export XHS_COOKIE='<redacted>', no longer leaking plaintext
  • media_local_path fallback logic: construct a scenario where video.mp4 already exists on disk but this download is judged a failure, confirm the path is still correctly retained rather than cleared

Completion receipt

Delivery order: first write the file to disk, then output the receipt below, not the other way around; uncertain metadata (e.g., failure to parse the author nickname) should be explicitly marked "unverified" in the receipt, do not fabricate.

Receipt contains:

  1. What was done — a one-sentence summary of the work completed.
  2. File changes — List the paths of newly created / modified vault notes, as well as the local media directory paths (or "not downloaded").
    • The receipt must include the actual disk directory; if it used the Articles/ fallback, it must explicitly note that this directory is not the official archive location for this vault, and recommend that the user configure --articles-dir、OBSIDIAN_ARTICLES or private config.yml and then relocate it.
  3. Next steps — Optional follow-up suggestions (add summary/topics/people, retry failed media downloads, etc.).