SuperAPI AI Directory

soia-pkm-clip-douyin

General & other

View repository

Updated 10-16 09:48

About this skill

soia-pkm-clip-douyin

Preserve valuable Douyin videos into an Obsidian vault: a lightweight Markdown note records metadata and copy, while the video file is stored separately in the local Downloads directory, so large binaries are not stuffed into the git-tracked knowledge base.

Customer-readable description

What this skill can do

Customer wants Skill will do Customer can see
Archive a Douyin video link Use Playwright to open the video page and intercept the signed API, obtaining the author/copy/duration/interaction counts and a downloadable direct link; download the MP4 locally; write a lightweight Markdown note in the vault (including media_local_path) A new .md file is added to the vault, and the terminal prints the file path, author, publish time, download path, and interaction data
Only want to first see whether this link can be parsed, without downloading the video for now Add --metadata-only: only scrape metadata and copy to write the note, skipping video download In the note, media_local_path is empty,media_fetched: false, and the log indicates the number of candidate direct links
Archiving the same video repeatedly Deduplicate by the frontmatter url: of existing notes in the vault Print ⚠️ Already archived and exit, without re-downloading or re-writing the note
Missing Playwright dependency Stop and provide the installation command, without guessing at a fallback ❌ error message starting with + pip install playwright && python -m playwright install chromium

How customers use it

  1. Provide a Douyin video link (https://www.douyin.com/video/<id>, a link with the modal_id=/resource_id= parameter, or a v.douyin.com short link).
  2. Run python3 scripts/archive_douyin.py <URL>in the vault root directory; when not inside the vault, add --vault <path>。
  3. The script automatically deduplicates, scrapes, downloads, and writes the note; if any part of the process fails, it will use ❌ to report the specific reason, and will not leave a half-finished note.
  4. After archiving is complete, follow the "Next step" prompt in the receipt to supplement ## 摘要、topics、people。

Dependencies and installation

Dependency Type Installation / configuration What to do when missing
playwright(Python package) Hard dependency pip install playwright && python -m playwright install chromium Stop and print the installation command, exit with a non-zero code, and do not perform any speculative fallback
Obsidian vault Hard dependency Any structure, but it must be locatable by one of three methods: --vault / OBSIDIAN_VAULT / cwd auto-discovery Report an error and explain how to specify the path
Python 3 stdlib(urllib) Hard dependency Comes with Python, no installation required —

Installation (recommended: install the entire domain plugin, setting up all skills in this repository at once):

claude plugin marketplace add soia-team/soia-open-skills
claude plugin install soia-pkm-vault@soia

When you only want this one skill, you can use the npx route. Note that the skill will land in the shared source of truth ~/.agents/skills; if the plugin is also installed, the same skill will appear as two indexes and drift separately, so it is recommended to choose one of the two:

npx skills add soia-team/soia-open-pkm-vault-skills -g -a '*' -s soia-pkm-clip-douyin -y

Configuration conventions:

~/.config/soia-skills/soia-pkm-clip-douyin/config.yml
SOIA_PKM_CLIP_DOUYIN_CONFIG_FILE=<custom-config-path>
  • If you only archive occasionally and do not want a fixed vault path, you can skip creating config.yml, and just pass it with --vault each time.
  • Supported variable names:OBSIDIAN_VAULT、OBSIDIAN_ARTICLES. The real path is only written in the private config.yml , refer to config.example.yml。
  • Priority:--vault/--articles-dir CLI argument > OBSIDIAN_VAULT/OBSIDIAN_ARTICLES process environment variable > SOIA_PKM_CLIP_DOUYIN_CONFIG_FILE private directory pointed to by config.yml > automatically discover vault upward from the current working directory (look for AGENTS.md / .obsidian) > general fallback Articles/(and print a warning to stderr).

WorkBuddy loads as role-based experts rather than plugins,npx skills add -a '*' cannot cover it; it needs to be installed separately, see docs/install/workbuddy.md。

Private information and intermediate data

  • The video file itselfdoes not go into the vault(the vault is a git-tracked Markdown knowledge base and is not suitable for storing large binaries); the MP4 lands at ~/Downloads/soia-pkm-clip-douyin/<video_id>/video.mp4, and the vault note only stores an index of this local absolute path.
  • The signed direct link intercepted by Playwright (including msToken / a_bogus and other token parameters)is not written to the vault, not written to logs, and not written to stdout—failure receipts only print the host+path with the query string removed, to avoid leaving authentication tokens in plaintext in logs/notes that may be shared.
  • The signed direct link itself is short-lived: by design, the script will not cache the raw original video direct link to disk or pass it across processes;--metadata-only afterward, if the user wants the video, the script must be rerun (see "Boundaries and Exceptions").
  • Playwright uses a one-off headless browser context (does not persist login state, does not use the user's real Douyin account cookies), and only performs anonymous web browsing; it does not write to a provider login-state directory.
  • Does not cache or persist to disk any privacy information of anyone other than the user; captured author nicknames/UIDs are used only for this note's frontmatter/body.

Logs and completion receipts

The video file being written to disk only means captured; afterward, organized → MOC/导航 → map → Basemust be completed, and transcript/OCR and other body searchability must be reported separately. The contract is detailed in the lifecycle management skill.

完成:<一句话说明本次归档了什么>。

日志摘要:
- started: 已定位 vault / video_id=<id>
- processed: 1 条抖音视频
- created/updated: <vault 内笔记相对路径>
- skipped/failed: <0 或具体原因>

文件变化:
- <vault 笔记绝对路径>
- <本地 MP4 绝对路径,或"未下载(--metadata-only)">

验证:
- 笔记内 `media_local_path` 文件是否存在、MP4 文件头是否为有效 ftyp box

问题与下一步:
- <缺 playwright / 视频不可见 / 需要客户确认 / 建议下一条命令;没有则写"无">

Location

Videos inside the Douyin App are not convenient to accumulate long-term into searchable textual knowledge assets: the copy gets buried in the feed, cannot be found, cannot be searched, and cannot be backlinked with other notes. This skill archives the metadata and copy of a single Douyin video into Obsidian with one click, while the video itself stays on the local disk, keeping the vault lightweight.

Prerequisite dependencies

  • Obsidian vault (any structure)
  • Python 3 + playwright(required for intercepting the signed API, see the installation command above); the download step itself is pure stdlib urllib, no additional dependencies required
  • Vault location priority:
    1. Command line --vault <path>
    2. Environment variable OBSIDIAN_VAULT
    3. Automatically discover vault upward from the current working directory (look for AGENTS.md / .obsidian)
  • Article directory location priority:
    1. Command line --articles-dir <subdir>
    2. Environment variable OBSIDIAN_ARTICLES
    3. Fall back to general default Articles; the script will print a warning to stderr, and the exit code remains 0

Note: tools such as Claude Code / Codex / Gemini CLI / opencode often execute commands in a non-login shell and may not inherit the exportfrom shell startup files. The script will first load the private config.yml, then automatically discover the vault from cwd; if the agent is already in the vault directory, just run it directly.

  • Private config.yml(optional,do not place in the vault or an open-source skill repository):

    Priority:$SOIA_PKM_CLIP_DOUYIN_CONFIG_FILE(or compatible alias $SOIA_PKM_CLIP_DOUYIN_ENV_FILE)> ~/.config/soia-skills/soia-pkm-clip-douyin/config.yml。

    Supported variable names:OBSIDIAN_VAULT、OBSIDIAN_ARTICLES. The real value is only written in the private config.yml , not written into this file.

Core mechanism: Playwright intercepts the signed API

The Douyin web playback address is dynamically generated by the client signature (msToken/a_bogus), and the page SSR JSON (RENDER_DATA) does not contain play_addr. There is no pure HTTP way to fetch the data; a real browser must be used once:

  1. playwright.chromium.launch(headless=True) + new_context(user_agent=<desktop UA>, locale="zh-CN")。
  2. page.on("response", handler) Listen for all JSON responses under the douyin.com domain.
  3. Matching strategyby content rather than endpoint name: recursively search each response body, looking for a dict that has str(node["aweme_id"]) == 目标id and itself carries the "video" key—regardless of whether it appears at the expected aweme/detail/ endpoint, or is mixed into the aweme/mix/ array returned by a collection endpoint such as aweme_list[] .
  4. Poll until a hit or timeout (default 60s,--timeout adjustable).
  5. After a hit, take the direct link by priority:play_addr → play_addr_h264 → download_addr, trying to download one by one until success.
Key field Purpose
aweme["video"]["play_addr"]["url_list"][0] Video direct link (h264/download_addr fallback)
aweme["desc"] Caption/title
aweme["author"]["nickname"] / ["uid"] Author
aweme["create_time"] Publish time (Unix seconds)
aweme["video"]["duration"] Duration (milliseconds)
aweme["statistics"] digg_count/comment_count/collect_count/share_count/play_count/admire_count/recommend_count

The download step uses pure stdlib urllib.request:User-Agent + Referer: https://www.douyin.com/ + Range: bytes=0-, streaming to disk and verifying the byte count and MP4 magic bytes (ftyp box)。

Trigger words

User says Invoke
"Archive this Douyin: URL" / "clip this Douyin video" Run archive_douyin.py <URL>in the vault root; when not inside the vault, add --vault <path>
"First see whether this link can be parsed" / "only the caption, no video" Add --metadata-only, skip the video download
"Re-archive this Douyin" (already exists) Add --force, will keep the existing topics/people/## 摘要

Workflow (default archive)

  1. Parse URL → video_id(short links first do one HTTP redirect resolution before matching)
  2. Call script archive_douyin.py <URL>
  3. The script checks whether the vault has already archived it (by whether the note frontmatter url: line contains that video_id)
    • Already exists → output SKIP, exit 0
    • Otherwise → start Playwright to intercept the signed response and get the aweme object
  4. When not --metadata-only : download the MP4 to ~/Downloads/soia-pkm-clip-douyin/<video_id>/video.mp4; if any step fails, do not write the note
  5. Write the vault note (frontmatter + body)
  6. AI fills in afterward:
    • 1-sentence Chinese ## 摘要
    • 1-3 topics double links + people double links
  7. The "My take" section is always left blank for the user

File naming

<YYYY-MM-DD>-抖音-<作者昵称前 20 字>-<文案前 50 字>.md

Multiple entries by the same author on the same day → append -<video_id 后 6 位>。

Output frontmatter

---
tags: [视频摘抄]
source: 抖音
url: <用户给出的原始链接>
author: <昵称>
published_at: YYYY-MM-DD HH:mm
captured_at: YYYY-MM-DD HH:mm
language: zh
type: video
topics: []   # AI 填
people: []   # AI 填
media_local_path: "<本地 MP4 绝对路径,--metadata-only 时为空>"
content_complete: true
metrics:
  likes: N
  comments: N
  collects: N
  shares: N
  plays: N
  admires: N
  recommends: N
---

Body structure:# <标题> → > [!source]- source collapsible block (source/fetch method/metadata/media status) → ## 摘要(AI fills in) → ## 原文(original Douyin caption, unprocessed) → ## 我的看法(left blank) → ## 关联。

Boundaries and exceptions

Scenario Handling
URL does not contain /video/<id>、modal_id=、resource_id=, nor is it a short link that can be redirected and resolved Reject, and the error message explains the supported formats
Playwright polling timed out without intercepting the target aweme The video has been deleted/private/region-restricted, or Douyin changed the API structure; report an explicit error, do not guess
Intercepted aweme but all candidate direct-link downloads failed Print the failure reason for each one (URL with query string removed), exit non-zero,do not write a note
After archiving with --metadata-only you want the video you must rerun the script completely once more — signed direct links are short-lived, and the script does not do a two-stage "record the direct link first, download later" design
The same video is archived again with a short link, having previously been archived with a full link The short-link text itself does not contain the numeric video_id; deduplication by frontmatter url: line substring matching may fail; it is recommended to prefer archiving with the full douyin.com/video/<id> link
Duplicate filename for the same author on the same day Append at the end -<video_id 后 6 位>
already archived in the vault Default SKIP and exit 0;--force only then overwrite (preserving existing topics/people/## 摘要)

Command-line reference

python3 scripts/archive_douyin.py <URL>
python3 scripts/archive_douyin.py <URL> --force                  # 覆盖已归档
python3 scripts/archive_douyin.py <URL> --vault /path/to/vault   # 覆盖环境变量
python3 scripts/archive_douyin.py <URL> --articles-dir <articles-subdir>
python3 scripts/archive_douyin.py <URL> --metadata-only          # 只抓元数据,跳过视频下载
python3 scripts/archive_douyin.py <URL> --timeout 90             # 拦截等待超时秒数(默认 60)

Verification and testing

Adversarial review (correctness + security dual lens) found 1 should-fix issue:find_existing_archive 's deduplication matching was originally a bare substring test against url: the entire line, without adding boundary anchors like archive_x.py 's /status/{id} —video_id when it happens to be a substring of a longer digit string in another already-archived link (or some query parameter value), it would falsely judge it as "already archived" and silently miss a real new video. Changed to (?<!\d){video_id}(?!\d) numeric boundary matching: it does not assume url: stores which link form (/video/、modal_id=、resource_id= or the form after short-link resolution are both possible), it only requires that neither side is a digit, balancing false-positive prevention with format independence.

Real end-to-end verification (non-mock, 2026-07-21, real public Douyin video):

  • Full archive: Playwright intercepted aweme/detail obtained a 132MB video candidate direct link, download completed,file confirmed a valid MP4, and the frontmatter like/comment/favorite/share counts matched the real-time numbers manually checked (the small difference between two readings of the same video within 40 minutes is real like growth, not a bug)
  • Rerun the same video (without adding --force): correctly SKIP
  • The deduplication boundary fix was verified with real archive results: normal scenarios (real /video/<id> links) were unaffected and already-archived was still correctly recognized

py_compile + audit_skills.py: no findings.


Receipt after completion

Delivery order: first write the file to disk, then output the receipt below, never the reverse; uncertain metadata (such as author or publish time parse failures) must be explicitly marked "unverified" in the receipt, do not fabricate.

The receipt includes:

  1. What was done — a one-sentence summary of the work completed.
  2. File changes — list the newly created vault note path and local MP4 path (full paths);--metadata-only , state that the video was not downloaded.
    • If the Articles/ fallback was used, you must explicitly note that this directory is not this vault's official archive location, and recommend that the user configure --articles-dir、OBSIDIAN_ARTICLES or a private config.yml then return to place.
  3. Next step — optional follow-up suggestions (e.g., add ## 摘要/topics/people, or connect soia-pkm-organize-article-moc)。