SuperAPI AI Directory
Partner中文AI newsFavorites
Skillssoia-pkm-clip-douyin

soia-pkm-clip-douyin

Office & productivity

View repository

Updated 10-16 09:28

About this skill

soia-pkm-clip-douyin

Preserve valuable Douyin videos into an Obsidian vault: a lightweight Markdown note records metadata and the caption, while the video file is stored separately in the local Downloads directory, so large binaries are not stuffed into a git-tracked knowledge base.

Customer-readable description

What this skill can do

Customer wants Skill will do Customer can see
Archive a Douyin video link Use Playwright to open the video page and intercept the signed API to get the author/caption/duration/interaction counts and a downloadable direct link; download the MP4 locally; write a lightweight Markdown note in the vault (including media_local_path） A new .md file is added to the vault, and the terminal prints the file path, author, publish time, download path, and interaction data
Only want to see whether this link can be parsed first, without downloading the video yet Add --metadata-only: only scrape metadata and caption to write the note, skipping video download In the note, media_local_path is empty,media_fetched: false, and the log indicates the number of candidate direct links
Archiving the same video repeatedly Deduplicate by the frontmatter url: of existing notes in the vault Print ⚠️ Already archived and exit, without re-downloading or re-writing the note
Missing Playwright dependency Stop and provide the install command, without guessing at a fallback ❌ error message starting with + pip install playwright && python -m playwright install chromium

How customers use it

  1. Provide a Douyin video link (https://www.douyin.com/video/<id>, a link with the modal_id=/resource_id= parameter, or a v.douyin.com short link).
  2. Run python3 scripts/archive_douyin.py <URL>in the vault root directory; if not inside the vault, add --vault <path>。
  3. The script automatically deduplicates, scrapes, downloads, and writes the note; if anything fails throughout the process, it will use ❌ to report the specific reason, and will not leave a half-finished note.
  4. After archiving is complete, follow the "Next step" prompt in the receipt to fill in ## 摘要、topics、people。

Dependencies and installation

Dependency Type Install / configure What to do if missing
playwright(Python package) Hard dependency pip install playwright && python -m playwright install chromium Stop and print the install command, exit with a non-zero code, and do not make any speculative fallback
Obsidian vault Hard dependency Any structure, as long as it can be located by one of the three: --vault / OBSIDIAN_VAULT / cwd auto-discovery Report an error and explain how to specify the path
Python 3 stdlib（urllib） Hard dependency Comes with Python, no installation required —

Install this skill (single skill):

npx skills add soia-team/soia-open-skills -g -a '*' -s soia-pkm-clip-douyin -y

Configuration conventions:

~/.config/soia-skills/soia-open-skills/soia-pkm/soia-pkm-clip-douyin/config.yml
SOIA_PKM_CLIP_DOUYIN_CONFIG_FILE=<custom-config-path>
  • If you only archive occasionally and do not want a fixed vault path, you can skip creating config.yml, and just pass --vault as an argument each time.
  • Supported variable names:OBSIDIAN_VAULT、OBSIDIAN_ARTICLES. The real path is only written in the private config.yml , see config.example.yml。
  • Priority:--vault/--articles-dir CLI argument > OBSIDIAN_VAULT/OBSIDIAN_ARTICLES process environment variable > SOIA_PKM_CLIP_DOUYIN_CONFIG_FILE points to the private config.yml > Auto-discover the vault upward from the current working directory (look for AGENTS.md / .obsidian) > Generic fallback Articles/(and print a warning to stderr).

Private information and intermediate data

  • The video file itselfdoes not go into the vault(the vault is a git-tracked Markdown knowledge base, not suitable for storing large binary files); MP4 lands in ~/Downloads/soia-pkm-clip-douyin/<video_id>/video.mp4, and the vault note only stores an index of this local absolute path.
  • Signed direct links intercepted by Playwright (including msToken / a_bogus and other token parameters)are not written to the vault, not written to logs, not written to stdout—failure receipts only print the host+path with the query string removed, to avoid leaving authentication tokens in plaintext in logs/notes that may be shared.
  • The signed direct link itself is short-lived: by design, the script will not cache the bare original video direct link to disk or pass it across processes;--metadata-only afterward, if the user wants the video, the script must be rerun (see "Boundaries and Exceptions").
  • Playwright uses a one-off headless browser context (does not persist login state, does not use the user's real Douyin account cookies), and only performs anonymous web browsing; it does not write a provider login-state directory.
  • It does not cache or persist to disk any private information of anyone other than the user; captured author nicknames/UIDs are only used for the frontmatter/body of this note.

Logs and completion receipts

完成：<一句话说明本次归档了什么>。

日志摘要：
- started: 已定位 vault / video_id=<id>
- processed: 1 条抖音视频
- created/updated: <vault 内笔记相对路径>
- skipped/failed: <0 或具体原因>

文件变化：
- <vault 笔记绝对路径>
- <本地 MP4 绝对路径，或"未下载（--metadata-only）">

验证：
- 笔记内 `media_local_path` 文件是否存在、MP4 文件头是否为有效 ftyp box

问题与下一步：
- <缺 playwright / 视频不可见 / 需要客户确认 / 建议下一条命令；没有则写"无">

Location

Videos within the Douyin App are inconvenient to accumulate over time into a searchable textual knowledge asset: the copy gets buried in the feed, cannot be found, cannot be searched, and cannot be bidirectionally linked with other notes. This skill archives the metadata and copy of a single Douyin video into Obsidian with one click, while the video itself stays on the local disk, keeping the vault lightweight.

Prerequisites

  • Obsidian vault (any structure)
  • Python 3 + playwright(required for intercepting the signed API, see the installation command above); the download step itself is pure stdlib urllib, no additional dependencies are needed
  • Vault location priority:
    1. Command line --vault <path>
    2. Environment variable OBSIDIAN_VAULT
    3. Auto-discover the vault upward from the current working directory (look for AGENTS.md / .obsidian）
  • Article directory location priority:
    1. Command line --articles-dir <subdir>
    2. Environment variable OBSIDIAN_ARTICLES
    3. Fall back to the generic default Articles; the script will print a warning to stderr, and the exit code remains 0

Note: tools such as Claude Code / Codex / Gemini CLI / opencode often execute commands in a non-login shell and may not inherit the exportfrom the shell startup file. The script will first load the private config.yml, then auto-discover the vault from cwd; if the agent is already in the vault directory, just run it directly.

  • Private config.yml(optional,do not place in the vault or an open-source skill repository）：

    Priority:$SOIA_PKM_CLIP_DOUYIN_CONFIG_FILE(or compatible alias $SOIA_PKM_CLIP_DOUYIN_ENV_FILE）> ~/.config/soia-skills/soia-open-skills/soia-pkm/soia-pkm-clip-douyin/config.yml。

    Supported variable names:OBSIDIAN_VAULT、OBSIDIAN_ARTICLES. The real values are only written in the private config.yml and are not written into this file.

Core mechanism: Playwright intercepts the signed API

The Douyin web playback address is dynamically generated by client-side signing (msToken/a_bogus); the page SSR JSON (RENDER_DATA) does not contain play_addr, there is no pure HTTP way to fetch the data, and a real browser must be used once:

  1. playwright.chromium.launch(headless=True) + new_context(user_agent=<desktop UA>, locale="zh-CN")。
  2. page.on("response", handler) listens for all douyin.com JSON responses under the domain.
  3. Matching strategyBy content, not endpoint name: recursively search each response body for a str(node["aweme_id"]) == 目标id dictionary that itself has a "video" key—regardless of whether it appears in the expected aweme/detail/ endpoint, or is mixed into the aweme/mix/ array returned by collection endpoints like aweme_list[] .
  4. Poll until a hit or timeout (default 60s,--timeout adjustable).
  5. After a hit, take direct links by priority:play_addr → play_addr_h264 → download_addr, trying downloads one by one until success.
Key field Purpose
aweme["video"]["play_addr"]["url_list"][0] Video direct link (h264/download_addr fallback)
aweme["desc"] Caption/title
aweme["author"]["nickname"] / ["uid"] Author
aweme["create_time"] Publish time (Unix seconds)
aweme["video"]["duration"] Duration (milliseconds)
aweme["statistics"] digg_count/comment_count/collect_count/share_count/play_count/admire_count/recommend_count

The download step uses pure stdlib urllib.request：User-Agent + Referer: https://www.douyin.com/ + Range: bytes=0-, streams to disk and verifies the byte count and MP4 magic bytes (ftyp box）。

Trigger words

User says Invoke
"Archive this Douyin: URL" / "clip this Douyin video" Run in the vault root archive_douyin.py <URL>; when not inside the vault, add --vault <path>
"First see if this link can be parsed" / "only the caption, no video" Add --metadata-only, skip video download
"Re-archive this Douyin" (already exists) Add --force, will keep the existing topics/people/## 摘要

Workflow (default archive)

  1. Parse URL → video_id(short links first undergo one HTTP redirect resolution before matching)
  2. Call script archive_douyin.py <URL>
  3. The script checks whether the vault has already archived it (by whether the note frontmatter url: line contains that video_id）
    • Already exists → output SKIP, exit 0
    • Otherwise → start Playwright to intercept the signed response and get the aweme object
  4. When not --metadata-only : download the MP4 to ~/Downloads/soia-pkm-clip-douyin/<video_id>/video.mp4; if any step fails, do not write the note
  5. Write the vault note (frontmatter + body)
  6. AI fills in afterward:
    • 1-sentence Chinese ## 摘要
    • 1-3 topics double links + people double links
  7. The "My take" section is always left blank for the user

File naming

<YYYY-MM-DD>-抖音-<作者昵称前 20 字>-<文案前 50 字>.md

Multiple entries by the same author on the same day → append -<video_id 后 6 位>。

Output frontmatter

---
tags: [视频摘抄]
source: 抖音
url: <用户给出的原始链接>
author: <昵称>
published_at: YYYY-MM-DD HH:mm
captured_at: YYYY-MM-DD HH:mm
language: zh
type: video
topics: []   # AI 填
people: []   # AI 填
media_local_path: "<本地 MP4 绝对路径，--metadata-only 时为空>"
content_complete: true
metrics:
  likes: N
  comments: N
  collects: N
  shares: N
  plays: N
  admires: N
  recommends: N
---

Body structure:# <标题> → > [!source]- source collapsible block (source/capture method/metadata/media status) → ## 摘要(AI fills in) → ## 原文(original Douyin caption, unprocessed) → ## 我的看法(left blank) → ## 关联。

Boundaries and exceptions

Scenario Handling
URL does not contain /video/<id>、modal_id=、resource_id=, and is not a short link that can be resolved by redirect Reject, with an error explaining the supported formats
Playwright polling times out without intercepting the target aweme Video deleted/private/region-blocked, or Douyin changed the API structure; report the error clearly, do not guess
Intercepted aweme but all candidate direct-link downloads fail Print the failure reason for each one (URL with query string removed), exit non-zero,do not write the note
With --metadata-only after archiving, if you want the video you must rerun the script completely once more — signed direct links are short-lived, and the script does not use a two-stage design of "record the direct link first, download later"
the same video is archived again using a short link, after it was previously archived using the full link the short-link text itself does not contain the numeric video_id, so deduplication by frontmatter url: line substring matching may fail; it is recommended to prioritize archiving with the full douyin.com/video/<id> link
same day, same author, duplicate filename append at the end -<video_id 后 6 位>
already archived in the vault default SKIP and exit 0;--force only then overwrite (preserving existing topics/people/## 摘要）

command-line reference

python3 scripts/archive_douyin.py <URL>
python3 scripts/archive_douyin.py <URL> --force                  # 覆盖已归档
python3 scripts/archive_douyin.py <URL> --vault /path/to/vault   # 覆盖环境变量
python3 scripts/archive_douyin.py <URL> --articles-dir <articles-subdir>
python3 scripts/archive_douyin.py <URL> --metadata-only          # 只抓元数据，跳过视频下载
python3 scripts/archive_douyin.py <URL> --timeout 90             # 拦截等待超时秒数（默认 60）

verification and testing

adversarial review (correctness + security dual lens) found 1 should-fix:find_existing_archive deduplication matching was originally a bare substring test against the entire url: line, without adding boundary anchors like archive_x.py 's /status/{id} —video_id when it happens to be a substring of a longer numeric string (or some query parameter value) in another already-archived link, it will be misjudged as "already archived," silently missing a genuinely new video. Changed to (?<!\d){video_id}(?!\d) numeric boundary matching: does not assume url: which link form is stored (/video/、modal_id=、resource_id= or the form after short-link resolution are both possible), only requires that neither side is a digit, balancing false-positive prevention and format independence.

real end-to-end verification (not mock, 2026-07-21, real public Douyin video):

  • full archive: Playwright intercepts aweme/detail obtains a 132MB video candidate direct link, download completes,file confirms a valid MP4, frontmatter like/comment/favorite/share counts match the manually verified real-time numbers (the slight difference between two readings of the same video within 40 minutes is real like growth, not a bug)
  • rerun the same video (without adding --force): correctly SKIP
  • deduplication boundary fix verified with real archive results: normal scenario (real /video/<id> link) unaffected, still correctly identifies as archived

py_compile + audit_skills.py: no findings.

receipt after completion

delivery order: first write the files to disk, then output the receipt below, not the other way around; uncertain metadata (such as author or publish time parsing failure) must be explicitly marked "unverified" in the receipt, do not fabricate.

after executionmustoutput to the user (do not do it silently):

  1. what was done — a one-sentence summary of the completed work.
  2. file changes — list the newly created vault note paths and local MP4 paths (full paths);--metadata-only when , explain that the video was not downloaded.
    • if Articles/ fallback was used, you must explicitly note that this directory is not the official archive location of this vault, and recommend that the user configure --articles-dir、OBSIDIAN_ARTICLES or private config.yml and then relocate it.
  3. next steps — optional follow-up suggestions (such as adding ## 摘要/topics/people, or linking to soia-pkm-organize-article-moc）。