Preserve valuable Douyin videos into an Obsidian vault: a lightweight Markdown note records metadata and copy, while the video file is stored separately in the local Downloads directory, so large binaries are not stuffed into the git-tracked knowledge base.
Customer-readable description
What this skill can do
| Customer wants | Skill will do | Customer can see |
|---|---|---|
| Archive a Douyin video link | Use Playwright to open the video page and intercept the signed API, obtaining the author/copy/duration/interaction counts and a downloadable direct link; download the MP4 locally; write a lightweight Markdown note in the vault (including media_local_path) |
A new .md file is added to the vault, and the terminal prints the file path, author, publish time, download path, and interaction data |
| Only want to first see whether this link can be parsed, without downloading the video for now | Add --metadata-only: only scrape metadata and copy to write the note, skipping video download |
In the note, media_local_path is empty,media_fetched: false, and the log indicates the number of candidate direct links |
| Archiving the same video repeatedly | Deduplicate by the frontmatter url: of existing notes in the vault |
Print ⚠️ Already archived and exit, without re-downloading or re-writing the note |
| Missing Playwright dependency | Stop and provide the installation command, without guessing at a fallback | ❌ error message starting with + pip install playwright && python -m playwright install chromium |
How customers use it
- Provide a Douyin video link (
https://www.douyin.com/video/<id>, a link with themodal_id=/resource_id=parameter, or av.douyin.comshort link). - Run
python3 scripts/archive_douyin.py <URL>in the vault root directory; when not inside the vault, add--vault <path>。 - The script automatically deduplicates, scrapes, downloads, and writes the note; if any part of the process fails, it will use
❌to report the specific reason, and will not leave a half-finished note. - After archiving is complete, follow the "Next step" prompt in the receipt to supplement
## 摘要、topics、people。
Dependencies and installation
| Dependency | Type | Installation / configuration | What to do when missing |
|---|---|---|---|
playwright(Python package) |
Hard dependency | pip install playwright && python -m playwright install chromium |
Stop and print the installation command, exit with a non-zero code, and do not perform any speculative fallback |
| Obsidian vault | Hard dependency | Any structure, but it must be locatable by one of three methods: --vault / OBSIDIAN_VAULT / cwd auto-discovery |
Report an error and explain how to specify the path |
Python 3 stdlib(urllib) |
Hard dependency | Comes with Python, no installation required | — |
Installation (recommended: install the entire domain plugin, setting up all skills in this repository at once):
claude plugin marketplace add soia-team/soia-open-skills
claude plugin install soia-pkm-vault@soia
When you only want this one skill, you can use the npx route. Note that the skill will land in the shared source of truth ~/.agents/skills; if the plugin is also installed, the same skill will appear as two indexes and drift separately, so it is recommended to choose one of the two:
npx skills add soia-team/soia-open-pkm-vault-skills -g -a '*' -s soia-pkm-clip-douyin -y
Configuration conventions:
~/.config/soia-skills/soia-pkm-clip-douyin/config.yml
SOIA_PKM_CLIP_DOUYIN_CONFIG_FILE=<custom-config-path>
- If you only archive occasionally and do not want a fixed vault path, you can skip creating
config.yml, and just pass it with--vaulteach time. - Supported variable names:
OBSIDIAN_VAULT、OBSIDIAN_ARTICLES. The real path is only written in the privateconfig.yml, refer to config.example.yml。 - Priority:
--vault/--articles-dirCLI argument >OBSIDIAN_VAULT/OBSIDIAN_ARTICLESprocess environment variable >SOIA_PKM_CLIP_DOUYIN_CONFIG_FILEprivate directory pointed to byconfig.yml> automatically discover vault upward from the current working directory (look forAGENTS.md/.obsidian) > general fallbackArticles/(and print a warning to stderr).
WorkBuddy loads as role-based experts rather than plugins,npx skills add -a '*' cannot cover it; it needs to be installed separately, see docs/install/workbuddy.md。
Private information and intermediate data
- The video file itselfdoes not go into the vault(the vault is a git-tracked Markdown knowledge base and is not suitable for storing large binaries); the MP4 lands at
~/Downloads/soia-pkm-clip-douyin/<video_id>/video.mp4, and the vault note only stores an index of this local absolute path. - The signed direct link intercepted by Playwright (including
msToken/a_bogusand other token parameters)is not written to the vault, not written to logs, and not written to stdout—failure receipts only print the host+path with the query string removed, to avoid leaving authentication tokens in plaintext in logs/notes that may be shared. - The signed direct link itself is short-lived: by design, the script will not cache the raw original video direct link to disk or pass it across processes;
--metadata-onlyafterward, if the user wants the video, the script must be rerun (see "Boundaries and Exceptions"). - Playwright uses a one-off headless browser context (does not persist login state, does not use the user's real Douyin account cookies), and only performs anonymous web browsing; it does not write to a provider login-state directory.
- Does not cache or persist to disk any privacy information of anyone other than the user; captured author nicknames/UIDs are used only for this note's frontmatter/body.
Logs and completion receipts
The video file being written to disk only means captured; afterward, organized → MOC/导航 → map → Basemust be completed, and transcript/OCR and other body searchability must be reported separately. The contract is detailed in the lifecycle management skill.
完成:<一句话说明本次归档了什么>。
日志摘要:
- started: 已定位 vault / video_id=<id>
- processed: 1 条抖音视频
- created/updated: <vault 内笔记相对路径>
- skipped/failed: <0 或具体原因>
文件变化:
- <vault 笔记绝对路径>
- <本地 MP4 绝对路径,或"未下载(--metadata-only)">
验证:
- 笔记内 `media_local_path` 文件是否存在、MP4 文件头是否为有效 ftyp box
问题与下一步:
- <缺 playwright / 视频不可见 / 需要客户确认 / 建议下一条命令;没有则写"无">
Location
Videos inside the Douyin App are not convenient to accumulate long-term into searchable textual knowledge assets: the copy gets buried in the feed, cannot be found, cannot be searched, and cannot be backlinked with other notes. This skill archives the metadata and copy of a single Douyin video into Obsidian with one click, while the video itself stays on the local disk, keeping the vault lightweight.
Prerequisite dependencies
- Obsidian vault (any structure)
- Python 3 +
playwright(required for intercepting the signed API, see the installation command above); the download step itself is pure stdliburllib, no additional dependencies required - Vault location priority:
- Command line
--vault <path> - Environment variable
OBSIDIAN_VAULT - Automatically discover vault upward from the current working directory (look for
AGENTS.md/.obsidian)
- Command line
- Article directory location priority:
- Command line
--articles-dir <subdir> - Environment variable
OBSIDIAN_ARTICLES - Fall back to general default
Articles; the script will print a warning to stderr, and the exit code remains 0
- Command line
Note: tools such as Claude Code / Codex / Gemini CLI / opencode often execute commands in a non-login shell and may not inherit the
exportfrom shell startup files. The script will first load the privateconfig.yml, then automatically discover the vault from cwd; if the agent is already in the vault directory, just run it directly.
-
Private
config.yml(optional,do not place in the vault or an open-source skill repository):Priority:
$SOIA_PKM_CLIP_DOUYIN_CONFIG_FILE(or compatible alias$SOIA_PKM_CLIP_DOUYIN_ENV_FILE)>~/.config/soia-skills/soia-pkm-clip-douyin/config.yml。Supported variable names:
OBSIDIAN_VAULT、OBSIDIAN_ARTICLES. The real value is only written in the privateconfig.yml, not written into this file.
Core mechanism: Playwright intercepts the signed API
The Douyin web playback address is dynamically generated by the client signature (msToken/a_bogus), and the page SSR JSON (RENDER_DATA) does not contain play_addr. There is no pure HTTP way to fetch the data; a real browser must be used once:
playwright.chromium.launch(headless=True)+new_context(user_agent=<desktop UA>, locale="zh-CN")。page.on("response", handler)Listen for all JSON responses under thedouyin.comdomain.- Matching strategyby content rather than endpoint name: recursively search each response body, looking for a dict that has
str(node["aweme_id"]) == 目标idand itself carries the"video"key—regardless of whether it appears at the expectedaweme/detail/endpoint, or is mixed into theaweme/mix/array returned by a collection endpoint such asaweme_list[]. - Poll until a hit or timeout (default 60s,
--timeoutadjustable). - After a hit, take the direct link by priority:
play_addr→play_addr_h264→download_addr, trying to download one by one until success.
| Key field | Purpose |
|---|---|
aweme["video"]["play_addr"]["url_list"][0] |
Video direct link (h264/download_addr fallback) |
aweme["desc"] |
Caption/title |
aweme["author"]["nickname"] / ["uid"] |
Author |
aweme["create_time"] |
Publish time (Unix seconds) |
aweme["video"]["duration"] |
Duration (milliseconds) |
aweme["statistics"] |
digg_count/comment_count/collect_count/share_count/play_count/admire_count/recommend_count |
The download step uses pure stdlib urllib.request:User-Agent + Referer: https://www.douyin.com/ + Range: bytes=0-, streaming to disk and verifying the byte count and MP4 magic bytes (ftyp box)。
Trigger words
| User says | Invoke |
|---|---|
| "Archive this Douyin: URL" / "clip this Douyin video" | Run archive_douyin.py <URL>in the vault root; when not inside the vault, add --vault <path> |
| "First see whether this link can be parsed" / "only the caption, no video" | Add --metadata-only, skip the video download |
| "Re-archive this Douyin" (already exists) | Add --force, will keep the existing topics/people/## 摘要 |
Workflow (default archive)
- Parse URL →
video_id(short links first do one HTTP redirect resolution before matching) - Call script
archive_douyin.py <URL> - The script checks whether the vault has already archived it (by whether the note frontmatter
url:line contains thatvideo_id)- Already exists → output
SKIP, exit 0 - Otherwise → start Playwright to intercept the signed response and get the
awemeobject
- Already exists → output
- When not
--metadata-only: download the MP4 to~/Downloads/soia-pkm-clip-douyin/<video_id>/video.mp4; if any step fails, do not write the note - Write the vault note (frontmatter + body)
- AI fills in afterward:
- 1-sentence Chinese
## 摘要 - 1-3
topicsdouble links +peopledouble links
- 1-sentence Chinese
- The "My take" section is always left blank for the user
File naming
<YYYY-MM-DD>-抖音-<作者昵称前 20 字>-<文案前 50 字>.md
Multiple entries by the same author on the same day → append -<video_id 后 6 位>。
Output frontmatter
---
tags: [视频摘抄]
source: 抖音
url: <用户给出的原始链接>
author: <昵称>
published_at: YYYY-MM-DD HH:mm
captured_at: YYYY-MM-DD HH:mm
language: zh
type: video
topics: [] # AI 填
people: [] # AI 填
media_local_path: "<本地 MP4 绝对路径,--metadata-only 时为空>"
content_complete: true
metrics:
likes: N
comments: N
collects: N
shares: N
plays: N
admires: N
recommends: N
---
Body structure:# <标题> → > [!source]- source collapsible block (source/fetch method/metadata/media status) → ## 摘要(AI fills in) → ## 原文(original Douyin caption, unprocessed) → ## 我的看法(left blank) → ## 关联。
Boundaries and exceptions
| Scenario | Handling |
|---|---|
URL does not contain /video/<id>、modal_id=、resource_id=, nor is it a short link that can be redirected and resolved |
Reject, and the error message explains the supported formats |
Playwright polling timed out without intercepting the target aweme |
The video has been deleted/private/region-restricted, or Douyin changed the API structure; report an explicit error, do not guess |
Intercepted aweme but all candidate direct-link downloads failed |
Print the failure reason for each one (URL with query string removed), exit non-zero,do not write a note |
After archiving with --metadata-only you want the video |
you must rerun the script completely once more — signed direct links are short-lived, and the script does not do a two-stage "record the direct link first, download later" design |
| The same video is archived again with a short link, having previously been archived with a full link | The short-link text itself does not contain the numeric video_id; deduplication by frontmatter url: line substring matching may fail; it is recommended to prefer archiving with the full douyin.com/video/<id> link |
| Duplicate filename for the same author on the same day | Append at the end -<video_id 后 6 位> |
| already archived in the vault | Default SKIP and exit 0;--force only then overwrite (preserving existing topics/people/## 摘要) |
Command-line reference
python3 scripts/archive_douyin.py <URL>
python3 scripts/archive_douyin.py <URL> --force # 覆盖已归档
python3 scripts/archive_douyin.py <URL> --vault /path/to/vault # 覆盖环境变量
python3 scripts/archive_douyin.py <URL> --articles-dir <articles-subdir>
python3 scripts/archive_douyin.py <URL> --metadata-only # 只抓元数据,跳过视频下载
python3 scripts/archive_douyin.py <URL> --timeout 90 # 拦截等待超时秒数(默认 60)
Verification and testing
Adversarial review (correctness + security dual lens) found 1 should-fix issue:find_existing_archive 's deduplication matching was originally a bare substring test against url: the entire line, without adding boundary anchors like archive_x.py 's /status/{id} —video_id when it happens to be a substring of a longer digit string in another already-archived link (or some query parameter value), it would falsely judge it as "already archived" and silently miss a real new video. Changed to (?<!\d){video_id}(?!\d) numeric boundary matching: it does not assume url: stores which link form (/video/、modal_id=、resource_id= or the form after short-link resolution are both possible), it only requires that neither side is a digit, balancing false-positive prevention with format independence.
Real end-to-end verification (non-mock, 2026-07-21, real public Douyin video):
- Full archive: Playwright intercepted
aweme/detailobtained a 132MB video candidate direct link, download completed,fileconfirmed a valid MP4, and the frontmatter like/comment/favorite/share counts matched the real-time numbers manually checked (the small difference between two readings of the same video within 40 minutes is real like growth, not a bug) - Rerun the same video (without adding
--force): correctlySKIP - The deduplication boundary fix was verified with real archive results: normal scenarios (real
/video/<id>links) were unaffected and already-archived was still correctly recognized
py_compile + audit_skills.py: no findings.
Receipt after completion
Delivery order: first write the file to disk, then output the receipt below, never the reverse; uncertain metadata (such as author or publish time parse failures) must be explicitly marked "unverified" in the receipt, do not fabricate.
The receipt includes:
- What was done — a one-sentence summary of the work completed.
- File changes — list the newly created vault note path and local MP4 path (full paths);
--metadata-only, state that the video was not downloaded.- If the
Articles/fallback was used, you must explicitly note that this directory is not this vault's official archive location, and recommend that the user configure--articles-dir、OBSIDIAN_ARTICLESor a privateconfig.ymlthen return to place.
- If the
- Next step — optional follow-up suggestions (e.g., add
## 摘要/topics/people, or connectsoia-pkm-organize-article-moc)。