clip of the familygeneric web page member: save blog / web articles into the vault.
Customer-readable description
What this skill can do
Archive web pages/blog articles to an Obsidian vault; when the web page has public PodcastEpisode metadata, also save shownotes and local audio at the same time. Articles continue to use readability/trafilatura to extract the body text, podcasts are parsed with JSON-LD, and both are stored according to the unified clip family specification. When the user says "archive and convert to PDF", after archiving, prefer to use Obsidian's built-in PDF within the Obsidian vault.
| Customer wants | Skill will do | Customer can see |
|---|---|---|
| Complete the work covered by this skill | Read the user request, necessary context, and this skill's main workflow, and execute the minimum reliable steps | The customer will see Obsidian/vault file changes, terminal logs, generated artifact paths, and a final receipt. |
| Missing dependencies, permissions, configuration, or key | Stop actions that require external state, and clearly state what is missing | Install commands, application addresses, configuration paths, or questions that require customer confirmation |
| Execution complete | Summarize successes, skips, failures, file changes, and verification results | A completion receipt that can be copied into a ticket/log |
How the customer uses it
- Describe the goal in natural language and provide the necessary inputs: files, URLs, repos, workspace, proposal, vault, or platform account status.
- For actions that support dry-run or preview, provide a preview first; when deletion, overwriting, sending, publishing, or writing remote state is involved, ask the customer for confirmation first.
Dependencies and installation
Installation (recommended: install the entire domain plugin, which installs all skills in this repo at once):
claude plugin marketplace add soia-team/soia-open-skills
claude plugin install soia-pkm-vault@soia
If you only want this one skill, you can use the npx route. Note that the skill will land in the shared source of truth ~/.agents/skills; if the plugin is also installed, the same skill will have two indexes and they will drift independently, so it is recommended to choose one of the two:
npx skills add soia-team/soia-open-pkm-vault-skills -g -a '*' -s soia-pkm-clip-web -y
Configuration conventions:
~/.config/soia-skills/soia-pkm-clip-web/config.yml
SOIA_PKM_CLIP_WEB_CONFIG_FILE=<custom-config-path>
- If this skill does not need private configuration, you can skip creating
config.yml。 - If an API key, cookie, session, provider home, or local path is needed, it can only be placed in private
config.yml, the process environment, or the provider's own login state; it must not be written into the repo, vault body text, or logs. - Third-party skills can only declare dependencies and installation methods; they must not directly modify third-party skill files.
WorkBuddy is loaded as role-based experts rather than plugins,npx skills add -a '*' cannot cover it; it needs to be installed separately, see docs/install/workbuddy.md。
Logs and completion receipt
Every execution must let the customer see the process and results. Minimum receipt format:
完成:<一句话说明本次完成了什么>。
日志摘要:
- started: <检查到的输入/配置/依赖,不打印秘密值>
- processed: <数量或范围>
- created/updated: <数量或路径>
- skipped/failed: <数量和原因>
文件变化:
- <绝对路径或“未改动文件”>
验证:
- <运行过的检查、命令或人工核对点>
问题与下一步:
- <缺 key / 缺依赖 / 需要客户确认 / 建议下一条命令;没有则写“无”>
Private information and intermediate data
- Share URLs are used only for this fetch; before persistence, remove query/fragment and do not write share identifiers into public repos, vaults, or logs.
- Media addresses with query parameters are used only within the process for downloading; notes save only the de-parameterized address and the path/hash of the downloaded file.
.partare temporary download files; clean them up on failure and atomically replace them with the final media file on success.
Fetching
- Input: any article URL (blog / Substack / Medium / news / Zhihu, etc.)
- Body extraction:
trafilaturaorreadability-lxmlextract the body (remove ads / navigation), extract title, author, and publication time. The two are not necessarily preinstalled:pip installis often blocked by PEP 668(externally-managed) blocked, per host rules use--break-system-packagesor a dedicatedvenvinstall; if the host lacksagent_browser/no browser, Cloudflare-type sites (such as Medium) can first use a public reader proxy (such asr.jina.ai) to retrieve Markdown as a fallback, but that source must be written into "fetch method" and the original site marked as protected. - Cannot fetch the body →
content_complete: false,never silently truncate。 - Ordinary articles are still executed by the agent according to the process in this section; podcast web pages are read and execute podcast web page archiving, using
scripts/archive_podcast.pydeterministic parsing, downloading, and landing. - On mobile, Obsidian Web Clipper can be used to land into
<vault-inbox-dir>/, then migrated in by this skill.
Podcasts and audio
- When the page publicly exposes
PodcastEpisodeJSON-LD and the client requests audio, prefer the dedicated script; do not hardcode private APIs for any single platform. - shownotes and transcripts are kept separate: a complete page description can only prove
content_complete: true, and when there is no transcript, you must writetranscript_status: not_provided_by_source。 - Podcast audio lands by default in
<vault>/_attachments/podcasts/<episode-id>/, and the vault note records the playable embed, vault-relative path, bytes, and SHA-256; whether large files enter Git is determined by the vault's local exclude, Git LFS, or a separate backup strategy. - By default, clear the sharing query/fragment before persisting the canonical URL. Media URLs with query are used only in memory for this download and are not written to the vault or logs.
- When you only want to verify parsing capability or only collect shownotes, use
--metadata-only; when the client explicitly requests audio, a failed audio download must not package the body note as a complete delivery.
Fixture regressions for JSON-LD, URL parameter stripping, duration parsing, file naming, and fake-IP/SSRF gating are in tests/test_archive_podcast.py; real delivery must also use actual episode pages and audio containers for end-to-end review.
Mandatory fetch quality review
A script exit code of 0 does not equal successful fetching—some sites return login walls, navigation shells, or anti-scraping block pages as normal responses, and the script will not report an error.Before writing to the vault, you must personally read the artifact, and verify: whether the title matches the original page (not generic site titles like "Login" or "Access Exception"), whether the body is the complete article (not a pile of navigation bars/sidebars, and not all links), and whether the word count roughly matches the original text's scale.
Failure signal checklist (hitting any one means failure, and it must not be delivered directly):
- Login wall/verification keywords such as "Login", "Please log in first", or "verify you are human" appear
- The body is obviously too short (it should be a long article but has only a few dozen characters)
- The body is almost entirely links, with no substantive narrative text
When a failure signal is hit, switch strategy (switch readability-lxml/trafilatura as mutual fallbacks, prompt the user to log in manually and retry, or truthfully mark content_complete: false and remind for manual verification against the original), rather than writing the block page into the vault as the body.
Landing (unified specification for the clip family)
- Path:
<vault-articles-dir>/<年>/YYYY-MM-DD-<来源>-<作者>-<标题>.md(sources such as blog / Substack / Medium) - frontmatter is the same as the clip family; body
## 摘要 / 原文 / 我的看法 / 关联。 - Non-Chinese body (
languagenon-zh): before关联prepend## 中文译文, translate freely rather than literally, write like natural Chinese, keep proper nouns/product names/code in English—this is a general clip-web rule, not limited to the X channel. - Article → long-term knowledge: if the article distills concepts/patterns reusable across projects, it can further go through
soia-pkm-extract-vault-knowledgeto be deposited into Zone 20 (new numbered subdirectory + update topic navigation + rebuild map/verify Base); the receipt must separately report "archived" and "long-term knowledge deposited", and must not treat archiving as already distilled. - Single-article archiving is by default not the endpoint: after the archive write succeeds and the body quality review passes, automatically hand the file to
soia-pkm-organize-article-mocfor minimal organization (summary/topics, year-month placement,only incrementally sync affected MOCs and index gating). A single-article task must not invoke a full rebuild process that first clears the entire_MOC. Only when the user explicitly says "archive only / no organization" should it stop at the clip result; batch web pages must first have a list drawn up, then confirm whether to batch-organize. - Podcast audio is by default written to the vault root's
_attachments/podcasts/<episode-id>/, and the archive note must include## 🎧 收听本地音频and Obsidian's native![[相对路径]]player. When a local file exists but the note contains only absolute path text, the status is stillaudio_embedded: false, and you must not claim it is playable within Obsidian.
Single-article archiving completion gate
The archive receipt must be reported separately:captured(file has been written to disk),organized(whether single-article organization succeeded),moc_synced、map_synced、base_verified. If any post-step fails, you can only say "archived, organization incomplete" and give the retryable file path; clip success cannot be equated with knowledge base organization completion.
Export PDF after archiving
When the user also requests "convert to PDF / export PDF", first complete the Markdown archiving, summary, topics, and month placement, then read and execute references/obsidian-pdf-export.md. As long as the target file is within the Obsidian vault, prioritize calling Obsidian's built-in "Export PDF"; external PDF engines can only serve as an explicit fallback.
Closed-loop position
★clip-web(收) → organize → distill → compose → publish。
Receipt after completion
Delivery order: first write the file to disk, then output the receipt below, not the reverse; uncertain metadata (such as author or publish time that failed to parse) must be explicitly marked "unverified" in the receipt, and must not be fabricated.
The receipt includes:
- What was done — a one-sentence summary of the completed work.
- File changes — list newly created / modified / moved files (full paths); if no files were changed, state "no files changed".
- Next steps — optional follow-up suggestions (such as the next skill to connect to).