SuperAPI AI Directory

soia-pkm-clip-web

General & other

View repository

Updated 10-16 08:45

About this skill

soia-pkm-clip-web

clip of the familygeneric web page member: save blog / web articles into the vault.

Customer-readable description

What this skill can do

Archive web pages/blog articles to an Obsidian vault; when the web page has public PodcastEpisode metadata, also save shownotes and local audio at the same time. Articles continue to use readability/trafilatura to extract the body text, podcasts are parsed with JSON-LD, and both are stored according to the unified clip family specification. When the user says "archive and convert to PDF", after archiving, prefer to use Obsidian's built-in PDF within the Obsidian vault.

Customer wants Skill will do Customer can see
Complete the work covered by this skill Read the user request, necessary context, and this skill's main workflow, and execute the minimum reliable steps The customer will see Obsidian/vault file changes, terminal logs, generated artifact paths, and a final receipt.
Missing dependencies, permissions, configuration, or key Stop actions that require external state, and clearly state what is missing Install commands, application addresses, configuration paths, or questions that require customer confirmation
Execution complete Summarize successes, skips, failures, file changes, and verification results A completion receipt that can be copied into a ticket/log

How the customer uses it

  1. Describe the goal in natural language and provide the necessary inputs: files, URLs, repos, workspace, proposal, vault, or platform account status.
  2. For actions that support dry-run or preview, provide a preview first; when deletion, overwriting, sending, publishing, or writing remote state is involved, ask the customer for confirmation first.

Dependencies and installation

Installation (recommended: install the entire domain plugin, which installs all skills in this repo at once):

claude plugin marketplace add soia-team/soia-open-skills
claude plugin install soia-pkm-vault@soia

If you only want this one skill, you can use the npx route. Note that the skill will land in the shared source of truth ~/.agents/skills; if the plugin is also installed, the same skill will have two indexes and they will drift independently, so it is recommended to choose one of the two:

npx skills add soia-team/soia-open-pkm-vault-skills -g -a '*' -s soia-pkm-clip-web -y

Configuration conventions:

~/.config/soia-skills/soia-pkm-clip-web/config.yml
SOIA_PKM_CLIP_WEB_CONFIG_FILE=<custom-config-path>
  • If this skill does not need private configuration, you can skip creating config.yml。
  • If an API key, cookie, session, provider home, or local path is needed, it can only be placed in private config.yml, the process environment, or the provider's own login state; it must not be written into the repo, vault body text, or logs.
  • Third-party skills can only declare dependencies and installation methods; they must not directly modify third-party skill files.

WorkBuddy is loaded as role-based experts rather than plugins,npx skills add -a '*' cannot cover it; it needs to be installed separately, see docs/install/workbuddy.md。

Logs and completion receipt

Every execution must let the customer see the process and results. Minimum receipt format:

完成:<一句话说明本次完成了什么>。

日志摘要:
- started: <检查到的输入/配置/依赖,不打印秘密值>
- processed: <数量或范围>
- created/updated: <数量或路径>
- skipped/failed: <数量和原因>

文件变化:
- <绝对路径或“未改动文件”>

验证:
- <运行过的检查、命令或人工核对点>

问题与下一步:
- <缺 key / 缺依赖 / 需要客户确认 / 建议下一条命令;没有则写“无”>

Private information and intermediate data

  • Share URLs are used only for this fetch; before persistence, remove query/fragment and do not write share identifiers into public repos, vaults, or logs.
  • Media addresses with query parameters are used only within the process for downloading; notes save only the de-parameterized address and the path/hash of the downloaded file.
  • .part are temporary download files; clean them up on failure and atomically replace them with the final media file on success.

Fetching

  • Input: any article URL (blog / Substack / Medium / news / Zhihu, etc.)
  • Body extraction:trafilatura or readability-lxml extract the body (remove ads / navigation), extract title, author, and publication time. The two are not necessarily preinstalled:pip install is often blocked by PEP 668(externally-managed) blocked, per host rules use --break-system-packages or a dedicated venv install; if the host lacks agent_browser/no browser, Cloudflare-type sites (such as Medium) can first use a public reader proxy (such as r.jina.ai) to retrieve Markdown as a fallback, but that source must be written into "fetch method" and the original site marked as protected.
  • Cannot fetch the body → content_complete: false,never silently truncate。
  • Ordinary articles are still executed by the agent according to the process in this section; podcast web pages are read and execute podcast web page archiving, using scripts/archive_podcast.py deterministic parsing, downloading, and landing.
  • On mobile, Obsidian Web Clipper can be used to land into <vault-inbox-dir>/, then migrated in by this skill.

Podcasts and audio

  • When the page publicly exposes PodcastEpisode JSON-LD and the client requests audio, prefer the dedicated script; do not hardcode private APIs for any single platform.
  • shownotes and transcripts are kept separate: a complete page description can only prove content_complete: true, and when there is no transcript, you must write transcript_status: not_provided_by_source。
  • Podcast audio lands by default in <vault>/_attachments/podcasts/<episode-id>/, and the vault note records the playable embed, vault-relative path, bytes, and SHA-256; whether large files enter Git is determined by the vault's local exclude, Git LFS, or a separate backup strategy.
  • By default, clear the sharing query/fragment before persisting the canonical URL. Media URLs with query are used only in memory for this download and are not written to the vault or logs.
  • When you only want to verify parsing capability or only collect shownotes, use --metadata-only; when the client explicitly requests audio, a failed audio download must not package the body note as a complete delivery.

Fixture regressions for JSON-LD, URL parameter stripping, duration parsing, file naming, and fake-IP/SSRF gating are in tests/test_archive_podcast.py; real delivery must also use actual episode pages and audio containers for end-to-end review.

Mandatory fetch quality review

A script exit code of 0 does not equal successful fetching—some sites return login walls, navigation shells, or anti-scraping block pages as normal responses, and the script will not report an error.Before writing to the vault, you must personally read the artifact, and verify: whether the title matches the original page (not generic site titles like "Login" or "Access Exception"), whether the body is the complete article (not a pile of navigation bars/sidebars, and not all links), and whether the word count roughly matches the original text's scale.

Failure signal checklist (hitting any one means failure, and it must not be delivered directly):

  • Login wall/verification keywords such as "Login", "Please log in first", or "verify you are human" appear
  • The body is obviously too short (it should be a long article but has only a few dozen characters)
  • The body is almost entirely links, with no substantive narrative text

When a failure signal is hit, switch strategy (switch readability-lxml/trafilatura as mutual fallbacks, prompt the user to log in manually and retry, or truthfully mark content_complete: false and remind for manual verification against the original), rather than writing the block page into the vault as the body.

Landing (unified specification for the clip family)

  • Path:<vault-articles-dir>/<年>/YYYY-MM-DD-<来源>-<作者>-<标题>.md(sources such as blog / Substack / Medium)
  • frontmatter is the same as the clip family; body ## 摘要 / 原文 / 我的看法 / 关联。
  • Non-Chinese body (language non-zh): before 关联 prepend ## 中文译文, translate freely rather than literally, write like natural Chinese, keep proper nouns/product names/code in English—this is a general clip-web rule, not limited to the X channel.
  • Article → long-term knowledge: if the article distills concepts/patterns reusable across projects, it can further go through soia-pkm-extract-vault-knowledge to be deposited into Zone 20 (new numbered subdirectory + update topic navigation + rebuild map/verify Base); the receipt must separately report "archived" and "long-term knowledge deposited", and must not treat archiving as already distilled.
  • Single-article archiving is by default not the endpoint: after the archive write succeeds and the body quality review passes, automatically hand the file to soia-pkm-organize-article-moc for minimal organization (summary/topics, year-month placement,only incrementally sync affected MOCs and index gating). A single-article task must not invoke a full rebuild process that first clears the entire _MOC . Only when the user explicitly says "archive only / no organization" should it stop at the clip result; batch web pages must first have a list drawn up, then confirm whether to batch-organize.
  • Podcast audio is by default written to the vault root's _attachments/podcasts/<episode-id>/, and the archive note must include ## 🎧 收听本地音频 and Obsidian's native ![[相对路径]] player. When a local file exists but the note contains only absolute path text, the status is still audio_embedded: false, and you must not claim it is playable within Obsidian.

Single-article archiving completion gate

The archive receipt must be reported separately:captured(file has been written to disk),organized(whether single-article organization succeeded),moc_synced、map_synced、base_verified. If any post-step fails, you can only say "archived, organization incomplete" and give the retryable file path; clip success cannot be equated with knowledge base organization completion.

Export PDF after archiving

When the user also requests "convert to PDF / export PDF", first complete the Markdown archiving, summary, topics, and month placement, then read and execute references/obsidian-pdf-export.md. As long as the target file is within the Obsidian vault, prioritize calling Obsidian's built-in "Export PDF"; external PDF engines can only serve as an explicit fallback.

Closed-loop position

★clip-web(收) → organize → distill → compose → publish。


Receipt after completion

Delivery order: first write the file to disk, then output the receipt below, not the reverse; uncertain metadata (such as author or publish time that failed to parse) must be explicitly marked "unverified" in the receipt, and must not be fabricated.

The receipt includes:

  1. What was done — a one-sentence summary of the completed work.
  2. File changes — list newly created / modified / moved files (full paths); if no files were changed, state "no files changed".
  3. Next steps — optional follow-up suggestions (such as the next skill to connect to).