obsidian-knowledge-filter

Skill: Obsidian Knowledge Filter

Description

This skill is designed to retrieve, filter, and extract knowledge from a knowledge base composed of a large number of Markdown files (such as an Obsidian Vault) through a precise and phased workflow. It uses YAML and Markdown files with numbered prefixes to manage the intermediate data of each step, and pauses at each critical step for user review, ensuring that the final output is highly relevant and non-duplicative.

Core Concept: The goal of this skill is, under the user's guidance, to precisely distill information related to a specific topic from the knowledge base through a series of independent scripts and analysis steps, present it in a structured manner, and ultimately form a comprehensive analysis report.

When to Use

Use this skill when the user requests that you solve complex problems by finding, associating, and synthesizing information scattered across a personal knowledge base. This workflow is particularly suitable for scenarios that require high precision, hope to reduce redundant information, and want to participate throughout the filtering process.

Workflow

The process is divided into four main phases, each with user review as a checkpoint.

Phase 1: Keyword Search and Analysis

Step 1.1: Run the analysis script `001_analyze_files.py`

This script recursively searches all .md files in the specified directory and performs the following operations:

File Exclusion: Automatically skip any file/directory whose path contains tmp-, _attachments, or .trash .
Keyword Analysis: Find files containing the specified keywords and record the frequency of occurrence of each keyword.
Context Merging: Merge the context of lines containing keywords (by default, 10 lines before and after) into continuous line number intervals.
Output: In a newly created tmp-YYYY-MM-DD-HH-MM-SS-analysis/ temporary directory, generate 001_analysis.yaml。

001_analysis.yaml Format Details: This is a YAML file containing a list of file information objects. Each object has the following structure:

- file_path: '文件夹/2025-12-29-示例文件.md'  # 相对于当前工作目录的路径
  keyword_counts:
    - word: '关键词1'
      count: 5
    - word: '关键词2'
      count: 2
  line_ranges:
    - [10, 30]   # 第一个内容提取区间
    - [45, 65]   # 第二个内容提取区间

Step 1.2: User reviews the analysis file

The Agent pauses and requests the user to review 001_analysis.yaml. The user can manually edit this file to add or remove files or modify line number intervals.

Phase 2: Content Extraction and Cleanup

Step 2.1: Run the extraction script `002_extract_content.py`

After user confirmation, this script reads 001_analysis.yamland, based on the file_path and line_ranges within it, extracts the corresponding content.

Content Cleanup: After extracting content, the script performs the following three automatic cleanup operations:
1. Merge Line Breaks: Merge multiple consecutive line breaks into one.
2. Remove Bold: Delete all Markdown bold markers (**)。
3. Remove Images: Delete all Markdown image links (![[...]])。
Output: In the same temporary directory, generate 002_content.yaml, whose structure is similar to 001_analysis.yaml , but each object has an additional content field.

002_content.yaml Format Details:

- file_path: '文件夹/2025-12-29-示例文件.md'
  keyword_counts:
    # ... (同上)
  line_ranges:
    # ... (同上)
  content: "这是从文件中提取并清理后的文本内容...\n..."

Step 2.2: User reviews the content file

The Agent pauses again and requests the user to review 002_content.yaml, to ensure the extracted and cleaned content meets expectations.

Phase 3: Content Insight Synthesis

Step 3.1: Agent performs chunked comprehensive analysis

After user confirmation, the Agent begins intelligent analysis of 002_content.yaml . To improve efficiency and ensure continuity of analysis, the Agent uses a chunked "read-analyze-write" loop mode:

Chunked reading: The Agent reads the 002_content.yaml file chunk by chunk in units of 1000 lines; for example, the first pass reads lines 1-1000.
Comprehensive analysis: For the entire content blockjust read (which may contain multiple file fragments), the Agent internally performs a one-time comprehensive analysis to extract all insights, viewpoints, and connections relevant to the user's task.
Single write: The Agent appends the analytical insights generated in 2 in a single operation to the 003_content_analysis.md file, clearly indicating which file each insight comes from.
Loop execution: Go to 1 to continue with the next chunk until all content chunks have been processed.

This method avoids the inefficiency and fragmentation of item-by-item analysis, ensuring the fluency and contextual relevance of the analysis process.

Output: 003_content_analysis.md, which contains comprehensive insights into all file fragments.

Step 3.2: User reviews the insight file

After the Agent completes analysis of all content chunks, it pauses and requests the user to review the final generated 003_content_analysis.md, confirming the accuracy and depth of the insights.

Phase 4: Final Report Synthesis

Step 4.1: Agent generates the final report

After the user completes the final review, the Agent will use 003_content_analysis.md as material to write a logically coherent and clearly structured final report.

Output: 004_report.md

Step 4.2: Submit results

The Agent submits the final 04_report.md report to the user.