SuperAPI AI Directory

doc-skill-generator

Coding & development

View repository

Updated 10-15 15:43

About this skill

Doc Skill Generator

Task Objective

This Skill is used to automatically extract documentation content from the official websites of technical frameworks or programming languages, and through intelligent analysis generate standardized Skill sub-skill packages, helping users systematically learn the tech stack.

Core Capabilities

  • Intelligent crawling of official website documentation (path filtering based on type)
  • Automatic extraction of version numbers and metadata
  • LLM content summarization and core capability graph generation
  • Templated sub-skill generation (language type/framework type/tool type)
  • Automatic validation of skill completeness

Trigger Conditions

Typical user expressions:

  • "Help me generate a Vue learning skill package"
  • "Create a corresponding skill based on the React official website"
  • "I need a Go language skill package"
  • "Generate a skill based on the TypeScript official website"

Prerequisites

Dependency Installation

crawl4ai>=0.4.0
playwright==1.40.0

Playwright browser initialization (first use)

playwright install chromium

Operation Steps

1. Information Collection and Preparation

Extract the following information from the user request:

  • Official website URL (required)
  • Tech stack name (optional, can be inferred from the URL)
  • Documentation type (language type/framework type/tool type, optional, can be intelligently inferred)
  • Crawl depth (optional, default 2 levels)

2. Website Documentation Crawling (Enhanced Version)

Call scripts/crawl_website.py to crawl the official website content:

python scripts/crawl_website.py \
  --url "https://vuejs.org" \
  --depth 2 \
  --output "/workspace/projects/vue-skills/raw_docs" \
  --type framework

Parameter Description:

  • --url: Official website address
  • --depth: Crawl depth (2-3 levels recommended)
  • --output: Documentation output directory
  • --max-pages: Maximum number of pages to crawl (default 50)
  • --type: Documentation type (language/framework/tool)

Enhanced Features:

  • Intelligent path filtering: only crawl documentation paths such as /docs/, /guide/, /api/, etc.
  • Automatic version detection: extract version number from the homepage
  • Metadata injection: record source URL, crawl time, and version number at the beginning of the file

3. Content Summarization and Analysis

Call scripts/summarize_pages.py to generate a summary report:

python scripts/summarize_pages.py \
  --input "/workspace/projects/vue-skills/raw_docs" \
  --output "/workspace/projects/vue-skills/summary.json" \
  --type framework

Output Content:

  • Basic page information statistics
  • Documentation structure analysis (API/guide/example distribution)
  • Core concept graph (basic version)
  • Skill type recommendation
  • Priority reading page list

4. Sub-skill Template Selection and Filling

Based on the summary report, the agent selects an appropriate template and fills in the content:

4.1 Select Template

Select from references/templates/ based on the documentation type:

  • language-skill.md - Programming language type (Go, Python, Rust)
  • framework-skill.md - Frontend framework type (Vue, React, Angular)
  • tool-skill.md - Tool type (Webpack, Docker, ESLint)

4.2 Fill in Metadata

Extract from the summary report:

---
name: <技术栈名称>-skills
description: <能力描述>
metadata:
  source_url: <官网URL>
  version: <版本号>
  crawled_at: <抓取时间>
  doc_type: <文档类型>
---

4.3 Fill in Core Content

Based on the summary report and the original documentation, the agent fills in:

  • Task Objective: generated based on the core concept graph
  • Core Capabilities: extracted from the documentation structure analysis
  • Operation Steps: Organize by learning path
  • Resource index: Link to the corresponding reference documentation

4.4 Generate reference documentation

Extract key content from the crawled documentation:

  • core-concepts.md - Detailed explanation of core concepts (>100 lines, including TOC)
  • api-reference.md - API reference documentation
  • best-practices.md - Best practices guide
  • examples.md - Practical example code

4.5 Generate code templates

Extract runnable examples from the official documentation:

  • assets/hello-world.* - Hello World example
  • assets/basic-template.* - Basic template
  • assets/advanced-example.* - Advanced example

5. Skill validation (optional)

Call scripts/validate_skill.py to validate the generated skill:

python scripts/validate_skill.py \
  --skill-path "/workspace/projects/vue-skills" \
  --output "/workspace/projects/vue-skills/validation_report.json"

Validation content:

  • Metadata completeness check
  • Directory structure specification check
  • Content quality check (description length, line count limits)
  • Automatically generate test questions

6. Package and deliver

Call the packaging tool to generate the .skill file:

package_skill(skill_dir_name="<技术栈名称>-skills")

Resource index

Required scripts

  • scripts/crawl_website.py

    • Purpose: Crawl official website documentation and convert it to Markdown
    • Parameters: url (required), depth, output, type
    • Enhanced features: path filtering, version detection, metadata injection
  • scripts/summarize_pages.py

    • Purpose: Generate content summaries and core capability maps
    • Parameters: input (required), output, type
    • Output: Summary report in JSON format
  • scripts/validate_skill.py

    • Purpose: Validate skill completeness and usability
    • Parameters: skill-path (required), output
    • Output: Validation report in JSON format

Domain reference

  • references/common-doc-patterns.md
    • When to read: When analyzing the structure of technical documentation
    • Content: Organizational patterns of common technology stack documentation

Skill templates

  • references/templates/language-skill.md

    • Applicable scenarios: Programming language skills (Go, Python, Rust)
    • Content: Basic syntax, data structures, concurrent programming, Web development
  • references/templates/framework-skill.md

    • Applicable scenarios: Frontend framework skills (Vue, React, Angular)
    • Content: Core concepts, component development, state management, route configuration
  • references/templates/tool-skill.md

    • Applicable scenarios: Tool skills (Webpack, Docker, ESLint)
    • Content: Configuration management, plugin system, performance optimization, project integration

Notes

Crawling strategy optimization

  • Automatically filter paths based on the documentation type to avoid crawling irrelevant pages
  • Control the crawling depth and number of pages to avoid putting pressure on the target website
  • Comply with the website's robots.txt rules

Content distillation

  • The agent should prioritize reading the summary report to quickly understand the documentation structure
  • The core capability map helps design the learning path for sub-skills
  • Focus on core documentation such as tutorials, API references, and best practices

Skill templating

  • Automatically select the appropriate template based on the documentation type
  • Dynamically populate metadata (source URL, version number, crawl time)
  • Maintain the consistency and standardization of templates

Validation mechanism

  • It is recommended to run the validation script before packaging
  • Check whether all validation items pass
  • Fix issues based on the validation report

Agent Responsibility Boundaries

  • ✅ Agent is responsible for: content analysis, template selection, document generation, example creation
  • ✅ Agent is responsible for: designing sub-skill structure based on the summary report
  • ❌ Agent is not responsible for: HTML parsing, link discovery, file format conversion
  • ❌ Agent is not responsible for: metadata extraction (done by script)

Usage Examples

Example 1: Generate a Vue learning skill (framework type)

# 1. 抓取 Vue 官网
python scripts/crawl_website.py \
  --url "https://vuejs.org" \
  --depth 2 \
  --output "./vue-skills/raw_docs" \
  --type framework

# 2. 生成摘要
python scripts/summarize_pages.py \
  --input "./vue-skills/raw_docs" \
  --output "./vue-skills/summary.json" \
  --type framework

# 3. 智能体选择 framework-skill.md 模板并填充内容
# 4. 生成参考文档和代码模板
# 5. 验证技能
python scripts/validate_skill.py \
  --skill-path "./vue-skills" \
  --output "./vue-skills/validation.json"

# 6. 打包交付
package_skill(skill_dir_name="vue-skills")

Example 2: Generate a Go language skill (language type)

# 1. 抓取 Go 官网
python scripts/crawl_website.py \
  --url "https://go.dev/doc" \
  --depth 3 \
  --max-pages 100 \
  --output "./go-skills/raw_docs" \
  --type language

# 2. 生成摘要
python scripts/summarize_pages.py \
  --input "./go-skills/raw_docs" \
  --output "./go-skills/summary.json" \
  --type language

# 3. 智能体选择 language-skill.md 模板并填充内容
# 4. 重点关注并发编程、Web 开发等内容
# 5. 打包交付

Example 3: Generate a Webpack skill (tool type)

# 1. 抓取 Webpack 官网
python scripts/crawl_website.py \
  --url "https://webpack.js.org" \
  --depth 2 \
  --output "./webpack-skills/raw_docs" \
  --type tool

# 2. 生成摘要
python scripts/summarize_pages.py \
  --input "./webpack-skills/raw_docs" \
  --output "./webpack-skills/summary.json" \
  --type tool

# 3. 智能体选择 tool-skill.md 模板并填充内容
# 4. 重点关注配置管理、插件系统、优化技巧
# 5. 打包交付

Technical Architecture Description

Crawling Strategy Optimization

  • Path Filtering: Preset different path filtering rules based on document type (language/framework/tool)
  • Version Detection: Automatically extract the version number from the homepage HTML
  • Metadata Injection: Inject the source URL, crawl time, and version number into the header of each Markdown file

Content Distillation

  • Summary Generation: Use a script to collect basic page information (word count, document type)
  • Document Structure Analysis: Identify different types of documents such as APIs, guides, and examples
  • Core Capability Graph: Extract potential concepts based on titles and content (basic version)

Skill Templating

  • Three Templates: Language type, framework type, and tool type, covering common technology stacks
  • Dynamic Filling: Automatically fill in metadata and core content based on the summary report
  • Consistency Assurance: Ensure that the generated sub-skills conform to a unified format specification

Verification Mechanism

  • Metadata Check: Verify the completeness of the YAML front matter
  • Structure Check: Verify that the directory structure conforms to the specification
  • Content Quality Check: Verify description length, line count limits, etc.
  • Test Question Generation: Automatically generate test questions to verify skill usability