SuperAPI AI Directory

concept-extractor

General & other

View repository

Updated 10-15 21:03

About this skill

Concept Extractor

Recommended execution model & parallelization

Model: sonnet Required (opus costs 3-5x more. The accuracy difference is small)

Parallelization: Split chapters into N parts and launch N Agents in parallel

Number of chapters Recommended parallelism Chapters per Agent
~12 chapters 2 Agent 6 chapters each
~24 chapters 4 Agent 6 chapters each
~36 chapters 4-6 Agent 6-9 chapters each
36 chapters+ 6 Agent 6-10 chapters each

Agent launch template:

Agent(
  model: "sonnet",
  mode: "bypassPermissions",
  run_in_background: true,
  prompt: "Process chapters NN-NN of [bundle_path]..."
)

Note: visual-indexer is the most cost-heavy because it Reads all images.Always use sonnet + parallelization. Running all chapters sequentially with opus is forbidden.


WHAT

Using the lecture course-bundle (manifest.json + each chapter's manual.md/transcript.txt) as input, extract structured knowledge in 6 categories.

Output files:

File Granularity Content
knowledge/chXX-knowledge.json Chapter unit concepts, key_quotes, tacit_knowledge, teaching_patterns, cross_references, resource_references
knowledge/knowledge-graph.json Entire course Prerequisite relationship graph, concept hierarchy, cross-cutting themes

Out of scope:

  • Analysis of images and slides (responsibility of visual-indexer)
  • Extraction of procedures and workflows (responsibility of procedure-extractor)
  • Creation and formatting of transcripts (responsibility of course-ingest)

WHY

Transcripts are long and unstructured. As they are, they cannot be used for RAG, and are difficult to utilize for manual generation. By structuring concepts, definitions, and tacit knowledge:

  1. The manual generation skill can cite accurate definitions
  2. The learning order can be optimized using the prerequisite relationship graph
  3. The speaker's tacit knowledge is made explicit and can be incorporated into the AI agent's knowledge base

HOW

Input requirements

Required Content
manifest.json Course configuration file generated by course-ingest
chapters/chXX/manual.md or transcript.txt Source text for each chapter (see priority order below)

If manifest.json is not found, confirm with the user and suggest running course-ingest.

Input priority order

For each chapter, select the source according to the following priority order:

Priority Source Reason
1 chapters/{id}/manual.md utage-manual output. Edited, structured, and images linked
2 chapters/{id}/page_text.md Description text of the lecture page. Includes supplementary information
3 chapters/{id}/transcript.txt Raw transcript. Fallback when there is no manual

If a manual exists, use the manual as the primary source, and use the transcript to supplement information not included in the manual.

BLOCKER: Reading the full source text

Always read the entire manual.md (or transcript.txt) for each chapter before starting extraction. Do not start by reading only the first 50 lines. When using the Read tool with offset/limit, always read through to the end in multiple passes.


Six extraction categories

Category What to extract Example
concept Name + definition + concrete example "AI Mindset = the mindset of removing AI blame. Do not blame AI; take responsibility for your own instruction ability."
key_quote Important statements by the speaker, verbatim "The moment you blame AI, you can no longer create anything."
tacit_knowledge Judgment criteria the speaker implicitly holds "The deeper a person's expertise, the better they can use AI. AI is a mirror."
teaching_pattern The speaker's teaching pattern "Present pain → identify cause → present principle → concrete example → re-emphasize mindset"
cross_reference References to other chapters and prerequisite relationships "The magic script prompt in Chapter 9 presupposes these four principles."
resource_reference Mentions of external resources "I store prompts in Notion." "Please download the PDF."

If you are unsure how to classify a category, refer to references/extraction-taxonomy.md .

resource_reference detection patterns

Detect the following patterns in transcripts and manuals:

  • "In Notion ~" "The link is ~" "In the description ~" "Download ~"
  • "Stored here" "template" "gift" "distributed materials"
  • URL mentions (notion.so, drive.google.com, etc.)

Output format in knowledge.json:

"resource_references": [
  {
    "id": "rr03-01",
    "text": "Notionの方にプロンプト格納してます",
    "type": "notion_page",
    "context": "PredictionXプロンプトの保管場所",
    "url_if_detected": null
  }
]

Processing steps

Step 1: manifest.json読み込み → 章リスト取得
Step 2: 各章のソース選択(manual.md優先 → transcript.txtフォールバック)+全文読み込み ← BLOCKER
Step 3: 6カテゴリ抽出(章ごと)
Step 4: chXX-knowledge.json生成(章ごと)
Step 5: 全章処理完了後 → knowledge-graph.json生成
Step 6: 自己検証

Step 1: Read manifest.json

Get the course name and chapter list from manifest.json.

Step 2-4: Processing per chapter

For each chapter, do the following:

  1. Source selection: If manual.md exists, use it as the primary source. If not, fall back to transcript.txt
  2. Read the full primary source(BLOCKER: partial reading is NG). If a manual exists, refer to transcript.txt as supplementary information when necessary
  3. concept extraction: Identify concepts that the speaker names and explains. Record name, definition, examples, key_quotes, speaker_framing, prerequisites, related_concepts
  4. key_quote extraction: Record strong claims, memorable expressions, and core statements verbatim. Never summarize or paraphrase
  5. tacit_knowledge extraction: Identify judgment criteria and values that the speaker does not explicitly define but implicitly assumes. Always attach a quote as evidence
  6. teaching_pattern extraction: Identify the structural pattern of the speaker's explanations (e.g., pain → cause → principle → example → re-emphasis)
  7. cross_reference extraction: Record references to other chapters and prerequisite relationships
  8. resource_reference extraction: Detect and record mentions of external resources (Notion, PDF, templates, download links, etc.)
  9. chXX-knowledge.json output

The schema is schemas/knowledge.schema.json Must comply with.

Step 5: knowledge-graph.json generation

BLOCKER: Execute only after knowledge.json generation for all chapters is complete. Do not create the overall graph by looking at just one chapter.

Integrate the knowledge.json of all chapters:

  1. prerequisite_graph: prerequisite relationships between concepts (built from cross_reference and prerequisites)
  2. concept_hierarchy: category classification of concepts
  3. cross_chapter_themes: themes that span multiple chapters

The schema is schemas/knowledge-graph.schema.json Must comply with.

Step 6: Self-verification

Check all of the following, and fix any problems:

  • Does each concept have name, definition, and examples?
  • Are key_quotes verbatim from the source (not summarized)?
  • Do the evidence entries of tacit_knowledge contain quotes from the transcript?
  • Are concept IDs unique across all chapters (including the chapter number, like c03-01, c04-01)?
  • Does cross_reference refer to chapters that actually exist?
  • Does the prerequisite_graph of knowledge-graph.json reference only concept IDs that actually exist?
  • Does the total_concepts value match the actual number of concepts?
  • In chapters that have a manual, was the manual used as the primary source?
  • Were mentions of resources (Notion, PDF, templates, etc.) detected?

NG / OK table

NG OK Reason
"An important concept is stated" Write the name + definition as "AI mindset = de-AI-accountability thinking" Abstract descriptions cannot be utilized
Summarize or paraphrase the key_quote Record the speaker's exact words For quotes, the original text is what has value
Read only one chapter and create the overall graph Create the graph after processing all chapters Prerequisite relationships cannot be understood without looking at the whole
Read only the first 50 lines of transcript.txt Start extraction only after reading the entire text Important concepts are often in the latter half
Make examples an empty array Record at least one concrete example given by the speaker A definition alone does not convey the concept
tacit_knowledge without evidence Quote the relevant passage from the transcript Speculation without evidence cannot be trusted
Number concept IDs sequentially as "c1", "c2" Include the chapter number, like "c03-01", "c04-01" To guarantee uniqueness across all chapters
Write definitions longer than 3 sentences Describe the core concisely in 1-2 sentences Definitions should be short and clear
Extract from the transcript even though a manual exists Use the manual as the primary source The manual is edited and high quality
Ignore resource mentions Record as resource_references Serves as a clue for resource acquisition when turning into a skill

Output directory structure

{course-bundle}/
├── knowledge/
│   ├── ch00-knowledge.json
│   ├── ch01-knowledge.json
│   ├── ...
│   └── knowledge-graph.json

Presentation to the user

After processing is complete, present the following summary:

## Concept Extraction Complete

- コース: {course_name}
- 処理章数: {n}章
- 抽出概念数: {total_concepts}
- 抽出引用数: {total_quotes}
- 暗黙知: {total_tacit}件
- 教え方パターン: {total_patterns}件
- リソース言及: {total_resources}件
- 横断テーマ: {total_themes}件

### 主要概念(上位5件)
1. {concept_name} - {definition}
...

### 横断テーマ
- {theme}: 章{chapters}
...