A skill that catalogs all screenshots in the course-bundle and performs classification, OCR, and metadata assignment.
Recommended execution model & parallelization
Model: sonnet Required (opus costs 3-5x more. The accuracy difference is small)
Reading all frames in a single execution is the most costly. Strictly use sonnet in parallel.
Parallelization: Split chapters into N parts and launch N Agents in parallel
| Number of chapters | Recommended parallelism | Chapters per Agent |
|---|---|---|
| ~12 chapters | 2 Agent | 6 chapters each |
| ~24 chapters | 4 Agent | 6 chapters each |
| ~36 chapters | 4-6 Agent | 6-9 chapters each |
| 36 chapters+ | 6 Agent | 6-10 chapters each |
Agent launch template:
Agent(
model: "sonnet",
mode: "bypassPermissions",
run_in_background: true,
prompt: "Process chapters NN-NN of [bundle_path]..."
)
Note: visual-indexer reads all images, so it is the most cost-heavy.Always use sonnet + parallel. Running all chapters sequentially with opus is forbidden.
WHAT — What it does
Reads each screenshot image contained in every chapter of the course one by one, and performs the following:
- Frame classification — Classify images using the 7 core types + course-specific types (custom_types)
- OCR extraction — Extract all text within the image
- Identification of visual elements — Record visual elements such as shapes, arrows, boxes, etc.
- Keyword assignment — Tag so that downstream skills can search easily
- Index generation — By chapter
visual-index.json+ overallvisual-catalog.json
WHY — Why it is necessary
- procedure-extractor needs OCR information and frame classification to link procedure images
- concept-extractor uses slide images as supporting material
- Reduce downstream processing costs by excluding talking_head and transition
- A catalog is needed to get an overview of the image assets of the entire course
HOW — Execution procedure
Phase 0: Preparation & verification
BLOCKER: manifest.jsonが存在するか確認
→ 存在しない場合は course-ingest を先に実行するよう案内して停止
manifest.jsonand obtain the chapter list and screenshot directories- Check the existence of the screenshot directory for each chapter
- Count all frames and create a processing plan
Phase 0.5: Discovery of course-specific types
Sample-read the first 5 frames of each chapter:
- Determine whether they fit the core types
- If there are 2 or more screens that do not fit → propose course-specific types
- Present the type definitions to the user or main session (only at the beginning of the chapter)
- Once confirmed,
visual-index.jsonのcustom_typesregister in - For subsequent frames, select from core types + course-specific types
Note: For full automation, the Agent itself decides and registers in Phase 0.5.
Phase 1: Chapter-by-chapter frame processing
For each chapter, execute the following:
Step 1.1: Retrieve frame list
ls chapters/{chapter_id}/screenshots/
Step 1.2: Batch processing (5-10 frames at a time)
BLOCKER: 各画像は必ずReadツールで読み取ってから分類する
→ ファイル名からの推測は NG
For each frame:
- Read the image with the Read tool
- Frame classification — classify with the following taxonomy:
Core types(common to all courses):
| type | Description | How it is used downstream |
|---|---|---|
title_slide |
Chapter title | Manual heading image |
content_slide |
Concept diagrams/slides | Reinforcement for concept-extractor |
browser_screenshot |
Browser screen | Procedure images for procedure-extractor |
terminal_screenshot |
Command line | Procedure images for procedure-extractor |
diagram |
Flowcharts/diagrams | Visualization of conceptual relationships |
talking_head |
Speaker video only | Low value, can skip |
transition |
Blank/transition frames | Skip |
Repeated screens that do not fit the core types should be course-specific types (custom_types) (described later).
Course-specific types (custom_types)
Screens that do not fit the core types but appear repeatedly within a course should becourse-specific typesand created dynamically.
Creation rules
-
First 3-5 framesto consider type candidates
-
"Does not fit the core types" and "appears 2 or more times" → register as a course-specific type
-
Name in snake_case, including keywords from the course
- Example:
tmux_pane,vscode_editor,ae_timeline,figma_canvas
- Example:
-
Per courseUp to 5 types(to prevent infinite proliferation)
-
Do not create types that appear only once; assign them to the closest core type
-
OCR extraction — store all text in the image in the
text_contentarray -
Visual element identification —
visual_elementsrecord in the array -
Keyword assignment — store keywords representing the content in the
keywordsarray -
Confidence determination —
confidenceをhigh/medium/lowset with -
UI operation determination — for browser/terminal screens, set
is_ui_walkthroughset
Step 1.3: Generate visual-index.json
Once all frames in the chapter have been processed, visual-index.json generate:
{
"chapter_id": "03",
"custom_types": {
"tmux_pane": "tmuxで複数paneに分割されたClaude Code画面(タイトルバーにtmux表記、画面分割あり)"
},
"frames": [
{
"filename": "frame_005.jpg",
"type": "content_slide",
"text_content": ["AIマインド", "壁打ち", "アイデア", "専門知識"],
"visual_elements": ["equation_diagram", "four_boxes", "arrow"],
"keywords": ["4原則", "マインド"],
"confidence": "high"
}
]
}
Output destination: chapters/{chapter_id}/visual-index.json
Schema: schemas/visual-index.schema.json Must conform to.
Phase 2: Generate visual-catalog.json
After processing all chapters, generate the catalog for the entire course:
{
"course_name": "YouTube Booster",
"total_frames": 245,
"course_custom_types": {
"tmux_pane": {"definition": "tmuxで複数paneに分割されたClaude Code画面", "frame_count": 18},
"vscode_editor": {"definition": "VS Codeのエディタ画面(ファイルツリー+エディタ+ステータスバー)", "frame_count": 8}
},
"type_distribution": {
"content_slide": 89,
"browser_screenshot": 45,
"talking_head": 67,
"diagram": 23,
"title_slide": 12,
"transition": 9
},
"chapters": [
{
"chapter_id": "03",
"frame_count": 31,
"high_value_frames": ["frame_005.jpg", "frame_012.jpg"],
"visual_index_path": "chapters/03/visual-index.json"
}
]
}
high_value_frames:content_slide,browser_screenshot,diagramamongconfidence: "high"withtype_distribution: Aggregate the frame types of all chapters
Output destination: project root's visual-catalog.json
Phase 3: Self-verification
All of the following checklist items must pass:
- All frames have
typeis set -
text_contentwith emptycontent_slideis missing (slides must always have text) -
talking_headframe count is reasonable (usually 20-40%) -
visual-catalog.jsonのtotal_framesmatches the actual number of files - Each
visual-index.jsonがschemas/visual-index.schema.jsoncomplies with -
type_distributiontotal matchestotal_framesmatches -
custom_typescount is 5 or fewer - Each
custom_typehas a definition statement - Each
custom_typehas 2 or more frames using it -
custom_typenaming is snake_case
検証失敗時:
→ 不一致があれば該当チャプターを再処理
→ text_contentが空のcontent_slideがあれば画像を再読み取り
BLOCKER gate
| Gate | Condition | Action when not met |
|---|---|---|
| G1 | manifest.json exists | Guide the user to run course-ingest first |
| G2 | Screenshot directory exists | Guide the user to run video-to-frames first |
| G3 | Each image was actually read with the Read tool | Guessing from filenames is prohibited; always read them |
| G4 | All frames were processed (all of them, not a sample) | If there are unprocessed frames, process them additionally |
| G5 | All self-verification checklist items pass | Fix failed items and re-verify |
NG / OK judgment table
| NG | OK | Reason |
|---|---|---|
| Classify based only on filenames | Read images with the Read tool before classifying | Filenames contain no content information |
| Miss talking_head and classify everything as content_slide | Frames with only a person are clearly classified as talking_head | Later skills end up processing unnecessary images |
| Leave text_content empty | Extract all text in the image | OCR information is needed for procedure-extractor linking |
| Process only a sample of frames | Process all frames without omission | If an omitted frame was a procedure image, the procedure is missing |
| Do not distinguish between content_slide and diagram | Clearly separate slide format from flowchart format | concept-extractor's processing method differs |
| Set confidence to high for everything | Set images you are unsure about to medium/low | Information is needed to prioritize in later stages |
| Omit visual_elements | Record arrows, boxes, tables, code, etc. | Affects downstream image selection accuracy |
| Forcing screens that don't fit into core types | Create course-specific types | Misclassification breaks downstream skills |
| Create a type that appears only once | Assign to the closest core type | Prevent infinite proliferation of types |
| Create more than 5 course-specific types in one course | Consolidate into the top 5 | Prevent catalog bloat |
Course-specific type naming uses generic terms (editoretc.) |
Contains course keywords (vscode_editoretc.) |
Does not conflict with other courses |
Frame classification criteria
For details, see references/frame-types.md Refer to. Overview:
- title_slide: Large text, chapter numbers, background design. Few text elements
- content_slide: Bullet points, diagrams, concept explanations. Large amount of text
- browser_screenshot: Browser UI elements (address bar, tabs, etc.) are visible
- terminal_screenshot: Black background with monospace font, command line
- diagram: Arrows, flows, connecting lines. Arrangement of structured shapes
- talking_head: Person's video occupies most of the screen. No/few text elements
- transition: Blank screen, fade, blur
Edge cases
- Small speaker video overlaid on slide →
content_slide(slide is primary) - Browser screen + commentary text overlay →
browser_screenshot - Half slide, half speaker → judge by text amount; if there is a lot of text,
content_slide
Output file list
| File | Location | Content |
|---|---|---|
| visual-index.json | chapters/{id}/visual-index.json | Frame information by chapter |
| visual-catalog.json | Project root | Catalog for the entire course |
Linked skills
| Skill | Relationship |
|---|---|
| course-ingest | Prerequisite: generate manifest.json |
| video-to-frames | Prerequisite: generate screenshots |
| procedure-extractor | Subsequent: use frame classification and OCR |
| concept-extractor | Can run in parallel: use content_slide for reinforcement |
Batch processing notes
- When reading images with the Read tool, process 5-10 images per batch
- Process all frames even if there are a large number (over 100)
- Report processing progress by chapter:
Chapter 03: 31/31 frames processed - Frames where an error occurred are recorded in
confidence: "low"and note the error details