knowledge-compiler

knowledge-compiler

Compiles a source into a personal working skill: not a retelling of the book, but a map of applicable ideas tied to locations in the source. Suitable not only for technical books, but also for management, marketing, sales, product, learning, internal manuals, reports and any materials with applied value.

Position

1. Personal circuit by default — the skill is assembled for individual work with a lawfully accessible source. Publication modes, legal questionnaires and complex permissions are not part of the main path.
2. Compilation, not republication — the result includes concepts, heuristics, questions, decision rules, playbooks and anti-patterns. Do not copy long quotes, tables, listings and diagrams by default.
3. Source map is mandatory — every significant thesis must have a pointer: segment, chapter/heading, lines or characters, confidence.
4. Output is portable — the finished skill is created in the selected folder, usually dist/<skill-name>/. Installation into a specific environment is a separate step.
5. Cache conserves context — the full text and segments are stored in cache/jobs/<job-id>/; do not output large fragments in the response to the user.

When to run

Run if the user says: "make a skill from a book", "turn PDF/EPUB into a skill", "build a knowledge map", "I want to apply the author's ideas", "make a personal advisor on the material", "extract frameworks from the document".

Do not run for public publication of someone else's commercial book without a separate discussion of rights. Do not bypass DRM, access protection, paywall or technical restrictions of the file.

Quick route

1. Check the environment

   sh scripts/doctor.sh
   

2. Extract the source

   sh scripts/prepare_source.sh --input "/path/to/source.pdf" --mode auto
   

   The script will create cache/jobs/<job-id>/source.txt и metadata.json. For EPUB it reads OPF metadata (title, creator) and a table of contents from toc.ncx/nav.xhtml, even if the file arrived with the extension .zip, but inside it is an EPUB.

3. Reconnoiter the structure

   uv run --script scripts/outline_scout.py \
     --input cache/jobs/<job-id>/source.txt \
     --out cache/jobs/<job-id>/outline-scout.json
   

   Pass to the model or subagent metadata.json, outline-scout.json, outline-head.txt, outline-tail.txt и outline-candidates.txt. Its task is to determine how the table of contents, parts, chapters, appendices, and back matter are designated in the source. Do not send the entire source.txt at this step. If outline-scout.json already contains script_observations.epub_toc_items, consider it the primary candidate for the structure: EPUB itself stores the table of contents more accurately than regexes over the text.

4. Split into segments

   uv run --script scripts/segment_text.py \
     --input cache/jobs/<job-id>/source.txt \
     --out cache/jobs/<job-id>/segments \
     --decision cache/jobs/<job-id>/outline-scout.json
   

   If the solution has toc_items, the segmenter cuts by the table of contents, normalizes non-breaking spaces and multi-line headings, and does not glue short appendices to neighboring segments.

5. Create a draft of the future skill

   sh scripts/build_skill_skeleton.sh \
     --name my-source-skill \
     --title "Название источника" \
     --author "Автор" \
     --job-dir cache/jobs/<job-id> \
     --output-dir dist
   

6. Compile the content Read metadata.json, segments/outline.json and the necessary files from segments/chunks/. Fill in:

   • SKILL.md — a short working protocol, up to 2000 words;
   • references/concepts.md — key concepts;
   • references/decision-rules.md — decisive rules "if X, do Y";
   • references/playbooks.md — applied scenarios;
   • references/anti-patterns.md — mistakes and risk indicators;
   • references/glossary.md — terms;
   • references/source-map.json — connection of theses to segments;
   • references/knowledge-manifest.json — source, hash, extraction method, limitations.

   Follow the convention on claim_id, headings, and links from references/OUTPUT_SCHEMA.md. Do not copy the source text or segments into the finished skill: they remain in the cache.

7. Check quality and create an index of sources

   uv run --script scripts/quality_gate.py \
     --source cache/jobs/<job-id>/source.txt \
     --skill-dir dist/my-source-skill \
     --write-source-index
   

   After successful checks, the command will create references/source-index.md from the filled-in source map. To check without changes, run it without --write-source-index; after changing the map, recreate the index with the same flag.

Build focus

If the user has not clarified the goal, ask in one question: "What will the skill be for: applying ideas at work, learning from the material, quickly handling terminology, or making decisions?" Do not turn this into a complex choice of modes.

Guidelines:

• applied advisor — more playbooks, diagnostic questions, and anti-patterns;
• decision map — more decision-rules.md, forks, and selection criteria;
• reference guide — stronger concepts.md и glossary.md;
• study outline — allows more explanations, but still without replacing the book.

Domains

For technical materials, look for architectural decisions, teams, APIs, constraints, selection criteria, and operational risks. For management — processes, teams, hiring, strategy, decision-making, and managerial traps. For marketing and sales — positioning, research, funnels, channels, copy, offers, and signs of off-target demand. For product and business — customers, metrics, economics, launch, growth, and trade-offs. For courses and manuals — exercises, procedures, verification criteria, and learning paths.

For legal, financial, medical, and other sensitive materials, only make a reference guide to the source and working questions for a specialist; do not issue a professional opinion.

Working with large sources

For long books, break down segments gradually. If the environment allows child sessions, you can parallelize the analysis of independent segments, but do not launch models below the level of the current orchestrator. Give child agents only their segment and require structured output: concepts, rules, playbooks, anti-patterns, source pointers.

What to read as needed

• references/OUTPUT_SCHEMA.md — the exact structure of a finished skill and the fields of the source map.
• references/COMPILATION_PLAYBOOK.md — how to turn text into applicable maps, not into a retelling.
• references/QUALITY_GATE.md — what quality_gate.py checks and how to fix failures.
• references/BACKLOG.md — deferred mechanics: OCR, Go accelerator, publication profile, processing comparison.

Constraints

• Run Python scripts via uv run --script; dependencies are declared directly in the script headers.
• OCR is still in the backlog. Scanned PDFs may produce empty or poor text.
• Similarity checking is currently lexical: word n-grams. Semantic similarity and an assessment of the degree of reworking are better added as a separate task.
• A Go binary is not needed yet: the current operations are run once per source. If frequent repeated runs over large texts appear, the hot path for markup/comparison can be moved to Go.