Produce a graded exercise ladder that verifies understanding of a specific topic, concept, method, derivation, or paper. The output is two separate files so the learner attempts the problems first and only then checks solutions — peeking is the failure mode this format is built to prevent.
This skill makes the assessment, not the lesson. It is the step flow-learn-new-topic and flow-deep-understanding route to at an exit gate to confirm the learner actually passed, instead of asserting they did.
When to use / not use
- Use when the learner has covered material and wants to test retention, mechanism, reasoning, or generative mastery.
- Use as the gate-check inside a learning flow.
- Don't use to teach the concept for the first time — that is
professor-mentor-technical-teaching. - Don't use to design research experiments or benchmarks — that is
experiment-design/benchmark-and-baseline-selector.
Before You Generate: Ground the Exercises
Blind exercises test a hallucinated version of the topic. Gather, or confirm you already have:
- The concept(s) to test, stated concretely (not "transformers" but "scaled dot-product attention and why it's scaled").
- The source material the learner actually studied — the paper, the lecture, the teaching output, the toy cases. Exercises must be answerable from that material, not from a generic textbook the learner never read.
- Current level — novice, can-read-papers, or can-build. This sets where the ladder centers.
- Depth target & budget — a 15-minute self-check is ~4 problems; a full mastery pass is 8–12 across all tiers.
If the source material is missing, ask for it or pull it from the upstream teaching output before generating. Do not invent facts to build a problem around.
The Difficulty Ladder
Four tiers, easy → advanced. Full detail, verbs, and per-domain templates are in references/difficulty-ladder.md and references/exercise-templates.md. For multiple-choice distractors and intuitive-trap problems, use references/assessment-design-principles.md. The short form:
| Tier | Name | Tests | Example verbs |
|---|---|---|---|
| 1 | Recall & Recognize | vocabulary, what-is, which-family | define, identify, state, match |
| 2 | Apply & Compute | mechanism on a toy case | compute, trace, predict, plug in |
| 3 | Analyze & Compare | why, when it breaks, trade-offs | contrast, diagnose, justify, choose |
| 4 | Build, Derive & Break | generative mastery | derive, construct a counterexample, critique, design a variant |
A learner who only clears Tiers 1–2 can recite and run the method but does not yet understand it; Tiers 3–4 are where real comprehension shows. Always include at least one Tier 3+ problem, even in a short set — that is the one that actually verifies understanding.
Process
- Scope — confirm concept(s), source material, level, and budget (above).
- Pick the spread — choose how many problems per tier from the budget and depth target. Defaults in
references/difficulty-ladder.md. Center the ladder on the learner's level but always reach one tier above it. - Write the problems — one concept per problem, unambiguous, answerable from the source material. Tag each with
[Tier N · concept]. Use the per-domain templates (conceptual, mathematical/derivation, coding, analysis/critique) inreferences/exercise-templates.md. If you use multiple choice, every distractor must encode a known misconception and diagnose which wrong model the learner holds; never use random wrong answers. When the topic has an attractive intuitive-but-wrong shortcut, include one CRT-style verification problem that forces deliberate checking. - Write the solutions separately — for each problem: the answer, the worked reasoning, a self-check rubric ("you've got it if…", "common wrong turns"), and a revisit pointer naming what to restudy on a miss. Format in
references/solution-and-rubric-format.md. - Emit two files — never interleave problems and solutions:
exercises.md— problems only, numbered, tier-tagged, with an instruction header.solutions.md— matching numbers, full solutions + rubric + revisit pointers.
- State the self-verification protocol to the learner (below).
Output: Two Files
exercises.md opens with a short header telling the learner how to use it:
# Exercises — <topic>
Attempt every problem before opening solutions.md. Write your answer down; a vague
mental "yeah I know this" is the thing these problems exist to catch. Then score
yourself against the rubric in solutions.md.
1. [Tier 1 · <concept>] <problem>
2. [Tier 2 · <concept>] <problem>
...
solutions.md mirrors the numbering exactly:
# Solutions — <topic>
1. [Tier 1 · <concept>]
Answer: ...
Why: ...
✅ You've got it if: ...
⚠️ Common wrong turn: ...
↩ Revisit if missed: <concept / section / stage to restudy>
Self-Verification Protocol (give this to the learner)
- Attempt every problem and write the answer down before opening
solutions.md. - Score each against its rubric — honestly. A near-miss on a Tier 3–4 problem is a miss.
- Map every miss to its revisit pointer; restudy those, don't just read the solution.
- Gate rule: understanding is verified only when Tier 3+ problems pass unaided. Clearing Tiers 1–2 alone means re-study, not progress.
Handoffs
- Learner missed problems and needs re-teaching →
professor-mentor-technical-teaching. - Misses cluster on computing minimal cases →
theory-to-toy-cases. - Tier-4 critique/derivation exposed a real gap in the source itself →
gap-finderortheorem-and-claim-audit. - Called from a learning flow → return the pass/fail-by-tier result to the orchestrator so it can decide the next stage.