maker-checker

Source Metadata

Original cangjie-skill frontmatter from the distillation run:

name: maker-checker
description: |
  设计 Loop 中的 Maker-Checker 模式: 用独立 agent 审查产出,避免自产自检。
  当用户发现 loop 产出质量不稳定、agent 自我评估过于宽容、或需要提升产出可信度时使用。
  不适用于: 产出可以客观验证 (测试通过/失败) 的任务、或成本极其敏感的场景。
  关键 trigger: "agent 自评不准确"、"产出质量不稳定"、"怎么让 agent 审查 agent"。
source_book: "Loop Engineering 视频合集"
source_chapter: 视频2 (Boris Cherny) / 视频3 (小木头)
tags: [maker-checker, verification, quality, separation-of-concerns]
related_skills: [goal-verification, loop-5plus1-architecture, loop-three-elements]

Maker-Checker pattern — use an independent Agent to review output

R — Reading (original text)

"Don't get agent to self-verify its own work. It just generally didn't work that well." — Boris Cherny (Video 2)

"The model that writes the code grades its own homework, and sometimes it is too lenient. So we need another agent to find faults." — Xiaomutou (Video 3)

I — Interpretation (self-description)

Core rule: do not let the same agent both do the work and check it.

The Maker-Checker pattern splits production and review into two independent agents:

Maker: executes the task (writing code, writing articles, generating plans)
Checker: reviews the output and gives feedback or a score

Why it is needed:

The same agent grading its own homework is usually too lenient
Agents have blind spots and cannot see their own mistakes
An independent checker can give a more objective evaluation

A1 — Past Application (cases in the book)

Case 1: Boris's code review (Video 2)

Maker: the agent that writes code -Checker: a read-only verifier agent with a detailed spec
Result: self-review often lets problems through, while an independent checker can find more defects

Case 2: thumbnail scoring (Video 1)

Maker: generates 10 thumbnail concepts
Checker: scores them using a Mr. Beast-style rubric
Lesson: if the maker and checker are the same agent, scoring will be too subjective

A2 — Future Trigger (future triggers)

When agent self-evaluation is inaccurate: "It says it did a good job, but it is actually very poor"
When output quality is unstable: "Sometimes good, sometimes bad, with no guarantee"
When designing the verification step of a loop: choose maker-checker as the verification strategy
When automating team code review: use an AI checker to replace part of manual review

Language signals: "agent self-evaluation is inaccurate", "how to make an agent review an agent", "output quality is unstable", "independent review"

Differences from adjacent skills:

goal-verification: focuses on "what to verify" (standard design); this skill focuses on "who verifies" (role division)
loop-5plus1-architecture: complete system architecture (this skill is a specific pattern at the sub-agent layer)
loop-three-elements: three-element framework (this skill is quality assurance for the action step)

E — Execution (executable steps)

Step 1: Determine whether Maker-Checker is needed

Can the output be objectively verified (tests/numbers)? → Not needed; use automated checks
Does the output require subjective judgment (writing/design)? → Maker-Checker is needed
Extremely cost-sensitive? → Be cautious, because one more agent doubles the cost

Step 2: Design the Maker

Clarify the task: what to do? What to output?
Provide context: rules, constraints, reference cases
Output format: ensure the checker can review it clearly

Step 3: Design the Checker

Must be read-only: cannot modify the output, can only evaluate
Provide a detailed spec: scoring criteria, common mistakes, quality rubric
Output: issue list + score + improvement suggestions

Step 4: Integrate into the Loop

Maker produces → Checker reviews → Pass? Submit / Fail? Feedback to Maker to redo
Set a maximum number of redos (to prevent infinite loops)

B — Boundary

Scenarios where this skill should not be used:

Output can be objectively verified: tests pass/fail → no need for an independent checker
Cost-sensitive: one more agent doubles token consumption
Simple tasks: if a single prompt can handle it, there is no need to divide the work

The author's blind spots and limitations:

Maker-Checker assumes "two agents are better than one agent," but does not provide a solution for "what if the checker is also unreliable"
The cost of an independent checker (tokens, time) is underestimated
The video cases are all code review; applicability to non-technical scenarios (content creation) is unknown

Adjacent but easily confused methodologies:

Code Review: human review of code; this skill is AI reviewing AI
A/B testing: "comparing two versions"; this skill is "division of labor between production and review"