goal-verification

Source Metadata

Original cangjie-skill frontmatter from the distillation run:

name: goal-verification
description: |
  设计 Loop 的 Goal 和 Verification 环节: 将模糊目标转化为可验证的停止条件。
  当用户发现 loop 产出质量不稳定、循环无法停止、或"不知道什么时候算做完"时使用。
  不适用于: 目标已经非常清晰可量的任务、或非循环场景。
  关键 trigger: "loop 停不下来"、"产出质量不稳定"、"怎么定义完成"、"goal 怎么写"。
source_book: "Loop Engineering 视频合集"
source_chapter: 视频1 (Adam Gillock) / 视频4 (Idoos Money)
tags: [goal-design, verification, stop-condition, quality]
related_skills: [loop-three-elements, maker-checker, loop-build-path]

Goal verifiability — the quality leverage point of loop design

R — Reading (original text)

"There's really two most important pillars: the goal (objective, not subjective) and then verification. How does the agent know what that stop condition is?" — Adam Gillock (Video 1)

"A loop is only going to be as good as its done check, as the done criteria." — Adam Gillock (Video 1)

I — Interpretation (in my own words)

The quality ceiling of a loop = the quality ceiling of its verification step. When designing a Loop, two questions must be answered:

1. Goal: What is the loop to achieve? It must beobjectively verifiable— not "do it well," but "reach metric X."
2. Verification: How to determine the goal has been achieved? There must beexecutable check steps。

Good Goal vs Bad Goal:

• ❌ "until you're satisfied" — subjective, not verifiable
• ❌ "do this feature well" — vague, cannot be judged
• ✅ "collect 5 entries of data" — objective, countable
• ✅ "average score ≥ 9 or at most 8 rounds" — objective + hard stop

Verification method spectrum (from objective to subjective):

• Purely objective: run the test suite, numerical metrics
• Semi-objective: another LLM judges
• Subjective: human-in-the-loop judgment
• Vague: cannot clearly judge good from bad

A1 — Past Application (cases in the book)

Case 1: Bad Goal — thumbnail generation (Video 1)

• Goal: "iterate until satisfied" → subjective, leading to an uncontrollable 27 minutes
• Lesson: it should be changed to "score ≥ 8/10 or at most 5 rounds"

Case 2: Good Goal — Abbey Road recreation (Video 1)

• Goal: "average score ≥ 9 or at most 8 rounds" → objective + hard stop
• Although the result was not perfect, the loop stopped within expectations

Case 3: Applying the verification spectrum (Video 4)

• Test suite → most objective, use first
• LLM judge → second choice, needs an independent checker
• Human judgment → last resort, high cost

A2 — Future Trigger (future triggers)

1. When the Loop cannot stop: "ran for 3 days and still not done" → check whether the stop condition is too subjective
2. When output quality is unstable: "sometimes good, sometimes bad" → the verification step may be unreliable
3. When designing a new Loop: first write the goal and verification, then write the action
4. When debugging a Loop: diagnosing "why the output is poor" → first check whether the goal is verifiable

Language signals: "the loop can't stop," "how to define done," "how to write the goal," "output quality is unstable"

Differences from adjacent skills:

• loop-three-elements: the overall framework of the three elements (this skill focuses on designing the stop condition)
• maker-checker: the concrete implementation of the verification step (this skill is the design principle of verification)
• loop-worthiness-test: Decide whether a loop is needed (this skill is about how to design it once decided)

E — Execution (executable steps)

Step 1: Turn a vague goal into a verifiable goal

When facing a goal like "do X well", ask:

• What is the concrete manifestation of "doing it well"?
• Can it be judged with a single number/boolean value?
• If not, can it be broken down into multiple verifiable sub-goals?

Step 2: Choose the verification method

Try in order from most to least objective:

1. Purely objective check: Tests pass? Values meet the target? File exists?
2. LLM judgment: Use an independent agent to score according to a rubric
3. Human judgment: The last resort when none of the above are feasible

Step 3: Set a Hard Stop

Even if verification is subjective, a hard limit must still be set:

• Maximum number of iterations (e.g., 8 rounds)
• Maximum runtime (e.g., 30 minutes)
• Maximum token consumption

Step 4: Independent check of the verification step

• Do not let the same agent produce and check its own work (see maker-checker)
• The verification agent should be read-only and have a clear spec

B — Boundary (boundaries)

Scenarios where this skill should not be used:

1. The goal is already verifiable: "Collect 5 items" does not need further design
2. Purely creative tasks: Poetry, art, etc. cannot be objectively verified; this skill does not apply
3. Non-loop scenarios: Goal design for a single task does not need to consider stop conditions

The author's blind spots and limitations:

• "Goals must be objective" is hard to achieve in creative tasks, but the author does not provide an alternative for creative tasks
• The verification spectrum assumes "the more objective, the better", but sometimes subjective judgment (human aesthetics) is precisely the goal
• The video cases are all technical tasks, with insufficient coverage of verification design for scenarios such as content creation and research

Adjacent but easily confused methodologies:

• SMART goals: A general goal-setting framework; this skill focuses on "verifiable stop conditions" in loop scenarios
• Test-driven development (TDD): "Write tests before writing code"; this skill's idea of "design verification before designing actions" is similar