test-designer

Test Designer / Test Designer

Independent test design orchestrator. Implements the 'independent evaluation' principle: the agent writing the tests must not be the agent implementing the feature, and must not inherit the implementation's assumptions.

When to Use / When to Use

• Forcomplex / non-trivialfeatures in the TDD red phase (multiple files, multi-branch logic, new subsystems)
• Requirements are vague enough that tests written by the implementer are likely to rationalize the implementation rather than catch defects
• The user explicitly requests 'independent test design', 'tests without a predetermined perspective', or runs /test-designer

Not applicable to:

• Trivial changes (one-line fixes, renames) — write tests inline directly
• Defect reproduction tests — write directly from the defect report
• Non-code changes (docs-only, config-only, prompt-only)

The Iron Law / The Iron Law

The agent designing the tests must not carry the context of the feature implementation. If you (the main agent) are about to implement the feature, you have lost the qualification to design its tests. Dispatch it out.

Violating this iron law = tests pass because they mirror a flawed implementation.

Must not (orchestrator scope) / Must not (orchestrator scope)

The iron law above states this principle from the positive side. The following are themain agent invoking this skill's corresponding negative rules — distinct from the 'process constraints' for dispatched agents in step 3.

• If you have already implemented or drafted the feature, you must not invoke this skill. Test design in the same context — whether done by you or by a sub-agent dispatched from this conversation — will inherit the blind spots of the implementation approach you have already discussed. Once implementation context exists, the independent evaluation guarantee is gone; defer test coverage to the PR stage deep-review 's test-quality reviewer to handle.
• Before dispatching, you must not modify requirements or acceptance criteria to fit an already-drafted implementation. The dispatched agent's input must be the original requirements as stated by the user, not a version already mapped to code. If the requirements are ambiguous, ask the user — do not disambiguate through implementation choices and then let that leak into the test design.

Steps / Steps

Step 1: Assemble the dispatch package / Step 1: Assemble the dispatch package

Collect only these inputs— nothing else:

1. Validation contract (primary source) — from spec-design 's docs/specs/<topic>/validation-contract.md. The VAL list (VAL-XXX-NNN: Behavior + Tool + Evidence）is the normative contract. Each VAL maps to one or more failing tests: choose the test level matching the Tool category, and assert against the Evidence definition. The dispatched agent must enumerate failing tests by walking the VAL list item by item, not by improvising.
2. Requirements description (fallback context) — docs/specs/<topic>/spec.md 's Why / Findings / What / Out of scope. Use when a VAL's Behavior is too brief to interpret clearly, or when the VAL list appears incomplete (spec.md § What's described behavior has no corresponding VAL). If the fallback path is triggered frequently, surface it—the specification contract has an under-specification problem, and it should be re-invoked spec-design, rather than covering up the gap with improvised tests.
3. Project testing rules — first check the repository's docs/rules/test/. The directory is anchored at the repository root: use git rev-parse --show-toplevel to resolve the repository root, and search for <仓库根>/docs/rules/test/; when the cwd is not at the repository root, and there is a closer docs/rules/test/(monorepo subpackage) between the cwd and the repository root, collect both levels—subpackage-level rules are treated as supplements/narrowing of repository-level rules, and in case of conflict the subpackage-level rules take precedence; for non-git repositories, fall back to searching relative to the cwd. Collect the rules relevant to the current module, test type, fixture, or runner, and set the rules file'sabsolute pathwrite them into the dispatch packet as well. If the directory does not exist or there are no relevant files, clearly state in the dispatch packet that "there are no project-specific testing rules." The dispatched agent must design tests according to these rules; if the rules conflict with the VAL, report the conflict first and do not make the trade-off on its own.
4. Relevant code file paths — read-only access to the code that the feature will touch or integrate with
5. Boundary scenario hints — when VAL coverage in these dimensions is thin, the categories the dispatched agent should enumerate:
   • Boundary inputs (empty values, maximum, minimum, off-by-one errors)
   • Concurrency / ordering (if applicable)
   • Resource lifecycle (cleanup on error, partial failure)
   • Invariants (data consistency, idempotency)
   • Adversarial inputs (malformed, oversized, encoding errors)

Explicitly excluded:

• The implementation plan or design you have been developing
• Any hints about the approach you chose
• Code excerpts from the in-progress branch
• Your own guesses about "the correct way to test"

VAL → test mapping rule: One VAL can expand into N test assertions (1:N), but not the reverse. Every generated test must be traceable in comments to a specific VAL-XXX-NNN id. Tests that cannot be traced are either: (a) covering entries in Findings/§What that have no corresponding VAL—report as a spec-design gap, or (b) over-extended (testing implementation details)—delete.

Step 2: Choose the executor / Step 2: Choose the executor

Task shape	Executor	Reason
Complex, involves architectural impact	Independent agent (e.g., using a new session's codex-agent or claude-code-agent）	True zero-context isolation; can use the strongest model and highest reasoning effort
Medium complexity, current conversation is clean	In-conversation subagent	More economical; still acceptable if the main agent has not yet proposed an implementation approach
Trivial	Do not dispatch—write the tests inline directly

When the main agent has already discussed or drafted an implementation approach, use an independent agent by default. Subagent isolation within the same conversation cannot eliminate prior context contamination.

Step 3: Dispatch with the strongest model and highest effort / Step 3: Dispatch with the strongest model and highest effort

Test design is a reasoning task crucial to correctness, not mechanical repetition. Use:

• Model: the strongest reasoning model the runtime can provide—if the main agent is already at that tier, inherit it; otherwise override it. Do not hardcode specific brand names
• Reasoning effort：xhigh(the highest level supported by the runtime). Escalation ladder:low → medium → high → xhigh
• Tools: use Read / Grep / Glob on code paths; use Write on test files
• Permissions: read-only for non-test files; writable for test files

Dispatch prompt skeleton—Test quality constraintssection is the standard the dispatched agent must satisfy, and it is also the basis for validation in step 4; treat it as critical content, not boilerplate:

你正在为一个功能设计失败测试。你不会看到、也不会编写实现代码。你的任务
是产出可执行的测试——这些测试今天会失败，只有当功能被正确实现后才通过。

## 需求 / Requirement
<粘贴需求描述 + 验收标准>

## 代码路径（只读，用于理解上下文）/ Code paths (read-only, for understanding context)
<文件路径列表 —— 公开 API 面、集成边界、已有测试约定>

## 先分析，再编写 / Analyze first, write second
在起草任何测试之前：
1. 阅读代码路径；识别公开 API / 可观测的行为面。
2. 阅读派发包中按绝对路径列出的项目测试规则文件（源自仓库根及 monorepo 子包
   的 `docs/rules/test/`）；派发包写明「无项目专属测试规则」时，在输出中明确说明。
3. 扫描仓库中已有的测试文件；匹配其框架、runner、fixture 模式和命名风格。
   如果 `validation-contract.md` 提供了 `## Toolchain` 表格，直接从中取用
   runner / driver 并通过扫描确认，而不是从头重新推断技术栈。
4. 按 5 个场景类别（happy / empty / boundary / error / concurrency）枚举
   待测行为。

## 测试质量约束 / Test quality constraints

测试必须满足以下标准。如有任何一项不满足，否决你自己的草稿。

### 1. 在正确层级测试 / Test at the right level
- 纯逻辑、无 I/O → unit test（小、毫秒级）
- 跨越边界（DB / 网络 / 文件系统 / 时钟）→ integration test
- 仅限端到端的关键用户流程 → E2E test
默认选择能捕捉该行为的最低层级。

### 2. 测试行为，而非实现 / Behavior, not implementation
- 对结果进行断言：返回值、持久化状态、可观测的副作用。
- 不要对内部方法调用顺序、私有辅助方法或精确的日志字符串进行断言。
- 仅对其自身 mock 出来的值进行断言的测试是一种 tautology（自证循环）——
  禁止。

### 3. 仅在边界处 mock / Mock at boundaries only
优先顺序：真实实现 > in-memory fake > stub > mock。
仅 mock：
- 外部 API / 网络端点
- 时间、随机性及其他不确定性来源
- 以真实形态运行会很慢或有破坏性的操作（真实邮件、生产 DB）
不要 mock 被测单元或纯内部依赖。

### 4. 必须覆盖的场景 / Required scenario coverage
对每个公开行为，纳入覆盖以下类别的测试：

| 场景 / Scenario | 示例 / Example |
|---|---|
| Happy path | 合法输入 → 预期输出 |
| Empty / null | "", [], null, undefined |
| Boundary | 0, 1, max, max+1, negative |
| Error path | 非法输入、超时、权限拒绝 |
| Concurrency / order | 快速重复、乱序响应、竞态 |

仅当某类别确实不适用时才跳过（如纯函数的并发场景）。跳过时在测试计划中
说明原因。

### 5. 结构质量 / Structural quality
- Arrange-Act-Assert：每个测试中三段可见地分隔开。
- 每个测试一个断言概念。名称中含「and」→ 拆成两个。
- 测试名称读起来像规格说明句子。好：
  `it('rejects empty email with "Email required"')`。差：`it('test1')`、
  `it('works')`、`it('handles errors')`。
- 测试中 DAMP 优于 DRY：每个测试自包含。不要把被验证的内容藏在共享的
  `beforeEach` / helper 后面。

### 6. 不稳定风险——禁止模式 / Flake risk — forbidden patterns
- 依赖时间却不用 fake timers（`setTimeout`、真实 `Date.now()`）
- 依赖顺序（测试间共享可变状态、依赖迭代顺序的断言）
- 真实网络 / 真实文件系统且无隔离
- 对未经审查的输出做 snapshot test

### 7. 测试必须真正失败 / Must actually fail
- 针对空实现就能通过的测试什么也没测。
- 因 `ImportError`、缺失 fixture、语法错误或「module not found」而失败的
  测试通常是 **fake red**——它并未真正执行其声称要测的行为。
- **例外——greenfield**：当被测模块 / 符号属于本功能、目前尚不存在时，
  import 必然失败，而你只能写测试文件、无权创建被测代码。此时不要删除
  这些测试——按下方「流程约束」在测试文件顶部声明你假设的签名，并在测试
  计划里标注「在主代理补齐空骨架前，此测试会红在 import」。主代理会在实现
  开始前补齐纯签名空骨架，届时失败落到断言上。
- 对当前代码运行每个测试；确认失败原因与你的预测相符。删除或修复因路径
  写错、拼写错误等**你自己的失误**导致的 fake-red 测试；greenfield 的
  import-红按上一条保留。

### 8. 属性断言优于示例断言 / Property over example
对特定 happy-path 值进行断言的测试（如断言某个 fixture 的输出恰好等于
`[1,2,3]`），只在 input==fixture 时通过，对任何合法变体都会失败。改用
属性断言（已排序、幂等、包含全部输入），或为示例配一个变体输入测试来
验证同一不变量。

## 产出 / Produce

返回单条响应，包含以下四个部分：

### 当前状态 / Current state
- 覆盖该行为面的已有测试文件：`<file:line list>`，或「none」
- 检测到的测试框架 / runner / 断言库

### 覆盖缺口 / Coverage gaps
- 尚未被测试的行为：项目符号列表，按 5 场景类别分组

### 推荐测试（含优先级）/ Recommended tests (with priority)
- **Critical** — 能捕捉数据丢失、安全回归或契约违反的测试
- **High** — 针对核心业务逻辑 / 验收标准的测试
- **Medium** — 针对边界场景和错误处理的测试
- **Low** — 针对工具 / 格式化辅助函数的测试

每项附一行理由，说明它能捕捉的缺陷。

### 可执行测试 / Executable tests
- 测试文件，可直接运行。
- 每个测试附一行理由注释。
- 测试对当前代码失败，且失败原因与预测相符（通过运行测试验证）。greenfield
  例外见 §7：被测符号尚不存在时 import-红即预期原因，主代理补齐空骨架后落到断言。

## 流程约束 / Process constraints
- 不要提出或草拟实现方案。
- 不要编辑测试目录以外的文件。
- 使用项目已有的测试框架、runner 和 fixture 约定。
- 如果你必须对代码做出某些无法从只读文件中验证的假设，在测试文件顶部
  逐条列出每个假设。

步骤 4：验证返回的测试 / Step 4: Validate the returned tests

Before handing the tests over to the implementation phase, the main agent runs the following checks. If any item fails, reject the deliverable and require it to be redone.

跑红之前：补齐空骨架 / Before running red: materialize signature-only skeletons

"Tests must genuinely fail" (§7) requires tests to fail on assertions, not on loading. The dispatched agent can only write test files and cannot create the code under test—so forbrand-new modules or brand-new symbols(functions / classes / methods imported by the tests that do not yet exist), before running the red validation below, the main agent first materializespure signature-only empty skeletons：

• The signature (name, parameters, return type) aligns with the "assumptions" declared at the top of the test file—by using these APIs, the tests have already defined their shape; this is test-first API design.
• The function body only raise NotImplementedError / throw new Error('not implemented')，does not write any implementation logic。
• This is not the implementation of step 5—it only makes the imports resolve and the code load, so that the failure lands on assertions rather than ImportError. Step 5 is where the function body is filled in.

This is standard TDD practice (in statically typed languages this step is mandatory—without creating skeletons the tests simply won't compile; in dynamically typed languages normative practice likewise creates them, to ensure failures are diagnostically meaningful assertion failures). It happens after test design is finalizedafterwards, and is executed by the main agent, so it does not retroactively contaminate the independent evaluation. When modifying existing modules, if the symbol already exists, this step is a no-op.

Run tests to check / Run-the-tests checks

1. The test fails (red). Every returned test fails against the current code.
2. The failure reason matches the rationale statement. After the empty skeletons are materialized as in the previous section, the failure should be an assertion failure—not ImportError, not "module not found", not a syntax error. The test fails because the asserted behavior is missing—the rationale comment should be able to predict the failure message.

Standards-based checks / Standards-based checks

These checks cover §1–§6 + §8 of the "Test quality constraints" in step 3 (§7 "must genuinely fail" is jointly enforced by the empty-skeleton prerequisite step above + the run-the-tests checks). Violating any one of them means the test is rejected.

1. Appropriate level(§1)—logic that can be unit tested should not be wrapped in E2E; cross-boundary work should not masquerade as a pure unit test.
2. Test behavior, not implementation(§2) — Where a state assertion suffices, do not use expect(spy).toHaveBeenCalled… chains; do not assert on private helper methods or exact log strings.
3. Pure mock tests with no self-proving loop(§2) — Flag any test that only asserts on its own stub values (tautology).
4. Mock granularity(§3) — Mocks sit at network / DB / clock / FS / randomness boundaries; the unit under test is the real implementation.
5. 5-scenario coverage(§4) — happy / empty / boundary / error / concurrency all covered, or the reason for not applying is explicitly stated.
6. Structural quality(§5) — AAA is clearly visible, one concept per test, names read like specifications, no over-DRY setup.
7. No flaky patterns(§6) — No real timers, no order dependence, no real network, no unreviewed snapshot tests.
8. Property assertions over example assertions(§8) — No happy-path assertions targeting only a single fixture; use property assertions or variant-pair tests where applicable.

Coverage checks / Coverage checks

1. Distinct failure modes — Scan the rationale notes; remove near-duplicates.
2. Critical path priorities covered — Every Critical and High item in the "Recommended tests" section has a corresponding executable test.

Step 5: Hand off to implementation / Step 5: Hand off to implementation

With the verified failing tests and empty skeletons in place, implementation proceeds according to the test-driven-development skill: fill the empty skeleton bodies with the minimal implementation to make the tests pass (green), then run regression verification.

Anti-patterns / Anti-patterns

The following govern the test design'sorchestration. Standards for individual tests are in the Step 3 constraints and the Step 4 checks.

• ❌ The main agent writes tests after drafting the implementation — the tests will mirror the implementation's assumptions
• ❌ Dispatching at medium reasoning effort / to a weaker model to save cost — test design quality continues to have an impact throughout the feature's lifecycle
• ❌ Passing the contents of an in-progress branch to the dispatched agent — undermines independent evaluation
• ❌ Accepting tests that would pass against an empty implementation — these tests constrain no behavior
• ❌ Skipping Step 4 verification — unverified tests will be merged in a fake green state

Relationship to other skills / Relationship to other skills

• spec-design → produces validation-contract.md(primary input) + spec.md(fallback context), consumed by this skill
• test-driven-development → manages the red-green-refactor loop (downstream; consumes failing tests)
• systematic-debugging → intervenes when tests unexpectedly fail after implementation (downstream)
• verification-before-completion → runs tests at the "Done" gate (downstream)

The deep-review plugin's test-quality reviewer applies the same §1–§8 standards above when reviewing tests in a PR — the front end (this skill, at the design stage) and the back end (the reviewer, at the PR stage) each independently hold this standard, but they echo each other.

Example invocation / Example invocation

User：开始做新的插件依赖解析器。先设计测试。
Assistant：
  1. 需求：「解析器接收一份插件清单，返回遵守依赖关系并能检测循环的
     安装顺序。必须处理：传递依赖、菱形依赖、自引用、缺失依赖、循环。」
  2. 代码路径：src/plugins.ts、.claude/plugins.json schema、tests/ 目录
  3. 派发给一个独立代理技能（新会话），推理强度 `xhigh`，src/ 只读、
     tests/ 可写
  4. 代理返回：当前状态 + 覆盖缺口 + 推荐测试
     （5 个 Critical、8 个 High、5 个 Medium）+ 带理由注释的
     tests/resolver.test.ts
  5. 主代理为测试引用、但尚不存在的符号补齐纯签名空骨架 → 运行测试 →
     全部 red 在断言 → 对照 12 点的步骤 4 检查清单验证 → 移交