Positioning
5-layer automated verification chain, from the project'sexisting test infrastructuremap the tools and commands for each layer—do not invent new commands, only orchestrate gates. The full chain must pass before delivery; each layer's result must include fresh command output (verification-before-completion). Layers areconcepts, tools/commands areproject mappings(see the "Project Mapping" section).
When to use
- Before delivery / after major changes / when the user asks "self-test everything passes before giving it to me"
- Before any "claim of completion"; triggered by "is there a QA chain" or "run the quality chain"
Five-layer verification chain + pyramid gradient
| Layer | Quality dimension | Verification target | Typical tool family | Quantity/speed/frequency (pyramid) | Gate trigger |
|---|---|---|---|---|---|
| L1 Domain/unit logic | Logic | Pure functions, business rules, value object invariants, state machines | Unit testing frameworks (Vitest/Jest/…) | Many·Fast·Every commit | Must run on logic changes |
| L2 Contract | Functionality | Inter-module/inter-process interfaces, API/channel integrity, schema | Type checkers (tsc/…) + contract testing (Pact/…) | Medium·Medium·Must run in CI | Must run on interface/contract changes |
| L3 Component interaction | Functionality | Component/module interaction logic, authorization/state machine flows | Component testing (Playwright/Testing Library/…) | Medium·Medium·Must run in CI | Must run on interaction logic changes |
| L4 End-to-end | Experience | Real user journeys (entry→action→result→delivery) | E2E frameworks (Playwright/Cypress/…) + real dependencies | Few·Slow·Before merge/delivery | Must run on any functional change |
| L5 Visual regression | Visual | UI rendering pixel baseline | Visual regression (toHaveScreenshot/Chromatic/…) | Few·Slow·On UI changes | Must run on UI changes |
Pyramid discipline: the lower layers are many and fast, the top layers are few and slow. If the number of test cases at some layer is inverted (more E2E than unit tests) → an imbalance signal: push assertions that can be moved down to the lower layers, and keep only the journeys that truly need a real environment at the top. Layers can be trimmed: projects without UI skip L5, those without IPC/RPC skip the L2 contract sub-item (keep type checking).
Verification and validation (V&V—two different questions)
- Verification: whether the implementation is correct (building it right)—assertions at each layer L1-L5.
- Validation: whether what is implemented is what is wanted (building the right thing)—all functional ACs pass + the implementation is traceable to the original definition (tickets/spec ↔ documentation).
- Both are indispensable: the full chain is green but the ACs are not implemented = verified, validation failed—do not deliver.
Project mapping (detect → adapt)
- [ ] 1. 探测测试基建：package.json scripts / CI 配置（GitHub Actions 等）/ 测试目录布局
- [ ] 2. 映射各层：L1 → 单测脚本；L2 → 类型检查 + 契约测试；L3 → 组件测试；L4 → e2e 脚本；L5 → 视觉脚本
- [ ] 3. 探测失败（无 scripts/自定义 runner）→ 问用户或查项目文档；禁止猜命令
- [ ] 4. 记录映射表（输出报告含每层实际命令）——下次复用
Each layer must be able to answer: "What command to run, how it counts as passing". Command layers that cannot be mapped → marknot covered(not pretend to pass).
Configurable slots:tool(runner),runner(command template),workdir、env(e.g. secret-type environment variables—paths not written as values).
Coverage and Sufficiency
- Must-run ≠ sufficient: after the layer gate passes, check whether that layer has a metric proving "enough"—statement/branch coverage thresholds (project convention, e.g. L1 ≥80% branch); key invariants may optionally usemutation testingspot checks (mutation kill rate—prevents "assertions that didn't test the semantics").
- Projects without a coverage convention: mark "not measured" (not counted as FAIL, but explicitly listed in the report—prevents false sense of security).
- Test data isolation: data between cases must not contaminate each other (in-memory adapters/per-case reset/factory data)—avoids "green only when depending on execution order".
Full-chain run-all (delivery gate)
# 按映射结果拼装：全部层依次执行，任一失败即停（或记录后汇总）
<L1 命令> && <L2 命令> && <L3 命令> && <L4 命令> && <L5 命令>
- Do not claim completion without running the full chain; any failure → locate root cause (systematic-debugging) → fix → rerun the full chain
- flaky governance: for intermittent failures, first characterize them (rerun the same command to confirm whether it is flaky)—flaky tests are themselves defects, record them as todos, and must not be left unaddressed long-term on the grounds of "passed on retry"
- Gate passing = release approval, not release: for projects with CI, run-all should become a required passing status check; production release (canary/feature flag) is a release decision outside CI and not within this gate
DoD Quality Gate (feature "Done" determination)
| DoD item | Check | Corresponding layer |
|---|---|---|
| Feature complete | All ACs of the feature pass (predicates/scenario assertions) | Confirm (backfill into tickets/spec) |
| Deterministic verification | Logic/contract/interaction, etc.deterministic parts—assertions must pass (non-probabilistic) | L1-L3 |
| Probabilistic verification | Model-driven behavior(LLM response quality/agent behavior)—threshold + sample verification (e.g. 5 trial runs with ≥4 passing), prevents drift | L4 (if applicable) |
| Quality chain passes | Affected layers + full-chain run-all all green | L1-L5 |
| Definition alignment | Implementation is traceable to the original definition (tickets AC ↔ documentation) | Confirm |
Iron rule (AI application specialization): deterministic parts must never be fudged with "the model should"; they must be asserted; probabilistic parts must never be passed off as "passed once"—require sample/threshold evidence (sample size and threshold are project conventions and can be parameterized).
Workflow
- [ ] 0. 项目映射（首次运行/基建变化时）：探测测试基建 → 各层命令映射表（见「项目映射」）
- [ ] 1. 确认改动范围 → 映射影响层（L1-L5 哪些）
- [ ] 2. 跑受影响层（+ 门禁要求的最小集合）——任一失败：定位根因（systematic-debugging）→ 修复 → 重跑该层
- [ ] 3. 交付前全链 run-all（flaky 定性：偶发失败重跑确认——flaky 是缺陷，记入待办）
- [ ] 4. 确认层：tickets AC 回填 + 定义对齐（product-doc-audit 如需）
- [ ] 5. 报告：每层结果 + 实际命令 + 证据（输出尾部）——未通过/未确认不交付
Boundaries (division of labor)
| Adjacent skills | Division of labor |
|---|---|
verification-before-completion |
Single-point verification discipline (evidence before assertion); this skill is its chain-level orchestration |
stage-gate |
Stage-level DoD aggregation (stage-spec executed item by item); this skill = delivery-level quality chain (feature granularity) |
systematic-debugging |
On failure, root cause first, then fix (this skill only orchestrates, does not debug) |
coverage-matrix |
Coverage matrix maintenance (invariants/events ↔ tests); this skill = execution gate |
playwright-best-practices / pixel-perfect / test-data-generation etc. |
TestingWritespecifications (anti-flaky/data isolation); this skill = orchestration and gating |
cicd-pipeline |
CI pipeline configuration (the run-all mapped by this skill lands as a CI job/status check) |
Completion criteria
- All affected layers pass (command output as evidence, mapping table visible)
- Full-chain run-all passes before delivery
- Confirmation layer: tickets AC backfill (implementation↔definition traceable)
- Report includes evidence per layer + explicit annotation of uncovered layers/unmeasured items
- Flaky identified and recorded in the backlog (if any)
Anti-patterns
- Guessing commands from memory (not probing project infrastructure)
- Running only L1 and skipping L4/L5 ("the change is small" is not a reason)
- Claiming delivery after a single layer passes (no run-all, no confirmation layer)
- Pretending to pass when the project has no tests for a layer (should be marked as uncovered)
- Inverted E2E case count (pyramid imbalance—assertions should be pushed down)
- Tolerating flaky via "passes on retry" (flaky is a defect, not luck)
- Treating this skill as a debugging tool (failures should be handed to systematic-debugging)
Methodology sources (2026-08 research)
- Test pyramid/layering: Martin Fowler (martinfowler.com/bliki/TestPyramid.html); Parasoft automation pyramid
- CI quality gates: SonarQube Quality Gates; GitHub Required Status Checks (Microsoft Code-with-Engineering-Playbook)
- Definition of Done: Scrum.org DoD resources; Microsoft Done/Undone
- V&V / IEEE 829 (test documentation and verification/validation distinction), ISO/IEC/IEEE 29119
- Contract testing: Pact consumer-driven contracts (docs.pact.io)
- E2E trade-offs and flaky: Google Testing on the Toilet ("good E2E is few and focused")
- Coverage/mutation testing: statement/branch vs mutation adequacy comparison (2023 empirical); A Formal Notion of Program-Based Test Data Adequacy (1982)
- Canary/release: feature flag canary practices (ConfigCat/Statsig)
- DoD academic: On the benefits and problems related to using Definition of Done (survey study)
- Quality model: ISO/IEC 25010 (non-functional dimension classification)