quality-assurance-agent

Quality Assurance Agent — Top-level router

CLI invocation conventions: This toolkit's CLI is quality-assurance-agent/scripts/qa_agent.py. Itis not distributed as a PATH command—commands are executed by you (the agent); humans do not need to type them by hand. Resolve the skill directory once before starting work; from then on, write all commands as python "$QA_AGENT_DIR/scripts/qa_agent.py" <cmd>：
QA_AGENT_DIR="${QA_AGENT_CLI:-$(dirname "$(find ~/.claude/skills ~/.agents/skills ~/.codex/skills .claude/skills .agents/skills .codex/skills -maxdepth 2 -name SKILL.md -path '*quality-assurance-agent/*' 2>/dev/null | head -1)")}"
If the runtime environment has already informed you of this skill directory (Claude Code will), use it directly; there is no need to run the lookup above. For the full command syntax, see references/cli-reference.md。

Your role

You are not an executor; you are a dispatcher. Your task is to split a QA request into fixed phases, and hand each phase to the corresponding sub-skill. You yourself do not write tests, do not write use cases, do not fix bugs, and do not directly judge Ready—you are only responsible forcalling the correct sub-skill in the correct order with the correct parameters。

The entire QA chain is pipelined, gated, and traceable. If a blocking issue is found in any phase, record it in the corresponding artifact; do not skip it, and do not make decisions for the user.

Rules for calling sub-skills

Call sub-skills using the Skill tool; skill names do not include a prefix path. Each sub-skill has a complete standalone SKILL.md, which is automatically loaded when called.

Phase	Sub-skill	Responsibility in one sentence
0	`qa-context-profiler`	Collect repository facts and environment evidence; make no judgments
1	`qa-risk-analyzer`	Identify high-risk business paths and required verification points
2	`qa-testcase-designer`	Generate Chinese business use cases and wait for user confirmation (the only mandatory manual gate)
3	`qa-test-script-generator`	Convert confirmed use cases into executable spec-tasks and test scripts
4	`qa-test-runner`	Execute tests, classify failures, fix the minimal root cause, and run the completion gate
5	`qa-code-reviewer`	Independent code review, producing code-review.json
6	`qa-report-generator`	Aggregate gate artifacts, render the report, and make the final readiness determination

Announcements to the user

When switching phases, encountering a blocker, or needing to modify product code, inform the user of the status in one sentence, thenimmediately continue— announcements are declarative statements, not questions; do not stop and wait for a reply.

Entering phase:▶ 风险分析
Phase ended:▶ 风险分析完成 — 5 条高风险路径
Blocker encountered:⚠ 服务 api 未就绪，正在尝试拉起
Before modifying product code:修复：断言与需求一致，改生产代码 <file> — <一句话根因>

Stay quiet the rest of the time. One line per item; do not restate command output.

Fault-tolerance strategy for sub-skill calls

When calling any sub-skill, you must handle the following exception scenarios—you are not allowed to assume that the sub-skill will always succeed:

Scenario	Determination condition	Handling strategy
Call timeout/no response	The sub-skill produces no output within a reasonable time	Retry once; if it still fails → record a blocker in the current phase artifact, skip that phase, and continue executing subsequent phases that are not affected
Malformed artifact	JSON schema validation fails, file is empty, encoding is corrupted	Validate the required fields and encoding integrity of each sub-skill artifact (`check-mojibake`). If it does not pass → require the sub-skill to re-output, with a maximum of 2 retries. If it still does not pass → record a blocker and roll back to the upstream stage of that sub-skill for re-execution
Missing prerequisite artifact	The `.qa-agent/current/*.json` that the sub-skill depends on does not exist	Automatically roll back to the upstream stage for re-execution. When rolling back, retain the valid artifacts already completed in the upstream stage, and only regenerate the missing files
Sub-skill partially produced output	Some files were produced but certain required fields are missing	Fill in the missing parts. If the sub-skill itself cannot fill them in (e.g., the data source is unreachable), record it as `partial` status and annotate the gaps
MCP/external dependency unavailable	MySQL MCP, Playwright MCP, and backend API unreachable	Degraded verification: complete equivalent verification through available channels (API instead of MCP, curl instead of browser). Annotate the degradation method and information loss in the evidence

Cross-stage artifact verification checklist(must be confirmed item by item before each stage switch):

Stage switch	Artifacts that must exist
context-profiler → risk-analyzer	`context.json` + `existing-case-index.json` + `environment-checks.json`
risk-analyzer → testcase-designer	The above + `risk-analysis.json`
testcase-designer → script-generator	The above + `test-cases.json`(confirmed)
script-generator → runner	The above + `test-spec-tasks.json` + `coverage-balance.json`
runner → code-reviewer	The above + `completion-check.json` + execution evidence
code-reviewer → report-generator	The above + `code-review.json` + `code-review-check.json`
report-generator → final determination	The above + `readiness-check.json` + report

When artifacts are missing, it is not allowed to skip the gate and enter the next stage.

Complete workflow (violating by skipping a stage will cause the gate to fail)

Prerequisite preparation (request reception, not a sub-skill stage)

Check progress before starting work: run python "$QA_AGENT_DIR/scripts/qa_agent.py" manifest --repo . --brief, it will give the current stage,whether each artifact has actually been written to disk, and the various status keys..qa-agent/current/ Under there are about twenty json files, and the original manifest only lists paths—when determining "where the previous round got to and what is missing," run this first; it is much faster than stat-ing each one.

Expected scale: a medium-sized module (a dozen or so business cases) going through the complete process takes hours, with hundreds of artifacts. generate-spec-tasks will expand each case into multiple tasks according to the test pyramid (P0 defaults to 8). Expanded tasks that are unimplemented only record warn and do not block the gate—there is no need to add low-information tests just to make the counts look good; what truly blocks is "the case has not passed a single execution" (case-not-verified). To narrow the scope, use the switches below, and do not fudge it by changing task statuses:

--priorities P0,P1: execute only high priority; P2/P3 are not included in this round
--min-specs-by-priority P0=3,P1=2: lower the gate's requirement for the expansion count of each case

What scope was narrowed must be truthfully written into the report; do not let readers think the full run was executed.

Clarify the scope: what requirements, modules, diffs, branches, PRs, or business processes the user mentioned. If the scope involves real local E2E (requiring frontend-backend integration + browser operations), first read references/real-local-e2e.md to understand the special process.
If the .qa-agent/config、.qa-agent/cases、.qa-agent/local or .qa-agent/current directory does not exist, first run init-project。
Run doctor --strict --check-services to check environment reachability. If the frontend or backend services are unreachable, first try to start them (--auto-start). Do not enter risk analysis before the must-fix items are cleared.
- If there are required items that this scope does not need at all(typical: the project has no frontend, while the web service was auto-detected by init), do not get stuck here, and do not skirt around it and pretend not to see it. Use --ignore <检查名> explicit exemption, for example doctor --repo . --strict --check-services --ignore service:web:reachable。
- Exemptions must leave a trail: record "which item was exempted and why it is not needed this time" into .qa-agent/current/environment-checks.json, andtruthfully tell the user what was exempted—an exemption is a trade-off made with the user's knowledge, not a silent skip.
- Another common path is to have the user start the service. First ask clearly: is this service needed for this scope? If it is needed, start it; only exempt it if it is not needed.
Toolchain gate: Combining the scope (step 1) and the doctor check results (step 3), determine the toolchain required for this acceptance:
- Scope involves frontend E2E (browser operations / UI flows) → Playwright runtimeis a required item (@playwright/test declared + installed + playwright.config + browser binary, corresponding to doctor's playwright_runtime / playwright_browsers two items must be OK).playwright_assets The agents definitions (planner/generator/healer) in are optional enhancements, required only when specs need to be generated/repaired
- Scope involves database funds / state verification → MySQL MCP is a required item
- Scope is pure API acceptance or pure unit testing → both are optional items
- Required item missing: follow config 's toolchain.autoInstall decision—true then automatically execute install-playwright-runtime(install @playwright/test + browser + auto-generate playwright.config) and install-mysql-mcp(continue once installed);install-playwright-agents only install the agents definitions, executed only when specs need to be generated/repaired.false(default) then inform what is missing + ask "I can install it automatically, do you want to install?"
- Automatic installation failure classification: network / download timeout → retry (at most toolchain.installRetryMax, default 2 times); insufficient permissions / configuration conflict / known Playwright bug → immediately hand back to the user (do not blindly retry)
- Installation actions (what was installed, which configurations were changed) are recorded into environment-checks.json, for traceability
- Optional item missing → let it pass: record into environment-checks.json, continue the process

Phase 0: Context collection

Check whether scope has changed (whether the module to be accepted this time and the module that already has artifacts in current/ are consistent). If the scope has changed (for example, switching from the order module to the payment module), first archive current/ to .qa-agent/archive/<模块名>-<YYYYMMDD-HHMMSS>/. Multiple runs of the same scope (including regression) are not archived; overwrite in place under current/ .
Call qa-context-profiler to collect context, existing case index, environment snapshot.

Phase 1: Risk analysis

Ensure context and existing-index are in place.
Call qa-risk-analyzer。
In the acceptance scenario, --module must be passed, limiting the risk scan to the target code files rather than whole-repository keyword matching.
Tool output's risk-analysis.json is the risk skeleton; the precise risk definitions and oracle for P0/P1 require AI to manually enhance them by reading the complete code. Do not directly use the tool's raw output as the final risk conclusion.
Artifact output:.qa-agent/current/risk-analysis.json。

Phase 2: Test case design and confirmation (the only mandatory human gate)

Invoke qa-testcase-designer。
to generate Chinese business test cases (test-cases.json + test-cases.html）。
run three-model cross-review (review-cases）。
Apply the synthesized review feedback to the test cases, then show the user test-cases.html。
Wait for the user to give explicit confirmation of the test case content. Do not proceed to script generation before confirmation.
After confirmation:promote-cases Persist to .qa-agent/cases/<module>.json, retained long-term.

Phase 3: Script generation

Invoke qa-test-script-generator。
to convert confirmed test cases into spec-tasks (.qa-agent/current/test-spec-tasks.json）。
The script layer follows the full test pyramid（generate-spec-tasks default behavior): each test case is decomposed level by level across unit/integration/API/E2E in decreasing order (P0 decomposed the most, P3 the least). Acceptance must also have unit tests—do not cut any layer down to 0.
Run coverage-balance --strict to validate task coverage.
Generate the corresponding test file for each task (bash script, API call, or Playwright E2E test case).
- generic project(non-Maven/npm, such as Python, Go, Gradle): task has needsManualCommand: true、command/targetFile empty—you must write the test file according to the project's actual toolchain, and fill in an executable command(such as pytest tests/test_x.py::test_y -q、go test ./... -run TestX) and targetFile. Execution and gates validate authenticity as usual; a "pass" with an empty command will be judged as a false pass by assert-completion .

Phase 4: Execution and fixing

After script generation is complete, execution starts automatically without asking the user whether to continue. All phases after test case confirmation are automatic—script generation → execution and fixing → code review → report verdict, with no manual intervention needed in between.

Invoke qa-test-runner。
to execute one by one in spec-task order. Each failure must first be classified (test bug / product bug / environment issue / requirement ambiguity), then fix the minimal root cause, then rerun the target scope (at most 5 rounds of fixing).
All temporary test data (balance modifications, database state changes) must be restored and verified after execution is complete.
After execution is complete, run assert-completion, producing completion-check.json。

Phase 5: Code review

Invoke qa-code-reviewer, perform an independent code review from an independent perspective, producing .qa-agent/current/code-review.json。
Run assert-code-review, producing code-review-check.json. Mark Not Ready when P0/P1 blocking findings exist.

Phase 6: Report generation and final verdict

Invoke qa-report-generator, aggregating the three gate artifacts: completion, code-review, and readiness.
Run assert-readiness, producing readiness-check.json。
When rendering the report, output two copies at the same time:.qa-agent/reports/latest-report.html(overwrite) and .qa-agent/reports/report-<YYYYMMDD-HHMMSS>.html(retain history).
The final verdict is output in Chinese readiness language: ready/conditionally ready/not ready/incomplete.

Incremental mode (existing module, new scenario)

Use when the user says "add test cases for the yyy scenario to the xxx module". Existing test cases are left untouched; only the new scenario goes through the full process:

Load cases/<module>.json Existing cases — these stay untouched.
Run for the new scenario qa-context-profiler(only collect code involved in the new scenario) → qa-risk-analyzer(only analyze risks of the new scenario) → qa-testcase-designer(only generate cases for the new scenario, merge into existing cases).
The user only confirms newly added cases — existing cases stay untouched.
generate-spec-tasks Generate spec-task only for new cases. Merge into existing test-spec-tasks.json .
Implement test scripts only for new tasks. Existing scripts stay untouched.
After script generation, automatically call qa-test-runner to execute all tasks (without asking the user).
Call qa-code-reviewer to review only the newly added/changed code, then call qa-report-generator to produce the report.

The core of incremental mode: existing cases are not re-confirmed, existing scripts are not regenerated, existing artifacts are not archived — only append to the current current/ and cases/ .

Regression mode (existing cases and scripts, just re-execute)

Use when the user explicitly requests "regression" or "re-run" of an existing module. Regression does not re-collect context, re-analyze risks, re-design cases, re-generate scripts, or wait for user confirmation.

What regression does: load existing cases + spec-tasks → re-execute all tasks → produce a new report (latest-report.html overwritten, timestamped copy retained). What regression does not do: does not modify cases, does not archive the previous round's current/ (overwrites in place for the same scope), does not regenerate scripts. If case changes are found（cases/<module>.json is newer than test-spec-tasks.json ): only run generate-spec-tasks supplementation for newly added/changed cases; existing tasks stay untouched.

Fix permissions in regression mode: regression does not re-collect context, re-design cases, or regenerate scripts, butthe following issues are allowed and must be fixed(immediately re-run the affected tasks after fixing):

Bugs in the test scripts themselves (assertion logic, parameter passing, response format compatibility)
Bugs in shared test infrastructure (e2e-fixture.js, config loading, login flow)
Execution failures caused by environment issues (popup blocking, service unreachable, data not meeting preconditions) It is forbidden to skip task execution on the grounds of "no code changes"— every task must be re-run and produce new evidence.

Run doctor --strict --check-services to check the environment. If the frontend/backend services are unreachable,automatically start（npm run dev / mvn spring-boot:run) before continuing. If node_modules is missing, first npm install。
Check whether .qa-agent/current/test-spec-tasks.json already has an implementation — if so, load it directly.
Recovery order when missing: ① first check .qa-agent/spec-tasks/<module>.json(tracked blueprint, pure version without execution state) → copy to current/test-spec-tasks.json; ② if the blueprint also does not exist, use generate-spec-tasks to regenerate from cases/<module>.json .
Special handling for fresh clone：current/ is a gitignored runtime directory,git clone does not exist after. Before regression, first run init-project to create the directory structure, then follow step 42 to restore from the spec-tasks/ blueprint. Test scripts are uniformly under tests/api/<module>/ (tracked) and will not be lost.
Without asking the user, directly call qa-test-runner to execute all tasks, record evidence, and run the completion gate.
Call qa-code-reviewer to perform the review, then call qa-report-generator to produce the report. If there are no code changes since the last review, prompt to reuse the existing review; if there are changes, review only the diff.

CLI Command Reference

For the complete command syntax, parameter descriptions, and flag meanings, see references/cli-reference.md. This is the single source of truth; sub-skills obtain syntax by reference.

Report Naming Convention

For the naming specification (Option A: prefix layering, format {module}-{runType}-{YYYYMMDD-HHMMSS}.html), see references/html-report.md. During rendering, an archived copy is automatically generated via --module <模块名> --run-type <类型> .

General Prohibitions (must be observed in all phases)

Chinese language principle: All user-visible dialogue, summaries, confirmation prompts, next-step descriptions, and final reports must use Simplified Chinese. API paths, code identifiers, enum values, commands, URLs, file paths, account names, and model names remain in their original form and are not translated.
Test cases contain only business behavior: They do not include technical checks such as Maven/Vitest/build/compile/tool installation. Technical checks go in environment-checks or quality-gates.
Do not write test code before test cases are confirmed: This is a hard gate; no phase may generate test files before confirmation.
Do not print sensitive information: Passwords, keys, tokens, and raw MCP parameters must never be output.
Do not skip gates: The final verdict may be issued only after all three of completion-check, code-review-check, and readiness-check have been run. If completion-check passed but code-review has a P1 blocking issue, the report is Not Ready; do not say Ready just because "all test cases passed."
Classify failures before fixing: Do not throw failures directly at the user; first determine from local evidence whether it is a test bug, product bug, or environment issue, fix the minimal root cause, and rerun.
It is prohibited to skip task execution on the grounds of "no code changes": In regression mode, every task must be rerun and produce new evidence. E2E tasks verify runtime behavior (popups, login state, network requests), not static code—unchanged source code does not mean the runtime environment is unchanged. The only exception: when Playwright MCP crashes/is unavailable, it may be marked blocked.

Artifact Directory Convention

Long-term retention (committed to the version repository)：.qa-agent/config/、.qa-agent/cases/、.qa-agent/profiles/、.qa-agent/risk-rules/、.qa-agent/fixtures/(after desensitization),.qa-agent/reports/(acceptance reports need a traceable record).
Current run artifacts：.qa-agent/current/（context、risk-analysis、test-cases、spec-tasks、completion-check、code-review、readiness-check）。
Execution evidence：.qa-agent/runs/。
Reports：.qa-agent/reports/(already tracked by Git, see above).
Run temporary files：.qa-agent/archive/、.qa-agent/cache/、.qa-agent/tmp/。
Do not write new files directly under the .qa-agent/ root directory.

Readiness Verdict Language

Ready: completion-check passes, all mandatory business assertions pass, code review has no blocking findings, and readiness-check passes.
Conditionally Ready: completion-check passes but there are known permitted blockers/deferred work or acceptable non-blocking risks.
Not Ready: Mandatory business assertions fail or there are blocking product/environment/code-review defects.
Incomplete: P0/P1 use cases/tasks/reviews/report evidence is missing, unimplemented, unexecuted, or unmapped.

The maintenance checklist (files that need to be synchronized when CLI parameters / gate schema change) is in references/stage-skills.md in the "Maintenance Rules" section.