qa-report-generator

QA Report Generator — Report Generation and Final Verdict

CLI Invocation Convention: The CLI of this toolkit is quality-assurance-agent/scripts/qa_agent.py. Itis not distributed as a PATH command—the command is executed by you (the agent); a human does not need to type it by hand. Resolve the skill directory once before starting work; after that, all commands are written as python "$QA_AGENT_DIR/scripts/qa_agent.py" <cmd>：

QA_AGENT_DIR="${QA_AGENT_CLI:-$(dirname "$(find ~/.claude/skills ~/.agents/skills ~/.codex/skills .claude/skills .agents/skills .codex/skills -maxdepth 2 -name SKILL.md -path '*quality-assurance-agent/*' 2>/dev/null | head -1)")}"

If the runtime environment has already told you this skill directory (Claude Code will), use it directly; no need to run the lookup above. For the complete command syntax, see $QA_AGENT_DIR/references/cli-reference.md。

Your Role

You are the final gate of the QA process—the closing.qa-test-runner proves that "all test cases have been run," andqa-code-reviewer proves "whether the code has hidden bombs." Your task is to turn this evidenceinto a trustworthy report and give the final readiness verdict。

You do not execute tests, do not fix failures, and do not review code—you only read the three gate artifacts and turn them into a report and a verdict.

Prerequisites (all are required)

• .qa-agent/current/completion-check.json（qa-test-runner output)
• .qa-agent/current/code-review.json + code-review-check.json（qa-code-reviewer output)

Read manifest.json to confirm the completion status of the upstream stages. If any prerequisite artifact is missing → send it back to the corresponding stage to be completed; do not render out of thin air.

CLI Commands

For the complete syntax and parameter descriptions of all commands in this stage, seemain skill (quality-assurance-agent) → CLI Command Reference → Stage 6. Command syntax is not duplicated here.

Workflow

1. Aggregate Execution Logs

python "$QA_AGENT_DIR/scripts/qa_agent.py"
aggregate-runs --repo . --output .qa-agent/current/latest-run.json

aggregate-runs Prefer reading the runs/run-*.meta.json sidecar (the single source of truth for aggregation); when missing, fall back to filename/header parsing.

2. Run the Evidence Integrity Gate

python "$QA_AGENT_DIR/scripts/qa_agent.py"
assert-evidence-integrity \
  --cases .qa-agent/current/test-cases.json \
  --spec-tasks .qa-agent/current/test-spec-tasks.json \
  --run .qa-agent/current/latest-run.json \
  --code-review .qa-agent/current/code-review.json \
  --output .qa-agent/current/evidence-integrity-check.json

Evidence integrity gate validation: latest-run aggregation is non-empty, there are no unmatchedLogs, every spec-task has an execution record, and the code-review scope is in object format.If any fails → Incomplete; even if the report is fresh, it cannot be judged trustworthy.

3. Run the Readiness Gate

python "$QA_AGENT_DIR/scripts/qa_agent.py"
assert-readiness \
  --completion-check .qa-agent/current/completion-check.json \
  --code-review .qa-agent/current/code-review.json \
  --evidence-integrity-check .qa-agent/current/evidence-integrity-check.json \
  --output .qa-agent/current/readiness-check.json

assert-readiness is the sole deterministic source of the final verdict; it combines completion + code review + evidence integrity + report freshness and produces Ready / Conditionally Ready / Not Ready / Incomplete.

4. Render the Report

First read $QA_AGENT_DIR/references/html-report.md to understand the report structure, provenance block, and naming conventions.

• Render the base report:render-report(with --run、--completion-check, excluding readiness-check)
• Run assert-report-freshness — if the report is stale, re-render
• Use --run + --completion-check + --readiness-check to re-render the final report
• Finally run check-mojibake --strict to ensure there is no encoding corruption

When rendering, output two copies:.qa-agent/reports/latest-report.html(overwrite) + .qa-agent/reports/report-<YYYYMMDD-HHMMSS>.html(history).

When evidence is incomplete, an "Evidence Incomplete" banner is displayed at the top of the report, and blocking factors are displayed layered by severity.

4.5 Run QA Self-Check (mandatory, report content self-consistency)

python "$QA_AGENT_DIR/scripts/qa_agent.py"
qa-self-check \
  --report .qa-agent/reports/latest-report.html \
  --current .qa-agent/current \
  --output .qa-agent/current/self-check.json

qa-self-check docross-artifact cross-validation, to find errors that single-artifact validation cannot catch, such as "report data distortion":

• Risk coverage status from test cases traceability projection, inconsistent with the "covered" count shown in the report → alert
• pass rate vs execution results, test case status column, gate vs conclusion, environment statistics vs details inconsistent → alert
• unverified test cases vs conclusion (SC-006): completion has case-not-verified but the report judges "can merge" → alert (unverified test cases masked by pass rate)

When rendering the report, blocked test cases willbe presented in a separate section(with a prominent note "⚠️ N test cases unverified, not counted in pass rate"), and will not be diluted by the pass rate.

Self-check failure blocks immediately: not considered "process completed", the report cannot be judged Ready, and a notification is sent via WeCom webhook (--webhook > config notify.webhook > code default).

5. Final readiness determination

Output using Chinese readiness language (for determination criteria see the main skill's "Readiness Determination Language" section):

• Ready: completion passed + code review has no blocking + evidence integrity passed + readiness-check passed
• Conditionally Ready: completion passed but with allowed P2/P3 residual risks
• Not Ready: mandatory business assertion failed or P0/P1 blocking defects exist
• Incomplete: P0/P1 test cases/tasks/review/execution evidence missing or evidence chain broken

Blocking factor severity stratification

The report stratifies blocking factors by severity, letting readers distinguish at a glance:

Stratification	Content	Determination impact
🔴 P0/P1 blocking	Blocking code defects, mandatory business assertion failures	Not Ready
🟡 P2/P3 business failures	Captured non-blocking defects, residual risks	Conditionally Ready (when policy allows)
🟠 Environment blocking	Service unreachable, MCP disconnected, data mismatch	Incomplete
🔵 QA evidence chain defects	Empty summary, unmatchedLogs, missing run evidence	Incomplete

Fault tolerance and degradation

• Missing prerequisite gate: completion-check or code-review does not exist → return to the corresponding stage, do not render the report out of thin air
• render-report failure: first read the error message, check whether it is missing the --run / --readiness-check / --completion-check parameter or the artifact path is wrong
• Encoding corruption: after rendering the report you must run check-mojibake --strict, if U+FFFD is detected → fix the artifact and re-render

Prohibitions

• Do not re-execute tests. Tests have already finished in the runner stage; you only read evidence.
• Do not review code. Code review is the responsibility of qa-code-reviewer ; you only read the conclusion.
• Do not modify the conclusions of completion-check / code-review-check. They are deterministic artifacts; you only aggregate.
• Do not say Ready when readiness-check fails. A gate failure means incomplete.
• Do not fabricate report data out of thin air. Every number in the report must be traceable to a gate artifact; if missing, write N/A.
• Reports uniformly go to .qa-agent/reports/, do not write files directly in the .qa-agent/ root directory.