test-strategy

Test strategy (test-strategy)

Answer "How should this feature be tested" — translate risk intotwo-domain test scope and depth. The strategy sits in the "requirements → risk analysis → strategy → test design → test cases" chain; skipping the strategy and writing test cases directly is explicitly opposed by this framework.

• Input: requirements model (需求模型.md; if absent, inline lightweight study of requirement sources), code repository, system architecture, historical bugs
• Output (persisted to disk)：{项目}/测试策略.md(Risk Map + two-domain scope + depth_budget); when there is a handoff / blocked / external executor axis, additionally produce {项目}/专项移交_{轴}_{日期}.yaml
• Boundaries: do not write concrete test cases (→ test-case-writing); do not make decisions on the user's behalf (automation proposals and budget decisions all ⏸ pending confirmation); coverage scope is limited tosystem-level black box(UI / API / manual / special handoff) — unit/integration testing is the development side's responsibility; this strategy's risk ratings and "covered" conclusions all take as an assumption thatthis layer already has safeguards(when the assumption is unverified, note it in the test report to prevent the misreading that "all system-level tests pass = quality is assured")

When to Use

• "How should this feature be tested" — requires strategy decisions and rationale for scope / type / depth / priority
• Need to identify risks and rate them (Risk Map), as the basis for test case priority and regression anchors
• Need to decide the trade-offs among test types (which of performance / security / reliability / compatibility etc. to test, how deeply, and which are explicitly not tested) and keep a record
• Need to formulate an automation plan proposal and a long-term regression strategy

When NOT to Use

• End-to-end testing of the entire requirement → qa orchestration
• Need systematic requirements modeling → requirement-analysis
• Existing strategy, write test cases directly → test-case-writing
• Determine regression scope after code changes → regression-testing(this skill's regression_plan is a long-term regression strategy, not the scope selection for a particular diff)

Test strategy Schema (output structure)

test_strategy:
  feature:
  functional_scope:            # 功能域：范围+深度+理由（含 state / data_consistency 两轴；rationale 每轴必带，示例仅演示写法）
    functional:        { include: true, depth: full,     rationale: "核心资损路径", risk_refs: [R3] }
    boundary:          { include: true, depth: standard, rationale: "入参边界密集（金额/时限）" }
    permission:        { include: true, depth: full,     rationale: "越权高危", risk_refs: [R2] }
    state:             { include: true, depth: standard, rationale: "券状态流转多分支" }
    data_consistency:  { include: true, depth: standard, rationale: "库存与订单跨表一致" }
    regression:        { include: true, depth: standard, rationale: "存量功能回归基线" }
  type_scope:                  # 类型域：十轴全轴必答，每轴单行 flow 风格（校验器按行解析）
    performance:        { decision: include, depth: full,     signals: ["PRD-4.2 SLA", "order_service.go:88"], risk_refs: [R3], executor: k6, execution_status: blocked, todo: "向运维索取独立压测环境", handoff_ref: "专项移交_性能_{日期}.yaml" }
    security_business:  { decision: include, depth: standard, signals: ["多角色", "内部信号:permission≥standard"] }
    reliability:        { decision: include, depth: standard, signals: ["retry: pay_service.go:41"] }
    concurrency:        { decision: include, depth: standard, signals: ["库存扣减", "S级复核: check-then-write 命中 cart_service.go:41"] }
    compatibility:      { decision: include, depth: light,    signals: ["有前端"] }
    accessibility:      { decision: include, depth: light,    signals: ["有前端"] }
    visual:             { decision: include, depth: light,    signals: ["有前端"] }
    i18n:               { decision: exclude,  rationale: "需求信号（海外/多语言/RTL）未命中；无代码仓库", scanned: ["需求信号(G)"] }
    migration:          { decision: include, depth: standard, signals: ["migrations/2026-08-x.sql"] }
    contract_integration: { decision: include, depth: standard, signals: ["外部风控依赖"] }
  depth_budget:                # full ≤3（两域合并计）；被裁 Critical 轴走预算裁决
    full_axes: [functional, permission, performance]
    ranking_rationale: "R3(资损 Critical) > R2(越权 High) > ..."
  automation_plan:            # 哪些用例自动化、用什么框架（提案，⏸ 等用户确认）
  regression_plan:            # 回归策略（锚点用例、回归节奏）
  risk_map_ref:               # 指向 Risk Map（本文档内嵌，每条风险含证据）

Field conventions:signals references requirement sections / 文件:行 / internal matrix signals;scanned is the G+S dual list of exclude (items suffixed with (G)/(S));executor defaults to agent; non-agent (k6 / locust etc.) must carry handoff_ref；execution_status ∈ ready / blocked / done; exclude axes are omitted. Depth levelstake only three values: full (cell-by-cell full coverage) / standard (main paths + key exceptions) / light (sampling + smoke) — "not tested" is not a depth but a scope decision: for the functional domain write include: false(with rationale attached), for the type domain write decision: exclude or handoff, both must carry a rationale (the validator will reject any other value appearing in depth).

Hard format constraints (to prevent format drift in weak models)：type_scope Each axis must be written as a single line flow style, in the form performance: { decision: include, depth: standard, signals: ["..."] }——Forbiddento split one axis into a multi-line block of decision / depth / signals (the validator parses by line; block style = validation failure = that axis is treated as undecided). Before writing, copy the shape of the previous line. Basis: in actual testing of weak models, nearly half of samples showed block-style drift causing parse failures, while the decision content itself was fine — pure format loss.

Workflow

1. Input study

• Read the requirements model (if none, read the original requirements source for a lightweight study); if there is code → request the repository (path/branch/diff), confirm the implementation form (whether there is a UI, interface surface, data flow)
• Collect historical bugs (same functional area/same module); historical defect density is an estimation input for Likelihood and a trigger input for the R3 upgrade/downgrade

2. Risk identification and rating (at this point load ../core/risk-model.md）

• Bydimensionscan for risks: functional domain dimensions (permission / data consistency / boundary / state transition / asset loss—for the correspondence between the five dimensions and the six axes of scope in step 3, see ../core/risk-model.md dimension comments) + type domain axis names (concurrency / reliability / security / performance / compatibility / migration / contract—same source as the matrix axes)
• Each risk (R1, R2…) is rated according to the model:Impact × Likelihood (each 1–5) → Critical/High/Medium/Low, and each must carry evidence—a risk rating without evidence is considered an invalid rating
• For method selection, refer to ../core/testing-principles.md Section 2 (select design methods by functional characteristics, determining the depth rationale for each axis of the functional domain)

3. Functional domain scope decision

Decide include / depth / rationale axis by axis (functional / boundary / permission / state / data_consistency / regression):

• The risk level → test case priority mapping follows ../core/risk-model.md Section 3 (Critical must have P0 and be a regression anchor…); deviations must be explicitly justified
• Even what is not tested must have a rationale—"why not test" is just as important as "why test"
• High depth for permission (≥standard) is also an internal inclusion signal for axis 2 business security (for the ruling, see Matrix Section 2)

4. Full-axis scan of the type domain (at this pointload by group ../core/test-type-matrix.md corresponding subsections)

Proceed by axis group(A funds and correctness → B dependencies and changes → C frontend experience; for each group, load only the subsections for that group's axes from the matrix, not the entire file):

1. G-level scan(when there is a code repository, run once at the start of work and share across the three groups):python3 ../core/scripts/scan_signals.py <仓库路径> → G-level signal for each axis + pre-filled table
2. Axis-by-axis decision (constrained choice, revised from the pre-filled table, not generated from blank): requirement signal → G-level check → S-level item-by-item review(for items marked 〔S〕 on each matrix axis, read the code item by item to confirm) → decision → depth
3. Prevent rubber-stamping: exclude is never pre-filled—exclude must complete the S-level review and write the G+S dual checklist into scanned; "no script hit" alone does not constitute a reason for exclude
4. Intra-group delivery check: proceed to the next group only after all axes in this group have decisions; once all ten axes are complete, proceed to step 5

No code repository: skip G/S levels and decide based only on requirement signals; for the rationale of exclude, note "no code repository" (the validator exempts the S-level requirement accordingly).

5. Depth calibration and budget ranking (at this point execute R1–R6 in Matrix Section 13)

• Upgrade/downgrade according to R1–R6 (risk upgrade / dual-source signal / historical defects / no-signal downgrade / cost gate / budget constraint)
• R6 > R1: when full (two domains merged) > 3, trim by risk ranking; a trimmed Critical axis triggers ⏸ budget ruling checkpoint—present the ranking and trimming impact; the user may expand the budget (when expanding the budget, depth_budget records budget_review: {approved_by: 用户}）。Under headless(pipeline-integration convention 1, clause 4): record faithfully budget_review: {headless 未决: 保守裁剪, 依据: …}— must not be written as user-approved; after manual reflow, append on the original line approved_by: 用户
• depth and execution_status are separated: missing environment / data / tools → depth is not lowered, record blocked + todo(from whom to request what) — "cannot do" must not masquerade as "no need to test"

6. Automation plan proposal (⏸ proposal, not a ruling)

• Propose based on use case characteristics: has UI main flow → Playwright (automated-e2e-testing); interface-level validation/idempotency/concurrency → API script (api-testing); execution artifacts of script-type axes (performance/visual, accessibility axe scan tasks) → specialized scripts / scan tasks; exploratory/one-off → manual
• automation_plan is aproposal: list the suggested checklist + framework + non-automation checklist + rationale, submit to the user for confirmation (in the qa pipeline this is the execution strategy checkpoint); do not start execution-type skills before confirmation

7. Regression strategy

• Designate from Critical/High risks and P0 use cases theregression anchors(must run every time)
• regression_plan: anchor set + extension rules by change type (interface change → full interface use case suite; UI change → main flow E2E…)

8. Persistence and delivery

1. Write {项目}/测试策略.md(Schema structure + embedded Risk Map; type_scope one-line flow style per axis + sufficiency statement: per-axis tier action checklist and completion status; opening paragraphscope assumption statement: this strategy is limited to system-level black box, unit/integration testing is the development side's responsibility, and conclusions are premised on this)
2. handoff axis / include+external executor / blocked axis → generate {项目}/专项移交_{轴}_{日期}.yaml handoff package (target, scenario parameters, thresholds/acceptance criteria).For {axis} in the file name, use the following axis Chinese short names(one-to-one correspondence with matrix axes; for axis 1 the full name is "performance efficiency" and the file name takes "performance"): performance / business security / reliability / concurrency consistency / compatibility / accessibility / visual consistency / internationalization / migration and upgrade / contract and integration, e.g. 专项移交_性能_20260901.yaml— the execution side (api-testing, etc.) consumes by Chinese-name glob; using the English axis id will break the chain
3. Deliver after validation：python3 ../core/scripts/validate_schema.py {项目}/测试策略.md(V1–V5; when there is a code repository, additionally overlay --repo-root {仓库根} to spot-check the authenticity of what each axis's signals refer to), and deliver only after errors are cleared. Note: if an axis that is included and carries a Critical risk ultimately remains below the full tier, the validator requires that axis name to appear in the budget_review text — a downgrade decision must leave a line of basis, and must not silently disappear
4. Deliver the index to downstream (test-case-writing): strategy path + included axis checklist (for type/tag requirements of use-case-type axes see matrix section 12) + priority mapping requirements

Common Mistakes

Error	Consequence	Correct practice
Risk rating without evidence (off-the-cuff High)	The rating cannot be reviewed, and the strategy loses its derivation chain	Attach evidence to each risk (../core/risk-model.md）
Type axis silently missing (missing axis)	A type blind spot is hidden as "never thought about it" and cannot be challenged	All ten axes must be answered (V1), and exclude must also leave a trace
include without signal / exclude without reason	Over-testing / missed testing	signals attach evidence; rationale + scanned dual lists (V2/V3)
Script has no hit, directly exclude	Script blind spots are institutionalized as missed tests	exclude only after S-level review (G+S dual confirmation)
full is everywhere	Resources diluted on the long tail, no focus	full ≤3 + attach risk number (V4); on conflict R6>R1 + budget adjudication
Missing environment recorded as exclude	"Cannot do it" masquerades as "no need to test"	depth/execution separated, blocked + todo (R5/V5)
scope only writes a boolean switch	"How deep to test" is lost, each interprets it their own way during execution	Scope + depth + rationale trio
Write test cases into the strategy	Out of bounds (that is the output of test-case-writing)	Strategy stops at "scope/depth/priority requirements"
automation_plan executes directly without waiting for confirmation	Deprives the user of the right to adjudicate the execution strategy	Proposal → ⏸ user confirmation → only then start execution
Risk levels continue to use P0/P1 naming	Ambiguous, same name as test case priority	Risks use Critical/High/Medium/Low (../core/risk-model.md）