SuperAPI AI Directory

combinatorial-testing

Testing & quality

View repository

Updated 10-16 01:34

About this skill

Combinatorial Test Design

After factors, values, and constraints are explicit, select high-risk multi-factor combinations and output CT-##. It only produces design candidates within the evidence boundary; it does not execute tests and does not claim coverage or passing.

When to Use

  • Need to analyze factors, values, combination constraints, interaction risks, and existing combinations.
  • Need to retain selection rationale, evidence gaps, priorities, and verification actions.
  • Input is incomplete but a limited initial version must be delivered, and items must be marked unassessed or blocked.

Output Format Options

  • Default output is Markdown; use tables, JSON, or CSV only when the user explicitly needs them or the existing delivery format requires them.
  • Separate static analysis, unexecuted items, evidence status, and Human decisions; when there is no execution evidence, keep unassessed, blocked, or NOT_RUN.

How to Use

  1. Read prompts/combinatorial-testing.md, and first list known, missing, conflicting, stale, out_of_scope, and assumptions.
  2. Form CT-## according to the method contract, retaining source, evidence status, impact, responsible role, closure conditions, and verification method.
  3. Facts, inferences, recommendations, and Human decisions must be separated.
  4. Only propose follow-up verification intent; do not write it as an executed result.

Domain Execution Points

  • Build the factor/value table and constraint expressions from the materials, distinguishing input, environment, and state; unknown value domains cannot be filled in as arbitrary binary values.
  • Deliver factors/constraints, strength rationale, the legal t-tuple universe, and gaps; declare static combination coverage only for models that have been enumerated.
  • When forming domain candidates, you must read and follow the domain operations and determination process of prompts/combinatorial-testing.md , retaining legal controls, evidence gaps, and verification actions item by item.

Core Constraints

  • Do not write the number of multi-factor combinations as coverage proof, do not ignore constraints, and do not fabricate values from experience.
  • File existence, names, templates, and Eval configuration do not equal real execution evidence.
  • Do not modify requirements, code, test assets, or the target system, and do not accept risk on behalf of the Human.

Pre-Delivery Self-Check

  • The six-category input audit is complete.
  • Each CT-## has source, evidence, impact/priority, responsible role, closure conditions, and verification method.
  • Unexecuted, unverified, unassessed, and pending-decision items are clearly identified.

Reference Files

  • During regression, read evals/eval.yaml and matching cases; configuration does not prove real project results.
  • For trigger verification, use evals/trigger-prompts.csv and evals/local-rules.json; when skill.selection evidence is missing, report BLOCKED.

Common Misconceptions

  • When scope or evidence is incomplete, do not write the method name, file existence, or number of candidates as evidence that tests were executed, covered, passed, or released.
  • Do not use common sense to fill in rules, thresholds, data, environments, or results that were not provided; retain unassessed, blocked, and pending-confirmation items.
  • Do not expand this Skill's specialized candidates into a complete strategy, complete test cases, real execution, or release decisions.

Best Practices

  • Complete the six-category input audit first, then select the smallest, traceable, and verifiable discovery scope.
  • Each finding retains the source, evidence status, impact/priority, responsible role, closure conditions, and verification method, and clearly states the residual risk.
  • Write verification recommendations as next-step actions, and do not upgrade the Skill package structure, number of candidates, or local Eval configuration into actual quality conclusions.