ai-feature-testing

AI Feature Testing (Chinese Version)

When to Use

You need to verify how an AI feature embedded in a product performs in terms of quality, safety, fairness, degradation, and user experience.
You need to conduct a risk review of an existing plan, result, or evidence, and form actionable improvement items.
The input is incomplete, but a usable first version with assumptions and information gaps still needs to be provided first.

The default output is Markdown, suitable for review, execution, and continuous supplementation.
When the user requests a table, CSV, JSON, or ticket format, retain the same risk, evidence, priority, and boundary fields.
If the output will enter an automated process, first confirm the field schema, enum values, and required items.

First read and follow the input, execution rules, minimum coverage checklist, and output order of prompts/ai-feature-testing.md .
Supplement the scope, environment, version, constraints, evidence, and success criteria that truly affect judgment.
First conduct an input audit, then distinguish confirmed facts, reasonable assumptions, and questions to be confirmed.
Sort by risk and evidence strength, and produce results that can be directly executed, reviewed, or verified.
When information is insufficient, do not stop at asking questions: first deliver a constrained first version, and explain which conclusions cannot currently hold.

When calibrating domain results, correct comparisons, or evidence gaps, read the desensitized input, item-by-item expectations, and recalculation basis of references/domain-decision-example.md .
Each execution must read prompts/ai-feature-testing.md; it is the complete execution specification of this Skill.
When this Skill needs to be evaluated or regressed, read evals/eval.yaml and the matching evals/cases/ test cases.
Only read references/、examples/、scripts/ or output-formats.mdwhen the directory actually exists and the task requires it; do not assume assets that do not exist.

Fix the model, Prompt, knowledge/evaluation data, product, and environment versions; establish product task × language × user group × risk slices, record the sampling source and sample size, and do not cover up small groups with overall averages.
For the manual rubric, write observable anchors, labelers, disagreement handling, and reference answer sources for each dimension; calibrate samples first, then aggregate by project scoring and weights, and do not invent scores for undefined rules.
Separate task quality, safety/privacy, should-refuse versus normally answerable, and degradation/fallback thresholds; record the source, denominator, and version of each threshold; a high refusal rate does not automatically mean safety.
When slices lack samples or thresholds, output UNKNOWN/to be confirmed; launch recommendations should retain undecided thresholds and Human decisions, and cannot conclude that something has passed based on a static plan.

Do not test only ideal inputs
Distinguish model limitations from product integration defects
High-risk outputs must be designed with refusal or human fallback
Do not fabricate system behavior, fields, data, metrics, or root causes that do not exist in the input.
Key conclusions must be linked to evidence; when evidence is insufficient, mark them as assumptions and provide verification methods.
Priority must explain the basis of business impact, likelihood of occurrence, or detectability.

Covered: task quality, factuality, stability, safety, fairness, privacy, latency cost, degradation, and human takeover.
Facts, assumptions, gaps, and recommendations are distinguished.
High-risk items have clear priority, evidence, owner, or next step.
The output includes verifiable judgment criteria rather than generalities.
No unauthorized production write operations or destructive actions were performed.

Listing only checkpoints without specifying input conditions, expected results, or evidence.
Marking everything as high priority, losing the value of trade-offs.
Replacing domain judgment with tool names or general theories.
Directly refusing when input is incomplete, or conversely pretending the conclusion is already certain.

Start with the paths most likely to cause business loss, safety issues, or release blockers.
Use the smallest verifiable experiment to reduce uncertainty, and record the conditions for reproduction.
Make the output directly executable and reviewable by another engineer.