llm-testing

LLM Testing (Chinese Version)

When to Use

You need to test the task quality, robustness, safety, factuality, and version regression of large language model applications.
You need to conduct a risk review of existing solutions, results, or evidence, and form actionable improvement items.
The input is incomplete, but a usable initial version with assumptions and information gaps still needs to be provided first.

The default output is Markdown, suitable for review, execution, and continuous supplementation.
When the user requests a table, CSV, JSON, or ticket format, retain the same risk, evidence, priority, and boundary fields.
If the output will enter an automated process, first confirm the field schema, enum values, and required items.

First read and follow the input, execution rules, minimum coverage checklist, and output order of prompts/llm-testing.md .
Supplement the scope, environment, version, constraints, evidence, and success criteria that truly affect judgment.
First conduct an input audit, then distinguish confirmed facts, reasonable assumptions, and questions to be confirmed.
Sort by risk and evidence strength, and produce results that can be directly executed, reviewed, or verified.
When information is insufficient, do not stop at asking questions: first deliver a constrained initial version, and explain which conclusions cannot yet hold.

Each execution must read prompts/llm-testing.md; it is the complete execution specification of this Skill.
When this Skill needs to be evaluated or regressed, read evals/eval.yaml and the matching evals/cases/ test cases.
Only read references/、examples/、scripts/ or output-formats.mdwhen the directory actually exists and the task requires it; do not assume assets that do not exist.

Avoid using only exact string assertions
Fix and record model parameters
Use repetition and distribution judgment for non-deterministic results
Do not fabricate system behavior, fields, data, metrics, or root causes that do not exist in the input.
Key conclusions must be linked to evidence; when evidence is insufficient, mark them as assumptions and provide verification methods.
Priority must state the basis in terms of business impact, likelihood of occurrence, or detectability.

Covered: instruction following, factuality, robustness, safety, bias, context length, multilingual, randomness, cost and latency.
Facts, assumptions, gaps, and recommendations have been distinguished.
High-risk items have clear priority, evidence, owner, or next step.
The output contains verifiable judgment criteria rather than generalities.
No unauthorized production write operations or destructive actions were performed.

Listing only checkpoints without stating input conditions, expected results, or evidence.
Marking everything as high priority, losing the value of trade-offs.
Replacing domain judgment with tool names or general theory.
Directly refusing when the input is incomplete, or conversely pretending the conclusions are already certain.

Start from the paths most likely to cause business loss, safety issues, or release blockage.
Use the smallest verifiable experiment to reduce uncertainty, and record reproduction conditions.
Make the output directly executable and reviewable by another engineer.