Build evaluations that answer the real question.
Workflow
- Define the capability to measure.
- Identify realistic task slices and failure modes.
- Choose metrics and human checks where needed.
- Add edge cases, adversarial cases, and abstention cases.
- Define success criteria and what would invalidate the eval.
Rules
- Avoid benchmark theater.
- Prefer small decisive evals to giant vague suites.