triton-operator-precision-eval

Triton Operator Precision Evaluation

Core Principles

Precision is the bottom line for operator correctness. Do not perform performance optimization before verification passes.

Workflow

1. Write a Torch reference implementation (benchmark)
2. Write test cases (covering multiple shapes × dtypes × edge cases)
3. Execute the Triton operator and the Torch reference implementation on the NPU
4. Use test_common.validate_cmp() for comparison and generate a precision report

MANDATORY - READ ENTIRE FILE: Before writing tests, read in full scripts/test_common.py。

Error Thresholds

Judgment condition:MERE < threshold and MARE < 10 × threshold

Data Type	Threshold	MERE upper limit	MARE upper limit
float16	2⁻¹⁰ ≈ 9.77e-4	9.77e-4	9.77e-3
float32	2⁻¹³ ≈ 1.22e-4	1.22e-4	1.22e-3
bfloat16	2⁻⁷ ≈ 7.81e-3	7.81e-3	7.81e-2
int8/uint8/int16/uint16/int32/uint32/int64/uint64	Exactly equal	—	—
bool	Exactly equal	—	—

Where:

• MERE = mean(|y_cal - y_ref| / |y_ref|), i.e. mean relative error
• MARE = max(|y_cal - y_ref| / |y_ref|), i.e. maximum relative error

The report must include

• Verification configuration: operator name, test shapes, dtype, number of cores
• Precision standard: thresholds for all dtypes and the judgment condition (MERE < threshold and MARE < 10 × threshold)
• Verification results: total passed/failed
• Error metrics: MERE (mean relative error), MARE (maximum relative error)
• Judgment condition

Output Conventions

• The precision report is written to the operator directory under precision_report.md(Markdown format)
• The precision evaluation script is written to the operator directory under precision_eval.py
• Refer to the example scripts and report format under the examples/ directory

Anti-Pattern Checklist (NEVER)

• ❌ Perform precision verification without providing a Torch reference implementation
• ❌ Use the wrong threshold (e.g. using the FP32 threshold for FP16)
• ❌ Not use the MERE/MARE judgment condition (using only rtol/atol is not strict enough)
• ❌ Do not promote precision to FP32 for reduction operations
• ❌ Test only one dtype/shape and assert correctness
• ❌ Skip edge cases (non-aligned dimensions)
• ❌ Perform performance optimization before precision passes

Checklist

• The Torch reference implementation is correct and verified on the NPU
• Cover multiple shapes × dtypes
• Use FP32 for reduction operations
• Generate a standardized precision report