triton-operator-performance-eval

Triton Operator Performance Evaluation (Ascend NPU)

Core Principles

Trust only msprof data, not intuition.

Performance Ratio (Ratio) Definition：Ratio = torch_npu 耗时 / Triton 耗时. Ratio > 1.0 means Triton is faster, Ratio < 1.0 means slower.It is not the direct ratio of elapsed times, but the reciprocal of the elapsed times. The target is usually Ratio ≥ 1.0x.

The only trustworthy collection methods:msprof(function-level) and msprof op(operator-level). Other methods (time.time()、torch.npu.Event、do_bench, etc.) include Host overhead and lack sufficient precision,absolutely must not be used for performance evaluation。

Command	Purpose	Typical Scenario
msprof --application="python x.py"	Function-level: multi-operator comparison, full-pipeline analysis	"Which operator is the slowest?"
msprof op --kernel-name=K python x.py	Operator-level: hardware utilization, Bank Conflict	"Why is this kernel slow?"

Decision: first msprof to locate hotspots, then msprof op for in-depth analysis.

Reference Resource Loading

Task	Must Load	Do Not Load
Function-level performance comparison	msprof-function-level.md	msprof-op-level
Operator-level hardware analysis	msprof-op-level.md, performance-data-analysis.md	msprof-function-level
Understanding hardware terminology	ascend-terminology.md	—

Performance Summary Template

After completing the evaluationmustoutput:

Module	Content
Basic Information	Operator name, input scale, hardware model, measurement method
Performance Metrics	Execution time, memory bandwidth utilization, Cube/Vector utilization, L2 Cache hit rate, Bank Conflict
Bottleneck Diagnosis	Bottleneck type + basis for judgment + CSV data evidence
Optimization Suggestions	Arranged by priority, with expected gains

Principle: All conclusions must be supported by Profiling data. Utilization < 30% is marked as Key Focus。

Bottleneck Classification (for diagnostic categorization only)

Bottleneck	Judgment Condition	Detailed Optimization Methods
Memory-Bound	AI < hardware balance point	→ hand over to performance-optim skill
Compute-Bound	AI > hardware balance point	→ hand over to performance-optim skill
Latency-Bound	Both bandwidth and compute utilization are low	→ hand over to performance-optim skill

Boundary: This skill is responsible for diagnosing the bottleneck type and outputting data. Optimization methods are the responsibility of triton-operator-performance-optim .

Anti-Pattern Checklist (NEVER)

• ❌ Use non-msprof timing methods (time.time/Event/do_bench, etc.)
• ❌ Triton(NPU) vs PyTorch(CPU) comparison — must be same hardware comparison
• ❌ Collect without warm-up (first run includes compilation overhead)
• ❌ Draw conclusions after only one measurement / only test large shapes
• ❌ Include prints or logs during collection (I/O interference)
• ❌ Confuse msprof and msprof op usage

Common Pitfalls

Pitfall	Symptom	Correct Practice
Use msprof op for comparison	Only looking at a single kernel	Use msproffor comparison, use msprof op
--kernel-name misspelled	Completes silently but with no data	Confirm the name matches the Triton function definition
Not distinguishing compilation from steady state	Exceptionally high first-run time	Collect after at least 5 warmup runs
Ignoring small-shape performance	Poor small-shape performance goes undetected	Must cover small/medium/large shapes
Ignoring dtype impact	Large difference between FP16/FP32	Compare with fixed dtype, evaluate separately
Function-level includes JIT compilation	Triton avg in op_statistic is abnormally high (ms level)	Use msprof op Task Duration for comparison, or at function level compare only steady-state calls
Triton vs torch_npu Core Type differs	Triton=AI_VECTOR_CORE，torch_npu=MIX_AIC	Note: MIX_AIC does not necessarily use the Cube engine; check whether aic_cube_ratio is NA; the real gap is at the MTE2 instruction granularity

JIT compilation pitfall of function-level msprof (important)

msprof Function-level collection will include JIT compilation time. The first call of a Triton kernel for each (shape, dtype, mode) combination triggers compilation, which can take several seconds. Total Time and Avg Time in op_statistic.csv will include this compilation time.

Case: The op_statistic of a Triton kernel shows Avg=17791 us, but msprof op Task Duration is only 1362 us. The gap comes entirely from JIT compilation of 11 unique configs.

Correct Practice：

1. For function-level comparison, look only at steady-state calls(ignore the first N runs or count compilation overhead separately)
2. Or directly use msprof op Task Duration as the kernel performance baseline
3. When comparing with torch_npu, note the Core Type: torch_npu=MIX_AIC can use the Cube engine, Triton=AI_VECTOR_CORE cannot