Operator Classification System
| Category | Characteristics | Typical Operators | Design Guide |
|---|---|---|---|
| Reduction category | Reduction along an axis (including index-tracking variants) | ReduceSum, Softmax, LayerNorm, ArgMax | ✅ Scenario routing(⚠️ must read first) / Algorithm |
| Elementwise category | Input and output Shapes are the same, elementwise independent computation | Sin, Cos, Abs, Exp | ✅ Scenario routing(⚠️ must read first) |
| Broadcast category | Input Shapes differ, broadcast alignment required | Add, Mul, Sub | ✅ Scenario routing(⚠️ must read first) |
| Conversion data conversion category | Change layout/shape, merge/split tensors | Transpose, Concat, Split | ⚠️ Scenario routing(currently only partially supported for Transpose) |
| Random category | Generate random numbers, seed management required | RandomUniform, Dropout | 📋 In planning |
| MatMul matrix multiplication category | Matrix multiplication, high computational density, uses the Cube unit | MatMul, BatchMatMul | ✅ Scenario routing(mxfp8 + eltwise fusion; other matmul forms are in planning) |
| Convolution category | Spatial convolution, sliding window computation | Conv2D, DepthwiseConv | 📋 In planning |
| NN neural network category | Dedicated to neural networks, combinations of multiple operations | FlashAttention, GroupNorm | 📋 In planning |
General Design Elements (required for all categories)
The following design elements must be considered and completed for all operator categories:
1. Multi-core partitioning strategy
Core question: How are tasks allocated to multiple AI Cores?
Design points：
- Load balancing: the task amount processed by each core should be as equal as possible
- Data locality: adjacent data should be allocated to the same core as much as possible
- Moderate granularity: tiles cannot be too small (high scheduling overhead) or too large (low parallelism)
Output：
- Total task partitioning method (which dimension to partition along)
- Task amount processed by each AI Core
- Number of AI Cores used
2. UB partitioning strategy
Core question: How much data can be processed at a time?
Design points：
- UB capacity limit (DAV_2201: 192KB, DAV_3510: 248KB)
- Amount of data processed at a time
- Whether chunked processing is needed
Output：
- Amount of data processed at a time
- Whether chunking is needed
- Chunk size calculation formula
3. Buffer planning
Core question: Which buffers are needed? How large is each?
Design points：
- Input buffer (inQueue)
- Output buffer (outQueue)
- Intermediate computation buffers (tmpBuf, workBuf, etc.)
- Double Buffer optimization
Output：
- Buffer list and purposes
- Formula for calculating the size of each Buffer
- Total UB usage
4. Branch scenario coverage
Core question: Which different scenarios need to be handled?
Common branch dimensions：
- Data type: FP32 / FP16 / BF16 / INT8
- Shape size: large shape / small shape
- Data alignment: 32-byte aligned / unaligned
- Boundary cases: minimum value / maximum value / special values
Output：
- Branch decision conditions
- Handling strategy for each branch
- Boundary test cases