SuperAPI AI Directory

tech-evaluation

General & other

View repository

Updated 10-15 10:19

About this skill

Tech Evaluation

The core of selection is not "which is better" but "which is more suitable in our scenario." There is no best technology, only the most suitable technology.


Step 1: Define the selection problem

Use AskUserQuestion to clarify the following information:

Element What to ask Why it matters
Problem to solve What is the selection meant to solve? Anchor evaluation criteria
Candidate solutions What candidates are there already? Do you need help discovering more? Determine evaluation scope
Hard constraints Conditions that must be met (license, language, compatibility) First eliminate unqualified options
Priority dimensions What is valued most? (performance / ecosystem / learning cost / cost) Determine weights
Decision time How deep is needed? Quick judgment or in-depth evaluation? Control investment

Step 2: Quick screening

2.1 Eliminate unqualified options

Use hard constraints for the first round of screening:

## 候选方案筛选

| 候选 | 约束 1(MIT 许可) | 约束 2(支持 TS) | 约束 3(活跃维护) | 结果 |
|------|-------------------|-------------------|-------------------|------|
| 方案 A | ✅ | ✅ | ✅ | 进入评估 |
| 方案 B | ✅ | ❌ | ✅ | 排除 |
| 方案 C | ✅ | ✅ | ❌(2 年无更新) | 排除 |

2.2 Quick judgment path

If after screening only 1-2 candidates remain, and the differences are obvious:

  • Directly give a recommendation + reasons; no need for a full evaluation
  • Just record the decision in the spec document

If after screening there are 2-3 evenly matched candidates → proceed to Step 3 full evaluation.


Step 3: Multi-dimensional evaluation

3.1 Build an evaluation matrix

Based on the dimensions the user cares about, build a weighted matrix:

## 评估维度与权重

| 维度 | 权重 | 说明 |
|------|------|------|
| 功能匹配度 | 30% | 是否满足核心需求 |
| 性能 | 25% | 对应场景下的实际表现 |
| 生态与社区 | 20% | 文档质量、社区活跃度、第三方集成 |
| 学习成本 | 15% | 团队上手难度 |
| 运维成本 | 10% | 部署复杂度、监控、升级成本 |

How weights are determined:

  • User explicitly stated priorities → use them directly
  • User did not say → give suggested weights and have the user confirm

3.2 Evaluate dimension by dimension

For each dimension, usefacts and datato evaluate, not "feelings":

Dimension How to evaluate Data source
Feature fit List requirements and check item by item whether each candidate supports them Official documentation, GitHub issues
Performance Find benchmark data, or run PoC tests yourself Official benchmark, third-party reviews, self-testing
Ecosystem and community GitHub stars/issues response speed, npm weekly downloads, Stack Overflow question count GitHub、npm、Stack Overflow
Learning cost Documentation quality, whether the API design is intuitive, whether there is a migration guide Official documentation, tutorial resources
Operations cost Deployment method, configuration complexity, upgrade history (whether there are breaking changes) CHANGELOG, upgrade guide

3.3 PoC validation (optional but recommended)

For key dimensions,writing code to verifyis more reliable than reading documentation:

## PoC 验证

### 验证目标
用方案 A 和方案 B 分别实现 <核心场景>,对比:
- 代码量和复杂度
- 实际性能数据
- 遇到的坑

### 验证结果
| 指标 | 方案 A | 方案 B |
|------|--------|--------|
| 代码行数 | 120 行 | 85 行 |
| 响应时间(p95) | 23ms | 18ms |
| 遇到的问题 | 文档缺失,靠看源码 | 顺利,文档完整 |

A PoC does not need to implement full functionality; it only needs to verifythe most uncertain dimensions。


Step 4: Comprehensive Scoring and Decision

4.1 Score Summary

## 综合评分

| 维度 | 权重 | 方案 A | 方案 B | 方案 C |
|------|------|--------|--------|--------|
| 功能匹配度 | 30% | 9 | 8 | 7 |
| 性能 | 25% | 7 | 9 | 8 |
| 生态与社区 | 20% | 8 | 7 | 9 |
| 学习成本 | 15% | 6 | 8 | 7 |
| 运维成本 | 10% | 7 | 8 | 6 |
| **加权总分** | | **7.65** | **8.05** | **7.50** |

4.2 Give a Decision

## 选型决策

**推荐:方案 B**

### 核心理由
- 加权总分最高(8.05)
- 在最看重的性能维度(权重 25%)上明显领先
- PoC 验证中开发体验最好

### 取舍说明
- 放弃方案 A 的原因:学习成本较高,团队没有相关经验
- 放弃方案 C 的原因:社区活跃但功能匹配度不足

### 风险提示
- 方案 B 的社区规模较小,未来可能面临维护风险
- 建议:核心功能不过度依赖其独有特性,保持可替换性

Step 5: Output the Selection Report

Write the selection conclusion into the spec document (invoke the writing skill's technical document mode), including at least:

  • Background and Motivation: why selection is needed
  • Candidate Solutions: what options exist
  • Evaluation Process: evaluation dimensions, weights, data sources
  • PoC Results: if verification was performed
  • Decision and Trade-offs: what was chosen, why it was chosen, and what was given up

Selection Principles

  • Scenario First: there is no "best" technology, only "the most suitable for the current scenario"
  • Let Data Speak: every score must be based on facts, not impressions
  • Verify Uncertainty: verify the most uncertain dimensions with a PoC, not guesswork
  • Consider the Team: good technology alone does not mean the team can use it well; learning cost is a real cost
  • Leave a Way Out: prioritize non-locking solutions and maintain replaceability
  • Do Not Over-Evaluate: if 2 candidates differ clearly, choose directly; no need for a 5-dimension 10-point matrix

Relationship with Other Skills

task-start(方案阶段遇到选型)→ tech-evaluation(评估决策)
deep-research(需要广度调研时)← tech-evaluation 按需调用
tech-evaluation → writing(输出选型报告到 spec)

tech-evaluation vs deep-research:

  • tech-evaluation:Must give a conclusion. "Choose A, because XYZ"
  • deep-research:Does not necessarily give a conclusion. "The current market landscape is like this, and the trend is like that"
  • If you do not understand the candidate solutions well enough before selection, you can first use deep-research for investigation, then use tech-evaluation for decision-making