Evaluation model: script calculates deterministic base score + AI semantically calibrates after reading content (±0.3) → final score. For detailed scoring criteria see references/scoring-criteria.md, for calibration rules see references/calibration-guide.md。
⚡ Beginner 30-second quick start
What does it do? Perform TRACE five-dimension quality scoring on any Agent Skill, output an evaluation report with sub-item scores.
When is it triggered?
- Just finished writing a new Skill and want to know how good the quality is → use directly
- User requests "check Skill quality", "TRACE evaluation", "skill scoring"
- Provides a Skill directory path + keywords like "evaluate"/"evaluation"/"score"
Trigger examples:
✅ "用 TRACE 评测 /path/to/skill"
✅ "TRACE 评测 ddd-architecture-awesome"
✅ "对 skill-trace-evaluation 做五维度评估"
✅ "生成 TRACE 报告 + HTML 雷达图"
✅ "严格评测 jimeng-prompt-text2image" （全部子项需 5.0）
✅ "快速检查这个 Skill 有哪些扣分项" （仅输出扣分项）
✅ "只检查 T 维度" （单维度聚焦）
✅ "我修改了 FAQ，重新评测一下规范性"
✅ "输出 TRACE 五维画像 + 基线对比"
✅ "技能评估报告 + 官方合规检查"
One-sentence flow: Run the script to get the base score → read SKILL.md → calibrate against the scoring details → produce a report.
Execution timing
The following scenarios are suitable for triggering TRACE evaluation:
- New Skill finished being written: Just finished writing SKILL.md and want to know the quality baseline
- After major Skill modifications: Modified core content such as feature descriptions, FAQ, boundary conditions, trigger words, etc.
- User explicitly requests: "check Skill quality", "TRACE evaluation", "skill scoring"
- Third-party review: Platform reviewers or community users perform quality evaluation on a Skill
Capability boundary description
✅ Good at handling
- Evaluate any Agent Skill, output 20 sub-item scores
- Locate specific reasons for point deductions, each score accompanied by evidence
- Generate standardized reports (dimension-level Chinese comments + sub-item score table + improvement suggestions)
- Verify whether modifications are effective: re-evaluate after modification, compare score changes before and after
- Compare differences between two versions: determine in which sub-items the new version has substantial improvement over the old version
- Supports four modes: standard/strict/quick/single-dimension
⚠️ Requires materials
- Complete evaluation requires a Skill directory path (including SKILL.md); evaluation is impossible based on name alone
- R1/E1/E3 scoring requires the AI to read the main text itself to judge semantic quality—the script only provides structural base scores
❌ Out of scope (with alternatives)
- Help you write Skill content → use
skill-awesome(specification knowledge) orskill-trace-checker(self-check before publishing) - Evaluate non-Skill documents → find the corresponding tool
- Automatically publish Skill → complete manually
Evaluation process
Step 1 ──── Step 2 ──── Step 3 ──── Step 4
收集基分 阅读技能 逐项校准 产出报告
────────────────────────────────────────────
trace_ev- AI 直接 基分 ±0.3 Markdown
aluate.py 阅读正文 附调整理由 + 可选 HTML
Step 1: Collect base scores
python3 scripts/trace_evaluate.py --skill-dir <path> --format json
Output includes base_scores: each sub-item base（1.0-5.0）、formula(calculation formula),evidence(evidence field).
Step 2: Read the skill
You must read the SKILL.md main text yourself + scan the references/ and examples/ directories. The script provides structural data; the AI judges content quality.
Example of the AI's checking approach when reading:
【T 维】
安全 → 查有无密钥/secrets/脚本，正文有无安全声明
国内 → 查全文中文化程度、示例是否基于国内平台
边界 → 查有无独立边界章节、三分类是否每类≥3例
隐私 → 查有无数据隐私说明（FAQ 或专项章节）
【R 维】
异常 → 查 Gotchas 是否包含"交互式引导模板"（先假设版本→列缺失项）
功能 → 查 workflow 步骤是否覆盖所有声明功能
稳定 → 查有无 validate-plan-execute 循环或等效约束
降级 → 查边界章节中超范围后是否给替代方案
【A 维】
边界定义 → 查三分类是否有场景化判断逻辑（"什么时候该用/不该用/模糊怎么判"）
触发 → 查 description 信息量，是关键词堆砌还是场景化路由
受众 → 查有无显式说明适用用户类型
定制 → 查有无风格/参数传递机制
【C 维】
文档 → 查 examples 数量是否达标（prompt≥10/cli≥4/doc≥5）
披露 → 查 body 行数 + references 文件数，是否三层结构
结构 → 查 name 规范 + refs 子目录≥2
反模式/FAQ → 查 Gotchas 数量是否≥5 + FAQ 是否≥6且非充数
【E 维】
准确 → 查有无"禁止胡编"规则或等效约束
完整 → 查 examples 数量是否达阈值（prompt≥25/cli≥4/doc≥5）
增值 → 查 refs 子目录≥2 + 是否有评估框架/决策树等深度领域知识
开箱 → 查有无快速开始章节 + ≥3 个可复制开场白
Step 3: Calibrate item by item
Check base → read the full-score standard for that sub-item in references/scoring-criteria.md + deduction reasons → determine which tier the main text content matches → according to references/calibration-guide.md Adjustment (±0.3) → add a sentence stating the reason for the adjustment.
Step 4: Produce the report ⚠️ Mandatory
After scoring is complete, a report must be output; it is not allowed to only verbally report the score without writing it to a file.
The report uses a unified template. The template file contains {...} placeholders; replace them with the actual evaluation results and then write to the target file.
Report template
| Format | Template file | Description |
|---|---|---|
| Markdown | examples/trace-report.generated.md |
Plain text, console-readable, suitable for archiving and diff |
| HTML | examples/trace-report.generated.html |
Visualized radar chart + Tailwind cards, suitable for browser display and sharing |
Let the user choose the format
After scoring is complete, you must use the AskUserQuestion tool to ask the user to choose the output format:
问题：评估完成，请选择报告输出格式
选项：
- "Markdown 报告" — 使用 examples/trace-report.generated.md 模板，输出纯文本报告
- "HTML 报告" — 使用 examples/trace-report.generated.html 模板，输出含 SVG 雷达图的交互式报告
- "两者都要" — 同时输出 Markdown + HTML
After the user chooses:
- Choose "Markdown" → based on the
trace-report.generated.mdtemplate, fill in the data → write to{skill-dir}/trace-report.md - Choose "HTML" → based on the
trace-report.generated.htmltemplate, fill in the data → write to{skill-dir}/evaluation-report.html - Choose "Both" → generate both of the above files at the same time
After writing is complete, you must report the file path to the user.
Markdown report structure
- Overall score table (5 dimensions + overall)
- One-sentence conclusion
- Detailed analysis of the five dimensions (each dimension has 4 sub-items: score + evidence + suggestions)
- no-skill baseline comparison table
- Official specification compliance table (10 agentskills.io checks)
- Optimization suggestions (priority order P1/P2/P3)
- Skill basic profile
HTML report includes
- SVG radar chart (T/R/A/C/E five axes)
- Five-dimension score cards + progress bars
- Detailed table of 20 sub-items (each sub-item: score + evidence + suggestions)
- Baseline comparison + official compliance + optimization suggestions + Skill profile
- Supports Print and mobile responsive layout
Improvement guidance after evaluation
TRACE evaluation is not just about giving a score; the ultimate goal is to help the Skill achieve higher quality. After the evaluation report is produced:
- Locate the point deductions: check the sub-items in the report with < 5.0, and compare them against the scoring-criteria.md "full-score standard" and "revision comparison" examples for that sub-item
- Modify the corresponding files: each point deduction must correspond to a substantive change in SKILL.md or a references/ file (it is forbidden to only change the wording of the report without modifying the files)
- Re-evaluate and verify: after modification, run the evaluation again to confirm that the score has substantially improved
- Reference cases: when you do not know how to change it, check the trace-anti-patterns.md for the correct way to fix similar problems
A score of 4.8 or above can be regarded as meeting the practical standard; there is no need to endlessly pursue a literal 5.0.
Scoring criteria quick reference
The full scoring rules are in references/scoring-criteria.md, and the full calibration rules are in references/calibration-guide.md. Below is a summary of the base score formula + full-score standard for the 20 sub-items.
T · Trust (trustworthiness)
| ID | Name | Base score formula | Full-score standard summary |
|---|---|---|---|
| T1 | Security scan | 4.5 + no_scripts(+0.3) + secdecl(+0.2) - secrets(-1.0) |
No keys + no scripts + has a security statement |
| T2 | Domestic adaptability | has_chinese ? 5.0 : 2.0 |
All Chinese, examples based on domestic platforms |
| T3 | Boundary transparency | boundary ? 4.5+0.3(when_to_use) : 3.0 |
Three-category classification (✅/⚠/❌), ≥3 examples per category |
| T4 | Data privacy standards | 4.5 + secdecl(+0.5) - scripts(-0.2) |
State "no data collection/no data processing" |
Common reasons for T-dimension deductions: All examples are English scenarios; no privacy usage description; boundary descriptions are vague (only saying "some scenarios are not supported").
R · Reliability
| ID | Name | Base score formula | Full-score standard summary |
|---|---|---|---|
| R1 | Exception handling | gotchas ? 4.0+0.5(val) : 3.0 |
Gotchas + interactive guidance template |
| R2 | Functional completeness | WF ? 4.5+0.3(steps≥5或exs≥10) : 3.5~4.0 |
Workflow + scenarios cover everything with no blind spots |
| R3 | Operational stability | 4.0 + gotchas/rules(+0.3) + val(+0.2) |
Constraint rules + validation checklist |
| R4 | Degradation fallback | boundary ? 4.5 : 3.5 |
Guide to alternative tools when out of scope |
Common reasons for R-dimension deductions: When user input is incomplete, the AI directly stops to ask without first providing an assumed version; out-of-scope requests are directly refused without guidance to alternatives.
A · Adaptability
| ID | Name | Base score formula | Full-score standard summary |
|---|---|---|---|
| A1 | Capability boundary definition | boundary+when_to_use ? 4.5 : 4.0 |
Scenario-based judgment logic |
| A2 | Trigger method | desc≥100 ? 5.0 : 3.5~4.5 |
Scenario-based routing, not keyword stacking |
| A3 | Audience breadth | 4.0 + chinese(+0.3) |
Explicitly state applicable user types |
| A4 | Customization support | type=cli或exs≥10 ? 4.5 : 4.0 |
Style cards/scenario parameters/configuration mechanisms |
Common reasons for A-dimension deductions: Trigger methods are only keyword lists, with no judgment logic for "which feature to use in which situation"; no explanation of how different types of users should use it.
C · Convention
| ID | Name | Base score formula | Full-score standard summary |
|---|---|---|---|
| C1 | Documentation quality | exs≥阈值(cli=4/prompt=10) ? 5.0 : 3.5~4.5 |
Examples can be copied directly, format is standardized |
| C2 | Progressive disclosure | refs≥8+body<200 ? 5.0 : 3.0~4.5+th(+0.2) |
Three-layer structure, get started in 30 seconds |
| C3 | Clear structure | 4.5 + name(+0.2) + refsd≥2(+0.3) |
refs subdirectory ≥2 |
| C4 | Anti-patterns and FAQ | gotchas ? 4.5+0.3(gc≥5) : 3.5 |
Gotchas ≥5 + FAQ ≥6, not filler |
Common reasons for C-dimension deductions: No references/ in-depth documentation; FAQ has only 3~4 questions, insufficient coverage; no anti-pattern cases, so users do not know which practices will lead to poor output.
E · Effectiveness
| ID | Name | Base score formula | Full-score standard summary |
|---|---|---|---|
| E1 | Output accuracy | WF ? 4.5+0.3(val) : 4.0 |
Clear scenario distinctions, prohibit making things up |
| E2 | Content completeness | exs≥阈值(prompt=25/cli=4/doc=5) ? 5.0 : 3.0~4.5 |
Full scenario coverage, E2E examples |
| E3 | Creativity and added value | refsd≥2 ? 4.5+0.2(refs≥10) : 4.0 |
Evaluation framework/decision tree/domain knowledge system |
| E4 | Out-of-the-box usability | 4.0 + WF(+0.3) + val(+0.3) + gotchas(+0.2) + cli(+0.2) |
Quick start chapter + copyable opening lines |
Common reasons for E-dimension deductions: No beginner onboarding guidance, so users do not know where to start; output examples are not realistic enough (using generic templates rather than real scenarios); lack of value-added features, only mechanically executing instructions.
Customization Usage Guide
The following parameters can be passed in at trigger time to customize evaluation behavior:
| Mode | Trigger keywords | Behavior |
|---|---|---|
| Standard mode | "TRACE 评测"(Default) |
Full 20-subitem evaluation, output a complete report |
| Strict mode | "严格评测" |
All subitems must reach 5.0; intermediate scores are considered failing |
| Quick mode | "快速检查" / "只看扣分项" |
Only output deduction items with < 4.5, skip full-score explanations |
| Focused mode | "只检查 [维度]" |
Focus on a single dimension, skip other dimensions |
| Comparison mode | "对比修改前后" |
Provide two versions of the path → output a difference comparison table |
| HTML report | "生成雷达图" / "可视化" |
In addition to Markdown, additionally generate HTML containing a radar chart |
Calibration rules quick reference
| # | Rule | Description |
|---|---|---|
| 1 | Adjust only when there is a semantic signal | Full-score patterns in the body → adjust upward; deduction patterns → adjust downward. If neither matches → do not adjust |
| 2 | Adjustment range ±0.3 | Single subitem max ±0.3. 4.5 cannot jump to 5.0 (can only reach 4.8) |
| 3 | Attach adjustment rationale | One sentence in Chinese, citing specific content |
| 4 | Deduplication downgrade | ref_names ≥3 same-version prefixes → C2/E2 downgrade by one level (max 4.5) |
| 5 | Perfect threshold buffer | All 4 subitems in the same dimension can reach 5.0 → check for hidden flaws → if present, max 4.8 |
| 6 | Type awareness | CLI type C2=150 lines E2=4exs. Already built into the script |
Overall score
Average of the five dimensions = Overall.
| Rating | Score |
|---|---|
| Excellent (优秀) | ≥ 4.5 |
| Good (良好) | 3.5 – 4.4 |
| Needs improvement (需改进) | < 3.5 |
Report template
# TRACE Evaluation Report
Target: `<path>`
## Overall Assessment
**Overall Score: X.X / 5**
**Overall Rating: Excellent (优秀) / Good (良好) / Needs improvement (需改进)**
One-sentence conclusion: ...（自然中文，具体不空洞）
## TRACE Dimension Explanation
SkillHub TRACE 评测体系从**可信任度、可靠性、适用性、规范性、有效性**五个维度全面评估。
[了解详情](https://skillhub.cn/tutorials#trace-evaluation)
评测基于 AI 自动化检测，结果供参考。
## Evaluation Details
### T · Trust — X.X / 5
（1-2 句中文）
| Sub-item | Score | Commentary |
|----------|-------|------------|
| 安全性扫描 | X.X | 一句话证据 |
| 国内适配性 | X.X | ... |
| 边界透明度 | X.X | ... |
| 数据隐私规范 | X.X | ... |
### R · Reliability — X.X / 5
（中文评语）
| Sub-item | Score | Commentary |
|----------|-------|------------|
| 异常处理 | X.X | ... |
| 功能完善性 | X.X | ... |
| 运行稳定性 | X.X | ... |
| 降级兜底 | X.X | ... |
### A · Adaptability — X.X / 5
| Sub-item | Score | Commentary |
|----------|-------|------------|
| 能力边界定义 | X.X | ... |
| 触发方式 | X.X | ... |
| 受众广度 | X.X | ... |
| 定制化支持 | X.X | ... |
### C · Convention — X.X / 5
| Sub-item | Score | Commentary |
|----------|-------|------------|
| 文档质量 | X.X | ... |
| 渐进式披露 | X.X | ... |
| 结构清晰 | X.X | ... |
| 反模式与FAQ | X.X | ... |
### E · Effectiveness — X.X / 5
| Sub-item | Score | Commentary |
|----------|-------|------------|
| 输出准确性 | X.X | ... |
| 内容完整度 | X.X | ... |
| 创造力与增值 | X.X | ... |
| 开箱即用度 | X.X | ... |
## Improvement Suggestions (prioritized)
1. ...（具体、可行动）
2. ...
3. ...
Comment standards
- dimension-level: 1-2 natural Chinese sentences, capture the core. ✅ "Pure documentation type with zero risk, all in Chinese specifically built for Jimeng. Clear boundaries but data privacy is not declared" ❌ "Overall performance is good"
- sub-item: One sentence citing evidence. ✅ "13 Gotchas + checklist, but lacks an interactive guidance template" ❌ "No problem with security"
Suggested wording for each dimension:
| Dimension | Positive wording example | Negative wording example |
|---|---|---|
| T | No hardcoded sensitive information found; security statement is complete | Suspected credential patterns found and need removal; missing data boundary description |
| R | Abnormal input has self-checks and prompts; failure path handling is clear | Lacks self-check prompts for abnormal input; lacks a degradation fallback strategy |
| A | Clearly defines inapplicable boundaries so it is not easily misused; has near-miss prevention | Description is too broad and causes false triggers; capability boundaries conflict with adjacent skills |
| C | Structure follows quick start → process → examples, with details pushed down to references/ | Content is piled into SKILL.md and lacks an index/table of contents; examples are not copyable |
| E | Examples cover mainstream scenarios and can be used directly; can significantly reduce rework | Insufficient guidance for advanced scenarios; can only complete shallow output |
Audience description
| User type | Usage method |
|---|---|
| Skill evaluator | Perform third-party quality assessment of Skills on communities or platforms, and output standardized reports |
| Platform reviewer | Use this standard as a unified review framework to ensure consistent audit criteria |
| Skill Author Self-Check | Self-check before publishing, identify weaknesses and make targeted revisions |
| Community Users | Quickly understand quality before choosing a Skill, or compare multiple Skills of the same type |
Frequently Asked Questions (FAQ)
Q1: The script base score and the AI calibration score are inconsistent. Which one should be used? The score in the final report = base score ± AI semantic calibration. Calibration is only allowed within ±0.3, so the two will not differ too much. If the difference exceeds 0.3, it means the full-score standard description for that sub-item is vague. It is recommended to refer to scoring-criteria.md the "4.5 vs 5.0 comparison" case for that sub-item to find the objective standard.
Q2: I only changed a few lines of text in the Skill. Do I need to redo TRACE? If you modified content that affects the user experience, such as feature descriptions, FAQ, boundary conditions, trigger words, etc. → re-evaluation is required. If you only corrected typos or formatting → not required.
Q3: Are TRACE evaluation results automatically scored by AI or manually scored? The script calculates the deterministic base score (structural detection), the AI reads the main text and performs semantic calibration (±0.3), and the final output is produced comprehensively by AI. It is consistent but not guaranteed to be completely identical to SkillHub's official evaluation results—the official evaluation also involves user behavior data dimensions.
Q4: What should I do if a sub-item still does not receive a full score after repeated evaluations? Refer to references/scoring-criteria.md the "full-score standard" and "4.5 vs 5.0 comparison" case for that sub-item. The comparison case shows the specific formatting requirements for exactly 5.0. After identifying the gap, revise the file accordingly.
Q5: If the file under references/ does not exist, how should that sub-item be scored? Does not exist = the standard is not met = the script base score can be at most 4.0. The corresponding file must be created and filled with substantive content; only then can it possibly reach 5.0 after AI calibration.
Q6: Can I check only one dimension? Yes. When triggering, clearly state "only check E·Effectiveness" or "only look at dimension C", and the evaluation will focus on scoring that dimension and provide improvement suggestions. For in-depth FAQ (questions 7-15), see references/trace-faq-deep.md。
Prohibited List
| Behavior | Reason |
|---|---|
| Adjusting the base score without evidence (purely to pad the numbers) | Calibration must be supported by semantic signals |
| Saying "performs well" without evidence | Every score must have citable evidence |
| Using the same standard for CLI and Prompt types | Type awareness must be applied |
| Awarding a full score for padded FAQ | Read the content to judge substantive value |
| Ignoring deduplication signals | ref_names A prefix ≥3 must be downgraded |
TRACE Evaluation System (Original Text)
The following content comes from the official announcement of the TRACE curated framework jointly released by Tencent Technology, SkillHub, and Tencent Xuanwu Lab on May 21, 2026.
It has been only half a year since Anthropic launched Agent Skills, and the number of Skills on the domestic Skill community SkillHub has already reached the 70,000 level. On May 21, Tencent Technology, SkillHub, and Tencent Xuanwu Lab jointly released TRACE — the first domestic curated evaluation system for real Skill usage scenarios。
T (Trust, safe and trustworthy) — red-line dimension.R (Reliability, reliable operation) — stability, reproducibility, and delivery reliability.A (Adaptability, scenario applicability) — Whether the Agent can naturally recognize and load the target Skill.C (Convention, structural规范) — This is not about judging whether it is written beautifully, but about having the foundation to be understood, run, evaluated, and maintained.E (Effectiveness, effect gain) — The result must be significantly better than the no-skill reference group, and the improvement must be worth the cost.
TRACE is a quality observation coordinate system, adopting a rigorous selection mechanism of "heat signals + time slices + system evaluation + editor's picks", and does not pursue full scoring rankings.
References
- SkillHub TRACE Evaluation System
- references/scoring-criteria.md — Full-score standards for 20 sub-items + reasons for deductions + revision comparisons + five-dimension quick reference (including the original trace-criteria-detail and trace-rubric content)
- references/calibration-guide.md — Calibration rules + anti-pattern cases + adjustment range guide
- references/trace-anti-patterns.md — A collection of anti-pattern cases for Skill creators' self-check (source: skill-trace-checker/东四联周博远)
- references/trace-skill-checklist.md — One-page self-check checklist (condensed version of scoring-criteria) (source: skill-trace-checker/东四联周博远)
- references/trace-sample-reports.md — A collection of real scoring report cases (source: skill-trace-checker/东四联周博远)
- references/trace-faq-deep.md — TRACE in-depth FAQ (source: skill-trace-checker/东四联周博远)
- examples/trace-report.generated.md — Real evaluation output sample (Markdown)
- examples/trace-report.generated.html — Real evaluation output sample (HTML radar chart)
Keywords
English: trace-evaluation, trace-scoring, trace-report, skill-quality, five-dimension-evaluation
Chinese: TRACE evaluation, TRACE scoring, TRACE report, five-dimension assessment, T/R/A/C/E assessment, skill quality evaluation, Trace rigorous selection, rigorous evaluation, quick check