SuperAPI AI Directory

skill-trace-checker

Coding & development

View repository

Updated 10-15 21:48

About this skill

Skill TRACE Quality Evaluation Process

Mandatory rule: after creating or modifying a Skill each time, this TRACE self-check must be completed, and only after all sub-items reach the full-score standard is the task considered complete. It is not allowed to deliver a Skill when the score has not met the standard, nor is it allowed to only generate a good-looking report without truly improving the Skill content.


⚡ Beginner 30-Second Quick Start

What is this Skill for? It helps you check whether the Skill you wrote is good enough in quality, where it falls short, and how to improve it.

When is it triggered?

  • Just finished writing a new Skill and want to know whether to revise it → use it directly
  • Modified the core content of an existing Skill → use it
  • The user requests "check Skill quality" → use it

Just copy these trigger examples to get started:

✅ "帮我检查刚写的 opc-skill 是否符合 TRACE 标准"
✅ "对这个 SKILL.md 做全量 TRACE 自检"
✅ "检查一下 skill-trace-checker 本身的质量有没有问题"
✅ "我修改了招聘 Skill 的 FAQ,重新评测一下"
✅ "这个 Skill 的反模式章节写得够好吗?帮我打分"

One-sentence process: Read the Skill file → score item by item against the 20 check items → find sub-items < 5.0 → revise → re-score → finish only when all are 5.0.


Capability Boundary Description

✅ Good at handling

  1. Checking newly created Skills: read SKILL.md and references/ files, score dimension by dimension, and output a structured scoring table
  2. Pinpointing specific reasons for point deductions: for each sub-item below 5.0, give precise guidance on "what the problem is, which file it is in, and how to fix it"
  3. Verifying whether revisions are effective: re-score after revision to confirm substantive improvement rather than only wording changes
  4. Batch-checking multiple files: comprehensively evaluate multiple files under SKILL.md + references/ at the same time
  5. Comparing differences between two versions: determine whether the new version has substantive improvement over the old version in a certain sub-item
  6. Generating a standard scoring table: output a TRACE scoring table in a unified format for easy recording and tracking

⚠️ Requires materials to do

  1. Full self-check: requires the SKILL.md file path or complete content; it cannot be evaluated from the Skill name alone
  2. Targeted suggestions: requires a description of the Skill's usage scenario (what users it serves and what problem it solves); otherwise only general suggestions can be given
  3. Multi-version comparison: requires the file content of two versions to determine the degree of improvement

❌ Out of scope (with alternatives)

  1. Writing Skill content for you: this Skill only scores; it does not generate Skill content for you → use skill-creator to create a Skill
  2. Evaluating non-Skill documents: such as README, product documentation, code quality → use the corresponding code review tool or document review Skill
  3. Executing the Skill's function: only responsible for quality evaluation, and does not execute the actual task in place of the Skill being checked
  4. Automatically publishing or installing a Skill: publishing operations after passing evaluation need to be completed manually

Execution Timing

After any of the following situations occurs,immediatelyexecute this checking process:

  1. Completed writing the SKILL.md of a new Skill
  2. Major content modifications to an existing Skill (adding modules, modifying behavior specifications, adjusting FAQs, etc.)
  3. The user explicitly requests "check Skill quality" or "TRACE evaluation"

TRACE five-dimension scoring standard

Score each sub-item from 1.0 to 5.0. If it does not reach 5.0, the Skill content must be immediately modified until it reaches full marks.


T · Trust (Target: 5.0)

Sub-item Full-mark standard Inspection method
Domestic suitability Fully Chinese interface; supports colloquial Chinese input; examples based on real domestic scenarios (WeChat/Taobao/Xiaohongshu, etc.) Check whether all examples and descriptions are in Chinese, and whether trigger words include Chinese keywords
Security No P0/P1 security risks; does not guide users to operate others' accounts; does not process real sensitive data Check whether the system prompt has a "Prohibited Behaviors" section and whether it includes privacy protection instructions
Boundary transparency Clearly distinguish the three categories of what can be done / requires materials / out of scope, and each category has specific examples Whether SKILL.md has an independent "Capability Boundary Description" section
Data privacy specifications The FAQ or a dedicated section clearly explains the principles for handling sensitive data and provides guidance on anonymization operations Check whether there are privacy/data security-related instructions

Common reasons for point deductions:

  • All examples are English scenarios
  • No privacy usage instructions
  • Boundary descriptions are vague (only saying "some scenarios are not supported")

R · Reliability (Target: 5.0)

Sub-item Full-mark standard Inspection method
Exception handling All error prompts use user language rather than technical error messages; the format is "missing [specific item] + how to supplement it" Check whether the system prompt has precise error prompt specifications and whether vague prompts are explicitly prohibited
Functional completeness Full coverage of main usage scenarios, with no obvious functional blind spots; each function has output format instructions Count the feature points and compare them with typical user needs to see if anything is missing
Operational stability When input is ambiguous, there is a clear degradation strategy (first provide an assumed version, then ask), without hanging or giving an empty response Whether the system prompt has handling rules for "when information is insufficient"
Degradation fallback When out of scope, provide guidance to alternative tools; when there are multiple tasks, sort them by priority Whether the capability boundary section explains how to handle out-of-scope situations

Common reasons for point deductions:

  • Errors directly throw technical exception information (ImportError / File not found)
  • When user input is incomplete, the AI directly stops and asks, without providing an assumed version
  • Out-of-scope requests are directly rejected, with no alternatives

A · Adaptability (Target: 5.0)

Sub-item Full-mark standard Inspection method
Capability boundary definition Three categories (✅ Good at / ⚠️ Requires materials / ❌ Out of scope) and each category has more than 3 specific examples Check the number of categories and the number of examples in the boundary section
Trigger Precision Each feature has "precise trigger conditions", including main triggers and submodule routing; users can determine which feature to use Check whether each feature module has independent trigger condition descriptions
Audience Breadth Clearly state the applicable user types, and explain how non-primary users (such as teams) can use it Check whether there is audience applicability guidance
Customization Support Provide mechanisms for users to convey personal preferences (such as style cards, scenario parameters) Check whether the FAQ or documentation contains customization usage guidelines

Common reasons for point deductions:

  • The trigger method is only a keyword list, with no judgment logic for "which feature to use in which situation"
  • No explanation of how different types of users should use it

C · Convention Normativity (Target: 5.0)

Sub-item Full-score standard Inspection method
Progressive Disclosure At least a three-layer structure (quick start → feature details → in-depth reference); users can go deeper as needed Check whether SKILL.md has a quick navigation or introductory chapter, and whether there are in-depth documents in references/
Structural Clarity File names are self-explanatory; SKILL.md has a clear internal hierarchy; tables/code blocks/collapsible sections are used appropriately Check whether the file naming in references/ is semantically clear
Anti-pattern Explanation At least 3 types of common incorrect usage + improved example comparisons; a list of prohibited usages Check whether references/anti-patterns.md exists and is substantial in content
FAQ Depth The main document FAQ covers general questions (≥6 questions), and the in-depth FAQ covers edge scenarios (≥8 questions) Count the number of FAQ questions, and check whether references/faq-deep.md exists

Common reasons for point deductions:

  • Only SKILL.md, with no in-depth documents in references/
  • The FAQ has only 3–4 questions, with insufficient coverage
  • There are no anti-pattern cases, so users do not know what practices will lead to poor output

E · Effectiveness (Target: 5.0)

Sub-item Full-score standard Inspection method
Output Accuracy The system prompt has a rule of "prohibit making things up in uncertain areas"; each output states the decision logic Check whether the system prompt contains output accuracy constraints
Content Completeness The number of modules × sub-feature points comprehensively covers typical scenarios; the in-depth FAQ supplements edge scenarios Whether the number of feature points covers more than 80% of common needs in the field
Creativity and Added Value Has value-added features beyond basic functions (style customization/multi-platform adaptation/proactively asking for adjustments/intelligent classification, etc.) List the value-added feature points in the Skill that go beyond "just completing the task"
Out-of-the-box Usability Has a beginner introductory chapter; has directly copyable opening examples (≥3); zero learning cost for first use Check whether there is a "Beginner Introduction" or "Quick Start" chapter, and whether there are example opening lines

Common reasons for point deductions:

  • There is no beginner onboarding guide, so users do not know where to start
  • The output examples are not realistic enough (generic template placeholders are used instead of real scenario examples)
  • It lacks any value-added features and merely executes instructions mechanically

Scoring and Revision Process

1. 逐项打分(1.0~5.0)
2. 找出所有 < 5.0 的子项
3. 针对每个扣分点,修改 SKILL.md 或 references/ 文件
4. 重新对修改后的内容打分
5. 重复 2~4,直到全部子项 = 5.0
6. 在回复中输出最终评分表,确认达标

⚠️ Shortcuts prohibited: It is not allowed to cover up actual point deductions by "polishing the wording of the report." Every sub-item below 5.0 must correspond to a substantive change in SKILL.md or a references file.


FAQ

Q1: Can this Skill only be used by the Skill author? Yes. This Skill is intended for Skill creators and maintainers. Ordinary users generally do not need to trigger it in everyday conversation scenarios.

Q2: I just changed a few lines of text. Do I also need to redo TRACE? If you modified content that affects the user experience, such as feature descriptions, FAQ, boundary conditions, or trigger words → yes; if you only corrected typos or formatting → no.

Q3: Are TRACE results scored automatically by AI or manually? This process is executed by AI against fixed standards. It is consistent but is not guaranteed to be completely consistent with Skillhub's official evaluation results. Official evaluation involves user behavior data, which is an additional dimension.

Q4: I revised a certain sub-item repeatedly, but AI still does not give it 5.0. What should I do? Refer to references/trace-criteria-detail.md for the "4.5 vs 5.0 real comparison case" for that sub-item and use it for self-check. The comparison case shows the specific formatting requirements for exactly 5.0.

Q5: If a file under references/ does not exist, how should that sub-item be scored? Does not exist = the standard for that item is not met = at most 4.0. The corresponding file must be created and filled with substantive content to get 5.0.

Q6: Can I check only one dimension? Yes. Clearly state "only check E·Effectiveness" or "only look at the FAQ depth item," and this Skill will focus on scoring that dimension and provide improvement suggestions.


Audience Description

User Type How to Use
Individual Skill Developer Use directly; provide the Skill file path or paste the content
Team Skill Maintainer Use as a CI check step before merging changes to ensure the quality baseline
Skill Reviewer Use this standard as a unified review framework to ensure consistent review criteria
Beginner (just wrote their first Skill) First read the "30-second beginner introduction" section, then refer to references/sample-reports.md for complete examples

Customization Usage Guide

The following parameters can be passed in at trigger time to customize the checking behavior:

  • Strict Mode:"严格评测" → All sub-items must reach 5.0 to pass; intermediate scores are considered failing
  • Quick Mode:"快速检查" → Output only sub-items below 5.0, skipping explanations for full-score sub-items
  • Focused Mode:"只检查 [维度名]" → Focus on a single dimension and skip other dimensions
  • Comparison Mode:"对比修改前后" + provide two versions of the file → output a difference analysis table

Reference Documents

  • references/trace-criteria-detail.md — Detailed scoring rules and cases for each TRACE sub-item
  • references/skill-checklist.md — Quick self-check checklist (one-page version)
  • references/anti-patterns.md — Common incorrect practices + improvement comparison cases
  • references/faq-deep.md — In-depth FAQ (edge cases/tool compatibility/security compliance)
  • references/examples.md — Complete usage example (trigger → inspection process → output triple)
  • references/sample-reports.md — Real scoring report case (qualified/unqualified/comparison after modification)

Usage workflow

Step 1: Environment preparation

Ensure the necessary dependencies and tools are installed in the development environment.

Step 2: Configuration initialization

Perform basic configuration according to project requirements.

Step 3: Core function usage

Implement the core functions according to the sample code.

Step 4: Test verification

Run tests to ensure the functions work properly.

Step 5: Deployment and launch

After completing development, proceed with deployment and monitoring.