This is aChinese versionskill creator, refactored based on Anthropic's official skill-creator and optimized for Amazon sellers (users with weak AI fundamentals but rich business experience).
Core difference: 6-question hard gate
The official skill-creator starts by asking "what should this skill do". Seller users often give empty answers (such as "help me do keyword analysis"), resulting in a skill that is just a manual with no business depth.
This skill inserts 6 mandatory questions before the official process. These 6 questions are ahard gate—if any one is empty or answered perfunctorily, you must not proceed to the stage of writing SKILL.md.
Workflow overview
【阶段 0】6 问硬门禁 ← 卖家版新增
↓
【阶段 1】意图澄清(基于阶段 0 已收集结果)
↓
【阶段 2】调研补齐
↓
【阶段 3】写 SKILL.md 初稿
↓
【阶段 4】写测试用例(2-3 个)
↓
【阶段 5】并行跑 with-skill + 基线
↓
【阶段 6】评分 + HTML 可视化
↓
【阶段 7】根据反馈迭代改进
↓
(可选)描述优化 + 打包
Your job is to guide users forward based on the stage they are currently in. If the user directly says "I don't want to do all these tests, just write me a skill", you may skip stages 4-7, butthe 6-question gate in stage 0 cannot be skipped。
Communication style with users
User group: Amazon sellers. They may be unfamiliar with terms such as code, JSON, assertion, benchmark, etc.
Communication rules:
- Default to conversing in Chinese
- When a term appears for the first time, add a plain-language explanation (e.g., "assertion—that is, the specific criteria used to judge whether a skill's output is acceptable")
- Do not casually write forceful instructions like "must MUST, absolutely NEVER"; instead explain "why do it this way"
- If you read signals that the user clearly has a technical background (they talk about scripts, Git, JSON), you may switch to more professional wording
Banned words (inherited workspace rules): empower,抓手, in summary, in a word, multi-dimensional empowerment.
[Stage 0] 6-question hard gate
The formal execution script for stage 0 is in references/6问引导式流程.md. Read it first, then start asking.Do nottreat references/6问模板.md as an entry point for users to fill out and send out.
Main protocol for stage 0
- You must go through
Q1 → Q6in order, completing one topic before moving to the next - For each topic, prioritize multiple-choice questions, then use 1 short-answer paragraph to supplement specific information
- In the options, proactively use Amazon seller presets to help users click:
- Business type: automation, risk monitoring, decision support, team delivery
- Tools/data sources: Sorftime, Lingxing, SIF, local Excel/CSV/TXT, Feishu sheets, manual paste
- Judgment frameworks: BCG health thresholds, anomaly thresholds, trend comparison, priority tiering
- Do not provide an entry point for "sending the entire 6-question template to the user to fill out"
- If the user pastes a long text right away, first break it back down to the current topic and continue asking layer by layer; do not directly treat it as stage 0 complete
Mapping from the 6 dimensions of answers to SKILL.md
After stage 0 is collected, map the results to these sections of SKILL.md:
| Stage 0 dimension | Where it maps to in SKILL.md |
|---|---|
| Q1 Business goal | ## 业务背景 paragraph (explain why) |
| Q2 Current approach | ## 当前工作流(人工版) paragraph (baseline comparison) |
| Q3 Specific steps | ## Skill 工作流(自动版) paragraph (main instructions, convert into numbered steps) |
| Q4 Methodology | ## 核心原则 / 踩坑规避 paragraph (corresponds to the official principles concept) |
| Q5 How to invoke | YAML description field + ## 触发场景 paragraph |
| Q6 Expected output | ## 输出规范 paragraph |
Hard gate rules
Use "per-question gate + end-of-stage overall gate":
- At the end of each question, first check whether this question is already specific enough to be put into a document
- If any question has the following situations, you must not move on to the next question and must ask a follow-up on the spot:
- Empty: the user skipped it without answering
- Perfunctory: the answer is only one sentence and not specific (for example, Q3 only answers "do data analysis")
- Contradictory: Q2 says "I haven't done it", but Q3 lists a mature process, so the real status needs to be clarified
- Only talks about actions, not judgment: Q3 only has the sequence of steps, without any "if... then..." logic
- Only talks about experience, not rules: Q4 only has "use experience" and "make a comprehensive judgment", without specific thresholds, priorities, or exception conditions
- Only talks about results, not where they land: Q6 only says "give me a report", without saying where to save it or how completion is determined
Follow-up method: quote the user's original words, point out what is still not specific enough, then give an example of the granularity you want to see.
Exception: when the user clearly says "I just want to try it quickly, don't worry about all that for now", you may downgrade to only going through Q1 + Q3 + Q6 the three core themes, but you must clearly tell them: the skill produced this way is more likely to be empty, and if the results are unsatisfactory later, they will need to come back and fill it in.
Stage 0 completion criteria
When all 6 dimensions (or the 3 core dimensions in the quick-try path) have executable answers, restate them to the user once (using a bullet list) and have them confirm. Only after confirmation do you enter Stage 1.
[Stage 1] Intent clarification
Prerequisite for entering this stage: the 6-dimensional answers from Stage 0 have already been collected in place through the guided process.
Based on the Stage 0 results, supplement 4 more technical details
The official skill-creator standard 4 questions are used here as a supplement:
- Does the output need testing? Skills with objectively verifiable output (file conversion, data extraction, code generation) are suitable for testing; skills with subjective output (writing style, design) usually are not. Give a recommendation based on the skill type and let the user decide.
- Are there any ready-made tools/scripts/MCPs? If the user has mentioned MCPs such as Sorftime, Lingxing, SIF, etc., or has Python scripts on hand, have them clearly state whether the skill should directly reuse them.
- Invocation frequencyIs it weekly, daily, or case by case? This affects whether description optimization is needed.
- Who will use it? Only yourself, shared by the team, or to be published? If shared by the team, consider differences in how different people phrase the trigger.
If the answer is already in the conversation history
A common situation: before entering this skill, the user has already demonstrated the manual process once in the chat (for example, "Last week you helped me analyze BCG's keywords, just follow that process"). In this case, prioritize extracting the answers from the conversation history: the tools used, the order of steps, the user's corrections, and the observed input/output formats. After extracting, restate them to the user for confirmation, and don't make them explain it all over again from scratch.
[Phase 2] Research and fill in the gaps
Building on phases 0-1, proactively ask about these things:
- Edge cases: What to do when the input is empty, data is missing, or an API fails?
- Concrete examples of the input format: The user provides a real file or pastes in an example
- Success criteria: What kind of output counts as "correct"? What counts as "wrong"?
- Dependencies: Which MCPs are used, which file paths, which external scripts?
If there are available MCPs in the environment (such as Sorftime, Lingxing, SIF) and they are helpful for researching this skill (finding similar skills, checking documentation, looking at best practices), you can dispatch sub-agents to research in parallel. The goal is to come back to the user with information and reduce their burden.
The goal of this step:Gather all the facts needed to write SKILL.md. By the time you actually start writing SKILL.md, you won't need to ask again and again.
[Phase 3] Write the first draft of SKILL.md
For detailed writing rules, see references/skill写作指南.md. Here we only cover the key actions.
Required YAML frontmatter fields
- name: the skill identifier (lowercase English + hyphens, for example
keyword-rank-report) - description: the primary trigger mechanism. It must include "what it does + when to use it".It's recommended to write it a bit more "pushy"—Claude currently has a tendency not to invoke skills, so if the description is written too conservatively it won't get triggered.
Counterexample (too conservative):
A skill used for keyword organic ranking analysis
Good example (with push):
Generate a keyword organic ranking analysis report. Invoke it when the user mentions "keyword ranking", "organic position ranking", "Sorftime reverse lookup", or "check ASIN exposure"; trigger it proactively even if they don't explicitly say "analysis".
Body structure (fill in according to the 6-dimension answers from phase 0)
# [Skill 名称]
## 业务背景 ← 映射问 1:业务目标
## 当前工作流(人工版) ← 映射问 2:过去怎么做
## Skill 工作流(自动版) ← 映射问 3:具体步骤
## 核心原则 / 踩坑规避 ← 映射问 4:方法论
## 触发场景 ← 映射问 5:调用方式
## 输出规范 ← 映射问 6:期望输出
## 引用文件 ← 如果有 references/、scripts/、assets/
Key writing techniques
- Use imperative sentences: not "you can use a script to process", but "use
scripts/xxx.pyto process" - Explain why: don't simply write "you must do X"; write "do X because of Y, otherwise Z will happen"
- Use templates for output formats: use
## 报告结构+ a complete template demonstration - Keep the SKILL.md body within 500 lines, and if it exceeds that, split the details into
references/xxx.md, leaving only "when to read" guidance in the body - Large reference files (>300 lines) should have a table of contents
Skill structure
skill-name/
├── SKILL.md (必需)
│ ├── YAML frontmatter
│ └── Markdown 正文
└── 可选资源
├── scripts/ - 固定/重复任务的可执行代码
├── references/ - 按需加载的文档
└── assets/ - 输出使用的素材(模板、图标、字体)
Three-layer loading mechanism (understanding this helps you write better)
- Metadata(name + description) — always in context (~100 words)
- SKILL.md body — enters context only when the skill is triggered (<500 lines)
- Bundled resources — loaded on demand (no limit; scripts can be executed directly without being loaded)
So "put commonly used instructions in SKILL.md, and rarely used details in references".
Multi-domain organization
When a skill supports multiple sets of solutions (e.g., SP/SB/SD ads), split by variant:
ads-report/
├── SKILL.md (主干流程 + 选择逻辑)
└── references/
├── sp.md
├── sb.md
└── sd.md
Claude only reads the reference it needs.
[Stage 4] Write test cases
After finishing the first draft of SKILL.md, think of 2-3things a real user would saytest prompts—not abstract "format the data," but "the ad share for this ASIN of mine at BCG has been dropping badly lately, can you take a look using that report template we used before."
Save the tests to evals/evals.json。At this step, only write the prompt, not the assertions—assertions are added in Stage 6 while running the tests.
{
"skill_name": "example-skill",
"evals": [
{
"id": 1,
"prompt": "用户的任务 prompt",
"expected_output": "期望结果的描述",
"files": []
}
]
}
See the full schema at references/schemas.md。
Show the test cases to the user:
I want to run these test cases, does this look okay to you? Should we add/remove any?
[Stage 5] Run tests and baseline in parallel
This section isa continuous action, do not stop in the middle.Do notuse /skill-test or any other testing skill; just follow the process below.
Result storage location:<skill-name>-workspace/ as the skill directory'ssibling directory. Inside, organize by iteration (iteration-1/、iteration-2/), with one subdirectory per test case (eval-0/、eval-1/). Do not create them all at once; create them as you go.
Step 1: Dispatch with-skill + baseline simultaneously in the same round
For each test case, dispatch two subagents—one with the skill, one without.Key: these two must be dispatched simultaneously in the same round, do not run with-skill first and then run baseline after the results come out. The reason is to let them finish at roughly the same time, reducing differences in system state.
With-skill run:
执行以下任务:
- Skill 路径:<skill 路径>
- 任务:<eval prompt>
- 输入文件:<eval 附带的文件;没有就写 none>
- 输出保存到:<workspace>/iteration-<N>/eval-<ID>/with_skill/outputs/
- 要保存的输出:<用户真正在意的文件,比如 "the .docx file" 或 "the final CSV">
Baseline run(same prompt, but the baseline differs by scenario):
- New skill: give no skill at all. Same prompt, no skill path, save to
without_skill/outputs/。 - Improve an existing skill: use the old version. Before making changes, snapshot a copy first (
cp -r <skill 路径> <workspace>/skill-snapshot/), point the baseline subagent at the snapshot, save toold_skill/outputs/。
At the same time, write for each eval eval_metadata.json(leave assertions empty for now). Give each evala descriptive name(based on the test content, do not call it "eval-0"), and use this name for the directory name too. If this iteration uses new or modified prompts, these files need to be rebuilt for each new eval directory—do not assume they will automatically inherit from the previous iteration.
{
"eval_id": 0,
"eval_name": "descriptive-name-here",
"prompt": "用户的任务 prompt",
"assertions": []
}
Step 2: While the tests are running, write assertions
Do not just wait. Use this time to write for each test casequantitative assertionsand explain them to the user. If evals/evals.json already has assertions, review them too before explaining to the user.
Characteristics of good assertions:
- Objectively verifiable(cannot rely on subjectivity)
- Descriptive names—a glance in the benchmark viewer tells you what it is verifying
- For subjective skills (writing style, design quality), use qualitative evaluation; do not force assertions
After writing, update eval_metadata.json and evals/evals.json. By the way, tell the user what he will see in the viewer—two things: qualitative output + quantitative benchmark.
Step 3: When a subagent finishes, immediately record the timing
Each time a subagent finishes, you'll receive a notification containing total_tokens and duration_ms。immediatelySave to the timing.json:
{
"total_tokens": 84852,
"duration_ms": 23332,
"total_duration_seconds": 23.3
}
This is the only chance to capture this data—once the notification is gone, it's gone. Process them one by one; don't plan to accumulate a batch and then process it.
[Phase 6] Scoring, aggregation, launching the HTML viewer
After all runs finish, do 4 things:
1. Score each run
Dispatch a grader subagent (or do it yourself inline) to read the instructions in agents/grader.md and judge each assertion using the outputs. Write the results to grading.json in each run directory.The field names must be text、passed、evidence(not name/met/details or other variants), because the HTML viewer depends on these three exact field names.
For assertions that can be verified by script,write a script and run itrather than eyeballing them—scripts are faster, more reliable, and reusable across iterations.
2. Aggregate into a benchmark
In the zach-seller-skill-creator directory, run:
python -m scripts.aggregate_benchmark <workspace>/iteration-N --skill-name <name>
Produce benchmark.json and benchmark.md, including pass_rate, time, and token count for each configuration (mean ± stddev and delta). If you need to manually generate benchmark.json, see the schema in references/schemas.md。
Placement order: each with_skill version goes before its corresponding baseline, to make it easy for users to compare.
3. Do a pass of analyst review
Read the benchmark data and look for patterns masked by the averages. Refer to agents/analyzer.md 's "Analyzing Benchmark Results" section, and pay attention to:
- An assertion that passes regardless of whether the skill is present—such an assertion has no discriminating power and is effectively useless
- Evals with particularly high variance—possibly unstable/flaky
- Time/token tradeoff
4. Launch the HTML viewer
View qualitative outputs + quantitative data together:
nohup python ~/.claude/skills/zach-seller-skill-creator/eval-viewer/generate_review.py \
<workspace>/iteration-N \
--skill-name "my-skill" \
--benchmark <workspace>/iteration-N/benchmark.json \
> /dev/null 2>&1 &
VIEWER_PID=$!
For the 2nd and later iterations, add --previous-workspace <workspace>/iteration-<N-1> for comparison.
No-display environment (Cowork / remote): use --static <输出路径> to generate a standalone HTML file; when the user clicks "Submit All Reviews", feedback.jsonwill be downloaded. After downloading, put it back in the workspace directory for the next iteration to read.
Don't reinvent the wheel by writing HTML yourself—just use generate_review.py .
Tell the user something like this:
I've opened the results in your browser. There are two tabs:
- Outputs — click into each test case to see the output, with a text box below for feedback
- Benchmark — see the quantitative comparison
After you've looked, just come back here and let me know.
What the user sees in the viewer
Outputs tab(one test case at a time):
- Prompt: the task given at the time
- Output: the files produced by the skill; anything that can be rendered inline is rendered directly
- Previous Output(iteration 2+): the collapsed output from the previous round
- Formal Grades(if grading is available): the collapsed assertion pass/fail
- Feedback: the auto-saved feedback text box
- Previous Feedback(iteration 2+): the previous feedback
Benchmark tab: a statistical overview of pass rate, time, and tokens, with per-eval breakdown and analyst observations.
Use the prev/next buttons or arrow keys to navigate. When done, click "Submit All Reviews" to save all feedback to feedback.json。
Step 5: Read feedback
After the user says they're done, read feedback.json:
{
"reviews": [
{"run_id": "eval-0-with_skill", "feedback": "图表缺坐标轴标签", "timestamp": "..."},
{"run_id": "eval-1-with_skill", "feedback": "", "timestamp": "..."},
{"run_id": "eval-2-with_skill", "feedback": "完美,继续这样", "timestamp": "..."}
],
"status": "complete"
}
Empty feedback = the user thinks it's OK. Focus your improvement effort on the test cases with specific complaints.
Finally, don't forget to kill the viewer:
kill $VIEWER_PID 2>/dev/null
[Phase 7] Iterative improvement
This is the entire loop'sheart. The tests have run, the user has reviewed, and now you need to fix the skill based on the feedback.
The 4 core principles of improvement
-
Generalize from feedback. Key point: we're building a skill that will be usedtens of thousands of times, and iterating here with the user is just using 2-3 examples to speed up the pace. The user is too familiar with these few examples, so evaluating the output is fast. But if the skill only works well for these few examples and breaks with a different prompt, it's useless.Don't make overfitted fiddly changes, and don't add "oppressively constrictive MUSTs". When you hit a stubborn problem, change your angle, change your metaphor, change your mode of working—trying costs little, and you might hit a flash of inspiration.
-
Keep the prompt concise. Delete the parts that aren't pulling their weight.Read the transcripts (not just the final output)—if you see the skill causing the model to spend a long stretch doing useless work, kill the instructions that make it do that and see what happens.
-
Explain why. Let the model understand why you're asking for this. Today's LLMs are smart and have theory of mind; giving them reasons beats giving them rules. If you find yourself writing ALWAYS, NEVER, or especially rigid structures, this is ayellow alert—try switching to phrasing that explains the reason. This is a more human, more effective way.
-
Look for repeated work across tests. Read the transcripts and see whether the subagent independently writes similar helper scripts each time, or takes the same multi-step process each time. If 3 tests all have the subagent write
create_docx.pyorbuild_chart.py, that's a strong signal:build this script into the skill(write it once and put it inscripts/, and tell the skill to use it in the instructions). Save on reinventing the wheel on every call.
This step is quite important (we're creating value here), and your thinking time is not the bottleneck—go slower, think it through. It's recommended to write a draft, then look at it again from a different perspective and improve it. Really put yourself in the user's shoes and understand what they want.
Iteration loop
After making changes:
- Apply the changes to the skill
- Rerun all test cases to the new
iteration-<N+1>/,including the baseline。- New skill: the baseline is always
without_skill(without loading the skill), unchanged across iterations - Improving an existing skill: use your own judgment—the baseline can be the very first version or the previous iteration, either is fine
- New skill: the baseline is always
- When starting the viewer, add
--previous-workspace <上一迭代目录> - Wait until the user finishes reviewing and says "done"
- Read the new feedback, change again, loop again
When to stop:
- The user says they're satisfied
- The feedback is basically all empty (all OK)
- There's no longer any meaningful progress
Advanced: Blind A/B Testing
When you need a more rigorous comparison of two versions of a skill (for example, if a user asks "is the new version really better than the old one?"), there is a blind comparison mechanism. See agents/comparator.md and agents/analyzer.md。
Core idea: hand two outputs to an independent agent,without telling it who is who,, let it judge the quality; then unblind and analyze why the winner won.
This is optional and requires a subagent; most users won't need it. A manual review loop is usually enough.
Advanced: description-triggered optimization
The description in SKILL.md frontmatter is the main mechanism that determines whether Claude invokes this skill. After creating or improving a skill, you can proactively ask the user whether to do description optimization.
Step 1: Generate queries for trigger evaluation
Create 20 eval queries—a mix of should-trigger and should-not-trigger. Save as JSON:
[
{"query": "用户的 prompt", "should_trigger": true},
{"query": "另一个 prompt", "should_trigger": false}
]
Queries must bereal and specific, things that a Claude Code or Claude.ai user would actually type. Don't use abstract requests; include details: file paths, the user's background, column names and values, company names, URLs, a bit of backstory. Some can be lowercase, some can have abbreviations or typos, some can sound like casual speech. Mix lengths, and focus on edge cases rather than the obvious ones.
Bad examples:"格式化一下这份数据"、"从 PDF 提取文字"、"做个图表"
Good examples:"我老板刚扔给我一个 xlsx 文件(在我下载目录里,叫 'Q4 sales final FINAL v2.xlsx' 之类的),她想让我加一列显示利润率百分比。我记得收入在 C 列、成本在 D 列"
- should-trigger(8-10): you want coverage. Different ways of saying the same intent—some formal, some colloquial.Deliberately do not directly mention the skill name or file type,but clearly need it. Add some rare scenarios, plus cases that compete with other skills but should win.
- should-not-trigger(8-10):The most valuable are near-misses—queries that are similar to the skill's keywords/concepts but actually need something else. Think adjacent domains, vague phrasing (where naive keyword matching would trigger but should not), and cases that touch on what the skill involves but should use a different tool.
Avoid: Don't make should-not-trigger obviously irrelevant. "Write a fibonacci function" as a counterexample for a PDF skill—too easy, it doesn't test anything. Counterexamples should begenuinely tricky。
Step 2: Review with the user
Use the HTML template to show the user the eval set:
- Read the template
assets/eval_review.html - Replace the placeholders:
__EVAL_DATA_PLACEHOLDER__→ JSON array of eval items (no quotes—it's a JS variable assignment)__SKILL_NAME_PLACEHOLDER__→ skill name__SKILL_DESCRIPTION_PLACEHOLDER__→ the skill's current description
- Write to a temp file (for example,
/tmp/eval_review_<skill-name>.html) and open it:open /tmp/eval_review_<skill-name>.html - The user can edit queries, toggle should-trigger, add or remove items, and then click "Export Eval Set"
- The file downloads to
~/Downloads/eval_set.json—if there is a name collision (eval_set (1).json), take the latest one
This step is critical—bad eval queries lead to a bad description.
Step 3: Run the optimization loop
Tell the user: "This will take a while—I'll run it in the background, come back later to check progress."
Save the eval set to the workspace, then run in the background:
python -m scripts.run_loop \
--eval-set <path-to-trigger-eval.json> \
--skill-path <path-to-skill> \
--model <当前会话的 model id> \
--max-iterations 5 \
--verbose
--model Use the current session's model id (it's in your system prompt), so that trigger testing matches the user's real experience.
While it runs, periodically tail the output to tell the user which round it's on and how the scores look.
This script automatically runs the full optimization loop: split the eval set into 60% train + 40% held-out test, run the current description 3 times (for stability), use extended thinking to have Claude propose improvements based on failure cases, re-evaluate each new description on train and test, up to 5 rounds. At the end it opens an HTML report in the browser and returns JSON (containing best_description——select using test scores rather than train scores, to prevent overfitting).
How the triggering mechanism works
Understanding the triggering mechanism helps you write good eval queries. Skills are shown in Claude's available_skills list with name + description, and Claude decides whether to use them based on the description.Key: Claude only consults a skill when the task isn't easy for it to handle on its own—for a simple one-step query like "read this PDF," it may not trigger even if the description matches perfectly, because Claude can do it itself with basic tools. Only complex, multi-step, specialized queries trigger reliably.
Soyour eval queries should have enough substanceto make Claude feel it needs to consult the skill. An overly simple query ("read file X") is a bad test case—no matter how good the description is, it won't trigger.
Step 4: Apply the results
Take the best_description from the JSON output and update it into the skill's SKILL.md frontmatter. Show the user the before/after and the scores.
Packaging and distribution
Only run when the present_files tool is available. If not, skip. If so, package the skill and give the .skill file to the user:
python -m scripts.package_skill <path/to/skill-folder>
After packaging, tell the user the path of the generated .skill file, and they can install it into their own environment.
Special environment adaptations
Claude.ai (no subagent)
The core flow is the same (draft → test → review → improve → loop), but because there's no subagent, the mechanism needs adjusting:
- Run tests: no parallelism possible. Each test case reads SKILL.md itself and completes the task according to its instructions. Do them one by one. This is less rigorous than independent subagents (you write the skill and run it yourself, so information is symmetric), but paired with human review it's still a useful sanity check. Skip baseline runs; just do it with the skill.
- Review results: if you can't open a browser, skip the browser viewer and present directly in the conversation. For each test case, show the prompt and output; if the output is a file the user needs to download, like .docx, save it to the filesystem and tell them the path so they can download and view it. "Is this OK? Anything to change?"
- Benchmark: skip the quantitative benchmark—it relies on baseline comparison, which is meaningless without a subagent. Focus on qualitative feedback.
- Description optimization: relies on
claude -p(only available in Claude Code), skip. - Blind comparison: requires a subagent, skip.
- Packaging:
package_skill.pyonly needs Python and a filesystem to run, so it works.
Cowork
- With a subagent, the main flow (parallel runs, baseline, grading) all works. If you hit severe timeouts, you can run the test prompts serially.
- No browser/display: when generating the viewer, use
--static <输出路径>to write standalone HTML and give the user a link to click. - In the Cowork environment, Claude oftendoes not proactivelygenerate an eval viewer; a reminder in caps here:after finishing the tests, before evaluating it yourself, first use
generate_review.pygenerate_review.py to generate an eval viewer for people to see. What you want is for people to see the examples as soon as possible! - The feedback mechanism is different: with no running server, the viewer's "Submit All Reviews" will download
feedback.json, and you read this file (you may need to request access first). - Packaging works.
- Description optimization (
run_loop.py/run_eval.py) can run in Cowork (it uses subprocess to callclaude -p, no browser needed), butdo it after skills like this are stable and the user says OK。
Reference files
agents/ The directory contains English instructions for specialized subagents (Claude reads them directly when dispatching subagents; English is more reliable):
agents/grader.md— How to evaluate assertions using outputsagents/comparator.md— How to do blind A/B comparisonsagents/analyzer.md— How to analyze why one version wins
references/ Directory:
references/schemas.md— JSON structures for evals.json, grading.json, etc. (Chinese)references/6问引导式流程.md— The only execution script for Stage 0 (question-by-question guidance + gate validation)references/6问模板.md— Internal reference for how Stage 0 results map to SKILL.mdreferences/skill写作指南.md— Complete rules and examples for writing SKILL.md
Finally, to repeat the core loop once more:
- User proposes an idea → pass the 6-question hard gate
- Write the first draft of SKILL.md → write 2-3 test cases
- Run with/without tests in parallel → launch the HTML viewer for the user to see
- Read user feedback → revise the skill
- Repeat until satisfied → package and distribute
When you encounter this, remember to add these steps to the TodoList to avoid missing them.
Wishing you a smooth creation!