Positioning: teaches the discipline of "the optimization process itself," not bound to any specific component type. Core idea = validation-driven optimization loop: score to locate weaknesses → single-variable change → after change pass quality gate + before/after comparison → keep only if strictly better (otherwise git revert) → stop at ceiling. Methodology soft references (borrowed, not depended upon; no import, no script calls): darwin-skill (9-dimension rubric + validation-gated + ratchet + stop at ceiling) · huashu-nuwa (independent sub-agent validation to prevent self-evaluation bias) · cangjie-skill (stress testing / decoy boundaries) · grill-me (confirm decisions question by question; look up facts yourself, wait for the user on decisions).
🔴 Hard rules (violation = failure)
- Any change to component content must pass the quality gate(mandated by the project CLAUDE.md):
Endpoint flakiness (400) retry loop (see memory claude-p-endpoint-flaky); if still failing after 3 attempts → manual verification + small rollback-able commit, marked "pending endpoint recovery for rerun."claude -p "<待测内容>" --output-format stream-json | jq -r 'select(.type=="result" and .subtype=="success") | .result' - validation-gated: keep only if strictly better: after the change pass the quality gate + before/after comparison (trigger accuracy / output quality / score)any regression →
git revert HEADroll back; no direct landing on "I think it's better." Fine-grained scores are untrustworthy; only gross signals (Δ>0). - Single-variable round: only one dimension (or one tightly related cluster) changed per round; changing multiple dimensions at once makes attribution fail.
- Independent validation to prevent self-evaluation bias: spawn an independent sub-agent (or independent session) for scoring and comparison; no self-evaluation/self-modification in the same context (optimism bias; LLM-as-judge empirically ~46% accuracy).
- Stop at ceiling: if Δ < 2 points for 2 consecutive rounds → break; no forcing MAX_ROUNDS (typical symptom = adding filler to make the LLM think it is more detailed; padding is the warning sign).
- No direct third-party dependencies: the scoring rubric / comparison scripts are all self-contained (see references/); the soft references darwin/nuwa/cangjie/grill are "methodology borrowing sources, not dependencies"; no importing their scripts.
Routing (deduplicate first to avoid overlap with skill-dev / plugin-dev)
| Input signal | Go to |
|---|---|
| Optimizea single skill / agent 's deep quality (9-dimension scoring / fix triggers / fill in failure modes / hill climbing) | 🛑 /skill-dev(process B) |
| Plugin-level manifest / component wiring / hook robustness / marketplace consistency | 🛑 /plugin-dev(process B) |
| Across any component, no clear direction, needs the general "score → single-variable change → validate → ratchet → stop at ceiling" discipline applied | ✅ This skill |
| Optimize a command / script / config / cross-component combination (neither a single skill nor plugin wiring) | ✅ This skill |
If the boundary is ambiguous (e.g., "this skill's triggers are inaccurate, and by the way help me optimize the whole process"): first apply this skill's loop discipline for framework-level optimization; if a single skill's deep scoring is the bottleneck, then hand off to
/skill-dev。
General optimization loop (4 phases)
Phase 0: Define scope (grill question by question, decisions await the user)
Confirm question by questionDecision items(Look up facts in the environment yourself, asking about facts is forbidden):
- Target component to optimize (plugin / skill / agent / command / combination) + path.
- Current pain points (not triggering? output off track? too long? false triggering? human review thinks it regressed?).
- Acceptance criteria (trigger accuracy / output quality / size limit / observable behavior) — write them as decidable before/after comparison items.
🔴 CHECKPOINT: Put the three items — target + pain points + acceptance criteria — in writing and get the user's nod before acting. A wrong direction means all subsequent work is redone.
Phase 1: Baseline scoring (score independently, locate the lowest dimension)
- Score against the general dimension table to locate the lowest dimension,scoring spawns an independent sub-agent(or independent session) to run; self-scoring in the same context is forbidden.
- For the dimension table quick reference + scoring details see references/scoring-matrix.md(created by s2 subtask, includes general dimensions such as trigger accuracy / structural clarity / failure mode coverage / specificity / redundancy / verifiability, with adapted weights by component type).
- Output a diagnostic table: dimension / score / weakness evidence (line number references) / suggested edit type (add/delete/replace) / related cluster.
🔴 CHECKPOINT: After the diagnostic table gets the user's confirmation on direction + priority, then design the edits.
Phase 2: Single-variable rounds (core — change → verify → ratchet → stop at ceiling)
Each round of the loop:
- Change one dimension(or one related cluster): the smallest verifiable change, prefer small steps over rewriting whole sections (unless ≥3 dimensions simultaneously score ≤4, triggering exploratory rewrite, and the user must confirm).
- Pass the quality gate(hard rule 1 command) + before/after comparison: see references/validation-gate.md(created by s3 subtask, includes should-trigger / should-not-trigger / edge case sets + independent sub-agent blind evaluation process + dry_run fallback).
- Keep only if strictly better: trigger accuracy does not regress · output has no negatives (redundancy / off track / weird format) · score gross Δ>0 →
git commit(branchoptimize/<comp>-YYYYMMDD); any regression →git revert HEADrollback (forbiddenreset --hardto lose the working tree), record the reason for the failed attempt (unclear attribution / Δ<0 / worse triggering). - Stop at ceiling: 2 consecutive rounds with Δ < 2 → break into Phase 3 (+0.x is a stop signal, not a continue signal).
Phase 3: Summary (score changes + keep / rollback stats + ceiling signal)
Output: before/after score change table + Δ per dimension + list of kept / rolled-back edits + rollback reasons + judge consensus + dry_run ratio + ceiling round. Destructive changes (trigger words / large-scale rewrite / merge) are explicitly flagged as requiring user confirmation.
Failure mode quick reference (trigger → first-line fix → fallback if still failing)
| Trigger | First-line fix | Fallback if still failing |
|---|---|---|
Quality gate claude -p returns 400 / empty |
Retry loop (endpoint flakiness, remember claude-p-endpoint-flaky) | Still failing after 3 times → manual review + small rollback-able commit, marked "pending endpoint recovery for re-run" |
| Self-evaluation optimistic bias (everything looks good after the change) | Scoring / comparison spawns an independent sub-agent or independent session | judge disagreement is large → add a 3rd judge or switch to full_test for actual measurement; if disagreement persists, mark as "evaluation untrustworthy" for human review |
| Multiple dimensions changed together makes it impossible to attribute which one took effect | Reduce to single-variable retries, change only one dimension per round (or one tightly related cluster) | Already mixed changes → revert all, redo dimension by dimension to establish causality |
| After hitting the ceiling, padding with filler to raise the score (volume inflation) | Inflation guardrail: if after change > original ×1.5, refuse to submit; streamline first, then evaluate | Delete redundancy / merge duplicates back within ×1.5 and still Δ<2 → accept ceiling hit and stop |
| Independent sub-agent unavailable | Degrade to dry_run (simulate execution approach), mark as dry_run | dry_run > 30% → ⚠️ evaluation invalid warning, only produce suggestions without changing the board, scores untrustworthy and require human review |
| Changed trigger words cause downstream breakage | Roll back trigger words, add keywords inside body | Create a new component instead of modifying in place (breaking changes must be confirmed by the user) |
Counterexamples (a hit = process error)
- Self-evaluate and self-modify in the same context (should spawn an independent sub-agent / independent session to prevent optimism bias).
git reset --hardrollback (loses the working tree, shouldgit revert HEADcreate a reverse commit).- Add redundancy / expand volume to pad the score (should use the inflation guardrail: > original ×1.5 refuse to submit, streamline first, then evaluate).
- Skip the quality gate and directly submit the changed content (violates hard rule 1).
- Change multiple dimensions in one round (attribution fails, should use single-variable rounds).
- Use this skill to evaluatesingle skill / agent depth(hand over to
/skill-dev) or checkplugin wiring(hand over to/plugin-dev）。 - Guess out of thin air without verifying / make decisions without letting the user decide (should grill: verify facts yourself in the environment, wait for the user on decisions).
Resources
- references/scoring-matrix.md — general dimension scoring table + weight adaptation by component type (created by s2 subtask, pointed to by this skill's Phase 1).
- references/validation-gate.md — before/after comparison process + independent sub-agent blind evaluation + dry_run degradation + held-out set construction (created by s3 subtask, pointed to by this skill's Phase 2).
- Methodology reference sources (not dependencies, no import and no script calls allowed): darwin-skill (rubric + ratchet + ceiling) · huashu-nuwa (independent verification to prevent self-evaluation bias) · cangjie-skill (stress testing / decoys) · grill-me (confirm decisions question by question).
Honest boundaries
- Fine-grained scores are untrustworthy: gross signals (Δ>0) are valid and can be used for rough judgment; important decisions (breaking / trigger words / large-scale rewrites) must have human review, and "merge because the score went up" is forbidden.
- No third-party benchmark: the rubric is a distillation of commonalities from darwin/nuwa/cangjie/grill plus an aggregation of experience, not peer-reviewed, with no independent third-party reproduction.
- Scores are inflated when there is a lot of dry_run: when independent sub-agents / real harnesses are insufficient, it degrades to dry_run; > 30% means the evaluation is invalid, only produce suggestions without changing the board.