SuperAPI AI Directory

skill-tuning

Data & analytics

View repository

Updated 10-15 21:36

About this skill

Skill Tuning

Universal skill diagnosis and optimization tool that identifies and resolves skill execution problems through iterative multi-agent analysis.

Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│  Skill Tuning Architecture (Autonomous Mode + Gemini CLI)                    │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  ⚠️ Phase 0: Specification  → 阅读规范 + 理解目标 skill 结构 (强制前置)       │
│              Study                                                           │
│           ↓                                                                  │
│  ┌───────────────────────────────────────────────────────────────────────┐  │
│  │                    Orchestrator (状态驱动决策)                          │  │
│  │  读取诊断状态 → 选择下一步动作 → 执行 → 更新状态 → 循环直到完成         │  │
│  └───────────────────────────────────────────────────────────────────────┘  │
│                              │                                               │
│     ┌────────────┬───────────┼───────────┬────────────┬────────────┐        │
│     ↓            ↓           ↓           ↓            ↓            ↓        │
│  ┌──────┐  ┌──────────┐  ┌─────────┐  ┌────────┐  ┌────────┐  ┌─────────┐  │
│  │ Init │→ │ Analyze  │→ │Diagnose │  │Diagnose│  │Diagnose│  │ Gemini  │  │
│  │      │  │Requiremts│  │ Context │  │ Memory │  │DataFlow│  │Analysis │  │
│  └──────┘  └──────────┘  └─────────┘  └────────┘  └────────┘  └─────────┘  │
│                 │              │           │           │            │        │
│                 │              └───────────┴───────────┴────────────┘        │
│                 ↓                                                            │
│  ┌───────────────────────────────────────────────────────────────────────┐  │
│  │  Requirement Analysis (NEW)                                            │  │
│  │  • Phase 1: 维度拆解 (Gemini CLI) - 单一描述 → 多个关注维度             │  │
│  │  • Phase 2: Spec 匹配 - 每个维度 → taxonomy + strategy                 │  │
│  │  • Phase 3: 覆盖度评估 - 以"有修复策略"为满足标准                       │  │
│  │  • Phase 4: 歧义检测 - 识别多义性描述,必要时请求澄清                   │  │
│  └───────────────────────────────────────────────────────────────────────┘  │
│                              ↓                                               │
│                    ┌──────────────────┐                                      │
│                    │  Apply Fixes +   │                                      │
│                    │  Verify Results  │                                      │
│                    └──────────────────┘                                      │
│                                                                              │
│  ┌───────────────────────────────────────────────────────────────────────┐  │
│  │                    Gemini CLI Integration                              │  │
│  │  根据用户需求动态调用 gemini cli 进行深度分析:                          │  │
│  │  • 需求维度拆解 (requirement decomposition)                             │  │
│  │  • 复杂问题分析 (prompt engineering, architecture review)               │  │
│  │  • 代码模式识别 (pattern matching, anti-pattern detection)              │  │
│  │  • 修复策略生成 (fix generation, refactoring suggestions)               │  │
│  └───────────────────────────────────────────────────────────────────────┘  │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

Problem Domain

Based on comprehensive analysis, skill-tuning addresses core skill issues and general optimization areas:

Core Skill Issues (automatic detection)

Priority Problem Root Cause Solution Strategy
P0 Authoring Principles Violation Intermediate file storage, State bloat, file relay eliminate_intermediate_files, minimize_state, context_passing
P1 Data Flow Disruption Scattered state, inconsistent formats state_centralization, schema_enforcement
P2 Agent Coordination Fragile call chains, merge complexity error_wrapping, result_validation
P3 Context Explosion Token accumulation, multi-turn bloat sliding_window, context_summarization
P4 Long-tail Forgetting Early constraint loss constraint_injection, checkpoint_restore
P5 Token Consumption Verbose prompts, excessive state, redundant I/O prompt_compression, lazy_loading, output_minimization

General Optimization Areas (on-demand analysis via Gemini CLI)

Category Issues Gemini Analysis Scope
Prompt Engineering Vague instructions, inconsistent output format, hallucination risk Prompt optimization, structured output design
Architecture Unreasonable phase division, chaotic dependencies, poor scalability Architecture review, modularization suggestions
Performance Slow execution, high Token consumption, repeated computation Performance analysis, caching strategy
Error Handling Improper error recovery, no degradation strategy, insufficient logging Fault-tolerant design, observability enhancement
Output Quality Unstable output, format drift, quality fluctuation Quality gating, validation mechanism
User Experience Unsmooth interaction, unclear feedback, invisible progress UX optimization, progress tracking

Key Design Principles

  1. Problem-First Diagnosis: Systematic identification before any fix attempt
  2. Data-Driven Analysis: Record execution traces, token counts, state snapshots
  3. Iterative Refinement: Multiple tuning rounds until quality gates pass
  4. Non-Destructive: All changes are reversible with backup checkpoints
  5. Agent Coordination: Use specialized sub-agents for each diagnosis type
  6. Gemini CLI On-Demand: Deep analysis via CLI for complex/custom issues

Gemini CLI Integration

Dynamically invoke Gemini CLI for in-depth analysis based on user needs.

Trigger Conditions

Condition Action CLI Mode
User describes a complex problem Invoke Gemini to analyze the root cause of the problem analysis
Automatic diagnosis discovers critical issues Request in-depth analysis confirmation analysis
User requests architecture review Execute architecture analysis analysis
Need to generate fix code Generate fix proposal write
Standard strategy not applicable Request customized strategy analysis

CLI Command Template

ccw cli -p "
PURPOSE: ${purpose}
TASK: ${task_steps}
MODE: ${mode}
CONTEXT: @${skill_path}/**/*
EXPECTED: ${expected_output}
RULES: $(cat ~/.claude/workflows/cli-templates/protocols/${mode}-protocol.md) | ${constraints}
" --tool gemini --mode ${mode} --cd ${skill_path}

Analysis Types

1. Problem Root Cause Analysis

ccw cli -p "
PURPOSE: Identify root cause of skill execution issue: ${user_issue_description}
TASK: • Analyze skill structure and phase flow • Identify anti-patterns • Trace data flow issues
MODE: analysis
CONTEXT: @**/*.md
EXPECTED: JSON with { root_causes: [], patterns_found: [], recommendations: [] }
RULES: $(cat ~/.claude/workflows/cli-templates/protocols/analysis-protocol.md) | Focus on execution flow
" --tool gemini --mode analysis

2. Architecture Review

ccw cli -p "
PURPOSE: Review skill architecture for scalability and maintainability
TASK: • Evaluate phase decomposition • Check state management patterns • Assess agent coordination
MODE: analysis
CONTEXT: @**/*.md
EXPECTED: Architecture assessment with improvement recommendations
RULES: $(cat ~/.claude/workflows/cli-templates/protocols/analysis-protocol.md) | Focus on modularity
" --tool gemini --mode analysis

3. Fix Strategy Generation

ccw cli -p "
PURPOSE: Generate fix strategy for issue: ${issue_id} - ${issue_description}
TASK: • Analyze issue context • Design fix approach • Generate implementation plan
MODE: analysis
CONTEXT: @**/*.md
EXPECTED: JSON with { strategy: string, changes: [], verification_steps: [] }
RULES: $(cat ~/.claude/workflows/cli-templates/protocols/analysis-protocol.md) | Minimal invasive changes
" --tool gemini --mode analysis

Mandatory Prerequisites

CRITICAL: Read these documents before executing any action.

Core Specs (Required)

Document Purpose Priority
specs/skill-authoring-principles.md Primary principles: concise and efficient, remove storage, context flow P0
specs/problem-taxonomy.md Problem classification and detection patterns P0
specs/tuning-strategies.md Fix strategies for each problem type P0
specs/dimension-mapping.md Dimension to Spec mapping rules P0
specs/quality-gates.md Quality thresholds and verification criteria P1

Templates (Reference)

Document Purpose
templates/diagnosis-report.md Diagnosis report structure
templates/fix-proposal.md Fix proposal format

Execution Flow

┌─────────────────────────────────────────────────────────────────────────────┐
│  Phase 0: Specification Study (强制前置 - 禁止跳过)                           │
│  → Read: specs/problem-taxonomy.md (问题分类)                                │
│  → Read: specs/tuning-strategies.md (调优策略)                               │
│  → Read: specs/dimension-mapping.md (维度映射规则)                           │
│  → Read: Target skill's SKILL.md and phases/*.md                            │
│  → Output: 内化规范,理解目标 skill 结构                                      │
├─────────────────────────────────────────────────────────────────────────────┤
│  action-init: Initialize Tuning Session                                      │
│  → Create work directory: .workflow/.scratchpad/skill-tuning-{timestamp}    │
│  → Initialize state.json with target skill info                             │
│  → Create backup of target skill files                                       │
├─────────────────────────────────────────────────────────────────────────────┤
│  action-analyze-requirements: Requirement Analysis                           │
│  → Phase 1: 维度拆解 (Gemini CLI) - 单一描述 → 多个关注维度                   │
│  → Phase 2: Spec 匹配 - 每个维度 → taxonomy + strategy                       │
│  → Phase 3: 覆盖度评估 - 以"有修复策略"为满足标准                             │
│  → Phase 4: 歧义检测 - 识别多义性描述,必要时请求澄清                         │
│  → Output: state.json (requirement_analysis field)                           │
├─────────────────────────────────────────────────────────────────────────────┤
│  action-diagnose-*: Diagnosis Actions (context/memory/dataflow/agent/docs/   │
│                      token_consumption)                                      │
│  → Execute pattern-based detection for each category                         │
│  → Output: state.json (diagnosis.{category} field)                           │
├─────────────────────────────────────────────────────────────────────────────┤
│  action-generate-report: Consolidated Report                                 │
│  → Generate markdown summary from state.diagnosis                            │
│  → Prioritize issues by severity                                             │
│  → Output: state.json (final_report field)                                   │
├─────────────────────────────────────────────────────────────────────────────┤
│  action-propose-fixes: Fix Proposal Generation                               │
│  → Generate fix strategies for each issue                                    │
│  → Create implementation plan                                                │
│  → Output: state.json (proposed_fixes field)                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│  action-apply-fix: Apply Selected Fix                                        │
│  → User selects fix to apply                                                 │
│  → Execute fix with backup                                                   │
│  → Update state with fix result                                              │
├─────────────────────────────────────────────────────────────────────────────┤
│  action-verify: Verification                                                 │
│  → Re-run affected diagnosis                                                 │
│  → Check quality gates                                                       │
│  → Update iteration count                                                    │
├─────────────────────────────────────────────────────────────────────────────┤
│  action-complete: Finalization                                               │
│  → Set status='completed'                                                    │
│  → Final report already in state.json (final_report field)                   │
│  → Output: state.json (final)                                                │
└─────────────────────────────────────────────────────────────────────────────┘

Directory Setup

const timestamp = new Date().toISOString().slice(0,19).replace(/[-:T]/g, '');
const workDir = `.workflow/.scratchpad/skill-tuning-${timestamp}`;

// Simplified: Only backups dir needed, diagnosis results go into state.json
Bash(`mkdir -p "${workDir}/backups"`);

Output Structure

.workflow/.scratchpad/skill-tuning-{timestamp}/
├── state.json                      # Single source of truth (all results consolidated)
│   ├── diagnosis.*                 # All diagnosis results embedded
│   ├── issues[]                    # Found issues
│   ├── proposed_fixes[]            # Fix proposals
│   └── final_report                # Markdown summary (on completion)
└── backups/
    └── {skill-name}-backup/        # Original skill files backup

Token Optimization: All outputs consolidated into state.json. No separate diagnosis files or report files.

State Schema

For detailed state structure definitions, please refer to phases/state-schema.md。

Core state fields:

  • status: Workflow state (pending/running/completed/failed)
  • target_skill: Target skill information
  • diagnosis: Diagnosis results for each dimension
  • issues: List of discovered issues
  • proposed_fixes: Suggested fix solutions

Reference Documents

Document Purpose
phases/orchestrator.md Orchestrator decision logic
phases/state-schema.md State structure definition
phases/actions/action-init.md Initialize tuning session
phases/actions/action-analyze-requirements.md Requirement analysis (NEW)
phases/actions/action-diagnose-context.md Context explosion diagnosis
phases/actions/action-diagnose-memory.md Long-tail forgetting diagnosis
phases/actions/action-diagnose-dataflow.md Data flow diagnosis
phases/actions/action-diagnose-agent.md Agent coordination diagnosis
phases/actions/action-diagnose-docs.md Documentation structure diagnosis
phases/actions/action-diagnose-token-consumption.md Token consumption diagnosis
phases/actions/action-generate-report.md Report generation
phases/actions/action-propose-fixes.md Fix proposal
phases/actions/action-apply-fix.md Fix application
phases/actions/action-verify.md Verification
phases/actions/action-complete.md Finalization
specs/problem-taxonomy.md Problem classification
specs/tuning-strategies.md Fix strategies
specs/dimension-mapping.md Dimension to Spec mapping (NEW)
specs/quality-gates.md Quality criteria