skill-evolution

Skill Evolution

Track tool and strategy effectiveness across research tasks, propose improvements to existing skills, and maintain a reusable pattern library inspired by VOYAGER's skill library architecture.

When to Use

• After multiple research sessions to identify improvement opportunities
• "Which search strategies work best for biomedical topics?"
• "Propose improvements to the literature-search skill"
• "Store this successful analysis pattern for reuse"
• When the science-evolution extension triggers post-session analysis

When NOT to Use

• During active research (focus on the task, reflect afterward)
• For one-off questions that don't need pattern storage
• For simple skill usage (just use the skill directly)

VOYAGER Skill Library Pattern

Inspired by VOYAGER (Wang et al., 2023), maintain a library of reusable research patterns:

Pattern Structure

{
  "pattern_id": "lit-review-biomedical-v2",
  "name": "Biomedical Literature Review",
  "domain": ["biology", "medicine"],
  "task_type": "literature_review",
  "version": 2,
  "description": "Optimized search strategy for biomedical topics",
  "steps": [
    "Search Semantic Scholar with MeSH-equivalent terms",
    "Search PubMed via NCBI Entrez with MeSH filters",
    "Cross-reference with ClinicalTrials.gov for ongoing trials",
    "Citation chain top 3 papers (forward + backward)",
    "Verify key genes/proteins in UniProt",
    "Verify drug interactions in ChEMBL"
  ],
  "tools_used": ["semantic-scholar", "ncbi-entrez", "uniprot-protein", "chembl-drug"],
  "success_rate": 0.85,
  "avg_quality_score": 21,
  "times_used": 12,
  "last_used": "2026-03-11",
  "lessons": [
    "PubMed MeSH terms are more precise than free-text for biomedical queries",
    "Always check ClinicalTrials.gov for therapy-related topics",
    "UniProt cross-references to PDB save a separate search step"
  ]
}

Pattern Operations

# Store a new pattern
curl -X POST http://localhost:18789/evolution/pattern \
  -H "Content-Type: application/json" \
  -d '{"pattern": {...}}'

# Search for patterns matching a new task
curl "http://localhost:18789/evolution/search?domain=biology&task_type=literature_review"

# Update pattern after use (increment times_used, update success_rate)
curl -X PATCH http://localhost:18789/evolution/pattern/lit-review-biomedical-v2 \
  -d '{"success": true, "quality_score": 22}'

Skill Improvement Proposals

Analysis Framework

After 5+ uses of a skill, analyze its performance:

## Skill Analysis: [skill-name]

### Usage Statistics
- Times used: N
- Average quality score: X/25
- Success rate: Y%
- Common failure modes: [list]

### Strengths
- [What the skill does well]

### Weaknesses
- [Where the skill falls short]
- [Missing capabilities]
- [Incorrect or outdated guidance]

### Proposed Changes
1. [Specific change to SKILL.md]
   - Rationale: [why]
   - Expected impact: [improvement area]

2. [Another change]
   - Rationale: [why]
   - Expected impact: [improvement area]

### Priority: [high/medium/low]

Automated Improvement Detection

Track these signals across research sessions:

Signal	Indicates	Action
Repeated tool failures	API endpoint changed or unreliable	Update SKILL.md with workaround
Consistent low scores in one dimension	Skill gap in that area	Add guidance for that dimension
User corrections	Skill provides wrong guidance	Fix the incorrect guidance
New API discovered	Opportunity to expand	Add new tool instructions
Cross-domain pattern success	Transferable knowledge	Create cross-domain pattern

Cross-Domain Knowledge Transfer

Identifying Transferable Patterns

Some research patterns work across disciplines:

1. Citation chain analysis — Works for any field with citation data
2. Database cross-verification — Applicable whenever primary data exists
3. Effect size reporting — Standard across quantitative disciplines
4. PICO framework — Adaptable beyond medicine (SPIDER for qualitative)
5. Visualization standards — Journal figure requirements are similar

Transfer Process

When a pattern succeeds in domain A, evaluate for domain B:

1. Are the tools available? (e.g., does domain B have equivalent databases?)
2. Are the methods appropriate? (e.g., meta-analysis needs comparable studies)
3. What adaptations are needed? (e.g., different search terms, different databases)
4. Store as a new domain-specific variant

Evolution Metrics

Skill Health Dashboard

| Skill | Uses | Avg Score | Trend | Health |
|-------|------|-----------|-------|--------|
| literature-search | 45 | 22/25 | +1.2 | Healthy |
| statsmodels-stats | 12 | 18/25 | -0.5 | Needs attention |
| semantic-scholar | 38 | 23/25 | +0.8 | Healthy |
| meta-analysis | 3 | -- | -- | Too few uses |

Trend Analysis

• Improving: Score trending up → skill guidance is effective
• Declining: Score trending down → investigate (API changes? outdated guidance?)
• Stable: No trend → working as expected
• Insufficient data: < 5 uses → collect more data before drawing conclusions

Integration with science-evolution Extension

This skill works with the science-evolution extension:

• Extension tracks tool usage and outcomes automatically
• Stores data in ~/.openclaw/science-evolution.db
• Provides API endpoints for pattern storage and retrieval
• Triggers post-session analysis when enough data accumulates

Best Practices

1. Don't optimize prematurely — wait for 5+ uses before proposing changes
2. Track both successes and failures for each pattern
3. Version patterns so you can roll back if a change hurts performance
4. Cross-reference reflections (research-reflection) with evolution data
5. Focus improvements on the highest-impact skills first
6. Keep the pattern library curated — remove patterns that are never reused