improvement-generator

Improvement Generator

Produces ranked improvement candidates from target analysis, feedback signals, and failure traces.

When to Use

• Generate structured improvement candidates for a target skill
• Inject the previous failure's trace into the next round (trace-aware reflection)
• Automatically lower the priority of candidates in the previously failed category based on the trace
• Generate high-priority candidates by combining multi-source signals from memory and feedback
• Batch-generate candidate lists for multiple skills for the discriminator to score
• Automatically invoked by the orchestrator in autoloop scenarios, injecting historical traces
• Used as a standalone tool when manually debugging the improvement direction of a single skill
• Compare generation results with/without trace to verify whether trace injection is effective

When NOT to Use

• Score candidates → use improvement-discriminator
• Evaluate skill structure → use improvement-learner
• Full pipeline → use improvement-orchestrator
• Execute approved changes → use improvement-executor
• Gate verification → use improvement-gate

Why Trace-Aware Generation Matters

Problem: Without trace injection, the LLM generates candidates from scratch every time. If the previous round failed on the accuracy dimension, the next round is very likely to generate candidates of the same category again — because the LLM does not know it failed last time. In measured tests, the repeated failure rate of retries without trace is as high as 60-70%.

Tradeoff: Trace injection increases prompt length (about 200-500 tokens), but greatly reduces the repeated failure rate. Because the trace contains three key signals — failure dimension, failure reason, and strategies already tried — the generator can avoid known dead ends during the generation stage, rather than only discovering them after the discriminator scores. This saves 1-2 iterations compared with the "generate → score → discover duplication → regenerate" loop.

Trace-Aware Generation

Previous failure on "accuracy" dimension
  → deprioritize candidates of the same category as the failed one
  → prioritize other dimensions' improvements instead
  → if same category failed ≥2 times, skip entirely and try adjacent dimensions

Trace JSON Structure

The trace file records the full context of the previous round's failure, and the generator parses it to adjust candidate priorities:

{
  "iteration": 2,
  "failed_dimension": "accuracy",
  "failed_category": "add_code_examples",
  "failure_reason": "code example added but not syntactically valid",
  "attempted_strategies": ["append_bash_example", "append_python_snippet"],
  "scores_before": {"accuracy": 0.67, "coverage": 0.85},
  "scores_after": {"accuracy": 0.63, "coverage": 0.85}
}

After receiving this trace, the generator will: (1) lower the priority of the add_code_examples category to the lowest, (2) look for candidates from dimensions that did not fail, such as coverage/trigger_quality, (3) if other categories under accuracy (such as add_output_artifacts) have not been tried, they can still be generated.

CLI

# Basic generation
python3 scripts/propose.py --target /path/to/skill --output candidates.json

# With failure trace (retry loop)
python3 scripts/propose.py --target /path/to/skill --trace failure.json --output candidates.json

# With memory/feedback sources
python3 scripts/propose.py --target /path/to/skill --source memory.json --output candidates.json

Output Artifacts

Request	Deliverable
Generate	JSON array of ranked candidates with category, risk_level, execution_plan
With trace	Same format, priorities adjusted based on failure analysis
With memory	Candidates informed by historical patterns and past successes
With feedback	Candidates prioritized by user correction hotspots

The JSON structure of each candidate includes category (improvement category), risk_level (low/medium/high), execution_plan (specific modification steps), priority_score (0-1 overall priority), and trace_adjusted (whether the priority was adjusted by the trace).

Related Skills

• improvement-discriminator: Scores the candidates this skill produces
• improvement-orchestrator: Calls generator as stage 1
• improvement-learner: Provides evaluation data that informs candidate selection
• improvement-executor: Executes the top-ranked candidate approved by gate
• session-feedback-analyzer: Generates feedback.jsonl that feeds into candidate prioritization