os-improvement-loop

Concurrent Agent Loop

Treats concurrent sessions as OS threads sharing a common event bus and memory address space. Every cycle includes execution, independent peer evaluation, friction tracking, self-assessment surveys, post-run metrics, and memory persistence.

When to Use This Pattern

Use when:

Coordinating continuous improvement across multiple concurrent agent sessions.
Evaluating and improving multiple skills, workflows, or templates in parallel.
You need every cycle to generate measurable accuracy gains and persistent memory.

Do NOT use for:

Single-session procedural tasks (use os-eval-runner directly).
Signal-only coordination with no evaluation, survey, or memory steps.

Key Invariants

No-Rollback Rule: Never manually roll back changes during a cycle unless evaluate.py registers an explicit accuracy regression.
Eval Gate Mandatory: Every modification must pass the independent evaluation gate (evaluate.py exit code 0). No manual bypasses.
NEVER STOP Discipline: Do not abort a running loop due to minor/moderate errors. Complete the loop close checklist and log unresolved issues as Map Debt.
Outer Loop Ownership: The outer loop owns session lifecycle. Inner loop tasks (os-eval-runner) must not prematurely close a session without running Stage 4 (memory promotion and survey collection).

Stage Pointers & Reference Protocols

The execution details are split across modular references:

Stage 0: Setup and Orientation — Pre-flight reads, registry, and packet design.
Stage 1: INNER_AGENT Execution — Strategy execution, friction logging, and local scoring.
Stage 2: PEER_AGENT Verification — Independent evaluation run and verdict formulation.
Stage 3: Decision Logic — KEEP/DISCARD actions and correction packets.
Stage 4: Loop Close Checklist — Surveys, ledger updates, memory promotion, and retrospectives.
Orchestrator Meta-Survey — Meta-evaluation of loop coordination patterns.
Lab Limits — Hard caps on iterations, eval sets, and critic invocations per run.

Smoke Test

Verify Event Registry: Run os-init or start a test loop. Assert that context/events.jsonl registers start events correctly.
Execute Scorer: Run python3 ./scripts/evaluate.py --skill skills/todo-check/ on a dummy check to verify that exit codes map correctly (0 for KEEP, 1 for DISCARD, 2 for path error).
Friction Event Test: Propose a manual edit, emit a mock friction event, resolve it with friction.resolved, and verify the metrics engine logs the resolution gate pass.

Gotchas

Conflation of Loops: Conflating the inner target skill loop with the outer OS-improvement loop. Outer loop changes the OS workflows; inner loop changes target skills.
Orphaned Sessions: Completing inner loop tasks but failing to run memory promotion and survey curation. Ephemeral findings are lost.
Directory Symlinks: Creating directory-level symlinks from skills to shared roots. This violates ADR-003. Use file-level symlinks.

HANDOFF_BLOCK Template

Every loop execution that completes a cycle must output this block in its handoff:

## HANDOFF_BLOCK
- **Cycle ID**: cycle-YYYYMMDD-HHMMSS
- **Target Skill**: [path/to/target]
- **Verdict**: KEEP / DISCARD
- **Score (Before -> After)**: [0.XX -> 0.YY]
- **Friction Events**: [N encountered / N resolved]
- **Outstanding Map Debt**: [list links or IDs]
- **Recommended Next Step**: [next hypothesis to test]