SuperAPI AI Directory

session-audit

Agents & models

View repository

Updated 10-16 05:46

About this skill

Triggers: /session-audit, "session audit", "auditar sesión", "audita la sesión", or automatically as the mandatory pre-close step (before mem_session_summary) when a session is being closed.


Purpose

Produce objective, cited findings on where a session violated the rules, and ROUTE each one to the place it will actually be acted on: a GitHub issue in agent-config for configurator-scoped findings, the project's own backlog for project-scoped ones. The audit is done by an independent sub-agent — never by the acting session agent — because the acting agent is blind to its own blind spots (judge-and-party).

The auditor remains read-only: it reports, it never fixes, and it never files anything itself. The ORCHESTRATOR routes what the auditor returns. Filing an issue is capture with a durable, actionable sink — not self-correction — so independence is preserved intact.

Findings are NOT accumulated on disk. Recurrence is visible through duplicate GitHub issues sharing a byte-identical [rule:<rule-id>] title prefix: the same rule violated twice reads as two adjacent issues in the list, with no filter and no synthesis step required.

Process

Step 1 — Resolve the transcript

  1. Read CLAUDE_CODE_SESSION_ID from the environment.
  2. Derive the transcript path: take the current working directory, replace every /, \, : with - to form the project-dir slug, and resolve ~/.claude/projects/<cwd-slug>/<CLAUDE_CODE_SESSION_ID>.jsonl.
  3. If the file does not exist, do NOT fail: report transcript unavailable — audit skipped in the close line, route nothing, and continue the close.

The transcript is JSONL, one record per line. A genuine user turn is type==="user" AND message.role==="user" AND typeof message.content==="string" AND message.promptSource present; array-content lines are tool_result echoes.

Step 2 — Launch the independent auditor sub-agent

Launch ONE auditor via the Agent tool (independent context — the orchestrator MUST NOT produce the findings itself). Rule-violations are near-deterministic string/turn matches, so one well-prompted auditor suffices; escalate to a second confirming auditor only if false-positives recur.

The auditor prompt MUST contain:

  • The transcript (the JSONL content, or a faithful digest of the turns).

  • A ## Rules to audit block — the VERIFIABLE rule-classes. Each carries its stable rule-id, which the auditor MUST echo verbatim on any finding matching that class. The vocabulary is CLOSED: never invent an id at close time, because a byte-identical id is what makes recurrence visible across sessions.

    rule-id Rule class Source
    commit-authorization Committed / pushed / amended without an explicit per-action user request CLAUDE.md Rules
    ai-attribution Added Co-Authored-By / AI attribution to a commit CLAUDE.md Rules
    build-after-changes Ran a build after changes CLAUDE.md Rules
    question-stop Continued past a question instead of stopping and waiting CLAUDE.md Rules
    close-summary-order Proposed to close/clear without first calling mem_session_summary CLAUDE.md Rules
    turn-closing-reflex Closed a turn with an unsolicited close offer (context under ~70% and the user did not ask) OR a commit/authorization request — including one emitted in a turn that itself poses an open question, or that would sweep in unrelated staged work CLAUDE.md Rules
    english-only-artifact Non-English content in an artifact (code/comments/commit/doc) CLAUDE.md Rules
    pushback-opener Used a banned pushback opener ("tienes razón", "you're right", "mi error", …) without a re-verify in the same turn ~/.claude/docs/protocols/pushback.md
    plan-mode-bypass Edited a complex / 2+-file / unverified-diagnosis task without entering plan mode ~/.claude/docs/protocols/plan-mode.md
    test-plan-dump Dumped a full test list in the first reply instead of asking step-by-step vs document ~/.claude/docs/protocols/test-plan.md
    afk-unsafe-question Used AskUserQuestion for an irreversible or high-cost decision ~/.claude/docs/protocols/afk-safety.md
    build-product-edit Hand-edited a build product (root CLAUDE.md, a spliced SKILL.md region, docs/wiki.html) instead of its source ~/.claude/docs/protocols/assembly-model.md
    close-report-narration Narrated machinery / phase names / duplicated status blocks at a cycle close ~/.claude/docs/protocols/close-report.md
    narration-gap A multi-tool-call stretch crossed a direction change (new goal, new target, state-changing pivot) with zero prose narration between tool calls CLAUDE.md Rules
    sdd-spec-citation Skipped a spec citation in an SDD artifact the sdd-* skills
    interpreted-grant-provenance An interpreted grant whose user_utterance does not appear verbatim (whitespace-normalized) in a genuine USER message of the transcript; or a grant record missing granted_by; or a marker provenance entry with granted_by: "model-interpretation" and no matching commit-grants.jsonl line core/protocols/interpreted-grants.md
    thread-hygiene ADVISORY — an orchestrating session (one that launched ≥1 pipeline phase sub-agent) ran mechanical shell WRITES inline (git add/commit/push, gh issue/pr edit-writes) instead of delegating a phase-close delivery unit, or launched a phase with no intent/outcome narration around it. EXCLUDES the three user-named inline actions: opening a PR, writing status:approved, and a push landing on the default branch agents/_shared/orchestrator-core.md
    delegation-inline-stretch Inline stretch crossed a tripwire threshold (~20 tool calls, 5 exploratory reads, or 2 non-mechanical edits, zero delegation), or a fired warning drew no inline-vs-delegate verdict ADR-052
    unclassified RESERVED — a real deviation matching none of the above n/a
  • The interpreted-grant verification procedure (rule interpreted-grant-provenance, source core/protocols/interpreted-grants.md) — the auditor runs it read-only:

    1. Read ~/.claude/logs/commit-grants.jsonl; keep lines whose cwd matches the session cwd and whose ts falls inside the transcript's time window; also read the live commit-auth marker's provenance entries with granted_by: "model-interpretation".
    2. For each record, normalize user_utterance (collapse whitespace runs to a single space, trim; case preserved) and search it as a substring of each normalized GENUINE user message — the transcript-shape rule Step 1 already pins: type==="user" AND message.role==="user" AND typeof message.content==="string" AND message.promptSource present. Array-content lines (tool_result echoes, task notifications) are NOT user messages, and a hit found only there is a FINDING, not a pass.
    3. Finding shape: rule_id: interpreted-grant-provenance, scope: configurator, moment: <grant ts + verb>, severity high for a fabricated or assistant-only citation, med for a marker/log mismatch (an interpreted marker grant with no matching log line), routed via the standard [rule:<id>] mechanism — duplicate issues are the recurrence signal.
  • A scope-classification instruction: every VERIFIABLE failure MUST carry a scope:

    • configurator — the violated rule's source of truth lives in the configurator (CLAUDE.md, core/protocols/*, a skills/* contract).
    • project — the finding concerns the code, conventions, or artifacts of the project the session was working in, and its correction belongs in that project's repository.
    • Ambiguous → configurator. State that the default was applied in scope_basis.
  • The return shape — the auditor returns this structure; the orchestrator parses it and does NOT re-derive any field:

    AuditResult:
      verdict:      "deviations" | "none" | "transcript-unavailable"
      failures:     Finding[]        # VERIFIABLE — routed
      observations: Observation[]    # SOFT — counted, never routed
    
    Finding:
      rule_id:      one of the ids above (never invented)
      rule_path:    e.g. "CLAUDE.md" | "core/protocols/pushback.md"
      scope:        "configurator" | "project"
      scope_basis:  short string; says so explicitly when the default was applied
      moment:       e.g. "turn 14" | "commit 4f2a1c"
      severity:     "low" | "med" | "high"
      description:  one line — becomes the human half of the issue title
      proposed_fix: one line
    
    Observation:
      description: string
      why_soft:    string
    

    Split findings into two groups: VERIFIABLE failures (citable with transcript evidence) and SOFT observations (persona, tone, response-length, style — judgment calls, NOT failures). If no deviation is found, return verdict: none.

  • A read-only constraint: the auditor reads and reports ONLY. It MUST NOT edit any file, fix anything, analyze other sessions, or file an issue / write a backlog entry itself — routing is the orchestrator's job (Steps 3 and 3.5).

Step 3 — Route each finding (orchestrator)

The orchestrator (NOT the auditor) routes what Step 2 returned. No file is written under docs/session-audits/ — that corpus no longer exists. SOFT observations are counted for the close line and are NEVER routed.

For each VERIFIABLE failure, resolve the destination by scope (Step 3.5), then emit:

GitHub issue (configurator-scoped, Rung 1):

Title:  [rule:<rule_id>] <description>
Labels: status:needs-review, type:chore
Body:
        **Rule**: <rule_path>
        **Moment**: <moment>
        **Severity**: <severity>
        **Proposed fix**: <proposed_fix>
        **Scope basis**: <scope_basis>
        ---
        Filed automatically by session-audit at close. Not user-requested —
        left at status:needs-review for triage.

Invocation: gh issue create --title "<title>" --body "<body>" --label "status:needs-review" --label "type:chore". Add --repo pointing at the configurator ONLY when the session's cwd is a different project; omit it when closing inside agent-config itself.

  • type:chore is the uniform type label. Every rule-class describes the assistant failing to follow a WORKING rule, not a malfunctioning rule mechanism, so no type:bug split is deterministically applicable at close.
  • status:approved MUST NEVER be applied. Per core/skills/issue-creation/logic.md:60 (Gate 1, row 2), an issue filed proactively by the assistant carries no user endorsement — and every audit finding is proactive by construction. The user applies it at triage.
  • When rule_id is unclassified, add to the body: **No rule-id matched** — the auditor found a deviation outside the enumerated vocabulary. Consider whether it needs a new entry.

Backlog item (any backlog rung, and all project-scoped findings) — appended via the project's backlog adapter so its ## Legend schema stays authoritative:

- [<next-id>] [rule:<rule_id>] <description> — <rule_path>, <moment>, severity <severity>.
  Proposed fix: <proposed_fix>. — state:Open north:audit:<rule_id>

The [rule:<rule_id>] prefix is retained VERBATIM in backlog lines too, so the recurrence signal reads identically in both destinations.

File a bare duplicate. Do NOT search existing issues before filing, do NOT comment on an existing issue instead of filing, and do NOT close, merge, or link duplicates. Repetition IS the recurrence signal — suppressing it would destroy the measurement.

No per-FINDING engram copy is written. Findings live in their destination: each finding lives only in its routed destination; a parallel engram row per finding would be a second, non-authoritative copy. This does NOT ban the run-level empty-ledger record in Step 4 — one row per RUN is a trace, not a finding copy.

Step 3.5 — Destination degradation chain

Evaluate in order, stop at the first rung that succeeds. Applicability is delegated to issue-creation Gate 0 (core/skills/issue-creation/logic.md:15-34) — do NOT restate its checks here.

Configurator-scoped:

  1. Gate 0 passes (github.com remote + gh authenticated + gate labels defined) → file the issue per Step 3.
  2. Remote + gh fine, but the label vocabulary is ABSENT → degrade SILENTLY to rung 3. Do NOT ask whether to bootstrap labels. This is a deliberate divergence from issue-creation Gate 0 (logic.md:33-34), which asks: at close the user is AFK by definition and the close MUST NOT gate on a prompt.
  3. No GitHub remote, gh unauthenticated, non-GitHub host, or the gh call fails → append to the configurator's backlog.store.
  4. No backlog store resolves → append to docs/BACKLOG.md (the floor).

Project-scoped: route via resolveStore(backlog.store) (core/protocols/backlog-capture.md:27-29): github → project-tracking; file → docs/BACKLOG.md; unset or unknown → file.

The chain is TOTAL: every finding reaches exactly one destination. A finding MUST NEVER be discarded — the floor always accepts.

Step 4 — Report to the user

Print ONE line naming the destinations, e.g. Session audit: 2 failure(s) routed (#128, #129), 1 observation(s) — or, when a rung degraded, Session audit: 1 failure(s) routed (BACKLOG.md BL-014), 0 observation(s). Then continue the close ritual (mem_session_summary, then session-handoff if requested).

Empty-ledger record (REQUIRED) — when the auditor returns verdict: none, persist ONE run-level record before the close line: mem_save (type discovery, topic_key: session-audit/{session-id}/run, capture_prompt: false) with content verdict: none — 0 failures routed, {N} observation(s). A clean audit MUST leave a durable trace.

No git operations at close — do NOT commit, branch, or push for a routed finding. Filing a GitHub issue is not a git operation and needs no branch, which is precisely why it is safe at an AFK close: committing would trip the branch-first reflex and strand a branch that push authorization (unavailable while AFK) can never resolve. A finding that degrades to a backlog file is left as an untracked working-tree change, exactly like docs/handoffs/.

Rules

  • The audit MUST be performed by an INDEPENDENT sub-agent (Agent tool); the orchestrator MUST NOT audit its own session inline (judge-and-party).
  • The auditor is READ-ONLY — it MUST NOT edit any source, config, or prior artifact, MUST NOT self-correct or re-run any action, and MUST NOT file an issue or write a backlog entry. It RETURNS findings; the ORCHESTRATOR routes them.
  • Every failure MUST cite the rule's source path and the moment — no uncited failures.
  • Every failure MUST carry a rule_id from the CLOSED vocabulary in Step 2. Never invent an id at close time: a byte-identical [rule:<id>] prefix is what makes recurrence visible across sessions. A real deviation matching no id uses unclassified and is still routed — never dropped.
  • close-summary-order and turn-closing-reflex are DISJOINT and MUST NOT be conflated. close-summary-order is about ORDER: a close was proposed without mem_session_summary running first — it presumes the close itself was warranted. turn-closing-reflex is about OCCASION: the close offer or commit request should not have been emitted at all, because no close signal existed (context under ~70%, user did not ask) or because the same turn poses an open question whose answer could change what would be committed. A single turn can violate BOTH, and then it yields TWO findings, one per id.
  • VERIFIABLE failures and SOFT observations MUST be reported SEPARATELY; soft/persona/style calls are observations, never failures. Observations are counted in the close line and are NEVER routed.
  • Every VERIFIABLE failure is ROUTED regardless of severity or recurrence count. There is no threshold gate: suppressing a first occurrence would prevent the duplicate-issue signal from ever starting.
  • ROUTING IS THE ACTUATION. Cross-session recurrence is NOT analyzed at close and needs no synthesizer — duplicate GitHub issues sharing the [rule:<id>] prefix ARE the recurrence signal, readable from the issue list without a filter. Findings that degrade to a local backlog do not aggregate across sessions; that is an accepted cost.
  • File a bare duplicate: no searching before filing, no commenting instead of filing, no closing/merging/linking. Dedup machinery would suppress the signal.
  • The degradation chain is TOTAL — every finding reaches exactly one destination; a finding MUST NEVER be discarded.
  • AFK-safe: routing MUST NOT gate the close on any prompt. The missing-label rung degrades SILENTLY to the backlog store — it MUST NOT ask whether to bootstrap labels.
  • A missing transcript is NOT a failure — report transcript unavailable — audit skipped, route nothing, and continue; never block the close.
  • The audit runs BEFORE mem_session_summary at close (per the CLAUDE.md pre-close rule).
  • Scope covers configurator-scoped AND project-scoped findings; ambiguous scope defaults to configurator.
  • No commit, no push, no branch, no git operations at close. Filing a GitHub issue is not a git operation and needs no branch — that is exactly why it is safe at an AFK close. A finding that degrades to a backlog file is left as an untracked working-tree change and rides along with the next session's commits, like docs/handoffs/. Committing at close would trip branch-first and strand a branch that push-authorization (unavailable while AFK) can never resolve — a specific skill rule that overrides the generic autonomous-commit allowance.