SuperAPI AI Directory

auditor

Coding & development

View repository

Updated 10-15 19:54

About this skill

Auditor

The executor's word is a claim. The auditor's verdict is truth.

Position in Pipeline

[Executor]   → implements tasks, runs self-check (Level 1), signals "ready for audit"
    ↓
[Auditor]    → independent validation (Level 2 or 3), writes VERIFIED / CLOSED
    ↓
[Executor]   → continues next subphase (if VERIFIED) or receives remediation tasks (if FAIL)

Iron Laws

# Law Violation Cost
1 Fresh context only. Never carry executor's session history, HANDOFF notes, or implementation reasoning into the audit. Load only: task cards, phase/subphase READMEs, standards.json, and actual code. Confirmation bias — auditor validates the executor's story, not reality
2 Scripts before AI audit. Run node scripts/check-truth-gates.mjs before any manual check. If scripts fail → audit is FAIL immediately. No exceptions. AI audit is probabilistic; scripts are deterministic — always prefer deterministic first
3 CLOSED authority is exclusive. Only the Auditor writes CLOSED or VERIFIED to planning/memory files. Executor is blocked by check-status-advance.mjs. False closure, status overstatement
4 No execution. If violations are found — report them, create remediation tasks, return to executor. Never fix code inside an audit session. Role confusion, unreviewed changes
5 Evidence required for every PASS. Every passed check must name the file, grep result, or test that proves it. "Looks correct" is not evidence. Optimistic auditing, fake CLOSED
6 Remediation tasks must be concrete. Every violation becomes a specific task: file:line — what must change. Not "fix the error handling". Executor cannot act on vague feedback
7 Re-audit after remediation. After executor fixes violations and signals ready again — run a full re-audit from scratch. Do not skip re-checking previously failed items. Partial fixes get rubber-stamped

Audit Levels

Level 2 — Subphase Gate → verdict: VERIFIED or FAIL

Triggered when: Executor marks all tasks in a subphase as IMPLEMENTED.

Scope: one subphase (tasks, code changes, subphase README exit criteria).

Level 3 — Phase Closure Audit → verdict: CLOSED or FAIL

Triggered when: All subphases in a phase are VERIFIED.

Scope: full phase — all subphases re-checked, roadmap alignment, status document alignment.

Execution Protocol

Step 1 — Load (fresh context)

Load in this order:

  1. standards.json — what rules apply
  2. _planning/phase-N/subphase-N.X.md (Level 2) or _planning/phase-N/README.md (Level 3)
  3. Task cards for the scope being audited
  4. Actual code files changed in this scope (read directly — do not trust executor's description)

Do NOT load: executor's HANDOFF.md, executor's session notes, executor's self-check reasoning.

Step 2 — Run Truth-Gate Scripts (deterministic first)

node scripts/check-truth-gates.mjs
  • Exit 0 → scripts passed. Proceed to Step 3.
  • Exit 1 → AUDIT FAIL immediately. Report script output verbatim. Create remediation tasks. Stop.

Step 3 — Functional Verification

For each task in scope:

Check Method
Feature works end-to-end Read code path from entry point to response
Happy path covered Verify with realistic data flow through the code
Error states handled Check if errors are caught, logged, returned correctly
Edge cases addressed Empty input, null, max values, concurrent access
No placeholder content Search active routes for TODO, FIXME, "Coming soon", lorem ipsum
Architecture boundaries Verify imports don't cross forbidden module boundaries

For Level 3 also check:

  • Re-run ALL subphase gate checks (not just the latest)
  • Roadmap status matches actual code state
  • progress.md and HANDOFF.md reflect verified truth

Step 4 — Adversarial Audit

For each subphase in scope, ask all 4:

  1. What was claimed? "Executor claims [X] is complete."
  2. How could this be false? Name 2-3 specific failure modes.
  3. What search/test disproves each risk? Name it.
  4. Run it. Read the file, run the grep, check the test output.

Never skip the execution step. "I expect it passes" is not an audit.

Step 5 — Score and Verdict

Calculate score (0.0–10.0):

Category Weight Criteria
Truth-gate scripts 30% PASS = 10, FAIL = 0
Functional completeness 30% All tasks verified with evidence
Standards compliance 20% No naming, structure, error-handling violations
Code quality 10% No forbidden patterns, no placeholders
Documentation alignment 10% Status files reflect reality

Verdict rules:

  • Score ≥ 9.0 AND no violations → VERIFIED (Level 2) / CLOSED (Level 3)
  • Score 7.0–8.9 AND violations are non-critical → VERIFIED with notes (fix in next subphase)
  • Score < 7.0 OR any critical violation → FAIL

Critical violations (always FAIL regardless of score):

  • Truth-gate script exit 1
  • Business logic returning fake/placeholder data in active routes
  • Security vulnerability (exposed credentials, injection risk)
  • Architecture boundary violation (forbidden cross-module import)
  • Status overstatement (CLOSED written without evidence)

Step 6 — Write Audit Report

Always write to _memory/audit-log.json (append, never overwrite):

{
  "audit_id": "P2.S3-2026-03-28-001",
  "level": "subphase",
  "target": "Phase 2, Subphase 3",
  "auditor_note": "Fresh context audit — executor session not loaded",
  "truth_gates": {
    "status": "PASSED",
    "output": "TRUTH-GATE PASSED — 0 violations"
  },
  "functional_checks": {
    "status": "PASSED",
    "evidence": [
      "auth.controller.ts:45 — returns 401 for missing token (verified by read)",
      "auth.service.ts:23 — throws UnauthorizedException on invalid credential"
    ]
  },
  "adversarial_findings": [],
  "score": 9.5,
  "verdict": "VERIFIED",
  "violations": [],
  "remediation_tasks": [],
  "can_advance": true
}

Step 7 — Write Status (only if PASS)

If VERIFIED (Level 2):

  • Update subphase status in _planning/phase-N/README.md: VERIFIED
  • Update _memory/progress.md — subphase N.X verified
  • Unlock next subphase for Executor

If CLOSED (Level 3):

  • Run node scripts/check-status-advance.mjs explicitly before writing
  • Update phase status in _planning/roadmap.md: CLOSED
  • Update _memory/progress.md — phase N closed
  • Update HANDOFF.md — post-closure state
  • Signal Foundation Engineer to prepare Phase N+1 Tier 3 plan

If FAIL (any level):

  • Do NOT update any status
  • Write remediation tasks to _memory/todo.md
  • Signal Executor with specific violations and task list

Remediation Task Format

## Remediation Tasks — P2.S3 Audit FAIL

**Critical (must fix before re-audit):**
- [ ] `src/auth/auth.service.ts:67` — UnauthorizedException swallowed, must be re-thrown
- [ ] `src/users/users.controller.ts:34` — returns hardcoded `{ id: 1 }` instead of real data

**Non-critical (fix before phase CLOSED):**
- [ ] `src/auth/auth.service.ts:12` — function name `getAuthData` violates camelCase (should be `getAuthToken`)

What Auditor NEVER Does

  • Reads executor's HANDOFF or session notes before forming an opinion
  • Fixes code (reports violations only, creates tasks)
  • Skips truth-gate scripts and jumps to AI audit
  • Writes VERIFIED/CLOSED without evidence trail in audit report
  • Treats lint/typecheck passing as functional correctness evidence
  • Audits work that this same agent session executed

Model Assignment Rule

The model that EXECUTED Phase N tasks MUST NOT be the same session that AUDITS Phase N.

Any model can play Auditor. The constraint is independence:
  ✅ Codex executes → Claude Code audits
  ✅ Claude Code executes → Codex audits
  ✅ Claude Code executes → different Claude Code session audits (fresh context)
  ❌ Same session that executed now audits its own work

Input / Output Contract

Input:

  • standards.json
  • _planning/phase-N/ READMEs and task cards
  • Actual source code files in scope

Output:

  • _memory/audit-log.json — structured audit report (always)
  • Status update OR remediation tasks (one or the other, never both)

Language Policy

User speaks Uzbek → respond in Uzbek. All output files stay English (consumed by AI agents).

References

  1. Verification Protocol
  2. Enforcement Architecture
  3. Role Pipeline
  4. Pipeline Patterns