SuperAPI AI Directory

context-gap-analyzer

Agents & models

View repository

Updated 10-15 16:49

About this skill

Context Gap Analyzer

Code is necessary but insufficient context. The delta between "what the code says" and "what a competent team member knows" is where agents fail most expensively. Identify that delta, prioritize by agent impact, surface focused questions a human can answer.

What agents cannot derive from code: Why (rationale), What conventions aren't enforced by tooling, How things interact beyond repo boundaries, When to break vs. follow a pattern, Who owns what.


PHASE 1 — Context Harness Discovery

Discover all context infrastructure already in place. Actively explore — don't rely on a checklist.

Layer 1: File-level — Scan repo root + 2-3 levels deep for "instructions for someone working here." For each: read it, inventory topics, note scope (repo/dir/module), format, audience.

Layer 2: Toolchain — MCP servers, skills/plugins, hooks/middleware (git hooks, CI, pre-commit, linting), IDE/editor configs.

Layer 3: Runtime — Connected services (Drive, Notion, Confluence, Slack), env/secrets, package metadata.

Layer 4: Delivery classification

Delivery Examples Implication
Static AGENTS.md, .cursorrules, system prompts Token cost; must be concise
On-demand MCP servers, search, skill files Richer; agent must know when to fetch
Triggered Git hooks, CI checks, pre-commit Enforcement without why
Implicit Types, tests, linter configs Partial; rationale missing

Output: inventory of static files (path, topics, scope, format), toolchain context (MCP, skills, hooks), runtime context (services, config injection), delivery map with counts, already-documented topics, primary delivery format (determines Phase 6 output).

Be thorough before concluding "nothing exists" — .eslintrc, Makefile, docker-compose.yml are all context. Question is sufficiency.


PHASE 2 — Codebase Topology Scan

Map: directory structure (modules, depth), technology fingerprint (languages, frameworks, build tools), complexity indicators (file counts, nesting, config variety), integration surface (external services, APIs, DBs, queues), entry points (routes, CLI, event handlers).

Output: codebase type, languages, framework, top-level modules ranked by complexity, integration surface, high-traffic areas for agents.


PHASE 3 — Gap Analysis

Cross-reference Phase 1 (documented) against Phase 2 (code). Evaluate nine categories:

Cat Name Agent needs Gap indicator
C1 Architecture & Boundaries Component topology, data flow, module dependency rules Cross-boundary imports, code in wrong module
C2 Domain Model & Business Rules Logic rationale, vocabulary, invariants Technically correct but domain-wrong code
C3 Conventions & Patterns Naming, file org, error handling, logging Code that works but "feels wrong"
C4 Integration & External Deps API patterns, retry/fallback, rate limits, auth Incorrect service calls, missing retry
C5 Operations & Deployment CI/CD, feature flags, rollback, monitoring Code that breaks CI, unmonitored failures
C6 Testing Philosophy Unit vs. integration, mocking, fixtures Wrong test type, mocks at wrong boundaries
C7 Security Model Auth patterns, data classification, secrets Auth bypasses, logged sensitive data
C8 Performance Constraints Bottlenecks, caching, query patterns, pagination N+1 queries, skipped caching, unbounded queries
C9 Historical Decisions & Debt Why this way, migrations, deprecated patterns Extending deprecated, building on doomed code

Scoring

Factor Range Meaning
Documentation coverage 0-3 0=nothing, 1=mentioned, 2=partial, 3=thorough
Code complexity 1-3 1=simple, 2=moderate, 3=complex/non-obvious
Agent exposure 1-3 1=rarely touched, 2=sometimes, 3=frequently modified

Gap severity = Complexity x Exposure - Doc coverage (range: -2 to 9)

Score Priority
>= 5 Critical
3-4 Significant
<= 2 Acceptable

Output: table with all 9 categories scored, overall coverage % (Doc sum / 27 x 100), critical gaps listed.


PHASE 4 — Prioritized Question Generation

For each gap >= 3, generate focused questions answerable in 2-5 minutes. Each targets one piece of implicit knowledge; the answer must be directly usable as agent context.

Q[N] — [Category] — Priority: [Critical/High/Medium]
[Specific, focused question]

Why this matters: [what goes wrong without this context]
Good answer example: [3-5 sentence template showing detail level]

Generate 10-20 questions. At least 2 per critical gap, 1 per significant gap. Order by gap severity x actionability.


PHASE 5 — Coverage Map

CONTEXT COVERAGE MAP — [repo] — [date]

C1 Architecture      [████░░░░░░]  40%  ⚠️ Significant
C2 Domain Model      [█░░░░░░░░░]  10%  🔴 Critical
...
OVERALL: [X]% | Questions: [N] | Est. time to close critical gaps: ~[N] min

Use interactive visualization (radar, heatmap) when supported.


PHASE 6 — Tracking & Integration

Create .context-coverage.json at repo root with: version, repo, dates, harness info (primary format, static sources, toolchain, delivery summary), per-category scores, questions (id, category, priority, question, status, answer, integrated_to), overall coverage.

When the user answers a question:

  1. Store answer, mark "answered"
  2. Write into the harness matching existing conventions (tone, structure, format, scope)
  3. Record integrated_to path, recalculate coverage, show updated map

Integration principle: Adapt to the harness, never prescribe. Match existing voice and format. If no writable harness exists, ask where to write.


Incremental Modes

  • First run: Full Phases 1-6, baseline + all questions
  • Answer session: Integrate answers, update coverage
  • Re-audit: Re-run Phases 2-5 after significant changes
  • Coverage check: Phase 5 only from existing .context-coverage.json

Detect mode from whether .context-coverage.json exists and what the user asks.


Calibration

  1. Agent-first: Frame every question as "what would an agent get wrong?" not "what's undocumented?"
  2. Precision over completeness: 10 high-impact questions beat 50 thorough ones. Human time is the bottleneck.
  3. Respect existing context: Cross-reference before asking — don't duplicate.
  4. Harness humility: Detect and work within existing systems. Don't prescribe.
  5. Actionable answers: The example answer prevents both one-word and novel-length responses.
  6. Coverage honesty: Score based on what an agent would find useful, not what technically exists.