SuperAPI AI Directory

skill-audit

Coding & development

View repository

Updated 11-05 18:17

About this skill

Skill Audit

Broad scan across all skills (or a filtered subset) to surface consistency issues, quality gaps, ownership conflicts, and ecosystem-level problems that individual reviews miss.

When to Use

  • Periodic health check on the skill ecosystem
  • After bulk edits or a new batch of skills is added
  • Before a release to validate ecosystem integrity
  • Investigating systemic issues (e.g., "why do none of my skills trigger?")

Inputs

  • Scope — all (default), or a filter: category (roles, meta, contracts, workflows, orchestrator), or a list of specific skill names
  • Focus (optional) — narrow the audit to specific checks (e.g., "just ownership", "just descriptions")

Process

Phase 1: Discovery

Enumerate all skills in scope:

  1. Glob for skills/**/SKILL.md
  2. Parse frontmatter from each
  3. Build a skill inventory table:
| Skill | Category | Version | Description Length | Body Lines | Refs Count |

Phase 2: Bulk Checks

Run these checks across all skills in scope. Use subagents to parallelize where possible — each check category can run independently.

2a. Frontmatter Consistency

For every skill:

  • Required fields present (name, version, description)
  • Version is valid semver
  • Name matches directory name
  • Description starts with action verb
  • Description length vs 200-char target

Produce a table of violations.

2b. Ownership Conflict Detection

For agent role skills only:

  • Collect all owns.directories declarations
  • Collect all owns.patterns declarations
  • Check for overlaps (two agents claiming the same directory)
  • Validate against the v1.1 resolved conflicts table in frontmatter-spec.md
  • Flag any new conflicts

Produce a conflict map.

2c. Description Quality Scoring

For every skill:

  • Has action verb? (boolean)
  • Has ≥3 trigger contexts? (count)
  • Has keyword variants? (boolean)
  • States exclusions if ambiguous? (boolean)
  • Estimated "pushiness" (low/medium/high)

Produce a ranked list from weakest to strongest descriptions.

2d. Progressive Disclosure Check

For every skill:

  • SKILL.md body line count
  • Number of reference files
  • Are references linked from body?
  • Any body >500 lines? (flag)
  • Any reference >300 lines without TOC? (flag)

2e. Cross-Skill Consistency

Check for ecosystem-level issues:

  • composes_with references that point to non-existent skills
  • spawned_by references that point to non-existent skills
  • Circular composes_with chains (A→B→A)
  • Skills declaring requires_agent_teams: true that don't degrade gracefully
  • Orphan reference files (exist but aren't linked from any SKILL.md)

2f. Coverage Gap Analysis

Compare the skill inventory against the project's canonical documentation (docs/architecture.md, CLAUDE.md, and the orchestrator's File Ownership Map):

  • Are all roles described in the architecture doc implemented as skills?
  • Are all workflows the project depends on covered?
  • Any capabilities the orchestrator's playbook references but no skill exists for?

Phase 3: Generate Audit Report

Produce a structured markdown report:

# Skill Ecosystem Audit Report
Audited: [timestamp]
Scope: [all / filtered description]
Skills scanned: [count]

## Executive Summary
- Total skills: X
- Passing: X (X%)
- Issues found: X (X critical, X high, X medium, X low)
- Ownership conflicts: X

## Skill Inventory
| Skill | Category | Version | Desc Length | Body Lines | Refs | Status |
|-------|----------|---------|------------|------------|------|--------|

## Ownership Map
[Visual or tabular representation of which agent owns what]

### Conflicts
[Any overlaps detected]

## Description Quality Ranking
| Rank | Skill | Score | Issues |
|------|-------|-------|--------|
[Weakest first]

## Bulk Issues

### Frontmatter Violations
| Skill | Field | Issue |
|-------|-------|-------|

### Progressive Disclosure Violations
| Skill | Body Lines | Issue |
|-------|-----------|-------|

### Cross-Skill Issues
| Issue Type | Skills Involved | Description |
|-----------|-----------------|-------------|

## Coverage Gaps
[Capabilities referenced in `docs/architecture.md`, `CLAUDE.md`, or orchestrator phase-guide that no existing skill covers]

## Recommendations
[Top 5 highest-impact improvements, ordered by priority]

## Per-Skill Scores
| Skill | Frontmatter | Description | Disclosure | Consistency | Overall |
|-------|-------------|-------------|------------|-------------|---------|
[Quick scores for each skill — details available via skill-deep-review]

Phase 4: Triage

After generating the report:

  1. Highlight the top 3–5 most impactful issues
  2. Suggest which skills would benefit from a skill-deep-review for further investigation
  3. Note any issues that skill-improvement-plan could address directly

Tell the user:

"Audit complete. [X] issues found across [Y] skills. The top priorities are [brief list]. Feed this report into /skill-improvement-plan to generate fix plans, or use /skill-deep-review on [specific skills] for deeper investigation."

Guidelines

  • Optimize for speed — this is a broad scan, not a deep dive. Score quickly, flag issues, move on.
  • Use subagents to parallelize check categories when available
  • Don't read reference file contents unless checking for specific issues (orphans, broken links) — frontmatter + body line count is enough for most checks
  • Focus on ecosystem-level patterns that individual reviews miss: conflicts, inconsistencies, gaps
  • If scope is filtered, still check cross-skill references against the full inventory

Reference Files

  • references/audit-checklist.md — Quick-reference checklist for all bulk checks