docs-freshness-audit

Docs Freshness Audit

A procedure document for auditing whether docs have diverged from the implementation and guiding their disposition as "update / delete / migrate to ADR". Spec documents left abandoned with banners, specs that keep describing modules that do not exist, and plan files that remain even though they have been merged will, if left in place, become a source of confusion rather than help.

Step 0: Confirm the deployment destination settings

The skill body itself contains only procedures and prompts. Start after gathering the following from the deployment destination's CLAUDE.md / rules / documentation maintenance policy (if absent, confirm with the user):

Item	Example
docs verification command	`make docs-okf`, `python scripts/validate_docs.py`
Audit exclusions	Bundles such as ADRs/designs (left to each party's own management rules), generated artifacts
Plan/design output directory	`docs/plans/`, `docs/superpowers/plans
Append destination for maintenance history	`docs/documentation-maintenance.md`
Deletion operations	Push destination rules for docs chores (whether direct push is allowed)

Phase 1: Mechanical scan (cheap, run every time)

Each scan is completed with bash. Consolidate the results into a "list of suspect files (with evidence)".

Dead source path references: Extract source-path-like strings in docs and check whether they exist

git ls-files docs | grep "\.md$" | while read doc; do
  grep -oE "src/[A-Za-z0-9_/.-]+\.[a-z]+" "$doc" | sort -u | while read p; do
    [ -e "$p" ] || echo "$doc -> $p"
  done
done

Dead internal links: Check whether the targets of markdown relative links (](path)) exist
Self-declared banner detection: DEPRECATED / Superseded / 要確認 / TBD / 未解決 / 確認事項 with grep. Documents left with banners attached are treated as suspects as "known sources of confusion"

Freshness gap: Compare the last commit date of the doc with the recent churn of the code it describes

git log -1 --format=%ad --date=short -- "$doc"   # doc 側
git log --since="$doc_date" --oneline -- "$src_area" | wc -l   # コード側の乖離量

Remnants invisible to git: Enumerate ignored / untracked files in the audit target tree. git rm only deletes tracked files, so generated artifacts you thought you deleted remain on disk
```
git status --ignored --porcelain docs/ | grep "^!!\|^??"
```
Forgotten plan files: Extract Issue/PR numbers from each file in the plan output directory and gh query their status.If all are closed/merged, treat as "leftover completed plans" and mark as deletion candidates. Plans with no number reference are treated as suspects based on elapsed days and sent to Phase 2
```
grep -oE "#[0-9]+" "$plan" | sort -u | while read n; do
  gh issue view "${n#\#}" --json state --jq .state 2>/dev/null \
    || gh pr view "${n#\#}" --json state --jq .state
done
```
Overlap with model default behavior and lint: Check whether each section of rules / CLAUDE.md restates matters that lint (Ruff / mypy) detects (type hint syntax, import order, line length, dangerous functions) or behavior that the current model already performs without instructions (running tests, fixing root causes, strictly adhering to scope). Since this is scaffolding that becomes stale with every model update, treat as suspects every 3~6 months and send to Phase 2

Phase 2: Semantic audit (suspect files only, parallel read-only agents)

For the suspect files from Phase 1, dispatch read-only agents in parallel to compare docs against the implementation. Mechanical scans cannot detect divergences of the kind where "an unused old algorithm is described as the correct one", so do not omit this phase (during deep audits).

Prompt template:

<repo> で、ドキュメント <doc> (最終更新 <date>) と現在の実装の乖離を監査してください。

手順:
1. <doc> を全文読む
2. doc が言及するモジュール/クラス/関数/ウィジェットを現在の <src candidates> で確認
3. 乖離を列挙: (a) doc に書かれているが実装に存在しない要素、
   (b) 実装にあるが doc に無い主要要素、(c) 名前・シグネチャ・アルゴリズムの不一致
4. 同領域をカバーしうる live ドキュメント (<live doc candidates>) の有無を確認

出力: 乖離項目の箇条書き (doc 行番号 + 実装 file:line)。最後に判定を1つ:
「概ね正確 / 部分修正で足りる / 全面書き直しが必要 / 削除可 (live docs がカバー)」+ 根拠 1-2 文。

Note: Re-verify for yourself whether the agent's "unreferenced" reports are tracked or ignored (there are real cases where generated artifacts were actually git-tracked and overlooked).

Phase 3: Disposition and execution (user approval gate required)

Decision matrix:

State	Disposition
live docs / implementation already covers the same area, no unique value	Delete (git history preserves it)
Specification of current functionality but description is outdated	Update (reflect implementation verification results)
Valuable only as a record of design decisions	Move to ADR and delete the original
Matches the implementation	Add an audit confirmation note (with date) and keep
Completed plan file	Delete
rules / CLAUDE.md sections that restate lint / model default behavior	Delete, or convert anything you want enforced every time into a hook

Present the candidate list to the user with supporting evidence, andafter obtaining approval,execute it (do not delete on your own). Grouping them (definite deletions / needs judgment / local only) and asking with multiple selection is faster
After execution, run the Step 0 verification command and leave an audit record (what was deleted/fixed and why) in the maintenance history
Tool default output locations (such as directories a skill automatically writes to) should "not be deleted even if they look cluttered" - deleting them just makes the tool regenerate them. Instead, clearly state the division of roles in the documentation maintenance policy

Execution mode

Light audit (monthly guideline): Phase 1 only -> if there are zero suspects, record and finish; if there are any, report
Deep audit (quarterly guideline / after large-scale refactoring or renaming): Phase 1 + 2 + 3 full set

The frequency is left to the adopter's operations (e.g., a person triggers it in the same slot as the monthly dependency review).