SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsbase-in-reality

base-in-reality

Writing & research

View repository

Updated 10-15 21:09

About this skill

base-in-reality

The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY in this skill are to be interpreted as described in BCP 14 (RFC 2119, RFC 8174) when, and only when, they appear in all capitals.

A read-only, research-grounded repository audit. It answers one question for each notable design decision in a repo: does this violate an established norm, standard, algorithm, or best practice — provably, against a real source?

The defining rule: no fabricated authority. Every finding MUST be tied to a source the agent actually fetched in-session. Ungrounded claims MUST be reported as UNCONFIRMED, and MUST NOT be reported as violations, because a violation with no fetched source is fabricated authority.

When to use

Use when you want to sanity-check a codebase, architecture, or business logic against the state of the art and against domain standards — e.g. "is our APR calculation correct per lending norms?", "does our crypto follow NIST?", "is this consensus approach sound?", "does our ML eval avoid known leakage pitfalls?". Works on any repo; adapts to its domain(s).

Invariants (do not violate)

  1. You MUST NOT edit code, because this is a read-only audit and every fix is the user's call. --annotate inserts comment markers only, never logic. The one file this skill creates on the default path is its own report under docs/base-in-reality/; you MUST name it before writing it, and MUST offer the in-conversation report instead if the user wants their tree untouched.
  2. No fabricated citations. You MUST cite only URLs/DOIs fetched this session, and MUST prove it: fetch with BIR_EVIDENCE_LOG set, then lint with --evidence. fetched: true is a claim the agent makes about itself — the evidence log is what makes it checkable. Ungrounded ⇒ UNCONFIRMED. See references/verdict-rubric.md.
  3. Adversarial gate. A VIOLATION/DEVIATION MUST NOT be reported unless it survives a refutation pass, so that a finding the first verifier over-read is caught before it reaches the report.
  4. No silent truncation. If --max-claims caps extraction, you MUST list what was dropped in the report's Dropped-claims log.

Flags

  • --annotate — also insert # BASE-IN-REALITY[<sev>]: <one-line> — see <report> markers at finding locations (the only code-touching mode). Default: off.
  • --layer algo|arch|biz — restrict to one layer. Default: all three.
  • --domain <x> — override auto-detected domain. Default: auto.
  • --since <ref> — audit only the diff vs <ref>. Default: whole repo.
  • --max-claims N — cap claims per run. Default: 40.

Procedure

Python interpreter. Most commands in this skill and its references are written with python3. Find the interpreter once, before the first of them. If $SKILL_PYTHON is set, use it. Otherwise, on macOS, if command -v python3 prints /usr/bin/python3 and xcode-select -p fails, that python3 is the install stub: skip it without running it. Use the first of python3, python and py -3 that is not skipped and for which <interpreter> -c 'import sys; sys.exit(sys.version_info < (3, 10))' exits 0. Write the interpreter you found in place of python3 in every command, and in any Bash(...) permission rule you suggest. Commands that start with uv run or {python} keep their own form. In a session resumed under a Bash(...) allowlist, skip this lookup and use the interpreter that the allowlist names. If none passes, follow this skill's own fallback for running without Python where it gives one; otherwise stop and tell the user that this skill needs Python 3.10 or newer.

Run these six stages. Dispatch subagents where noted (fan-out). On Claude Code you may instead feed assets/workflow.mjs to the Workflow tool as a deterministic accelerator — it performs stages 2/4/5 with parallel fan-out; the steps below are the portable path.

Locating the source helper (do this first). The bundled fetch_sources.py is referenced by a path relative to THIS skill's directory, but the verify subagents (stage 4) run from the target repo, not from here — a path like assets/fetch_sources.py will not resolve for them. Before dispatching any subagent, resolve the helper's ABSOLUTE path once and pass it into each subagent's prompt:

  • Use this skill's base directory (your harness provides it when the skill loads) and set FETCH="$SKILL_DIR/assets/fetch_sources.py".
  • If you don't have the base directory, discover it: FETCH=$(find ~/.claude ~/.config ~/.agents -path '*base-in-reality*/assets/fetch_sources.py' 2>/dev/null | head -1)
  • Verify it resolves: python3 "$FETCH" --source openalex --query test --limit 1 should emit JSON. (uv run "$FETCH" is equivalent; the script declares dependencies = [] and imports only the stdlib, so python3 works on any host and does not make uv a prerequisite.)
  • You MUST hand subagents the literal absolute $FETCH value, because a relative assets/-prefixed form will not resolve from the target repo.
  • Open the evidence log before any fetching, and export it so every subagent inherits it: export BIR_EVIDENCE_LOG="$(mktemp -t bir-evidence-XXXXXX.jsonl)". Each retrieval appends the URL/DOI actually returned, and stage 6 reconciles the report against it. Without this, a fetched: true flag is only the agent's word for itself and the linter says so.

  1. Scope & domain detection. Survey languages, dependency manifests, directory layout, README/docs, DB schemas/migrations, config. Classify the domain(s) using references/domains.md detection signals; fix the norms surface. Honor --domain and --since. Produce the domain map.

  2. Claim extraction (fan-out). Dispatch subagents over repo regions. Each returns discrete, FALSIFIABLE claims tagged {layer (algo|arch|biz), location (file:line), confidence, domain-tag}. A claim is a checkable assertion about what the code does (e.g. "uses MD5 for password hashing", "synchronous fan-out to 12 services on the request path", "APR computed as simple interest"). Merge and dedup. Enforce --max-claims; record any surplus for the Dropped-claims log.

  3. Source routing. For each claim, pick a ranked source plan from references/source-routing.md (algorithm→arxiv/Semantic Scholar; security→NIST/RFC/ OWASP; medical→PubMed; fintech→regulator+standards; general→Scholar/JSTOR/OpenAlex).

  4. Verify (fan-out, grounded). Per claim, dispatch a subagent that queries keyless APIs first via the bundled source helper at the absolute $FETCH path resolved above (python3 "$FETCH" --source <s> --query "<q>" --limit <n> — pass the literal absolute path into the subagent; a skill-relative assets/-prefixed path will not resolve from the target repo), then WebSearch/WebFetch for standards and paywalled sources. It MUST hold ≥1 fetched source before asserting anything stronger than UNCONFIRMED. Output a finding conforming to assets/findings.schema.json: verdict ∈ {VIOLATION,DEVIATION,OUTDATED,UNCONFIRMED}, severity ∈ {critical,high,medium,low}, citations[] (each with the fetched URL/DOI and a supporting quote; mark abstract_only where only the abstract was available), recommended_fix.

  5. Refute (fan-out, adversarial). For each candidate VIOLATION/DEVIATION, dispatch independent refuters across distinct lenses (correctness, citation-applicability, severity) per references/verdict-rubric.md. Each defaults to skeptical. Downgrade to UNCONFIRMED when ≥2 of 3 refute a medium/low finding, or when any refuter refutes a finding of any other severity (critical, high, or a missing or unknown one). A refuter that returns nothing counts as a refute. Record the votes in the finding's refutation field, whether it survives or not.

  6. Synthesize. Before filling the report, lint the merged findings array with the bundled contract linter: write the findings to a temp JSON file and run python3 "$SKILL_DIR/assets/report_lint.py" <findings.json> --evidence "$BIR_EVIDENCE_LOG". It deterministically enforces the schema enums and the grounding invariant (a VIOLATION/DEVIATION with no fetched citation is rejected — downgrade it to UNCONFIRMED rather than shipping it), and with --evidence it checks each fetched: true citation against the URLs a retrieval actually returned, so a plausible-looking but never-fetched DOI fails instead of rendering as grounded. It also checks refutation: a surviving VIOLATION/DEVIATION with no recorded refutation, or with votes that stage 5 says downgrade it, is rejected. A PASS means the recorded votes agree with the verdict, not that the votes were honest. You MUST run it WITH --evidence: the result line states which mode ran, and a report linted without it is only shape-checked. Fix every ERROR: line, then fill assets/report-skeleton.md. Writing the report to docs/base-in-reality/<YYYY-MM-DD>-audit.md is this skill's one expected write outside --annotate; you MUST say so before creating it, and emit the report in-conversation instead if the user would rather keep their tree untouched. The report contains: executive summary, domain map, findings (ordered by severity then layer), sources appendix, dropped-claims log. If --annotate, insert the comment markers at each finding's location.

Output style. Write reports and explanations for the user in about 80% ASD-STE100 Simplified Technical English. Use short sentences, common words, active voice and one action per step.

References

  • references/source-routing.md — claim-class → source-class table, API usage, etiquette.
  • references/verdict-rubric.md — verdict taxonomy, severity, grounding rule, refutation.
  • references/domains.md — domain-detection signals and per-domain norms surface.

Assets

  • assets/fetch_sources.py — keyless scholarly-source query helper (stdlib; uv run).
  • assets/findings.schema.json — the finding schema verification subagents emit.
  • assets/report_lint.py — stdlib linter that gates findings on the schema, the grounding invariant and the refutation rule (stage 6).
  • assets/report-skeleton.md — the report template filled in stage 6.
  • assets/workflow.mjs — optional Claude Code Workflow accelerator.

Edge cases

  • Uneven source coverage. arxiv/PubMed are strong; JSTOR/ISO are reachable only via abstracts — mark those citations abstract_only so evidence depth is visible.
  • Domain misdetection. Surface the domain map at the top of the report; --domain is the override.
  • Large repos. Deep mode fans out twice per claim (verify + refute) and is token-heavy. Scope with --layer, --since, or a lower --max-claims.
  • "Best practice" is contested. Prefer DEVIATION over VIOLATION for defensible departures; the refutation pass exists to stop the skill asserting one true way.