SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsharness-tour

harness-tour

Coding & development

View repository

Updated 10-15 23:24

About this skill

Skill: harness-tour

WHAT

Purpose and context

  • Purpose: To let the user SEE (not read) the harness in action: you — Claude — deliberately violate a rule on a real project, get blocked by the hook in real time, and narrate it. End without leaving behind junk.
  • Trigger (when to use): user types /harness-tour or says "harness tour", "see what the harness does", "test short", "test full" (disable-model-invocation: true — only runs when the user calls it). Two lengths: SHORT (default, R1·R2·R3) and FULL (R1–R10).
  • Non-goals: do not install/modify the harness (only suggest the install command when MISSING), do not fake it when there is no hook, do not touch the project's existing files, do not replace demo.sh/test-broad.sh (the machine-performed version does not need a session).

Mental model

ARGUMENTS → độ dài (SHORT|FULL) → pre-check harness (POC|PROD|MISSING) → mỗi cảnh: tường thuật → vi phạm có chủ đích trên path map 1-rule → hook chặn live → trích nguyên văn → khắc phục → dọn dẹp tour-demo* + row index cùng lượt → kết màn trích audit.jsonl. Rules split into 3 groups: block-live (R1,R2,R3,R5,R7,R9) · write/remind without blocking (R4,R8,R10) · repo layer (R6).

Input and output contract

Field Required? Meaning
In ARGUMENTS no contains full/10/"full" → FULL; otherwise → SHORT
In session open at the root of a project with the harness installed yes .claude/settings.json must be loaded, otherwise MISSING
Out narration + block stderr quoted verbatim for each scene yes e.g. [R1 no-write-raw] …, [R3 index-sync] …
Out clean project yes every tour-demo* that PASSed is deleted + the corresponding row index; harness artifacts kept intact
Out closing yes summarize the rule just blocked + cat the last few lines of .claude/audit/*.jsonl; FULL adds a table of 10 rules

Rules and capabilities

  • RULE-01 (MUST): Each scene: narrate → violate → quote the block reason verbatim → remediate. No abridged performance.
  • RULE-02 (MUST): Only touch files named tour-demo*; absolutely do not modify/delete the project's existing files.
  • RULE-03 (MUST): A scene that should-block but is NOT blocked → stop the tour, report "fence broken at rule X" — that is a real bug that needs handling (DOES NOT apply to the non-blocking group R4/R6/R8/R10 — they inherently do not block).
  • Capabilities: write/delete demo file tour-demo* in llmwiki/ + edit wiki/index.md (demo row); read audit artifacts; run local harness scripts. Depends on the host's hooks (PreToolUse/Stop) — without hooks, no performance.

Failure boundaries

  • Pre-check prints nothing → blocked (MISSING): STOP, suggest PoC install command + open a new session at root, no fake demo.
  • A scene that should be blocked but isn't → failed: stop the tour, report "guard broken at rule X" (except R4/R6/R8/R10).
  • Cleanup deletes files but leaves an index row → Stop hook R3 blocks that very cleanup turn → delete the row in the same turn and finish.

HOW

Main workflow

Step Type Inputs Action Outputs/exit Failure/next
W01 deterministic ARGUMENTS Choose tour length SHORT or FULL —
W02 deterministic root session Pre-check harness (command grep/test below) POC / PROD prints nothing → MISSING, stop
W03 effect map path → rule Perform the SHORT scenario (3 scenes) or FULL (R1–R10) sequentially, narrating BEFORE each scene stderr blocked verbatim + file PASS not blocked → stop, report guard broken
W04 effect file tour-demo* already PASS Cleanup in the same turn: delete file + index row clean project Stop R3 blocks → delete the remaining row
W05 deterministic audit Closing: summarize rules + cat .claude/audit/*.jsonl report —

Details of each step (source of truth for W01–W05):

Choose tour length (read ARGUMENTS)

  • Contains full / 10 / "full" → FULL (R1–R10).
  • Otherwise (default, or short/3) → SHORT (3 scenes: R1 · R2 · R3).

Pre-check — detect which harness is live in the session

Run at the ROOT of the session:

{ grep -q llmwiki-validate .claude/settings.json 2>/dev/null && test -f harness/poc-vendor-neutral/bin/llmwiki-validate.py && echo POC; }
test -f llmwiki/.claude/hooks/pre_tool_use.py && echo PROD
  • Prints POC → PRIMARY way. PoC now has enough R1–R10 (5 rules block at PreToolUse + R3/R4/R8/R10 via event hooks).
  • Prints PROD → production harness. Same R1–R10, same short/full scenario below; only differs in hook script name (pre_tool_use.py/post_tool_use.py/stop.py) and R1 may block via permissions.deny. Perform identically.
  • Prints NOTHING → MISSING. STOP (don't fake a demo), suggest installation (PoC is the primary way): 
    curl -fsSL https://raw.githubusercontent.com/Rheinmir/setup/orca/harness/poc-vendor-neutral/bootstrap.sh | bash
    then open a NEW Claude session at that project root (so .claude/settings.json is loaded) → retype /harness-tour.

Hook only blocks when the session load .claude/settings.json in the correct root project where it was installed. Open a session in the source repo / another root → no hook → the tour does NOT happen (happening without being blocked = staging, forbidden).

The Harness is a HOOK, NOT MCP (explain if the user asks "why isn't it in /mcp")

The blocking layer is PreToolUse hook → verify with /hooks or the block hooks in .claude/settings.json, NOT in /mcp. MCP is something else (a tool for the model to call). PoC: the hook calls llmwiki-validate claude-hook. Production: pre_tool_use.py…

Table mapping path → rule (USE THE CORRECT path so each scene triggers only 1 rule)

Multiple rules watch the same wiki/ → choosing the wrong path means 2 rules block together, messy. Standard map (per policy.yaml):

Rule File to violate Why ONLY that rule triggers
R1 no-write-raw llmwiki/raw/tour-demo.md deny_write glob raw/**
R2 origin-required llmwiki/wiki/sources/draft/tour-demo.md = ---\ntype: draft\n---\n# Tour Demo (HAS frontmatter, NO ## Origin, NO ## Plan) frontmatter → R9 passes; missing Origin → R2 blocks
R3 index-sync (Stop) the very file R2 just created, NOT yet in wiki/index.md m_stop scans concepts/entities/sources/draft against the index
R5 folder-structure llmwiki/wiki/tour-demo.md (directly under wiki/, odd basename) forbid_root: wiki/*.md not in the allow-list
R7 proposal-complete llmwiki/wiki/sources/draft/tour-demo-proposal.md — HAS frontmatter + ## Origin + ## Plan + proposed, MISSING ## Agent Task Assignment & Sequence diagram frontmatter→R9 passes, Origin→R2 passes → R7 is the one blocking
R9 okf-frontmatter llmwiki/wiki/concepts/tour-demo.md — HAS ## Origin but does NOT have YAML frontmatter has Origin so R2 passes; missing type: → R9 blocks

⚠️ R9 now covers BOTH sources/ + draft/ (matching global production — drafts also need frontmatter). Since R2/R9 watch every content dir → isolate by CONTENT, not by path: a file that HAS frontmatter but MISSING Origin → only R2; a file that HAS Origin but MISSING frontmatter → only R9.

Non-blocking-live (only for VIEWING artifacts, state clearly "does not block"): R4 (.claude/audit/audit.jsonl + log.md), R8 (line [harness] N rule… at SessionStart), R10 (.claude/audit/.docs-gate.json counts prompts). Repo layer: R6 (pre-commit + CI .github/workflows/harness.yml + skill /verify-before-commit).

SHORT SCRIPT (default) — 3 scenes, performed SEQUENTIALLY, narrate BEFORE each scene

Opening: "I will deliberately violate 3 rules. Watch the hook block me in real time — it's not me backing off."

Scene 1 — R1 (PreToolUse · deny writing to raw/):

  • Narration: "raw/ is the human's inbox, the agent must not write."
  • Write llmwiki/raw/tour-demo.md → exit 2 → BLOCK. Quote verbatim stderr [R1 no-write-raw] …. Do NOT retry. The file is never created.

Scene 2 — R2 (PreToolUse · enforce ## Origin):

  • Narration: "Create the source file but 'forget' ## Origin → origin not traceable."
  • Write llmwiki/wiki/sources/draft/tour-demo.md = ---\ntype: draft\n---\n# Tour Demo (HAS frontmatter so R9 passes, but NO ## Origin) → exit 2 → BLOCK. Quote [R2 origin-required] ….
  • Fix: Write that same file again, add ## Origin\n- harness-tour demo (keep the frontmatter) → this time PASS, the file is written.

Scene 3 — R3 (Stop · index lies):

  • Narration: "The tour-demo file was just created but I deliberately did NOT write it into wiki/index.md. Now try to end the turn."
  • End the response here without updating the index → Stop hook (harness-events.py stop) returns exit 2, brings back the drift list.
  • When blocked: quote verbatim [R3 index-sync] wiki/index.md chưa liệt kê: …, then add exactly 1 row for tour-demo to wiki/index.md, move on to cleanup.

→ next Cleanup (below).

FULL SCRIPT (when ARGUMENTS has full) — R1–R10 in sequence

Perform 6 scenes live-block (R1,R2,R3,R5,R7,R9) + 3 non-blocking ones to show artifacts (R4,R8,R10) + 1 repo (R6). State clearly before the non-blocking group: "The following 3 rules do NOT block while typing — they WRITE/REMIND; I will just show the artifacts as proof."

R1 — Write llmwiki/raw/tour-demo.md → BLOCK (deny raw/). Quote, do not retry.

R2 — Write llmwiki/wiki/sources/draft/tour-demo.md (HAS frontmatter ---\ntype: draft\n--- so R9 passes, but missing ## Origin) → BLOCK → add ## Origin → PASS.

R3 — (Stop) end the turn while the R2 file is not yet in index.md → Stop block → add index row → continue. (In FULL, for continuity you can merge R3 into the end: after creating the PASS files below, try to end a turn with the index still drifting → block → update the index.)

R4 — log-append (does NOT block): after a PASS Write, just .claude/audit/audit.jsonl has a new line (tool+path+timestamp) and .claude/audit/log.md is regenerated. cat the last few lines as proof — "the machine writes, I can't delete."

R5 — Write llmwiki/wiki/tour-demo.md (right under wiki/) → BLOCK [R5 folder-structure] … (wiki files must be in concepts/entities/…). Do not retry.

R6 — verify-before-commit (repo, does NOT block live): point out the commit gate consisting of pre-commit hook + CI .github/workflows/harness.yml (calls the core files mode on changed .md) + skill /verify-before-commit (Claude self-checks new code before committing). This is the repo layer, not the session — no direct block demo.

R7 — Write llmwiki/wiki/sources/draft/tour-demo-proposal.md has frontmatter + ## Origin + ## Plan + proposed but is MISSING ## Agent Task Assignment & Sequence diagram → BLOCK [R7 proposal-complete] … → fix those 2 more items → PASS.

R8 — pattern-health (does NOT block): run python3 harness/poc-vendor-neutral/bin/harness-events.py session to show the line [harness] N rule đang gác… (+ drift warning if policy diverges from remote). This is what gets printed at SessionStart.

R9 — Write llmwiki/wiki/concepts/tour-demo.md has ## Origin but does NOT have YAML frontmatter → BLOCKS [R9 okf-frontmatter] … (missing type) → add ---\ntype: concept\n--- at the top → PASS.

R10 — docs-gate (does NOT block): cat .claude/audit/.docs-gate.json shows the prompt counter; explains that every LLMWIKI_DOCS_GATE_EVERY (default 5) turns, UserPromptSubmit inserts a directive suggesting to call /docs-site-macos.

FULL wrap-up: a table of 10 rules — which one blocks-live, which one records/reminds, which one stays in the repo; cite .claude/audit/*.jsonl as evidence.

Cleanup (MANDATORY — same turn, otherwise R3 blocks)

Delete EVERY file tour-demo* that PASS (was created) and delete the corresponding row in wiki/index.md in the same turn (deleting the file but leaving the row → index mismatch → Stop hook blocks the cleanup turn itself):

  • SHORT: wiki/sources/draft/tour-demo.md + its row.
  • FULL: add wiki/sources/draft/tour-demo-proposal.md, wiki/concepts/tour-demo.md + the rows.
  • The files that were BLOCKED (raw/tour-demo, wiki/tour-demo R5) do not exist → no need to delete.
  • Harness artifact (audit.jsonl, log.md, .docs-gate.json) is a valid machine record → keep as is, do not delete.
  • ABSOLUTELY do not touch other existing files of the project.

Finale: summarize in 1 paragraph the rules that just blocked + "every tool call just now is in .claude/audit/*.jsonl — machine-recorded, I cannot forget it" (cat a few last lines). The machine-performance version needs no session: bash harness/poc-vendor-neutral/demo.sh (13 assertions) · test-broad.sh (63).

Branches

ID Kind Guard Behavior Skip / failure Rejoin
B01 user_optional ARGUMENTS has full / 10 / "full" FULL SCENARIO R1–R10 instead of SHORT default → SHORT W04
B02 conditional_required pre-check in PROD performs exactly the same, only the hook script name differs (pre_tool_use.py/post_tool_use.py/stop.py), R1 can block via permissions.deny — W03
B03 recovery pre-check prints nothing (MISSING) STOP, suggest the curl command bootstrap.sh PoC (Pre-check block) then open a NEW session at the project root does not perform ends
B04 user_optional user asks "why can't I see /mcp" explain that the harness is a HOOK (checks /hooks), not MCP — W03

Validation and stopping

Each live-blocking scenario counts only when the real hook returns exit 2 and stderr is extracted verbatim; no retry after blocking (except for the defined remediation step). The tour stops immediately when a live-blocking rule does not block. Done when there are no more tour-demo* and wiki/index.md matches.

Examples

  • Positive: /harness-tour on a project with PoC installed → pre-check in POC → Write llmwiki/raw/tour-demo.md is blocked [R1 no-write-raw] … → R2 blocks then PASS after adding ## Origin → end of turn is stopped [R3 index-sync] … → add row → clean up wiki/sources/draft/tour-demo.md + row in the same turn.
  • Boundary/failure: /harness-tour full opens in the source repo (does not load the project's .claude/settings.json ) → pre-check prints nothing → MISSING, stops, suggests the curl bootstrap command + opens a new session; no scenario is performed.