cawplan-internal-qa-coding-humaninputs-sample

CawPlan Internal QA — Human Input Sample & Label

Bootstrap

cawplan skill check

Task

Fetch human-input rows via the CawPlan CLI, emit a fixture JSON aligned with production classify context rules, then open the local label editor so a human can set expected_categories, expected_topic, optionally expected_reason, and review_required per turn.

Workflow

1. Resolve scope (same disambiguation rules as humaninputs-test)

Member (optional): cawplan session members → exact --member git key. If ambiguous, ask the user to pick — do not guess.

Product (optional): cawplan products list --search "<name>" → unique_id. If ambiguous, ask the user to pick.

User within product (optional): needs PRM user_id for product-human-input-logs --user-id, resolved via cawplan users query.

Date range (required): --from / --to, or --date for a single day. Default to last 2 days if the user gives no range.

Count: --limit N (default 20).

Session shape (pick one):

--one-session — auto-pick the session with the most rows in range (best for multi-turn prev chains, like the 20-turn Slack fixture).
--session-id <id> — keep one session explicitly.
Neither — first N rows after sort (prev still computed per session_id).

2. Build the fixture JSON

From this skill directory:

SKILL_DIR="/media/spx/work/github/flow-cawplan-skill/skills/cawplan-internal-qa-coding-humaninputs-sample"

node "$SKILL_DIR/scripts/build_sample.mjs" \
  --from <YYYY-MM-DD> --to <YYYY-MM-DD> \
  [--member "<exact_member>"] \
  [--product "<product name>" | --product-id <id>] \
  [--user-id <prm_user_id>] \
  [--session-id <session_id> | --one-session] \
  --limit <N> \
  --output "<path>/human-input-sample.json" \
  --copy-for-editor

Output shape (matches eval fixtures under /media/spx/work/2026/0830/):

Field	Meaning
`items[].content`	Raw human input
`items[].assistant_message`	Raw paired assistant reply (full text from API)
`items[].prev_assistant_message`	Processed last paragraph of the previous turn's assistant in the same session (production `prev` slot). Resolved from all rows fetched in the date range, not only the selected N — so a recent slice can still get correct prev even when earlier turns of that session are not in the output
`items[].expected_categories`	Empty `[]` until labeled
`items[].expected_topic`	Empty until labeled
`items[].expected_reason`	Optional free-text rationale for reviewers (empty until filled)
`items[].review_required`	`false` by default; set `true` when category/topic labels need review
`items[].cloud_category` / `cloud_topic`	Optional hints from cloud enrichment

Preprocessing mirrors uid.core-product ClassifyPrevAssistantTail / ClassifyAssistantSnippet (see scripts/classify_context.mjs).

3. Open the label editor

After --copy-for-editor, start a static server and open the page:

cd "$SKILL_DIR/assets"
python3 -m http.server 8765

Browser: http://localhost:8765/label-editor.html

Load latest sample — reads assets/samples/latest.json (written by --copy-for-editor).
Load JSON — pick any fixture file.
Download labeled JSON — exports *-labeled.json with expected_categories / expected_topic filled in; expected_reason / review_required included when set.
Draft auto-saves to browser localStorage.

Tell the user the output path, editor URL, and that they should export the labeled JSON when done (save under their eval folder, e.g. /media/spx/work/2026/0830/).

Output

Report to the user:

Resolved filters (member / product / dates / limit / session).
Path to the generated JSON and assets/samples/latest.json.
Label editor URL (http://localhost:8765/label-editor.html).
Reminder: export labeled JSON from the page when annotation is complete.

References

scripts/build_sample.mjs — fetch + preprocess
scripts/classify_context.mjs — production-aligned paragraph rules
assets/label-editor.html — web UI
references/CATEGORY_TAXONOMY.md — category label definitions
references/TOPIC_TAXONOMY.md — topic label definitions