SuperAPI AI Directory
Partner中文AI newsFavorites
Skillstc-run

tc-run

General & other

View repository

Updated 10-16 01:12

About this skill

Skill: tc-run — from a full test case file to a report for each batch

Extracted from the Payroll SIT session 15/09/2026 (335 TC, 7 sheets). Deterministic tool: harness/scripts/tc-xlsx.py (extract · classify · plan · report). Batch runtime: /orca-graph. Browser: claude-in-chrome on the user's already-logged-in session (SSO). Report: MCP tester-kit → tao_report_test.

WHAT

Purpose and context

  • Purpose: turns ONE full test case file .xlsx into PASS/FAIL/BLOCK results with a screenshot for each TC and a tester-kit .xlsx report for each batch.
  • Trigger (when to use): The user provides a test case file .xlsx (SIT/UAT/regression) and wants to "run it", "break it down", "export a report".
  • Non-goals: NOT used to write new test cases (that is /uat-nonit-testcase), not used for unit testing code.

Mental model

xlsx → tcs.json (extract) → nhãn ui|calc|perf|security|gap (classify) → PLAN theo lô → orca-graph (t1 hitl đăng nhập → các lô song song → report) → results/<lô>.json + shots/<id>.png → report-<lô>.xlsx → summary.md.

Input and output contract

Field Required? Meaning
In test case file .xlsx yes multiple sheets, columns Test Case ID/Description/Steps/Expected/Priority/Type
In app URL + login session yes the user logs in themselves in the claude-in-chrome tab (HITL)
In calc input data no persona/pay period prepared by the user; if missing then TC calc does not run
Out tcs.json + classification table by sheet yes number ui/calc/perf/security/gap
Out PLAN.md + graph yes batch ≤ 8 TC, graph ≤ 20 nodes
Out results/<lô>.json + shots/<id>.png yes PASS/FAIL/BLOCK + actual + evidence per TC
Out report-<lô>.xlsx + summary.md yes tester-kit report for each batch; aggregated by batch

Rules and capabilities

  • RULE-01 (MUST): Each TC result must have a real screenshot + observable actual; without a screenshot it must not be marked PASS.
  • RULE-02 (MUST): calc runs only when input data (persona/pay period) prepared by the user is available; it does not fabricate numbers.
  • RULE-03 (MUST): security and perf state "outside automation scope" clearly in the summary, not silently omit it.
  • RULE-04 (MUST): Batches ≤ 8 TC so that screenshots and context do not balloon; graph ≤ 20 nodes (orca-graph limit) → a UI file > ~150 TC is split into multiple PLANs (parent graph → child).
  • RULE-05 (MUST): Report follows the tester-kit schema: priority High/Medium/Low (P0,P1→High, P2→Medium, P3→Low), description includes STEPS/EXPECTED/ACTUAL/STATUS.
  • Capabilities: read xlsx + deterministic classification; control the browser on the user's logged-in session; write result files/local images; call the report export service.

Failure boundaries

  • Encounter login/SSO page → blocked wait for the user to log in in that same tab; do not enter passwords, do not click SSO on the user's behalf.
  • Steps requiring Save/Delete/Update on real data → that TC is BLOCK "needs a separate SIT environment", unless the user explicitly allows it.
  • TC calc missing input data, gap steps empty → do not run, list in summary (partial).
  • Cannot capture a screenshot → TC must not be PASS.

HOW

Main workflow

Step Type Inputs Action Outputs/exit Failure/next
W01 deterministic file .xlsx Extract + classify (tc-xlsx.py extract then classify), report the table by sheet tcs.json + table heuristic wrong → fix label auto, state clearly which TC
W02 deterministic tcs.json Generate PLAN by batch then build graph (orca-graph.py build --strict) PLAN + graph build error → fix PLAN then repeat
W03 effect App URL HITL login session, write app.json → set t1 done_user_reported already logged in see login/SSO → stop and ask the user
W04 effect batch Run each batch: lock → dispatched → do → set done, capture screenshot, compare Expected results/<lô>.json write real data → BLOCK that TC
W05 effect results Report each batch via tao_report_test + summary.md report-<lô>.xlsx, summary —
W06 deterministic graph Track progress on the cockpit, audit model responses progress —

Details of each step (source of truth for W01–W06):

  1. Extract + classify (0 token): 
    python3 harness/scripts/tc-xlsx.py extract <file.xlsx> -o scratchpad/<tên>/tcs.json
python3 harness/scripts/tc-xlsx.py classify scratchpad/<tên>/tcs.json
    Report to the user the table by sheet: how many ui (runnable via browser), calc (need input data, controls), perf, security (SSO/login, cannot be safely automated), gap (empty steps/waiting on business). If the heuristic is wrong, fix label auto in tcs.json and state clearly which TC was fixed.
  2. Generate PLAN by batch then build graph: 
    python3 harness/scripts/tc-xlsx.py plan tcs.json --app "<tên app (URL)>" --batch 8 --only ui -o llmwiki/wiki/sources/draft/DDMMYY-<tên>-PLAN.md
python3 harness/scripts/orca-graph.py build <PLAN> --strict && orca-graph.py ask <id> parallel
    Task 1 is always hitl (app survey + login session); batches run in parallel; the final task exports the report. Each batch is a node with verify = results file with enough TCs.
  3. Login session — HITL, must not exceed: open the URL in the claude-in-chrome tab; if you see the login/SSO page → stop, ask the user to click login in that same tab. Do not enter the password, do not click the SSO button on the user's behalf. After login → record app.json (logged_in: true, actual menu) → set t1 done_user_reported.
  4. Run each batch (lock → dispatched → do → set done): each TC according to Test Steps using find/computer/read_page, take a screenshot shots/<id>.png, compare Expected → PASS|FAIL|BLOCK + actual + evidence. Stop before any Save/Delete/Update on real data → BLOCK "needs a separate SIT environment", unless the user explicitly allows it. Record results/<lô>.json.
  5. Report per batch: tc-xlsx.py report results/<lô>.json --tcs tcs.json --url <URL> → payload → call MCP tao_report_test → copy the xlsx to report-<lô>.xlsx. Finally summary.md: PASS/FAIL/BLOCK per batch, TCs needing a human, remaining calc/perf/security TCs.
  6. Track progress on the cockpit (control-room.html); audit model answers like orca-graph.

Branches

ID Kind Guard Behavior Skip / failure Rejoin
B01 recovery heuristic classify assigns the wrong label fix the label auto in tcs.json, state clearly which TCs were fixed — W02
B02 conditional_required file > ~150 TC ui (exceeds 20 nodes) split into multiple PLANs (parent graph → child) — W03
B03 user_optional user explicitly allows writing real data run all Save/Delete/Update steps not allowed → BLOCK "needs a separate SIT environment" W04
B04 conditional_required has TC calc and the user has prepared the persona/pay period run the calc TCs, check the numbers missing data → do not run, record a summary W05

Validation and stopping

Each batch is a node with verify = results file with enough TCs; PASS only when there is a real screenshot + actual. Stop when every batch node is done and each batch has a report; remaining TCs (calc/perf/security/gap, BLOCK) are listed in summary.md.

Examples

  • Positive: SIT Payroll file 335 TCs, 7 sheets → classify into the part ui → plan --batch 8 --only ui → user logs in via SSO in the tab → each batch of 8 TCs has shots/<id>.png + results/<lô>.json → report-<lô>.xlsx + summary.md PASS/FAIL/BLOCK per batch.
  • Boundary/failure: TC requires clicking "Save" on the payroll table with real data, the user has not allowed it → that TC is BLOCK "needs a separate SIT environment", do not click; TC cannot take a screenshot → not PASS.

Recap

/tc-run = xlsx → tc-xlsx.py extract/classify → PLAN per batch → orca-graph → run on the logged-in session → results JSON + screenshots → tao_report_test each batch → summary. Unexpected use case: run classify on the team's test case file to immediately see how many % are gaps/waiting for business before someone promises a completion date.