SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsdouble-check

double-check

Coding & development

View repository

Updated 10-15 21:41

About this skill

double-check

Purpose

Standalone second-pass PR review, stage-partitioned into an ICM procedure. Fetch the PR + the first review + the diff and check it out (setup), then run ONE cohesive critical-judgement review in a clean context (verify the first review's claims, find missed edge cases, and check tests/docs together → a single consolidated verdict), then apply fixes and re-run tests if needed (fix), then post the curated comment, apply the double-checked label, and write the report (post). Behaviorally equivalent to the original double-check skill, just isolated so the judgement step is not polluted by the orchestrator's history.

Arguments

Param Required Default Notes
pr yes — PR number, e.g. 742
repo yes — org/repo, e.g. fellowship-dev/booster-pack

Parse from $ARGUMENTS: first token is pr, second is repo. GitHub auth is ambient — no token env var. The pod's git-credential-pylot helper and the gh shim mint short-lived App installation tokens per operation, so git URLs must stay plain (https://github.com/<org>/<repo>.git); inline credentials bypass the helper and expire mid-run.

What it does

4-stage SEQUENTIAL ICM procedure (no parallel stages):

Stage Mode Description
01-setup subagent Fetch PR metadata including current HEAD, classify the incoming first-review receipt (its Head reviewed line) as current/stale/absent, capture comments + full diff, and checkout PR branch + merge base
02-review subagent ONE cohesive critical review in clean context: verify first review's claims, find missed edge cases, check tests/docs → consolidated verdict (ready iff zero open MUST-FIX code items) + curated findings
03-fix subagent Apply MUST-FIX (and worthwhile NICE-TO-HAVE) fixes, re-run tests, push — only if fixes are needed
04-post inline Re-fetch the live head, promote when it equals the exact 40-hex head stage 02 reviewed or carries the same patch-id, otherwise perform one restart; then post curated review comment and apply labels only for the matching head

Patch-id carry and delta review (pylot#3738)

Every verdict receipt records the head and the PR's patch-id (git diff base...head | git patch-id --verbatim). A head change alone never costs a review:

  • Same patch-id (rebase, merge-from-base): the verdict and double-checked carry to the new head with a one-line "verdict carried: patch-id unchanged" note. Setup reports review_scope: carry, stages 02 and 03 are skipped, and stage 04 runs its carry branch.
  • New commits (rework, stage-03 fixes): stage 02 reviews only the files whose hunks changed since the last verdict, plus the findings they are meant to fix (review_scope: delta). Over 30% of the PR's files changed, or no reachable prior verdict: full review.
  • A changed diff on a conflict resolution has a new patch-id, so it is always re-reviewed.

Handoff locations

All handoffs live in the repo working directory:

.procedure-output/double-check/{stage}/handoff.md

The setup stage records the local checkout dir (REPO_DIR) in its handoff so the fix stage operates on the same working tree. Each subagent stage receives only the handoffs it needs — never the full orchestrator context.

Execution

Resolve DC_SKILL_DIR to the absolute directory containing the loaded SKILL.md, using its supplied resource path before changing the working directory. Use that same root for stage instructions, templates and helpers; do not infer it from the reviewed repository's cwd or prefer another installed version. Pass it explicitly to each stage and export it in that stage's shell. If the loaded resource path is unavailable, stop and resolve it before running stages.

Exact-head cycles (sequential subagents)

Run one Task per stage, one after another. Do NOT launch any stages in parallel. Do not start the next stage until the current one completes. Start with restart_count=0. If stage 04 sees a different full remote SHA with a different patch-id, it stops without posting a verdict or touching labels, and the orchestrator runs a complete conditional 01 → 02 → 03 → 04 cycle at the observed SHA with restart_count=1, passing delta_base_head (the head the previous cycle reviewed) to stage 01 and first moving the previous stage-02 handoff to .procedure-output/double-check/prior-review.md, so the new cycle reviews only the delta and the findings it fixes. Stage 02 still receives only its new setup handoff and artifacts: never orchestration history. A second transition, or any unreadable live head, stops blocked. A moved head with the same patch-id is not a transition: stage 04 carries the verdict.

Each Task prompt must be self-contained:

  • Include only the stage's input handoff paths
  • Include the path to the stage's CONTEXT.md
  • Pass pr and repo values
  • Do NOT pass orchestrator history or prior reasoning

Task prompt template:

You are running stage {NN}-{name} of the double-check procedure.

PR: {pr}    REPO: {repo}
DC_SKILL_DIR: {absolute skill directory}
Export DC_SKILL_DIR to this absolute path before executing shell snippets.

Read your stage instructions:
  {DC_SKILL_DIR}/stages/{NN}-{name}/CONTEXT.md

Your inputs:
  {list only the input handoff paths from that stage's CONTEXT.md}

Write your output to:
  .procedure-output/double-check/{NN}-{name}/handoff.md

Execute all steps in CONTEXT.md. Write handoff.md before exiting.

Stage gating:

  • Stage 02 is the isolated critical-judgement step. Its prompt MUST carry only the setup handoff (PR + first review + diff) — nothing else. This is the clean-context window the whole proc exists for.
  • After stage 01, read its handoff. If review_scope: carry, SKIP stages 02 and 03 and run stage 04's carry branch (the PR's own diff is unchanged since a ready verdict).
  • After stage 02, read its handoff. If fixes_needed: false, SKIP stage 03 (no fixes to apply) and go straight to stage 04. Otherwise run stage 03.

Stage 04 (inline)

Run stage 04 yourself in the orchestrator context — do NOT spawn a Task. Read CONTEXT.md:

{DC_SKILL_DIR}/stages/04-post/CONTEXT.md

Run the live exact-head gate (gh pr view), then only for a matching head post the comment and apply the label. Verify labels/comment actually landed, write the report file, and emit the [pylot] outcome=... marker from the orchestrator (never from a subagent). If stage 04 exits 3, set RESTART_COUNT=1 and run a new complete conditional Stage 01 → 02 → 03 → 04 cycle at the current live head; do not pass the old Stage 02 or Stage 03 handoff. If it exits 2, it is terminal blocked: do not run a promotion path.

Stage handoff chain

01-setup ─► 02-review ─► 03-fix ─► 04-post (inline, reads 01+02+03)
                  │                    ▲
                  └── fixes_needed:false ┘  (skip 03)

Exit paths

  • First-check fail closed (at least one open MUST-FIX code item): stage 04 posts a <!-- pylot:first-check-fail-closed --> comment, removes/withholds double-checked, adds or retains needs-work, and creates no positive follow-on. It emits: [pylot:$PYLOT_OUTCOME_NONCE] outcome="double-check {repo}#{pr} — verdict {verdict}, double-checked withheld, needs-work retained" status=success
  • Re-check PASS (PR had needs-work, verdict=ready): stage 04 removes needs-work, re-toggles double-checked (remove + re-add), and emits: [pylot:$PYLOT_OUTCOME_NONCE] outcome="double-checked re-check PASS {repo}#{pr} — loop closed, cto-review re-fired" status=success
  • Re-check FAIL (PR had needs-work, at least one open MUST-FIX code item): stage 04 leaves needs-work in place, does NOT re-toggle double-checked, posts a structured verdict comment with a <!-- pylot:recheck-fail --> marker (idempotent — skipped if marker already present), and emits: [pylot:$PYLOT_OUTCOME_NONCE] outcome="double-checked re-check FAIL {repo}#{pr} — needs-work retained" status=success
  • First-check success: only an explicit ready verdict at the exact live 40-hex head may apply double-checked; it emits: [pylot:$PYLOT_OUTCOME_NONCE] outcome="double-checked {repo}#{pr} — verdict ready, {N} findings curated, {N} fixes pushed" status=success
  • Verdict carried (review_scope: carry): stage 04 posts the one-line carry note, leaves the labels, and emits: [pylot:$PYLOT_OUTCOME_NONCE] outcome="double-check {repo}#{pr} — verdict carried: patch-id unchanged, no re-review" status=success
  • Failure: failing stage emits [pylot:$PYLOT_OUTCOME_NONCE] outcome="double-check failed at stage NN: {reason}" status=failed
  • Blocked: setup cannot fetch/checkout the PR (e.g. merge conflict, missing PR), the live head cannot be read, or a second head transition occurs → [pylot:$PYLOT_OUTCOME_NONCE] outcome="double-check blocked: {reason}" status=blocked (a deliberate stop — blocked is its own terminal state, not a failure)

Rework Follow-up Mode

Whether — and how — a "produce the fix" follow-up dispatch exists at all is repo policy, not protocol: some orgs wire a CTO-rework automation on top of this skill, some don't. Read the repo playbook (GET /admin/playbooks/<org>/<repo>) for a rework-dispatch section; if it names a contract, it looks like this:

A gate rule dispatches back into this skill when a prior review left needs-work, with a task string identifying it as a PRODUCE follow-up (not a re-judge). The playbook names the exact trigger string/label to match and the automation's identity — treat those as resolved values below, not literal text to hardcode here.

When dispatched under that contract, do the following BEFORE running the stages above (the playbook is the source of truth for the trigger and rule name — keep this protocol in sync with it, don't let the two drift):

  1. Scan the PR's comments first. If a prior attempt already posted a ⚠️ NEEDS HUMAN marker, or a previous rework already tried and failed on this SAME gate, do NOT loop — leave it for a human and stop. Never re-check and re-apply needs-work without producing anything.
  2. Read the latest CTO review comment for its specific action items.
  3. Actually ADDRESS them. Code/test/doc fixes: make the changes and push. Requests for PR-body edits or staging evidence are not rework items: staging is a release-train step (pylot#3389), and the body never decides the verdict. If no code item remains, run the stages below; zero MUST-FIX items is a pass. If the ask genuinely exceeds this skill's scope (needs a specialized runner, a human decision, or credentials you lack): do NOT silently re-apply needs-work — post a comment beginning ⚠️ NEEDS HUMAN stating exactly what's blocking and what would unblock it, then STOP.
  4. Only after the items are actually addressed: gh pr edit <number> --repo <repo> --remove-label "double-checked,needs-work", then run the stages above so the label chain re-fires. Setup compares the reworked diff with the last verdict's patch-id, so the review covers only the rework delta plus the findings it addresses (full review when the delta exceeds 30% of the files).

No rework-dispatch section in the playbook? This mode does not apply — run the stages above as a normal review/fix/post pass.

Hard Rules

  1. SEQUENTIAL ONLY — one Task per stage, run one after another. NO parallel Task launches, ever.
  2. The review is ONE cohesive stage — do NOT split stage 02 into per-file or per-dimension subagents. Correctness, edge cases, tests, docs, deps, and security are judged together in a single verdict.
  3. Stage 02 gets a clean context — only the setup handoff and its artifact files (PR body, first review, changed files, diff). Never pass orchestrator history into it.
  4. Stage 04 runs inline — the [pylot] outcome=... marker MUST come from the orchestrator.
  5. Never pass full orchestrator context into subagent Task prompts — inputs only.
  6. Each stage writes handoff.md before the next stage reads it.
  7. Do not skip stages except stage 03 when fixes_needed: false (an explicit, allowed skip).
  8. NO Quest. Reporting is the local report file only — no Quest POST, no 127.0.0.1:4242, no quest.fellowship.dev, no QUEST_TOKEN.
  9. Apply labels only after the comment posts successfully (stage 04). On re-check PASS, remove needs-work BEFORE re-adding double-checked — this is the structural loop-break. On re-check FAIL, do NOT touch labels or re-toggle double-checked.
  10. Code decides the verdict; zero MUST-FIX is a pass. The verdict is ready exactly when no MUST-FIX code item is open, on a first check and on a re-check alike; stage 04 normalizes a needs-work with must_fix_open: 0 to ready. The PR body is intent only: there is no claims-reconciliation step, and a stale body is at most a one-line note. (Replaces the pylot#2649 claims gate: on a 30-day sweep most of its needs-work flags were stale bodies on clean diffs, each costing a rework.)
  11. Stage 04 verifies its own side effects — after labelling, gh pr view the PR and confirm the expected labels/comment are actually there. Reporting success on unverified side effects is the failure this skill exists to catch in others.
  12. Curate the first review, never re-derive it blind — setup captures the first review's comments verbatim and its Head reviewed receipt line; stage 02 curates those findings at tier-scaled depth (escalate-only). No first review found → full-depth fresh review.
  13. A stale first review is historical evidence, not a blocker or current coverage — compare its head_sha with the post-rebase PR HEAD. Continue the cohesive review against the complete current diff, re-check prior findings, and post a new current-head receipt. Never restart the whole pipeline merely because the incoming receipt is stale.
  14. Promotion binds to the final full SHA — immediately before every verdict comment or label mutation, fetch headRefOid. The 40-character live SHA must equal the Stage 02 reviewed_head_sha, or carry the same full patch-id as the reviewed diff (then the live head becomes the promoted head), fail closed. On the first patch-id change restart into a delta review; on a second or a failed retrieval stop blocked. A file list, diff stat, short SHA, or local HEAD never substitutes for SHA or patch-id equality. A stale or blocked run never mutates double-checked or triggers downstream automation.
  15. First-check promotion is explicitly positive only — apply double-checked only when the reviewer verdict is exactly ready and bound to the exact live head. Any negative, missing, malformed, stale, or conflicting signal fails closed: remove/withhold double-checked, add or retain needs-work, and do not create CTO, FlowChad, staging, or merge follow-ons.
  16. Success markers come from stage 04's templates ONLY (pylot#3392). The five documented status=success markers (four verdict branches plus the carry branch) in stage 04's "Emit outcome marker" section are the complete set. Never synthesize a free-form success marker from the orchestrator, and never emit ANY status=success unless stage 04 ran to completion — receipt comment posted and post-action verification passed. On 2026-09-05 an orchestrator emitted "queued-worker result ok" without running stage 04: the mission terminalized done with no receipt and the completed work was invisible for ~6h. If you cannot run stage 04, the outcome is status=failed or status=blocked with the reason — a fabricated success is the worst possible exit.
  17. Verbatim artifacts are written by shell, never retyped. Stage 01 redirects the PR body, first review, manifest and diff into files beside its handoff. Copying them through the Write tool was the largest setup cost and can silently drift from the source.
  18. Wait on processes, not clocks. A long command (tests, a hooked git push) runs in the foreground with the tool's longest timeout, or in the background with its PID recorded and a blocking wait that returns the moment it exits (stage 03's "Waiting on a long command"). Never poll with a fixed sleep N of 30 seconds or more: measured runs slept up to 10 minutes past completion.
  19. Verification follows the owning repo contract. Stage 03 selects meaningful checks for its fix delta and affected consumers, preserving required broad/full, dependency, transformation, integration and release gates. CI owns a gate only when current repo policy and configured workflows establish that. Avoid duplicate runs on the same revision only when policy allows reuse and scope/environment match; rerun after changes, failures or unresolved concerns. Hook bypass requires explicit repo permission and proof that all checks the bypass omits passed. Stage 02's independent review and stage 04's exact-head acceptance remain required; focused testing does not waive either.

Reference files

  • CONTEXT.md — architecture overview
  • stages/NN-name/CONTEXT.md — per-stage inputs, task, output contract
  • shared/review-comment-template.md — curated PR comment template (stage 04)
  • shared/report-template.md — local report file template (stage 04)