SuperAPI AI Directory

gsdl-verify-gates

Testing & quality

View repository

Updated 10-16 07:24

About this skill

Verify Against Deterministic Gates

Runs whatever deterministic checks the project actually has, and produces a gate report — the evidence that Step 8 (Open PR) puts in front of a reviewer and Step 9 (Close Loop) records. This is Step 7: Pick up source → Brainstorm → SPEC.md → Slices → Review → Branch → Execute → Verify → PR + bot → Close loop

Why this is a standalone skill and not a checkbox inside gsdl-execute: most repos don't yet have CI that enforces this automatically. Until CI/local parity and unbypassable ratchets exist, this skill is the manual crutch that makes "verify" a deliberate, evidenced step rather than something that quietly gets skipped.

Tier

MEDIUM. The gates are deterministic by definition — discover the project's real checks, run them, capture the output verbatim. There's no judgment to spend a LARGE model on. When the gsdl orchestrator runs Step 7 it spawns a MEDIUM subagent for it.

Running as a subagent means non-interactive: don't ask the user anything, and don't try to fix failing gates. Write the report, then return the failures with their verbatim output to the caller. Triage and remediation happen in the orchestrator context, which can loop back into Step 6.

Prerequisites

  1. Project exists: .planning/[project-name]/
  2. Implementation complete: all parent slices in .planning/[project-name]/tasks.md are [x] (run gsdl-execute first if not)
  3. SPEC.md exists: read its "Verification Plan" section (Step 3) for what the user already said should be checked

Step 1 — Discover the Gates

Look for deterministic checks in this order, and use all that are discoverable — don't stop at the first one found:

  1. SPEC.md's Verification Plan — the user may have already named specific commands or manual steps
  2. package.json scripts — test, lint, typecheck/type-check, build, format:check
  3. Other package manifests — Makefile targets (test, lint, build), pyproject.toml/tox.ini (pytest, ruff, mypy), Cargo.toml (cargo test, cargo clippy), go.mod (go test ./..., go vet), *.csproj/*.sln (dotnet test, dotnet build), Gemfile (rspec, rubocop), pom.xml/build.gradle (mvn verify, gradle check)
  4. CI config — .github/workflows/*.yml, .gitlab-ci.yml, .circleci/config.yml, azure-pipelines.yml. Mirror what CI actually runs; if CI runs something not exposed as a local script, note the CI-only gate and try to run its underlying command directly
  5. Pre-commit / lint config — .pre-commit-config.yaml, .eslintrc*, .golangci.yml, ruff.toml

If you find no discoverable gates at all: this is itself a finding, not a silent pass. Say so explicitly — it's a verification-contract gap — and ask what to check manually (e.g., "run the app and click through X", "check the staging deploy").

Step 2 — Run Each Gate

Run each discovered command. Capture:

  • The exact command run
  • Pass/fail
  • A trimmed excerpt of the output (full output for failures; just the summary line for passes)

Run gates in a reasonable order: install/build first if needed, then lint/typecheck (fast, cheap), then tests (slower), matching whatever the repo's CI does if that's discoverable.

Do not skip a gate because it's slow. If a gate is genuinely too expensive to run in full (e.g., a full e2e suite), say so and ask the user whether to run a subset or skip with their explicit sign-off — never skip silently.

Step 3 — Produce the Gate Report

Write to .planning/[project-name]/verify-report.md:

# [Project Name] — Verify Report

> Generated by gsdl-verify-gates on [YYYY-MM-DD]

## Gates Run

| Gate | Command | Result | Notes |
|------|---------|--------|-------|
| Lint | `npm run lint` | ✅ Pass | |
| Typecheck | `npm run typecheck` | ✅ Pass | |
| Tests | `npm test` | ❌ Fail | 2 failures in `auth.test.ts` — see below |
| Build | `npm run build` | ✅ Pass | |

## Failures

### `auth.test.ts`
```
[trimmed failure output]
```

## Manual Checks (if no automated gate covers them)

- [ ] [Manual check description] — performed by: [user/agent], result: [pass/fail]

## Gaps Identified

- [Any check SPEC.md's Verification Plan called for but no gate exists to run automatically — flag it, don't just note and move on]

Omit the "Manual Checks" or "Gaps Identified" sections if empty.

Step 4 — Gate on the Result

This is a ratchet, even if a manual one: do not report Step 7 as complete while any gate is failing.

  • All gates pass: tell the user verification passed, show the report path, and say the project is ready for Step 8 (gsdl-open-pr) — or Step 9 directly if no forge is configured.
  • Any gate fails: stop here. Report the failure(s) clearly, and do not proceed to PR creation or close-out — a PR is never opened on a red branch. Remediation is the caller's call, not yours (see Tier above) — return the failures and let it choose between:
    • Fixing the issue and re-running this skill, or
    • Going back to gsdl-execute if the fix is substantial enough to need its own sub-task tracking

Never soften a failing gate into "mostly passing" — the report is evidence a human or reviewer may later rely on.

Rules

  1. Discover gates from the repo, don't assume a stack — a Python repo has no package.json; check what actually exists
  2. No silent gaps — if something can't be verified automatically, say so in the report, don't omit it
  3. Never mark Step 7 done with a failing gate — this is the one hard stop in the whole pipeline
  4. Re-run from disk — if tasks.md or SPEC.md changed since this skill last ran, re-read them before re-verifying