SuperAPI AI Directory

qa

Testing & quality

View repository

Updated 10-16 06:21

About this skill

QA — Establish the Test Cage

A test cage is the suite + conventions that hold a codebase's behavior so you can change code without fear. This skill builds or strengthens that cage for this repo. Inspired by Uncle Bob: tests are first-class; the cage is what makes production code keepable.

Different job from Matt Pocock's skills. Those move an idea → ship (wayfinder / grill / to-spec / implement / tdd). This skill is codebase upkeep: put a trustworthy cage in place (or repair it). Once the cage exists, Matt's tdd writes the next feature inside it.

Leading words

  • Test cage — the automated harness that constrains behavior (unit lane, acceptance lane, plumbing lane, CI gates, doubles policy)
  • Semantic stability — meaningful production changes turn the suite red
  • F.I.R.S.T. — Fast, Isolated, Repeatable, Self-validating, Timely
  • Kind / ownership — who owns the expectation (programmer / business / architect) → kinds.md
  • CRAP — complexity × missing coverage; where the cage is weakest

Process — put the cage up

Copy and track:

Cage progress:
- [ ] 1. Survey
- [ ] 2. Policy
- [ ] 3. Lanes & seams
- [ ] 4. Doubles policy
- [ ] 5. Trust gates
- [ ] 6. Hotspot closure
- [ ] 7. Record the cage

1. Survey

Map what exists today (commands, folders, CI jobs, approximate runtime, flakes, coverage tooling, mutation tooling). Note gaps: no fast programmer lane, acceptance missing, mocks everywhere, CI only runs e2e, etc.

Read CONTEXT.md / ADRs if present so domain language matches the cage.

2. Policy

Agree with the user (short, written) what "done" means for this repo:

  • Coverage goal (asymptotic 100% line/branch — mutation.md)
  • Whether mutation and/or CRAP run locally, in CI, or on-demand
  • Which kinds are in-scope now vs later (kinds.md)
  • F.I.R.S.T. non-negotiables for the fast lane (first.md)

3. Lanes & seams

Define the lanes of the cage (not every codebase needs all on day one):

Lane Owner Cadence
Programmer / unit Devs Every save / every PR — must be Fast
Acceptance / customer Business / QA PR or main — business rules
Integration / system Architects PR or nightly — plumbing

Name the seams each lane tests at. Prefer few, high seams (codebase-design vocabulary when that skill is present). Confirm seams with the user before adding a pile of tests.

4. Doubles policy

Lock where doubles are allowed: architectural / system boundaries only. Details: doubles.md. Prefer hand-rolled fakes at those edges.

5. Trust gates

Coverage alone is not the cage. Wire what the repo will actually run:

  1. Fast suite on every PR
  2. Coverage report (honest map of execution)
  3. Mutation and/or CRAP on the modules that matter (mutation.md)

Mutants must die. High-CRAP functions are the holes in the cage — characterize before refactoring.

6. Hotspot closure

From CRAP / mutation survivors / known flakes: add characterization tests, delete or quarantine rotten tests, split slow tests out of the fast lane. If the real blocker is "no seam to cage against," hand off to improve-codebase-architecture / codebase-design — then return here.

7. Record the cage

Leave durable breadcrumbs the next agent can follow, e.g.:

  • docs/testing.md or a Testing section in CONTEXT.md — lanes, commands, doubles policy, gates
  • Scripts / CI config that encode the gates
  • Optional ADR if a contested decision was made (e.g. "no UI in the fast lane")

When not to use this skill

You want… Use instead
Idea → tickets → feature Matt: wayfinder / grill-with-docs → to-spec → implement → tdd
One red→green behavior slice tdd (inside an existing cage)
Conversational bug filing triage (Matt's old qa session skill is deprecated)
One stubborn bug diagnosing-bugs

Completion criteria

  • Survey of current suite/CI captured
  • Written policy (kinds in scope, F.I.R.S.T. for fast lane, trust gates)
  • Lanes + seams named and confirmed
  • Doubles policy explicit
  • At least one trust gate runnable (fast suite; mutation/CRAP if in policy)
  • Cage recorded in-repo for the next session

Reference