SuperAPI AI Directory

deterministic-test-harness

Testing & quality

View repository

Updated 10-16 08:02

About this skill

Deterministic Test Harness

Control clocks, randomness, providers, storage, and concurrency for repeatable tests; inject deterministic dependencies at the boundary.

Sources of Nondeterminism

  • Time, time zones, calendars, date cutoffs, settlement windows
  • Random ids, UUIDs, references, sampling, shuffling
  • External providers: payments, market data, bank files, email, SMS, KYC
  • Background jobs, queues, scheduled tasks, retries, polling
  • Database ordering without explicit ORDER BY
  • Parallel writes, locks, optimistic concurrency, race-prone state

Test Controls

Prefer existing patterns, else smallest boundary: frozen-time clock, deterministic id factory, provider fake (success/decline/timeout/duplicate), in-memory queue/sync runner, fixture reset, stable sort. Freeze time, seed ids, drive providers from fakes, assert observable behavior.

Prove Stability

Run repeatedly with the repo's runner:

npm test -- <target>
pytest <target> -q
go test ./... -count=20

Guardrails

  • Do not add sleeps to fix timing, or hit real payment/market-data providers from tests.
  • Do not assert on unordered collections without sorting, use wall-clock dates in expected values, or over-mock internal collaborators.