SuperAPI AI Directory

diagnosing-bugs

General & other

View repository

Updated 10-16 09:45

About this skill

Diagnose defects

Discipline for tricky bugs. Skip a phase only when you explicitly state the reason.

When exploring the codebase, first read GLOSSARY.md(if it exists), build a clear mental model of the relevant modules, and check the ADRs for the area you will touch.

Redaction

This skill will ask you to show commands, outputs, and captured artifacts.First redact every secret: replace it with <REDACTED> . Build loops against env vars so credentials stay in the environment and not in what you display. Captured artifacts with auth headers: quote only the lines that carry signal.

If the redacted output is not enough to diagnose the bug, explain the situation and ask the user.

Phase 1 - Build a feedback loop

This is the core of this skill. Everything else is mechanical steps. If you have a tight pass/fail signal for the bug, that is, it turns red on this bug, you can find the cause; bisection, hypothesis-testing, and instrumentation all just consume this signal. Without it, staring at the code for any amount of time will not save you.

Invest disproportionate effort here.Be tough, be creative, refuse to give up.

Ways to build the loop - try roughly in the following order

  1. Failing test, placed on a seam that can reach the bug: unit, integration, e2e all work.
  2. Curl / HTTP script, hitting the running dev server.
  3. CLI invocation, using fixture input, and diff stdout against a known-good snapshot.
  4. Headless browser script(Playwright / Puppeteer), drive the UI, and assert DOM/console/network.
  5. Replay a captured trace. Save the real network request / payload / event log to disk, and replay it into the code path in an isolated environment.
  6. Throwaway harness. Start the smallest subset of the system (one service, mocked deps), and trigger the bug code path with a single function call.
  7. Property / fuzz loop. If the bug is "sometimes wrong output", run 1000 random inputs and look for the failure mode.
  8. Bisection harness. If the bug appears between two known states (commit, dataset, version), automate "boot at state X, check, repeat", so you can git bisect run。
  9. Differential loop. run old-version vs new-version (or two configs) with the same input, then diff outputs.
  10. HITL bash script. Last resort. If a human must click, use scripts/hitl-loop.template.sh to drive the human, so the loop still stays structured. The captured output is fed back to you.

Build the right feedback loop and the bug is 90% fixed.

Tighten the loop

Treat the loop as a product. As long as you have one loop, keep tighten it:

  • Can I make it faster? (Cache setup, skip irrelevant init, narrow the test scope.)
  • Can I make the signal sharper? (Assert the specific symptom, not "didn't crash".)
  • Can I make it more deterministic? (Pin time, seed RNG, isolate filesystem, freeze network.)

A 30-second flaky loop is almost no better than no loop at all; a 2-second deterministic loop is a tight debugging superpower.

Non-deterministic bugs

The goal is not a clean repro, but higher reproduction rate. Loop triggering 100x, parallelise, add stress, narrow timing windows, inject sleeps. A 50%-flake bug can be debugged; a 1% one cannot. Keep raising the reproduction rate until it is debuggable.

When you truly cannot build a loop

Stop and state it clearly. List what you tried. Ask the user for: (a) access to an environment that reproduces it, (b) a redacted captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation.Do not keep hypothesising without a loop.

Completion conditions - a tight loop that can turn red

Phase 1 completion conditions: loop tight and red-capableYou can point to one command(script path, test invocation, curl), and you have already run it at least once (show the invocation and output, redacted), and it satisfies:

  • Red-capable - it drives the real bug code path and asserts the user's exact symptom, so it can turn red on that bug and green after the fix. Not "runs without erroring", but it must be able to catch this specific bug。
  • Deterministic - the same verdict on every run (flaky bugs: pin to a high reproduction rate as above).
  • Fast - seconds, not minutes.
  • Agent-runnable - you can run it unattended; human in the loop only via scripts/hitl-loop.template.sh。

If you find yourself reading code and building theories before a command exists,stop; jumping straight to hypothesis is exactly the failure this skill is meant to prevent. No red-capable command, no Phase 2.

Phase 2 - Reproduce + minimise

Run the loop. Watch it turn red, i.e. the bug appears.

Confirm:

  • The failure mode the loop produces is the user's described one, not some other nearby failure. Wrong bug = wrong fix.
  • The failure reproduces across multiple runs (or for non-deterministic bugs, the reproduction rate is high enough to debug with).
  • You have captured the exact symptom (error message, wrong output, slow timing), and later phases can verify that the fix truly resolves it.

Minimise

Once it turns red, shrink the repro to the smallest scenario that still turns red. Cut inputs, callers, config, data and steps one by one, re-running the loop after each cut; keep only the load-bearing parts of the failure.

Reason: a minimal repro narrows the hypothesis space in Phase 3 (fewer moving parts to suspect) and becomes a clean regression test in Phase 5.

Completion conditions:every remaining element is load-bearing, removing any one of them turns the loop green.

Do not continue before you have reproduced and minimised.

Phase 3 - Form hypotheses

Before testing any hypothesis, generate 3-5 ranked hypotheses. A single hypothesis anchors you to the first plausible idea.

Each hypothesis must falsifiable: state what prediction it makes.

Format: "If is the cause, then will make the bug disappear / will make it worse."

If you cannot state the prediction, it is vibe; discard or refine it.

Show the ranked list to the user before testing. Users often have domain knowledge and can immediately reorder ("we just deployed a change to #3"), or know which hypotheses have been ruled out. Cheap checkpoint, big time saver. Do not block on it; if the user is AFK, continue with your ordering.

Phase 4 - Instrumentation

Every probe must map to a specific prediction from Phase 3.Change only one variable at a time.

Tool preference:

  1. Debugger / REPL inspection, if the environment supports it. One breakpoint beats ten logs.
  2. Targeted logs, placed at boundaries that can distinguish hypotheses.
  3. Never "log everything and grep".

Give every debug log a unique prefix, for example [DEBUG-a4f2]. Then cleanup can be done with a single grep. Untagged logs survive; tagged logs get deleted.

Perf branch。 For performance regressions, logs are usually wrong. Instead, first establish a baseline measurement (timing harness,performance.now(), profiler, query plan), then bisect. Measure first, then fix.

Phase 5 - Fix + regression test

Write the regression test before the fix, but only if there is a correct seam。

The correct seam is where the test can trigger real bug pattern in the way it actually happens at the call site. If the only available seam is too shallow (the bug requires multiple callers, but the test has only a single-caller; the unit test cannot replicate the chain that triggers the bug), the regression test there will give false confidence.

If no correct seam exists, that itself is a finding. Record it. The codebase architecture prevents you from pinning down the bug. Flag it for the next phase.

If a correct seam exists:

  1. Turn the minimised repro into a failing test at that seam.
  2. Watch it fail.
  3. Apply the fix.
  4. Watch it pass.
  5. Run the Phase 1 feedback loop again against the original (unminimised) scenario.

Phase 6 - Cleanup

Before declaring done, you must:

  • The original repro no longer reproduces (rerun the Phase 1 loop)
  • The regression test passes (or the missing seam is recorded)
  • All [DEBUG-...] instrumentation has been removed (grep the prefix)
  • Throwaway prototypes have been deleted (or moved to a clearly marked debug location)
  • The correct hypothesis has been written into the commit / PR message, so the next debugger can learn from it