SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsdiagnosing-bugs

diagnosing-bugs

General & other

View repository

Updated 10-15 21:08

About this skill

Diagnose Bug

A discipline for tricky bugs: a phase may be skipped only when an explicit justification is stated.

While exploring the codebase, read GLOSSARY.md(if present) to get a clear mental model of the relevant modules, and check the ADRs for the areas you touch.

Redaction

This skill will have you show commands, outputs, and captured artifacts.Redact all secrets first: replace with <REDACTED> . Build loops over environment variables so credentials stay in the environment rather than appearing in what you show. Captured artifacts may carry auth headers: quote only the few lines that carry the signal.

If the redacted output is not enough to diagnose the bug, say so explicitly, and ask the user for more material.

Phase 1: Build a feedback loop

This is the skill itself. Everything else is just mechanical action. If you have atightpass/fail signal for this bug (a signal that turns red for _this_ bug), you will find the root cause; bisection, hypothesis testing, and instrumentation are all just consumers of this signal. If you do not have this signal, staring at the code forever will not save you.

Invest a disproportionate amount of effort in this step.Be aggressive. Be creative. Never give up.

Several ways to build a feedback loop (roughly in this order)

  1. Failing test: write at any seam that can reach the bug — unit, integration, e2e.
  2. Curl / HTTP script: against a running dev server.
  3. CLI invocation: use fixed input, diff stdout against a known-good snapshot.
  4. Headless browser script(Playwright / Puppeteer): drive the UI and assert on DOM/console/network.
  5. Replay a captured trace. Write the real network requests / payloads / event logs to disk, and replay them through the code path in isolation.
  6. Throwaway harness. Spin up a minimal subset of the system (one service, mocked dependencies) so a single function call triggers the buggy code path.
  7. Property / fuzz loop. If the bug is "sometimes the output is wrong", run 1000 random inputs and observe the failure mode.
  8. Bisection harness. If the bug appears between two known states (commit, dataset, version), automate "start at state X, check, repeat" so that git bisect run。
  9. Differential loop. Run the same input through both the old and new versions (or two configurations) and compare the outputs.
  10. HITL bash script. Last resort. If a human must click, use scripts/hitl-loop.template.sh to drive _them_, so the loop is still structured. The captured output is then fed back to you.

Get the feedback loop right and the bug is already 90% solved.

Tighten the loop

Treat the loop as a product. Once you have _one_ loop,tightenit:

  • Can it be made faster? (Cache initialization, skip irrelevant init, narrow the test scope.)
  • Can you make the signal sharper? (Assert on the specific symptom, not "it didn't crash.")
  • Can you make it more deterministic? (Fix time, seed the RNG, isolate the filesystem, freeze the network.)

A 30-second flaky loop is only slightly better than no loop; a 2-second, deterministic loop is truly tight—it's a debugging superpower.

Non-deterministic bugs

The goal is not a clean reproduction, buta higher reproduction rate. Run the trigger condition in a loop for 100 rounds, parallelize, add stress, tighten the time window, inject sleeps. A flaky bug with a 50% reproduction rate is debuggable; 1% is not, so keep raising the reproduction rate until it is debuggable.

When you really can't build a loop

Stop, and say so explicitly. List everything you tried. Ask the user for: (a) access to an environment that can reproduce the bug, (b) a sanitized capture artifact (HAR file, log dump, core dump, timestamped screen recording), or (c) authorization to add temporary instrumentation in production.Do notresort to armchair theorizing without a loop.

Completion criterion: a tight, red-capable loop

The sign that Phase 1 is complete is a loop that istightandred-capable: you can point toa single command(script path, a single test invocation, a curl), andyou have actually run it at least once(give the invocation and the sanitized output), and it satisfies:

  • Red-capable: drives the real bug code path and asserts onthe exact symptom the user described—so it can go red for this bug and go green after the fix. Not "no errors"; it must _catch this specific bug_.
  • Deterministic: gets the same verdict on every run (for flaky bugs: pinned to a high reproduction rate as described above).
  • Fast: seconds, not minutes.
  • Agent-runnable: you can run it unattended; only put a human in the loop when passing through scripts/hitl-loop.template.sh .

If you have already started reading code and building theories before writing this command,stop: skipping hypotheses and acting directly is exactly the failure mode this skill is meant to prevent. Without a red-capable command, there is no Phase 2.

Phase 2: Reproduce + minimize

Run the loop. Watch it go red as the bug appears.

Confirm:

  • The loop is producing theuser'sdescribed failure mode, not some other failure that happens to be nearby. Finding the wrong bug = fixing the wrong bug.
  • The failure is reproducible across multiple runs (or for a non-deterministic bug, has a high enough reproduction rate to debug based on it).
  • You have captured the exact symptom (error message, wrong output, slow timing) so that later phases can verify the fix actually addresses it.

Minimize

Once it goes red, shrink the reproduction case tothe smallest scenario that still goes red. Trim inputs, callers, configuration, data, and steps one by one, re-running the loop after each trim, keeping only the parts that bear the weight of the failure.

Why do it: a minimized reproduction compresses the hypothesis space of Phase 3 (fewer moving parts remain to be suspected), and in Phase 5 it becomes a clean regression test.

Done whenevery remaining element bears weight: removing any one of them makes the loop go green.

Do not advance to the next phase before both reproducingandminimizing are complete.

Phase 3: List hypotheses

Before testing any hypothesis, generate3–5 ranked hypotheses. Generating only a single hypothesis anchors on the first plausible idea.

Each hypothesis must befalsifiable: explicitly state the prediction it makes.

Format: "If is the cause, then <changing Y> will make the bug disappear / <changing Z> will make it worse."

If you cannot state the prediction clearly, then it is just a vibe: discard it or refine it.

Show the ranked list to the user before testing. They often have domain knowledge that can instantly reorder it ("we just deployed a change to #3"), or know hypotheses they have already ruled out. This is a cheap checkpoint but it saves a lot of time. Do not block on the user; if the user is AFK, proceed with your own ranking.

Phase 4: Instrumentation

Every probe must correspond to a specific prediction from Phase 3.Change only one variable at a time.

Tool preferences:

  1. Debugger / REPL inspection: use it if the environment supports it. One breakpoint beats ten logs.
  2. Targeted logging: place it at boundaries that can distinguish hypotheses.
  3. Never "log everything and grep".

Give every debug log a unique prefix, for example [DEBUG-a4f2]. Final cleanup then becomes a single grep. Logs without the prefix survive; logs with it die.

Performance branch. For performance regressions, logs are usually unsuitable. Instead: first establish baseline measurements (timing harness,performance.now(), profiler, query plan), then bisect. Measure first, then fix.

Phase 5: Fix + regression test

Write the regression testbefore the fix, but only if acorrect seam。

A correct seam is a location where the test can, at the call site, triggera true reproduction of the bug pattern. If the only available seam is too shallow (single-caller test, but the bug requires multiple callers; a unit test cannot reproduce the entire chain that triggers the bug), a regression test there will only provide false confidence.

If no correct seam exists, that is itself a finding. Record it. The codebase architecture is preventing the bug from being pinned down. Flag this for the next phase.

If a correct seam exists:

  1. Turn the minimized reproduction into a failing test at that seam.
  2. Watch it fail. If you force a red light by modifying the code or fixture, first do diffwith a clean copy, confirm the modification actually took effect, and only then trust the test result.
  3. Implement the fix.
  4. Watch it pass.
  5. Rerun the Phase 1 feedback loop to verify the original (non-minimized) scenario.

Phase 6: Cleanup

Things that must be done before declaring completion:

  • The original reproduction no longer reproduces (rerun the Phase 1 loop)
  • Regression tests pass (or the missing seam has been documented)
  • All [DEBUG-...] instrumentation has been removed (grep that prefix)
  • One-off prototypes have been deleted (or moved to a location explicitly marked as debug)
  • The assumptions that actually held have been written into the commit / PR message, so the next debugger can learn from them