diagnosing-bugs

Diagnosing Bugs

A discipline for hard bugs. Skip phases only when you explicitly state the reason.

Before starting: load project context

Use the loaded CONTEXT terminology and RULEs. If not yet loaded, follow the project knowledge protocol; if knowledge is unavailable, continue and state so honestly. Build a mental model of the relevant modules using project terminology, and keep subsequent reproduction, hypotheses, instrumentation, and fixes consistently compliant with the loaded rules.

Phase 1 — Establish a feedback loop

The core is a compact pass/fail signal:This bug turns red when it appears. Bisection, hypothesis testing, and instrumentation all consume only this signal; without it, reading code cannot locate the root cause. Invest disproportionate effort here.

Methods for building a feedback loop — try roughly in this order

1. Failing test— at any seam that can reach the bug: unit, integration, end-to-end.
2. Curl / HTTP script— send requests to a running dev server.
3. CLI invocation— with fixed input, diff stdout against a known-correct snapshot.
4. Headless browser script(Playwright / Puppeteer) — drive the UI and assert on DOM/console/network.
5. Replay a captured trace. Save real network requests/payloads/event logs to disk; replay through the code path in an isolated environment.
6. One-off test harness. Start the minimal subset of the system (one service, mocked dependencies) and execute the buggy code path with a single function call.
7. Property/fuzz loop. If the bug is "sometimes outputs wrong", run 1000 random inputs to find the failure pattern.
8. Bisection tooling. If the bug appears between two known states (commit, dataset, version), automate "start at state X, check, repeat" so that git bisect run。
9. Differential loop. Run the same input through the old and new versions (or two configurations) separately and diff the output.
10. HITL bash script. Last resort. If a human must click, use scripts/hitl-loop.template.sh to drive _them_, so the loop remains structured. The captured output is fed back to you.

Establish the right feedback loop, and the bug is already 90% solved.

Iterate on the loop itself

Treat the loop as a product. Once you have a loop, ask:

• Can it be faster? (Cache setup, skip irrelevant initialization, narrow the test scope.)
• Can the signal be clearer? (Assert the specific symptom, not "it didn't crash".)
• Can it be more deterministic? (Fix time, set random seeds, isolate the filesystem, freeze the network.)

A 30-second flaky loop is almost as bad as no loop. Aim for a second-level, deterministic loop.

Non-deterministic bugs

The goal is not a clean reproduction, but ahigher reproduction rate. Trigger the loop 100 times, parallelize, add stress, narrow the time window, inject sleeps. A 50% reproduction rate is debuggable; 1% is not — keep increasing it until it is debuggable.

When you truly cannot establish a loop

Stop and state it explicitly. List what you tried. Ask the user for: (a) access to an environment that can reproduce it, (b) captured artifacts (HAR file, log dump, core dump, timestamped screen recording), or (c) permission to add temporary production instrumentation. Do not keep assuming without a loop.

Completion criteria — a tight loop that can turn red

Phase 1 is complete only when the feedback loop is bothtightandable to turn red: point to a command that has actually been run at least once — a script path, test command, or curl — and show the invocation and output. It must simultaneously satisfy:

• Able to turn red— it executes the real bug code path and asserts the exact symptom the user described, so it can turn red when this bug exists and turn green after the fix. Merely "no error at runtime" does not count.
• Deterministic— each run yields the same verdict; for nondeterministic bugs, fix a sufficiently high reproduction rate as described above.
• Fast— measured in seconds, not minutes.
• Agent-runnable— can run unattended; human action is allowed only through scripts/hitl-loop.template.sh introduction.

Before establishing this command, limited reading of project documentation, test configuration, entry files, and related call paths is allowed to locate how to run it, the reproduction entry point, and testable seams. At this point, the goal of reading can only be to establish the feedback loop, and you must not use it to generate or lock in a root-cause theory; if you start reading broadly and guessing at causes, immediately return to the feedback loop. Without a command that can turn red, you must not enter Phase 2.

Phase 2 — Reproduce + minimize

Run the loop. Observe the bug appear.

Confirm:

• the loop produces theuser-described failure mode — not another failure that happens to be nearby. Wrong bug = wrong fix.
• The failure is reproducible across multiple runs (or, for nondeterministic bugs, the reproduction rate is high enough to support debugging).
• The exact symptom has been captured (error message, incorrect output, slow response time) so that later phases can verify the fix actually resolves it.

Minimize

After the loop turns red, reduce the reproduction to theminimal scenariothat still turns red. Remove one input, caller, configuration, piece of data, or operation step at a time, then rerun the loop; keep only the elements indispensable for triggering the failure.

The minimal reproduction narrows the hypothesis space for Phase 3 and becomes the seed for future regression tests. Minimization is complete when every remaining element is indispensable — removing any one of them turns the loop green. Do not continue before reproduction and minimization are complete.

Phase 3 — Hypotheses

Before testing any hypothesis, generate 3 to 5 ranked hypotheses. Single-hypothesis generation anchors on the first plausible idea.

Each hypothesis must befalsifiable: state the prediction it makes.

Format: "If is the cause, then <changing Y> will make the bug disappear / <changing Z> will make it worse."

A hypothesis that cannot state a prediction is just intuition — discard it or sharpen it.

Show the ranked list to the user before testing. They usually have domain knowledge to immediately reorder, or know which hypotheses have already been ruled out. Don't block waiting—if the user isn't there, continue with your ordering.

Phase 4 — Instrumentation

Every probe must map to a specific prediction from Phase 3.Change only one variable at a time.

Tool preferences:

1. Debugger / REPL inspection—if the environment supports it. One breakpoint beats ten logs.
2. Targeted logging—at the boundaries that distinguish hypotheses.
3. Targeted recording, not logging everything for later retrieval.

Tag every debug log, using a unique prefix such as [DEBUG-a4f2], so that during the fix phase a single grep cleans them up.

Performance branch. For performance regressions, logging is usually the wrong tool. Instead: establish baseline measurements (timing tools,performance.now(), profiler, query plan), then bisect. Measure first, fix later.

Authorization boundary

Once Phase 4 is complete, the diagnostic conclusion can already be delivered: reproduction method, root cause evidence, impact scope, and minimal fix recommendation.

Continue to Phase 5 when the user explicitly asks to "fix," "resolve," or implement a change. When only diagnosis is requested, remove temporary instrumentation and one-off artifacts, then stop. When intent is unclear, ask for confirmation before any corrective write.

Phase 5 — Fix + regression test

Write the regression testbeforethe fix, but only if acorrect seamexists: the test can exercise the real bug pattern at the call site. A seam that is too shallow, for example a bug that requires multiple callers but only has a single-caller test, or a unit test that cannot reproduce the call chain that triggers the bug, only produces false confidence.

If no correct seam exists, that itself is a finding. Record that the codebase architecture is preventing the bug from being locked down, and explain it in Phase 6.

If a correct seam exists:

1. Turn the minimal reproduction into a failing test at that seam.
2. Observe the test fail.
3. Apply the fix.
4. Observe the test pass.
5. Verify the fix complies with the loaded RULE; when a deviation from the rule is needed, state the conflict explicitly and leave the decision to the user.
6. Re-run the Phase 1 feedback loop against the original, unminimized scenario.

Phase 6 — Cleanup

Before declaring completion, confirm:

• the original reproduction no longer occurs;
• the regression test passes, or the absence of a correct seam has been recorded;
• all [DEBUG-...] instrumentation has been removed;
• one-off prototypes have been deleted, or moved to a clearly marked debug location;
• the final correct hypothesis has been written into the commit or PR message, so the next debugger understands it.

Finally, explain what could have prevented this bug. If the answer involves architectural changes, such as missing test seams, tangled callers, or hidden coupling, record specific follow-up improvement suggestions after the fix is complete.