SuperAPI AI Directory
Partner中文AI newsFavorites
Skillssystematic-debugging

systematic-debugging

Testing & quality

View repository

Updated 10-16 08:46

About this skill

Systematic Debugging

Overview

Random fixes waste time and create new bugs. Quick patches mask the underlying problem.

Core principle: Always find the root cause before attempting a fix. Symptom fixes fail.

**Violating the letter of this process is violating the spirit of debugging.**

Iron Law

NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST

If you have not completed Phase 1, you cannot propose a fix.

When to Use

Use for any technical problem:

  • Test failures
  • Bugs in production
  • Unexpected behavior
  • Performance issues
  • Build failures
  • Integration issues

Especially use this when:

  • Under time pressure (emergencies make it easy to guess)
  • "Just one quick fix" seems obvious
  • You have already tried multiple fixes
  • Previous fixes did not work
  • You do not fully understand the problem

Do not skip when:

  • The problem looks simple (simple bugs also have root causes)
  • You are in a hurry (haste guarantees rework)
  • The manager wants an immediate fix (systematic is faster than chaos)

Four Phases

You must complete each phase before moving to the next.

Phase 1: Root Cause Investigation

Before attempting any fix:

  1. Read error messages carefully

    • Do not skip past errors or warnings
    • They often contain the exact solution
    • Read the stack trace in full
    • Note down line numbers, file paths, error codes

  2. Reproduce consistently

    • Can you trigger it reliably?
    • What are the exact steps?
    • Does it happen every time?
    • If it is not reproducible → gather more data, do not guess

  3. Check recent changes

    • What change could have caused this?
    • Git diff, recent commits
    • New dependencies, configuration changes
    • Environment differences

  4. Gather evidence in multi-component systems

When the system has multiple components (CI → build → signing, API → service → database):

Before proposing a fix, add diagnostic tooling:

For EACH component boundary:
  - Log what data enters component
  - Log what data exits component
  - Verify environment/config propagation
  - Check state at each layer

Run once to gather evidence showing WHERE it breaks
THEN analyze evidence to identify failing component
THEN investigate that specific component

Example (multi-layer system):

# Layer 1: Workflow
echo "=== Secrets available in workflow: ==="
echo "IDENTITY: ${IDENTITY:+SET}${IDENTITY:-UNSET}"

# Layer 2: Build script
echo "=== Env vars in build script: ==="
env | grep IDENTITY || echo "IDENTITY not in environment"

# Layer 3: Signing script
echo "=== Keychain state: ==="
security list-keychains
security find-identity -v

# Layer 4: Actual signing
codesign --sign "$IDENTITY" --verbose=4 "$APP"

**This reveals:** which layer failed (secret → workflow ✓, workflow → build ✗)

  1. Trace the data flow

When the error goes deep into the call stack:

Seeroot-cause-tracing.mdin this directory for the complete backward tracing technique.

Quick version:

  • Where does the bad value come from?
  • What calls this value?
  • Keep tracing until you find the source
  • Fix from the root, not from the symptom

Phase 2: Pattern Analysis

Find the pattern before fixing:

  1. Look for working examples

    • Find similar working code in the same codebase
    • What works are similar to the broken work?

  2. Compare with reference materials

    • If implementing the pattern, read the reference implementation in full
    • Do not skim - read every line
    • Fully understand the pattern before applying it

  3. Find the differences

    • What is different between the working and broken versions?
    • List every difference, no matter how small
    • Do not think "that's not important"

  4. Understand the dependencies

    • What other components does this need?
    • What settings, configuration, environment?
    • What assumptions does it make?

Phase 3: Hypotheses and Testing

Scientific method:

  1. Form a single hypothesis

    • State clearly: "I think X is the root cause because Y"
    • Write it down
    • Be specific, not vague

  2. Minimal test

    • Make the smallest possible change to test the hypothesis
    • One variable at a time
    • Do not fix multiple problems at once

  3. Verify before continuing

    • Did it work? Yes → Phase 4
    • Didn't work? Form a new hypothesis
    • Do not add more fixes on top

  4. When you don't know

    • Say "I don't understand X"
    • Don't pretend to know
    • Ask for help
    • Research more

Phase 4: Implementation

Fix the root cause, not the symptoms:

  1. Create a failing test case

    • The simplest reproduction
    • Automate the test if possible
    • If there is no framework, a one-off test script
    • Must exist before the fix
    • Usesuperpowers:test-driven-developmentThe skill of writing a correct failing test

  2. Implement a single fix

    • Fix the identified root cause
    • Change one at a time
    • No "while I'm here" improvements
    • No bundled refactoring

  3. Verify the fix

    • Does the test pass now?
    • Are the other tests not broken?
    • Is the problem really solved?

  4. If the fix doesn't work

    • Stop
    • Count: how many fixes have you tried?
    • If < 3: return to Phase 1 and re-analyze with the new information
    • If ≥ 3: stop and question the architecture (step 5 below)
    • Do not attempt fix #4 without an architecture discussion

  5. If more than 3 fixes fail: architecture problem

Patterns indicating an architecture problem:

  • Each fix reveals new shared state/coupling/problems in different places
  • The fix requires a "large-scale refactor" to implement
  • Each fix creates new symptoms elsewhere

Stop and ask about the fundamentals:

  • Is this pattern fundamentally reasonable?
  • Are we "sticking with it purely out of inertia"?
  • Should we refactor the architecture or continue fixing symptoms?

Discuss with your human partner before trying more fixes

This is not a failed assumption—it is a wrong architecture.

Red flags - stop and follow the process

If you find yourself thinking:

  • "Quick fix now, investigate later"
  • "Try changing X to see if it works"
  • "Add multiple changes, run tests"
  • "Skip the test, I'll verify manually"
  • "It's probably X, let me fix this"
  • "I don't fully understand, but this might help"
  • "The pattern says X, but I'll adjust it differently"
  • "Here are the main issues: [list uninvestigated fixes]"
  • Propose a solution before tracing the data flow
  • "Try one more fix" (when 2+ have been tried)
  • Each fix reveals new problems in different places

**All of these mean: Stop. Return to phase one. **

If 3 or more fixes fail: Question the architecture (see phase 4.5)

Signals from your human partner indicate you are doing it wrong

Watch for these redirects:

  • "Did that happen?" - you assumed without verifying
  • "Would it tell us...?" - you should add evidence collection
  • "Stop guessing" - you are proposing fixes without understanding
  • "Ultrathink this" - question the fundamentals, not just the symptoms
  • "Are we stuck?" (frustration) - your approach is not working

When you see these: Stop. Return to phase one.

Common rationalizations

Sorry Reality
"The problem is simple, no process needed" Simple problems also have root causes. For simple errors, handling is fast.
"Emergency, no time to process" Systematic debugging is faster than guessing and checking repeatedly.
"Try it first, then investigate" The first fix sets the pattern. Do it right from the start.
"I will write tests after confirming the fix works" Untested fixes do not last. Test first to prove it.
"Doing multiple fixes at once saves time" Cannot isolate what works. Causes new bugs.
"The reference is too long, I'll adapt the pattern" Partial understanding will definitely lead to errors. Read it completely.
"I see the problem, let me fix it" Seeing symptoms ≠ understanding the root cause.
"Try one more fix" (after two or more failures) 3+ failures = architecture problem. Question the pattern, do not fix again.

Quick reference

Phase Main activity Success criteria
1. Root cause Read error, reproduce, check changes, gather evidence Understand what and why
2. Pattern Find working examples, compare Find the differences
3. Hypothesis Form a theory, minimal testing Confirmed or new hypothesis
4. Implementation Create test, fix, verify Error resolved, test passes

When the process shows "no root cause"

If the system investigation shows the problem is indeed environmental, time-related, or external:

  1. You have completed the process
  2. Record what you investigated
  3. Implement appropriate handling (retry, timeout, error message)
  4. Add monitoring/logging for future investigation

However: 95% of "no root cause" cases investigations are incomplete.

Supporting techniques

These techniques are part of systematic debugging and can be found in this catalog:

  • root-cause-tracing.md - Trace the error backward through the call stack to find the original trigger
  • defense-in-depth.md - Add multiple layers of validation after finding the root cause
  • condition-based-waiting.md - Replace arbitrary timeouts with conditional polling

Related skills:

  • superpowers:test-driven-development - Used to create failing test cases (Phase 4, Step 1)
  • Superpower: Verify before completion - Verify that the fix works before declaring success

Real-world impact

From a debugging session:

  • Systematic approach: 15-30 minute fix
  • Random fix approach: 2-3 hours of struggle
  • First-fix rate: 95% vs 40%
  • Introducing new bugs: near zero vs common