Debug

Systematic Debug

Structured, hypothesis-driven debugging. No guessing. No "let me just try this." Every action has a reason, every observation narrows the search space.

When To Use

• A test fails and the reason isn't obvious
• A fix introduced a regression
• Behavior doesn't match expectations
• A trailing verification agent found something wrong
• "It works locally but not in CI/prod"
• Something that was working stopped working

Philosophy

The goal is not to fix the bug. The goal is to understand the bug. The fix follows naturally.

Rules:

1. Never change code to "see if this helps" — change code because you know WHY it will help
2. Read before guessing. Most bugs are obvious once you read the right 20 lines of code
3. Shrink the search space with each step. If a step doesn't eliminate possibilities, it was wasted
4. Reproduce first. If you can't trigger it, you can't verify your fix

---

Phase 1: Stabilize (What exactly is broken?)

Before investigating, nail down the observable symptoms.

Collect:

• Error output — exact error message, stack trace, log lines (copy verbatim, don't paraphrase)
• Expected behavior — what SHOULD happen?
• Actual behavior — what DOES happen?
• Reproduction — can you trigger it reliably? What's the minimal reproduction?
• Recency — when did this last work? What changed since then?

If recency is known: git log --oneline from the last known-good state. The diff between then and now is your search space.

If recency is unknown: Skip ahead, but flag this — you'll need to bisect later.

Output: A one-paragraph problem statement. Example:

"The storychief-push-draft task returns 403 after the auth middleware refactor in commit abc123. It worked on commit def456. The error is Forbidden: invalid API scope. Expected: draft pushes successfully."

---

Phase 2: Narrow (Where is the bug?)

Systematically reduce the search space. Pick the strategy that fits:

Strategy A: Recent Change (you know what changed)

1. Read the diff (git diff <last-good>..<current> or specific files)
2. For each changed file, ask: "Could this change cause the observed symptom?"
3. Rank changes by likelihood (closest to the error path first)
4. Read the top 2-3 suspect files in full (not just the diff — context matters)

Strategy B: Stack Trace (you have one)

1. Start at the TOP of the stack trace (the throw/error site)
2. Read that file at that line. Understand what triggered the error
3. Walk UP the call chain — what passed bad data? What state was wrong?
4. The bug is usually 1-3 frames up from where the error surfaces

Strategy C: Bisect (no stack trace, no obvious change)

1. Find a known-good state (commit, config, input)
2. Find the known-bad state (current)
3. Test the midpoint
4. Repeat until you've isolated the breaking change
5. For git: git log --oneline <good>..<bad> then manually test the midpoint commit

Strategy D: Input/Output Trace (behavior is wrong, no error)

1. Identify the entry point (API call, function call, event trigger)
2. Trace the data flow through each transformation step
3. At each step: "Is the input correct? Is the output correct?"
4. The bug is at the first step where output diverges from expected

Output: A specific location (file:line) or narrow area (2-3 files) where the bug lives.

---

Phase 3: Hypothesize & Test (Why is it broken?)

Now you have a location. Form a hypothesis and test it — without changing production code yet.

Hypothesis format:

"The bug is caused by [specific mechanism] in [file:line]. This happens because [chain of causation]. If I'm right, then [testable prediction]."

Testing a hypothesis (pick one):

• Read test: Re-read the code path with your hypothesis in mind. Does every step confirm it?
• Log test: Add a temporary log/print at the suspect location. Run the reproduction. Does the output confirm your hypothesis?
• Minimal reproduction: Write a small test that isolates just the suspect behavior. Does it fail the way you predict?
• Counter-test: If your hypothesis is right, what ELSE should be broken? Check those cases.

If the hypothesis is CONFIRMED: Move to Phase 4.

If the hypothesis is REJECTED:

• What did you learn? What possibilities were eliminated?
• Form the next hypothesis based on the narrowed search space
• Do NOT loop more than 3 hypotheses without stepping back to Phase 2 to re-narrow

---

Phase 4: Fix & Verify (Make it right, prove it's right)

Fix:

1. Make the minimal change that addresses the root cause (not the symptom)
2. If the fix is more than ~20 lines, pause — you may be fixing the wrong thing or over-engineering
3. Do NOT fix adjacent issues you noticed. One fix per debug cycle. File other issues separately.

Verify:

1. Run the original reproduction — does it pass now?
2. Run the full test suite — did the fix break anything else?
3. If the bug was a regression, check the specific area that originally triggered it
4. Explain the fix in one sentence. If you can't, you may not fully understand it yet.

Fix statement format:

"The bug was [root cause] in [file:line]. It was caused by [mechanism]. Fixed by [what you changed]. Verified by [what you ran]."

---

Sub-Agent Mode

When this methodology is used by a sub-agent (e.g., a trailing verification agent in Ralph Review or audit):

The sub-agent MUST:

• Include the full problem statement (Phase 1 output) in its report
• Include the specific location (Phase 2 output)
• Include the confirmed hypothesis (Phase 3 output)
• Include the fix statement (Phase 4 output)
• Never say "I fixed it" without the fix statement

The parent agent MUST:

• Not accept "fixed" without a fix statement
• Verify the fix independently (run tests, read the diff)
• If the sub-agent looped 3+ hypotheses without resolution, escalate to the user

---

Anti-Patterns (Banned)

Pattern	Why It's Banned	Do This Instead
"Let me try changing X and see if it works"	Guess-and-check doesn't build understanding	Form a hypothesis first, test it without code changes
Changing multiple things at once	Can't tell which change fixed it (or broke something else)	One change per cycle
"It works now, not sure why"	You'll be back here next week	Keep investigating until you can write the fix statement
Searching the whole codebase for the error string	Unfocused; wastes time	Use Phase 2 strategies to narrow first
"Probably a race condition / caching issue"	These are cop-out hypotheses	Be specific: which race? Which cache? What state?
Deleting and rewriting the broken code	Hides the bug instead of understanding it	Understand first, then decide if rewrite is warranted