Debugging playbook

Debugging Playbook

Purpose

Debugging is a search problem, not a guessing game. This skill enforces a disciplined loop — reproduce, isolate, hypothesize, change one variable, verify the root cause — so you fix the actual cause instead of a symptom, and know you fixed it. It also tells you how to tell code bugs from environment and dependency bugs, and when the framework itself is the thing to remove.

When to Use / When NOT to Use

Use when:

• Anything is broken, failing, crashing, hanging, flaky, or producing wrong output.
• A test fails and you don't yet know why.
• Something "worked yesterday" and doesn't now.
• You are tempted to "just try a fix" — stop and use this instead.

Do NOT use (load the complementary skill instead):

• You already know the root cause and are about to make the change → change-control.
• The bug is "no test caught this" and you need a test strategy → validation-and-testing.
• The failure is a dependency version conflict / install break → dependency-management (return here if the diagnosis is unclear).
• The failure is in the build/release/publish pipeline → build-and-release.
• You're reconstructing the history of a long-standing/recurring failure → failure-archaeology.
• The bug is unreliable LLM output (refusals, cutoffs, malformed JSON) → llm-integration-reliability (this skill's loop still applies; that skill has the specifics).

Core Methodology

Ground rules (universal, any stack): change ONE variable at a time; a fix belongs at the root cause, not the symptom; when a framework abstraction hides the bug, drop to primitives; reproduce in the smallest environment that exhibits the failure before theorizing about larger ones.

The loop

1. Reproduce. Get a deterministic repro before touching anything. Exact command, exact input, exact environment. If it's flaky, find the conditions that make it reliable (order, timing, data). No repro → no fix; you'd only be guessing.
2. Read the error message in FULL. Whole stack trace, top to bottom AND bottom to top. The real cause is often mid-trace, not the last line. Read the actual message text, the file, the line, the type. Most "mysterious" bugs are stated plainly in an error nobody finished reading.
3. Isolate by binary search. Halve the search space each step (see tree). Comment out half, git bisect across history, or strip the repro to the minimal input that still fails.
4. Hypothesize. State one specific, falsifiable guess: "the value is null because X returns None when Y." Vague ("something's off with the parser") is not a hypothesis.
5. Test ONE variable. Change exactly one thing that would confirm or kill the hypothesis. Two changes at once and you learn nothing from the result.
6. Verify the ROOT cause. When it works, confirm you fixed the cause, not a coincidence: revert the fix → bug returns; re-apply → bug gone. If reverting doesn't bring the bug back, you fixed the wrong thing.
7. Then hand off to change-control to apply the smallest correct diff at that root cause and leave a check behind (validation-and-testing).

Decision tree: isolation strategy

Reproduced the failure. Isolate it:
├─ Did it work at some past commit?
│   └─ YES → git bisect between good and bad commit.
├─ Is it a big input / big file / big function?
│   └─ YES → minimize: cut the input in half repeatedly until
│           the smallest failing case remains.
├─ Many code paths could cause it?
│   └─ YES → comment out / short-circuit half; re-run; recurse
│           into the failing half.
└─ Same code fails here but works elsewhere?
    └─ Suspect ENVIRONMENT/DEPENDENCY, not code → next tree.

Decision tree: environment vs code vs dependency

Where does the bug live?
├─ Fails on your machine but not another (or vice versa)?
│   └─ ENVIRONMENT: versions, env vars, OS, paths, locale, network.
│      Diff the two environments before touching code.
├─ Started failing with no code change on your part?
│   └─ DEPENDENCY: a transitive upgrade moved under you.
│      Check lockfile / installed versions vs constraints.
│      (hand off to dependency-management)
├─ Fails deterministically from your input, everywhere?
│   └─ CODE: it's yours. Isolate with binary search.
└─ Fails only THROUGH a framework, not when you call the
   underlying primitive directly?
    └─ The ABSTRACTION is the bug. Drop to primitives:
       call the stdlib/DOM/SQL directly, confirm it works,
       then the framework layer is where the fix or removal goes.

Debugging checklist

• I have a deterministic reproduction (exact command + input).
• I read the ENTIRE error message and stack trace, not just the last line.
• I stated one falsifiable hypothesis before changing anything.
• I changed exactly one variable per experiment.
• I classified the failure as code / environment / dependency.
• I confirmed the root cause by reverting the fix and seeing the bug return.
• The fix goes at the cause (shared root), not at the symptom's call site.

Discovery Commands

# --- Reproduce & read the failure ---
pytest path/to/test.py::test_name -x -vv       # single test, verbose, stop on fail, full assert diff
pytest --tb=long -x                            # full tracebacks
npm test -- -t "test name"                     # Node: run one test by name (jest/vitest; the node --test runner uses --test-name-pattern instead)
<the-exact-failing-command> 2>&1 | tee /tmp/repro.log   # capture full output incl. stderr

# --- Binary-search isolation across history ---
git bisect start
git bisect bad                                  # current commit is broken
git bisect good <known-good-hash>               # last commit known to work
# git checks out midpoints; test each, then:
git bisect good      # or: git bisect bad
git bisect reset                                # when done

git log --oneline -20                           # find candidate good/bad commits
git diff <good>..<bad> -- path/to/suspect       # what changed between working and broken

# --- Environment vs dependency triage ---
python --version && which python                # Python interpreter identity
pip list | grep -i <package>                    # installed version vs constraint
node --version && npm ls <package>              # Node version + resolved dependency
env | grep -i <VAR>                             # env var actually set?
git stash push -m repro-check && <run> ; git stash pop   # does your uncommitted change cause it? (needs a dirty tree — if push reports "No local changes", do NOT pop: you'd apply an unrelated stash)

# --- Drop to primitives (framework-is-the-bug) ---
python -c "import json; print(json.loads(open('x.json').read()))"  # test the primitive directly
# In a REPL/console, call the underlying stdlib/DOM/SQL without the framework wrapper.

# --- Verify root cause ---
git stash push -m verify-fix   # revert your fix → expect bug RETURNS (only if the fix is uncommitted and the tree is dirty)
<run repro>        # confirm red
git stash pop      # re-apply fix → expect bug GONE
<run repro>        # confirm green

Ecosystem variants: git bisect run <cmd> automates the search if you have a scripted pass/fail; Python uses pdb/breakpoint(), Node uses node --inspect; both honor "read the whole trace".

Failure Modes & Anti-patterns

Symptom	Mistake	Correction
Fix "works" but bug comes back	Never confirmed root cause	Revert the fix; if the bug doesn't return, you fixed a coincidence
Hours lost on a "mysterious" error	Read only the last line of the trace	Read the ENTIRE message + stack, both directions
Two changes, still broken, no idea why	Changed multiple variables at once	One variable per experiment
Endless "try this" edits	No hypothesis, guessing	State a falsifiable hypothesis first
Blamed the code, it was the environment	Never diffed environments	Fails-here-not-there ⇒ compare versions/env/OS first
Broke overnight with no code change	Assumed it was your code	Suspect a transitive dependency; check lockfile/versions
Fought the framework for hours	Assumed your usage was wrong	Test the primitive directly; if that works, the abstraction is the bug
Can't reproduce, "fixed" anyway	Skipped step 1	No deterministic repro ⇒ no verifiable fix
Fixed the symptom line, siblings still fail	Patched where it surfaced	Trace to the producer; fix the shared root (see change-control)

Repository Examples

Repo facts below are EXAMPLES of the loop in action, not assumptions for other repos.

ORPHY — the framework abstraction WAS the bug (as of 2026-07-04) ~/orphy (Vite + React vocal coach) carries commit "fix(frontend): replace Tailwind centering with reliable inline styles" — a rendering bug resolved by dropping from framework utility classes down to CSS primitives (inline styles). This is the "drop to primitives" branch of the environment/code/dependency tree: when behavior only misbehaves through the abstraction, test/replace it at the primitive layer. (The precise causal chain is a hypothesis — requires verification.) Orphy's broader design also shows verification discipline: contract-first phases where DSP measures numbers and the LLM only interprets, with deterministic validators gating each phase — a way to isolate which layer produced a bad result.

RAGIT — a textbook "dependency moved under me" failure (as of 2026-07-04) ~/ragit: chromadb 0.4 lacked a numpy<2 constraint, so numpy 2.0 was installed transitively and broke at RUNTIME with no code change by the owner. This is the DEPENDENCY branch of the triage tree exactly. The diagnosis path: fails after an install, not after an edit ⇒ check resolved versions against constraints ⇒ pin. It took two commits (a1ad63ed pin chromadb, then 0a06ae9b pin numpy) because the first fix addressed only the direct dependency, not the transitive one that actually broke — a reminder to verify the true root cause before declaring victory. No CI meant it surfaced only under manual testing.

PRISM — designed so failures are distinguishable at the boundary (as of 2026-07-04) ~/prism distinguishes failure classes by design, which makes debugging deterministic: OllamaUnavailableError is a fatal abort (the whole review stops), whereas a per-chunk timeout merely skips that chunk. When something goes wrong you can immediately tell "the model backend is down" (environment/dependency) from "one chunk was slow" (transient) rather than guessing. Reproduce with the documented flow: pip install -e ., run prism, tests via pytest tests/.

AGENTIX — the loop survives bad inputs so you can observe them (as of 2026-07-04) ~/agentix turns tool exceptions into observation strings so the ReAct loop survives (with a hard cap of 10 iterations), and its malformed-JSON retry recovers on 1 strike but surfaces the raw output on 2 — meaning a debugging session gets the actual bad payload to read instead of a swallowed error. Timeouts are explicit (shell 30s, python 15s, web 15s), so a hang is bounded and attributable. Tested with pytest using monkeypatch + tmp_path.

Validation Criteria

You applied this skill correctly if:

1. You had a deterministic reproduction before you theorized about causes.
2. You can quote the specific line/message in the error that pointed at the cause.
3. You can state the one hypothesis you tested and the one variable you changed.
4. You classified the failure as code, environment, or dependency, with evidence.
5. You confirmed the root cause by reverting the fix and watching the bug return.
6. Your fix lands at the shared cause, not the line where the symptom appeared.
7. If a framework was involved, you verified the primitive works before blaming/removing the abstraction.

Provenance & Maintenance

Sources: ~/orphy, ~/ragit, ~/prism, ~/agentix, investigated 2026-07-04. Methodology is repo-independent.

Assumptions made:

• Orphy's Tailwind→inline-styles commit is interpreted as an abstraction-was-the-bug case; the exact causal mechanism is a hypothesis — requires verification.
• ragit's numpy break is reconstructed from the pinning-commit chain, not from a captured stack trace.
• Repos are local-first with zero CI, so failures surface under manual runs, not automated ones.
• Commit hashes/messages reflect the fact pack as of 2026-07-04; not re-verified in this session (git access unavailable at authoring time).

Re-verification commands:

git -C ~/orphy log --oneline | grep -i "inline styles"   # confirm the fix commit exists
git -C ~/ragit log --oneline | head -6                    # expect a1ad63ed, 0a06ae9b
grep -rn "OllamaUnavailableError" ~/prism                 # confirm fatal-vs-skip distinction
grep -rn "iteration" ~/agentix                            # confirm 10-iteration cap

Likely to drift: orphy's styling approach (may migrate again); ragit's pins (lifted when upstream fixes constraints); agentix timeout/iteration values; whether any repo adds CI (changes how failures first surface).

Maintenance checklist:

• Re-run re-verification commands; update hashes/paths that moved.
• If the orphy causal chain gets verified, remove the hypothesis marker.
• Confirm prism's fatal-vs-skip error taxonomy still holds.
• Re-stamp Repository Examples with the new verification date.