Negative result branches

Negative-Result Branches

Every nontrivial optimisation project accumulates a graveyard of "this should have worked but didn't". The default fate of that work is: branch deleted, commit history rebased away, lessons lost. Six months later you (or a colleague) have the same idea, build the same thing, get the same negative result. This skill is the discipline that prevents that cycle.

The pattern

1. One branch per experimental hypothesis. Branch name encodes the hypothesis: experiment/foo-rewrite, feat/bench-bar, try/algorithm-baz.
2. The commit message captures the empirical number, not the intent.
3. The branch lives forever (or until a major repo cleanup), even — especially — when the experiment doesn't pan out.
4. The main docs reference the negative-result branches so future-you / future-someone-else can find them by topic.

When this skill applies

• You have an optimisation hypothesis (algorithmic, structural, representation, hardware-acceleration).
• You're not sure whether it'll pay off — that's why you're considering an experiment.
• The work is more than 30 minutes — small enough to throw away, big enough to be worth recording even if it fails.

If you're certain it'll work, just do it on main. If you're certain it won't, don't waste the time. This skill is for the in-between.

Workflow

Phase 1: Branch and bench

# Branch from main
git checkout main
git checkout -b experiment/<hypothesis-shorthand>

# Implement. Doesn't have to be production-quality — it has to be correct
# enough to measure. Wire it into your bench harness.

# Measure. Capture numbers in a scratch file in the branch so the bench
# is reproducible later:
echo "..." > scratch/bench-results.md

Phase 2: Commit with the empirical number in the subject

The commit subject is the first thing future-you sees in git log --all --oneline. Make it carry the headline.

Good commit subjects:

experiment(strategy): beam-search K=5 at depth 5 — -43% vs baseline (6 seeds, 30 moves)
experiment(sim): bitboard mark_matches — 1.18× speedup, not 5-10× (8x8 too small)
experiment(opt): alpha-beta + B&B — same quality, slower (no-cache penalty > prune saving)

Bad commit subjects:

WIP: try beam search
beam search experiment
try bitboards

The body of the commit should expand: what was tried, what was the bench setup, what were the numbers, what's the hypothesis about why it failed (or succeeded).

experiment(strategy): beam-search K=5 at depth 5 — -43% vs baseline (6 seeds, 30 moves)

Top-level beam at depth 5, K=5: sort all root candidates by depth-1
score (including spawn-bonus credit), keep top K, run full depth-4
follow-up on each. Implementation correctness verified — K=50 (no
actual pruning) matches the exhaustive depth-5 search exactly.

Empirical, 6 seeds at 30 moves:

  seed   |  baseline  |  beam-K=5  |   Δ
  12345  |   186,900  |    77,800  | -58%
  1      |   150,100  |   110,400  | -26%
  2      |   100,200  |    77,400  | -23%
  3      |   133,600  |    66,600  | -50%
  4      |   130,600  |    95,200  | -27%
  5      |   175,900  |    72,300  | -59%
  mean   |   146,200  |    83,300  | -43%

Two compounding failures:
1. K=5 prune over-commits to depth-1 ranking; moves with low immediate
   score but great tails get pruned.
2. Deeper search doesn't outscore baseline on 30-move trajectories
   anyway — the "+18%" benchmark was 60-move; live games cap at ~25.

Branch preserved as a documented dead end.

Phase 3: Push the branch; don't merge

git push -u origin experiment/<hypothesis-shorthand>

That's it. The branch is its own documentation. No PR, no merge to main, no rebase.

Phase 4: Reference in main docs

In whatever doc on main discusses the topic (docs/perf.md, README.md, a blog post), include a section that names the branches and links to specific commits:

## What we tried that didn't work

Three optimisation hypotheses were tested on their own branches and didn't
pay off. They're preserved so future work doesn't reinvent the same dead
ends:

- [`experiment/beam-search`](https://github.com/.../commit/abc1234) — beam-K=5 at depth 5 scored -43% vs baseline. Combination of depth-1 prune mis-ranking AND deeper search not helping at the operating horizon.
- [`experiment/bitboards`](https://github.com/.../commit/def5678) — bitboard match detection gave 1.18× speedup, not the estimated 5-10×. The technique helps on larger boards; 8×8 is too small for the constant cost to dominate.
- [`experiment/alpha-beta`](https://github.com/.../commit/ghi9abc) — alpha-beta with admissible upper bound matches exhaustive quality but is slower because caching had to be disabled (lower-bound poisoning). A correct AB cache needs bounds-typed entries; out of scope.

Synthesis: the existing JIT + Zobrist transposition cache is at the
optimisation ceiling for this problem at this scale.

Link to specific commits, not branches — branches can move, commits are stable.

Why this matters

Future-you doesn't reinvent the failure

Six months later, your colleague says "wait, would bitboards help here?" and you can point at the commit with the numbers. The conversation ends in ten seconds instead of two days of re-implementation.

Blog material

Negative results are often the most interesting content. "Here's what we tried, here's why it didn't work" is more useful to readers than "here's what we shipped". The synthesis across multiple negatives ("the existing foundation is at the optimisation ceiling for this problem at this scale") is often the deepest insight in a write-up.

Invalidated assumptions surface

The failure mode itself often IS the finding. Bitboards being 1.18× instead of 5-10× isn't just "bitboards didn't help here" — it's "bitboards' published speedup is grid-size-dependent in ways that aren't obvious from the literature". That's a generalisable lesson.

The next reviewer trusts the chosen path

When you ship a feature, the reviewer's natural question is "did you consider X?". If you can answer "yes, on experiment/X, here's the diff and numbers", the review goes faster and you've demonstrated you considered the alternatives honestly.

Anti-patterns

"I'll commit it later"

You won't. The branch ends up rebased away in a cleanup. Commit before you start the next thing.

Commit message is the intent, not the result

"experiment: try bitboards" tells future-you nothing. The result has to be in the subject line.

Branch named after the technique with no result

feat/bitboards doesn't tell you whether it shipped or was tried-and-rejected. Use prefix conventions:

• feat/<name> — intended to ship.
• experiment/<name> — hypothesis under test.
• try/<name> — quick spike, may or may not survive.

Reserve feat/ for things you intend to merge. Use experiment/ for things explicitly meant as hypotheses.

Force-push to clean up history before "saving"

The exploratory commits ARE the value — they show what was tried in what order. Don't rebase them. Push as-is.

Documenting only positive results

If your docs only describe what worked, readers will assume everything else was untried. Document the failures too, with numbers.

Example: a synthesis paragraph

After three negative-result experiments, the synthesis is the headline. From a real perf-optimisation project:

Three negative results in a row. The synthesis is sharper than any individual finding: the existing foundation (JIT + transposition cache) is already at the optimisation ceiling for this problem at this scale. Marginal improvements over it (beam, bitboards, AB without cache) are slower or worse-quality. The optimisations that worked were the foundational ones (Tier 1 JIT for 21×, Tier 2 cache for 1.6× and worst-case bounding); the speculative algorithmic refinements didn't compose into further gains.

This kind of paragraph in a doc is high-value: it tells the reader what's not worth trying, which is rarer and more useful than telling them what worked.

A small process variant: the "tried and abandoned" registry

For teams, a single docs/tried-and-abandoned.md file that lists every negative-result experiment with one line each works well:

# Tried and abandoned

| Hypothesis | Branch | Result | Year |
|---|---|---|---|
| Bitboards on match-detection | [experiment/bitboards](...) | 1.18× (expected 5-10×); 8×8 too small | 2026 |
| Beam search K=5 at depth 5 | [experiment/beam-search](...) | -43% vs baseline; depth-1 prune mis-ranks | 2026 |
| Alpha-beta + B&B | [experiment/alpha-beta](...) | Correct, slower; cache disable cost > prune saving | 2026 |
| Switch to async I/O for the X path | [experiment/async-X](...) | +5% latency due to context switching | 2025 |

Easy to grep, easy to read, hard to lose track of.

Scope reminder

This skill is about engineering hygiene, not anything technical. It applies to any performance, structural, or algorithmic experiment where you'd otherwise just delete the branch when it failed. The cost is one commit and one line in a doc; the benefit is permanent.

Pairs with

This skill is meta to the others — it's the discipline you apply around any of them when the work doesn't pay off. Specifically valuable companion patterns:

• bit-exact-sim-validation — if you're optimising a port and a refactor breaks the trust anchor, the failed-refactor branch documents what didn't work.
• closed-loop-live-demo — sometimes the optimisation looks good offline but fails the live demo. Branch it, document the gap, move on.