Reasoning budget router

Reasoning Budget Router — escalate only the chunk that breaks

What this is (honestly): a routing technique, not a capability upgrade. The cheap model does not get smarter. What changes is WHERE compute is spent: instead of running the whole task on an expensive model "just in case," or running it all cheap and eating the errors, you decompose the task, draft everything cheap, catch the exact sub-part where the cheap model demonstrably fails or disagrees with itself, and send ONLY that sub-part — with the minimum context it needs — to a stronger model. This is the well-known "model cascade" pattern (cheap-first, escalate-on-signal), applied by hand and reported transparently.

What it isn't: it is not fable-hacendado (that wraps ONE cheap model in a heavier reasoning harness and never calls a stronger model). It is not "run everything twice for safety." It escalates a minority of the work, on evidence, not on hunch — and it says so.

When to use

• The task decomposes into independent-ish chunks (sections, sub-questions, steps, functions) where most chunks are low-risk (formatting, tone, extraction, summarization) and one or two are known trouble spots (arithmetic with several stacked rules, nested conditionals, long-range constraint tracking, edge-case code, precise counting).
• You have access to two model tiers: a cheap/fast default and a stronger/slower escalation option (same vendor or not).

When NOT to use

• The task is uniformly hard end-to-end with no clean chunk boundaries → just use the strong model; routing overhead won't pay for itself.
• The task is uniformly easy → skip the harness entirely, pure cheap, no ledger needed.
• It's a single tiny one-off task where the extra round trip costs more in latency than the escalation would ever save.

Inputs required

1. Task — full instructions plus acceptance criteria.
2. Cheap model — the default tier (e.g. Haiku-class).
3. Strong model — the escalation tier (e.g. Sonnet/Opus-class, or a bigger third-party model).
4. Decomposition hint (optional) — how to split the task into chunks; if omitted, split by the task's natural units (sections / sub-questions / functions / paragraphs).
5. Self-consistency sample count k (default 3) — how many independent cheap-model attempts to compare on checkable chunks.
6. Escalation cap (optional) — max % of chunks or max N escalations allowed, to bound cost.
7. External verifier (optional, preferred when available) — anything non-model that checks a chunk directly: run the code, recompute the arithmetic, confirm a citation/URL/date exists.

Procedure

1. Decompose. Split the task into the smallest chunks that can be drafted and graded independently. Label each one (A, B, C…).
2. Draft everything cheap. Run every chunk once on the cheap model. This is the full first draft — don't wait to see if escalation is needed before drafting.
3. Flag ceiling chunks, using whichever signal applies (checklist below). Do this per chunk, never for the whole task at once.
4. Escalate only the flagged chunk(s). Give the strong model: the original instructions for that chunk, the cheap model's attempt(s) and why they were flagged, and only the surrounding context needed so it doesn't have to re-derive the whole task (e.g. figures/decisions from untouched chunks it must stay consistent with).
5. Splice. Replace the flagged chunk's content with the strong model's answer. Leave every non-flagged chunk exactly as the cheap model wrote it.
6. Reseal, cheap. Run one more cheap-model pass over just the seams — where escalated content meets untouched content — to check referential consistency (same numbers, names, claims on both sides of the splice). This is a check, not a rewrite, so it stays cheap.
7. Report the ledger. Always output which chunks ran where and why (format below). This is the receipt that proves the escalation was evidence-based, not a vibe.

Escalation signal checklist (apply per chunk)

• Self-consistency disagreement — for chunks with a checkable answer (numbers, structured extraction, code output), run the cheap model k times (default 3). If the answers disagree beyond a stated tolerance, escalate.
• Self-declared low confidence — ask the cheap model to flag any step in its own draft where it had to guess an assumption or is under ~90% sure. Escalate flagged steps.
• External check fails — if a verifier exists (run the code, recompute the math, confirm a fact) and it fails, escalate. This signal outranks the two above when available.
• Structural complexity heuristic — pre-emptively treat as risk any chunk that requires holding more than ~4-5 simultaneous rules/constraints, nested conditionals, or long-range dependencies. Apply the self-consistency check to these proactively instead of waiting for a visible failure.
• Known weak spot for the cheap tier — multi-digit arithmetic, precise counting, subtle negation, order-of-operations across several stacked rules. Treat these chunk types as risk by default for that model.

Escalate the smallest span that contains the failure — never the whole document because one paragraph in it needs a stronger model.

Budget guardrails

• Hard cap on k (self-consistency samples): 3, unless the task explicitly needs more precision and the user accepts the extra cost.
• Escalation is a chunk-level decision — never "escalate the whole task because one part looked risky."
• If more than ~40% of chunks need escalation, stop routing and just run the whole task on the strong model; the routing overhead has stopped paying for itself.

Output format

## Result
<the final deliverable, merged>

## Escalation ledger
| # | Chunk | Model used    | Escalated? | Trigger |
|---|-------|---------------|------------|---------|
| A | ...   | cheap         | No         | — |
| B | ...   | cheap→strong  | Yes        | <signal that fired> |
| C | ...   | cheap         | No         | seam-checked against B |

## Budget summary
- Chunks total: N — Escalated: M (X%)
- Estimated share of tokens run on the strong tier: ~Y% (vs 100% if run entirely on strong)

Worked example

Task: Answer a customer's cancellation ticket — summarize the ticket, compute the refund per policy (prorate unused days of a $240/year plan, add a 5% Gold-tier loyalty bonus, convert to EUR at 0.92), and draft an empathetic reply.

1. Decompose: A = ticket summary/tone, B = refund calculation, C = reply draft.
2. Draft cheap (Haiku-class): A and C come out fine on one pass. For B: prorate 235/365 unused days of $240 → $154.52, then apply the loyalty bonus.
3. Flag: B has a checkable numeric answer → run k=3. Two runs apply the 5% bonus to the prorated amount ($7.73 → total $162.25 → €149.27); one run applies it to the original $240 instead ($12.00 → total $166.52 → €153.20). Disagreement → escalate B.
4. Escalate: send the strong model only the policy clause text, the two candidate totals, and the ticket's raw numbers, asking which order of operations the policy actually specifies.
5. Splice: the strong model confirms the bonus applies to the prorated amount per the clause wording → €149.27. That figure replaces the disputed number in chunk B.
6. Reseal cheap: one pass over C confirms the reply quotes €149.27, not the earlier wrong figure.
7. Ledger:

#	Chunk	Model used	Escalated?	Trigger
A	Ticket summary/tone	cheap	No	Low ambiguity, single pass sufficient
B	Refund calculation	cheap→strong	Yes	3-sample self-consistency disagreed on bonus base (2/3 vs 1/3)
C	Reply draft	cheap	No	Reworded only to insert the verified figure from B

Budget summary: 2 of 3 chunks stayed cheap; only the refund math (roughly 15% of total tokens) ran on the strong tier, versus 100% if the whole ticket had been routed to the strong model "just to be safe."

See also

fable-hacendado (heavier harness, same cheap model throughout, no escalation), plan (decomposition before building), ponytail (shortest path once inside a chunk).