Karpathy task routing

Karpathy Task Routing

Karpathy: "Software 2.0 easily automates what you can verify" (karpathy.bearblog.dev/verifiability, 17 Nov 2025). Set autonomy per task from two questions: can I verify the output cheaply, and is this on-distribution or novel? The human is the bottleneck of the generation-verification loop, so task size scales to verification bandwidth, not generation bandwidth.

Decision procedure

1. Verifiability test. Is there an objective, machine-checkable success signal — test, metric, diff, command output? If not, restructure the task until there is one, or do not delegate: judgment and taste stay human. Grant full autonomy only where the check is automatic — give objective, metric, and boundaries, then leave the inner loop.
2. Distribution test. Boilerplate and internet-common patterns: delegate wholesale. Novel, dense, or unconventional code: hand-write with autocomplete, or document every deviation from convention in context before delegating. On nanochat, Karpathy reported agents were "net unhelpful" and gave the reason that the repo was possibly too far off the data distribution (nanochat HN thread, Oct 2025). Separately, on ordinary repos he reports bloated abstractions, dead code left behind, and edits to code orthogonal to the task (X status/2015883857489522876, 26 Jan 2026, agent-coding notes).
3. Set the slider, do not max it. Pick a rung from the slider below and name it before starting.
4. Granularity. Delegate at macro level — implement this feature, refactor this subsystem, research this library — declaratively: goals plus success criteria, not step-by-step imperatives. Specific beats vague, because a failed verification costs more than a longer prompt. Co-write a spec first for anything substantial; the spec is the durable artifact. Karpathy's version: with LLM agents there is less need to share the specific code or app, you share the idea (X status/2040470801506541998, Apr 2026, idea-file thread).
5. Parallelism. For parameter or config search, one sequential agent doing binary search beats parallel sweeps, which waste compute; for open-ended work, run a planner ("chief scientist") agent that emits concrete tasks for worker agents (HN comment 47293311, autoresearch thread, 8 Mar 2026). Run parallel agents only on disjoint functionality with no shared files or modules (house rule).
6. Escalation ladder when stuck. Climb one rung at a time; do not jump straight to the top.
   • Autocomplete, then a coding agent on the same task with a sharper success criterion.
   • Strongest reasoning model, with the FULL problem context pasted in — the failing code, the exact error, the docs.
   • For high-stakes questions, a council of frontier models; their cross-ranking is input, your own qualitative read is the final ranking.
7. On failure, assume skill issue first. Karpathy's framing: when agent workflows do not work, it all feels like a skill issue (No Priors ep. 80, 20 Mar 2026). Iterate the prompt, the context, the decomposition, and the harness before concluding the model cannot do it. To turn an observed failure into a standing rule, use karpathy-context-engineering — that skill owns the codify-mistakes rule.

Autonomy slider

• Autocomplete — routine code you already know how to write.
• Scoped agent task — a well-specified chunk with one named success criterion.
• Agent loop — an objective metric exists; the agent iterates unsupervised against it.
• Vibe mode, Accept-All, diffs unread — throwaway prototypes ONLY, consciously entered AND consciously exited. Karpathy: "You are not allowed to introduce vulnerabilities because of vibe coding." (karpathy.bearblog.dev/sequoia-ascent-2026, 30 Apr 2026; Karpathy states in that post that he had a model produce the cleaned transcript, so its wording is LLM-cleaned rather than raw speech).

Task -> setting

Task	Setting
CRUD endpoint, tests, glue code	Agent, wholesale
Novel algorithm, precise architecture	Hand-write + autocomplete
Unfamiliar language or framework	Agent (best leverage-to-risk)
Kernel or perf tuning with a metric	Autonomous loop
Naming, API design, strategy	Human
Weekend prototype	Vibe mode, entered and exited on purpose

Model & tool routing

Need	Route
Routine, low-stakes query	Cheap fast model
Math, debugging, multi-step logic	Reasoning model; accept the latency
High-stakes hard question	Council of frontier models (his council ships with four: gpt-5.1, gemini-3-pro-preview, claude-sonnet-4.5, grok-4); your own read is final
Any arithmetic or data work	Force code execution; never in-head model math
Post-cutoff or dynamic facts	Force web search and click the citations
Multi-source survey	Deep research as a first draft; read the primary sources it cites
Reading a paper	Load the current chapter and interrogate it section by section
Repeated preferences	Memory or custom instructions, pruned periodically

These rows follow the practice described in Karpathy's "How I use LLMs" (youtube EWvNQjAaOHw, 28 Feb 2025). The transcript could not be retrieved for verification, so treat every row as sourced paraphrase, not quotation. The four council models are from X status/1992381094667411768 (22 Nov 2025, llm-council announcement); in that thread the council ranked GPT-5.1 top while he preferred Gemini 3's answer.

Excuse -> reality

Excuse	Reality
Parallel will be faster	Only if the agents touch disjoint files; otherwise they interfere and you pay twice
The model just can't do this	Skill issue first: prompt, context, decomposition, harness
I'll review the output after	Can you actually check it? If not, it was never delegable
One clean run, ship it	works.any() is not works.all(); budget for the march of nines
A 10k-line diff is fine, it's generated	You cannot audit it, so nobody has

Everyday LLM usage and learning

When the question is how to work with AI day to day, how to structure a prompt, when to build a throwaway single-use tool, or how to learn a subject — including a bare "help me learn X", where the answer is a study approach and not a summary — read references/everyday-llm-usage.md before answering. It also holds the working habits and learning practices absorbed from the retired karpathy-llm-usage, plus the three routing rows too rare for the table above: dense reading turned into audio, voice input, and camera capture.

Sources

Karpathy: karpathy.bearblog.dev/verifiability (17 Nov 2025) for the verifiability test; karpathy.bearblog.dev/sequoia-ascent-2026 (30 Apr 2026, LLM-cleaned transcript) for the vibe-coding quality bar and macro-level delegation; nanochat HN thread (Oct 2025) for "net unhelpful"; X status/2015883857489522876 (26 Jan 2026, agent-coding notes) for the bloat and out-of-scope-edit failure modes; HN comment 47293311 (autoresearch thread, 8 Mar 2026) for bisection-over-sweeps and the chief-scientist planner; No Priors ep. 80 (20 Mar 2026) for skill issue; X status/2040470801506541998 (Apr 2026, idea-file thread) for share-the-idea; X status/1992381094667411768 (22 Nov 2025, llm-council announcement) for the council; "How I use LLMs" (youtube EWvNQjAaOHw, 28 Feb 2025) for the routing table. Third-party: latent.space/p/s3 (17 Jun 2025) records "Demo is works.any(), product is works.all()" as slide text from the YC Software 3.0 talk, and the march of nines is from the Dwarkesh interview (17 Oct 2025) — both are stated in full in karpathy-verification. MIT notice for this skill set, covering text adapted from multica-ai/andrej-karpathy-skills (formerly forrestchang/andrej-karpathy-skills): see karpathy-coding-loop/references/examples.md.