Loop engineering

Loop engineering

Overview

The kit runs 3 first-party bounded loops: Goal, Debug, Execute. It also runs a bounded-revise side-flow, test-plan-review-team (SPEC-203/204). This skill extracts the reusable shape from that fourth loop. Use it to design a new loop against a known pattern, instead of starting from scratch.

How to use this skill

You do not need to design the stop condition or the output shape before you start. Describe the artifact and the itch. This skill runs the rest.

Step 1 needs: the artifact, and why now. Example: "the README's feature list keeps drifting from the code." This skill checks it against the four gate criteria below and reports pass or fail.

Step 2 needs: what good and bad look like, in plain words. You do not need to sort checks into Tier 1 or Tier 2 yourself. Describe what wrong looks like, for each check you want. This skill sorts them and shows you the split, so you can correct it.

Step 3 needs: what the artifact should look like when done. Say whether it is a written-back section, a standalone report, or an applied edit. Say what verdict shape you want. If you do not say, this skill defaults to the kit's own shape: a written-back section, a three-way verdict (SOLID / REVISE / RECONSIDER), and an honest halt on non-convergence.

How it fires: name it directly ("use the loop-engineering skill"), or describe the problem close to the trigger phrases in this file's frontmatter. No slash command is required.

A caveat on reuse across sessions: a plugin skill fires in other sessions only after its change merges to master, and the installed plugin copy refreshes (claude plugin marketplace update). Until then, it runs only in a session that reads this file directly off its branch.

Step 1: Gate, before designing anything

docs/PHILOSOPHY.md's "Loop boundaries" and "Feature rejection criteria" set the bar. This skill invents no new one:

• Bounded-in-session only. The loop keeps working inside the current session, under a verifiable stop condition. An unbounded outer loop (an external while that re-spawns sessions) belongs to autonomous-runtime tools like GSD v2, not this kit.
• Serves 2+ lifecycle phases. Or names a real external consumer, like visual-team / ui-design does, as a downstream-facing exception. "Might be useful someday" does not count.
• Explainable in one sentence. If the README-table line for it runs long, the scope is too complex.
• Has a source citation. Trace it to a proven implementation, or to an existing kit pattern (per "synthesize, do not originate"). Do not invent a mechanic from nothing.

If any of these fail, the idea is not a loop. It may still work as a one-shot side-flow, like kit-health or absorb. A side-flow is simpler and does not need this skill.

Step 2: Pick the shape

Three shapes exist. Do not conflate them. The routing question: does the loop make ONE thing good (engine), check MANY existing things (campaign, or the audit-loop pattern), or find the BEST VARIANT of one thing (search-select)?

• Bounded-revise engine. Use this when the loop converges on one artifact. Dispatch N scanners against the artifact. Merge findings by severity. Revise the artifact. Re-check. Repeat until clean, or until the round cap hits. This generalizes test-plan-review-team.md Step 2-4:

  dispatch N scanners against <ARTIFACT>
                │
                ▼
     count findings (K), by severity
                │
   ┌────────────┼────────────┐
  K = 0    severity fell    neither
  

(clean) (even if K flat) fell │ │ │ │ revise <ARTIFACT> halt honestly │ round += 1 (cap 3) (report as-is) │ │ └───────────┴──── or round cap hit │ ▼ verdict: SOLID / REVISE / RECONSIDER

Parameterize three things: the artifact, who scans it (N lenses), and who revises it. The
reviser must be distinct from every scanner. The convergence rule tracks severity, not raw
count. A flat K still counts as progress if the worst severity dropped. See
`test-plan-review-team.md` Step 4.3 for the exact wording.

**The scan step runs in two tiers. It does not dispatch all N lenses every round.**
`docs/WORKFLOW.md` already states a cheap-first verification-cost-routing principle. This
section applies that principle inside the engine's own scan step.

Tier 1 runs a deterministic check: grep or bash, zero model cost. Use it for any criterion
that reduces to a mechanical yes-or-no, like a row that exists, a command that is present, or
a field that is set. Tier 1 runs every round and decides on its own.

Tier 2 runs the N-lens model critique. Dispatch it only for the criteria Tier 1 cannot reduce
to a script, like oracle quality, ambiguity, or determinism risk. Skip any lens whose whole
finding-space Tier 1 already cleared.

The stop condition becomes: Tier 1 is all clean, and Tier 2 hits K=0, or its severity drops,
or the round cap hits.

This fixes the engine's real cost problem. N parallel model dispatches every round cost too
much when most findings are mechanical. See `docs/research/2026-07-30-loop-engine-prior-art.md`
for the worked cost comparison.

- **Campaign / worklist iteration.** Use this when you drive an existing loop across many
items. It is not a new engine. It wraps an existing one.

Shape: build a worklist of untreated items. Run the existing loop or command on each item in
turn. Track progress. Stop when the worklist runs out, or a budget or blocker hits.

This reuses the Goal loop's own shape: "keep working one objective until a verifiable stop
holds." Here it points at a list instead of one objective. Do not build a second convergence
mechanic. Reuse the Goal loop. For the audit flavor of this shape, enumerate a set, verdict
each item, gate through a PR, see `docs/patterns/audit-loop.md`.

- **Bounded search-select.** Use this when the goal is the best VARIANT of one thing, not the
repair of one thing. Mutate a candidate, score it, keep it if the score improves, discard it
if not. Repeat under a fixed iteration budget. This is Karpathy's autoresearch loop
([github.com/karpathy/autoresearch](https://github.com/karpathy/autoresearch)), bounded.

All three preconditions must hold, or this shape is wrong:

1. **A cheap, unambiguous numeric metric exists.** A training loss, a false-positive rate
   against a corpus, a test pass count, a benchmark score. If a model must judge quality,
   that judge becomes the metric, and the corpus bar below applies.
2. **Evaluation is fast enough to afford many shots.** Karpathy's budget math: about 5
   minutes per experiment, about 100 overnight. One-hour evaluations make hill-climbing the
   wrong economics.
3. **Losers carry no information the next attempt needs.** A discarded variant is thrown
   away whole. If a failed attempt should teach the next one, use the bounded-revise engine
   instead.

Two adaptations are mandatory, per `docs/PHILOSOPHY.md` "Loop boundaries": a fixed iteration
budget (never "loop forever until interrupted"), and an honest-halt report at the end, what
improved, what plateaued, which variant won. Kit targets already queued for this shape, and
their corpus bar (10-30+ real transcripts, not yet met): `docs/PHILOSOPHY.md` "AutoResearch
optimization". Full source verification and adoption verdict:
`docs/research/2026-07-31-karpathy-autoresearch-loop.md`.

## Step 2b: The loop's survival set (mandatory before "designed")

<!-- scenario-gen --> A loop is not designed until its own survival scenarios
are named, per `docs/patterns/scenario-generation.md` (the actor of move 1 is
the operator driving the loop). Minimum set, every new loop:

- convergence (the happy path reaches the stop condition)
- non-convergence (the cap hits; the honest-halt report is right)
- bad input (the teach-on-bad-input path fires, never a dry failure)
- interrupted run (killed mid-round; state on disk lets a fresh session resume
or cleanly restart, and says which)
- gamed metric (the scanner/checker can be satisfied without the artifact
actually improving; name the cheapest gaming move and the counter)

These rows ride into the loop's spec and become its first test plan. The
gauntlet's rules 7-10 are this set's worked answers for one engine; a new loop
answers the same five for itself.

## Step 3: Where it lands once shaped

- A new bounded-revise engine becomes a new `/kit:<name>` side-flow. Register it in
`docs/WORKFLOW.md`'s "Opt-in side-flows" table, and in `docs/workflow-map.md`'s mirror.
Update both in the same commit, per that section's own companion-drift note.
- A new campaign wrapper usually becomes a `/kit:<name>` command. That command invokes the Goal
loop machinery. It does not become a new engine.
- An idea with no commitment yet goes into `_meta/BACKLOG.md`'s "v2 candidates" tier. The
entries already logged there, feature-list reconciliation, doc-drift, agent-effectiveness,
dependency patch, and the test-plan backfill campaign, show what Step 1 and Step 2 output
looks like.

## Lineage (satisfies Step 1's source-citation gate)

The engine's coarse shape is not original. It comes from Anthropic's own **Evaluator-Optimizer
/ Reflection-loop** workflow (["Building Effective Agents"](https://www.anthropic.com/research/building-effective-agents)).
Self-Refine (Madaan et al. 2023), Reflexion (Shinn et al. 2023), and CRITIC (Gou et al. 2023)
share the same lineage. A 2026-07-30 web sweep found no framework, paper, or Claude Code skill
that packages the specific bundle used here:

- **Artifact-agnostic parameterization.** This matches Evaluator-Optimizer's own framing. The
kit did not invent it.
- **Severity-aware convergence.** A flat finding-count still counts as progress if the worst
severity dropped. No surveyed literature or tool does this. The kit added it.
- **Distinct reviser, never a scorer.** Practice shows this incidentally, one model reviews, a
different model fixes, but no source states it as an enforced rule. The kit added it.
- **Hard cap with an honest-halt reporting path.** The loop reports "not converging" as a real
outcome, not a silent stop. Pieces of this exist, max-round caps, keep-best fallbacks, but no
source packages them as an explicit contract. The kit added it.

Cite Evaluator-Optimizer for the shape. Do not claim the shape itself is new. The three deltas
above are the real contribution. Defend those three if this pattern gets challenged later.

## Reference

`commands/test-plan-review-team.md` is the worked engine instantiation. `_meta/BACKLOG.md`'s v2
candidates sketch four more engine instantiations and one campaign. None are built yet.