Research loop

research_loop

Find specific items in a large corpus by walking it in chunks, dispatching a cheap routed model per chunk, checkpointing every chunk, and running the zero_context_audit skill on the consolidated findings before the architect reviews. Built for overnight runs on a cost ceiling.

Status v0.1.0

This skill is currently spec-only. The runner code (scripts/run_research_loop.sh plus the Python orchestrator) is deferred to v0.2.0 to allow the architect to confirm the target manifest format and corpus shape on a real workstream first. The spec below is implementation-ready.

When to use this skill

• When the architect names specific items they need found and points at a corpus that is too large to read in one Opus-context pass.
• When the corpus is heterogeneous (PDFs, Markdown, code, JSON) and a single grep or single-model pass would miss items.
• When the search has accept criteria that require a model judgment rather than a regex (e.g., "find textbooks that cover N+1 query optimization, even if the term is not used verbatim").
• When the run can be checkpointed and resumed (overnight, multi-hour).

When NOT to use this skill

• When a regex or filename glob is sufficient.
• When the architect has not yet defined the target manifest.
• When the corpus has not yet been declared (no path, no document IDs).
• When the cost ceiling has not been agreed.

Inputs required before running

The skill refuses to run without ALL FIVE of the following:

1. Corpus declaration: a list of paths, document IDs, repo URLs, or knowledge-base IDs. May include exclude patterns. Format in packets/<workstream>/corpus.yaml.
2. Target manifest: a list of items to find. Each target has:
   • id: stable identifier
   • description: one-paragraph plain English
   • exemplar: a known example (for the model to anchor on)
   • accept_criteria: bullet list of properties that qualify a match
   • reject_criteria: bullet list of properties that disqualify a match Format in packets/<workstream>/targets.yaml.
3. Cost ceiling: USD value above which the loop halts and surfaces to the architect.
4. Synthesizer model: routed model slug for chunked synthesis. Default moonshotai/kimi-k2-thinking (research lane).
5. Auditor model: routed model slug for the final pass. Must be a different family than the synthesizer. Default deepseek/deepseek-v4-pro.

Architect input pending

The first concrete workstream needs the architect to specify:

• WHAT to find (target manifest contents)
• WHERE to look (corpus declaration)
• HOW MUCH to spend (cost ceiling)

This is the open input that gates the v0.2.0 runner build.

Mechanism

The loop runs in five phases:

Phase 1: Manifest and corpus validation

• Load corpus.yaml and targets.yaml.
• Confirm every corpus path or ID is reachable.
• Confirm every target has all required fields.
• Compute corpus size: total tokens, total documents, total bytes.
• Compute estimated cost: corpus_tokens times synthesizer_input_rate plus expected output_tokens times synthesizer_output_rate, summed across chunks. Compare to cost ceiling. Halt if estimate exceeds ceiling, surface estimate to architect.

Phase 2: Chunking

• Split the corpus into chunks sized to the synthesizer's input window with 10 percent overhead for the prompt.
• Each chunk gets a chunk_id and a manifest of which corpus items it contains.
• Save chunk plan to packets/<workstream>/chunk_plan.json.

Phase 3: Per-chunk dispatch

• For each chunk, dispatch the synthesizer with a prompt that contains:
  • The targets manifest
  • The chunk content
  • Output format spec (JSON list of findings, each with target_id, excerpt, document path, accept_criteria_met)
• Capture output to packets/<workstream>/chunks/<chunk_id>.json.
• Append a checkpoint line to packets/<workstream>/findings.jsonl with the chunk's findings.
• On any failure (network, API error, malformed output), retry once, then mark the chunk failed and continue.

Phase 4: Consolidation

• Merge all chunk outputs into packets/<workstream>/consolidated.jsonl.
• Deduplicate findings by document path plus target_id.
• Compute summary statistics: findings per target, false-positive candidates (findings flagged ambiguously by the synthesizer), failed chunks.

Phase 5: Audit and verdict

• Run the zero_context_audit skill on consolidated.jsonl. The auditor checks:
  • Each finding's excerpt actually exists in the cited document (sample 5 percent at random)
  • Each finding's accept_criteria_met claim is plausible
  • The synthesizer did not invent target_ids not in the manifest
• Surface the consolidated findings plus audit verdict to the architect. The architect issues the final verdict.

Output

packets/<workstream>/
  corpus.yaml             input
  targets.yaml            input
  chunk_plan.json         phase 2 output
  chunks/                 phase 3 outputs (one per chunk)
  findings.jsonl          phase 3 checkpoint stream (append-only)
  consolidated.jsonl      phase 4 output
  audit.md                phase 5 output (zero_context_audit verdict)
  dispatch_log.json       cost and elapsed per chunk
  summary.md              architect-facing summary

Cost mechanics

• Idle parent wait while a chunk dispatches is zero cost (per references/cost_mechanics.md).
• Synthesizer cost is dominated by input tokens (corpus chunks). Use routed models on OpenRouter to keep per-token cost low. Kimi-k2-thinking at $0.60 per M input, DeepSeek-v4-pro at $0.435 per M input.
• Audit cost is bounded by consolidated.jsonl size. Typically under $0.05.
• Cost ceiling enforcement: every chunk dispatch records cost; if the running total exceeds 0.9 times the ceiling, the loop pauses and surfaces a partial-results decision to the architect.

Failure modes the loop watches for

• Synthesizer drift: per-chunk outputs that ignore the target manifest and produce general summaries instead. Detected by the audit phase.
• False-positive flooding: synthesizer marks too many borderline matches as found. Detected by per-target finding density (if more than 5 percent of corpus items are flagged for one target, surface for review).
• Silent chunk failures: synthesizer returns empty output without error. Detected by checkpoint stream gaps; failed chunks are retried once then marked.
• Cost overrun: the running total tracker halts the loop at 90 percent of ceiling.

Pairing with zero_context_audit

The loop calls zero_context_audit at Phase 5. The auditor receives:

• The consolidated findings file
• A 5-percent random sample of the original chunks (for excerpt verification)
• The targets manifest
• An audit prompt that asks specifically for excerpt-existence and accept-criteria plausibility checks.

If the audit returns any LIKELY FABRICATED finding, the loop halts.

Citations for this skill

• Agent Skills spec, Anthropic, agentskills.io.
• The empirical run on 2026-04-29 that produced this bundle, paraphrased in references/audit_pattern_origin.md.
• The shell-out dispatch reference at references/dispatch_via_or.md.

End research_loop.