Zero context audit

zero_context_audit

Audit a synthesis output before any human or downstream agent acts on it. The auditor is a model from a different family than the synthesizer, spawned with zero parent context, given only the artifact and a rubric. This is a structural property: the auditor cannot fall in love with the parent's reasoning because it never sees it.

When to use this skill

• After any research synthesis that contains citations, repos, or commands.
• After any plan that lists specific install instructions or model slugs.
• After any code generation that imports unfamiliar libraries.
• Before publishing any document that downstream readers will trust.
• Whenever the workflow is about to end and the architect has not yet reviewed the artifact.

Do not use this skill for inputs that are already verified (datasheets, official documentation, pasted user content). The skill is for outputs.

When NOT to use this skill

• For pure stylistic edits or grammar fixes (no claims to verify).
• For artifacts shorter than roughly 200 words (verification overhead exceeds value).
• For artifacts the architect has already reviewed (architect verdict is authoritative; do not double-check).

Mechanism

1. The synthesizer produces an artifact. Save it to a known path.
2. Choose an auditor model from a different family. If synthesizer was Kimi, auditor is DeepSeek. If synthesizer was DeepSeek, auditor is Kimi. If synthesizer was Claude, auditor is DeepSeek or Kimi. The different-family rule prevents the auditor from sharing the synthesizer's training-data biases.
3. Render the audit prompt template (below) with the rubric scoped to the artifact's domain.
4. Dispatch the auditor as a one-shot completion. The dispatch path is documented in references/dispatch_via_or.md.
5. Capture the auditor's output. Parse the structured CHECK sections.
6. Halt the workflow on any LIKELY FABRICATED finding. Surface to the architect for verdict.

Audit prompt template

The template below is the canonical audit prompt. Adjust the CHECK sections to match the artifact's claim categories.

You are a zero-context auditor. You have NO prior knowledge of the
project that produced the document below, no relationship with the
model that produced it, and no incentive to be polite. Be specific,
terse, and skeptical.

Your job is to verify each claim in the document. For every claim,
mark it as one of:
- VERIFIED: claim is correct to your knowledge.
- WRONG ID / WRONG TITLE / WRONG ORG: claim references a real thing
  but with incorrect attributes; give the correct attribute if known.
- UNVERIFIABLE: you cannot confirm or deny; do NOT assume real.
- LIKELY FABRICATED: claim is plausible-sounding but you have no
  recollection and the pattern matches known hallucination shapes.

Group your output into these CHECK sections (adjust per artifact):
- CHECK 1: arXiv IDs and paper titles
- CHECK 2: GitHub repositories (org/repo)
- CHECK 3: command syntax and CLI invocations
- CHECK 4: model slugs, library names, version numbers
- CHECK 5: numeric claims (benchmarks, costs, performance)

End with OVERALL FINDINGS: 5 to 10 bullets ranking the most load-bearing
fabrications by danger. A wrong title is minor. A fabricated repo that
the user might pip install is dangerous.

If you suspect a claim is fabricated but cannot prove it, mark
UNVERIFIABLE and note the suspicion. Do not bluff certainty.

Hard cap output at 2500 words. Most audits are under 1500 words.

Output schema

The auditor produces Markdown with these sections in order:

## CHECK 1 - <category>
- <claim>: <VERDICT>: <one-line justification>
- <claim>: <VERDICT>: <one-line justification>
...

## CHECK 2 - <category>
...

## OVERALL FINDINGS
- <load-bearing finding ranked by danger>
- <next finding>
...

A downstream parser can extract verdict counts and trigger the workflow halt on any LIKELY FABRICATED finding.

Provenance fields

Every audit run records the following metadata to a sibling JSON file:

{
  "synthesizer_model": "<slug>",
  "synthesizer_family": "<family>",
  "auditor_model": "<slug>",
  "auditor_family": "<family>",
  "artifact_path": "<path>",
  "audit_path": "<path>",
  "elapsed_seconds": <float>,
  "cost_usd": <float>,
  "verdicts": {
    "verified": <int>,
    "wrong": <int>,
    "unverifiable": <int>,
    "likely_fabricated": <int>
  },
  "halt_triggered": <bool>,
  "timestamp": "<ISO 8601>"
}

Failure modes the auditor watches for

The auditor itself can be wrong. The skill mitigates this by requiring:

• Different family (auditor and synthesizer share fewer training biases).
• Zero context (auditor cannot inherit synthesizer's framing).
• Explicit UNVERIFIABLE option (auditor must distinguish "I do not know" from "I know it is wrong").

The auditor's output is itself an artifact. The architect is the only authority that can issue a final verdict. Two-auditor consensus is a v0.2.0 enhancement.

Cost mechanics

The audit is a one-shot completion. Input is the artifact (typically under 15 KB) plus the audit prompt (under 5 KB). Output is the audit report (under 10 KB). Total cost is dominated by output tokens at the auditor's rate.

Empirical sample, 2026-04-29, audit of a research report (32 KB):

• Auditor: deepseek/deepseek-v4-pro
• Input: roughly 5000 tokens
• Output: roughly 3400 tokens
• Cost: under $0.02
• Elapsed: 147 seconds

For artifacts under 50 KB, expect under $0.05 per audit.

Citations for this skill

• Agent Skills spec, Anthropic, agentskills.io.
• SelfCheckGPT (zero-resource hallucination detection), arXiv:2303.08896, github.com/potsawee/selfcheckgpt. This skill is a different mechanism (separate-family model receiving only the artifact) but shares the zero-resource intuition.
• Constitutional AI, arXiv:2212.08073, Anthropic. Conceptual lineage for principle-prepended scoring.
• Compound Engineering plugin, EveryInc, MIT, attribution-only reference. The plan-execute-review-test loop pattern.

End zero_context_audit.