Build the oracle before the answer
Skill XyraSinclair/ideonomy/skills/build-the-oracle-before-the-answer
Computational ideonomy: Gunkel's science of ideas as inference-time machinery — a 37-primitive organon, an MDL-ratcheted respiratory engine, and 14 gated agent skills (Claude Code plugin)
npx -y skills add XyraSinclair/ideonomy --skill build-the-oracle-before-the-answerAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Defeat answer-then-rationalize: when a claim is checkable more cheaply than it is producible (a row count, a benchmark number, "this refactor preserves behavior", a combinatorial count, "this query returns X"), write the independent truth-check AND commit its expected value BEFORE you produce the answer. The failure it prevents: an LLM emits a fluent answer and then manufactures support for whatever it already said — verification collapses into self-agreement because the check was built after the answer. Triggers on any quantitative or verifiable claim, especially counts, latencies, "this works", and anything you'd be tempted to assert without running.
SKILL.md
4.0 KB, as published. Nobody here has run it
build-the-oracle-before-the-answer
Feynman's master move, independently rediscovered by von Neumann (the recompute-script gate) and by production code-quality systems (the gated-not-attested ledger). The single differentiating insight is temporal: build the truth-signal before and independently of the answer, and commit to its expected value blind. Justification produced after an answer is rationalization, not verification.
Organon: P19 oracle + P3 fault-model + P20 self-verify, M1 generator/critic split. Respiratory: pre-commits the compression target before the breath in.
When NOT to use it
If the question is irreducibly normative/aesthetic/strategic and no honest independent check exists, do not fabricate one — that is the exact Goodhart failure. Route to triangulate-without-oracle instead. The discipline is: oracle-first when an honest oracle exists; named-judgment when it doesn't.
Procedure
-
Before answering, write the oracle. The smallest independent computation that returns a value or TRUE/FALSE for the claim: a
SELECT count(*), a five-line script, a closed-form recomputation, a unit / order-of-magnitude derivation, a property the answer must satisfy. It must not reuse the answer's intermediate values — if it does, it's circular and disqualified. -
Commit the expected value now, blind. Write down what the oracle should return if you are right — before you have the answer. Timestamp it (a file, a note). This commitment is what makes the check honest; an expectation formed after seeing the output proves nothing.
-
Pick the cheapest honest oracle for the shape of the claim:
- counts / "returns N rows" → run the query, check
ROW_COUNTand a monotonicity/uniqueness property. - quantities / latency / cost → units must algebra-out to the demanded units, and the answer within ~3× of a one-significant-figure mental estimate; sanity-check limiting cases (0, ∞).
- "behavior preserved" → a test or a re-run of the originating check.
- combinatorics → hand-compute the smallest case (see compute-the-smallest-nontrivial-case) and require the formula to reduce to it.
- counts / "returns N rows" → run the query, check
-
Produce the answer by your main reasoning path.
-
Run the oracle. Three-way compare expected vs oracle vs answer. Any disagreement → stop; the answer is unproven; reconcile before reporting.
The gate
A timestamped expected-value artifact exists before the answer, the oracle is independent of the answer's derivation, it executes, and its value matches the committed expectation and the answer. "I considered whether it's right and it seems fine" (narrated) does not pass; "I committed N=10 blind, ran it, got 10" (executed) does. If no oracle was writable, log that fact and route to triangulate-without-oracle — do not silently skip the gate.
Example
Task: "this SQL returns the 10 most-cited papers." Default writes the query and asserts it. This skill first commits the oracle blind: "if right, exactly 10 rows, citation_count monotone non-increasing, no duplicate paper id." Then runs it: 11 rows — a JOIN fan-out duplicated one paper. Caught only because the "= 10" was committed before seeing output. The narration-only path ships the dup with full confidence.