Audit the oracle coverage

Skill XyraSinclair/ideonomy/skills/audit-the-oracle-coverage

Computational ideonomy: Gunkel's science of ideas as inference-time machinery — a 37-primitive organon, an MDL-ratcheted respiratory engine, and 14 gated agent skills (Claude Code plugin)

Install

npx -y skills add XyraSinclair/ideonomy --skill audit-the-oracle-coverage

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

The dangerous case is not "no oracle" — it is a WEAK oracle you falsely believe is exact. Before trusting any test suite, metric, benchmark, eval, or proof as if it settled the question, state explicitly what it CERTIFIES and what it does NOT — the gap between the proxy and the target. The failure it prevents: false certainty from a passing check that silently misses the spec (tests green but the spec was wrong), proxies value (offline metric up, user value flat), or has drifted (backtest on a system that has since changed). Triggers whenever a check comes back positive and you are tempted to call the thing "verified", and as the mandatory middle step between build-the-oracle-before-the-answer and triangulate-without-oracle.

SKILL.md

4.2 KB, as published. Nobody here has run it

audit-the-oracle-coverage

The matched pair build-the-oracle-before-the-answer (oracle exists) vs triangulate-without-oracle (no oracle) is a false binary. Most real work lives in the middle: a partial or proxy oracle that certifies something but not the whole claim, and the characteristic failure is treating that partial certificate as a full one. This skill is the three-way partition the binary was missing — exact / partial-proxy / none — and it routes each part to the right discipline instead of laundering a weak signal into confidence.

Fuses build-the-oracle-before-the-answer (P-1) with prove-the-coverage-denominator (P-11) applied to the oracle itself, and routes the uncovered residual to triangulate-without-oracle (P-9). Organon: P3 fault-model + P11 + P19 oracle + P37 residue-seed. Phase: JUDGE.

Procedure

State what the oracle actually certifies. Write the precise claim the check establishes when it passes — not the claim you wish it established. "All 412 tests pass" certifies the behaviors those tests assert, not "the code is correct."
Name the gap between proxy and target (P3). What is true of the target that the oracle does not see? Spec ambiguity it never probed, inputs it never ran, the distribution it was built on but no longer matches, the value it proxies but is not. If you cannot name a single thing the oracle misses, the audit did not happen — that itself is the red flag.
Partition the claim (P11 over the oracle). Label every part of what you are asserting as:
- covered — the oracle directly certifies it;
- proxied — the oracle correlates with it but with a known, named gap;
- uncovered — the oracle says nothing about it.
Route each part to its discipline. Covered → trust it. Proxied → either strengthen the oracle (build-the-oracle-before-the-answer on the gap) or carry the gap as explicit risk. Uncovered → if it is testable, build an oracle for it; if it is irreducibly normative/strategic, route to triangulate-without-oracle and name the owner.
Report the partition, not a binary verdict. "Verified" is replaced by "covered: X; proxied with gap: Y; uncovered, routed to Z."

The gate

You may not report a claim as "verified" until its parts are partitioned into covered / proxied / uncovered, with at least one named thing the oracle does not certify. An audit that finds the oracle covers everything is presumed to have not looked — partial oracles always miss something; name it or the gate fails. The one escape: an oracle that provably enumerates the claim's entire finite case space may be declared exact, and then the finiteness argument itself is what the audit must produce. The proxied and uncovered parts must be routed, not folded into the covered set.

Example

A migration's test suite passes, and the instinct is "settled, ship it." The audit: covered = the 412 asserted behaviors; proxied = "preserves user settings" (the tests check three settings shapes, correlated but not the full space); uncovered = users with null locale and deleted workspace memberships (no test exists). Routing the uncovered case to a targeted fixture (build an oracle for the gap) finds one settings row that silently drops — exactly the case "all tests pass" was about to certify away. The binary "verified vs not" would have shipped it; the coverage partition caught it.