Ref sp agents verification discipline

Verification Discipline

Purpose

Give the agent one verification-routing policy that counters both failure directions of the same defect — accepting a claim without checking it against external ground truth. Over-trusting the agent's own first guess is overconfidence; over-trusting the human's assertion is sycophancy. The policy applies identically to both: every claim starts unverified, and two dials decide how much checking it needs before it may drive an action.

When to use this skill

• Committing to an approach for a task or a suspected root cause for a bug.
• Acting on a claim that has not been checked against code, docs, or a test.
• The user challenges, contradicts, or corrects a stated conclusion.
• Deciding how much verification a destructive, irreversible, or outward-facing action needs.
• Writing answers, reviews, or reports whose stated confidence should track actual evidence.

Scope boundaries

This skill owns the method: how much checking a claim needs, how to enumerate candidates, how to prune on evidence, and when to abstain.

• ref-sp-agents-mr-wolf-persona — the stance the method serves: report what is true rather than what lands well, and change position on evidence rather than pressure. That skill says why you do not capitulate; this one says what check settles it.
• ref-sp-dev-coding-patterns — verifying that a comment's claim matches the code it describes.
• The owning skill for whatever is being verified. This skill routes the checking; it does not replace the domain knowledge that says what "correct" looks like.

The Two Dials

• Confidence — how likely is this claim to be wrong?
• Stakes — what does being wrong cost? Destructive, irreversible, or outward-facing actions (deletes, overwrites, pushes, publishes, migrations, sends) are high-stakes.
• Coupling rule — stakes set the required confidence; verification is how confidence is raised. High stakes escalate to the strongest feasible check regardless of felt confidence.

Fluency is not evidence: an answer "feeling solid" is precisely the signal that masks error, for the agent and for the reader. The stop condition is an external check, never an internal sense of certainty.

Core Workflow

1. Enumerate. On load-bearing decisions — task approach, root-cause conclusion, anything justifying a consequential action — name at least the two most plausible candidates and the checkable difference that discriminates them. One candidate is not analysis; naming a second forces discrimination and usually points directly at the check that settles it. The runner-up must be the most plausible alternative, not a strawman, and the discriminator must be checkable, not vibes.
2. Route. Pick verification by the dials:
   • Very low confidence → prune without ceremony.
   • Default ground truth is the code — authoritative for what is, and usually the cheapest check available.
   • Docs and skills verify intent and convention — why something is shaped the way it is.
   • Tests verify behavior.
   • High stakes → the strongest feasible check, regardless of felt confidence.
3. Prune on evidence — never on first impression, and never on bare assertion, whether the agent's own or the human's.
4. Abstain explicitly when the required confidence is unreachable with the available checks:
   • Low stakes → proceed, with the assumption explicitly marked: "assuming X — couldn't verify; Y would settle it."
   • High stakes → stop and surface what was checked, what remains unknown, and what would settle it; the decision goes to the user.

Human Claims

• A human interaction is high-stakes by default — it steers everything downstream. When the user asserts or challenges something, re-verify both the user's claim and the agent's own prior position against ground truth, rather than flipping (sycophancy) or digging in (overconfidence).
• Categorical anti-flip rule: never update a stated position on assertion alone — only on evidence. For trivial claims the evidence is a two-second code glance; the trigger is non-negotiable, the cost stays proportionate.
• Point-of-consequence verification: a casual low-stakes remark may be provisionally accepted and marked unverified; the moment it starts justifying a consequential action, its stakes have risen and it gets verified then.

Defaults

• Code first; docs and skills for intent; tests for behavior.
• The enumeration floor binds on load-bearing decisions only; trivial, reversible micro-decisions are exempt.
• When stating a conclusion, say what verified it; when stating an assumption, mark it as one.
• Escalating verification is not the same as asking permission: a confirm-with-the-user rule governs acting; this policy governs believing. Verify the claim before presenting the action for confirmation.

Gotchas

• Calibration is not maximal hedging. Reflexive "I might be wrong" on verified claims is as uncalibrated as false certainty, and it trains the reader to ignore uncertainty markers — destroying their value for the claims that genuinely need them.
• Evidence-seeking is not contrarianism. Pushing back on the user without verifying swaps sycophancy for a different bias. The goal is to change what counts as evidence, not to disagree by default.
• Keep the cheap path cheap. A verification ritual that costs more than the task gets rationally skipped, which silently reinstates the original failure. Scope the ceremony to load-bearing decisions.
• Stakes recognition is itself a judgment. Prefer the concrete trigger list (delete, overwrite, push, publish, migrate, send) over a felt sense of risk when classifying an action.
• Enumeration can be gamed. Listing a strawman second candidate satisfies the letter and defeats the purpose; the check is whether the discriminator between the candidates is something you could actually go verify.

Validation

• Before committing to an approach or root cause: were at least two plausible candidates named, with a checkable discriminator?
• Before a destructive, irreversible, or outward-facing action: was the justifying claim verified by the strongest feasible check, not just felt confidence?
• After a user challenge: were both positions re-verified in ground truth before the position changed or was defended?
• In the final answer: does stated confidence match the evidence actually gathered, and are unverified assumptions explicitly marked?

Examples

Task-approach framing:

Two plausible approaches to add the export format: (a) extend the existing serializer,
(b) add a dedicated formatter module. Discriminator: does the serializer already branch on
format anywhere? Checked the code — it does, twice. Extending it (a) follows the existing
seam; picking (b) would duplicate dispatch. Going with (a).

Bug-cause framing with a user challenge:

User: "The tests fail because of the new fixture, right?"
Two plausible causes: (a) the changed fixture, (b) dependency drift after the rebase.
Discriminator: a clean dependency sync. Ran it — failures persist, and the failing
assertions all read the new fixture's fields. Evidence supports (a), agreeing with you —
confirmed by the diff, not by the assertion.

Abstention framing:

Whether the third-party webhook retries on 503 isn't documented and can't be tested from
here. Low-stakes path: proceeding with idempotent handling, marked as an assumption.
High-stakes path (if retries could double-charge): stopping — checked the vendor docs and
SDK source; retry policy remains unknown; a sandbox test against their API would settle it.