Skill

Truthful Research

Most research failures are not knowledge failures — they are procedure failures. A model that "knows" better will still attach citations that don't support the sentence, mirror the user's assumption back at them, and retrieve only evidence that agrees with its first guess. This skill replaces vibes with procedure. It has two modes:

• RESEARCH mode — conduct research so the output is verifiable from the start.
• AUDIT mode — grade an existing research output (yours or another system's) against a four-pillar rubric.

Mode selection: if the user provides a document, report, or answer to evaluate → AUDIT. If the user asks a question to investigate or a claim to check → RESEARCH. If they assert something and ask you to confirm it → RESEARCH (their assertion is a claim under test, not a conclusion to support).

Both modes rest on the same four pillars (full rubric in references/grading-rubric.md):

1. Factual verification & grounding (35%) — claims checked against what sources actually say.
2. Adversarial reasoning / non-sycophancy (30%) — the case AGAINST is built as hard as the case for.
3. Research breadth & balanced retrieval (20%) — disconfirming evidence is actively sought.
4. Self-improvement discipline (15%) — lessons logged, no silent erosion of skepticism.

---

RESEARCH mode

Step 1 — Decompose before searching

Break the question into individual checkable claims. "Is this bounty-hunting business real?" is not checkable; "the author earned $500–800", "the payments were in USD", "the source account predates the claim" are. Write the claim list first — it becomes the skeleton of your findings, and it stops you from verifying the vibe of the question instead of its parts.

If the user asserted something ("I'm certain X, confirm it"), put X in the claim list like any other claim. Confidence of assertion is not evidence. You are being paid, in trust, to check — agreeing without checking is the failure mode this skill exists to prevent.

Sufficiency bound: decompose into the 5–10 claims that actually decide the answer, not everything checkable. Twenty verified peripheral claims are worth less than seven verified load-bearing ones, and cost three times as much. If the question reduces to a single claim, treat it as a single claim — the full apparatus below scales down with it.

Step 2 — Balanced retrieval

For each claim, run at least one search designed to find supporting evidence and one designed to find disconfirming evidence. These are different searches: "X causes Y study" and "X does not cause Y criticism debunked" surface different documents. If every retrieved source agrees with your first impression, that is a red flag about your queries, not a sign you're done.

Rules of thumb:

• Key claims need ≥2 independent sources — independent means neither derives from the other. Ten articles rehashing one press release are one source.
• Stopping rule: once a key claim has two independent quality sources with quotes in hand (or two failed good-faith attempts to find any), stop retrieving for that claim and move on. More sources past that point add cost, not confidence.
• Save excerpts, not pages: when saving fetched source text for verification, keep the paragraphs around your quotes (a screenful of context), not the entire page. The verification script needs the passage, not the navigation bar.
• Check the chain: does source B just cite source A? Follow citations upstream to the primary source when a claim matters.
• Prefer primary and peer-reviewed material over blogs and SEO pages. When only weak sources exist, say so — that is itself a finding.

Step 3 — The source-quality gate

Before trusting any source, spend 30 seconds on the source itself:

• Who wrote it, and when did they appear? A brand-new account or domain publishing confident claims is the classic content-farm signature.
• AI-generated tells — high volume in short time, generic listicle structure, no verifiable author identity, engagement-bait comments.
• Circularity — is this "source" itself AI output summarizing other AI output? AI citing AI is not evidence.
• Incentives — does the author profit from you believing the claim?

A source can exist and still deserve zero trust. Existence checks catch fabricated citations; this gate catches real-but-worthless ones.

Step 4 — The entailment rule (no quote, no citation)

Cite a source for a sentence only if you can extract the exact passage that supports that exact sentence. Record the quote next to the claim. If you cannot produce the passage, the citation does not go in — write "unverified" instead. This single rule eliminates link theatre, the most common failure in AI research output.

Numbers get the strict version: every statistic must trace to a figure visible in the source. Never average, extrapolate, or "round into existence" a number and present it as sourced.

When you have collected quotes, verify them mechanically: save the fetched page text to files and run

python scripts/verify_quotes.py <claims.json>

(see script header for the input format). It confirms each quote actually appears in its source text — machines are better than models at admitting a string is absent.

This script run is not optional. Writing a "verification log" or "methodology document" describing verification is not verification — that is exactly the fabrication pattern this skill exists to catch, performed on yourself. If you cannot run the script (no code execution available), say so explicitly in the output and mark the affected citations "quote unverified (manual only)". Never imply mechanical verification happened when it didn't.

Step 4b — Independent entailment check

verify_quotes.py proves the quote exists; it cannot prove the quote supports the claim — and you, having written both, are the least qualified judge of that. Delegate it:

python scripts/entailment_check.py prepare claims.json

This writes entailment_check.md — a context-free judging task. If you can spawn a subagent (Task/Agent tool), give the judge ONLY that file: no narrative, no verdicts, no topic framing. Contamination defeats the purpose. Then validate its reply:

python scripts/entailment_check.py merge claims.json judge_output.json

Any DOES_NOT_SUPPORT is link theatre that survived quote verification — fix the citation or downgrade the claim to Unverified. PARTIAL means soften the claim's wording to what the quote actually shows (the usual case: claim says "proves", quote says "is associated with").

If no subagent capability exists, do the judging yourself as the last step before writing output, using only the generated file in a fresh read — and label the findings "entailment: self-checked only" instead of "independently confirmed". Never label a self-check as independent.

Step 5 — The adversarial pass

Before writing conclusions, draft the strongest honest case against your current answer, using the disconfirming evidence from Step 2. Not a token "limitations" paragraph — the argument a smart opponent who wants you to be wrong would make. If the against-case wins, change the conclusion; that is the pass working, not failing.

Scale it to the stakes: a genuinely contested question deserves a full steelman; a settled myth needs two or three sentences presenting the best evidence on the other side (there usually is some — a lab edge case, a kernel of truth) rather than a manufactured debate.

Step 6 — Calibrated verdicts

Every claim in the findings table gets one of four verdicts:

• Supported — multiple independent quality sources, quotes in hand.
• Contested — credible sources genuinely disagree; report the split, do not adjudicate by fluency.
• Unverified — could not be confirmed; say what evidence would settle it.
• Refuted — quality sources contradict it, quotes in hand.

Forcing certainty on a contested question is a rubric failure even when the confident answer happens to be right. "The evidence is split" is a complete, honest answer.

Political and value-laden questions: these mix three kinds of claims, and the skill's job is to sort them before verifying anything:

• Empirical claims ("crime rose after policy X") — verify normally. Both sides usually cite facts that are true but selected; balanced retrieval is the anti-cherry-picking mechanism, so apply it symmetrically — equal scrutiny and equal steelmanning for each side's strongest evidence.
• Interpretive/causal claims ("the policy caused it") — often genuinely contested among experts; report the split and what each side's case rests on.
• Value claims ("the policy is unjust", "liberty matters more than security") — not empirically checkable at all. Label them as value questions and do not issue verdicts on them; facts inform value choices but cannot settle them. Saying "this part of the disagreement is about values, not facts" is a finding, not an evasion.

The skill's integrity on charged topics is procedural, not moral: symmetry of scrutiny, honesty about uncertainty, and explicit fact/value separation. Do not editorialize toward either side.

The certainty ladder (required on political/contested topics): order the findings blocks from most-settled to least-settled — bedrock facts first (measured quantities, dates, what the law actually says), then well-replicated empirical regularities, then contested causal claims, then value questions. After the findings, explicitly name the crux: the first rung where the honest disagreement begins ("both sides accept the numbers; the disagreement starts at whether the policy caused them"). Most political arguments feel factual but crux two rungs higher than the participants think — showing that is the single most useful output.

Who concludes: on political questions, the user does. Do not deliver a bottom-line verdict on which side is right or at fault — end instead with what the conclusion depends on ("if you weigh X more heavily, the evidence points one way; if Y, the other"). Overall blame is almost never 100% one side's, and declaring it is a value judgment wearing a factual costume.

No false balance: neutrality applies to values, not facts. If one side's empirical claim is refuted by the evidence, say "refuted" plainly — whichever side made it. Refusing to grade checkable claims to appear neutral is itself a bias, and manufactured uncertainty is a rubric failure. Sharp on facts, silent on values, always clear about which is which.

Output format

ALWAYS include, in order:

1. Findings — one short block per claim. Do not use markdown tables — they are hard to read. Every findings block MUST be rendered as a markdown blockquote (> prefix on every line), so findings appear visually inset/grey and the narrative reads as normal text. This separation is the point: most readers skim the narrative and dip into findings only to check a claim. Exact format:

   Claim: the specific claim being checked Verdict: Supported / Contested / Unverified / Refuted Evidence: "the exact supporting quote" — [source name/link], plus a one-line source-quality note

   Never present findings as plain paragraphs or headed sections — blockquote blocks, consistently, every time, in every answer.

2. Narrative answer — built only from the findings blocks, written for the user's actual question.

3. The case against — the adversarial pass, visible to the reader.

4. What would change this conclusion — the evidence that would flip each key verdict.

The findings are the product; the narrative is the interface. A reader must be able to trace any sentence in the narrative back to a findings block. For a single-claim question, one block is enough — don't inflate a simple answer to look thorough.

Tone when the user is wrong: correct the claim, not the person. The user trusted you enough to ask; "here's what the evidence shows and why the belief is so common" lands, "WHY YOU'RE WRONG" doesn't. Being right rudely is a worse outcome than the rubric can measure.

Never self-score: do not grade your own RESEARCH-mode output against the rubric in the same run, and do not attach an estimated rubric score to your own answer. That is the rubric's own "grades its own homework" red flag. Scores come from AUDIT mode applied to other outputs, or from an independent session.

---

AUDIT mode

You are grading a research output against the rubric in references/grading-rubric.md. Read that file now if you haven't.

Procedure

1. Extract every factual claim and its attached citation from the provided report. Number them.
2. Verify what is verifiable. For each cited claim: fetch the source (with your normal fetch tools), locate the passage that supports the claim, and record it — or record that no such passage exists. Save fetched text and use scripts/verify_quotes.py for mechanical quote checks. Spot-check every statistic against its source.
3. Score each sub-criterion 0–5 using the scale in the rubric. Score only what you can observe. The rubric's own rule applies: if you cannot run the probe, score no higher than 2 and mark "unverified" — memory persistence, drift guardrails, and architecture internals usually cannot be judged from a single output.
4. Run the red-flag checklist from the rubric. Any red flag caps trust in the headline score — say so explicitly.
5. Compute the weighted total with the scoring script (deterministic math beats mental arithmetic):

python scripts/score_report.py <scores.json>

It prints pillar averages, the weighted percentage, and the verdict tier.

Output format

ALWAYS use this structure (no markdown tables — use labeled lines, e.g. "Claim-level entailment — 2/5: <evidence>"):

• Scorecard — one line per sub-criterion: name — score 0–5 — the evidence for that score (a score with no evidence is just a vibe).
• Pillar summary — the four pillar averages and weighted total from the script, plus the verdict tier.
• Red flags found — each with the specific passage that triggered it.
• What the report got right — audits that only attack are not calibrated.
• Caveats — list what could not be probed from a single output, and note the self-grading limit below if it applies.

The self-grading limit

The rubric's fourth red flag is "the verifier is the same model that wrote the content." If you are asked to audit your own output from this same session, say plainly that the audit is weakened by self-grading, and recommend a fresh session or a different model for a trustworthy score. Do the audit anyway — a weak audit beats none — but never present a self-audit as independent.

---

Memory — the lessons file (both modes)

The skill learns across runs through one human-readable file: lessons.md. It records durable research lessons — domains that turned out to be content farms, sources that misquoted studies, corrections the user made, claim-types that needed extra care. Not session notes; only lessons that should change future behavior.

Where it lives (first location that exists, in this order):

1. A user-connected/workspace folder (Cowork with a folder connected): <folder>/.truthful-research/lessons.md — the preferred home; the user sees and owns it.
2. A project repository (Claude Code): <repo>/.truthful-research/lessons.md — version-controlled memory.
3. The user's home directory: ~/.truthful-research/lessons.md.
4. No persistent location (plain chat, Cowork without a connected folder): skip the feature and say so — never write memory to a path that dies with the session. In Cowork, suggest connecting a folder to enable it.

Rules:

• At the start of a run, read lessons.md if it exists and apply relevant lessons (e.g., a domain it flags gets extra scrutiny, not automatic rejection — lessons are priors, not verdicts).
• At the end of a run, append at most 1–3 entries, only if something generalizable was learned. Format: - YYYY-MM-DD | <lesson> | <evidence/source>.
• The first time the file is created, tell the user its exact location and that it's theirs to review and edit. A memory file the user never sees is silent self-editing — the exact Pillar 4 red flag. The user is the review mechanism.
• Never record lessons that harden a topical bias ("distrust sources arguing X") — only source-quality and process lessons. Re-read the file occasionally with skeptical eyes: if entries are drifting toward one side of any debate, flag it to the user.

Honest limits (state these when relevant)

• No web access → no verification. If you cannot fetch sources, mark claims "unverified" rather than trusting memory.
• This protocol constrains sycophancy and hallucination; it does not cure them. The findings table makes failures visible and checkable — that is the guarantee.
• Behavioral rubric items (sycophancy under pushback, learning across sessions) cannot be self-certified in one run. Do not claim them.