Citation faithfulness
Skill Hash-7777/Hash-Medical-Reasearch-Agent-Skills/skills/citation-faithfulness
Drop-in agent skills that make an AI appraise medical literature like a reviewer — grade every citation against its source, run reproducible PRISMA searches, pool studies safely, and defend RAG against prompt injection. Framework-agnostic, MIT.
npx -y skills add Hash-7777/Hash-Medical-Reasearch-Agent-Skills --skill citation-faithfulnessAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
2 things to look at
- 23 days oldThe repository was created 23 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
- 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Grade every [S#] claim in an answer against the exact evidence snippet it cites, and flag any claim the source does not actually support. Use whenever an agent produces medical or scientific claims tagged with numbered sources.
SKILL.md
3.1 KB, as published. Nobody here has run it
Citation Faithfulness
A citation is a promise: "the source I tagged says this." A model that breaks that promise is worse than one with no citations at all, because the tag manufactures trust. This skill makes the agent keep the promise by grading each claim against its own evidence before the answer is shown.
When to use
Any time the output contains claims tagged to numbered sources ([S1], [S3], [12], etc.) and a reader might act on them. Especially: clinical claims, drug interactions, effect sizes, safety statements.
The rule
No [S#] claim ships until it has been graded against the snippet it cites.
For each sentence that carries a source tag:
- Pull the exact evidence the tag points to — the retrieved snippet, the abstract sentence, the table cell. Not the paper's title. Not your memory of the paper. The specific text.
- Grade the claim → evidence relationship into one of:
- Supported — the snippet states the claim (or entails it directly).
- Partially supported — the snippet supports a weaker or narrower version. Rewrite the claim to match what the snippet actually says.
- Not supported — the snippet does not contain the claim. Remove the tag, and remove or rewrite the claim.
- Contradicted — the snippet says the opposite. This is a hard stop: fix the claim, and surface the contradiction to the reader.
- Never upgrade certainty. If the snippet says "may reduce," the claim cannot say "reduces." If the snippet is a single small trial, the claim cannot speak for the field.
How to check well
- Match the specific quantity, direction, and population. "Lowers LDL by 40%" needs a snippet with that number, that direction, in that population. A snippet about a different dose or a different group is not support.
- Watch for the fabricated middle step. Two real sources, joined by a causal claim neither one makes, is the most common failure. Grade the joining claim on its own.
- Distinguish the source from your prior. If you believe the claim but the cited snippet does not say it, the grade is not supported — go find a snippet that does, or drop the claim. Your prior is not a citation.
Output
For a graded answer, keep a short audit the reader can expand:
[S3] "Metformin reduces all-cause mortality in T2DM"
→ snippet: "...associated with lower all-cause mortality (HR 0.83, 95% CI 0.76-0.90)..."
grade: SUPPORTED (observational; claim softened to "is associated with")
[S7] "Adding the second agent doubles efficacy"
→ snippet: "...no statistically significant difference between arms (p=0.21)..."
grade: CONTRADICTED — claim removed, tag removed
The point is not to produce the audit. The point is that producing it forces the check, and the check is what makes the answer safe to read.