Source reliability

Skill Hash-7777/Hash-Medical-Reasearch-Agent-Skills/skills/source-reliability

Drop-in agent skills that make an AI appraise medical literature like a reviewer — grade every citation against its source, run reproducible PRISMA searches, pool studies safely, and defend RAG against prompt injection. Framework-agnostic, MIT.

Install

npx -y skills add Hash-7777/Hash-Medical-Reasearch-Agent-Skills --skill source-reliability

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

23 days oldThe repository was created 23 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Rank evidence by study design and track record so the strongest source drives the answer, and disagreements between sources are surfaced rather than averaged away. Use whenever an agent weighs multiple sources of differing quality.

SKILL.md

2.6 KB, as published. Nobody here has run it

Source Reliability

Not all evidence is equal, and treating it as equal is how a preprint ends up outvoting a Cochrane review. This skill gives the agent an explicit hierarchy, so the weight of a claim tracks the strength of what backs it — and so conflicts between sources are shown to the reader, not silently resolved.

When to use

Whenever the answer rests on more than one source and those sources differ in quality, recency, or conclusion.

The rule

Weight each source by its design and record. When strong sources disagree, surface the disagreement — do not average it into a false consensus.

A working hierarchy (strongest first)

Systematic reviews / meta-analyses of RCTs (e.g. Cochrane)
Individual well-powered RCTs
Cohort and case-control studies
Case series / case reports
Mechanistic, in-vitro, or animal studies
Expert opinion, narrative reviews
Preprints and unreviewed sources — usable, but flagged as unreviewed

Design is the starting point, not the whole story. Adjust for:

Directness — does the study answer this question, in this population?
Recency — has newer, stronger evidence superseded it?
Record — retractions, failed replications, or known conflicts of interest lower trust.
Consistency — does it agree with the rest of the strong evidence, or stand alone?

Surfacing conflict

When high-quality sources point different directions, the honest output is not a blended number — it is the split, shown:

The strong evidence is divided:
  • Meta-analysis of RCTs (2024)   → benefit  (RR 0.85 [0.78–0.93])
  • Large RCT (2026, newer)        → no effect (RR 0.99 [0.90–1.09])
  Likely reason: the newer trial used a different endpoint / population.
  A reader should weigh recency and directness, not an average of the two.

Averaging these would erase the most important fact about the evidence: that it is unsettled.

What to put on the answer

Attach a compact trust signal to each claim or to the answer as a whole:

the tier of the strongest supporting source,
whether refuting evidence exists and how strong it is,
and a one-line reason the top source was trusted.

The reader should be able to see why the answer leans the way it does, and change their mind if they weigh the sources differently.