Evidence coverage

Skill Hash-7777/Hash-Medical-Reasearch-Agent-Skills/skills/evidence-coverage

Drop-in agent skills that make an AI appraise medical literature like a reviewer — grade every citation against its source, run reproducible PRISMA searches, pool studies safely, and defend RAG against prompt injection. Framework-agnostic, MIT.

Install

npx -y skills add Hash-7777/Hash-Medical-Reasearch-Agent-Skills --skill evidence-coverage

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

23 days oldThe repository was created 23 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Check whether the retrieved evidence actually covers every concept in the question before answering — score the coverage, name the gaps, and re-retrieve when a core concept is missing instead of answering around it. Use after retrieval and before writing the answer.

SKILL.md

3.1 KB, as published. Nobody here has run it

Evidence Coverage

An AI will happily answer a three-part question using evidence that covers only one part — and the reader can't tell. The gap is invisible in a fluent paragraph. This skill makes the agent check its own retrieval: does the evidence in hand actually touch every concept the question asked about? If a core concept has no supporting chunk, the honest move is to go back and search, not to improvise.

When to use

After retrieval, before composing the answer. Pairs naturally with query-planning: the concepts the planner produced are exactly what coverage checks against.

The rule

Before answering, map the question's concepts against the retrieved evidence. Score the coverage, list what's missing, and re-retrieve when a core concept has no support.

List the concepts the question requires — the same concept groups from planning (drug, outcome, population, comparator), each expanded with its synonyms so a match counts even when the paper uses a different word.
Mark each concept covered or missing — covered if at least one retrieved chunk genuinely addresses it (synonym match counts), missing otherwise.
Score coverage — the fraction of concepts with support (0 to 1). A high-level number the reader and the agent can act on.
Decide: answer or re-retrieve — re-retrieve when coverage is poor or when any core concept (not a peripheral qualifier) is missing. A missing core concept means the answer would be built on a hole.

Doing it well

Core vs peripheral. A missing comparator may be tolerable; a missing drug or outcome is not. Weight core concepts harder — losing one of them triggers re-retrieval on its own, even if the overall score looks fine.
Count synonym matches. Coverage that only matches exact words undercounts real support and forces needless re-retrieval. Expand each concept before checking.
Ignore filler. Generic academic words ("study", "effect", "results") are not concepts and must not count toward coverage.
State the gap in the answer. If you answer with partial coverage (because re-retrieval found nothing), tell the reader which concept is unsupported. An honest gap beats a confident guess.

Output

Question concepts:  metformin · all-cause mortality · heart failure · type 2 diabetes

  metformin            covered   (3 chunks)
  all-cause mortality  covered   (2 chunks)
  type 2 diabetes      covered   (2 chunks)
  heart failure        MISSING   (0 chunks)   ← core concept

Coverage: 0.75  →  re-retrieve for "metformin AND heart failure AND mortality"

The value is not the score. It is that the agent notices the hole before it writes a confident paragraph that pretends the hole isn't there.