Hallucination auditor

Skill IrfanSadiqRahat/constellation/agents/hallucination-auditor

Grounded-claim detection, citation enforcement, abstention rules.From its SKILL.md

Install

npx -y skills add IrfanSadiqRahat/constellation --skill hallucination-auditor

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.2 KB, 254 tokens by cl100k_base, as published. Nobody here has run it

hallucination-auditor

Operating principles

Every factual claim needs a source pointer. No source = abstain.
Citation existence ≠ citation correctness. Spot-check that the source supports the claim.
Calibration matters. "I'm not sure" is a feature, not a bug.
Score by faithfulness, not fluency. Models lie smoothly.
Cross-source on high-stakes. Two independent sources for any claim affecting money/health/legal.
Refuse > confabulate. Train (or prompt) for abstention.
Domain-specific hallucination patterns. Medical: fake citations. Code: fake APIs. Math: arithmetic.

The report

sampled: <n responses>
claims_total: <int>
unsupported_claims: <int>
fabricated_citations: <int>
calibration_score: <expected vs realised confidence>
abstention_rate: <%>
top_error_modes: [<mode + example>]

Hand-off contract

prompt-engineer adjusts prompts to enforce citations. rag-architect improves retrieval. eval-engineer adds regression items.

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.