Rag eval guardrails

RAG eval & guardrails (verified, zero-API)

Prove a RAG/LLM feature is good enough to ship — and isn't leaking identifiers — with checks that run and exit non-zero on failure, not assertions. The eval scores a precomputed predictions file, so the gate needs no API access.

Core principle

Quality is measured, not claimed. The loop is: eval against a frozen golden set → read the failures → fix the prompt / retrieval / data → re-eval, until thresholds are met and nothing regressed versus the baseline. A separate PII/PHI guard fails the build if structured identifiers leak.

Be honest about scope (this is the rule that keeps the skill correct): the metrics here are lexical proxies — token overlap, containment, F1. They catch gross failures (hallucination, fabricated citations, retrieval misses) but do not measure truth. A correct paraphrase can score low; a wrong answer that reuses context words can score high. True faithfulness/correctness needs human review or an LLM-as-judge. The PII guard catches structured identifiers (emails, cards, SSNs…), not names or contextual PHI. Report "thresholds met on lexical proxies; no regression; no structured-identifier leakage" — never "the AI system is correct" or "compliant." → references/01-eval-driven-rag.md, references/02-metrics.md

When to use vs. not

• Use for: evaluating or regression-testing a RAG/LLM feature; measuring groundedness/hallucination, citation validity, retrieval hit@k, answer correctness, refusal rate, latency; adding an eval gate to CI; scanning prompts/answers/logs for PII/PHI leakage in regulated domains.
• Not for: training/fine-tuning models; a clinical or regulatory validation of an AI system (this assists, it does not certify); detecting personal names or free-text PHI (needs NER + human review).

Inputs to gather first

1. Golden set — questions with optional expected_answer / expected_sources as golden.jsonl. The contract everything keys off. → references/03-building-the-golden-set.md
2. Predictions — a precomputed predictions.jsonl ({id, answer, contexts?, citations?, latency_ms?}). Generate once via your endpoint, then score offline. → references/05-running-it-and-ci.md
3. Thresholds — the bars in rageval.config.json; set them just under an acceptable baseline, don't invent numbers.
4. What to scan for leakage — which prompts/answers/logs the PII guard runs over. → references/04-guardrails-and-pii.md

Workflow

Load each reference when you reach its step.

1. Adopt eval-driven development & accept the honest scope. Lexical proxies are necessary, not sufficient. → references/01-eval-driven-rag.md

2. Build the golden set — cover core facts, multi-hop, unanswerable, and adversarial/injection cases; freeze it like test code. → references/03-building-the-golden-set.md

   cp scripts/golden.example.jsonl golden.jsonl   # then edit to your domain
   

3. Generate predictions — the only step that touches your model. Wire your endpoint into the provider-agnostic stub, or supply any predictions.jsonl. → references/05-running-it-and-ci.md

   cp scripts/rageval.config.example.json rageval.config.json   # edit thresholds
   python3 scripts/client.example.py --golden golden.jsonl --out predictions.jsonl
   

4. Run the eval gate offline (no API); read report.md. → references/02-metrics.md

   python3 scripts/rag_eval.py --golden golden.jsonl --predictions predictions.jsonl \
     --config rageval.config.json --out-dir eval-report
   

5. Fix the root cause of the lowest-groundedness / failing items — retrieval, prompt, or data — and re-eval. Repeat until thresholds are met. → references/05-running-it-and-ci.md

6. Gate regressions against a trusted baseline report so upgrades/tweaks can't quietly degrade quality. → references/05-running-it-and-ci.md

   python3 scripts/rag_eval.py --golden golden.jsonl --predictions predictions.jsonl \
     --config rageval.config.json --baseline baseline/report.json --out-dir eval-report
   

7. Run the PII/PHI guard over answers and the prediction/log artifacts; it fails the build on any leak. → references/04-guardrails-and-pii.md

   python3 scripts/pii_guard.py --input predictions.jsonl --fields answer,contexts
   

What's in this skill

• scripts/rag_eval.py — the gate backbone: scores a precomputed predictions file (groundedness, F1/EM, hit@k, citation presence/validity, refusal rate, latency p50/p95), compares to thresholds and an optional --baseline, writes report.json+report.md, exits non-zero on any failure. STDLIB only, no network.
• scripts/pii_guard.py — PII/PHI leakage gate: regex + Luhn over JSONL/text for emails, phones, SSNs, MRN/DOB labels, IBANs, cards; redacts findings; exits non-zero on any leak. STDLIB only.
• scripts/client.example.py — provider-AGNOSTIC stub with a marked TODO showing where to call your LLM/RAG endpoint, plus a helper that dumps predictions.jsonl. Runs offline (emits refusals) until wired.
• scripts/golden.example.jsonl / predictions.example.jsonl — a 6-item self-test set with a correct refusal and one deliberately hallucinated answer + fabricated citation.
• scripts/rageval.config.example.json — thresholds + regression deltas.
• scripts/requirements.txt — stdlib-only; nothing to install for the runnable path.
• references/01–05 — eval-driven development & honest scope, metrics (how each is computed + its limits + the optional LLM-judge path), building the golden set, guardrails/PII/injection, and running it in CI.

Definition of done

• rag_eval.py runs on the golden + predictions and exits 0 with all configured thresholds met.
• A --baseline regression run passes (no metric drop beyond delta).
• Golden set covers core, multi-hop, unanswerable/refusal, and adversarial/injection cases; it's frozen and version-controlled.
• Lowest-groundedness items in report.md reviewed; failures traced to retrieval / prompt / data, not papered over by lowering a threshold.
• pii_guard.py runs on answers and the persisted prediction/log artifact and exits 0 (no structured-identifier leakage).
• CI runs both gates on the production endpoint's predictions; reports archived. Results reported as proxies, not as correctness/compliance.

Guardrails — avoid these mistakes

• Don't claim "correct" or "compliant" from a green eval. State "thresholds met on lexical proxies; no regression; no structured-identifier leakage." Overclaiming is the cardinal error here.
• Don't lower a threshold (or edit the golden set) to go green. Fix the retrieval/prompt/data root cause; baselines and golden items are the contract.
• Groundedness is overlap, not truth. A high score means lexically supported by the contexts, not factually correct — confirm with human review or an LLM judge for high-stakes outputs.
• Refusals cut both ways. Gate refusal rate with an upper bound; an over-cautious model is a regression other metrics miss.
• The PII guard misses names and contextual PHI. It's an early tripwire, not a HIPAA/GDPR attestation; pair with NER de-identification and human review.
• Don't log raw prompts/answers in regulated domains, and run the PII guard over the artifacts you persist — the most common leak is the log line.
• Keep the scoring path API-free and deterministic so CI stays cheap and reproducible; only generation should touch your model.