Research results auditor
Audit ML/statistics experiment outputs for validity, confounds, statistical support, calibration, and mismatch between measured results and claimed conclusions. Use when asked to interpret results, sanity-check benchmarks, review ablations, assess robustness claims, or decide whether an experiment actually supports a paper or project claim.From its SKILL.md
npx -y skills add kenantang/codex-and-claude-skills --skill research-results-auditorAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
2 things to look at
- no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
1.9 KB, 321 tokens by cl100k_base, as published. Nobody here has run it
Research Results Auditor
Quick start
- Collect the result artifact, the claim it is supposed to support, and the evaluation protocol.
- Audit metrics, baselines, controls, uncertainty, and confounds before interpreting the headline result.
- Separate what the data shows from what the author wants it to imply.
- Produce an audit using
references/results-audit-template.md.
Audit workflow
1) Reconstruct the intended claim
- Write the target claim in one sentence.
- Identify the exact numbers, plots, or tables meant to support it.
- Flag any missing result artifact needed for verification.
2) Check protocol integrity
- Verify that metrics match the task.
- Check whether baselines, splits, and data filters are comparable.
- Confirm that ablations isolate the claimed factor rather than multiple changes at once.
3) Check inferential quality
- Look for class imbalance, calibration problems, threshold sensitivity, unstable aggregates, and cherry-picked best runs.
- Require statistical tests or uncertainty intervals when claims compare conditions.
- Treat non-significant or noisy deltas as weak evidence, not wins.
4) Check confounds and claim drift
- Ask what else could explain the result.
- Compare the measured quantity with the stated conclusion.
- Flag any jump from benchmark score to real-world robustness, safety, or causality without additional support.
References
references/results-audit-template.mdreferences/metrics-and-tests-checklist.mdreferences/tabmol-ddi-ood-adapter.md
What ships with it: 4 files
1.7 KB alongside SKILL.md
agents/
- openai.yaml320 B