Eval output
Scores a research output against the 5-criteria rubric via Sonnet.From its SKILL.md
npx -y skills add dbc-oduffy/coordinator-claude --skill eval-outputAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
2 things to look at
- no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
- 5 stars5 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
1.0 KB, as published. Nobody here has run it
Evaluate Research Output
Usage
/coordinator:eval-output <path-to-research-output>
Process
- Read the eval rubric at
${CLAUDE_PLUGIN_ROOT}/pipelines/deep-research/eval-rubric.md - Read the research output at the provided path
- Dispatch a Sonnet agent (model: sonnet, tools: Read, WebFetch, Write) with:
- The rubric
- The research output
- Instruction to: read the output, sample 3-5 cited URLs via WebFetch to verify citation accuracy, score each of the 5 criteria with a 0.0-1.0 score and 2-3 sentence justification, provide overall pass/marginal/fail grade
- Present the scores to the PM
Notes
- This is a post-hoc quality check, not a gate. Use it to calibrate prompt improvements.
- Start by running it on recent pipeline outputs to establish a baseline.
- Anthropic found a single LLM call with a single prompt was most consistent.
What ships with it
Read from the repository
Just SKILL.md. No reference files, no scripts.
Gives 0 of the 12 instructions most evals benchmarks skills give
Counted across 499 of the 513 authors here whose files we hold, read 2026-09-06
- Spawn with-skill and baseline runs in the same turnin 31 of 499, across 24 files
- Keep SKILL.md under 500 linesin 31 of 499, across 24 files
- Draft assertions while test runs are in progressin 31 of 499, across 24 files
- Compare against the baseline after changesin 31 of 499, across 13 files
- Define evals before codingin 26 of 499, across 17 files
- Run evals frequently during developmentin 25 of 499, across 16 files
- Keep evals fastin 24 of 499, across 15 files
- Version evals with codein 24 of 499, across 15 files
- Generate the eval viewer before evaluating outputs yourselfin 24 of 499, across 17 files
- Generate an eval report after runsin 24 of 499, across 15 files
- Track pass@k metrics over timein 22 of 499, across 14 files
- Save a baseline before making changesin 21 of 499, across 9 files
Said here and by no other author read
- Read the research output file
- Dispatch a Sonnet agent with rubric and output
- Verify citations by sampling 3-5 cited URLs via WebFetch
- Score each criterion 0.0-1.0 with a justification
- Provide an overall pass/marginal/fail grade
- Present the scores to the PM
Grouped from the skills themselves: near-identical wordings counted once, and counted by distinct author, so one author publishing three of these counts once.