Eval
Checkpoint a change against its spec's acceptance criteria — run the machine-checks, adjudicate the prose ones, and gate ship until every criterion passes. Opt-in; not wired into the default pipeline. Use late in Verify, alongside /test and /verify.From its SKILL.md
npx -y skills add vasuag09/harness-claude --skill evalAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 2 stars2 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
3.1 KB, as published. Nobody here has run it
/eval — acceptance-criteria checkpoint
Goal: prove the work satisfies the spec it was built against, criterion by criterion. Green tests show the code runs; this shows it meets what was promised.
Opt-in. This skill is not invoked by any default-pipeline skill or hook. Running it is always explicit; it changes no baseline behavior. (Fulfills v0.4 AC-6 / v0.3 AC-E5.)
Do this
- Locate the spec. Default to the latest
docs/specs/*.md; let the user name one if the change targets an older spec. Confirm it's the spec this change was built against. - Run the deterministic gate:
It prints a per-criterion table (PASS / FAIL / MANUAL) and exits 0 (all PASS, none manual) · 1 (a check FAILED) · 2 (no failures, but MANUAL criteria remain).node scripts/eval/checkpoint.js [spec.md]- Each criterion may carry an inline
```evalfenced shell command — that's its machine-check. Add one to an AC when a deterministic check exists (port it from the spec's own test commands).
- Each criterion may carry an inline
- On FAIL (exit 1): the criterion is objectively unmet. Fix the implementation (not the criterion) and re-run. Do not proceed while any check FAILs.
- Adjudicate MANUAL (exit 2): for each prose criterion the script can't decide, gather
concrete evidence (run the app, read the diff, check
.claude/traces/for what fired) and judge PASS or FAIL yourself. The script never auto-passes prose — closing these is your job. If a MANUAL criterion is genuinely verifiable, prefer adding an```evalfence so it's deterministic next time. - Gate. Ship only when every criterion is PASS — checks green and every MANUAL adjudicated PASS with evidence. Report the verdict criterion by criterion.
Security & trust
```evalfences are shell-executed (via_lib.sh→execSync) with your privileges. They are author-authored, committed markdown — trust them exactly like the repo's own test scripts or build hooks. Review specs from untrusted branches/PRs before running/eval.checkpoint.js --listis the safe inspect-before-run path: it parses and shows which ACs carry a check, but executes nothing.- Secret-safe: a check's output is never printed. On FAIL the report shows only the exit code + the command (committed text), so you re-run it yourself to inspect output.
- Each check is bounded by a 60s timeout.
Notes
- Self-contained: the runner is pure Node and needs no companions (mgrep/graph/context7).
- Complements
harness-claude:test(suite + coverage) andharness-claude:verify(observe real behavior); this one ties the change back to its acceptance criteria.
Exit criterion
Every acceptance criterion in the spec is PASS (machine-checked or evidence-adjudicated).
Then harness-claude:ship.