Evaluation and quality harness
Skill alainlebret/claude-agents/higher-ed-teaching-agents/skills/evaluation-and-quality-harness
Modular skill and workflow ecosystems for AI coding assistants. Structured, multi-agent pipelines for specialized professional domains.
npx -y skills add alainlebret/claude-agents --skill evaluation-and-quality-harnessAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Measures exactitude, quality, and stability of runs produced by the teaching-agent system using programmatic checks, rubric-based grading, and benchmark cases. Use when validating workflows, benchmarking outputs, or comparing runs.
SKILL.md
3.1 KB, as published. Nobody here has run it
Evaluation and Quality Harness
Goal
Systematically verify that a workflow run produced the expected artefacts, that those artefacts meet the quality criteria defined in the shared policies, and that results are stable across multiple runs (no random drift).
When to use
- After completing a workflow run to certify it before publication or student distribution.
- When comparing two versions of the same course or exam to measure improvement.
- When benchmarking correction stability (same submission graded twice should yield the same score).
Inputs
run-manifest.json— list of produced artefacts and run metadataquality-report.json— programmatic gate results fromcheck_quality_gate.pylanguage-report.json— language consistency results fromcheck_language_policy.pymission.json— source of truth for expected quality level and constraintsrubric.md— used to verify grading reproducibilityshared/quality-metrics/quality-metrics.md— the quality dimensions to evaluate
Output contract
| File | Description |
|---|---|
benchmark/harness-report.md | Human-readable report: each quality dimension with a pass/review/fail verdict and evidence |
benchmark/benchmark-cases.json | List of benchmark cases run, inputs used, scores obtained, stability delta |
benchmark/stability-delta.json | If run twice: score difference per criterion; acceptable delta ≤ 0.5 points |
Quality dimensions evaluated
For each dimension, produce a verdict (pass / review / fail) with evidence:
| Dimension | What to check |
|---|---|
| Specification coverage | Are all learning_outcomes from mission.json covered by at least one rubric criterion? |
| Scientific exactitude | Do technical claims in statement.md and teacher-notes.md match established facts? |
| Internal coherence | Does the rubric total match the stated maximum? Do criteria refer to tasks that appear in the statement? |
| Level adequacy | Is the expected cognitive load consistent with the academic level in mission.json? |
| Workload realism | Can a student realistically complete all tasks in the stated duration? |
| Evaluability | Can every rubric criterion be applied reproducibly without subjective interpretation? |
| Stability | Re-running the correction on the same submission yields scores within the acceptable delta. |
Rules
- Never invent evidence. If a dimension cannot be evaluated without executing code, state that explicitly and mark it
review. - Always cite the specific artefact line or section that supports each verdict.
- If
overall_statusis notpassafter all checks, emit a prioritised list of fixes. - Distinguish programmatic failures (wrong output, missing file) from judgement failures (vague criterion).