agentsclimarketplace

Golden benchmark uplift loop

Skill g-shevchenko/agentic-quality-skills/skills/golden-benchmark-uplift-loop

Production-grade quality skills for AI coding agents: red-first TDD, quality gates, and golden benchmark uplift loops.

Install
npx -y skills add g-shevchenko/agentic-quality-skills --skill golden-benchmark-uplift-loop

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Use when improving a prompt, skill, eval, benchmark, scorer, calibration, judge, golden dataset, or agent workflow quality, and when the user says golden benchmark, golden dataset, golden regression, blind validation, unbiased evaluator, answer key contamination, uplift loop, before/after measurement, scorer calibration, prompt eval, or skill eval. Requires blind validation when the author has seen the answer key, deterministic grading, before/after measurement, and honest sample-size caveats.

SKILL.md

1.7 KB, as published. Nobody here has run it

Golden Benchmark Uplift Loop

Use this skill when quality must improve measurably, not just feel better.

Core Rule

If you wrote or saw the expected answer, prior passing output, golden labels, scorer threshold, or answer key, you are contaminated. Do not grade your own output by eye.

Instead:

  1. Write a general, principled guard or improvement.
  2. Use a deterministic grader or runner.
  3. Have a fresh executor produce output without seeing the answer key.
  4. Grade with the runner.
  5. Improve.
  6. Re-run blind with a fresh executor.
  7. Report before/after results and the sample-size caveat.

Read references/uplift-loop.md for templates and benchmark rules.

Golden Case Discipline

A golden is useful only when it captures a real behavior boundary:

  • valid case;
  • malformed case;
  • ambiguous case;
  • missing evidence case;
  • injection/adversarial case;
  • regression case from a real failure.

Do not overfit instructions to one golden. Add a general rule that would have prevented the bug class.

Done Criteria

Report:

  • what was measured;
  • who/what was blind to the answer key;
  • runner command;
  • before result;
  • after result;
  • sample size and residual risk.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.