agentsclimarketplace

Reward process integrity

Skill jpoindexter/reward-seeking-safety-skills/skills/reward-process-integrity

Agent skills and eval utilities for reward-seeking, grader-targeting, and oversight-dependent behavior.

Install
npx -y skills add jpoindexter/reward-seeking-safety-skills --skill reward-process-integrity

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

  • 15 days oldThe repository was created 15 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Audit and harden the reward path around an AI agent. Use when an agent might read or modify graders, hidden tests, verifier code, score files, budgets, model selection, monitors, audit logs, release thresholds, or other evaluation machinery; also use for reward hacking and grader gaming controls.

SKILL.md

1.5 KB, as published. Nobody here has run it

Reward Process Integrity

Map the complete path:

task generation -> agent observation/action -> evidence capture -> verifier ->
score aggregation -> release decision

For each boundary, record owner, read access, write access, indirect influence, failure behavior, and receipt. Assume the plan failed because the agent obtained a passing score without completing the intended task; identify the exact path.

Apply these controls:

  • Separate principals for generation, execution, verification, and release.
  • Keep holdouts, expected outputs, and condition labels unavailable to the agent.
  • Make verifier, budgets, model selection, logs, and thresholds immutable from the evaluated workspace.
  • Detect indirect influence through tools, network calls, environment, generated reports, and skipped checks.
  • Log denied attempts and preserve them even if the task later passes.
  • Make errors actionable values; do not let verifier failure become a pass.
  • Require independent evidence before release.

Read ../reward-safety/references/deployment-guardrails.md. Return a data-flow map, access matrix, demonstrated attack paths, blocked paths, residual risks, and the smallest executable proof that the boundary holds.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.