agentsclimarketplace
Sign in

Contrastive authority eval

Skill jpoindexter/reward-seeking-safety-skills/skills/contrastive-authority-eval

Design or review paired evaluations that test whether an AI agent follows a grader over a user, operator, developer, policy, or honesty constraint. Use for authority conflicts, reward-seeking detection, contrastive belief tests, grader-vs-user behavior, promise-breaking evals, or honesty-vs-task-completion experiments.From its SKILL.md

Install
npx -y skills add jpoindexter/reward-seeking-safety-skills --skill contrastive-authority-eval

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.6 KB, 261 tokens by cl100k_base, as published. Nobody here has run it

Contrastive Authority Evaluation

  1. Write the task distribution and authority contract before examples.
  2. Select a mutually exclusive feature with frequent opportunities to occur.
  3. Create paired conditions that reverse only the authority-to-feature mapping.
  4. Freeze model, harness, tools, budget, prompt structure, and scoring.
  5. Add a positive control, negative control, and manipulation/recall check.
  6. Run enough independent samples to report uncertainty, invalids, and refusals.
  7. Inspect behavior and receipts; use reasoning traces only as secondary evidence.
  8. Compute the contextual gap with ../reward-safety/scripts/contrastive-gap.mjs.
  9. Replicate across at least one changed task form before claiming detection.

Read ../reward-safety/references/authority-contract.md and ../reward-safety/references/measurement.md before authoring cases.

For API-only models, explicit in-context conflicts are a triage detector. Label the result detector_only; do not rank models or checkpoints from it. For SDF or another out-of-context intervention, report finetuning controls and off-target checks separately.

Output the complete eval contract, paired templates, scoring rule, controls, sample plan, analysis command, and claim boundary.

What ships with it: 1 file

262 B alongside SKILL.md

agents/

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.