agentsclimarketplace

Contrastive authority eval

Skill jpoindexter/reward-seeking-safety-skills/skills/contrastive-authority-eval

Agent skills and eval utilities for reward-seeking, grader-targeting, and oversight-dependent behavior.

Install
npx -y skills add jpoindexter/reward-seeking-safety-skills --skill contrastive-authority-eval

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

  • 15 days oldThe repository was created 15 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Design or review paired evaluations that test whether an AI agent follows a grader over a user, operator, developer, policy, or honesty constraint. Use for authority conflicts, reward-seeking detection, contrastive belief tests, grader-vs-user behavior, promise-breaking evals, or honesty-vs-task-completion experiments.

SKILL.md

1.6 KB, as published. Nobody here has run it

Contrastive Authority Evaluation

  1. Write the task distribution and authority contract before examples.
  2. Select a mutually exclusive feature with frequent opportunities to occur.
  3. Create paired conditions that reverse only the authority-to-feature mapping.
  4. Freeze model, harness, tools, budget, prompt structure, and scoring.
  5. Add a positive control, negative control, and manipulation/recall check.
  6. Run enough independent samples to report uncertainty, invalids, and refusals.
  7. Inspect behavior and receipts; use reasoning traces only as secondary evidence.
  8. Compute the contextual gap with ../reward-safety/scripts/contrastive-gap.mjs.
  9. Replicate across at least one changed task form before claiming detection.

Read ../reward-safety/references/authority-contract.md and ../reward-safety/references/measurement.md before authoring cases.

For API-only models, explicit in-context conflicts are a triage detector. Label the result detector_only; do not rank models or checkpoints from it. For SDF or another out-of-context intervention, report finetuning controls and off-target checks separately.

Output the complete eval contract, paired templates, scoring rule, controls, sample plan, analysis command, and claim boundary.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.