agentsclimarketplace

Eval triggers

Skill datocms/agent-skills/.agents/skills/eval-triggers

Run the trigger evaluation pipeline — classify, analyze, and optionally compare against a baseline. Only run when explicitly asked — evals are expensive.From its SKILL.md

Install
npx -y skills add datocms/agent-skills --skill eval-triggers

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
  • 3 stars3 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.9 KB, as published. Nobody here has run it

IMPORTANT: This skill is expensive (makes many LLM API calls). Only run when the user explicitly asks for it. Never run proactively.

Before running, ask the user which eval source to run unless they already specified it in $ARGUMENTS:

  • Claude Code only — uses run_trigger_eval.py --track claude
  • Codex only — uses run_trigger_eval.py --track codex
  • Both — runs both tracks sequentially

Step 1 — Classify:

The runner writes to the canonical layout at evals/results/trigger/<skill>/<track>/<source>/results.json. You do not pass an output directory.

For Claude Code:

python3 evals/scripts/run_trigger_eval.py --track claude --source combined

For Codex:

python3 evals/scripts/run_trigger_eval.py --track codex

Step 2 — Analyze:

python3 evals/scripts/analyze_trigger_results.py \
  --track claude --source frontmatter

This writes the cross-skill summary to evals/results/trigger/_summary/<track>/<source>/summary.{json,md}. Report:

  • The gate verdict (pass/fail at F1 ≥ 0.90, with the list of skills below threshold).
  • Per-skill precision, recall, F1.
  • Any noteworthy false negatives / positives.

Step 3 — Compare (optional):

If the user provides a baseline summary or you have one from a previous run:

python3 evals/scripts/compare_trigger_runs.py \
  --baseline <baseline-summary>.json \
  --candidate evals/results/trigger/_summary/<track>/<source>/summary.json \
  --output-markdown local/comparison.md

Summarize regressions and improvements. For ad-hoc baselines that should not be committed, store them under local/ (gitignored).

Keep looking

Skills are one crate of 609,856. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.