agentsclimarketplace

Agentv eval migrations

Skill EntityProcess/agentv/skills-data/agentv-eval-migrations

Light-weight AI agent evaluation and optimization framework

Install
npx -y skills add EntityProcess/agentv --skill agentv-eval-migrations

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 14 stars14 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Migrate AgentV eval YAML across breaking schema changes, especially workspace contract updates and portable-vs-local runtime binding cleanup.

SKILL.md

2.4 KB, as published. Nobody here has run it

AgentV Eval Migrations

Use this skill when updating existing AgentV eval YAML, examples, docs, or generated eval authoring guidance after a schema-breaking change.

Before editing, read references/breaking-changes.md. For stale evals from AgentV v4.42.4, use that reference as the migration map: it lists the v4.42.4-era shape, current shape, migration steps, verification commands, and compatibility notes for each major breaking authoring change. Then compare the eval file against the current portable contract:

  • Treat migrations as rule-based rewrites first. Do not rely on a manual "looks current" pass when the old field has a direct current replacement.
  • Rewrite authored prompt data into top-level prompts plus tests[].vars; keep input only for documented raw-case import compatibility.
  • Move sibling expected_output to vars.expected_output, then add or keep an explicit assertion that consumes it, usually llm-rubric with {{ expected_output }}.
  • Rewrite target identity to id, keep provider for the backend/adapter kind, put provider settings under config, and rewrite env references to {{ env.NAME }}.
  • Remove use_target, eval_cases, evalcases, providerPromptMap, and provider_prompt_map from authored eval/config YAML. Current AgentV rejects these fields, so do not preserve them as compatibility aliases.
  • Keep committed eval YAML portable: prompts, cases, assertions, workspace templates, repos, hooks, env checks, Docker preflight/container config, and workspace.scope.
  • Do not put machine-local existing workspace directories in eval YAML. Use --workspace-path for one-off runs or .agentv/config.local.yaml with execution.workspace_path for persistent local binding.
  • Use workspace.scope: suite | attempt for portable workspace lifetime. Docker config is not a replacement for workspace folder lifetime.
  • Keep wire-format fields in snake_case and TypeScript internals in camelCase.

After migration, validate with bun apps/cli/src/cli.ts validate <file> when working in the AgentV repo, or agentv validate <file> from an installed CLI. Run the repo's parser/schema tests for generated examples and fixtures when the change affects shared skill data or examples.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.