agentsclimarketplace

Skill quality eval

Skill citedy/skills/skills/skill-quality-eval

Curated collection of skills by Citedy — AI-powered SEO content automation

Install
npx -y skills add citedy/skills --skill skill-quality-eval

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Validate and audit quality of AI agent slash commands (.claude/commands/*.md) — checks YAML frontmatter, description length, jargon detection, directive language, and duplicate detection. Use when creating, editing, or auditing project commands and skills. Run proactively after adding new commands.

SKILL.md

2.6 KB, as published. Nobody here has run it

Skill Quality Evaluator

Validates .claude/commands/*.md files for quality, triggering accuracy, and best practices from Phil Schmid's (Google DeepMind) eval harness methodology.

What it checks

  1. YAML frontmatter — every command must start with --- block containing description:
  2. Description length — minimum 40 characters (short descriptions don't trigger well)
  3. Jargon detection — flags internal terms that confuse users (dogfood, --dangerously, etc.)
  4. Directive language — descriptions must contain action verbs (Run, Fix, Scan, Extract, etc.) because passive descriptions don't trigger Claude's skill routing
  5. Duplicate detection — similar commands must have distinct descriptions (first 30 chars normalized)

How to run

Execute the bundled script — it requires zero dependencies (pure Node.js):

node .claude/skills/skill-quality-eval/scripts/run-eval.js [commands-dir]
  • Default commands-dir: .claude/commands (relative to cwd)
  • Pass an absolute path to scan a different directory

The script outputs a structured report with PASS/FAIL per file, summary stats, and specific fix suggestions.

When a check fails

CheckHow to fix
Missing frontmatterAdd ---\ndescription: "..."\n--- at top of file
Description too shortExpand to 40+ chars — explain WHEN to use and WHAT it does
Jargon detectedReplace internal terms with user-facing language
No directive verbAdd action verbs: "Run...", "Fix...", "Scan...", "Use when..."
Duplicate descriptionMake the first 30 chars unique between similar commands

Customization

Edit scripts/run-eval.js to add:

  • Custom jargon patterns (line ~15, JARGON_PATTERNS array)
  • Additional directive verbs (line ~30, DIRECTIVE_PATTERNS array)
  • Differentiation pairs (line ~50, DIFF_PAIRS array)

Integration with CI

The script exits with code 1 if any check fails, making it suitable for pre-commit hooks or CI pipelines:

node .claude/skills/skill-quality-eval/scripts/run-eval.js || echo "Fix command quality issues!"

Background

Based on Phil Schmid's (Google DeepMind) eval harness methodology. See references/phil-schmid-eval-harness.md for the full framework and best practices.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.