Aicrew benchmark

Skill AKSoftCode/aicrew/codex-skills/aicrew-benchmark

TDD-first AI dev pipeline for Claude Code, Cursor, Codex, Gemini CLI & Antigravity. /dev /fix /quick + token-saving hooks. npx aicrew install

Install

npx -y skills add AKSoftCode/aicrew --skill aicrew-benchmark

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

3 stars3 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Run a benchmark to measure skill quality, response time, and token efficiency for aicrew skills.

SKILL.md

2.2 KB, as published. Nobody here has run it

aicrew-benchmark (Codex)

Use when: measuring aicrew skill output quality, latency, or token usage across skills or pipelines.

Equivalent actions

Method	Command
CLI	`aicrew benchmark` (planned)
Codex skill	`aicrew-benchmark` (this skill)
Claude Code slash	`/benchmark`

Default output

Caveman/lean style. See ~/Agents/agents/caveman.md.

Source of truth:

~/Agents/commands/harness-audit.md (for harness health)

What to benchmark

Ask the user which dimension to measure:

What should we benchmark?

Skill trigger accuracy — does the right skill fire for a given prompt?

Output quality — does a skill produce the expected sections/format?

Token efficiency — lean vs normal mode token counts for the same task

Pipeline timing — wall-clock time for each /dev or /fix phase

All of the above (full harness benchmark)

Wait for answer.

Steps per benchmark type

1. Skill trigger accuracy

List all skills and their description fields
For each skill, present a sample prompt and confirm the right skill triggers
Flag any ambiguous descriptions

2. Output quality

Run a skill against a standard input
Check required sections are present (e.g. SCOUT: block, HANDOFF: block, phase gates)
Flag missing or malformed sections

3. Token efficiency

Compare output token count with lean vs normal mode
Run the same task in both modes; measure diff

4. Pipeline timing

Record start/end of each phase in /dev or /fix
Output phase timings as a table

5. Full harness

Run all checks above in sequence
Output a summary table: skill → trigger ✓/✗ → quality ✓/✗ → tokens → timing

Output format

BENCHMARK:
Skill/Phase   | Trigger | Quality | Tokens | Timing
--------------|---------|---------|--------|-------
aicrew-dev    | ✓       | ✓       | 1200   | 4.2s
aicrew-fix    | ✓       | ✓       | 800    | 2.1s
...

Checkpoint

After each benchmark type completes: report results and ask "Continue with next type?" before proceeding.