Artificial analysis skill

Skill mrloldev/artificial-analysis-skill

Agent skill that picks the right AI model using live benchmarks from artificialanalysis.ai. Works with Claude Code, Cursor, Codex, and any skills.sh-compatible agent.

Install

npx -y skills add mrloldev/artificial-analysis-skill

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

2 stars2 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Pick the right LLM or media model for a task, backed by live benchmark data from artificialanalysis.ai. Use when the user asks "which model should I use for X", "what's the best/fastest/cheapest model", "compare model A vs B", "model leaderboard", or anything about model intelligence / speed / price / context / coding-or-math ability. Also for text-to-image, image-editing, text-to-speech, text-to-video, and image-to-video rankings. Triggers on "which model", "best model for", "cheapest LLM", "fastest model", "compare models", "model pricing", "artificial analysis", "MMLU score", "model ELO".

SKILL.md

6.7 KB, as published. Nobody here has run it

Artificial Analysis: pick the right model

Live benchmark data from https://artificialanalysis.ai — intelligence index (10 evals), coding index, math index, output speed, time-to-first-token, pricing, and context-window, for ~400+ LLMs plus media-model ELO leaderboards.

Your job: when the user needs a model recommendation, run recommend.sh with the right profile — do not guess from memory. Model rankings change weekly; never answer from training data alone.

Prerequisites

AA_API_KEY must be exported. Free key (1,000 req/day) at https://artificialanalysis.ai/documentation. If missing, scripts prompt on TTY or fail with a clear pointer.
curl and jq installed.

Responses cached 24h at ~/.cache/artificial-analysis/ — repeated calls are free of rate-limit cost.

Decision flow — ALWAYS follow this for "which model" questions

1. Classify the user's need → pick ONE profile (see table below).
2. Extract hard constraints: budget, min context, required provider, required quality floor.
3. Run:  recommend.sh <profile> [--budget $/M] [--min-context N] [--min-intel N] [--creator X] --top 5
4. Report the top pick with its trade-offs vs. #2 and #3.
5. If the user names two or more specific models, follow with: compare.sh <slug> <slug> ...

Do NOT skip step 3. Benchmark numbers change; the API is the source of truth.

Profile selection

User says / implies	Profile	Optimizes for
"best / smartest / most capable model"	`smart`	intelligence_index (tie-break: coding)
no strong steer, general production use	`balanced`	intelligence + price + speed
"fast / low latency / streaming / realtime"	`fast`	output t/s + low TTFT
"cheap / bulk / high-volume / budget"	`cheap`	price (with a sane quality floor)
"coding / code-gen / codegen agent"	`coding`	coding_index
"math / reasoning-heavy / proofs"	`math`	math_index
"long docs / 200k+ / large context"	`long-ctx`	context window, then intelligence
"agent / tool loop / long-running"	`agentic`	intelligence + speed + context

When in doubt, use balanced. If the user has two conflicting hints (e.g. "fast and smart"), pick the dominant one and say so.

Hard constraints → flags

Budget → --budget 5 means ≤ $5/M tokens blended (3:1 input:output).
Min context → --min-context 200000.
Quality floor → --min-intel 60 (intelligence_index is ~0–100-ish).
Specific vendor → --creator anthropic (matches slug or name, case-insensitive substring).

Core scripts

`recommend.sh` — the one you'll use most

# Pick the best general model
"${CLAUDE_SKILL_DIR}/scripts/recommend.sh" balanced --top 5

# Cheapest decent model for batch processing
"${CLAUDE_SKILL_DIR}/scripts/recommend.sh" cheap --min-intel 55

# Fast Anthropic model for streaming UI
"${CLAUDE_SKILL_DIR}/scripts/recommend.sh" fast --creator anthropic

# Smart model that fits a 500k-token document
"${CLAUDE_SKILL_DIR}/scripts/recommend.sh" long-ctx --min-context 500000

# Best coding model under $10/M
"${CLAUDE_SKILL_DIR}/scripts/recommend.sh" coding --budget 10

Output: ranked table with score, slug, creator, intel/code/math, t/s, TTFT, $/M, ctx.

`models.sh` — leaderboard

# Top 10 by raw intelligence
"${CLAUDE_SKILL_DIR}/scripts/models.sh" --top 10 --sort-by evaluations.artificial_analysis_intelligence_index

# Just the slugs (for piping into compare.sh)
"${CLAUDE_SKILL_DIR}/scripts/models.sh" --top 20 --slugs

# Inspect available fields on the first model
"${CLAUDE_SKILL_DIR}/scripts/models.sh" --path

Sort keys containing latency, price, ttft, or time_to_first are treated as lower-is-better automatically.

`compare.sh` — head-to-head

"${CLAUDE_SKILL_DIR}/scripts/compare.sh" claude-4-5-sonnet gpt-5-xhigh gemini-3-1-pro-preview

Prints rows for creator, intel, coding, math, t/s, TTFT, blended / input / output price, and context window.

`media.sh` — image / audio / video leaderboards

"${CLAUDE_SKILL_DIR}/scripts/media.sh" image --top 15
"${CLAUDE_SKILL_DIR}/scripts/media.sh" edit  --top 10
"${CLAUDE_SKILL_DIR}/scripts/media.sh" tts   --top 10
"${CLAUDE_SKILL_DIR}/scripts/media.sh" ttv   --categories --top 10
"${CLAUDE_SKILL_DIR}/scripts/media.sh" i2v   --categories --top 10

Ranked by ELO rating.

`fetch.sh` — raw access (for custom jq)

"${CLAUDE_SKILL_DIR}/scripts/fetch.sh" data/llms/models --raw \
  | jq '[.data[] | select(.context_window >= 1000000)] | length'

Guidance when reporting to the user

Name the top pick by slug, link to its AA page (https://artificialanalysis.ai/models/<slug>).
Justify with the two or three metrics that drove the choice — e.g. "intelligence 72, $3.20/M blended, 190 t/s".
Mention the runner-up in one line with the tradeoff ("If you want lower latency, try X at $/M but intel Y").
Attribute: say "per Artificial Analysis benchmarks" — that's the data source and their ToS expects it.
If slugs are unfamiliar to the user, offer compare.sh as a follow-up.

Failure handling

401 → remind user about AA_API_KEY and link to https://artificialanalysis.ai/documentation.
429 → the script auto-falls-back to stale cache; warn the user it's stale.
Empty filter result → loosen a constraint (usually --budget or --min-intel).
Unknown slug in compare.sh → it warns on stderr; list candidates with models.sh --slugs | grep -i <partial>.

Notes on authentication

The Artificial Analysis API requires an API key. There is no unauthenticated public endpoint — the website renders data server-side through authenticated calls. A free key (1,000 req/day) is explicitly offered for this use case; the skill caches aggressively to stay under that limit.