Analyze model
A curated collection of autonomous AI agent skills powered by Vercel's `skills` npm cli package. Specialized tools for productivity, document management, and custom LLM workflows.
npx -y skills add arendon1/agent-skills --skill analyze-modelAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
2 things to look at
- no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Fetches live model data from OpenRouter and Artificial Analysis APIs, merges benchmarks with pricing, runs deep cost analysis on usage logs, exports usage from LLM surfaces (OpenCode, Pi, VS Code, etc.), analyzes subscription efficiency, and forecasts future spend with cheaper-model alternatives. Use when comparing LLM costs or quality, analyzing total API spend from usage logs, exporting agent usage for cost analysis, evaluating subscription cost efficiency, forecasting future LLM costs, finding cheaper alternatives, or when the user asks about model pricing, intelligence index, speed, or token costs across providers.
SKILL.md
8.4 KB, as published. Nobody here has run it
analyze-model
Queries OpenRouter and Artificial Analysis APIs to build a live model catalog, then uses that catalog as the source of truth for cost analysis and usage forecasting based on real usage logs.
Authentication Setup
API keys resolved in this order for each provider:
.envfile in the current workspace- Shell environment variable
| Provider | Variable | Required? |
|---|---|---|
| OpenRouter | OPENROUTER_API_KEY | Optional (public models endpoint works without it) |
| Artificial Analysis | ARTIFICIAL_ANALYSIS_API_KEY | Required |
Workflows
analyze-model fetch-catalog
Fetches and merges model data from both APIs into a local catalog JSON.
script: fetch_models.py
python scripts/fetch_models.py --output catalog.json
# OpenRouter only:
python scripts/fetch_models.py --or-only --output catalog.json
# Artificial Analysis only:
python scripts/fetch_models.py --aa-only --output catalog.json
Output fields per model:
| Field | Source | Description |
|---|---|---|
id | OpenRouter | Unique model ID (provider/slug) |
name | OpenRouter | Human-readable name |
context_length | OpenRouter | Max context window (tokens) |
pricing.prompt | OpenRouter | USD per input token |
pricing.completion | OpenRouter | USD per output token |
aa.evaluations.artificial_analysis_intelligence_index | AA | Overall intelligence score |
aa.evaluations.artificial_analysis_coding_index | AA | Coding benchmark |
aa.median_output_tokens_per_second | AA | Generation speed |
aa.median_time_to_first_token_seconds | AA | Latency |
analyze-model analyze-costs
Reads a usage log file and the catalog, then produces a cost breakdown by
model with totals, percentages, and per-call averages. Resolves private-provider
model IDs (e.g. opencode-go/*) to OpenRouter equivalents via references/aliases.json.
Computes subscription efficiency for token-budget providers by comparing
subscription cost vs. OpenRouter pay-per-token equivalent.
Agent pre-flight: Before running, check if subscription pricing is fresh:
python scripts/fetch_subscriptions.py
If the exit code is 1, dispatch subagents to research current pricing, then
write references/subscriptions.json. Proceed once cache is fresh.
script: analyze_costs.py
python scripts/analyze_costs.py usage.json --catalog catalog.json
python scripts/analyze_costs.py usage.csv --catalog catalog.json --output report.json
Usage log format — see references/usage-format.md
Output includes:
- Total USD spent
- Calls, input tokens, output tokens (global + per model)
- Cost % share per model (sorted by spend) — rendered as horizontal bars
- Unknown models that could not be priced
- Subscription efficiency — monthly-bucketed with per-provider cap logic
- Monthly sparklines — cap usage trends per provider
- Model jumping detection — sessions with model switches flagged
- Alias resolution — private models matched to OpenRouter catalog
analyze-model forecast
Projects future costs over a configurable horizon (default 30 days) using linear trend extrapolation from historical usage. Includes cheaper-model alternative recommendations.
script: forecast.py
python scripts/forecast.py usage.json --catalog catalog.json --days 30
python scripts/forecast.py usage.csv --catalog catalog.json --days 90 --output forecast.json
Output includes:
- Projected total cost, calls, tokens for the horizon period (up to 365 days)
- Per-model projected cost with usage share — rendered as horizontal bars
- Multi-baseline projections — trend, worst-case, and average with comparison bars
- Up to 3 cheaper alternatives per top-cost model (requires
aaenrichment in catalog)
analyze-model export-usage
Exports usage data from an LLM surface into the standard usage log format that
analyze_costs.py and forecast.py consume.
script: export_usage.py
python scripts/export_usage.py --source opencode --output usage.json
python scripts/export_usage.py --source pi --output usage.json
python scripts/export_usage.py # interactive: pick a source
python scripts/export_usage.py --list-sources # show available bridges
python scripts/export_usage.py --list-all # include planned sources
How agents should use this: When a user asks about their LLM usage, costs,
or forecasting, prompt them to select a source. Ask: "Which LLM surface are you
using? I can extract from any supported surface (see --list-sources)." Then run the dispatcher with --source.
Typical Analysis Pipeline
# Step 0 (agent): refresh subscription pricing if stale
python scripts/fetch_subscriptions.py
# Step 1: Build the catalog
python scripts/fetch_models.py --output catalog.json
# Step 2: Export your usage from an LLM surface
python scripts/export_usage.py --source opencode --output usage.json
python scripts/export_usage.py --source pi --output usage.json
# (or manually provide a usage log — see references/usage-format.md)
# Step 3: Cost breakdown (now with aliases + subscription efficiency)
python scripts/analyze_costs.py usage.json --catalog catalog.json --output cost_report.json
# Step 4: 30-day forecast + alternatives
python scripts/forecast.py usage.json --catalog catalog.json --days 30 --output forecast.json
Quick Reference
| Action | Command |
|---|---|
| Fetch catalog | python scripts/fetch_models.py --output catalog.json |
| Analyze costs | python scripts/analyze_costs.py <usage> --catalog catalog.json |
| 30-day forecast | python scripts/forecast.py <usage> --catalog catalog.json --days 30 |
| OpenRouter only | python scripts/fetch_models.py --or-only --output catalog.json |
| Export usage (opencode) | python scripts/export_usage.py --source opencode --output usage.json |
| Export usage | python scripts/export_usage.py --source pi --output usage.json |
| List export sources | python scripts/export_usage.py --list-sources |
Error Handling
| Code | Meaning | Action |
|---|---|---|
| 401 | Invalid/missing API key | Check .env or env variable |
| 429 | Rate limit (AA: 1000 req/day) | Wait, cache responses locally |
| 500 | Server error | Retry up to 3x with backoff |
References
| Topic | File |
|---|---|
| OpenRouter API schema | references/openrouter-api.md |
| Artificial Analysis API schema | references/artificialanalysis-api.md |
| Usage log format spec | references/usage-format.md |
| Model aliases | references/aliases.json |
| Subscription pricing | references/subscriptions.json |
Scripts
script: export_usage.py — Dispatcher for usage extraction from LLM surfaces.
script: fetch_subscriptions.py — Cache-gate for subscription pricing (30-day TTL).
script: chart_utils.py — ASCII chart rendering (horizontal bars, sparklines, gauges).
script: bridges/opencode.py — SQLite usage bridge for that surface.
script: bridges/pi.py — JSONL usage bridge for that surface (per-message usage + logged cost).
script: bridges/base.py — Bridge protocol definition.
script: client_openrouter.py — OpenRouter HTTP client (models endpoint).
script: client_aa.py — Artificial Analysis HTTP client (LLMs endpoint).
script: fetch_models.py — Fetches from both APIs and merges into a unified catalog.
script: analyze_costs.py — Cost analysis engine; accepts JSON or CSV usage logs.
script: forecast.py — Linear-trend forecasting + cheaper-alternative finder.