Reduce llm cost
Skill ContextJet-ai/awesome-llm-observability/skills/reduce-llm-cost
Use this to cut the cost of an LLM app using observability data. Trigger on "my OpenAI/Anthropic bill is too high", "reduce token usage", "the app is expensive", "optimize LLM cost", "why am I spending so much on the API". Find the expensive spans first (measure), then apply the cheapest wins. Don't guess - the trace tells you where the money goes.From its SKILL.md
npx -y skills add ContextJet-ai/awesome-llm-observability --skill reduce-llm-costAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
What its file declares
Copied from the file, not written here
The file declares its own license as CC0-1.0. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.
SKILL.md
3.0 KB, 636 tokens by cl100k_base, as published. Nobody here has run it
Reduce LLM cost (measure first)
Most LLM bills are dominated by a few patterns you can see in traces. Measure before optimizing - the biggest cost is rarely where people assume.
Step 1 - find where the money goes
From your observability tool, sort spans by cost (or input_tokens). You're looking for:
- The highest-token spans - usually bloated context or a whole chat history re-sent every turn.
- Retry storms - the same call repeated N times (rate limits / transient errors) multiplying cost.
- The most-frequent call × its per-call cost - a cheap call made 10,000×/day beats one expensive call.
- Model overkill - using a frontier model for a task a small/cheap model handles fine.
If you have no cost data yet, add tracing first (see instrument-llm-observability) - you can't optimize what you can't see.
Step 2 - apply wins, cheapest-effort first
- Right-size the model. Route easy calls (classification, extraction, routing) to a small/cheap model; reserve the frontier model for hard reasoning. Biggest lever for most apps.
- Trim the context. Stop re-sending the full history/system prompt every turn. Send only what's needed; summarize old turns. For RAG, retrieve fewer/better chunks, not more.
- Cache. Enable prompt caching (Anthropic/OpenAI) for stable prefixes; cache identical requests (a gateway like Helicone/Portkey/LiteLLM does this for free).
- Cap
max_tokens. Unbounded outputs cost unbounded money; set a sane ceiling. - Fix retry storms. Cap retries + honor
Retry-After; a backoff bug can 10× cost silently. - Batch / async where the API supports it (batch endpoints are often ~50% cheaper).
- Shorten prompts. Few-shot examples and verbose instructions are pure input-token cost - trim to what actually changes behavior (measure with an eval so quality holds).
Step 3 - protect quality while cutting
Every cost cut is a potential quality regression. Gate changes with an eval suite (see add-llm-evals): make the cheap change, run evals, keep it only if quality holds. Then watch cost + quality together on a dashboard so a future change doesn't silently trade one for the other.
Quick math to prioritize
monthly_cost_of_a_span = per_call_tokens × price × calls_per_month. Optimize the span with the biggest product, not the one that looks expensive per call.
Anti-patterns
- Optimizing the model choice while ignoring a 20k-token context that's the real cost.
- Cutting cost with no eval → shipping a cheaper, worse app you find out about from users.
- Turning off logging "to save money" (observability cost is tiny vs the model bill it helps you cut).
What ships with it
Read from the repository
Just SKILL.md. No reference files, no scripts.