Api spend recovery

API Spend Recovery

Use this when the team uses hosted APIs, coding agents, chat/RAG products, voice APIs, media APIs, or managed model platforms.

You do not need to own GPUs to have an inference optimization problem.

Common Leaks

1. Premium model overuse.
2. Prompt and context bloat.
3. Duplicate RAG chunks.
4. Repeated tool calls.
5. Failed retries.
6. Weak routing.
7. Missing exact or semantic cache.
8. Human rework from bad outputs.
9. Rejected media renders.
10. Unresolved voice calls.

Fix Order

Prefer low-risk changes first:

1. Measure cost per successful task.
2. Add route labels: easy, hard, final, fallback, human.
3. Stabilize prompt prefixes.
4. Dedupe retrieval.
5. Cache safe tool results.
6. Add retry budgets.
7. Route drafts to cheaper paths.
8. Use premium models only where success requires them.
9. Replay before/after traces.

Coding-Agent Pattern

Before: broad search, full files, noisy logs, premium model for every step, repeated tests.

After: context map, targeted reads, cheap scout, premium patch/review only when needed, test budget, tool-result cache, evidence packet.

Chat/RAG Pattern

Before: full chat history, broad chunks, duplicate docs, premium answer model for everything.

After: intent classification, stable policy prefix, dedupe, rerank, cache, route, structured output, replay.

Voice Pattern

Before: one premium realtime path for every caller.

After: endpointing, short tool schemas, FAQ route, cached static TTS, semantic trigger, human handoff, per-turn trace.

Media Pattern

Before: final-quality render on first attempt.

After: classify, draft, cache, evaluate, human approve, final render only after direction is accepted.

Output

Write an OPTIMIZATION_RECIPE.md with:

• baseline;
• top leak;
• one first fix;
• verification plan;
• rollback;
• next test.

Do not claim a savings percentage unless traces prove it.