Performance and cost optimization

Performance and Cost Optimization

Overview

Define what "fast enough" and "cheap enough" mean before optimizing anything. Output is .forge/performance-budget.md — latency budgets per request path, LLM cost budgets per call type, the caching strategy (key schema, TTLs, invalidation), frontend bundle budgets, and the profiling targets. Pairs with observability (the dashboards that measure compliance) and scalability-analysis (the growth projection that decides when to invest).

When to Use

• A new service is being designed and there's no latency or cost target
• An LLM-heavy feature is shipping and per-call cost is unbounded
• The frontend bundle is growing organically and ship time is increasing
• A query or endpoint is approaching its SLO and the team is debating where to optimize
• Caching is being added ad-hoc with no key convention or invalidation story

When NOT to Use

• A one-off script with no user-facing latency
• Pure refactoring or rename work with no perf surface
• "It feels slow" with no measurement — start with observability first

Common Rationalizations

Thought	Reality
"Premature optimization is evil"	Setting budgets isn't optimizing — it's defining done. You can't tell what's premature without a target.
"LLM costs are fine for now"	Unbounded token usage scales linearly with users. The first viral spike turns a $50/day bill into a $5000/day bill overnight.
"We'll cache later"	Cache invalidation designed after the fact creates stale-data bugs you find in production. Design the invalidation strategy with the cache.
"Bundle size doesn't matter, our users have fast internet"	You don't know that. Mobile carriers, hotel wifi, rural backhaul, parking-garage signal — all real.
"Profile when it's slow"	"Slow" is a feeling. Without a budget, you'll profile randomly or never.
"Auto-scaling fixes performance"	Auto-scaling fixes capacity, not per-request latency. A slow request stays slow under any cluster size.

Red Flags

• An endpoint in production with no latency SLO and no dashboard
• An LLM call with no max_tokens, no model selection rationale, and no cost tracking
• A cache layer with no documented TTL or invalidation event
• A frontend route loading >500KB of initial JS
• A backend hot path that has never been profiled
• A "background job" that takes >30s with no chunking or progress reporting
• Optimization work happening before measurement
• An LLM prompt that grows unbounded with conversation history

Core Process

Step 1: Define latency budgets per request type

In .forge/performance-budget.md:

Path type	p95 budget	p99 budget	Source of truth
Interactive UI action (click → visible response)	200ms	500ms	Frontend RUM
Public API read	500ms	1s	Backend traces
Public API write	800ms	2s	Backend traces
Search / aggregation	1s	3s	Backend traces
Background job	30s	5min	Job runner
Email / webhook delivery	5min	30min	Queue worker

Adjust per project, but write the numbers down. Every hot path gets a budget.

Step 2: Set LLM cost budgets per call type

For every LLM call in the system, document:

• Model selection rationale (why Sonnet vs Haiku vs local)
• Input token cap (truncate or summarize context past N tokens)
• Output token cap (max_tokens always set)
• Cost target per call (e.g., "<$0.01 for the autocomplete call, <$0.05 for the summary call")
• Cost-per-user-per-day budget (the rate-limit threshold)
• Caching strategy if the same prompt recurs (prompt cache, response cache, or both)

Append to .forge/performance-budget.md.

Step 3: Design caching strategy

For every cache layer:

• Key schema — explicit, deterministic, includes version (v1:user:{id}:profile)
• TTL — short for frequently-changing data (60s), long for slowly-changing (1h), forever for immutable (with versioned keys)
• Invalidation events — which mutations bust which cache keys, documented as a table
• Negative caching — cache "not found" for short TTL to absorb thundering-herd lookups
• Stale-while-revalidate for read-heavy paths where eventual consistency is acceptable
• Failure mode — what happens when the cache is down (fall through to source, or fail closed)

A cache without a documented invalidation event is a stale-data bug waiting to ship.

Step 4: Set frontend bundle budgets

• Initial JS for any user-facing route: <200KB gzip.
• Initial CSS: <50KB.
• LCP-impacting resources (above-the-fold images): preloaded, properly sized.
• Code-split per route. Lazy-load heavy components (charts, editors, maps).
• Bundle analyzer in CI; budget breach blocks the build.

Step 5: Identify profiling targets

For each service:

• Top 3 endpoints by traffic — profiled with production-shaped data
• Top 3 endpoints by latency — flame graphs captured, reviewed
• Top 3 database queries by cost — EXPLAIN plans attached to .forge/database-design.md
• Any code path inside a setInterval, setTimeout, or render loop

Profile before optimizing. Optimization without a baseline is decoration.

Step 6: Define cost-tracking dashboards (cross-ref observability)

• Per-endpoint cost per request (compute + DB + cache + LLM + egress)
• Cost-per-active-user per day
• LLM spend per feature per day
• Top 10 most-expensive users per week (for usage-based pricing or abuse detection)

Verification

• .forge/performance-budget.md written
• Every hot-path endpoint has a documented p95 and p99 latency budget
• Every LLM call type has model rationale, max_tokens, and cost target
• Every cache layer has documented key schema, TTL, and invalidation events
• Frontend bundle is <200KB initial JS per route (CI-enforced)
• Top traffic and top latency endpoints have been profiled with flame graphs
• Cost-per-user-per-day dashboard exists and has an alert threshold
• No LLM prompt grows unbounded with user input or conversation length