Agent llm

Agent & LLM Patterns (Claude)

Guidance for writing application code that calls Claude well. This is about how to structure the integration, not the API surface.

Before writing any code that names a Claude model, calls the Anthropic SDK, or picks a model tier, load the claude-api skill — it has current model IDs, pricing, limits, and parameter details. Do not hardcode model IDs or pricing from memory; they go stale.

Prompt structure

• Put stable, cacheable content first (system prompt, tool defs, long context); put the variable turn last. This maximizes prompt-cache hits.
• Be explicit about the output contract. If you need JSON, say the exact shape and provide one example. Don't ask for "JSON" and hope.
• Give the model an escape hatch ("if you cannot determine X, return null") so it stops fabricating to satisfy the format.
• Separate instructions from data with clear delimiters (XML tags work well). Never interpolate untrusted user text directly into an instruction sentence — treat it as data inside a tag.

Tool design

• One tool = one capability with a crisp verb name. Overlapping tools cause the model to dither.
• The description is the prompt the model reads to decide when to call — write it for that decision, not as API docs. State when to use it and when NOT to.
• Make schemas strict: enums over free strings, required fields marked, no ambiguous optional soup. A tight schema removes a class of retries.
• Return errors the model can act on. {"error":"row not found","hint":"check the id"} beats a stack trace. The model reads tool results as its next observation.
• Prefer returning structured data over prose from a tool; let the model narrate.

Agentic loops

• The loop is: model → tool_use → you execute → tool_result → model, until it stops calling tools. Keep executing until there are no more tool calls, then surface the final text.
• Bound the loop (max iterations / token budget) so a confused model can't spin forever. Log each step's tool + args for debuggability.
• For structured final output, force a terminal tool call (a submit/finish tool with the result schema) rather than parsing prose — validation happens at the tool layer and the model retries on mismatch.
• Isolate side-effecting tools (writes, sends, deletes) behind confirmation or a dry-run flag when the loop is autonomous.

Context management

• Don't dump whole files/tables into context — retrieve the relevant slice. More context is slower, costlier, and dilutes attention.
• For long-running agents, summarize prior steps into a compact running state rather than carrying the full transcript.
• RAG: retrieve, then cite. Have the model quote the source span it used so answers are checkable; return null/"not found" when retrieval is empty instead of guessing.

Prompt caching

• Cache the system prompt, tool definitions, and any large shared context (5-minute TTL). Structure calls so the cached prefix is byte-identical across requests.
• A single changed byte early in the prompt busts the whole downstream cache — keep volatile content (timestamps, user turn) at the end.

Evals — don't ship a prompt you haven't measured

• Build a small labeled set of real inputs with expected outputs before tuning the prompt. "It looked good in the playground" is not a signal.
• Use an LLM-judge only for fuzzy criteria; for anything checkable (JSON valid? field present? number in range?) assert it in code.
• Track a regression set. When you change the prompt or model, re-run it — prompt changes have non-local effects.
• Measure cost and latency alongside quality; the cheapest model that passes the eval wins.

Common failure modes

• Refusals / cutoffs / streaming / tool-call bugs: load claude-api and check the model's limits and the exact request shape before assuming a logic bug.
• Truncated output: the response hit max_tokens — raise it or ask for less, don't retry blindly.
• Model ignores the format: your instruction and your example disagree, or the format is buried mid-prompt. Move it to the end and make the example match exactly.
• Non-determinism in tests: set temperature: 0 for eval runs; assert on structure/invariants, not exact wording.