Production agent design

Production Agent Design

A blueprint for building stable, effective production agents. Every principle here earns its place — nothing is optional polish.

How to use this skill

Default mode — passive reference. When the user is actively building, designing, or debugging an agent, surface only the principles relevant to what they're working on. Don't dump the whole blueprint. If they're writing a tool, point them at Section 2. If they're debugging a long-horizon failure, point them at Sections 7 and 9.

Phased mode — active workflow. When the user says "walk me through agent design," "phase this build," "audit my agent against this," or similar, run the explicit phased workflow:

Phase	Sections to apply	Output
1. Design	1, 4	Architecture choice, memory layer plan, planning approach
2. Build	2, 3	Tool specs, context budget, model routing
3. Harden	5, 6	Verifier strategy, security boundaries, guardrails
4. Ship	7, 8, 9, 10	Observability plan, escalation triggers, eval set, decision log, sanity checklist

In phased mode, finish each phase before moving to the next. Surface decisions for the user to confirm at phase boundaries, not at every micro-step.

The shipping checklist at the bottom is always relevant. Run it before any production deploy.

---

1. System Design & Orchestration

• Default to simplicity. Favor systems-level design over model modifications. Production agents overwhelmingly use prompting on off-the-shelf models — fine-tune only when prompting demonstrably fails. Weight tuning is expensive insurance for a problem you probably don't have yet.
• Neurosymbolic by default. Blend deterministic, symbolic kernels (IF-THEN rules, regex/AST matchers, classical planners, validators) with LLM reasoning. Use the LLM for ambiguity. Use the symbolic layer for anything that must be reliable. The LLM is the judgment; the code is the spine.
• Require an explicit planning phase. For any task longer than ~3 steps, the agent produces an inspectable plan before acting. Persist the plan; let it be edited or rolled back. No silent reactive loops on multi-step work — this is the single highest-leverage stability win.
• Plan revision is an explicit event, never silent. Most long-horizon drift is the agent quietly abandoning its plan mid-run. Allow revisions only through an explicit, logged revise_plan action with a stated reason. Two or more revisions in a single run is itself an escalation signal — the agent is confused.
• Fresh-context subtask execution for long jobs. Past ~8 planned steps, don't run one rolling context. An orchestrator holds the plan; each step executes in a fresh context carrying only the goal, its subtask, and needed state from external storage. Drift can't compound across subtasks because context doesn't carry over.
• Discipline subagents. Each subagent carries its own context window — costs multiply fast. Orchestrate sequential steps inside one agent. Spawn subagents only when work is genuinely parallel and isolated (e.g., enriching 50 leads, fanning out research queries). When that fan-out is also latency-insensitive, run it as an async batch job instead — same models, ~50% cost, its own rate-limit pool. Batch is a billing decision, not a quality one; it never belongs in the interactive loop.
• Cap autonomous steps — as a circuit breaker, not a review cadence. Errors compound geometrically (95% per-step reliability ≈ 60% success at 10 steps). Derive the cap from the plan (planned steps × 1.5, floor ~10); exceeding it means something is wrong — halt and surface the trace. Put humans at irreversible actions (staging gates on sends, deletes, charges), not at step counts: same safety, without killing async value. Client-facing outbound stays human-gated regardless.
• Autonomous and unsupervised are different settings — never ship them welded together. Autonomous means the agent takes steps without per-step approval. Unsupervised means nobody reads the output afterward. The first is a design decision; the second is a staffing decision, and it is almost never the right one. Autopilots have flown transatlantic routes for decades with two humans awake in the cockpit, and nobody calls that a failed autopilot. Decide the two independently and write both down: what the agent may do alone, and who reads what, how often.
• Guardrails live in code, not prompts. Building more than one agent? Codify tool validation, caps, staging gates, and verifiers as a shared library/chassis. Prompts drift with edits; code doesn't.
• Multi-tenancy from day one for client-serving agents. Key every state row, memory fact, and trace by client_id, and test cross-tenant isolation explicitly. Retrofitting tenancy is the expensive path.

2. Tool Design

Tools are the agent's API to the world. How they're written shapes behavior more than almost anything else.

• One job per tool. No swiss-army endpoints. get_invoice and update_invoice, not manage_invoice(action=...). The agent reasons better about narrow primitives than wide dispatchers.
• Treat the description as a prompt. Include when to use it, when not to, what the output shape is, and any preconditions. A vague description produces vague behavior.
• Schema-validate inputs and outputs. Reject malformed inputs at the boundary. Return structured data with stable shapes — don't make the agent parse prose.
• Idempotency by default. For destructive or non-idempotent ops (send, delete, charge, post), require an explicit confirmation argument or two-step commit. Make the safe path the default path.
• Error messages that teach recovery. "Field email invalid: expected RFC 5322 format, got user@" beats "400 Bad Request." The agent's next attempt should be informed by your error.
• Cap response size. Truncate, paginate, or summarize before returning. A 50KB tool result eats context budget for the rest of the run.

3. Context Window & Token Management

• Strict tool budget. Keep <10 MCP servers enabled and <80 active tools per project. Every tool description is a permanent context tax.
• Compact at logical breakpoints. After research phases, after milestones — never mid-implementation. Mid-task compaction loses variable state, file paths, and partial progress.
• Externalize state. Anything the agent will reference repeatedly belongs in a scratchpad file, structured note, or DB row — not the rolling context. Treat context as RAM, not disk.
• Route by complexity. Cheap, capable models as the default. Route to expensive models only for deep reasoning, architectural decisions, or hard debugging. Most steps don't need the heaviest model.
• Cap thinking tokens. Hidden inference costs scale silently. Set explicit ceilings per request.

Prompt caching (biggest single cost lever for agents)

Numbers below are Anthropic's; the structural rules generalize to any provider with prefix caching. Cache hits cost 10% of base input; 5-min writes cost 125%, 1-hour writes 200%. An agent loop that replays tools + system + history every step is the exact shape caching was built for.

• Order the prompt for caching: tools → system → messages. That's the cache hierarchy; a change at any level invalidates it and everything after. Static content first, varying content last.
• Put the breakpoint on the last stable block, not the last block. Cache writes happen only at the breakpoint. A breakpoint on a block containing a timestamp or the incoming user message writes a fresh entry every request and never reads. Automatic top-level caching hits this same trap — use an explicit block-level breakpoint when the tail varies.
• Automatic caching for plain multi-turn chat; explicit breakpoints for agents. A top-level cache directive moves the breakpoint forward as the conversation grows. Agents with tools + retrieved context want explicit breakpoints instead.
• Breakpoints are cheap — use them. Only cached/read tokens are billed. Typical agent layout: (1) end of tools, (2) end of static system instructions, (3) end of retrieved/knowledge context, (4) last message. Each segment is reused independently.
• Mind the lookback window. Cache lookups scan a bounded number of recent blocks. A conversation that grows faster than that window silently stops hitting. Add a second breakpoint further back so a write accumulates there before you need it.
• Instrument it. Log cache-read and cache-creation token counts per request. Both zero means you aren't caching — usually the prefix is under the provider's minimum cacheable length. Uncached input tokens count only the tail after the last breakpoint, so total input is the sum of all three. Track hit rate per run; a regression means something upstream started varying.
• Know what breaks it. Tool-definition edits, toggling built-in tools or citations, adding or removing images, changing tool-choice, thinking config, or reasoning-effort settings. Keep these fixed across a run.
• Short TTL by default; long TTL when the gap is longer. A short (minutes) cache refreshes on every hit — fine for tight loops. Use an extended TTL for side-agents that run longer than the default window, batch jobs, or conversations with slow human replies. When mixing TTLs, longer-lived breakpoints must come before shorter-lived ones.
• Batch discounts don't reliably stack with caching. Cache hits are best-effort inside a batch job — the two are separate levers, not compounding ones. Model the savings as one or the other, not both.
• Pre-warm latency-sensitive entrypoints. A zero-output request with the breakpoint on the system/tools prefix writes the cache without generating tokens. Match thinking/effort config to real traffic or you warm an entry nobody hits.

4. Memory Architecture

Be explicit about which memory each piece of information lives in. Conflating these is the source of most "the agent forgot" bugs.

• Short-term: active context window. Volatile, fixed budget. Use for the current task and immediate history.
• Working: scratchpad files, structured notes (e.g., PLAN.md, STATE.json). Survives compaction within a session.
• Long-term: vector store or structured DB. Cross-session knowledge, retrieved on demand.
• Episodic: archived past trajectories. Source for few-shot examples, post-hoc learning, and skill extraction.

Rule of thumb: if it must survive compaction, it doesn't live in context.

5. Verification & Pre-Action Checks

• Critic/verifier step before irreversible actions. A second pass — same model or a smaller dedicated one — validates the proposed action against the goal and known constraints. Cheap; catches a surprising number of mistakes.
• Confidence thresholds gating execution. Below threshold → escalate to human or trigger a clarifying step. Don't let the agent guess on low confidence.
• Dry-run mode for high-stakes operations. Show the diff, plan, or payload to the user (or a logging layer) before commit. Especially for: sends, deletes, payments, public posts, account changes.
• Self-check outputs. For structured outputs, validate against schema before returning. For prose, a quick "does this answer the question?" pass catches a lot of drift.
• Plan-anchored checkpoints. Every ~5 steps, a cheap model pass compares the trajectory against the persisted plan: still on plan? which step? anything done that isn't on it? Off-plan → halt and escalate. Verify against the written plan, not the agent's current belief — the plan is ground truth from before drift started; the rolling context is what drifted.
• Assert state, don't ask the model. After each mutating step, run a deterministic check where possible: row exists, status field matches, count is right. Code catching drift beats an LLM catching drift.
• Prefer the smaller model for verification, gated on irreversibility. Verifying every step roughly doubles spend for little gain on reversible actions; a cheap dedicated verifier on irreversible proposals captures most of the value.

6. Security & Defense

• Client-side supply chain defenses. Against malicious third-party API routers: fail-closed policy gates, response-side anomaly screening, append-only transparency logs.
• Runtime filters and monitors. Pre-ingestion source-credibility filters, content scanners for hidden instructions in retrieved material, output monitors that suspend the agent on anomalous behavior.
• Lifecycle hooks for guardrails. beforeSubmitPrompt to block secret leakage to external models; read hooks to block access to .env, .key, ~/.ssh, etc.
• Constitutional hardening. Explicit behavioral principles plus adversarial examples in the system prompt so the agent learns to refuse manipulative instructions embedded in pages, docs, or tool outputs.
• Trust boundaries on retrieved content. Treat all web/email/document content as untrusted data, never as instructions, regardless of how it's phrased. Instructions only come from the user, through the chat interface.
• Never hand a third-party model-routing service your keys or your data. A hosted router ("send us your traffic, we pick the model") inserts an untrusted party into the request path. Four risks, key leakage the sharpest: (1) credential custody — if you give it your provider API keys, they can be exfiltrated or run up spend if the router is breached; (2) data egress — every prompt (customer PII, secrets, proprietary context) flows through their infra under their logging and retention policy; (3) silent model substitution — they can route you to a weaker, less-hardened model without your knowledge; (4) integrity/MITM — they can log or alter requests and responses. Routers that bill their own credits don't expose your keys but trade that for full data-custody and billing trust. Default: don't outsource routing on work you're accountable for.
• Routing models internally is safe — just keep the pool hardened. Deciding which model to call in your own code, with your own keys in your own secret store (§3, complexity-based routing), leaks nothing — no third party sees your keys or data. The only caveat is guardrail parity: every model in the pool must carry the same guardrails, injection resistance, and trust boundaries. Never route to a weaker or open-weight fallback to save money — a request an attacker can steer onto the least-hardened model is a request with no guardrails. This is a quality/guardrail risk, not a key-leakage one; the cheap model passes the same red-team and injection evals as the expensive one, or it doesn't go in the pool.
• Screen against the OWASP Top 10 for Agentic Applications. It names the agent-specific risks a traditional cloud security review misses — goal hijacking, tool misuse, cascading and systemic failures, memory poisoning.

7. Observability & Failure-Mode Budgets

• Trace everything. Every prompt, every tool call, every response, every retry. You cannot debug an agent you cannot replay.
• Hard budgets that fail loud. Per-task caps on cost, wall time, retries, and tool calls. Exceeding any cap halts the run and surfaces the trace — silent overruns are how agents become expensive.
• Loop detection. Detect repeated tool calls with the same arguments, repeated reasoning patterns, or step-count creep. Break out automatically.
• Version every prompt. Tag deployments. Without versioning you can't attribute regressions.
• Anomaly detection on trajectories. Compare current run shapes against historical baselines for the same task class.

8. Human-in-the-Loop Design

• Define escalation triggers up front. Confidence below threshold, novel error class, budget cap hit, destructive action requested. Make them explicit, not vibes.
• Surface decisions, not raw reasoning. Humans should review what's about to happen, not wade through chain-of-thought.
• Cheap course-correction. One-click approve/reject/edit. If correcting the agent is more expensive than doing the task manually, the agent has negative ROI.
• Default to reversible. Where possible, agents take reversible actions (drafts, staged changes, soft deletes) and humans commit.

9. Evaluation & Continuous Improvement

• Eval-first: golden tasks are the spec. Write ~10 golden tasks before building the agent — including at least one prompt-injection case. They define scope, give a builder (human or model) an objective "done," and become the regression set for free. Gate promotion to autonomous on passing them across two consecutive prompt versions.
• Trajectory-grounded LLM-as-Judge for scalable failure attribution on long-horizon, interdependent tasks. Generic per-turn evals miss compounding errors.
• Safe live testing. When benchmarking on real production sites, preserve full dynamic complexity but use a lightweight interception layer to block the final commit (submit, send, charge). Real environment, no real consequences.
• Auto-extract winning patterns. Mine successful sessions for reusable skills, instincts, or templates. Promote them into the agent's standard toolkit.
• Regression evals on every prompt change. A small set of golden trajectories that must pass before deploy. Cheap insurance against silent drift.
• Run eval and judge passes as batch jobs. Golden tasks, regression sets, and LLM-as-judge grading are large, offline, and latency-insensitive — the exact shape async batching was built for, at roughly half the cost. That discount is what keeps "evals on every prompt change" a habit instead of an aspiration. Grade against the same model config you ship; a cheaper batch model as judge is a different evaluator.
• Separate task success from process quality. An agent that succeeds via 40 tool calls when 4 would do is failing — just slowly. Track both.
• Read raw output on a schedule — the control that finds what you didn't predict. Every technique above checks the agent against cases you already thought of. None of them find the failure class you never imagined. Pull 15–20 random production trajectories a week and read input and output together, end to end. Never the agent's own label, status, or summary — those are the thing under test. Budget under an hour. Do it even when the dashboard is green, especially when the dashboard is green: an agent that confidently does the wrong thing reports success, so a 100% pass rate is exactly the condition under which sampling has the most to tell you.
• "Handled" is not "handled correctly." Containment, deflection, and completion rates all measure whether the agent acted, not whether it acted rightly. A deterministic post-check confirms the email sent; it cannot confirm the email should have been sent. Any metric that a wrong-but-executed action still increments is a metric that will stay green through an incident.

10. Decision Logs (the anti-black-box layer)

§7 traces are for you: full, noisy, replayable, private. A decision log is the second, narrower layer built for someone else to read — a customer, an auditor, a regulator, a teammate three months from now. Most agent projects stall on trust, not capability. This is the artifact that unsticks them.

Reference spec, JSON Schema, adapters, and a retrofit guide for existing agents live in reference/.

Three normative rules. Everything else is implementation detail.

1. Log at tool boundaries, never chain-of-thought. The tool-call payload is the decision. Reasoning traces are unfaithful to actual behavior, and publishing them creates a liability you cannot defend when they contradict the outcome. Never show a customer raw reasoning.
2. Capture action before execution; derive outcome from a deterministic check. Never ask the model to narrate what it did after the fact — that is post-hoc rationalization, not evidence. Row exists, status matches, event ID returned.
3. Append-only. Corrections are new records pointing at the old one. Enforce it at the database GRANT level, not by convention — a log your own service role can rewrite is not an audit trail. Revoke TRUNCATE alongside UPDATE and DELETE: it is a separate privilege, it is not gated by row-level security, and it empties the whole log in one statement — leaving it granted is the most common way this control is silently defeated. Then prove the control by attempting a write as the application's role; table owners are never bound by grants, so testing as the owner always falsely succeeds.

Record the restraint, not just the actions. A dedicated field for what the agent declined to do and which rule stopped it ("did not send SMS — no opt-in on file") is the single most persuasive element of the whole log. A wall of successes reads as marketing; a documented refusal reads as control. Design agents to emit it explicitly — it is a prompt and tool-contract change, not a schema change, and it is the part builds usually skip.

Core fields. run_id · step · ts · agent · prompt_version · trigger · inputs_cited (source IDs, not prose — crm:person/123, kb:hours.md#L4) · decision (one sentence) · action (tool + args) · reversible · gates (rule name + pass/fail) · not_done · outcome (verified state) · human (auto / approved / escalated) · cost · latency · model.

Keep tenancy outside the envelope. The record describes one decision; the host system owns client_id, auth, and row-level security. That split is what makes the same log format portable across projects and safe to publish.

Minimize at write time, don't rely on deletion. Argument payloads are the highest-PII surface in the whole system. Redact known-sensitive keys on the way in and store a hash if you need matching. A retention job is a backstop, not a strategy — backups and point-in-time-recovery windows outlive your stated deletion policy, so the honest claim is "operational retention," and the strong control is never having written the value.

Know what the log does not do. It makes behavior legible, not self-policing. outcome.verified confirms the action executed and the state changed as claimed — it says nothing about whether the action was the right one. An agent that misreads intent and confidently sends the wrong template produces a record that verifies clean, with no failed gate and an empty not_done. The log is what lets a human spot that in seconds instead of hours; catching it automatically is a §1 gate, a §5 verifier, or a §9 golden task. Don't let a well-instrumented log substitute for any of the three.

Surface it, or it doesn't count. An unread log is a cost center. Give it a read-only stakeholder-facing view filtered to three things — escalated, declined, human-approved — plus a periodic digest of five numbers. The digest is what gets read; the view is proof it exists.

---

Field notes

Patterns that hold up in production — reference material, not a rewrite of the sections above.

• Agentic RAG = an agent/tool router, not a model router. The production pattern is four layers: (1) an orchestrator agent decomposes the goal and plans retrieval; (2) a retrieval tool suite it chooses among — semantic vector search, keyword/BM25 for exact codes and IDs, graph traversal for relationships, live API calls; (3) an evaluate-and-reflect loop that checks retrieved context for groundedness and reformulates if it fails; (4) structured, cited generation grounded only in verified context. The routing decision is which tool/data source, made by the agent — this is the approved shape. It is categorically different from a model router picking which LLM answers (see §6 — that's the attack surface).
• Six-stage agent lifecycle for anything customer-facing. Requirement Approval (map the business metric, name a human owner, define the autonomy boundary) → Guarded Build (secure containers, least-privilege API scopes) → Human Sign-Off (validate prompt logic and tool-routing rules) → Automated Deployment (provision infra, inject secrets) → Real-Time Monitoring (input drift, unexpected API calls, action frequency) → Structured Retirement (decommission cleanly, revoke data and API permissions). The two most-skipped gates — named owner at the start, clean permission teardown at the end — are where incidents originate.
• Knowledge-graph gate. A graph/knowledge-graph layer is usually the wrong choice for a simple setup — few distinct data systems, no recurring structural or relationship questions, or immature entity resolution. (Rules of thumb that circulate — roughly <8 source systems or <85% entity-resolution accuracy — are directional heuristics, not validated thresholds; treat them as a smell test, not a spec.) Traversing a poorly-governed graph infers false relationships and produces confident hallucinations at scale. Default to vector + BM25 until there are genuine cross-system relationship questions.
• Cost is a trap metric. Klarna automated roughly 700 agents' worth of volume (2.3M conversations/month, handle time 11 min → under 2 min), then reversed course in 2025 and rehired humans after service quality degraded. Optimizing containment and cost over quality builds a hidden satisfaction deficit that surfaces as churn later. Bake quality into evals, and keep the human boundary at the emotionally-nuanced, high-value slice of interactions — don't chase 100% deflection.

---

Pre-Ship Sanity Checklist

Run this before any production deploy. If any answer is "no" or "I'll get to it later," you have homework.

• Planning phase exists for multi-step tasks
• Every tool has schema validation on inputs and outputs
• Hard caps on cost, wall time, retries, and step count
• Verifier or dry-run before any irreversible action
• Full trace logging with prompt versioning
• Escalation triggers defined and tested
• Constitutional principles in the system prompt
• Regression eval set passes
• Trust boundaries enforced on all retrieved content
• Memory layer assignments explicit and documented
• Tool budget under ~80 active tools, MCP servers under ~10
• Plan-anchored checkpoint runs every ~5 steps; plan revisions logged, never silent
• Golden tasks written before build, passing (including injection case)
• Prompt caching wired: breakpoint on last stable block, cache hit rate logged and non-zero
• Decision log emitting at every tool boundary, append-only enforced at the DB grant level
• Agents populate not_done when a gate blocks them — verified in the golden tasks
• PII redaction on argument payloads happens at write time, retention job is a backstop only
• Autonomy boundary and review cadence written down separately — what runs alone, and who reads what, how often
• Weekly raw-output sampling scheduled with a named owner, not just dashboards
• Cross-tenant isolation tested (client A cannot read client B), if multi-client
• Cap breach forced in staging and observed to halt loudly

---

How to apply this skill — examples

User says: "Help me design a tool for my agent to update CRM records." → Surface Section 2 (Tool Design). Walk through single-purpose, schema validation, idempotency, error messages. Don't dump the rest.

User says: "My agent keeps looping on the same call. What's wrong?" → Surface Section 7 (loop detection, hard budgets) and Section 5 (verifier step). Diagnose, then fix.

User says: "Walk me through designing this voice agent for a client." → Switch to phased mode. Run Design → Build → Harden → Ship with the user, confirming at phase boundaries.

User says: "Audit this automation workflow against best practices." → Run the Pre-Ship Checklist against the workflow. Surface specific section guidance for any failed item.

---

Built by Super Logic AI — production AI automation. superlogicai.com