Agent architecture design

Agent Architecture Design Patterns

These patterns address the fundamental configuration challenges that determine whether your agent will succeed or stall. They come from production experience across dozens of agent deployments.

Core Principle

Teams struggle not because an agent can't handle their use cases, but because they didn't break down the problem in a way that maps to buildable systems.

Pattern 1: Whiteboard Agent Capabilities

Problem: Agent feature overload

You have dozens of processes to automate. Where do you start?

Solution: Organizational design for your agents

Think of it like hiring a team. Apply the same organizational design:

1. Write down everything you want your agent to do. Keep asking "What are we missing?"
2. Group similar capabilities together:
   • Pulling from the same data sources
   • Could be performed by the same job title
   • Returned by the same API call
3. Figure out the natural divisions:
   • Different responsible departments
   • Type of task (data fetching vs. synthesis vs. triggering actions)
   • Different steps in a business process
4. Group related capabilities into agents

By the end, you have a list of agents with tools, rank-ordered by priority.

Example

A sales agent breaks into: support agent + sales agent (with 3 subagents: customer discovery, account synthesis, next steps). Each has a focused toolset and clear success criteria.

Pattern 2: Evolve Your Agent Architecture

Problem: Monolithic mega-agents

As you add more tasks, you end up with a mega-agent that handles everything poorly. The likelihood of choosing the wrong tool goes up with the number of tools.

Solution: Discover your architecture iteratively

Do NOT design the final architecture upfront. Instead:

1. List the tasks you want your agent to perform
2. Start with the one burning problem
3. Build that agent really well
4. Notice what users ask for next
5. If it's separate, build a new agent
6. If your agent becomes unwieldy, split it
7. If you have multiple agents, add routing logic
8. Repeat

Each specialized agent has a cohesive toolchain, focused on a specific domain, with clear success criteria.

Key Insight

The magic is that each agent is really good at its format. You never build a "master content agent" that writes mediocre everything. You discover the architecture by solving one problem at a time.

Architecture Evolution Pattern

Single Agent --> Split into Specialists --> Add Router --> Add Coordinator

Example flow: LinkedIn Agent --> + Social Media Agent --> Router Agent --> Content Coordinator --> Router --> Specialists

Pattern 3: Dynamic Agents

Problem: User realities change constantly

Different users need different agent behaviors. You can't create separate agents for every user type.

Solution: Agents that adapt at runtime

A dynamic agent adjusts its configuration based on runtime signals:

• System prompt: Adjust instructions based on user role or context
• Tools: Enable/disable tools based on user tier or permissions
• Memory: Vary retrieval depth (e.g., topK = 8 for free, topK = 15 for enterprise)
• Model: Use different models for different user tiers or task complexity

Example: User tier differentiation

• Free tier: basic support with documentation links
• Pro: detailed technical support
• Enterprise: priority escalation to humans with custom solutions

Trade-offs

Reduces redundancy and increases customization, but introduces complexity in logic, testing, and consistency.

Pattern 4: Human-in-the-Loop (HITL)

Problem: Full agent autonomy is often untenable

Agent performance is heterogeneous -- it performs well on some tasks, poorly on others. Legal, regulatory, and ethical considerations require human oversight.

Solution: Agents and humans taking turns

Three HITL patterns:

1. In-the-loop (human provides context to agent)

   • Agent pauses mid-execution for human input
   • Used for: decisions, clarifications, approvals
   • Example: verifying a financial transaction

2. Post-processing (agent presents draft to human)

   • Human reviews, approves, or revises agent output before it's finalized
   • Used for: editing emails, reviewing legal/healthcare outputs
   • Example: generated email reviewed before sending

3. Deferred tool execution (async feedback)

   • Agent collects human feedback asynchronously without pausing
   • Best aligned with real workflows -- humans don't babysit agents step-by-step
   • Example: agent pushes PR to GitHub, asks for feedback, continues other work

Key Challenge

In almost any HITL architecture, humans become the bottleneck. Design around this -- agents don't sleep, but humans do.

Decision Framework

Situation	Recommended Pattern
Starting a new agent project	Whiteboard Capabilities first
Agent trying to do too much	Evolve Architecture (split it)
Same agent, different users	Dynamic Agents
High-risk or regulated domain	Human-in-the-Loop
Agent accuracy varies by task	HITL for weak areas, autonomy for strong

Gotchas

• Don't design the final multi-agent architecture upfront. Start with one agent, iterate.
• When splitting agents, each specialist should have a cohesive toolset -- don't split arbitrarily.
• Dynamic agent complexity grows fast. Test each configuration variant independently.
• For HITL, design the human review UX carefully -- SMEs won't use a bad interface.
• Deferred tool execution is the most production-friendly HITL pattern, but hardest to implement.

For detailed examples and case studies, see references/architecture-examples.md.