Start with one agent unless a real boundary or measurable workload gives you a reason to split the system.
A multi-agent design can separate permissions, contexts, teams, and specialist work. It also creates handoffs, duplicated context, more credentials, more traces, and more ways for an error to travel. The second agent should therefore solve a named problem that better tools, retrieval, prompts, or deterministic workflow logic cannot solve cleanly.
The architecture decision is not about how complicated the business process sounds. It is about where responsibility and state need to live.
One agent can still use many tools
A single-agent system does not mean one prompt with no structure. One agent can call databases, search services, calculators, workflow APIs, and approval tools. Deterministic code can route steps around it. Human review can pause protected actions.
That design has one main context and one decision trace. It is usually easier to test because the team can follow a request from input to result without reconstructing several conversations.
Use one agent when the domain is bounded, the same permission envelope can serve the work, and the steps benefit from shared context. The agent may be sophisticated. The coordination model remains simple.
Multi-agent architecture buys separation
The strongest reasons to use multiple agents are not theatrical roles such as “researcher,” “critic,” and “writer.” They are boundaries the system should enforce or workload shapes it should exploit.
Microsoft’s current architecture guidance recommends considering multiple agents when work crosses security or compliance boundaries, involves independently owned domains, or needs modular separation. It also warns that handoffs add latency, state management, monitoring, and debugging work. Microsoft’s single-versus-multi-agent guidance is explicit about that tradeoff.
Four boundaries can justify the split:
- Different data or action permissions must remain isolated.
- Contexts are large, unrelated, or harmful when mixed.
- Teams need independent ownership and release cycles.
- Work can run in parallel and the measured gain exceeds coordination overhead.
If none applies, extra agents may be labels wrapped around one workflow rather than necessary architecture.
Coordination has a bill
Every handoff needs a contract. The sending agent must state what it knows, what it wants, and what the receiver may do. The receiving agent must return a result the next step can interpret. Someone must own shared state when the two disagree.
That creates costs beyond model calls:
| Coordination cost | What the team must design |
|---|---|
| Context transfer | Which facts, sources, constraints, and versions cross the boundary |
| State ownership | Where the current truth lives and who may change it |
| Error propagation | How a bad upstream result is detected before it spreads |
| Permission transit | Whether a handoff broadens authority or exposes restricted data |
| Retry behavior | How to prevent loops, duplicates, and conflicting updates |
| Observability | Trace IDs, handoff records, timings, decisions, and outcomes |
| Evaluation | Tests for each agent plus end-to-end tests for the whole chain |
These are operating costs, even if the software framework makes agent creation easy.
Test the claimed benefit directly
Architecture debates often use words such as scalability or specialization without naming a measurable constraint. Replace the label with a test.
If the claim is better specialization, compare one well-instructed agent with scoped tools against specialist agents on the same truth set. If the claim is parallel speed, measure end-to-end latency under realistic load, including reconciliation. If the claim is stronger security, confirm that separate identities and tool scopes create a boundary the single-agent design cannot provide.
The comparison should record:
- Task success and unacceptable failures
- Total model and tool calls
- End-to-end latency
- Context passed between components
- Handoff and retry failures
- Human review effort
- Trace completeness
- Recovery behavior
Do not rely on a demonstration in which each architecture receives different prompts, tools, or test cases. That tests two implementations, not the architecture choice.
Distinguish workflow stages from agent identities
A business process may have planning, checking, approval, and execution stages. Those stages do not automatically require separate agents.
Deterministic workflow logic can enforce order. A policy service can check permissions. A human can approve a protected action. One agent can switch instructions between bounded steps while retaining a single trace.
Split the stages only when separation changes something important: access, context, ownership, independent scaling, or fault containment. Otherwise, multiple names can make the diagram clearer while making the system harder to operate.
Design the handoff before adding the agent
When multiple agents are justified, define the interface first. A useful handoff packet contains:
- Task and completion condition
- Input facts and source references
- Allowed tools and actions
- Data classification and permission scope
- Output schema
- Uncertainty or refusal rules
- Deadline and retry limit
- Trace ID and parent task ID
- Escalation owner
The receiver should not need to infer which source is authoritative or whether it may act. The orchestrator should not accept a free-form answer when the next step requires structured fields.
This contract also gives the evaluation team something stable to test. Without it, failures get blamed on “agent coordination” when the actual defect is missing state, ambiguous ownership, or an unconstrained output.
Choose the smallest architecture that survives the tests
Use a single agent when shared context and a unified permission scope make the job simpler. Use multiple agents when hard boundaries, independent ownership, or verified parallel work earn the added coordination surface. Use deterministic code for rules that should not depend on model judgment.
The right result may be hybrid: one agent for ambiguous work, ordinary workflow steps for routing, a separate service for policy checks, and a person for high-impact approval. Not every box needs to think.
Brainiac can help turn a workflow into an architecture decision record, compare the smallest viable designs, and define the handoffs and controls when separation is justified. The goal is not to maximize agent count. It is to make every added boundary pay for itself.