- Multi-agent is a cost–quality trade, not an upgrade. Anthropic measured a 90.2% gain over a single agent on its internal research eval — at roughly 15× the tokens of a chat.
- Seven patterns cover almost everything: orchestrator–workers, pipeline, router/handoff, parallel fan-out, evaluator–optimizer, blackboard and hierarchical teams.
- The failures are architectural. In the MAST study, specification issues (41.77%) and inter-agent misalignment (36.94%) dominate; better models alone do not fix them.
- Context is the hard part. Delegate with self-contained contracts and pass artifacts by reference, not lossy prose summaries.
- Every agent is a security principal. Scope its tools, treat peer messages as untrusted data, and log every action against the agent that took it.
Every team building agents eventually hits the same wall. The single agent that handled the prototype starts to sag under a growing tool list, a context window stuffed with half-relevant documents, and tasks that would obviously go faster if three things happened at once. The instinct is to add more agents. Done well, that instinct produces systems that are faster, more capable and easier to secure. Done badly, it produces a distributed system with all of the failure modes of microservices and none of the determinism.
The pressure to get this right is real. Gartner named multiagent systems one of its top strategic technology trends for 2026, and the same firm has forecast that over 40% of agentic AI projects will be scrapped by the end of 2027 on cost, unclear value and weak risk controls. Architecture is where those three outcomes are decided.
This guide is the architecture layer of our agentic AI series. It assumes you know what an agent is; if not, start with AI agents as employees. It pairs with our deep dives on why AI agents fail in production, context engineering and the Model Context Protocol.
What a Multi-Agent System Actually Is
Four building blocks, and the line between workflows and agents
Strip away the marketing and a multi-agent system is a set of LLM-driven components, each with a narrow job, plus the machinery that makes them cooperate. In practice every design is a combination of four decisions:
- Agents. Each has a role prompt, a model, a tool allowlist and its own context window. Different agents can use different models — a strong model to plan, cheaper ones to execute.
- Topology. Who can talk to, delegate to, or hand off to whom: a star around an orchestrator, a line, a tree, or a shared hub.
- Communication. Direct messages, transfers of control (handoffs), or reads and writes against shared state. This is where context is gained or lost.
- Control plane. Budgets, termination rules, verification gates, permissions, tracing and human approval. Most production failures trace back to gaps here.
Workflows vs. agents: who decides what happens next?
Anthropic's widely cited Building Effective Agents draws the most useful line in this space. In a workflow, LLMs and tools are orchestrated through predefined code paths. In an agent, the model dynamically directs its own process and tool use. Multi-agent systems sit on both sides of that line: a pipeline of agents is a workflow, while an orchestrator that decides at runtime how many workers to spawn is agentic. The distinction matters because every decision you hand to a model is a decision you must evaluate, trace and bound.
Where MCP and A2A fit
Two open protocols now frame the plumbing. The Model Context Protocol connects an agent to tools and data — it became a founding project of the Linux Foundation's Agentic AI Foundation in December 2025. The Agent2Agent (A2A) protocol, introduced by Google in 2025 and since moved to Linux Foundation governance, connects agents to other agents: capability discovery through agent cards, task delegation and streaming status updates. In August 2026 Axios reported that A2A would join MCP under the same foundation. The practical split: MCP inside every agent, A2A only where an agent crosses an ownership boundary.
Single Agent or Multi-Agent?
The evidence for both sides, and how to decide
The strongest public case for multiple agents comes from Anthropic's write-up of its multi-agent research system. A lead agent running Claude Opus 4 with Claude Sonnet 4 subagents outperformed a single Opus 4 agent by 90.2% on the company's internal research evaluation, and parallel tool calling cut research time by up to 90% on complex queries. The same post is candid about the bill: agents use about 4× the tokens of a chat interaction, and multi-agent systems about 15×.
That last number is the key to reading the result honestly. Much of the multi-agent advantage comes from spending more tokens in parallel, separate context windows — breadth that a single context cannot hold. It is a genuine architectural benefit, but it is bought, not free.
Actions carry implicit decisions, and conflicting decisions carry bad results.
The counter-argument is just as well founded. Cognition's essay argues that when parallel subagents cannot see each other's full traces, they make unstated choices that clash at merge time — and that for tightly coupled work like writing a program, a single-threaded agent with careful context compression is more reliable. Anthropic's post agrees on the boundaries: multi-agent designs are a poor fit when all agents need the same context, when there are many dependencies between subtasks, and for most coding tasks.
A practical decision test
Both camps are describing the same variable: coupling. Go multi-agent when you can answer yes to at least one of these, and the task value justifies a several-fold cost increase:
- Breadth exceeds one context. The information needed genuinely will not fit, or degrades the agent's focus when it does.
- Subtasks are independent. They can be completed without seeing each other's intermediate decisions — research threads, document batches, separate systems.
- Tool sprawl hurts selection. One agent with dozens of tools picks the wrong one; specialists with five each do not.
- Privileges must be separated. The step that reads untrusted email must not be the step that can wire money.
| Dimension | Single agent | Multi-agent |
|---|---|---|
| Context | One window holds everything; risk of overflow and distraction | Each agent gets a clean, focused window; risk of lost context at handoffs |
| Parallelism | Sequential tool calls (parallel calls within a turn at best) | Independent subtasks run concurrently, cutting wall-clock time |
| Token cost | Lower — roughly 4× a chat interaction per Anthropic | Higher — roughly 15× a chat interaction per Anthropic |
| Coherence | Every decision is made with full history | Implicit decisions can conflict between agents |
| Least privilege | One agent holds every tool and credential | Tools and secrets can be scoped per role |
| Debugging | One trajectory to read | Distributed trace across agents, handoffs and shared state |
The Seven Core Multi-Agent Design Patterns
Topologies that cover nearly every production system
Framework names differ — supervisors, crews, swarms, graphs — but the underlying topologies are few. The five workflow patterns in Anthropic's guide (prompt chaining, routing, parallelization, orchestrator–workers and evaluator–optimizer) plus two structures common in larger systems give you a complete vocabulary.
Select a pattern. Ratings are Thundrom's qualitative engineering assessment (1–5), not benchmark results.
A lead agent decomposes the goal at runtime, spawns specialised workers with narrow instructions, and synthesises their results. The decomposition itself is a model decision, not code.
Open-ended work whose subtasks you cannot enumerate up front — research, due diligence, broad codebase audits.
The lead becomes a bottleneck and a single point of misunderstanding; vague delegation produces duplicated or conflicting work.
1. Orchestrator–workers (supervisor)
The default for open-ended work. A lead agent plans, delegates to workers in parallel, and synthesises. Anthropic's research system adds a detail worth copying: the lead saves its plan to external memory so it survives context truncation, and a separate citation agent attributes claims after the research is done. The make-or-break skill is delegation — each worker needs an objective, an output format, guidance on tools and sources, and clear task boundaries, or workers duplicate each other.
2. Sequential pipeline
The most underrated pattern. When the steps are known, encode them in code and let each agent do one transformation well. Insert programmatic gates between stages — schema validation, required-field checks, confidence thresholds — so an error stops where it starts rather than compounding downstream.
3. Router and handoff
A triage agent classifies the request and transfers control to a specialist. The OpenAI Agents SDK makes handoffs a first-class primitive; LangGraph models them as edges between nodes. Design each handoff to carry a structured summary — who the user is, what was tried, what is being asked — and cap the number of transfers per conversation to prevent ping-pong.
4. Parallel fan-out
Split in code, run concurrently, reduce deterministically. Two variants matter: sectioning, where workers take different slices of the input, and voting, where several agents attempt the same task and you take consensus — useful for high-stakes classifications. Running a guardrail agent in parallel with the main response is a cheap, effective form of sectioning.
5. Evaluator–optimizer
A generator and a critic iterate until the critic passes the output. It works when the criteria are explicit. Ground the evaluator wherever possible — run the tests, validate the schema, check against retrieved facts — because a model judging another model's prose drifts towards approval. Always bound the number of rounds.
6. Blackboard and shared state
Agents coordinate through a shared, versioned store instead of messaging each other — an idea borrowed from classic AI blackboard systems and now common as graph state in LangGraph or as event streams on a broker such as Kafka. It suits asynchronous, long-running processes. Treat the store like any shared database: one owner per field, optimistic concurrency, and an append-only history of every write.
7. Hierarchical teams
Supervisors managing supervisors. It keeps any one agent's tools and context manageable at very large scale, but every layer adds latency, cost and a lossy summarisation hop. Adopt it only after a flat orchestrator has measurably run out of room.
Context, Contracts and Communication
How work moves between agents without losing meaning
Every boundary between agents is a compression step. The orchestrator knows why a subtask exists; the worker sees only what it is told. Most coordination bugs are information that failed to cross that boundary. Three rules keep it intact:
- Delegate with contracts, not chat. Each subtask carries a self-contained objective, an output schema, a tool allowlist, and step and token budgets. A worker should never need the orchestrator's history to understand its job.
- Pass artifacts by reference. Workers write full results to storage — files, rows, documents — and return a pointer plus a short structured summary. The synthesiser reads originals rather than a game of telephone.
- Make shared decisions explicit. If parallel workers must agree on a convention — naming, units, style, an API version — the orchestrator decides it first and puts it in every contract.
Here is that shape as a framework-neutral orchestrator–workers skeleton. The interesting parts are not the model calls; they are the contract, the hard budget, the validation of every worker output, and the refusal to synthesise over a missing piece.
import asynciofrom dataclasses import dataclass # make_agent, planner, synthesizer and validate stand in for your framework# (LangGraph, OpenAI Agents SDK, Microsoft Agent Framework, Google ADK, ...). @dataclass(frozen=True)class SubTask: id: str objective: str # self-contained: no "as discussed above" output_schema: dict # machine-checkable definition of done tools: tuple # least privilege: this worker sees only these max_steps: int = 12 max_tokens: int = 40_000 class BudgetExceeded(Exception): pass class Budget: def __init__(self, tokens): self.left = tokens def charge(self, used): self.left -= used if self.left < 0: raise BudgetExceeded() # hard stop, not a suggestion async def run_worker(task, budget, trace): agent = make_agent(tools=task.tools, max_steps=task.max_steps) with trace.span("agent.worker", task_id=task.id): res = await agent.run(task.objective, schema=task.output_schema) budget.charge(res.usage.total_tokens) ok = validate(res.output, task.output_schema) return {"task": task, "artifact_ref": res.artifact_ref, "ok": ok} async def orchestrate(goal, trace, total_tokens=400_000): budget = Budget(total_tokens) plan = await planner.decompose(goal) # list of SubTask, max 5 results = await asyncio.gather( *(run_worker(t, budget, trace) for t in plan), return_exceptions=True, ) good = [r for r in results if isinstance(r, dict) and r["ok"]] if len(good) < len(plan): # never let the synthesizer paper over a missing piece return escalate_to_human(goal, results) return await synthesizer.run(goal, artifacts=[r["artifact_ref"] for r in good])How Multi-Agent Systems Fail
What the MAST study found across seven frameworks
The most rigorous public failure analysis is “Why Do Multi-Agent LLM Systems Fail?” (Cemri et al., NeurIPS 2025). The authors annotated traces from seven popular multi-agent frameworks across more than 200 tasks, reaching an inter-annotator agreement of κ = 0.88, and organised what they found into the MAST taxonomy: 14 failure modes in three categories.
Step repetition 17.14% · disobeying task spec 10.98% · unaware of termination conditions 9.82% · loss of conversation history 3.33%
Reasoning–action mismatch 13.98% · failure to ask for clarification 11.65% · task derailment 7.15% · conversation reset 2.33%
Premature termination 7.82% · no or incomplete verification 6.82% · incorrect verification 6.66%
Read the leading modes as design defects. Step repetition and unawareness of termination conditions are missing control-plane rules. Failure to ask for clarification and reasoning–action mismatch are contract problems at agent boundaries. The verification category is simply the absence of checks. The authors conclude that better base models, while helpful, are insufficient on their own to guarantee reliable multi-agent performance.
Mapping failure modes to fixes
- Repetition and runaway loops → hard step ceilings, loop detection on repeated tool calls with identical arguments, and explicit done-criteria in every contract.
- Misalignment at handoffs → structured handoff payloads, a clarification path back to the orchestrator, and shared decisions written into each contract.
- Premature or missing verification → programmatic checks against world state and schemas after every consequential step, not a model's assurance that it is finished.
- Silent partial failure → a synthesiser that refuses to merge when inputs are missing, and one trace that spans every agent so the gap is visible.
Security and Governance Across Agents
Every agent is a principal with its own blast radius
Splitting work across agents is also a security decision — potentially your best one. Prompt injection sits at the top of the OWASP Top 10 for LLM applications, and there is still no reliable way to make a model ignore instructions hidden in content it reads. The defence is structural: limit what a compromised agent can do.
Simon Willison's “lethal trifecta” gives the design rule: an agent that combines access to private data, exposure to untrusted content, and the ability to communicate externally can be tricked into exfiltrating that data. Multi-agent architecture lets you make sure no single agent ever holds all three.
- Per-agent identity and least privilege. Each agent gets its own service identity, short-lived credentials and the minimal tool allowlist for its role — never a shared super-token.
- Quarantine untrusted input. The agent that reads web pages, emails or uploads has no write tools. It returns structured, validated data, which a privileged agent consumes as data.
- Treat peer messages as untrusted. An instruction arriving from another agent is not an instruction from your user. Validate it against a schema and the original task.
- Human gates on irreversible actions. Payments, deletions, outbound messages and permission changes pause for approval, with the full trace attached.
- Append-only audit trail. Every action records the agent, model version, tool call, inputs and the upstream request that caused it, so incidents can be reconstructed and responsibility assigned.
An Implementation Playbook
From one agent to a governed multi-agent system
The teams that succeed treat multi-agent as an optimisation they earn with evidence, not a starting architecture. This is the sequence we use on client engagements.
- Step 01
Baseline with one agent
Build the single-agent version first, with the full tool set and a strong model. Capture 30–50 real tasks with known-good outcomes as your evaluation set. This is the bar every multi-agent design has to beat on quality, latency and cost per successful task.
- Step 02
Find the seams
Read the traces. Split only where the evidence points: context windows overflowing, tool lists too long for reliable selection, subtasks that are provably independent, or permissions that must not live in the same agent.
- Step 03
Pick the simplest topology that fits
Known steps → pipeline. Separable domains → router. Independent slices → fan-out. Unknowable decomposition → orchestrator–workers. Reach for hierarchy or a blackboard only after a flatter design has measurably failed.
- Step 04
Write contracts, not prompts
Every delegation gets an objective, an output schema, a tool allowlist, and step and token budgets. Workers return artifacts by reference with a trace ID, not prose summaries of what they think they did.
- Step 05
Instrument and gate
One trace per task spanning every agent, tool call and handoff. Programmatic checks between stages. Release only when the multi-agent version beats the baseline on the same evaluation set, run repeatedly.
- Step 06
Govern before you scale
Per-agent identities and credentials, human approval on irreversible actions, an append-only audit log, and a kill switch. Then — and only then — widen the rollout.
Architecture Is the Agent Strategy
Multi-agent systems are not a more advanced kind of AI; they are a distributed-systems answer to three concrete problems — context that will not fit, work that can run in parallel, and privileges that must be separated. When those problems are real, the evidence says the gains are large. When they are not, the same architecture multiplies cost and creates new places to fail.
The durable skills are the unglamorous ones: choosing the simplest topology, writing contracts instead of prompts, putting hard limits in code, verifying against the world rather than the model, and giving every agent an identity you can audit. Get those right and the choice of framework — and of next year's model — becomes a detail.
Frequently Asked Questions
What engineering leaders ask before committing to multi-agent
Designing a multi-agent system? Get the architecture right first.
We help teams decide whether they need multiple agents at all, choose the topology, and ship it with tracing, evaluation and governance built in.

