Multi-Agent Systems

The intuition

One expert with one desk can only hold so many open binders. Multi-agent systems hire specialists with separate desks: each gets a clean workspace, a focused job, and a thin report back to a lead — or control of the whole conversation is handed to the next specialist at the counter.

That buys three real things: more desk space (context isolation), several people researching at once (parallelism), and specialists who only carry the tools for their craft (specialization). It also buys coordination meetings, conflicting decisions when two people decorate the same room without talking, and a bill that can jump to ~15× a simple chat.

Why it exists

A single agent hits hard limits that more prompt engineering cannot always fix.

1. Context windows fill. Deep research or multi-repo exploration can burn 100k tokens of tool output. Subagents explore in a clean window and return a 1–2k summary so the lead stays sharp.
2. Wall-clock breadth. Independent investigations in sequence are slow. Parallel workers trade tokens for latency on breadth-first work.
3. Tool and prompt interference. One mega-prompt with 80 tools selects poorly. Focused agents with small toolsets raise reliability in distinct domains.
4. But coordination is a new failure class. Isolated agents make conflicting implicit decisions; token cost multiplies (~15× chat per Anthropic's multi-agent measurement); debugging spans N transcripts.

The alternative is almost always better until proven otherwise: one agent, compaction, and better tools. Cognition's restraint argument and Anthropic's research system are the two poles — both are right for different task shapes.

The core idea

A multi-agent system splits work across several LLM agents, each with its own context window, prompt, and toolset. Three architectures cover practice: supervisor / orchestrator-workers — a lead agent decomposes the task, spawns worker agents (often in parallel), and synthesizes their results; workers report to the supervisor, never to each other. Handoffs — peer agents transfer control of the whole conversation (triage agent hands a billing question to the billing agent); one agent is active at a time, so this is really dynamic routing, not parallelism. Swarm / peer networks — agents messaging each other without a central coordinator; elegant on slides, rarely shipped, because coordination failures multiply with every edge.

The honest engineering framing: multi-agent buys exactly three things — context isolation (each agent gets a clean window; a research subagent can burn 100k tokens exploring and return a 1–2k summary, so the lead's context stays clean), parallelism (independent subtasks run concurrently — Anthropic's research system runs 3–5 subagents at once and beat a single-agent baseline by 90.2% on their research eval), and specialization (focused prompt + small toolset per domain beats one prompt juggling everything). The costs are equally concrete: Anthropic measured multi-agent at ~15x the tokens of a chat interaction (single agents ~4x), latency governed by the slowest worker plus synthesis, and a new failure class — coordination — where isolated agents make conflicting implicit decisions. Cognition's "Don't Build Multi-Agents" is the canonical counterargument: subagents that can't see each other's work produce incoherent output. Default answer: start with a single agent; go multi only when a specific bottleneck (context, parallelism, or specialization) demands it.

How it actually works

Supervisor mechanics: the supervisor is an agent whose "tools" are its workers — spawning a worker is a tool call whose arguments are the task brief. The brief is the whole game: Anthropic found workers need explicit objectives, output format, tool guidance, and task boundaries, or two workers silently duplicate or gap. Workers return condensed findings (not transcripts) as tool results; the supervisor synthesizes. Anthropic's research system adds an economics detail: Opus lead + Sonnet workers — spend on the coordinator, save on the fan-out. Token usage alone explained ~80% of performance variance on their eval — multi-agent largely wins by spending more compute in parallel, which is precisely why it's expensive.

Handoff mechanics: a handoff is a tool call (transfer_to_billing) that swaps the active agent's system prompt and toolset, keeping (all or a filtered view of) the conversation history. In LangGraph this is a node returning Command(goto="billing_agent", update={...}). One context, sequential control — cheap, no parallel coordination risk.

The context tradeoff (the deep part): parallel workers don't see each other's actions, and actions carry implicit decisions. Cognition's example: two subagents building one game produce a bird and a background with clashing visual styles — each locally correct, jointly incoherent. Mitigations: share full traces rather than summaries (costly), decompose so subtasks are truly independent, or keep all write work in one agent. Harrison Chase's read/write rule operationalizes it: parallelize reads, serialize writes. Research (reading) decomposes cleanly — conflicting reads waste tokens but don't corrupt anything; writing (code, a report) accumulates interdependent decisions, so conflicts produce incompatible artifacts. Anthropic's system obeys this: parallel subagents search; one agent writes the final report.

Decomposition rule: split by context needs, not by job title. Right: "these three investigations don't need each other's findings → three workers." Wrong: "a planner agent, a coder agent, a reviewer agent" mimicking a human org chart — those roles share almost all context, so splitting them maximizes information loss at each boundary.

On the governed enterprise platform, multi-agent shows up sparingly: e.g. parallel retrieval workers over entitlements-filtered corpora for deep research, with a single synthesis agent writing the answer — not a swarm of writers editing policies concurrently.

The flows

A worked example

Deep research on internal docs for the governed enterprise platform:

"Summarize how change-management policy interacts with emergency production access across the last two years of runbooks and incidents."

Single-agent baseline: one loop, sequential search, context fills with raw chunks, compaction fights tool dumps. Illustrative: ~4× chat tokens, long wall-clock.

Supervisor design (illustrative):

Anthropic-style economics: token usage drives coverage; multi-agent can approach ~15× chat cost and still win on research evals when breadth is the bottleneck (their reported 90.2% gain vs single-agent baseline, with token usage explaining ~80% of variance).

Common drill-downs

Questions that matter once you are past "we should use multi-agent because it sounds advanced."

When do multiple agents beat one agent? When one of three bottlenecks is real: context — the task's working set exceeds one window and subtasks decompose cleanly (research across many sources); parallelism — independent subtasks with wall-clock pressure; specialization — domains distinct enough that focused prompts/toolsets measurably raise reliability. Absent one of these, a single agent with good tools wins on cost, latency, and coherence.

Supervisor vs handoff — pick for a support bot. Handoff. Support is sequential — the user talks to one specialist at a time — so triage-then-transfer keeps one coherent conversation with no parallel coordination cost. Supervisor fits breadth-first work (research, analysis fan-out) where subtasks run concurrently and get synthesized.

Steelman "Don't build multi-agents." Actions carry implicit decisions; parallel agents can't see each other's decisions; conflicting decisions compound into incoherent output. So share full context in a single-threaded agent, and use compaction — not fragmentation — when context runs out. It's strongest for write-heavy generative tasks; it bends for read-heavy parallel research, which is exactly the case Anthropic ships.

Why did Anthropic's multi-agent researcher outperform a single agent by 90%? Mostly compute: token usage explained ~80% of variance — parallel subagents let the system spend 15x tokens exploring more sources within one wall-clock budget, each in a clean context. The lesson is not "multi-agent is smarter"; it's that for breadth-first search, parallel token spend converts to coverage — at 15x cost.

How do agents pass information without blowing up tokens? Contract at the boundary: task brief down (objective, output schema, tools, scope limits), condensed summary up (1–2k tokens), artifacts (files, DB rows) referenced by id instead of inlined. Never forward raw transcripts by default; escalate to full-trace sharing only when conflicts prove the summaries lose too much.

One coding agent or several in parallel? One, for the writing. Code accrues interdependent decisions (naming, structure, style), so parallel writers produce merge-conflict soup — Cognition's core case. Parallelize the reads around it: codebase exploration, doc lookup, test-log analysis as subagents feeding one implementer.

A worker returns confidently wrong findings. What guards the system? Boundary validation: structured output schemas, require citations/evidence with claims, spot-check verifiable facts (a checker step or tool), and a supervisor prompted to weigh confidence rather than paste. Plus tracing so post-hoc you can find which brief and context produced the error.

Same model everywhere, or mixed? Mixed, by role: strongest model for the supervisor (decomposition quality caps the whole run) and for final synthesis/writing; cheaper-faster models for high-volume workers. Anthropic's Opus-lead/Sonnet-workers split is the reference pattern — it attacks the 15x cost directly.

Production concerns

• Token multiplication: ~15x chat cost means multi-agent only pays on high-value tasks. Cheaper models for workers, prompt caching on shared prefixes, and strict summarization contracts at boundaries are the standard mitigations. See cost.
• Latency: parallel workers cut wall-clock for breadth-first work, but the supervisor waits for the slowest worker; a sequential multi-agent pipeline is strictly slower than one agent doing the same steps. Set per-worker timeouts and degrade gracefully on partial results. See latency.
• Coordination failure modes: duplicated work (overlapping briefs), gaps (each worker assumed the other covered it), conflicting implicit decisions, and stale context (worker acts on state another worker changed mid-flight). All trace back to information boundaries — log every brief and every summary so you can see what each agent knew.
• Error propagation across boundaries: a worker's subtly wrong summary becomes ground truth in the supervisor's context, laundering the error past inspection. Validate worker outputs (schemas, citation checks, spot verification) before synthesis. See agent reliability.
• Debugging is genuinely harder: one task now spans N transcripts. Distributed tracing with a run id spanning supervisor and workers is non-negotiable; without it you cannot reconstruct which brief produced which mistake. See observability.
• Reliability engineering: durable execution matters more — a 5-minute multi-agent run that dies at 90% is expensive to lose; checkpoint supervisor state so completed workers aren't re-run. Anthropic also runs async ("fire-and-forget with a mailbox") rather than blocking the lead on every worker. See orchestration and reliability.

Test yourself

Go deeper

• How we built our multi-agent research system — Anthropic — the canonical production writeup: 90.2% gain, 15x tokens, prompt-engineering the orchestrator.
• Don't Build Multi-Agents — Cognition — the counterargument: share full traces; actions carry implicit decisions.
• How and when to build multi-agent systems — LangChain — reconciles the two: parallelize reads, serialize writes; context engineering as the real problem.
• When to use multi-agent systems — Claude blog — the three criteria (context, parallelism, specialization) and context-centric decomposition.
• Building more effective AI agents — Anthropic — Erik Schluntz (multi-agent research co-lead) on multi-agent patterns and what changed since the essay.

Where this connects

• Agents vs workflows — apply the same "simplest thing that works" test before multiplying agents.
• Orchestration — graph primitives (Send, Command, checkpoints) that implement supervisors and handoffs.
• Harness vs orchestration — why adding agents multiplies per-turn weaknesses, and how to tell a sub-agent trick from real composition.
• Agent reliability — coordination failures and error laundering across agent boundaries.
• Tokens and context — why clean windows, summaries, and compaction are the real multi-agent design problem.