What a multi-agent system is — and what a monolithic model is
Enterprise multi-agent systems are having a moment. Vendors demo teams of agents that research, draft, check and act — and the architecture is real: Anthropic’s own research system, built this way, beat a single model by 90.2% on its internal research evaluation. But the same write-up reports the price: about 15 times the tokens of a chat interaction. The question is not whether multi-agent systems work. It is when the work is worth the price — and when a single model, or a fixed workflow, is the better engineering choice.
A monolithic model is the simplest shape: one model call, or one model plus tools, handling the whole task end to end. A multi-agent system is several specialised agents that split the work, hand off to each other and coordinate toward a shared goal. Between them sits a spectrum, not a wall. Anthropic’s guide to building effective agents draws the line that the rest of the industry now quotes:
Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.
On that spectrum, a single model with good retrieval is the bottom rung. Above it sit fixed workflows — Anthropic names five patterns, including prompt chaining, routing, parallelisation and orchestrator-workers — where code decides the path. At the top sit agents, and multi-agent teams are the most complex form: several agents, each directing its own process, under an orchestrator. Anthropic’s advice: find the simplest solution possible and only increase complexity when needed, because agentic systems trade latency and cost for better task performance.
When enterprise multi-agent systems win
The task decomposes into genuinely different specialisms. Anthropic’s research system is the reference case: a lead agent plans, then spawns parallel subagents that each pursue one direction and return summaries. On breadth-first research queries, that shape beat a single Claude Opus 4 by 90.2% on the company’s internal research evaluation. The key word is independent: the sub-directions must be separable, with clean interfaces between them.
Work can genuinely run in parallel. The same system ran three to five subagents at once and cut wall-clock time by up to 90% on suitable queries. Parallelisation is also a fixed-workflow pattern — Anthropic lists it alongside prompt chaining and routing — but a multi-agent team applies it to open-ended work where the sub-directions only become clear at runtime, as in breadth-first research.
Different steps need different tools, data or permissions. In the OpenAI Agents SDK, each agent carries its own tool list and guardrails — input guardrails run only for the first agent in a chain, output guardrails only for the agent producing the final output. A research agent that can read the web and a payments agent that can issue a refund should never share one permission set.
Fault isolation matters — in the orchestrated shape. The reliable shape that emerged by 2026 is one stateful orchestrator holding the full context, with narrow, stateless subagents that return summaries — not a chatty peer-to-peer mesh. In that shape, fault isolation becomes a design objective rather than a guarantee: it needs failure detection, contained state changes and explicit retry or rerouting, since retaining the plan alone does not contain a worker failure.
When a single monolithic model is the better choice
The task is well understood, narrow or repeatable. Microsoft’s AI Decision Framework is blunt: if the work is deterministic, repeatable, or can be expressed as a clear function or workflow, do not use an agent — use a workflow, an API integration or traditional code. Its Agent Framework documentation puts it even more plainly: if you can write a function to handle the task, do that instead of using an AI agent. OpenAI’s practical guide gives three signals that a task has resisted ordinary automation and might suit an agent — complex decision-making, rules too difficult to maintain, and heavy reliance on unstructured data — and adds that before committing, you should validate the use case meets those criteria clearly, because otherwise a deterministic solution may suffice.
Latency and cost budgets are tight. Anthropic’s data puts a single agent at about four times the tokens of a chat interaction, and a multi-agent system at about fifteen times. One team’s measurement of a LangGraph supervisor setup put the cost at two to three times a single agent, with about 1.5 to 2 times the wall-clock time.
Correctness depends on a single coherent context. Anthropic states that multi-agent systems are a poor fit today where all agents must share the same context, or where there are many dependencies between agents. Every handoff risks context being lost or misread. When every step needs the full picture, that dependency is a reason to prefer — and test — a single model or a fixed workflow as the baseline, rather than a multi-agent team.
The team is small or the system is early. Princeton NLP’s research found that a single agent matched or outperformed multi-agent systems on 64% of benchmarked tasks when given the same tools and context. Cognition — the makers of Devin — published “Don’t Build Multi-Agents” in June 2025, arguing the default should be one agent until the evidence forces otherwise.
The real costs of going multi-agent
The price of a team is not only the token bill. Four costs show up in production.
Orchestration overhead. Handoffs, shared state and message passing add latency and failure points. Coordination latency grows from about 200 milliseconds with two agents to over four seconds with eight or more — failures that never appear in single-agent benchmarks. The gains plateau, too: a Google DeepMind study across 180 configurations found coordination benefits level off beyond four agents, and unstructured networks amplify errors up to 17.2 times compared with single-agent baselines.
Debugging a team is harder than debugging one model. Failures hide in the conversation between agents. The MAST taxonomy — 1,642 execution traces across seven frameworks — found failure rates from 41% to 86.7%: about 42% specification and system-design problems, about 37% coordination failures, about 21% verification gaps. The most common single modes were step repetition (15.7% of failures), termination unawareness (12.4%) and task specification violations (11.8%).
Evaluation must cover the system, not just each agent. A team of passing agents can still fail together: a 20-step multi-agent workflow where each agent is 95% reliable succeeds end to end only about 36% of the time; at 90% per step, it drops to about 12%. Single-agent benchmarks do not predict multi-agent behaviour.
Cost grows with the number of agents times their steps. Budget it per run, not per model. On Anthropic’s BrowseComp evaluation, token usage alone explained 80% of performance variance; with tool calls and model choice, 95%. That is an association in one benchmark analysis, not proof that spending more tokens causes better results — but it is a reminder that the bill arrives every run, whether or not the extra spend pays off.
A decision checklist for your problem
Four questions, run over your own use case before choosing an architecture:
- Can the task be split into parts with clean interfaces, or does every step need the full context? If every step needs the full context, prefer and test a single model or a fixed workflow first.
- Do the parts need different tools, data or permissions? If one agent would need read access to everything and write access to the consequential things, splitting is a safety win.
- Is the path unpredictable enough that fixed workflows keep breaking? If a flowchart of the task would need a box labelled “figure out what to do next” that loops back on itself, that box is an agent — and it should be as small as possible.
- Is the task valuable enough to pay roughly fifteen times the tokens of a chat? Anthropic’s own framing is that multi-agent only pays off when the value of the task is high enough to cover the extra spend.
Take a support escalation: a failed delivery, a refund request, and a case that touches order history, refund policy and a human queue. As a single model with tools, one agent reads the ticket, checks the order, applies the policy and drafts the reply — one context, one thing to debug, and it stops when it is done. As a multi-agent team, a triage agent routes the message to an order or refund specialist through a handoff, an input guardrail blocks prompt injection before the main model runs, and an escalation carries structured metadata into the human queue. The team version is worth testing when the specialists genuinely need different tools or permissions, or when parallel work would cut the resolution time — and the decision should rest on measured cost, latency and success rate against the single-model baseline, not on a rule of thumb. For a support desk, the single agent with a guardrail is the baseline to beat.
Common mistakes when moving to multi-agent
Over-decomposition. Splitting a task into too many agents adds latency and error surface — and in the reported 180-configuration DeepMind study, coordination gains plateaued beyond four agents, so test agent counts on your own workload rather than treating four as a hard ceiling.
Chatty agents. Agents that negotiate endlessly instead of acting burn tokens and time: step repetition and termination unawareness are two of the most common multi-agent failure modes. Every handoff needs a termination rule — the supervisor pattern ends when the supervisor returns without naming a next worker — and a hard cap on turns.
Shared-state pitfalls. Agent A writes to shared state; agent B reads it before the write has propagated and acts on stale information. This is the MAST study’s coordination-failure category, invisible to any evaluation that tests agents one at a time.
Skipping guardrails. A team of agents must never hold broader permissions than any single agent would get. Microsoft tiers agent risk from low (individual productivity) through medium (internal services) to high (business-critical or customer-facing), with a decision-rights framework setting what the agent may decide alone and what needs a human. Put the input guardrail on the first agent and the output checks on the agent that produces the final answer — in the OpenAI Agents SDK, that is where they run — and treat a tripped guardrail as a halt, not a warning.
How to start if you choose multi-agent
If the checklist points to multi-agent, start small enough that the pilot can fail cheaply.
Start with two agents and one handoff, not a team of five. OpenAI’s incremental path: start with a single agent, add tools before adding agents, and move to multi-agent designs only when needed. A triage agent and one specialist, connected by a single declared handoff, is a complete pilot. The fragment below shows only the handoff declaration — imports, specialists and runner omitted — but the shape is the whole design decision: the model chooses when to hand off, and the handoff is a tool like any other.
# Fragment: the handoff declaration is the design.
# The model calls transfer_to_<agent> when the case belongs elsewhere.
triage_agent = Agent(
name="triage",
instructions="Route each message to the right specialist.",
handoffs=[refund_agent],
)Keep a human approval step on consequential actions until reliability is proven. High-risk actions — payments, policy decisions, anything irreversible — need real-time human approval while the pilot runs. Escalations should carry structured metadata into the human queue, so the reviewer is not starting from zero.
Log every handoff and tool call so the team is debuggable from day one. Configure logging that covers runs, handoffs and tool calls, and verify it works before the pilot goes live.
Measure the pilot against the single-model baseline before scaling. Track cost, latency and success rate per run, and use Anthropic’s multiples — about four times the tokens of a chat for an agent, about fifteen for a multi-agent system — as a sanity check, not a forecast for your workload. If the team does not win on success rate by enough to pay for its cost, keep the single model.
Gartner predicted that over 40% of agentic AI projects will be cancelled by the end of 2027, and Camunda’s 2026 survey found that 71% of organisations use AI agents but only 11% have production deployments. Those figures describe adoption, not its cause: orchestration maturity is a practical concern supported by the failure-mode research above, but the statistics alone do not prove it is the cause, or rule out other factors such as model selection. A multi-agent system is worth considering when the work splits into independent directions, the parts need different tools or permissions, or the task’s value justifies the extra spend — weigh those factors against measured cost, latency and success rate, and pilot before committing. For everything else, the simplest solution that works is still the right one.
