A multi-agent system is a setup in which multiple autonomous software entities — most often AI agents, but the concept is broader — interact with each other and their environment to achieve goals: dividing the labour, communicating with each other, and coordinating their actions so that, collectively, they can solve problems no single agent could handle alone. That includes competitive systems where each agent optimizes its own objective (as in market or auction-based coordination); this guide focuses on the more common case of cooperative teams of model-based agents. If a single AI agent is one clever assistant with a set of tools, a multi-agent system is the whole team: specialists with different roles, a way to pass work between them, and a way to keep that work coherent.
But “team of AI agents” is a phrase that currently gets used for everything from genuinely useful orchestration to expensive over-engineering. The distinction matters, because multi-agent systems are expensive to get wrong. That’s the criterion used consistently throughout this guide: fixed task order alone never settles whether agents are needed — a workflow whose steps happen to call an LLM at fixed nodes still isn’t a team of agents. Check whether each component is a separately autonomous agent, controlling its decisions and tool use within its assigned task, then test whether separate agents add demonstrated value.
Multi-agent systems in one paragraph
The classical definition, attributed to Wooldridge (2009) and quoted in a ScienceDirect overview of the field, describes multi-agent systems as “groups of agents that communicate between themselves to share information and resources, coordinate their activities and cooperate in the joint efficient solution of a distributed problem.” A survey of the field (arXiv 2402.01968) puts it more structurally: multiple autonomous agents interacting with each other and their environment to achieve goals, where autonomy, communication, and society are the three features that separate a multi-agent system from an expert system or plain object-oriented code. Agents have their own goals, perceive their environment, and act on it — and critically, they relate to each other through cooperation (a shared objective, requiring coordination, communication protocols, and consensus) or competition (each optimizing its own objective, as in market or auction-based coordination).
The consistent dividing line: whether the components are agents. A scripted pipeline is a deterministic code path — fixed steps, fixed routing, fixed tool calls — and an LLM can sit at a fixed node without making it multi-agent: the generated content at a fixed node need not be predetermined, but that model judgment alone doesn’t establish an agent. A multi-agent system has separately autonomous agents: even when the step order is fixed (the sequential pattern below), each agent controls its decisions and tool use within its assigned task. And a single agent with tools is just one model, one chain of decisions, one context window.
How multi-agent systems actually work
Every working multi-agent system has the same three moving parts: agents, communication, and coordination.
Agents: roles, capabilities, autonomy boundaries. Each agent is an autonomous entity — in this guide’s focus, a model-based agent — with a defined role (“you are the research specialist”), a bounded set of tools, and an autonomy budget — what it may do without asking, enforced with hard limits like a maximum number of steps. CrewAI makes this explicit: every agent carries a role, a goal, a backstory, and a tool set.
Communication: message passing, shared state, handoffs. The classical methods are direct (agent-to-agent), mediated (via a coordinator), broadcast (one-to-many), and stigmergy (coordinating through a shared environment). In practice: message passing (a supervisor routes a task to a worker; the worker reports back), shared memory or a blackboard (AutoGen’s group chat thread; LangGraph’s shared typed state), and tool-mediated handoffs (one agent hands control directly to a peer, skipping the return trip through a coordinator).
Coordination: orchestrators vs emergent peer behaviour. Someone — or no one — is in charge. In orchestrated systems a central agent decomposes tasks, delegates them, and synthesizes results. In decentralized peer systems agents negotiate among themselves using protocols like task allocation or consensus (the Contract Net Protocol is the classic example). And where you choose an orchestrated design, one rule is near-universal within it: exactly one agent must be designated as the orchestrator — two that both believe they are coordinating produce duplicated work, contradictory instructions, and race conditions that are miserable to debug. (Decentralized peer networks, covered below, are the deliberate exception: they have no orchestrator by design.)
A concrete example makes the message flow tangible. Take a weekly competitive-intelligence brief: the supervisor decomposes it into (1) a Web-Research Agent gathering recent announcements, (2) a Data Agent pulling pricing and market numbers, (3) a Writer Agent drafting the brief, and (4) a Critic Agent checking every claim in the draft against the sources. Each worker operates autonomously within its brief and reports back; the supervisor never writes the brief itself — it plans, routes, evaluates, and synthesizes. That division is the entire point.
The main architecture patterns
Most real systems reduce to five patterns. LangGraph’s documentation names three of them explicitly, practitioners treat a fourth — sequential pipelines of agents — as the default starting point, and a fifth — market or auction-based allocation — comes from the classical competitive literature.
1. Supervisor / orchestrator (hub-and-spoke). A central supervisor agent receives the task, routes it to specialist workers, and each worker reports back to the supervisor, which decides the next step or terminates. LangGraph describes this as the right choice for the common case — roughly four specialists, one orchestrator. The pattern’s weakness: the supervisor can become a bottleneck if it has to “think too hard” about every routing decision, and every domain hop costs two LLM calls (out to the worker, back to the supervisor). An illustrative sketch of the routing loop:
# Illustrative sketch — adapt names, models, and tools to your stack.
def run_supervisor(task, specialists, max_steps=10):
state = {"task": task, "inbox": [], "steps": 0}
while state["steps"] < max_steps: # hard exit limit — never trust FINISH alone
decision = supervisor.decide(state) # a specialist name, or FINISH
if decision == "FINISH":
return supervisor.summarize(state)
result = specialists[decision].run(state) # delegate
state["inbox"].append(result) # workers report back
state["steps"] += 1 # bounded autonomy
raise RuntimeError("max_steps exceeded")The hard step counter is not optional: without it, a supervisor that never emits FINISH will happily exhaust your API budget. Parallel work is a natural extension — the supervisor fans out independent subtasks to several workers at once and joins their results (LangGraph's Send primitive exists precisely for this).
2. Hierarchical teams of specialists. When the team grows, a supervisor of supervisors takes over — a top-level orchestrator delegating to mid-level supervisors, each managing its own workers. Practitioners warn this triples the cost and is only worth it past roughly eight specialists; below that, it's over-engineering.
3. Decentralized peer networks. No central orchestrator: agents negotiate, bid, or hand off to each other directly (the "swarm" pattern). This saves LLM calls — one per domain instead of two — but sacrifices the supervisor's global view, making routing mistakes harder to catch and full evaluation nearly impossible. It suits genuinely peer-to-peer collaboration.
4. Sequential patterns. CrewAI's first-class sequential process runs tasks in order, each feeding into the next — research, then write, then review — and production practitioners recommend it as the default because when it fails you know exactly which task and which agent failed. Note the distinction from a scripted pipeline: the order of steps is fixed, but each step is handled by an agent with real autonomy — choosing its own tools and sources, handling its own errors, and deciding when to hand off. A fixed sequence of plain LLM calls at fixed nodes is still a workflow; the same sequence staffed by genuinely autonomous agents is a multi-agent system.
5. Market or auction-based coordination. This describes a different design dimension — how work gets allocated rather than the order it runs in. It comes from the classical literature's competitive side: agents bid for tasks (as in the Contract Net Protocol), which fits scenarios like resource allocation across autonomous parties pursuing their own objectives.
When to use a multi-agent system
Multi-agent systems earn their complexity when several of these conditions hold at once:
The task genuinely decomposes. The work splits into independent or specialized sub-tasks that share one objective and converge on a single result — research plus coding plus testing, or several independent research threads that a synthesis step joins into one brief. Direct dependencies between specialists are common but not required: independent contributions that aggregate into one combined result count as a team, as the parallel-generation example in the next paragraph shows. What doesn't qualify is two unrelated jobs wearing one hat. Role labels alone (planner, reviewer, executor) are not decomposition — that split needs a real boundary behind it, as the counterweight below explains.
Parallelism buys real speed. Independent subtasks can run simultaneously — breadth-first research, or parallel generation with a final synthesis — where a single agent would serialize them.
Roles must stay separated by enforced boundaries. A planner that must not see execution details, a critic that must not be the author — or the hard version, different security contexts, where one agent may touch customer data another must never see. Practitioner write-ups identify security contexts as a genuine trigger for separate agents, because access boundaries are harder to enforce inside a single agent's context. Note this means enforced boundaries, not mere role labels.
One agent's context can't hold the job. A red flag from a practitioner deep-dive: a single agent's system prompt ballooning past roughly 2,000 tokens of instructions because it's covering too many roles. Treat this as a heuristic, not a measured threshold — when the work outgrows one context window, decomposition becomes a memory strategy too.
Verification raises quality. Anthropic's own internal research found that agents are significantly more effective at catching each other's errors than at catching their own — the empirical backbone of the writer-plus-reviewer and coder-plus-tester pairings. In MultiAgentBench (Zhu et al., ACL 2025), which evaluated star, chain, tree, and graph topologies across six domains, graph-structured teams performed best in the research scenario.
When a single agent (or no agent at all) is the better choice
Here is where the industry's most experienced builders are bluntest. Anthropic's Building Effective Agents states: "We recommend finding the simplest solution possible, and only increasing complexity when needed." The frontier labs agree: OpenAI says to maximize a single agent's capabilities first; Microsoft says to start with a single-agent prototype unless security boundaries or compliance domains demand otherwise (planner/reviewer/executor roles alone don't justify separate agents — one agent can emulate all three); Google warns a sub-agent where a plain function would do creates overhead that compounds. As LangChain's Harrison Chase puts it: "Most teams that come to us with multi-agent problems actually have a single-agent tool-design problem."
Concretely, don't build a multi-agent system when:
A single well-prompted agent with the same tools answers correctly. This is the case more often than teams expect, and adding a coordinator "just to add agents" is the most common waste pattern.
The task is sequential and single-domain. One research question, one coding task, one customer interaction that stays on topic — inter-agent coordination is pure overhead here.
Latency is critical and the task can't be parallelized. Multi-agent overhead only makes a serial pipeline slower, and every added agent adds another LLM call's cost and another failure probability.
You can draw the flowchart. "Workflows offer predictability and consistency for well-defined tasks, whereas agents are better for open-ended problems where it's hard to predict required steps." If the steps, the routing, and the tool calls are all fixed in code, the work belongs in a scripted pipeline — even if an LLM at one of those fixed nodes writes undetermined text. A fixed sequence of steps staffed by separately autonomous agents — research, then write, then review, each choosing its own tools and handling its own errors — is a sequential multi-agent pattern, not a scripted workflow; judge by whether the components are agents, not whether you can draw the diagram. (The research behind this guide turned up no reputable benchmark of deployment economics, so treat cost trade-offs qualitatively: every added agent multiplies LLM-call cost and failure surface.)
The honest starting position: prove the single agent fails — on context limits, role confusion, or domain breadth — then add agents to fix the specific shortcoming you can demonstrate with data.
Common failure modes and how to avoid them
Multi-agent systems fail in ways worth knowing before you build:
Cascading errors. One agent's plausible-but-wrong output becomes another agent's trusted input, and the error compounds down the chain. Mitigation: validate between agents — a critic, a schema check, a verify-against-sources pass — so poison doesn't flow silently.
Runaway loops. A supervisor that never decides to finish, or two agents bouncing work back and forth, burns budget while producing nothing. Defenses: hard step counters (as in the sketch above), bounded autonomy, and an explicit termination signal in the supervisor's instructions.
Coordination conflicts and supervisor bottlenecks. In supervised designs, two self-declared orchestrators produce duplicated work and contradictory instructions; a single supervisor forced to micromanage every decision becomes the throughput ceiling. Keep the orchestrator's decisions coarse — route and synthesize, don't micromanage.
Debugging distributed reasoning. A failure that emerges from an interaction between agents can't be reproduced by re-running one of them. In designs that use one, treat the orchestrator's state as the source of truth; without an orchestrator, the shared event history plays that role. Either way, checkpoint after every agent step, and keep each step inspectable. If you can't trace it, you can't debug it.
More agents, worse results. MultiAgentBench found that adding more agents can actually decrease performance — coordination complexity grows faster than capability. Every agent you add should earn its place by solving a demonstrated problem.
The through-line is the same: bounded autonomy, validated handoffs, and full observability — boring engineering virtues, and what separates a system that ships from a demo that loops.
A decision checklist for your architecture
Use these questions, in order, the next time you're tempted to build a team of agents:
1. Can this be a pipeline? If you can draw the flowchart and the routing, tool calls, and handoffs are all fixed in code, write a scripted workflow. As Anthropic puts it: "Workflows offer predictability and consistency for well-defined tasks". LLM calls at fixed nodes don't make a pipeline multi-agent. But a fixed sequence of separately autonomous agents — each controlling its decisions and tool use within its task — is a sequential multi-agent pattern, not a pipeline.
2. Can one agent do it? Give a single agent the tools, good instructions, and disciplined context management. Build this version first and measure where it breaks — context limits, role confusion, or domain breadth.
3. Does the task decompose into sub-tasks under one shared objective? If yes, you're in multi-agent territory — provided the sub-tasks converge through meaningful coordination or aggregation: a synthesis step, a supervisor, or a shared state. Independent contributions to one combined result qualify; two unrelated jobs wearing one hat don't.
4. Does parallelism, enforced role boundaries, or verification raise the ceiling? Breadth-first work, security boundaries, or generate-and-review quality gains are the legitimate reasons. "More agents feels more powerful" is not.
5. Can you afford the coordination tax? More LLM calls, more latency, a harder-to-debug system. Start sequential — failures are attributable — and add dynamism only when the task set genuinely can't be predetermined.
If the checklist above points to a multi-agent design, prototype the smallest thing that could work: a sequential pipeline of two or three agents with validated handoffs, a hard step limit, and full tracing. If it doesn't, prototype the scripted workflow or single agent the earlier questions led to. Multi-agent systems are a team of specialists — hire the team only when the job is genuinely too big for one very good generalist, and you'll get architecture that scales instead of overhead that compounds.
