Planning autonomous agents — the loop of breaking a goal into steps, acting on them, checking the result, and replanning when the world changes — is what separates a chatbot from a true autonomous agent. A simple reactive chatbot answers the question in front of it. An agent carries a goal across many steps, decides what to do next, and keeps going until the job is done or it has to give up.
The two kinds of system fail differently, cost differently, and suit different jobs. This article sets out what a simple reactive chatbot does, what planning adds, how the planning loop works, and where the line between the two really sits.
What a simple reactive chatbot does
A simple reactive chatbot is the deliberately simple end of the spectrum: one input in, one response out, and then the job is done. A single Chat Completions API call is stateless: the caller must store the transcript and send the accumulated messages array on each request, because the API retains no conversation state between calls. OpenAI’s documentation puts it plainly: “In Chat Completions, conversation state must be managed manually.”
{
"model": "gpt-4o",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"},
{"role": "assistant", "content": "The capital of France is Paris."},
{"role": "user", "content": "And its population?"}
]
}The model sees only what the caller sends in that request. It carries no goal state between turns, does no decomposition, sequences no tools, and recovers from nothing — because there are no steps. That is the baseline this article uses. It is not a description of every chatbot: many real chatbots add memory, tools, or multi-turn state. The baseline is the stimulus-response case, chosen so the contrast with a planning agent is clear.
What planning means for an agent
Planning is the mechanism that turns a goal into a sequence of steps: decomposition, sub-goal generation, ordering, and replanning when the world changes. Anthropic defines it in one sentence: “An agent is an application that completes a task by planning its own steps and calling tools that read files, run commands, or edit code.”
Decomposition is the first piece. Plan-and-Solve prompting (Wang et al., 2023) explicitly devises a plan to divide a task into smaller subtasks and then carries them out according to the plan — proposed to fix a specific failure of plain chain-of-thought reasoning: missing-step errors, where the model skips a step it needed. Chain-of-thought itself (Wei et al., 2022) is the reasoning base underneath: the model generates intermediate reasoning steps before answering, significantly improving reasoning performance.
Replanning is the second piece, and it is what makes a plan more than a script. In ReAct (Yao et al., 2022), reasoning traces are interleaved with actions: the traces “help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources such as knowledge bases or environments”. That interleaving paid off — ReAct beat imitation and reinforcement learning baselines by 34 percentage points on ALFWorld and 10 points on WebShop, Reflexion (Shinn et al., 2023) adds a verbal reflection after failure, stored in an episodic memory buffer and read on the next trial, improving decisions without any weight updates — 91% pass@1 on HumanEval, against 80% for GPT-4.
One distinction matters for the rest of this article: planning is a capability that contributes to autonomy, not a sufficient test for it. A system can plan while a human executes every step and decides when to continue — the planning is real and the autonomy is not. Autonomy needs more: control of execution, goal maintenance, observation of results, and the decision to continue or replan without a human prompting each step.
The planning loop in practice
The working anatomy of an autonomous agent is a loop: perceive the current state, plan the next steps, act through tools, check the result, and repeat until the goal is reached or the agent gives up. No vendor names the loop exactly that way, so this article presents it as a synthesis of the vendor framings plus ReAct, not as a quoted phrase: Anthropic states it as “gather context, take action, verify work, repeat”; OpenAI’s Agents SDK drives a “reason, call tool, observe, answer” loop, stopping only at a final answer.
Anthropic’s pattern is a while-loop wrapping alternating LLM API and tool calls, one loop per task. The shape of the loop, simplified to a sketch, is this:
# Sketch of an agentic loop — tool definitions and result handling omitted.
import anthropic
client = anthropic.Anthropic()
messages = [{"role": "user", "content": "File last month's expenses."}]
while True:
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
tools=[...], # tools the model may call
messages=messages,
)
# ... run any requested tool, append the result, repeat
# until the model returns a final answer or gives up.Ask a simple reactive chatbot to “file the expenses for last month’s conference trip” and it will explain how to do it, or draft the form. Give the same request to a planning agent and it forms a plan — gather the receipts, check the travel policy, fill in the form, submit it, verify the confirmation — then works through it, checking each step. If a receipt is missing, it replans, asking you for it or searching the inbox. OpenAI’s Responses API runs this kind of loop by default; its internal evals found a 3% SWE-bench improvement and 40–80% better cache utilization against manual loops.
How planning autonomous agents changes what a system can do
Three things change: multi-step goals, recovery from failed steps, and tool sequencing so earlier results inform later steps.
Multi-step goals are the clearest change: a task spanning many tool calls and taking minutes or hours is not a simple reactive chatbot’s job. METR’s 2025 measurement put the 50%-success time horizon for Claude 3.7 Sonnet at around 50 minutes — where the horizon is measured in the time a human would typically need for the task, not in how long the agent runs — and found the frontier horizon doubling roughly every seven months since 2019. METR attributes that growth primarily to greater reliability and ability to adapt to mistakes — the horizon is growing because agents are getting better at the planning loop.
Error recovery is the second change — a capability under suitable conditions, not a guarantee. In LangGraph, retry policies attach to individual nodes, so a failed API call, database query, or LLM call is retried with backoff instead of collapsing the whole graph. In LangGraph’s code-generation example, a failed check passes the stack trace back to the generation node to reflect and retry, up to three times — reaching 81%, against 55% for a single generation. These are different approaches to recovery, with different feedback paths: LangGraph retries a failed node with backoff, without changing the plan; the code-generation loop feeds the failure back so the next attempt can change; ReAct’s reasoning traces track and update the action plan as exceptions arise; and Reflexion stores a verbal reflection in an episodic memory buffer to change the next trial. Retrying a fixed step is recovery; only the approaches that revise the plan count as replanning.
But the reliability ceiling is still low. On TheAgentCompany’s 175 long-horizon workplace tasks, even the best frontier agent — OpenHands with Gemini 2.5 Pro — completed only 30.3% of tasks, taking an average of 27.2 steps at $4.20 per task; failures included agents getting stuck in loops or aimlessly exploring the environment. And the reliable version of the capability lags the headline: METR’s 80% horizon — again a human task-duration measure, not agent elapsed time — is roughly five times shorter than its 50% horizon. Planning makes multi-step work possible; it does not make it reliable.
Where the line sits: a spectrum, not a switch
Systems sit at different points along two dimensions — planning depth and execution control — not in two clean boxes. Simple scripts, single-step tool use, scripted multi-step flows, and full replanning agents are points on that spectrum.
The classification test is therefore not “does the system form its own plan?” but “who forms the plan and who executes it, maintains the goal, observes results, and decides whether to continue or replan?” Planning alone is not enough: a conversational system could generate and revise a plan while leaving all execution to the user — that system plans, but it is not autonomous. A scripted multi-step flow, meanwhile, executes without a human at each step but follows a plan a human wrote. The autonomous agent has both: it forms the plan and carries it out, watching the results and deciding for itself whether to continue, replan, or stop.
The distinction matters practically. It sets expectations: a system that plans but cannot execute will hand you a beautiful plan and leave you to do the work. It sets cost: repeated model calls and growing input context can raise cost, as the coding-workflow measurements later in this article show. A tool execution or verification step is not necessarily another model call — the Responses API, for example, lets the model call multiple tools within a single API request — so model calls, tool actions, and client API requests are three different things, not one. And it sets failure modes: a planning agent can fail by planning badly, by executing badly, or by looping forever, where a simple reactive chatbot can only fail by answering badly.
When you need planning, and when a chatbot is enough
Reach for planning when the goal is multi-step, open-ended, or needs recovery from failure: filing expenses or investigating a bug across a codebase.
A simple reactive chatbot is enough when the task is one question and one answer, with no state to carry: explain a concept, draft a paragraph, summarise a document. The test is whether the task needs anything decomposed, observed, or recovered from — not whether one response comes out at the end, since producing a single answer can still involve intermediate reasoning and subtasks. When there is nothing to decompose, observe, or recover from, a single response is enough.
The cost is worth stating with real numbers. Across 500 SWE-bench Verified coding tasks, agentic coding averaged about 3,500 times the tokens of a single-round reasoning task — 4.17 million tokens and $1.857 per task, against 1,190 tokens and $0.016 — because the accumulated context is re-sent to the model on every round. A separate benchmark found a median agent time of 66 seconds and an average model cost of $0.130 per task, with unresolved attempts costing about three times as much as resolved ones ($0.2417 against $0.0799).
Three caveats belong with those numbers. First, the SWE-bench token, latency, and cost comparisons concern coding tasks, while TheAgentCompany measures workplace tasks — and neither establishes general planning overhead across non-coding use cases. No reputable general-purpose benchmark measures planning overhead for non-coding tasks. Second, the 3,500-times figure comes from Bai and colleagues’ April 2026 preprint, which has not been peer reviewed — strong early evidence, not settled fact. Third, the research for this article did not identify a head-to-head benchmark comparing a planning agent against a non-planning agent on identical tasks outside coding, so the claim that planning enables multi-step success rests on the ReAct, Reflexion, and Plan-and-Solve ablations and the METR horizon trend, not on a direct comparison.
Common mistakes and limits of planning
Over-planning is the first mistake: long or inflexible plans can go stale before they are executed. A plan written at the start of a task can go stale if the world changes while the agent works through it. When it does — a tool returns something unexpected, a page has changed, a file is missing — the plan can become a liability, and the agent may spend steps undoing its own assumptions.
Brittle plans are the second: a plan that assumes the world will not change can break at the first surprise, and the benchmarks show it — agents getting stuck in loops, or aimlessly exploring an environment instead of progressing toward the goal. Replanning is the defence, but it is a capability under suitable conditions, not a guarantee — sometimes the agent cannot recover, and the loop ends with the agent giving up or escalating to a human.
Planning without grounding is the third: plans that name tools or facts that do not exist. A confident plan that calls a tool that is not available, or cites a fact that is not true, can fail at the first step.
And there are cases where planning alone cannot help: unclear goals, missing tools, or unobservable state. Clarification, information gathering, or alternative available tools may remove some of these obstacles — the missing-receipt example earlier shows an agent asking for what it needs instead of stopping. But where essential information, capabilities, or feedback remain unavailable, planning alone cannot overcome that: no amount of planning turns such a task into a solvable one.
Planning is what makes behaviour goal-directed rather than reactive — it lets a system carry a goal across many steps and decide for itself what to do next. But planning on its own is not autonomy. The system that separates a true autonomous agent from a chatbot is the one that plans and executes: it forms the plan, carries it out, watches the results, and decides whether to continue, replan, or stop, without a human prompting each step.
