AI agent error handling is what separates an agent that fails safely from one that fails silently. A chatbot that gives a wrong answer produces one bad paragraph. An agent that acts — calling tools, writing records, sending messages — can turn one bad step into a chain of wrong actions, each built on the last. This article explains how agents handle errors and exceptions: how they notice something has gone wrong, what they try next, and when they should stop and hand over to a human.
The central principle, stated plainly in LangChain’s fault-tolerance documentation, is that not all errors should be handled the same way. Transient failures should be retried automatically, errors the language model can recover from should be fed back to the model, and errors that need human input should pause the agent. Everything in this article follows from that classification.
What counts as an error in an agent system
Agent failures fall into three broad classes, and each one needs a different response. Confusing them leads to mismatched responses: a failure retried when it should have been escalated, or escalated when a retry would have fixed it.
Tool failures are the most familiar. An API returns an error, a request times out, a rate limit is hit, or a credential lacks permission for the requested action. These are failures of the outside world, and they usually arrive with a machine-readable signal: a status code, an error payload, or a timeout.
Model failures are unique to language-model agents. The model may produce output that is malformed, missing a required field, or simply wrong — including hallucinated tool arguments, such as a plausible-looking identifier that does not exist. The output may look fluent and confident while being unusable or incorrect.
State and logic failures are the hardest to see. The agent loses context, repeats a step it already completed, gets contradictory results from two tools, or leaves a partial side effect behind — for example, a record created but never confirmed. These failures rarely announce themselves with an error message, which is why detection matters as much as recovery.
How agents detect errors
An agent cannot recover from a failure it never notices, so detection is the first job of any error-handling design.
The easiest failures to detect are the ones tools report themselves. A well-designed tool returns a structured result that distinguishes success from failure, with an error code and a message the agent can reason about. In OpenAI’s agent API, for example, a failed function call is reported with success set to false plus an error message the agent can use. Contrast that with a tool that returns free text like “something went wrong”: the agent must guess what happened, and guesses compound.
Output validation is the second detection layer. Before the agent acts on a model’s output, the output is checked against a schema: required fields present, types correct, values within range. Schema-constrained generation, such as OpenAI’s Structured Outputs, makes malformed arguments structurally unlikely by constraining what the model can emit. When validation fails, the failure is caught before it becomes an action.
The third layer is observability. Tracing, logging, and step-level checkpoints make failures visible to the people running the system. Production agent failures often do not arrive as clean errors at all: as Splunk’s incident-response guidance notes, production AI systems rarely fail with a clear server error and instead degrade quietly, producing fluent, confident responses that ignore safety instructions or hallucinate policy. Without traces showing each step, tool call, and result, that kind of failure is much harder to diagnose and reconstruct, even though output guardrails and other checks may still catch some of them.
Retry, fallback and recovery strategies for AI agent error handling
Once a failure is detected and classified, the agent picks a route based on the failure type and the safety of the next action: retry, model repair, fallback, or immediate escalation. Some failures go straight to a pause, with no retries or fallbacks first, and the limits on every route matter.
Retries are for transient failures only — network timeouts, rate limits, temporary provider errors. A retry should use exponential backoff, waiting longer after each attempt, and it must be bounded. LangChain’s retry middleware, for instance, defaults to three total attempts. An unbounded retry is itself a failure mode: an agent stuck retrying a permanently failing call burns tokens, holds up the user, and may repeat side effects. Scope matters too: retrying a timed-out web search may help, while retrying a deterministic failure, such as a file read of a missing file, will not.
When retries are exhausted, the failure is handed back to the model as an error message it can reason about, rather than swallowed. The model may then choose a different approach: rephrase a query, use a different tool, or ask the user for the missing piece. If the primary model provider is down entirely, the system can fall back to an alternative model. And if no recovery path works, the agent can return a degraded but safe answer — a partial result clearly labelled as partial — instead of a confident fabrication.
Checkpointing and idempotency protect the retry path. Before an irreversible step, the agent records its state so a resumed run does not repeat completed work — but a checkpoint alone does not establish whether the external action succeeded. Tools with side effects should be idempotent, meaning that running them twice with the same request has the same effect as running them once, and every action should carry durable call and result tracking, keyed by session, turn, and call identifier. When an action’s outcome is unknown — a timeout with no receipt, a result never saved — the agent must check the external outcome before repeating the action; if the outcome cannot be established, it must stop, preserve the evidence, and escalate rather than guess. A payment, a booking, or a record creation that is retried blindly can be applied twice, and that second application is a new failure caused by the recovery itself.
{
"success": false,
"error": "rate_limit_exceeded",
"message": "The search API returned HTTP 429. Retry after a short backoff, or narrow the query and try a different source.",
"retryable": true
}The payload above shows the shape of a useful tool error: a machine-readable code, a human-readable message, and an explicit statement of whether retrying makes sense. That last field is what lets the agent choose correctly instead of guessing.
Handling bad model output
Bad model output gets its own section because it is the failure class other software does not have. The model is both the reasoning engine and a source of errors, so the system must check its work.
The standard pattern is a validation-and-repair loop. The output is validated against its schema; if validation fails, the error is spelled out and sent back to the model, which tries again with the problem named. This works because many model errors are fixable once named: a missing field or a wrong type. An argument that passes its schema can still refer to nothing real, so record existence and other semantic constraints need separate checks beyond schema validation. Returning the exact validation error, rather than a generic “try again”, gives the model a better chance of correcting the output, but it guarantees nothing: the loop may still fail, which is why it needs an attempt limit and a failure path.
Prevention beats repair where it is available. Schema-constrained generation reduces malformed output by constraining what the model can emit, and validating tool arguments before execution catches type errors and missing fields before they reach a real system. Schema checks alone do not prevent injection-style misuse or prove that a referenced record exists; those need separate input and existence checks.
Repair loops need their own limit. A loop that retries indefinitely, each time producing a slightly different wrong answer, is a runaway process wearing a helpful disguise. Cap the number of repair attempts, and when the cap is reached, treat the output as a failure: fall back, return a partial result, or escalate. Burning tokens chasing a valid answer that is not coming helps nobody.
When to stop and escalate to a human
Escalation is not a failure of automation. It is the boundary that makes automation trustworthy: the point where the agent admits it cannot proceed safely and asks a person to decide.
Escalate on repeated failure. If retries and repair loops have been exhausted, further attempts are unlikely to help and increasingly likely to cause harm. Escalate on high stakes: any irreversible or outward-facing action — sending a message to a customer, moving money, deleting a record, changing access — deserves human approval before it happens, not an apology afterwards. Escalate on ambiguity: instructions that are unclear, contradictory tool results, or uncertainty about what the user actually wants. And escalate on safety-relevant uncertainty, where proceeding could leak data or violate policy.
A good handoff contains three things: what was tried, what failed, and the state left behind. “The payment step failed” is not a handoff. “The payment step timed out twice, the order record was created but not confirmed, and no charge receipt was received, so the charge outcome was checked against the payment provider before any retry was considered” is a handoff a human can act on. While waiting, the agent should return a partial result or a safe default, clearly labelled, rather than leaving the user with nothing or with a guess presented as fact.
The practical rule for deciding what needs approval is reversibility. Cheap, reversible reads can proceed automatically. Anything destructive, public, financial, or hard to undo pauses for a person, and the person approves the exact action — the precise message, the precise record change — not a vague summary of intent.
Designing for graceful degradation
Individual tactics are not enough. A production agent needs system-level patterns, borrowed from distributed systems, that keep the whole thing useful when parts of it fail.
Timeouts bound every wait: no tool call, model call, or user wait should last forever. Circuit breakers stop calling a dependency that is failing repeatedly, giving it time to recover instead of hammering it. Step, token, and spend caps bound the blast radius of a runaway agent: a cap on tool calls bounds tool activity, while separate model-call, token, and spend limits bound the rest of the run. Reaching any limit must route through an explicitly implemented stop and handoff path; the cap itself does not escalate anything. Repetition detection adds a finer signal: repeated identical tool calls, evaluated alongside retry budgets and whether the run is making progress, can indicate a loop — but bounded retries may legitimately repeat arguments, so repetition alone does not prove a runaway.
Idempotent tools and confirmation before irreversible actions, covered above, are design decisions made when the tools are built, not when the failure happens. So is least privilege: a tool that only needs to read should not be able to write, and a database credential should be scoped to the tables the agent actually needs. A failure in a least-privilege system is smaller by construction.
Finally, test failure modes deliberately. Fault-injection suites — a tool that always times out, a model that always returns malformed output, a provider that goes down mid-run — reveal whether the recovery paths work before real users depend on them. An error-handling design that has never been exercised under failure is a hypothesis, not a safeguard.
Common mistakes and an error-handling checklist
Three mistakes recur in agent error handling. The first is treating every error as retryable: retrying a deterministic failure, an authentication failure, or a logically impossible request wastes resources and delays the correct response, which is usually to stop and report. The second is swallowing errors and continuing on stale state, so the agent reasons confidently on top of a result it never actually got. The third is letting the model decide silently whether a failure matters, with no rule, no log, and no escalation path — the failure disappears into the conversation and resurfaces later as a wrong action.
Use this checklist on any agent you build or review. Classify every failure before choosing a response. Detect failures with structured tool results, output validation, and tracing — never rely on the model noticing. Bound every retry, repair loop, and run with explicit caps. Choose the recovery route by failure type: retry, repair, or fallback where the failure allows it, and pause or escalate immediately where it does not, including any action whose outcome cannot be established. Escalate repeated, high-stakes, ambiguous, or safety-relevant failures to a human with a complete handoff. And log enough context to investigate failures, since missing traces make diagnosis and reconstruction much harder.
Done well, AI agent error handling is unglamorous and mostly invisible — which is exactly the point. The user sees an agent that sometimes pauses, sometimes asks, and sometimes returns a partial answer with its limits stated. What they do not see is the chain of silent failures that good error handling prevented.
