Controlling ai agent api costs is the difference between an agent that is useful in production and an agent that quietly spends a month’s budget in an afternoon. An agent does not make one call and stop: it decides, calls a tool, reads the result, decides again, and repeats until it believes the task is done, with every turn billed, usually per token, with separate rates for input, cached input and output. When the loop has no limit, no budget and no one watching, a small misunderstanding can turn into hundreds of paid calls before anyone notices. This article sets out the control stack that prevents that: budgets set before deployment, hard stops inside the loop, smaller inputs, live monitoring, and retry rules that do not pay twice for the same failure.
Why agent API bills run away
Four mechanisms do most of the damage, and they often combine. First, an unbounded loop: an agent that retries, re-plans or re-reads until it succeeds can multiply calls per task, especially when it misunderstands a tool result and tries the same approach again. Second, context growth: history and tool outputs appended each turn make later calls carry larger inputs than early ones. Third, tool-call amplification: one agent step can trigger several model calls plus external API calls, each billed separately. Fourth, a retry storm, where naive retries on failures pay repeatedly for work that never completes. None of these requires a bug in the model. They are what happens when a system that decides at runtime how many calls to make is given no boundary.
Cost is driven by the tail, not the average: most runs may be cheap, while the runs that hurt are the ones that loop or use a far more expensive model than the step requires. That is why the controls below are layered: a budget catches what a loop limit misses, a loop limit catches what a budget allows too slowly, and monitoring catches what both miss.
Set a budget before you deploy
Set budgets at three levels, because each catches a different failure. A per-run budget caps a single task, so one looping run cannot consume the month. A per-user budget stops one heavy user from crowding out everyone else. A per-day or per-month budget caps total exposure. Set all three before the agent runs unsupervised, and decide in advance what happens at each limit: stop the run, degrade to a cheaper model, or queue the task for human approval. A limit with no defined action gets ignored at the worst moment.
Pick the starting numbers from measurement, not from a guess. Run the agent on representative tasks, record the cost of a typical run and of an expensive run, and set the per-run budget above the typical run with room for genuinely hard tasks. One cost guide recommends hard caps on experimental projects and alert thresholds on production ones: experiments can stop hard, while production should warn before it stops. Provider tooling helps as a backstop — OpenAI supports project-level spend limits and rate limits, and Anthropic offers a console spend limit — but enforce your own per-run and per-user budgets in your own code, because only your code knows which run, user and step a call belongs to.
A budget policy is easier to review when it is written down as data rather than scattered through code. The example below is a complete JSON policy object: it states the three budget levels, the action at each limit, and an alert threshold set below the hard limit so a human or an automated action can intervene first.
{
"policy_name": "agent-run-budget",
"budgets": {
"per_run_usd": 0.5,
"per_user_per_day_usd": 5.0,
"per_month_usd": 500.0
},
"alert_at_fraction_of_budget": 0.8,
"action_at_limit": "stop_run_and_notify",
"degraded_model": "small-model-for-routine-steps"
}The numbers in that example are illustrative, not a recommendation for any particular workload: replace them with figures measured from your own runs. The structure is the point. Every budget level, the alert point and the action at the limit are explicit, so a reviewer can see exactly what the system will do when money runs out.
Put hard stops inside the agent loop
Budgets are measured in money, but loops are measured in steps, and you need both. Put hard stops inside the agent loop that bound a single run regardless of what the model decides. Cap the number of model decisions or tool executions, cap the maximum tokens per run, and set a wall-clock timeout. These are independent limits on purpose: a step cap stops a loop that makes cheap calls forever, a token cap stops a run whose calls keep growing, and a timeout stops a run that is stuck waiting on slow tools.
Frameworks give you a starting point, not a finished policy. In LangGraph, the legacy AgentExecutor maximum iteration count maps to a recursion limit counted in super-steps, and in LangGraph.js that default is 25 super-steps. Because one tool round trip costs two super-steps, that default buys roughly a dozen tool calls, not 25. Treat the framework default as a safety net, set your own explicit step, token and time limits on top of it, and check the default for your own runtime rather than assuming the JavaScript and Python defaults match. Define stopping conditions too: success, failure and escalation states should be explicit, so the agent can report a partial result with an explanation instead of looping until a limit kills it.
Progress checks belong alongside the caps: detect repeated tool calls with identical arguments or unchanged state, and stop or escalate when the agent repeats itself. Raising the iteration ceiling when a loop appears only delays the crash while compounding the cost. When a run hits a limit, log which limit fired and what the run had already spent, so a stopped run becomes a fixable bug instead of a mystery.
Control the inputs that drive ai agent api costs
Every token you do not send is a token you do not pay for. The most reliable lever is prompt caching: providers store the processed beginning of a prompt, so a repeated prefix such as a system prompt or tool definitions is not reprocessed every turn. Cache reads cost about a tenth of the standard input price on Anthropic, OpenAI recent models and Gemini, while cache writes carry a premium of about 25 percent on Anthropic. Caching is prefix-only, so any change early in the prompt breaks reuse, and Anthropic enforces minimum token thresholds before a cache entry is created, with a default lifetime of about five minutes that refreshes on each hit. Put stable content first, keep it unchanged, and confirm in the response usage fields that caching is actually hitting.
Context management is the second lever. Prune or summarise history instead of appending everything forever, truncate tool results to the part the agent actually needs, and avoid stuffing whole documents into context when a short excerpt would do. A step where input tokens jump sharply usually means a large document or full history was stuffed into context, and that jump is visible in traces long before the invoice arrives. The third lever is model routing: send routine steps such as classification and simple extraction to a cheaper model, and reserve the strongest model for steps where it changes the outcome. Output tokens typically cost several times input tokens, so capping output length on routine steps also pays off.
Monitor spend while runs are live
Monitoring turns a budget from a hope into a control. Record token counts on every model span and aggregate them per run, per user, per tool and per step. Provider invoices are aggregated by account and cannot tell you which run or step drove a spike, so your own traces must carry that attribution. Each API response reports its usage, including input, output and cache token counts, so log those figures with the model name, run identifier and step. Managed observability tools do the same job if you prefer them to your own logging, but the requirement is the same: cost must be sliceable by run and step, not just visible as a monthly total.
Set alert thresholds below the hard budget, and attach each alert to a named responder or an automated action. An alert that nobody owns is noise that teaches a team to ignore the next one. When a spike fires, open the most expensive run and read it top to bottom: which step grew, which tool was called repeatedly, and whether the run had far more model spans than the happy path. A run with far more spans than normal is usually a loop that did not terminate; a step where input tokens jump is usually a context problem. Fix the mechanism the trace shows. Finally, compare cost across agent versions: a prompt change that improves quality while doubling cost is a trade-off to make deliberately, not a surprise at billing time.
Handle retries and failures without paying twice
Failures are where small bills become large ones, because the natural response to a failed call is to try again. Bounded retries are the fix. Retry only failures likely to be transient, such as rate limits and temporary server errors, with a small fixed attempt count and exponential backoff plus jitter so retries do not arrive in a synchronised storm. The example below is a complete, runnable Python sketch: a retry helper with three attempts and backoff capped at eight seconds, demonstrated with a function that fails twice before succeeding. A production client would also respect rate-limit headers and log every attempt, and would only pass retry-safe operations to this helper.
import random
import time
def call_with_retry(call, max_attempts=3):
"""Retry only retry-safe operations, and only on transient errors.
call must be safe to run more than once. Any exception that is not
a transient TimeoutError or ConnectionError propagates immediately.
"""
for attempt in range(max_attempts):
try:
return call()
except (TimeoutError, ConnectionError):
if attempt == max_attempts - 1:
raise
delay = min(2 ** attempt, 8) + random.uniform(0, 0.25)
time.sleep(delay)
attempts = {"count": 0}
def flaky_call():
attempts["count"] += 1
if attempts["count"] < 3:
raise TimeoutError("temporary failure")
return "success"
result = call_with_retry(flaky_call)Two further rules prevent paying twice. First, make tool calls with side effects idempotent where the tool allows it, so a retry does not repeat a paid action such as sending a message or creating a record. If the tool itself is not idempotent, do not rely on a local completed-call record alone: a remote action can succeed while its response, or the local write recording it, fails, leaving the outcome uncertain. Use a provider-supported idempotency key where one exists, or reconcile the remote outcome (for example, by listing or reading back the created record) before retrying; when neither is available, stop or escalate an ambiguous state-changing call instead of retrying it blindly. Second, use a circuit breaker for a tool or route that keeps failing: pause that route instead of hammering it, and fall back to a cheaper route or a graceful failure. A retry policy that respects these rules turns a transient failure into a small delay; one that ignores them turns it into a bill.
When a bill spikes anyway: stop, diagnose, fix
Even with every control in place, a spike will eventually happen, so have a playbook ready. First, pause the agent or affected route so the bleeding stops. Second, sort runs by cost and open the most expensive one to identify the loop, tool or context jump behind it. Third, fix the mechanism, not the symptom: a loop needs a progress check or tighter step cap, context growth needs pruning, a retry storm needs a tighter retry policy, and a routing mistake needs a cheaper model. Re-run the same task and confirm the cost fell before restoring full traffic.
The common mistakes recur: limits set but never enforced; alerts with no owner and no action, which train teams to ignore the warning that matters; budgets covering only the model while tool APIs and retrieval services bill separately; and per-run budgets never measured, so they sit far above any real run and never fire. Run a short pre-launch checklist before any agent goes unsupervised: budgets at run, user and monthly levels with a defined action at each limit; step, token and time caps inside the loop; context pruning and caching confirmed in usage data; per-run cost attribution logging; bounded retries with backoff; and an alert threshold below the hard budget with a named responder. An agent with that checklist, enforced and monitored in practice rather than merely configured, is far less likely to fail expensively and silently — but no checklist removes that risk entirely, and only measured spend and working alerts can show whether the controls are actually holding.
