Ask how AI agents function calling systems work and you will usually get one of two answers: a vague wave at “the model decides to call a tool,” or a pile of API specifics with no mental model. This guide is the missing middle. The core idea is simple and non-negotiable: the model never executes anything. It emits a structured request naming a function and its arguments as JSON. Your code runs that function, feeds the result back into the conversation, and asks the model what to do next. That round trip — request, execute, observe — is the entire mechanism. Everything else, from parallel calls to tool_choice modes to MCP, is machinery built around it.
This article is about the how: the exact mechanism by which a model decides to call a function, how tools are described to it, and how arguments and results flow back and forth.
The core loop: the model asks, your code acts
Ask for the weather in Boston and the model does not fetch anything — it produces a structured request to call get_weather (full request below), arriving in the API response as a tool_calls entry: the model’s decision, not an action.
Your code runs the function and appends the result — role: "tool" with the matching tool_call_id (OpenAI), or a tool_result block matched to the tool_use id (Anthropic).
You call the model again with the updated history, and it either calls another function or answers directly — request → execute → observe until the model answers with no tool call.
How you describe a tool to the model
A tool is described to the model as name + description + JSON parameter schema. This triple is the entire interface the model sees — it never sees your implementation.
{
"name": "get_weather",
"description": "Get the current weather for a city. Use this when the user asks about current conditions or temperature for a specific place.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. 'Boston, MA'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit for the result"
}
},
"required": ["location"]
}
}The description is the model’s documentation for your function, and its quality drives call accuracy — “descriptions are the model’s documentation, and careful ones are cheap insurance.”
The schema side aims at one failure mode: hallucinated arguments. Explicit enums and marked required fields are the most effective defense.
A worked example: AI agents function calling in one round trip
The mechanism in code:
from openai import OpenAI
import json
client = OpenAI()
get_weather_tool = {
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "The city and state, e.g. 'Boston, MA'"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
},
}
messages = [{"role": "user", "content": "What's the weather like in Boston?"}]
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=[get_weather_tool],
tool_choice="auto", # let the model decide whether a tool is needed
)
response_message = response.choices[0].message
print(response_message.tool_calls)
# [ChatCompletionMessageToolCall(
# id='call_abc123',
# function=Function(name='get_weather',
# arguments='{"location": "Boston", "unit": "fahrenheit"}'))]The model returned a name and a JSON string — nothing executed. Your code parses, validates, runs, and returns one matched result per call:
def get_weather(location, unit="celsius"):
"""Your real implementation: an API call, a database lookup, etc."""
return {"location": location, "unit": unit, "temp": 62, "condition": "sunny"}
ALLOWED_UNITS = {"celsius", "fahrenheit"}
def validate_weather_args(args):
"""Validate the parsed object against the declared schema."""
if not isinstance(args, dict):
return "arguments must be a JSON object"
if not isinstance(args.get("location"), str):
return "location must be a string"
if "unit" in args and args["unit"] not in ALLOWED_UNITS:
return f"unit must be one of {sorted(ALLOWED_UNITS)}"
unexpected = sorted(set(args) - {"location", "unit"})
if unexpected:
return f"unexpected arguments: {unexpected}"
return None
# tool_choice="auto" allows a direct answer with NO tool calls
tool_calls = response_message.tool_calls or []
if not tool_calls:
print("Direct answer:", response_message.content)
tool_results = []
for call in tool_calls: # execute EVERY call requested, not just the first
try:
args = json.loads(call.function.arguments) # parse, never hard-code
err = validate_weather_args(args)
if err is None and call.function.name != "get_weather":
err = f"unknown tool: {call.function.name}"
result = get_weather(**args) if err is None else None
except Exception as e:
result, err = None, f"execution failed: {e}"
tool_results.append({
"role": "tool",
"tool_call_id": call.id,
"content": json.dumps({"error": err} if err else result),
})
if tool_calls:
messages.append(response_message) # the assistant's request, verbatim
messages.extend(tool_results) # every result, matched by id
final = client.chat.completions.create(model="gpt-4o", messages=messages)
print(final.choices[0].message.content)
# "It's 62 degrees and sunny in Boston right now."Always append the assistant’s message exactly as returned, and keep the tool_call_id match — it pairs each result with its request.
The agentic loop: plan, call, observe, repeat
The single round trip scales into real agent behavior by wrapping it in a loop: each turn the model reads the history and decides whether to call another tool or answer. The loop below uses Anthropic’s API, so it needs its own client, tool definitions, and handler mapping first:
import anthropic
client = anthropic.Anthropic() # the Anthropic SDK client, separate from OpenAI's
anthropic_tools = [{
"name": "get_weather",
"description": "Get the current weather for a city.",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "The city and state"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
}]
def get_weather(location, unit="celsius"):
"""Your real implementation: an API call, a database lookup, etc."""
return {"location": location, "unit": unit, "temp": 62, "condition": "sunny"}
HANDLERS = {"get_weather": get_weather} # tool name -> your implementation
def validate_tool_call(name, tool_input):
"""Reject unknown tools and schema-violating arguments before dispatch."""
if name not in HANDLERS:
return f"unknown tool: {name}"
if not isinstance(tool_input, dict):
return "tool input must be an object"
if not isinstance(tool_input.get("location"), str):
return "location must be a string"
if "unit" in tool_input and tool_input["unit"] not in {"celsius", "fahrenheit"}:
return 'unit must be "celsius" or "fahrenheit"'
unexpected = sorted(set(tool_input) - {"location", "unit"})
if unexpected:
return f"unexpected arguments: {unexpected}"
return Nonedef run_agent(user_message, max_iterations=10):
messages = [{"role": "user", "content": user_message}]
for _ in range(max_iterations):
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=2048,
tools=anthropic_tools,
messages=messages,
)
messages.append({"role": "assistant", "content": response.content})
if response.stop_reason == "end_turn":
return "".join(b.text for b in response.content if b.type == "text")
if response.stop_reason == "max_tokens":
raise RuntimeError("truncated response: refusing to execute a partial tool call")
if response.stop_reason != "tool_use":
raise RuntimeError(f"unexpected stop reason: {response.stop_reason}")
results = []
for block in response.content:
if block.type != "tool_use":
continue
try:
err = validate_tool_call(block.name, block.input)
if err is not None:
raise ValueError(err)
out = HANDLERS[block.name](**block.input) # validated, then executed
results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": json.dumps(out),
})
except Exception as e: # tool failures become matched error results
results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": json.dumps({"error": str(e)}),
"is_error": True,
})
messages.append({"role": "user", "content": results})
raise RuntimeError(f"agent did not finish within {max_iterations} iterations")Three production details, all visible above: explicit stop_reason handling; tool exceptions become matched error results; exhausting max_iterations raises explicitly.
How models actually emit function calls
A function call is structured model output — tokens that conform to a schema. A malformed call is a generation error, not an execution error.
tool_choice controls how aggressively the model uses tools:
// OpenAI
tool_choice: "auto" // model decides (default)
tool_choice: "required" // must call at least one tool
tool_choice: { "type": "function", "function": {"name": "get_weather"} } // force this one (Chat Completions)
tool_choice: "none" // no tools, plain text answer
// Anthropic
tool_choice: { type: "auto" } // model decides (default)
tool_choice: { type: "any" } // must call some tool
tool_choice: { type: "tool", name: "get_weather" } // force this tool
tool_choice: { type: "none" } // no tools
// Google
mode: "AUTO" / "ANY" / "NONE"Parallel calls: emission vs execution. When calls are independent, the model emits all of them in a single turn — OpenAI does this by default, Anthropic returns multiple tool_use blocks, Google returns multiple functionCall parts. Emission is the model’s job; execution is yours: the host schedules the independent calls concurrently — with asyncio.gather, a thread pool, or similar — waits for every result, and returns them as one matched set. The sequential for-loops in this article’s examples are the simplest correct version, not the fastest. And calls that depend on each other’s results cannot be parallelized this way at all — they wait for a later model turn, once the earlier result is in the history.
Two portability caveats. Forced tool use is less portable than it used to be: references report 400 rejections on Anthropic’s newest models but flag this as unverified and version-dependent — re-verify for the model you ship, and prefer auto plus an explicit instruction where forcing is unreliable. Parallel emission is the default on OpenAI models; limit a turn to one call with parallel_tool_calls: false (or disable_parallel_tool_use on Claude) when strict validation matters more than latency.
What breaks: error handling and common failure modes
Bad arguments. The model may omit required parameters, send wrong types, or hallucinate values — invented IDs or emails that look plausible but do not exist. Fix it in code: declare required fields, validate against typed schemas, check identifiers against authoritative systems, use explicit enums.
Error loops. A tool error can trigger endless retries of identical arguments. Fix it two ways: instruct the agent not to retry identical arguments, and enforce a hard max-iteration guard.
Timeouts and truncation. These masquerade as model bugs: a timed-out call looks like an agent stuck in a loop, and gateway idle timeouts (~60s) can truncate long traces mid-stream.
Rate limits. Providers throttle tool-heavy loops like any other traffic — handle 429s in the orchestrator with retries and backoff.
Weak descriptions and ambiguous requests. A tool described as “Create event” gives the model nothing to work with. Let the agent ask clarifying questions instead of forcing a call from insufficient information.
Design rules for agent-ready tools
One tool, one job. Keep tools granular and deterministic.
Make side effects explicit. The model cannot tell a read from a write unless your description tells it.
Keep parameter lists small and well-typed. The LLM’s JSON is untrusted input like any other.
No hidden state. If a tool depends on state the model cannot see, the model will guess at it.
Design for the error path. Return structured error payloads, no silent fallbacks.
Make side-effecting tools idempotent. Retries and error loops mean the same call can arrive twice — a send_email that runs twice sends two emails. Operations with side effects should behave idempotently: the standard approach is a stable operation identifier that the caller keeps the same across every retry of one logical operation, so the tool can recognize a repeat and return the original result instead of re-executing. This makes retries safer, though it is one safeguard among several, not a guarantee on its own.
Where this is going: MCP and the tool ecosystem
Everything so far assumed you hand-write each tool’s schema for each model provider. That is the N×M problem: N agents times M tools, each pair needing its own bespoke integration. The Model Context Protocol (MCP) is the industry’s answer, and it has moved fast.
MCP is an open standard, published by Anthropic in November 2024, for how AI applications connect to tools and data over JSON-RPC 2.0. Build the server once and any compatible app can use it — the “USB-C of the AI world.”
Adoption: OpenAI (March 2025), Google DeepMind (April 2025), Microsoft and GitHub (May 2025), then donation to the Agentic AI Foundation under the Linux Foundation (December 2025). By March 2026: 97 million monthly SDK downloads, 10,000+ public servers.
Two caveats: MCP covers agent-to-tool connections (agent-to-agent is Google’s A2A), and security is still maturing — tool poisoning and rug-pull attacks are demonstrated, and the NSA and CISA published joint guidance in June 2026.
The through-line: the model asks, your code acts. Build the round trip well once, and every agent you ship inherits it.
