What Tool Use Actually Is

Strip the marketing away and tool use is a structured protocol. The model does not actually “call” anything. It returns a JSON object describing what tool to invoke, with arguments it has extracted from the conversation. Your code receives that JSON, executes the real function (database query, API call, calculation, file read), and feeds the result back to the model as part of the next message. The model then continues the conversation with that tool result in its context.

This is the loop. The model is reasoning, decides it needs external information or action, emits a structured tool call request, your application runs the function, the result comes back, and the model uses it to produce the next user-facing response. The model never directly touches your database, never makes HTTP calls on its own, and cannot execute code. It only describes what should happen in a format your code can parse.

Why this matters is that it turns the model from a text generator into something closer to a controller or orchestrator. It can decide which function to call, what arguments to pass based on user intent, and how to chain multiple calls together. The model becomes a thin reasoning layer over a set of real capabilities you provide.

There are three moving parts in any tool use implementation. The first is the tool schema, which is a JSON Schema document describing the function name, description, and parameters. The second is the runtime loop, which handles sending messages, receiving tool calls, executing them, and feeding results back. The third is the model itself, which has been trained to emit tool calls in the expected format when given a tools array. Different providers (OpenAI, Anthropic, Google, Mistral, local models via Ollama) all support the pattern with minor syntax differences.

Setup and Authentication

For this walkthrough I will use the OpenAI Python SDK because the tool use surface is well documented and stable. The same patterns translate to the Anthropic SDK with a different request shape, and to most other providers.

Install the SDK and set your key. The current version of the OpenAI Python library is 1.x, and the API surface is consistent across that line.

pip install openai
export OPENAI_API_KEY=sk-...

If you are on a hosted environment, set the key in your secrets manager rather than as an environment variable on a shared box. For local development a .env file with python-dotenv is fine.

You will also need to pick a model that supports tool use. The current production-tier models from OpenAI, Anthropic, and Google all support function calling. Smaller or older models may support it inconsistently or refuse certain schemas. If you are running local models through Ollama or vLLM, check the model card to confirm function calling is in the supported features list, because many open models ship without a working tool calling mode.

First Working Example

Here is a complete, runnable example. It defines a single tool, the model, and the loop. The tool returns a fake weather report based on a city argument. This is the smallest useful tool use program you can write.

The tool definition is a JSON Schema with three fields. The type is “function”. The function has a name, a description that the model reads to decide when to call it, and a parameters object describing each argument. Always write the description as if you are explaining to a new engineer on your team when to use this function. The model uses those descriptions to make routing decisions, and vague descriptions lead to wrong calls.

The runtime loop looks like this. Send the user message plus the tools array. Read the response. If the response contains a tool call, execute the corresponding function, append the result to the message history under a tool role, and send the updated history back. Repeat until the model returns a message without a tool call. That final message is what you show to the user.

In Python with the OpenAI SDK this is roughly 30 lines. In TypeScript it is similar. The Anthropic SDK uses a slightly different shape where you pass tools in the request and the response includes a stop_reason of tool_use, then you push a tool_result block back. The mental model is identical.

A common mistake at this stage is forgetting to append the assistant message that contained the tool call before appending the tool result. Both the OpenAI and Anthropic APIs require you to replay the assistant turn that requested the tool, then the tool result turn. Skip that and you get a 400 error about message ordering. The message history is a strict alternating sequence and tool turns slot into it like any other role.

Key Settings That Matter

There are four settings most people ignore and then wonder why their tool use system misbehaves.

The first is tool_choice. This parameter controls when the model is allowed to call a tool. The default is “auto”, which lets the model decide. “required” forces at least one tool call, useful when you have a routing agent that must always dispatch somewhere. “none” forces the model to answer without tools, useful for safety rails. You can also pin a specific function with {"type": "function", "function": {"name": "exact_name"}}. Use the pinning option when you want a deterministic routing decision, because “auto” can sometimes pick the wrong tool when several look similar.

The second is description quality. The model picks between tools based on the description field. If two tools have overlapping descriptions the model will sometimes flip between them, sometimes on the same prompt. Write descriptions that describe the boundary, not just the capability. “Use this to look up current weather for a city” is fine. “Use this to look up current weather for a city, not historical data” is better. Adding negative cases to the description is a legitimate technique and the current generation of models use it well.

The third is argument validation. Even when the model emits valid JSON, the values may be wrong. A city field might be a real city or a typo, a date field might be the wrong format, a numeric field might be a string. Validate arguments against your own schema before executing the function. The model’s job is to produce plausible arguments. Your job is to decide whether they are actually usable. Treat the tool call as untrusted input even though it came from a model you control.

The fourth is the parallel tool calls setting. Most providers let the model emit multiple tool calls in a single response, and you can disable this. For workflows that need strict ordering, disable parallel calls. For workflows that benefit from batching (fetching weather for ten cities at once), leave it on. The default is on, and most production code keeps it on.

There is also a context-length consideration. Every tool definition eats tokens, and every tool result is appended to the message history. If you define 20 tools each with a 500-word description, that is real cost on every request. Trim descriptions to what the model actually needs to make a routing decision. If a tool has a parameter the model rarely fills, mark it optional in the schema with a sensible default.

Where It Shines

Tool use genuinely excels at three patterns.

The first is read-only data access with natural language interface. A model connected to a SQL tool, a search tool, and a calculator can answer ad hoc business questions without anyone writing a custom dashboard. The model writes the query, you execute it against a read replica, the result comes back, the model explains it in plain English. This is the highest-leverage use of tool use in most companies.

The second is structured extraction from messy text. Take a contract, an invoice, or a support email and ask the model to extract fields into a typed schema. You can give it a tool definition that says “use this function to record the extracted record” and it will emit a structured call. This is more reliable than asking for raw JSON because the schema acts as a contract the model has to satisfy.

The third is multi-step workflows where each step has a real side effect. A booking agent that checks availability, reserves a slot, charges a card, and sends a confirmation is a chain of tool calls. The model reasons about which step comes next based on prior results, and your code makes sure each step is atomic and reversible. This is the pattern behind most AI agent demos and it works well when the tool surface is small and well described.

In all three cases the common factor is that the model is acting as a thin reasoning layer over real capabilities. The capabilities do the work. The model decides which capability to invoke.

Where It Fails

Tool use is not magic, and there are specific failure modes you should plan for.

The model hallucinates tool names or argument keys that look right but do not exist in your schema. Providers usually reject these at the API level, but if you are running local models with a weak function calling implementation you may get a JSON object that does not match what you expected. Always validate the shape before dispatching.

The model calls the right tool with the wrong arguments in a plausible-looking way. “Get me the user’s last 10 orders” might become a call with limit=1000 and no status filter, which is technically valid but expensive. This is the failure mode that argument validation catches, and it is the one most teams underestimate.

Long tool chains degrade. After 10 to 15 tool calls the model starts losing track of the original goal, repeating calls, or fabricating results. Most production systems cap the number of tool calls per turn, and many cap the total per session. A common pattern is a hard cap of 20 calls per turn with a fallback that says “I could not complete this in one pass, want to continue”.

Cost is the other honest limitation. Each tool result that gets appended to the message history is paid for on every subsequent turn. A long agent session can run into thousands of dollars of input tokens for a single