Tool Calling

by kirupa (https://www.kirupa.com/me/index.htm) | filed under AI Fundamentals (https://www.kirupa.com/ai_fundamentals/index.htm)

A language model knows a lot, but it's frozen in time and sealed in a box. It doesn't know what's on the shelves of your bakery right now, what the weather is doing, or which tickets are open in your bug tracker. It can't do anything either. It can't send an email, place an order, or run a database query. All it can do is produce text.

So how do AI assistants look things up and take actions? They use a pattern called tool calling (aka function calling). The surprising part is that the model never runs anything itself. It writes a precisely formatted request, and our code does the actual work. Let's see how that plays out, step by step.

Onwards!

A Question the Model Can't Answer

Our help desk bot Biscuit from The Anatomy of a Chat Request (https://www.kirupa.com/ai_fundamentals/anatomy_of_a_chat_request.htm) is chatting with a customer named Kenji, who types:

Do you have any cinnamon rolls left?

Here is how that exchange looks in Biscuit's chat window once tool calling is wired up, with the stock check happening between the question and the answer:

[Image: The Biscuit chat window on the Crumb & Co. bakery website. Kenji asks, Do you have any cinnamon rolls left? Between the two messages, a small dashed note with a tool icon shows Biscuit's tool step: check_stock, which comes back with 6 left. Biscuit replies, Yes! We have 6 left.] (https://www.kirupa.com/ai_fundamentals/images/tool_calling_chat_cinnamon_rolls.png)

The model has no idea. The bakery's stock changes every few minutes, and it certainly wasn't in the model's training data. If we just send Kenji's question to the model, the best it can do is guess or apologize. A guess is worse, because it might sound confident.

What we want is for the model to say, in effect, I need to check the stock system, and here's exactly what to look up. Then something that actually has access to the stock system does the looking.

Describing a Tool

The first step happens before Kenji types anything. Along with our system prompt (https://www.kirupa.com/ai_fundamentals/system_prompts_vs_user_prompts.htm), we send the model a list of tools it's allowed to ask for. Each tool has a name, a plain-English description, and a JSON schema (https://www.kirupa.com/ai_fundamentals/structured_outputs_json_schema.htm) for its arguments:

{
  "name": "check_stock",
  "description": "Look up how many of a bakery item Crumb & Co. has on the shelf right now.",
  "parameters": {
    "type": "object",
    "properties": {
      "product": {
        "type": "string",
        "enum": ["cinnamon_roll", "sourdough_loaf", "baguette", "croissant"]
      }
    },
    "required": ["product"]
  }
}

Notice that this is only a description. There's no code in it and no address for our stock database. The model reads the name and description the same way it reads everything else, as text, and decides whether this tool would help with the question in front of it.

The Round Trip

Here's the whole exchange between our code, the model, and the bakery's stock system:

[Image: Three lanes: our code, the model, and the stock system. 1: our code sends the customer's question and the tool list to the model. 2: the model replies with check_stock(cinnamon_roll). 3: our code looks it up in the stock system. 4: the stock system answers 6 left. 5: our code sends the tool result back to the model. 6: the model writes the answer, Yes! We have 6 left.] (https://www.kirupa.com/ai_fundamentals/images/tool_calling_round_trip.png)

Let's walk through it. In step 1, our code sends Kenji's question along with the tool list. In step 2, instead of answering in prose, the model responds with a tool call, which is a structured message that names the tool and fills in its arguments:

{
  "type": "tool_call",
  "name": "check_stock",
  "arguments": { "product": "cinnamon_roll" }
}

The model turned Kenji's cinnamon rolls into cinnamon_roll, which is one of the exact values our schema allows. That translation from loose human words into precise arguments is the model's real job here. (The exact shape of this message differs a bit from provider to provider, but every major API has the same pieces.)

In step 3, our code receives the tool call, checks it, and runs the real stock lookup. The model isn't involved in this step at all. In step 4, the stock system says there are 6 left. In step 5, our code adds that result to the conversation as a tool result and calls the model again. Now the model has what it needs, so in step 6 it writes a normal reply: Yes! We have 6 left.

From Kenji's side, this was one question and one answer. Behind the scenes, it was two calls to the model with our code doing the real work in the middle.

The Model Only Writes Notes

A good way to picture this is to imagine the model sitting in a room with no windows and no phone. It can't leave the room. All it can do is pass notes out. On the other side is a clerk, which is our code, who reads each note, walks over to the shelves, and passes the answer back in:

[Image: The model sits inside a closed room and can only pass notes out. Its note, check_stock(...), goes out to our code, drawn as a clerk. The clerk does the real lookup and passes back a note that says 6 left.] (https://www.kirupa.com/ai_fundamentals/images/tool_calling_notes.png)

This picture explains a lot about how tool calling behaves. The model can only ask for tools we listed. If its note is malformed, or asks for a product that doesn't exist, our clerk can refuse. And if the clerk is careless and does whatever any note says, that's a bug in our code. All the model did was write a note.

More Than One Call

Real conversations often need more than one tool. Suppose Kenji follows up with Great, can you hold two for me? I'll be there at 3. We might give the model a second tool, hold_item(product, quantity, pickup_time).

Now the model may chain calls together: check the stock, place the hold, and then confirm. Each tool call goes through the same round trip, and our code keeps looping until the model replies with plain text instead of another tool call:

[Image: A loop of three boxes: the model decides, our code runs the tool, and the result goes back into the conversation. An arrow returns from the result to the model, repeating up to a step limit. When the model replies with plain text and no tool call, the loop exits and the answer is done.] (https://www.kirupa.com/ai_fundamentals/images/tool_calling_loop.png)

There are a couple of practical details to know about this loop. Many models can ask for several tools in a single turn. If Kenji asked about cinnamon rolls, croissants, and baguettes all at once, the model might request three check_stock calls together, so our code should be ready to run them side by side. We also always put a ceiling on the number of rounds, so a confused model can't loop forever. That ceiling becomes a big deal once a model is running long, multi-step tasks on its own, which is what people usually mean when they say agent, and Agent Step Limits and Stop Conditions (https://www.kirupa.com/ai_fundamentals/agent_step_limits.htm) covers it in detail.

Being a Careful Clerk

Since our code does the real work, our code is where the safety lives. The model picks arguments based on text it read, and some of that text came from customers or from documents we don't control. A few habits help a lot:

- Check every argument before running anything. Is the quantity a sensible number? Does the bakery actually have that many on the shelf?
- Give each tool only the access it needs. check_stock should be able to read the stock system and nothing else.
- Treat tools that change things differently from tools that only read. Placing a hold, issuing a refund, or sending an email deserves a confirmation from a person when it costs money or is hard to undo (see Human Approval for Risky Actions (https://www.kirupa.com/ai_fundamentals/human_approval_for_risky_actions.htm)).
- Send back clear errors instead of crashing. A tool result like {"error": "only 6 cinnamon rolls left"} lets the model explain the problem or adjust on the next round.

Tool descriptions matter too. The model picks a tool and fills in its arguments mostly from the name, the description, and the schema. A vague description like inventory function leads to vague calls. Look up how many of a bakery item Crumb & Co. has on the shelf right now tells the model exactly when the tool is useful.

Note: What about MCP?
 You'll probably run into the Model Context Protocol (https://www.anthropic.com/news/model-context-protocol) (MCP), an open standard that Anthropic introduced in November 2024 for packaging tools so that many different AI apps can use them. MCP changes how tools are described and connected. It doesn't change the round trip we just walked through. The model still writes a request, and some code still runs it.

Conclusion

Tool calling lets a model look things up and take actions without ever running anything itself. We describe each tool with a name, a description, and a schema. The model responds with a structured request that fills in the arguments. Our code runs the real function and sends the result back, and the model uses it to write its answer, looping as many times as the task needs.

Because the model only writes notes, everything that actually touches the world goes through our code. This is good news, because it means validation, permissions, and confirmations all live in code we control.

So far, we've treated the model as one thing. In practice, we get to choose between small, fast models and large, capable ones, and not every job needs the biggest one. That choice is what Small vs. Large Model Trade-offs (https://www.kirupa.com/ai_fundamentals/small_vs_large_models.htm) is about.