What Is Function Calling in AI? How LLMs Use Tools in 2026

What is function calling in AI explained

Ask a chatbot what the weather is right now and, on its own, it can't actually check. A language model only generates text based on patterns it learned during training. It has no built-in way to look at a live weather service, query your database, or book a meeting.

Function calling is the mechanism that closes that gap. It is the reason an AI assistant can look up an order, search the web, or edit a file instead of just talking about it. If you are building with AI, or just trying to understand how AI agents work, this is one of the most important concepts to get right.

Quick answer:

Function calling (also called tool calling) lets an AI model request an action from your software instead of only writing text. You describe your tools to the model, the model replies with the tool name and the arguments it wants to use, your code runs the tool, and the result goes back to the model so it can write the final answer. The model never runs the code itself.

What Is Function Calling in AI?

Function calling is a structured way for a language model to ask for something to be done. Instead of replying with a paragraph, the model replies with a small piece of structured data that says, in effect, "please run this function with these inputs."

OpenAI's documentation describes it as a way for models to interface with external systems and access data outside their training data. Anthropic's documentation calls the same idea tool use and treats the two names as equivalent. In everyday developer conversation, "function calling" and "tool calling" mean the same thing.

One point is easy to miss and matters a lot: the model does not execute anything. It only decides which tool to use and with what arguments. Your application does the actual work. That detail shapes everything else in this guide, especially the security side.

How the Function Calling Loop Works

Every function calling setup follows the same basic loop, no matter which provider you use:

  1. You define the tools. You send the model a list of available tools, each with a name, a description, and a schema for its inputs.
  2. The user asks something. The model reads the request and decides whether a tool would help.
  3. The model requests a tool call. If it needs a tool, it returns the tool name and arguments instead of a normal answer.
  4. Your code runs the tool. Your application executes the function, such as calling an API or querying a database.
  5. You send the result back. The model reads the result and either writes the final answer or asks for another tool call.

Step 5 is where agents get interesting. The model can chain several tool calls in a row, checking a result and deciding what to do next. With Anthropic's API, for example, your code keeps looping while the response signals that a tool was requested, and stops when the model signals it is finished. That loop, repeated with good judgment, is the core of what people call an AI agent.

What a Tool Definition Looks Like

A tool definition is basically a small contract. Here is a simplified example. Exact field names vary a little between providers (some use "parameters," Anthropic uses "input_schema"), but the idea is the same everywhere:

{
  "name": "get_order_status",
  "description": "Look up the shipping status of a customer order by order ID. Use this when the user asks where their order is.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "The order ID, for example A-1042"
      }
    },
    "required": ["order_id"]
  }
}

When a user asks "Where is order A-1042?", the model does not answer from memory. It returns something like this:

{
  "name": "get_order_status",
  "arguments": { "order_id": "A-1042" }
}

Your code then calls your real shipping system, gets the status, and sends it back to the model. Only then does the user see a normal sentence like "Your order shipped yesterday and arrives Friday."

Notice how much work the description does. The model decides whether to use the tool based mostly on that text. A vague description leads to missed or wrong tool calls.

Function Calling vs Tool Calling vs Structured Outputs

These terms get mixed up, so here is a simple way to separate them.

  • Function calling and tool calling are the same idea: the model asks your software to do something.
  • Structured outputs is a related feature where the model's reply is forced to match a JSON schema, which is useful when you just want clean data back.

The two overlap. OpenAI's strict option for function definitions is built on its structured outputs feature. According to OpenAI's documentation, setting strict to true makes function calls reliably follow your schema instead of being best effort, and OpenAI recommends always enabling it. Even so, validating arguments on your side before running anything is still good practice.

Client Tools vs Server Tools

Not every tool runs in your code. Anthropic's documentation draws a useful line between two kinds:

Feature Client tools Server tools
Who runs it Your application The provider's infrastructure
Your job Run the code and send the result back Just read the result
Typical examples Your database, internal APIs, file editing Web search, code sandbox, web fetch
Control level Full control, full responsibility Simpler, but less customizable

Client tools give you complete control, but error handling is your job. Server tools are easier to set up, and many real agents mix both: web search from the provider, plus your own database queries.

Function Calling vs MCP

If you have read about the Model Context Protocol, you may wonder how it fits in. They work at different layers. Function calling is how a model asks for a tool to be used. MCP is a standard way to package, describe, and discover tools so you do not have to write a custom integration for every service. An MCP server exposes tools, and the model still ends up making tool calls to use them.

So you do not need MCP to use function calling. You can define tools directly in your API request. MCP becomes valuable when you want to reuse the same tools across many apps. If you want the full picture, our guide on what MCP is and how it works covers that side in detail.

Why Tool Design Matters More Than You Think

Here is the part most beginner tutorials skip. Getting the loop working is easy. Getting a model to pick the right tool, with the right arguments, reliably, is the hard part.

Anthropic's engineering team has shared some telling numbers. In one example setup with five connected services (GitHub, Slack, Sentry, Grafana, and Splunk), the tool definitions alone came to 58 tools and roughly 55,000 tokens, before the conversation even started. They also reported seeing tool definitions reach 134,000 tokens before optimization. That is a huge chunk of the model's working memory spent just describing what it could do.

Their fix was a Tool Search approach, where the model loads only the tools it needs instead of everything upfront. On Anthropic's own internal evaluations of MCP-style tool use, accuracy for one model rose from 49% to 74%, and for another from 79.5% to 88.1%. Keep in mind these are the vendor's own tests on their own setup, so treat them as a strong signal rather than a universal guarantee. The practical takeaway is clear, though: more tools do not automatically make an agent better. Anthropic suggests looking at this kind of approach once tool definitions pass about 10,000 tokens or when you see tool selection mistakes.

A few design habits make a real difference:

  • Write descriptions for the model, not for humans. Say what the tool does, when to use it, and when not to.
  • Build tools around tasks, not raw API endpoints. One tool that checks availability and books a slot beats three separate tools the model has to chain correctly every time.
  • Return only useful information. A tool that dumps thousands of rows into the conversation wastes context and confuses the model. Filtering, pagination, and truncation help.
  • Use clear, distinct names. Two tools with near-identical names or descriptions are a common source of wrong calls.
  • Keep the starting tool list small. OpenAI's guidance suggests aiming for fewer than 20 functions available at the start of a turn, though it describes this as a soft suggestion rather than a hard rule.

This connects closely to a bigger idea: what you put in front of the model shapes how well it performs. We cover that in Context Engineering Explained, and tool definitions are a major part of it.

The Security Side of Function Calling

Because the model only suggests actions and your code performs them, your code is the safety layer. That is good news, since you stay in control. It is also a responsibility, since a model's output can be wrong, ambiguous, or manipulated.

OWASP, the well-known application security organization, lists this under "Excessive Agency" in its Top 10 for LLM applications. It describes the root causes as excessive functionality, excessive permissions, and excessive autonomy. In plain terms:

  • Excessive functionality: giving the model tools it does not need.
  • Excessive permissions: letting a tool do more than its job requires. OWASP's own example is an extension meant only to read data that connects with an identity that can also update and delete it.
  • Excessive autonomy: letting the model take high-impact actions without a human check. A simple example is requiring a person to review and press send on every email the AI drafts.

The safest habits are the boring ones: give each tool the minimum access it needs (read-only wherever possible), validate every argument before running it, and require human approval for anything destructive or irreversible, such as deleting data, spending money, or sending messages.

There is another risk worth knowing. If a tool returns text from the outside world, like a web page or an email, that text can contain hidden instructions aimed at the model. That is called prompt injection, and it becomes more dangerous the more powerful your tools are. Our guide to prompt injection and AI coding tools explains how it works and how to reduce the risk.

Common Mistakes Developers Make

  • Vague tool descriptions. "Gets data" tells the model almost nothing.
  • Too many overlapping tools. Fewer, clearer tools usually beat a giant catalog.
  • Trusting arguments blindly. Always validate what the model sends before running it.
  • Returning huge raw outputs. Summarize or filter before sending results back.
  • Skipping error handling. Tools fail. Return clear error messages so the model can recover or explain the problem.
  • No limit on the loop. Set a maximum number of tool calls so a confused agent cannot loop forever.
  • Over-permissioned tools. If read access is enough, do not grant write access.

Frequently Asked Questions

Does the AI model actually run the function?

No. The model only outputs the tool name and arguments. Your application, or in the case of server tools the provider's infrastructure, runs the function and returns the result.

Is function calling the same as tool calling?

In practice, yes. OpenAI and Anthropic both treat the terms as interchangeable. "Function calling" is the older and more widely searched name, while "tool calling" and "tool use" are increasingly common.

Do I need MCP to use function calling?

No. You can define tools directly in your API request. MCP is optional and mainly helps when you want to share tools across multiple apps or connect to existing services.

How many tools should I give a model?

There is no magic number, but fewer and clearer is the safer rule. Every tool definition uses up context, and similar-looking tools make wrong selections more likely. If your tool list grows large, look into loading tools on demand instead of all at once.

Final Thoughts

Function calling is simple at its core: describe your tools, let the model ask for one, run it yourself, and return the result. That small loop is what turns a chatbot into something that can actually do work.

The difference between a demo and a reliable product comes down to the details. Clear tool descriptions, a small and well-designed tool set, validated arguments, and tight permissions matter far more than the code that wires the loop together. Get those right and your agent will be both more capable and safer to run.

Sources and Further Reading

You Might Also Like

Comments

Popular posts from this blog

AI Evaluation Explained: How to Know If Your AI Agent Is Actually Working

Prompt Injection Explained: The Security Risk Every AI Coding Tool User Should Know

What Is Open-Source AI? Why It's Reshaping the Industry in 2026