Function Calling in LLMs Explained

Introduction

Function calling in LLMs explained plainly is the feature that turned chat models into software that can act. A large language model no longer just predicts text, because it can now request that your code run a real operation. That single shift underlies the current wave of AI agents, copilots, and automated workflows across the software industry. The stakes are large, since the enterprise AI agent market reached USD 6.65 billion in 2025 on its way to a projected 142 billion by 2035. Yet the mechanism itself is simple enough to explain in a single afternoon of focused reading. This guide walks through the loop, the schemas, the benchmarks, and the security tradeoffs that shape real deployments. By the end you will understand the mechanism from first principles all the way to production practice.

Quick Answers on Function Calling in LLMs

What is function calling in LLMs?

Function calling in LLMs explained briefly: the model outputs a structured request naming a function and its arguments, and your application runs that code and returns the result.

Does the model run the function itself?

No, the model never executes anything. It only decides which function to call and with what arguments. Your own code performs the action and hands the result back to the model.

When should I use function calling instead of plain text?

Use it whenever you need reliable structured data or a real action, such as booking, lookups, or calculations. Plain text is fine for open conversation, but tool calling gives you machine-readable output.

Key Takeaways

• Function calling lets an LLM output a structured, machine-readable request that your application executes as real code.
• The model chooses the function and arguments; your system runs the action and returns the result for a final answer.
• Strict schema modes pushed JSON reliability from under 40 percent to 100 percent on complex schemas.
• Real tools add real risk, so prompt injection and over-trusted execution demand guardrails before you ship.

What Is Function Calling in LLMs Explained

Function calling in LLMs explained is the capability of a large language model to emit a structured request, naming a function and its arguments, which your application then executes and feeds back so the model can finish its answer.

How the Tool-Call Loop Actually Works

Building on that definition, the loop behind function calling has four clean stages that repeat as needed. First your application sends the user prompt along with a list of available tools to the model API. The model reads the request and decides whether a tool is needed to answer it well. If it is, the model returns a structured tool call instead of ordinary prose text. Your code then executes that function, captures the result, and sends the output back to the model. The model reads the result and writes a final natural language reply for the user.

That end-to-end handshake is function calling in LLMs explained at the level of a single request. Notice that the model never touches your database, your payment system, or your file store directly. It only proposes an action, and your runtime keeps full authority over what actually executes. This separation is a feature, because it lets you validate, log, and reject calls before they run. You can also inspect the arguments, enforce permissions, and add rate limits at the boundary. The OpenAI function calling guide describes this same request and response cycle in detail.

Reading the flow once makes what a bot really is click into place for anyone new to agents. A plain chatbot answers from its training, while a tool-calling model can fetch and act on live facts. The difference shows up the moment a user asks for something the model cannot know from memory. A weather lookup, an order status, or a database query all require an external call. Function calling is the bridge that carries intent from language into a concrete, executable operation. Once that bridge exists, the same pattern scales from one tool to dozens of them.

The loop can also repeat several times within a single user turn when a task needs it. A model might call a search tool, read the result, then call a calculator with those numbers. Each cycle feeds new information back, so later calls build on the output of earlier ones. This is how a simple primitive grows into multi-step reasoning without any special new machinery. The model is not planning in a human sense, but the loop produces planning-like behavior. Understanding this repetition early prevents a lot of confusion when you first read agent code.

The Anatomy of a Tool Schema

Turning to the schema itself, a tool definition is just a structured description the model can read. Each tool carries a name, a short description, and a typed list of parameters it accepts. The description matters more than beginners expect, since the model uses it to decide when to call. Parameters are written in JSON Schema, which specifies types, required fields, and allowed values. A clear schema is the difference between reliable arguments and a stream of malformed guesses. Good naming and precise descriptions do most of the work of steering the model correctly.

{
  "type": "function",
  "function": {
    "name": "get_order_status",
    "description": "Look up the shipping status of an order by its id.",
    "parameters": {
      "type": "object",
      "properties": {
        "order_id": {"type": "string", "description": "The order identifier, for example A1B2C3."}
      },
      "required": ["order_id"]
    }
  }
}

Consider a function that looks up an order by its identifier and returns the shipping status. Its schema would declare an order_id string as required and perhaps an optional locale field. When a shopper asks about a delivery, the model fills those fields from the conversation. The data extraction with LLMs work shows how structured schemas turn messy text into clean records. That same discipline underlies every production tool, from calendar bookings to financial lookups. Weak schemas invite hallucinated fields, so tightening them is the fastest reliability win available.

Descriptions should read like instructions to a careful new teammate, not like terse code comments. Spell out units, formats, and edge cases, because the model treats the text as ground truth. If a date must be ISO 8601, say so plainly inside the parameter description. Enumerated values help too, since a fixed set of options removes ambiguity from the model. Reliable arguments are the quiet core of every dependable production tool-calling setup. Teams that invest in schema quality spend far less time debugging strange downstream failures.

Why Structured Outputs and Strict Mode Changed Everything

Beyond the basic loop, a second breakthrough made function calling dependable enough for real products. Early models often returned JSON that was almost valid but broke on a missing brace. Strict mode changed that by constraining the model to follow the supplied schema exactly. OpenAI reported that strict structured outputs hit 100 percent schema adherence on hard cases. The older gpt-4-0613 model had scored under 40 percent on the same demanding tests. That jump from unreliable to guaranteed output is what unlocked serious enterprise adoption.

The trick behind strict mode is constrained decoding, which limits token choices to valid ones. At each step the model may only emit tokens that keep the output on schema. This turns a probabilistic guess into a structurally guaranteed result without retraining the model. There is a subtle cost, since heavy format restriction can slightly dampen open-ended reasoning. Some tasks also see extra latency the first time a brand new schema is compiled. Even with those tradeoffs, strict outputs are now the default choice for most tool pipelines.

Function Calling Versus Alternative Integration Methods

Given the range of options, it helps to place function calling next to its common alternatives. Plain prompting asks the model to answer from memory, with no access to live systems. Retrieval augmented generation adds documents to the context so answers stay grounded in real sources. Fine-tuning bakes new behavior into the weights, which is powerful but slow and costly to update. Function calling is different, because it lets the model trigger actions and fetch fresh data on demand. These approaches are complements, not rivals, and strong systems often combine several of them.

Retrieval and function calling pair especially well in knowledge-heavy products and assistants. A model can call a search tool, then read the retrieved passages before answering. The choice between GraphRAG versus traditional RAG often shapes how that retrieval step is built. Fine-tuning still matters when you need a specific tone or a narrow domain skill. It rarely replaces tools, though, since weights cannot hold today’s order status or live price. Thinking in layers keeps each method doing the job it does best.

Cost is another axis that separates these methods in practice. Every tool call adds tokens for the schema, the arguments, and the returned result. Teams watching their budgets often study how to reduce LLM inference costs before scaling agents widely. Retrieval carries its own cost in embedding storage and vector search at query time. Fine-tuning front-loads cost into training but can lower per-call spending afterward. Weighing these tradeoffs early prevents an expensive redesign once traffic grows.

Multi-Step Tool Chains and Agentic Reasoning

Moving on from single calls, the most interesting behavior appears when tools chain together. An agent can call one function, inspect the result, then choose a different tool next. This loop of act, observe, and decide is the heart of mastering agentic AI workflows today. Each step narrows the problem, so a vague request slowly becomes a concrete sequence of actions. The model is not conscious of a plan, yet the chain produces goal-directed results. That emergent structure is why tool calling feels like a leap over plain chat.

Chaining also raises the stakes for memory, since later steps depend on earlier context. A robust AI agent memory architecture keeps track of what was tried and what already failed. Without that recall, an agent can loop forever or repeat a call that already errored. Reliability compounds across steps, so a 90 percent success rate falls fast over many calls. Five chained calls at 90 percent each land near 59 percent end-to-end success. This math is why serious teams cap chain depth and add checkpoints between risky actions.

Putting Function Calling to Work in Your Stack

In practice, wiring function calling into a product follows a short and repeatable recipe. You define your tools, expose them to the model, and write a handler for each name. The handler validates arguments, runs the real operation, and returns a compact result object. You then loop until the model stops requesting tools and produces its final answer. Wiring that loop into your stack is function calling in LLMs explained as an engineering task. None of it requires exotic infrastructure, just clean interfaces and careful error handling.

messages = [{"role": "user", "content": user_input}]
while True:
    resp = client.chat.completions.create(model="gpt-4o", messages=messages, tools=tools)
    msg = resp.choices[0].message
    if not msg.tool_calls:
        break
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = run_tool(call.function.name, args)
        messages.append(tool_result(call.id, result))

Validation deserves special attention, because the model can and will produce surprising arguments. Treat every incoming call as untrusted input, exactly as you would a public web request. Check types, ranges, and permissions before any function touches a real system of record. Good logging helps too, since you will want to replay strange calls during debugging. Teams that skip validation ship demos that break the first time a user gets creative. The habits from coding agents and live docs transfer directly to building reliable tool handlers.

Observability separates a prototype from a system you can operate at scale. Track how often each tool is called, how long it runs, and how often it fails. Those metrics reveal which functions the model misuses and which schemas need tightening. They also expose cost, since a chatty agent can quietly triple your token spend. The people who once did prompt engineering as a role now spend as much time on tool design. That shift reflects how central function calling has become to applied AI work.

Rollout should be gradual, with a small set of low-risk tools exposed first. Read-only functions like lookups and searches are a safe place to begin building trust. Only after those prove stable should you add tools that write data or move money. Feature flags let you disable a misbehaving tool without redeploying the whole service. A staged path turns a scary launch into a series of small, reversible steps. This discipline is what separates a durable deployment from a viral demo that collapses.

A Worked Example From Prompt to Result

With that groundwork, a single worked example makes the whole loop concrete and easy to remember. Imagine a travel assistant that a user asks to check the status of a booked flight. The developer exposes one tool named get_flight_status with a required confirmation code parameter. The user types a casual sentence that includes the code buried in the middle of it. The model reads the request, recognizes the intent, and emits a structured call to that tool. It fills the confirmation code argument by extracting the value from the natural language input. The application receives the call, queries the airline API, and returns a compact status object. The model then turns that raw data into a friendly sentence for the traveler to read.

Notice how much work the schema quietly did in that short exchange between user and model. The parameter description told the model exactly what a valid confirmation code looks like. Without that hint, the model might have grabbed the wrong token or invented a plausible code. The OpenAI function calling guide shows this extraction pattern with similar concrete samples. The application layer also validated the code format before spending money on an API request. If the code failed validation, the handler could ask the model to request a correction. That small guard prevents wasted calls and keeps the conversation moving smoothly for the user. Every production tool follows this same rhythm of extract, validate, execute, and summarize.

Now imagine the traveler asks a follow-up question that needs a second, different tool. They want to know whether a delayed flight still connects to their next leg. The model can call a connection-checker tool using the output of the first status call. This chaining is where a MCP developer workflow starts to feel genuinely powerful in daily use. Each tool stays small and focused, while the model composes them into a real answer. The developer never wrote branching logic for every possible question the traveler might ask. Instead, the tools plus the model together cover a wide space of related requests. That composability is the quiet superpower that makes tool calling worth the added complexity.

Best Practices for Reliable Tool Calling

Turning to day-to-day practice, a handful of habits separate reliable tool systems from fragile ones. Keep each tool small, with a single clear responsibility and a tightly typed schema. Write descriptions for a smart newcomer, spelling out formats, units, and every important edge case. Return compact, structured results, because verbose blobs waste tokens and confuse the model. Always validate arguments before execution, and treat the model as an untrusted caller by default. Add timeouts and retries with backoff, so a slow tool does not stall the whole conversation. Log every call with its arguments and result, which makes debugging strange behavior far easier. These habits sound obvious, yet skipping any one of them causes most real production incidents.

Reliability also depends on how you handle the inevitable failures that tools will produce. When a tool errors, return a clear message the model can read and reason about. A good error lets the model retry, choose another tool, or ask the user for help. Guides for AI agents guide for leaders stress that graceful failure is a product feature, not an afterthought. Set a hard cap on how many tool calls a single request may trigger. That cap stops runaway loops from burning your budget when the model gets stuck. Test each tool in isolation, then test the whole loop with messy, realistic user inputs. The teams that follow these practices ship features that keep working after the demo ends.

How Function Calling Reshapes Everyday Software Work

Beyond the codebase, function calling is changing how ordinary software gets built and used. Support teams now field a majority of routine questions through tool-using assistants. Analysts ask plain questions and let an agent call the database instead of writing queries. The latest agent adoption data shows most organizations already use agents somewhere in their stack. Roughly 85 percent of organizations report an AI agent in at least one workflow today. That reach means tool calling touches millions of everyday interactions across many industries. The change is quiet, since users rarely see the schemas working behind a simple chat box. Yet the shift in who does routine digital work is large and still accelerating.

This wave also reshapes the skills that software teams hire for and value. The craft of prompt engineering as a role now blends into tool design and evaluation work. Engineers spend less time on brittle scripts and more on clear interfaces for models. Product managers learn to think in terms of what actions an assistant may take. Support leaders learn to measure deflection rates and escalation quality with fresh rigor. New roles appear around agent safety, evaluation, and the governance of automated actions. The job market shifts steadily as tool calling moves from novelty into core infrastructure. None of this erases human work, but it does move where that work adds the most value.

There is a real risk of over-automation that thoughtful teams must actively manage. When an assistant handles everything, hard cases can slip through without a human ever noticing. Klarna’s later reversal is a reminder that pure automation can quietly degrade customer experience. The broader debate about autonomous agents captures this tension between speed and human judgment. Good deployments keep a clear path for a person to step in at the right moment. They also monitor quality closely, since silent failures are the most dangerous kind. Used well, tool calling frees people from drudgery and lets them focus on judgment. Used carelessly, it can hollow out service in ways that only show up much later.

Choosing a Model and Provider for Tool Calling

Given so many options, choosing a model for tool calling deserves more thought than a coin flip. Proprietary leaders currently top the accuracy charts, but they charge more per token. Open models have narrowed the gap and can run on your own hardware for privacy. The choice hinges on your accuracy needs, your budget, and your data residency rules. For high-stakes actions, the extra reliability of a top model is usually worth the cost. For simple lookups, a cheaper open model may handle the load at a fraction of the price. Studying how to reduce LLM inference costs helps teams pick a tier without overspending on every call. There is rarely one right answer, only the right tradeoff for a specific workload.

Provider lock-in is a second factor that quietly shapes long-term flexibility. A shared standard like the Model Context Protocol reduces how tightly you bind to one vendor. Writing tools against that standard lets you swap models with far less rewriting later. Evaluation is the tiebreaker, so test your finalists on your own realistic tasks. A model that tops a public leaderboard may still trail on your specific domain. Run a small benchmark of your actual tools before committing to a provider at scale. Measure accuracy, latency, and cost together, since optimizing one alone can hurt the others. The right process turns a risky bet into an informed and defensible engineering decision.

Common Mistakes When Adding Tools

From there, teams still stumble on a familiar set of mistakes when they add their first tools. The most common error is exposing too many tools at once to a single model. A crowded tool list confuses the model and pushes it toward the wrong choice. Vague descriptions are a close second, since the model cannot read your unspoken intent. Skipping argument validation is a dangerous third, because the model will eventually surprise you. Another mistake is trusting the first successful demo as proof that the system is ready. Demos use friendly inputs, while real users are messy, adversarial, and endlessly creative. Each of these mistakes is easy to fix once you know to watch for it.

A subtler mistake is ignoring cost until the monthly bill arrives as an unpleasant shock. Long chains and chatty agents multiply token usage in ways that are hard to predict. Watching how to coding agents and live docs run in production reveals where the spending really goes. Teams also forget to version their schemas, which breaks tools when descriptions quietly change. A schema change can silently alter model behavior if no one tracks it carefully. Treat tool definitions as code, with reviews, tests, and a clear change history. That discipline catches regressions before they reach users in a live product. Small process habits prevent the majority of painful and avoidable tool-calling failures.

The final mistake is deploying tools that write data before the read-only versions are solid. Write actions are irreversible in a way that read actions almost never are. A wrong lookup wastes a moment, while a wrong payment creates a real financial problem. Start every rollout with safe, reversible tools and earn trust before granting more power. Add confirmation steps for any action that moves money or changes important records. Keep a human in the loop for the highest-stakes decisions in the early days. Loosen those guardrails slowly, only as evidence of reliability accumulates over real traffic. This cautious path is how careful teams avoid the headlines that follow a careless launch.

Function Calling and Retrieval Working Together

Given the overlap, function calling and retrieval are strongest when they work together, not apart. Retrieval brings relevant documents into context, while tools take actions and fetch live values. A support agent might retrieve a policy page, then call a refund tool to apply it. The model reads the retrieved text, decides what to do, and issues the right structured call. This pairing grounds the model in facts while still letting it act on the world. Comparing GraphRAG versus traditional RAG helps teams choose how to structure the retrieval half of the system. Without retrieval, the model may act confidently on knowledge it simply does not have. Without tools, the model can cite a policy but cannot actually enforce it for the user.

Designing the two layers together avoids a common trap where they quietly duplicate work. Retrieval should handle open-ended knowledge, while tools handle precise lookups and actions. A price is a tool call, not a document, since it changes far too often to cache. A refund policy is retrieval, since it is stable text the model should quote accurately. Drawing that line clearly keeps each subsystem simple and easy to reason about. It also makes debugging easier, because you know which layer produced any given answer. The best assistants blend both fluidly, so users feel one coherent and capable system. That blend, done well, is what separates a toy demo from a dependable product.

Where Function Calling Falls Short and the Risks It Adds

Despite the momentum, handing a model real tools introduces risks that plain chat never had. The largest is prompt injection, where hostile text hijacks the model into unwanted actions. The OWASP Top 10 for LLMs ranks prompt injection as the number one threat today. A poisoned document or web page can quietly instruct an agent to call a dangerous function. Because tools execute code, a successful injection can leak data or trigger real transactions. Reported attack success rates range widely, and no current model is fully immune.

Over-trusted execution is a related danger that teams underestimate constantly. If a handler runs whatever the model asks, one bad argument can cause real damage. A recent agent flaw opens attack vector showed how a single message can pivot into a broader breach. Hallucinated arguments are another failure mode, since the model can invent plausible but wrong values. Strict schemas reduce this, yet they cannot guarantee that a valid value is a correct one. Defense in depth, with validation and least privilege, is the only workable strategy.

Cost and latency are quieter risks that still derail projects at scale. Every tool call adds round trips, and long chains can feel sluggish to users. Runaway loops can also burn a budget fast when an agent keeps retrying a broken call. Multi-turn reliability remains weak, which limits how much autonomy you can safely grant. These constraints are not reasons to avoid tools, only reasons to design carefully. Naming the risks plainly is the first step toward containing them in production.

The Ethics of Handing Models Real Tools

Stepping back from features, tool calling raises questions that go beyond pure engineering. When a model can act, responsibility for its actions becomes an urgent and concrete concern. Responsibility for actions is function calling in LLMs explained as a governance question, not only a technical one. If an agent sends a wrong refund, the accountable party must be clear in advance. The debate around AI agents and their promise captures how quickly capability can outrun oversight. Ethical deployment starts with limits on what tools an agent may ever touch.

Transparency is a second ethical pillar that users increasingly expect from automated systems. People deserve to know when an action was taken by a model rather than a person. Audit logs of every tool call make that transparency practical and verifiable after the fact. Guidance for decision makers, such as an AI agents guide for leaders, helps set these boundaries early. Consent and reversibility matter too, since some actions cannot simply be undone later. Treating ethics as design input, not an afterthought, produces systems people can actually trust.

How Function Calling Performs on Independent Benchmarks

With that context, independent benchmarks give the clearest picture of how well tool calling works. The Berkeley Function Calling Leaderboard tests models on real and synthetic tool tasks regularly. It scores accuracy across single calls, live execution, and harder multi-turn interactions. The strongest proprietary models cluster near 90 percent overall accuracy on that benchmark. Numbers like these are function calling in LLMs explained through the lens of measurable accuracy. Public leaderboards keep vendors honest and give teams a shared yardstick for comparison.

The multi-turn results are where the story turns more sober for autonomy fans. Accuracy on long, stateful tasks can fall toward 55 percent even for capable models. That gap explains why single-call features ship faster than fully autonomous agents. Open models have closed much of the distance, with several scoring in the sixties overall. Evaluation tooling such as evaluating Bedrock agents helps teams measure their own systems too. Measuring against a benchmark beats trusting a vendor’s marketing accuracy claim.

Benchmarks also reveal a subtle truth about how reliability should be reported. An overall score hides wide variance between easy lookups and complex chained tasks. A model near 90 percent on single calls may still struggle on a five-step workflow. Teams should test on their own tasks, since generic scores rarely match a specific domain. Small, targeted evaluation sets catch failures that a broad leaderboard will always miss. The habit of measuring locally is what turns a promising demo into a dependable feature.

The Model Context Protocol and the Standardization Race

Looking across the ecosystem, tool integration is quickly converging on shared open standards. The Model Context Protocol from Anthropic connects models to external systems through one interface. It sits above function calling, standardizing how tools are discovered, described, and invoked. Function calling remains the model-level primitive, while the protocol handles the integration layer. The protocol has spread fast, now spanning more than 10,000 public servers across the industry. Most modern stacks use both, so the two are partners rather than competitors.

Adoption accelerated once major vendors agreed to support the same protocol. Editors, assistants, and cloud platforms wired it in over the course of a single year. Anthropic’s earlier open source connection protocol seeded much of this rapid momentum. A shared standard means a tool written once can serve many different models and clients. That reuse cuts integration work and reduces the lock-in that worried early adopters. Standardization rarely feels dramatic, yet it quietly decides which platforms win.

The standard is not without its critics and open questions. Early versions lacked strong built-in authentication, which raised real security concerns. Giving many servers access to a model widens the surface that attackers can probe. Governance is now shifting toward a neutral foundation to keep the standard open and safe. That move signals maturity, since durable standards outlive the company that first proposed them. How this race resolves will shape tool calling for years to come.

The Future of Function Calling and Agentic Systems

Looking ahead, function calling is set to move from single actions toward orchestrated systems. The road ahead keeps function calling in LLMs explained evolving from single calls toward orchestrated systems. Agents will coordinate many tools, hand work to each other, and recover from their own errors. Governance is following that shift, with new foundations forming to steward open agent standards. On-device function calling is also advancing, letting small models act without a cloud round trip. The direction is clear, even if the exact timeline remains genuinely hard to predict.

Reliability on multi-turn tasks is the frontier that will decide how much autonomy we grant. As those scores climb, agents will take on longer and more valuable workflows safely. Advances in coding agents and live docs hint at where general tool use is heading next. Expect tighter security, better evaluation, and clearer accountability to arrive together. The winners will treat tools as a first-class part of the product, not a bolt-on. Function calling has already reshaped applied AI, and its second act is just beginning.

Key Insights

• The enterprise AI agent market hit USD 6.65 billion in 2025, a scale that explains why every major vendor now ships function calling as a core primitive.
• OpenAI reports that strict structured outputs reached 100 percent schema adherence, up from under 40 percent for the older gpt-4-0613 model.
• On the Berkeley Function Calling Leaderboard, the strongest proprietary models cluster near 90 percent overall accuracy while multi-turn scores drop toward 55 percent.
• Klarna said its AI assistant handled two-thirds of chats in month one, cutting resolution time from 11 minutes to under 2 minutes.
• DoorDash reported that its tool-using support system cut hallucinations by 90 percent and reduced compliance issues by 99 percent in production.
• The OWASP Top 10 for LLMs ranks prompt injection first, because tool wiring lets a crafted input trigger unintended real-world actions.
• Anthropic’s Model Context Protocol now spans more than 10,000 public servers, signaling that tool integration is standardizing across the whole industry.

Taken together, these numbers tell a consistent story about where the technology stands today. Function calling has matured from a demo feature into the load-bearing interface for agents and copilots. The reliability gains are real, yet they concentrate in single-call tasks rather than long multi-step chains. That gap is exactly why security and evaluation now matter as much as raw model quality. Seen this way, function calling in LLMs explained is less a trick and more an operating discipline. The teams that treat it that way are the ones shipping systems that survive contact with users.

Dimension	Function Calling	Plain Prompting	RAG Retrieval	Fine-Tuning
Structured output	Guaranteed with strict mode	Unreliable	Depends on prompt	Depends on training
Real-world actions	Yes, via tools	No	No	No
Live or fresh data	Yes, on demand	No	Yes, from documents	No, frozen in weights
Setup effort	Low to moderate	Very low	Moderate	High
Reliability control	High with schemas	Low	Moderate	Moderate
Cost per request	Extra tokens per call	Lowest	Retrieval overhead	Low after training
Auditability	Strong, every call logged	Weak	Moderate	Weak
Best fit	Actions and structured data	Open chat	Grounded answers	Fixed style or domain

Function Calling in Practice Across Shipping Products

From there, it helps to see function calling inside products that people actually use today. The examples below each pair a concrete implementation with a measured outcome and an honest limitation. None of them is magic, and each shows a real tradeoff worth studying closely. Together they map the current state of tool calling across research and industry. Read them as evidence, not as marketing, since every number here comes from a cited source.

OpenAI Structured Outputs and Strict Schemas

OpenAI rolled out structured outputs with a strict flag that forces generated arguments onto the supplied schema. In its own testing the gpt-4o-2024-08-06 model reached 100 percent adherence on complex JSON schemas. That result, documented in OpenAI’s structured outputs announcement, contrasted with under 40 percent for gpt-4-0613. The gain came from constrained decoding, which limits each token to keep output on schema. The limitation is that strict formatting can slightly reduce reasoning quality on open-ended tasks. New schemas also add first-request latency, so heavy users cache compiled schemas to avoid it. Even so, strict mode is now the default for teams that need dependable machine-readable output.

The Berkeley Function Calling Leaderboard

Researchers at Berkeley built a public leaderboard that runs models through thousands of real tool tasks. It evaluates function calls with abstract syntax tree checks plus live execution against real APIs. On that Berkeley leaderboard, Claude 3.5 Sonnet scored about 90 percent overall among proprietary models. GPT-4-0125-Preview landed near 88 percent, showing how close the top proprietary tier has become. The limitation is stark, since multi-turn accuracy falls toward 55 percent even for strong models. That single gap explains why autonomous multi-step agents still lag behind simple tool features. The leaderboard updates over time, so its snapshots capture a fast-moving field honestly.

Anthropic’s Model Context Protocol

Anthropic open-sourced the Model Context Protocol to standardize how models connect to external tools. Within roughly a year the standard spread to editors, assistants, and major cloud platforms. By 2026 the protocol spanned more than 10,000 public servers, a rapid increase across the industry. Adoption by ChatGPT, Cursor, and Gemini turned a single vendor idea into a shared layer. The limitation was security, because early versions lacked strong built-in authentication controls. Wide server access widened the attack surface, so governance moved to a neutral foundation. The tradeoff of openness against safety still shapes how the protocol evolves today.

Lessons From Teams That Deployed Tool-Calling Systems

Among the teams shipping systems, a few public deployments show what tool calling does at scale. Each case below states the problem, the solution, the measured impact, and an honest limitation. The subjects differ from the earlier examples, so together they broaden the evidence base. Customer support dominates here, because it offers clear metrics and high transaction volume. Read the limitations as carefully as the wins, since both are part of the real record.

Case Study: Klarna's OpenAI Assistant

Klarna faced a mounting support workload across dozens of markets and many languages at once. The company built a GPT-4-class assistant that used tools to resolve orders, refunds, and disputes. According to Klarna's own announcement, the assistant handled two-thirds of chats in month one. It ran 2.3 million conversations and cut average resolution time from 11 minutes to under 2. The firm estimated a 40 million dollar profit improvement from the system in its first year. Customer satisfaction held roughly on par with human agents across those early months. The scale of the rollout made it one of the most cited tool-calling deployments anywhere.

The limitation surfaced later, once quality and control problems became visible in practice. Reporting from industry coverage noted that Klarna began rehiring human agents for hard cases. The reversal showed that heavy automation can trade away nuance that customers still value. It did not erase the early gains, but it tempered the original headline claims sharply. The lesson is that tool calling scales support, yet humans remain essential for edge cases.

Case Study: Intercom's Fin AI Agent

Intercom needed to deflect a large share of repetitive support tickets without hurting customer trust. It built Fin, a tool-using agent that reads help content and calls back-end functions to resolve issues. The company reported that Fin 3 reached a 67 percent average resolution rate in 2025. Across its history the agent has handled more than 40 million resolved conversations. The limitation is that documented production deployments often land near 45 to 53 percent instead. Resolution also drops sharply on complex, multi-turn, and enterprise tickets in the field. Fin shows strong deflection on simple questions, yet vendor averages can overstate real performance.

Case Study: DoorDash Dasher Support

DoorDash fielded hundreds of thousands of Dasher support contacts every single day. The team needed to deliver accurate answers without the hallucinations that plague naive chatbots. It built a retrieval system whose evaluation wraps each check as a function over the transcript. As described in a DoorDash engineering case study, the system cut hallucinations by 90 percent. It also reduced compliance issues by 99 percent while automating support at large scale. A new testing framework ran thousands of automated checks per hour, a fiftyfold capacity gain. Those numbers came from careful evaluation rather than a single optimistic demo.

The limitation is that this reliability depends on heavy guardrails and constant monitoring. Escalations to human agents still happen, and quality drifts when the knowledge base ages. The system needs ongoing evaluation, since a static setup slowly degrades over time. That maintenance cost is easy to forget when you only read the headline metrics. DoorDash shows that tool calling works at scale, but only with disciplined operations behind it.

Common Questions About Function Calling in LLMs