How language models use tools and take actions.
A model on its own can only write words. Give it tools, and it can look something up, do exact math, or move a step forward. Here is the field that makes that happen, in plain language.
Tool use is how a language model reaches outside itself: instead of answering from memory, it writes a structured request, something runs that request, and the model reads the result and continues. We walk it the way the field is built: what a tool call is, why deciding to call one is hard, how a model is taught to do it, what it unlocks, and where it is heading. The papers behind each idea are linked, so you can read the source yourself.
get_order_statusorder_id: “A-4471”
A tool is anything outside the model the model can ask to run.
A search, a calculator, a unit converter, a database query, an API that reads or writes a record. The everyday name for the mechanism is function calling.
A language model, working from memory alone, can only write words. Ask it for today's exchange rate, the status of an order, or to add a note to a record, and it will either guess or politely decline. Its knowledge stopped at training time, and writing text is the only move it has.
Tool use changes what the model can do. Instead of answering everything from memory, it is given a small menu of tools it is allowed to call. When a question needs one, the model stops writing prose and writes a structured request instead: the tool's name plus the inputs to pass it. A runtime outside the model executes that request and hands the result back, and the model reads it and keeps going. It is the difference between a person who can only talk and a person who can also pick up the phone, open a spreadsheet, or file a form.
Two distinctions run through the rest of this page. First, deciding when a tool is needed, which tool, and what to put in it is a learned skill, not a given. Second, a lookup is not an action: a lookup can be retried freely, while an action changes the world and has to be done with care. Picture one running question that needs a fact looked up, a number computed, and a record updated, none of which memory can supply. That is the task tools are for.
{
"tool": "convert_currency",
"amount": 4200,
"from": "EUR",
"to": "USD"
}4,557 USD at an illustrative rate of 1.085 USD/EUR.
Reaching for the right tool, with the right inputs, at the right moment.
A model that can call tools faces choices a pure question-answering model never has to make, and several ways to get each one wrong.
Memory is stale and approximate.
Facts stop at training time, and a model asked to compute can produce fluent but wrong arithmetic. Tools exist to cover exactly these gaps, but only if the model reaches for one.
Most of the time, just answer.
The skill is recognizing the minority of moments that genuinely need a tool, and not calling one when none is needed. Over-reaching is its own failure, not only under-reaching.
The arguments can be invented.
Even when the right tool is chosen, models can pass malformed or made-up arguments. Gorilla (Patil et al., 2023) names this directly: hallucinated or wrong API usage is a core failure mode.
A handful fits. Thousands do not.
A few tools fit in the prompt. A large catalog does not. The model has to find the right one from a big set, which turns into a retrieval problem of its own.
One action is rarely enough.
The model has to plan, act, read what came back, and adjust, without losing the thread or repeating itself. Reasoning and acting have to stay in step across several turns.
Some moves change the world.
A lookup can be retried. Booking, sending, or writing changes state. That raises permission, confirmation, and recovery questions a pure question-answering system never faces.
From a structured request to a loop that can act.
Write the call as structure. Learn when to make it. Interleave thinking and acting. Route across modules. Narrow a large tool set. Close the loop on anything that changes state.
1The tool call as a structured request
Name the tool, type the inputs.
Rather than free text, the model emits a named call with typed inputs. The runtime validates that call against the tool's schema, runs it, and returns a result the model treats as new context. This is the mechanism behind what is commonly called function calling.
The structure is what makes it dependable. A schema says which tool exists, which inputs it expects, and what shape the answer takes, so the runtime can reject a malformed call before anything runs. The model's job is to fill that shape correctly, then read back what the tool returns.
- Structured request
get_order_status with order_id A-4471.
- Validation and authorization
Check argument types and access to this order.
- Execution
Call the order service.
- Result
Return the actual status for the model to use.
2Teaching the model when to call
A model can teach itself.
Toolformer (Schick et al., 2023) shows a model can learn this from a handful of examples, with no hand-labeled data. It proposes where a tool call would help, tries the call, and keeps only the calls that actually improved its next prediction.
The result is a model that decides which API to call, when, and with what arguments, trained on its own filtered attempts. The signal is simple and honest: did calling the tool make the model's continuation better. If not, the example is dropped.
- Propose a tool call
Insert a candidate call into training text.
- Run it
Obtain the external result.
- Measure usefulness
Check whether the result improves prediction of the continuation.
- Keep useful examples
Use retained calls for further training.
3Interleaving thought and action
Think, act, observe, think again.
ReAct (Yao et al., 2022) alternates short reasoning steps with actions. The reasoning helps the model plan the next move and recover from a bad step. The actions keep it grounded in real results instead of its own assumptions.
This is the loop tool-using agents commonly run. A single action is rarely enough for a real task, so the model reasons about what it has, takes one step, reads what came back, and decides the next step from there.
- Choose the next step
The task needs the order’s return status.
- Act
Call get_order_status(A-4471).
- Observe
Two of three items have been returned.
- Continue
Use that result to determine the next permitted step.
4Routing across modules
Send each input to the part that should handle it.
MRKL (Karpas et al., 2022) frames the broader pattern. A router sits in front of expert modules, some neural and some symbolic, a calculator, a database, an API, and sends each input to the module that should handle it.
It is the conceptual ancestor of tool routing. The lesson it carries forward: a language model does not have to do every job itself. It can recognize the kind of job in front of it and hand that job to the part built for it.
| Request | Handler |
|---|---|
| 42 × 19 | Calculator |
| Current order status | Order API |
| Summarize this paragraph | Language model |
5Finding the right tool among many
Narrow the menu before choosing.
Gorilla (Patil et al., 2023) pairs the model with a document retriever, so it can pick the correct API from a large, changing set and adapt when the documentation changes. ToolLLM (Qin et al., 2023) scales this to thousands of real APIs, with a retriever that shortens the menu before the model chooses.
The pattern is the same at both sizes. Retrieve a short list of plausible tools first, then let the model choose from a handful it can actually hold in context. That also keeps the system honest when the underlying APIs are versioned and shifting.
A customer wants to return two items.
- Full catalog
- Orders, billing, email, calendar, contacts, and more.
- Retrieved candidates
- get_order · get_return_policy · create_return
Check the chosen tool’s contract before constructing its arguments.
6Closing the loop on actions
A lookup can be retried. An action cannot.
For steps that change state, the field leans on a few practices: validating the arguments, asking for explicit confirmation, scoping permissions to what the task needs, and keeping a record of what was done so it can be checked or undone. This is general practice across the field, not any one product's design.
The reason is the distinction from the top of the page. A lookup is safe to repeat. Booking, sending, or writing happens once and stays happened, so an action runs behind a checkpoint that a lookup does not need.
Lookup
get_order_statusOften safe to repeat without changing state.
State-changing action
refund_orderUse authorization, validation, and an idempotency key or outcome check.
Tools move a model from "here is what I would say" to "here is what I did."
At the field level: current answers instead of stale ones, exact operations instead of approximations, write actions that complete a task, and one request that spans several tools in sequence.
The four gains below are field-level capabilities a tool-using model has that a model working from memory alone does not. For a sales co-pilot, the bridge is at the outcome level: a model that can look something up, do exact math, and take a careful action is what turns a conversation into a finished task.
Current answers, not stale ones.
A model that can search or query answers from live data, instead of frozen memory. The closely related pattern of grounding answers in retrieved sources is covered in Retrieval-augmented generation.
Exact operations, not approximations.
Arithmetic, conversions, and structured lookups become reliable because a tool does them, not the model's best guess at the answer.
Doing, not only describing.
With write actions, a model can update a record, draft and file a note, or move a step forward, turning a conversation into a completed task. See Live deal assist.
One request, several tools.
A reason-and-act loop lets a single ask trigger a lookup, a calculation, and an update in sequence, fused into one outcome.
This explainer covers published research. To see how AI REV Voice helps with sales conversations, explore Prism.
The direction the field is heading.
We describe the direction, not a roadmap of our own.
From single calls to whole tasks.
The frontier is sequences of actions that hold up across a whole task, not one clean call in isolation. An agent that takes the right first step but loses the thread is not yet dependable.
Consistency, not just accuracy.
τ-bench (Yao et al., 2024) checks the end state of a task and adds a repeated-trial metric. As an attributed field result, it reports that even strong function-calling agents succeed on under half of realistic tasks and behave inconsistently across repeated trials. Expect evaluation to weigh consistency as heavily as one-shot success.
Thousands of versioned APIs.
ToolLLM (Qin et al., 2023) points at thousands of real, versioned APIs, where retrieval and graceful handling of change matter as much as the call itself.
Permission and recovery as baseline.
As models take more real actions, permission boundaries, confirmation, and recoverability move from nice-to-have to baseline. We frame this as a field-wide direction.
| Task | Trial 1 | Trial 2 | Trial 3 |
|---|---|---|---|
| A | Pass | Pass | Pass |
| B | Pass | Fail | Pass |
| C | Fail | Pass | Fail |
At least one success is different from success on every trial.
The papers behind this page.
- Karpas et al. (2022). MRKL Systems: A Modular, Neuro-Symbolic Architecture That Combines Large Language Models, External Knowledge Sources and Discrete Reasoning. arXiv:2205.00445.
- Yao et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629.
- Schick et al. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv:2302.04761.
- Patil et al. (2023). Gorilla: Large Language Model Connected with Massive APIs. arXiv:2305.15334.
- Qin et al. (2023). ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs. arXiv:2307.16789.
- Yao et al. (2024). τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045.
Keep reading.
Sibling explainers in the Research section.
Talk. Done.
A model that can look something up, do the math, and move a step forward is what turns a conversation into a finished task. See it on a real call.
No app required. Just your phone number.