Dialogue state tracking.
Picture someone helping you book a trip. By the end they are holding a short list in their head: north of town, cheap (you changed your mind from moderate), a table for two near the hotel. They did not memorize the conversation. They kept a tidy record of the facts that matter, and updated it the moment you changed something.
Dialogue state tracking is the name for the part of a conversational system that keeps that running record. After each thing you say, it updates a compact summary of what you want so far, and keeps it correct as the conversation changes its mind. We walk it the way the field is built: what it is, why keeping it correct is harder than it sounds, how systems do it, what it unlocks, and where it is heading. The papers behind each idea are linked, so you can read the source yourself.
A small running record of what you want so far.
The field name is dialogue state tracking, also called belief tracking. The record it keeps is the dialogue state.
The dialogue state is a compact, structured summary of everything you want so far, carried forward across turns. Most often it is a set of slots and their values, the belief state: area = north, food = italian, price = cheap. After each user turn, a tracker updates that record to reflect the new turn plus everything before it.
It sits in the middle of a task-oriented dialogue pipeline. Natural language understanding reads a single turn and pulls out values. State tracking accumulates those across turns into one current record. A policy decides the next action from the record. Response generation says the next thing. The key line: the rest of the system acts on the record, not on the raw transcript. State tracking is what keeps that record correct.
“Find a hotel in the north.”
“Moderate price. Actually, make it cheap.”
- Area
- North
- Price
ModerateCheap- Dates
- Not provided
Keeping the record correct is harder than it sounds.
People change their minds, point back at earlier turns, and speak messily. The record has to stay right through all of it.
People change their minds.
"Moderate. No, make it cheap." The tracker has to overwrite the value, not append it. A record that only ever adds facts ends up holding two answers and trusting the wrong one.
The value lives in an earlier turn.
"The same area as the hotel." "Make it two as well." "That one." The word you need was said three turns ago, not in the current sentence, so the tracker has to look back to fill the slot.
A value can point at another slot.
"Take a taxi to the restaurant I just booked." The destination is not a literal phrase. It is whatever value another slot already holds, carried across from one domain to the next.
Spoken input is imperfect.
In a voice system the tracker reads imperfect speech recognition, so it has to stay robust to mis-heard words rather than trust a clean string. This is part of why the field moved toward reading recognition results straight into the tracker. See Speech-to-text.
One wrong slot fails the turn.
The common metric, joint goal accuracy, requires the whole state to be exactly right at each turn. A single wrong slot fails it, and errors compound: a wrong value early can poison the turns that follow.
New domains, unseen values.
Real values like names, times, and places can be unbounded and were not all seen in training, and assistants add new services constantly. Retraining a bespoke tracker for each one does not scale.
From hand-built rules to models that read a description.
The field moved in layers: hand-crafted trackers, then neural ones, then a shared benchmark, then generation, schemas, and language models. Each layer made adding a new domain look less like training a tracker.
1Hand-crafted and early statistical trackers
Rules, a hand-built lexicon, and a distribution over states.
The first systems mapped phrases to slot values with rules and hand-built semantic dictionaries, then maintained a distribution over possible states. This was brittle, hard to extend, and dependent on someone curating the lexicon. The Dialog State Tracking Challenge series (Williams et al., 2016) gave the field its first shared testbeds and metrics, and documents the move from generative to discriminative trackers.
| Observed phrase | Rule output |
|---|---|
| somewhere cheap | price = cheap |
| in the north | area = north |
| not too pricey | No rule: ask or defer |
2Neural, lexicon-free tracking
Learn the match instead of curating it.
Neural trackers learned to compare the user's words against candidate slot-value pairs using pre-trained word vectors, which removed the hand-built lexicon and generalized to phrasings nobody wrote down. The Neural Belief Tracker (Mrkšić et al., 2017) is a lexicon-free tracker of this kind, the foundational case for learned rather than curated matching.
I need a budget place.
Nothing fancy, please.
Interpret each phrase using its conversation context.
3A shared benchmark forces multi-domain
One conversation that crosses hotel, restaurant, and taxi.
A large multi-domain corpus let the field train and compare trackers on conversations that cross domains and carry values between them. MultiWOZ (Budzianowski et al., 2018) is a common benchmark of human-human dialogues with state annotations, and the place to look for how the field measures this. It also reports the unforgiving metric: joint goal accuracy, where the entire state must be right at each turn.
| Turn | State change |
|---|---|
| A cheap hotel in the north. | hotel.area = north; price = cheap |
| Moderate, actually. | price: cheap → moderate |
| And a taxi to that hotel. | taxi.destination = selected hotel |
4Generation and transfer beyond a fixed list
Generate the value, and lift unseen ones from the conversation.
Instead of scoring every candidate from a closed ontology, a generative tracker produces the value directly, with a copy mechanism that can pull an unseen value straight out of what was said. Sharing parameters across domains then lets it transfer to a new domain with few or zero examples, as in TRADE (Wu et al., 2019).
Book Saffron & Co. for two.
Closed list
Can select only restaurant names already in the ontology.
Generated value
restaurant: Saffron & Co.
people: 25Schema-guided tracking
Describe a new service, do not build a new tracker.
Describe each service with a schema, its slots and intents written in natural language, and have a single model read that description at run time. One model then serves many services, including ones it never saw in training, instead of a tracker per domain. That is the design behind Schema-Guided Dialogue (Rastogi et al., 2020).
| Service slot | Description |
|---|---|
| destination | Where the taxi should take the passenger. |
| depart_at | Requested pickup time. |
| passengers | Number of people travelling. |
Take two of us to the hotel at six.
destination: hotel · depart_at: six · passengers: 2
6Language models as zero-shot trackers
The schema becomes a function signature.
Recent work frames state tracking as function calling: give a language model the schema as a function signature and let it emit the filled slots as arguments, tracking state in a new domain without fine-tuning, as in FnCTOD (Li et al., 2024). The throughline across all six layers: the field moved from fixed lists and hand-built rules toward models that read a description of what to track and generate the values. This connects state tracking to tool use and structured output.
A cheap place in the north, for two.
find_restaurant({
area: "north",
price: "cheap",
people: 2
})A conversation you can act on needs a record you can trust.
At the field level: a conversation that holds together, self-correction handled gracefully, the right action on the right data, and a clean handoff to systems of record.
For a live setting like a sales call, the bridge is at the outcome level. The few facts that just changed, a renewal date, a budget number, a competitor mentioned, a next step agreed, form a small running record. Keeping that record correct as the call changes, and getting it to the app quickly, is the same shape of problem this page describes. These are examples of why the field matters to the person, not claims about any one system.
The conversation holds together.
A constraint set three turns ago is still in the record, so the system does not re-ask for what you already said.
Self-correction is graceful.
"Not next quarter, this one" overwrites the fact cleanly, instead of leaving two conflicting answers in the record.
Right action, right data.
Because downstream steps act on the structured record, the question asked next and the query that runs reflect what you actually want.
A clean handoff to the app.
The record is already slots and values, so it maps onto fields an application can store. That is the bridge from a spoken exchange to fresh data, fast. See Structured output.
The direction the field is heading.
We describe the direction, and stay honest about the open problems, not a roadmap of our own.
Beyond closed ontologies.
Generative tracking that handles values never seen in training, instead of choosing only from a fixed list of allowed values.
Schema-guided and zero-shot.
Tracking that adapts to a new service from its description, so adding a domain looks more like writing a schema than training a tracker.
Tracking through function calling.
Language models doing state tracking with structured output, blurring the line between understanding, tracking, and tool use.
Tied to retrieval and long context.
State that can include references grounded in records and earlier history, so "the one I mentioned" resolves against what was actually said.
Metrics, error spread, noisy input.
Whether the strict joint-accuracy metric reflects real usefulness, how errors propagate across long conversations, implicit updates, and inconsistent annotations even in widely used corpora.
Joint goal accuracy is strict: the whole record must be right, so one wrong slot fails the turn. This illustrates an evaluation metric, not a product quality score.
The papers behind this page.
Any number in a paper is a fact about that paper, not about us.
- Williams, Raux, Henderson (2016). The Dialog State Tracking Challenge Series: A Review. Dialogue & Discourse 7(3).
- Mrkšić et al. (2017). Neural Belief Tracker: Data-Driven Dialogue State Tracking. arXiv:1606.03777.
- Budzianowski et al. (2018). MultiWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling. arXiv:1810.00278.
- Wu et al. (2019). Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems (TRADE). arXiv:1905.08743.
- Rastogi et al. (2020). Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset. arXiv:1909.05855.
- Li et al. (2024). Large Language Models as Zero-shot Dialogue State Tracker through Function Calling (FnCTOD). arXiv:2402.10466.
Keep reading.
Sibling explainers in the Research section.
Talk. Done.
A conversation you can act on needs a record you can trust, kept current turn by turn. See it on a real call.
No app required. Just your phone number.