Entity resolution: one customer, one record.
The same company is in your database three times under three spellings. Entity resolution is the field that decides those rows are one real thing, and brings them together.
Entity resolution, also called record linkage and entity matching, decides which records, written differently, refer to the same real-world entity, and merges them into one. We walk it the way the field is built: what it is, why it is hard, how a system goes from blocking to matching to merging, what it makes possible, and where it is heading. The papers behind each idea are linked, so you can read the source yourself.
Deciding which records are the same real thing.
One field, three names. Record linkage in statistics, entity matching in databases, entity resolution as the umbrella term. The job is the same.
Open almost any sales database and the same customer is in there more than once. "Acme Corp." and "Acme Corporation, Inc." and "ACME" are three rows, one company. A contact named "Bob Smith" with no email sits a few lines from "Robert Smith" with a work email. None of these rows is wrong on its own. The problem is that the system does not know they belong together.
Entity resolution is the work of deciding, across messy records, which ones describe the same real-world thing, a company, a person, a household, and then pulling them into one clean record. It shows up in two shapes: deduplication, finding duplicates inside one list, and linkage, matching records across two lists, for example a CRM export and a marketing list. The page uses the Acme and Bob Smith pairs throughout, and shows how a system reaches the conclusion that each pair is one entity.
| Source record | Observed values |
|---|---|
| CRM | Bob Smith · Acme |
| Robert Smith · [email protected] | |
| Import | B. Smith · phone 555-0142 |
Compare evidence before deciding whether these observations belong together.
"Same" and "equal" are not the same thing.
If every row carried the same clean customer ID, this would be a join, not a research field. In practice the ID is missing, the data is messy, and the scale is unforgiving.
There is rarely an ID to match on.
Two systems almost never share a clean customer ID. So a system cannot just check whether two rows are equal. It has to infer sameness from imperfect attributes: names, emails, addresses.
People type things differently.
Typos, abbreviations, nicknames, reordered names, blank fields, stale addresses, different formats. "Acme Corp." and "Acme Corporation" look different and mean the same company.
Comparing everything to everything explodes.
The naive approach grows with the square of the list size. Two lists of a hundred thousand rows is ten billion comparisons. You cannot afford to check them all. This is exactly what blocking exists to fix.
A real slice of pairs is genuinely ambiguous.
Most pairs are clearly a match or clearly not. A meaningful middle is honestly unclear. A good system models that uncertainty instead of pretending every call is clean.
A wrong merge and a missed merge both cost you.
Merging two different customers corrupts a record. Failing to merge two copies of one customer leaves the duplicate in place. The field measures both, with precision and recall, further down.
Examples of "same / not same" are slow to make.
Learned matchers need labeled pairs, and producing them by hand is slow. Much of the practical effort goes here, into the data and the judgment, not into the model.
A recognizable pipeline, stage by stage.
Group the candidates, judge each pair, then build one record per entity. The field has converged on this shape. Name it once, then walk it.
- 01Collect
Bring candidate records together.
- 02Block
Reduce the comparison set.
- 03Match
Weigh identity evidence.
- 04Merge
Apply field survivorship.
The pipeline the survey (Christophides et al., 2019) lays out: four jobs in a row, each doing one thing.
1Blocking
Shrink the problem first.
Instead of comparing every record against every other record, blocking groups records that share a cheap signal, the same first three letters of the company, the same email domain, the same postal code, and only compares within a group. This trades a little recall for an enormous cut in work, and it is what makes entity resolution tractable at scale.
The survey (Christophides et al., 2019) treats blocking as the load-bearing first stage, and frames modern blocking as having to handle volume, velocity, and variety at once. It stays an active research area rather than a solved preprocessing step.
- Organization domain
- acme.com → Acme Corp · ACME Corporation
- Another domain
- globex.com → Globex · Globex LLC
- Name key
- smi → Smith · Smithson
2Weigh the evidence
The classic decision rule.
The foundational framing is Fellegi and Sunter (1969). For each candidate pair, compare the fields. Each agreement or disagreement carries a weight that depends on how telling it is: two records agreeing on a rare surname is strong evidence, agreeing on a common first name is weak, agreeing on country is almost nothing.
Sum the weights into a score and apply two thresholds, giving three outcomes: link, non-link, and possible link, sent to a person. This three-way decision, with a deliberate "I am not sure" zone, is still the spine of the field. It is also the honest answer to the ambiguous middle from the section above.
| Signal | Interpretation |
|---|---|
| Verified email agrees | Strong identifying evidence in this example. |
| Company suffix differs | Likely formatting variation. |
| Country agrees | Weak evidence when most candidates share it. |
Non-link · human review · link
3Learned and deep matching
From hand-set weights to trained matchers.
Rather than hand-set the weights, a learned matcher trains a classifier on labeled pairs to predict match or no-match from field-level similarities: string distance on names, exact match on email, numeric closeness on amounts. Magellan (Konda et al., 2016) frames this as a guided, end-to-end workflow with tooling for the whole pipeline, not just the classifier, and makes the honest point that the unglamorous steps, labeling, cleaning, debugging, carry most of the weight.
DeepMatcher (Mudgal et al., 2018) maps a design space of neural matchers and shows where deep learning helps most, on messy, textual, free-form attributes where simple string rules fall short, and where simpler methods are already enough. As an attributed finding from that paper, it is a useful guide to when to reach for a neural model. Any specific accuracy figure belongs to the paper that reports it, not to anyone reading this page.
| Field | Comparison |
|---|---|
| Name | Bob Smith ↔ Robert Smith |
| Missing on one side | |
| Phone | Same normalized number |
| Company | Same resolved organization |
Score the pair, then apply calibrated decision thresholds.
4Language-model matching
Bringing outside knowledge of language.
Ditto (Li et al., 2020) casts a pair of records as a single sequence and fine-tunes a pre-trained language model to classify "same or not." Pre-training lets the model bring outside knowledge of language, so "Inc." and "Incorporated", or a nickname and a full name, read as related without a hand-written rule. As the paper's own claim, it reports gains over prior methods and the ability to reach comparable quality with fewer labels. Those are the paper's numbers, attributed to it, never figures about any product here.
[A] name: Bob Smith; org: Acme Corp
[B] name: Robert Smith; org: Acme Corporation- Language evidence
- Bob can be a variant of Robert; Corp can expand to Corporation.
- Identity evidence
- Check identifiers and conflicting details before linking.
5Merge and survivorship
Building the surviving record.
Once a cluster of records is judged to be one entity, the system builds the single surviving record: keep the most complete name, the most recent address, the verified email, and fill the gaps from siblings. This last step is where "one record" actually appears. The field calls the rules that pick the winning value for each field survivorship.
It is worth keeping two ideas distinct here. A valid match, deciding two rows are the same, is separate from a correct merge, picking the right value for every field. Both have to go well for the result to be trustworthy.
| Field | Record A | Record B |
|---|---|---|
| Name | Bob Smith | Robert Smith |
| Missing | [email protected] | |
| Phone | 555-0142 | Missing |
Robert “Bob” Smith · [email protected] · 555-0142
6How we know it works
Precision and recall, reported together.
The field measures matching against ground truth with two numbers. Precision asks: of the pairs we called a match, how many really were? Recall asks: of the pairs that truly match, how many did we catch? Good practice reports both, because optimizing one alone hides the other failure mode.
This is the honest counterpart to "errors cut both ways" from earlier. High precision with low recall means few wrong merges but many duplicates left in place. High recall with low precision means few duplicates left but some distinct customers wrongly merged. Naming both is how the field avoids implying that matching is ever perfect.
Precision = 8 / 10 = 80%
Recall = 8 / 12 ≈ 67%Clean records are raw material for everything built on top.
At the field level: a database with one record per real customer, counts you can trust, cleaner outreach, and a single view of a customer across systems.
This is the field that turns a pile of duplicate, half-filled, stale records into one trustworthy record per customer. Described at the outcome level, here is what that cleanliness looks like for a seller. These are examples of the problem the field solves, not claims about how any one product solves it.
The same account stops showing up twice.
When duplicates collapse into one record, a pipeline stops listing the same company under three spellings, and territory coverage stops being inflated by copies.
A half-filled contact comes back complete.
Matching a contact across lists means a record that was missing an email or a title can be filled from its twin, the survivorship step from above.
Counts you can trust.
Account counts and pipeline value reflect real customers, not copies. Reporting becomes believable because the underlying records are one-per-thing.
A single view across systems.
Record linkage is how a CRM record, a billing record, and a support record get recognized as the same customer, the precondition for one view of that account.
Everything here describes the field of entity resolution, not the internals of any product. This page explains how the work is done in general; it does not disclose how a given system is built.
The direction the field is heading.
We describe the direction, not a roadmap of our own.
Matchers that generalize across domains.
From Ditto (2020) onward, pre-trained language understanding lets a matcher carry across domains with fewer hand-built rules and fewer labels.
General models, used directly.
Peeters and Bizer (2023) study using general large language models for matching, including with little or no task-specific training data.
Good matching from fewer examples.
A steady push to reach quality with fewer human-labeled pairs, through transfer learning and active learning, since labeling is the practical bottleneck.
Keeping up with volume and variety.
The survey frames modern blocking as having to handle volume, velocity, and variety at once, which keeps it an active area.
Precision and recall, never one alone.
Good practice keeps reporting both against ground truth, naming the tradeoff rather than implying matching is ever perfect.
More eager linking
Finds more true duplicates, but can merge distinct entities.
More cautious linking
Avoids some false merges, but can leave duplicates unresolved.
Catching every duplicate and never merging two real customers pull against each other. The uncertain middle is a person's call.
The papers behind this page.
- Fellegi and Sunter (1969). A Theory for Record Linkage. Journal of the American Statistical Association.
- Christophides, Efthymiou, Palpanas, Papadakis, and Stefanidis (2019). An Overview of End-to-End Entity Resolution for Big Data: A Survey. arXiv:1905.06397.
- Konda, Das, Suganthan G. C., Doan, et al. (2016). Magellan: Toward Building Entity Matching Management Systems. PVLDB.
- Mudgal, Li, Rekatsinas, Doan, et al. (2018). Deep Learning for Entity Matching: A Design Space Exploration (DeepMatcher). SIGMOD.
- Li, Li, Suhara, Doan, and Tan (2020). Deep Entity Matching with Pre-Trained Language Models (Ditto). arXiv:2004.00584.
- Peeters and Bizer (2023). Entity Matching using Large Language Models. arXiv:2310.11244.
Keep reading.
Sibling explainers in the Research section.
Talk. Done.
One customer, one record is the goal of the whole field. See what clean, complete records feel like on a real call.
No app required. Just your phone number.