Normalizing names, addresses, and companies.
The same account, three ways in. One clean record out. Here is the field that makes that happen, in plain language.
Open any busy CRM and you will find one company living in it three times, a contact with half its fields blank, a deal parked against an account that merged months ago. Normalization is how the field turns records that look different but mean the same thing into one agreed, canonical form. We walk it the way the field is built: what it is, why "the same" is so hard to pin down, how it is done, what it unlocks, and where it is going. The papers behind each idea are linked, so you can read the source yourself.
The two ideas to carry: standardize a single value, then match whole records.
| Received | Transformation | Normalized |
|---|---|---|
Company nameAcme Corp. | → | Acme Corp.Collapse extra spaces |
Phone · US record(415) 555-0142 | → | +14155550142Use the known country |
DateSep 5, 2026 | → | 2026-09-05Year, month, day |
Original values remain available alongside the normalized fields.
One clean record per real-world thing.
The goal is plain: one contact, one address, one account, even when the source records arrive looking nothing alike.
Records get typed by hundreds of people, pulled from a dozen tools, and copied across years. So "Acme Corp," "ACME Corporation," and "Acme Corp. (West)" all point at one real business, but the system holds them as three. A contact arrives from a web form with a name and an email and nothing else. None of it is anyone's fault. It is what messy data looks like.
There are two jobs underneath the word, and it helps to keep them apart. The first is standardizing a single value: parse an address into its parts, fix the casing on a name, strip the "Inc." so two spellings line up. The second is matching, also called entity resolution, record linkage, or deduplication: deciding that two whole records, neither identical, refer to the same real person, place, or company, and should be merged. Standardizing makes the comparison fair. Matching makes the decision. The field survey (Christophides et al., 2019) is the map for this framing.
Normalize one value
"Smith, Robert"
→ "Robert Smith"Compare two records
Record A ↔ Record B
→ link, review, or non-link"The same" is a judgment, not a lookup.
There is no universal id stamped on a lead. Sameness has to be inferred from messy values, with a real chance of error in both directions.
Nothing arrives pre-keyed.
A lead does not show up stamped with a unique number that says which account it belongs to. The system has to infer that two records are the same from the values themselves, which is the whole difficulty.
Two records can agree on one field and disagree on the rest.
Two rows can share an email and conflict on everything else. Two people can share a name and not be the same person. Sameness is decided on evidence, with a genuine risk of a wrong call either way.
The same value gets written many ways.
Names invert ("Smith, Robert" against "Robert Smith"). Companies carry or drop "Inc." and "LLC." Addresses abbreviate ("St" against "Street") and reorder across countries. Comparing raw strings fails before it starts, which is why standardizing comes first.
Fields are often blank.
A contact with only a name and email. An account missing its industry or region. Matching has to work with gaps, not assume every field is present and filled.
Every record against every other is quadratic.
A million records is half a trillion comparisons. That does not run. The field cuts the work down with blocking, grouping plausibly related records so only those get compared in full. The survey (Christophides et al., 2019) treats blocking as a first-class stage, not a footnote.
A clean shape is not a correct merge.
A perfectly parsed address still does not prove two contacts at it are the same person. "Standardized value" and "correct match" are separate guarantees. We keep them separate all the way down this page, the way a careful reader expects.
Standardize, block, score, decide.
Make each value comparable, cut the comparisons down to plausible pairs, score how alike two records are, and route the confident calls automatically while a person handles the uncertain middle.
1Standardize first, so the comparison is fair
Parse the value into its parts.
Lowercase, trim, expand abbreviations, and split a value into the pieces that mean something. For an address that is parsing: label each token as house number, street, unit, city, region, postal code. The subword address-parsing work (Yassine et al., 2020) showed this can be a learned labeling task trained on large open address data, rather than a brittle stack of hand-written rules, and that it generalizes across many countries.
This step buys a clean shape, and nothing more. A tidy, well-labeled address does not yet tell you whether two contacts at it are the same person. That decision comes later, and we keep it separate.
123 main st apt 4, austin tx 78701
| Component | Parsed value |
|---|---|
| House number | 123 |
| Street | Main St |
| Unit | Apt 4 |
| City / region | Austin / TX |
| Postal code | 78701 |
2Block to make the problem tractable
Group the plausible pairs, then compare only those.
Before any careful comparison, group records that could plausibly match: same postal code, same first three letters of a company name, same email domain. Then compare only within a group. This trades a small risk of missing a pair for a large cut in work, and it is what lets resolution run on real data sizes. The survey (Christophides et al., 2019) lays this out as a named stage in the pipeline.
| Normalized key | Candidate group |
|---|---|
| Postal code: 78701 | Records sharing that postal code |
| Company prefix: acm | Acme name variants |
| Domain: acme.com | Records associated with that domain |
3Score how alike two records are
Confident calls automate. The uncertain middle goes to a person.
Within a block, compare records field by field with similarity measures: exact match on email, edit distance on names, token overlap on company. The foundational statistical frame is Fellegi and Sunter (1969): combine the per-field agreements into a likelihood ratio, then set two thresholds. Above the upper one, call it a match. Below the lower one, call it a non-match. In between, send it to a human to review.
That three-way decision, match, non-match, possible-match, is the honest core of the field. It does not pretend to certainty. It automates the confident cases and routes the genuinely ambiguous ones to a person, by design.
| Score region | Action |
|---|---|
| Below lower threshold | Non-match: keep separate |
| Between thresholds | Review the evidence |
| Above upper threshold | Link under the validated policy |
Thresholds are chosen against error costs and evaluation data.
4A worked example
Two rows, one canonical record.
Take the two rows from the hero. After standardizing, the company suffixes line up, the inverted name is reconciled, and the emails match exactly. Field by field, the agreements add up. The combined score lands above the upper threshold, so the system links the rows and keeps one canonical record, with the duplicate merged behind it.
Worth saying plainly: this verdict is a confident call, not a proof. The same machinery that merges this pair would route a thinner one, say a shared email but a conflicting name, to the review band instead.
| Field | Record A | Record B |
|---|---|---|
| Company | Acme Corporation | Acme Corporation |
| Name | Robert Smith | Robert Smith |
| [email protected] | [email protected] |
Check for conflicting identifiers before applying the link.
5Let a model learn the comparison
Learned matchers, and records read like language.
Hand-tuning rules and weights is slow and brittle. Mudgal et al. (2018), the DeepMatcher work mapped out how to learn the matcher with neural networks and reported an honest result: learned models help especially on textual and "dirty" records where values are noisy, while classic methods stay strong on clean, structured data. Deep learning is not a free win everywhere, it is a win where the text is messy.
The next step casts a pair of records as a single piece of text. Li et al. (2020), the Ditto work asks a pre-trained language model whether two records describe the same entity. Because the model already knows that "Corp" and "Corporation" are related, it handles variation that exact rules miss. Any accuracy gain reported in that paper is its result on its own benchmarks.
[A] company: Acme Corp; name: Robert Smith
[B] company: ACME Corporation; name: Smith, Robert- Language knowledge
- Recognize common abbreviations and ordering conventions.
- Matching evidence
- Check identifiers, context, and disagreements.
6Manage it as a system, not a one-off script
Resolution is an engineered workflow.
Real entity matching is a pipeline with stages: sampling, labeling, blocking, matching, and evaluation, each one to run, measure, and maintain. Konda et al. (2016), the Magellan work makes this case directly and builds tooling for the whole arc. The takeaway for a reader: resolution holds up because it is managed like a system, not because one clever function solved it in a single pass.
- Sample
Inspect real formats and edge cases.
- Define
Set parsing, formatting, and validation rules.
- Compare
Use normalized fields for candidate matching.
- Evaluate
Measure bad merges and missed matches; monitor new formats.
Clean, canonical records are what everything else assumes it has.
At the field level: a deduped account list, a contact you can actually use, a pipeline you can report on, and a foundation that analytics, enrichment, and automation all build on.
This family of techniques is how the industry turns messy, duplicated records into one clean, trustworthy record. Here is what that cleanliness looks like as an everyday outcome for a seller, stated as outcomes, not as a claim about how any one system is built.
One account, not three.
Every real company appears once, with its duplicates linked and merged, so two reps stop emailing the same prospect under two records.
A contact you can actually use.
Gaps filled from the matched records, name and address in standard fields, so the row is usable by the next system instead of half-blank.
A pipeline you can report on.
When deals attach to clean, single accounts, totals, territory splits, and forecasts stop double-counting. The numbers start to mean something.
A foundation to build on.
Clean, canonical records are the input that analytics, enrichment, and automation all assume they have. Normalization is what makes that assumption true.
This explainer covers published research. To see how AI REV Voice helps with sales conversations, explore Prism.
The direction the field is heading.
We describe the direction, not a roadmap of our own.
Reading variation as meaning.
The clear direction is models that understand that "Corp" and "Corporation," or two phrasings of one address, mean the same thing, so resolution holds up on noisy, free-text data.
Less labeled data needed.
Matching has long needed many human-labeled example pairs. The frontier is doing more with fewer labels, through transfer from pre-trained models and active learning that asks a human only about the genuinely uncertain pairs.
Automate the confident, route the ambiguous.
The mature posture is not "automate every decision." It is automate the confident ones and send the ambiguous ones to a person. The possible-match zone from Fellegi and Sunter (1969) is still the right shape for this.
Shared benchmarks, real numbers.
The field increasingly measures resolution on shared datasets, so "it works" becomes a precision and recall figure you can compare, rather than a claim.
Clean shape plus a correct merge.
The open frontier is pairing a clean, canonical shape with confidence that the merge was actually right, so a tidy record is never mistaken for a proven one.
The papers behind this page.
- Fellegi and Sunter (1969). A Theory for Record Linkage. Journal of the American Statistical Association.
- Christophides, Efthymiou, Palpanas, Papadakis, and Stefanidis (2019). End-to-End Entity Resolution for Big Data: A Survey. arXiv:1905.06397.
- Konda, Das, et al. (2016). Magellan: Toward Building Entity Matching Management Systems. PVLDB.
- Mudgal, Li, Rekatsinas, Doan, et al. (2018). Deep Learning for Entity Matching: A Design Space Exploration (DeepMatcher). SIGMOD.
- Li, Li, Suhara, Doan, and Tan (2020). Deep Entity Matching with Pre-Trained Language Models (Ditto). arXiv:2004.00584.
- Yassine, Beauchemin, Laviolette, and Lamontagne (2020). Leveraging Subword Embeddings for Multinational Address Parsing. arXiv:2006.16152.
Keep reading.
Sibling explainers in the Research section.
Talk. Done.
Clean records are the quiet payoff behind a co-pilot you can just talk to. See it on a real call.
No app required. Just your phone number.