PII redaction and privacy-preserving ML.
Useful data is usually personal too. Here is the field that lets machines learn from it without exposing the people inside.
A transcript, a support ticket, a sales call: the value is in the words, and the words carry names, phone numbers, and account IDs. Two related fields keep the signal while reducing the chance any individual can be picked back out. PII redaction finds and removes the identifying parts of text. Privacy-preserving machine learning trains and serves models so the model itself does not leak who was in the data. We walk both the way the field is built, and link the papers behind each idea.
Call Dana Ruiz at (415) 555-0182 about account AC-90432.
Call [PERSON] at [PHONE] about account [ACCOUNT].
Two layers, one goal.
Redaction works on the data. Privacy-preserving ML works on the learning process. Both matter, and redaction alone is not the whole story.
PII, personally identifiable information, is anything that can single someone out, on its own or combined with other facts. The direct case is familiar: a name, a phone number, an email, a government ID, an account number. The subtle case is a combination of innocuous fields. A date plus a ZIP plus a job title can point to one person even with every obvious identifier removed, which is why the field talks about quasi-identifiers.
It helps to name three things the field keeps distinct, and that regulators treat differently. De-identification removes or obscures identifiers. Pseudonymization swaps an identifier for a consistent token that can be reversed with a key. Anonymization aims for data where re-identification risk is low enough that it is no longer personal data. These are not synonyms.
Finding PII in free text is mostly a named-entity-recognition problem: label each span by type, such as PERSON, PHONE, or ID. Real systems combine machine-learned recognizers with deterministic pattern rules and context, an approach made concrete by the open framework Microsoft Presidio and by neural de-identification work such as Dernoncourt et al. (2017).
Redaction
- Target
- Sensitive spans in documents, transcripts, or outputs.
Privacy-preserving learning
- Target
- What training or inference may reveal about individual data.
Privacy and usefulness pull against each other.
Removing the obvious names is not a guarantee. The hard part is what remains, what a model remembers, and how to reason about it.
Models can repeat what they learned.
Large models can store and later regurgitate verbatim training examples, including rare PII. So "we removed the names" is not a privacy guarantee on its own. This is the central reason privacy-preserving ML exists.
A missed span leaks a real person.
A detector that misses one span exposes someone; one that over-redacts destroys the meaning. The trade-off is precision against recall, and the cost of a miss is asymmetric. No detector has perfect recall on open-ended text.
Identity hides in combinations.
Remove the direct identifiers and quasi-identifiers can still point to one person through linkage to outside data. Classic results show a large share of a population identifiable from a few innocuous fields. Anonymization is a risk estimate, not a binary state.
More privacy, less utility.
More noise or more masking means more privacy and less usefulness. The field's job is to make that trade-off explicit and tunable rather than pretending it does not exist.
"Anonymous" is too vague to reason about.
Privacy needs a definition with a knob you can turn and verify. That is what differential privacy (Dwork et al., 2006) contributes: a precise, composable definition that lets practitioners make and check claims.
Listing entity types is not enough.
The text-anonymization community is moving from "tag the spans" toward estimating disclosure risk directly, accounting for quasi-identifiers and linkage, as surveyed in Lison et al. (2021).
From hiding the obvious to reasoning about risk.
Redaction handles the visible text. Differential privacy, DP-SGD, and federated learning handle what a model can learn and leak.
1The redaction pipeline
Detect, then transform.
Redaction is two steps, not one. Detect: NER models plus pattern recognizers plus context find the sensitive spans. Then transform: remove, mask, hash, encrypt, or replace each span with a realistic stand-in so the text still reads naturally. Keeping detection separate from transformation keeps each step auditable, which is the structure Presidio makes concrete and that neural de-identification such as Dernoncourt et al. (2017) demonstrates on free text.
Call Dana Ruiz at 415-555-0182.
| Detected span | Type | Replacement |
|---|---|---|
| Dana Ruiz | Person | [PERSON_1] |
| 415-555-0182 | Phone | [PHONE_1] |
Call [PERSON_1] at [PHONE_1].
2Reasoning about linkage
k-anonymity and quasi-identifiers.
An early structured-data idea bridges "hide the obvious" and "reason about linkage." k-anonymity generalizes or suppresses quasi-identifiers until every record is indistinguishable from at least k-1 others, so no single record stands out by its combination of fields. It is useful to name as the step before a precise definition, and as the thing differential privacy was designed to improve on.
Direct identifiers
Quasi-identifiers
A rare combination can single out a person after the direct identifiers are removed.
3Differential privacy
Give privacy a number.
A randomized process is differentially private if adding or removing any single individual's data barely changes the distribution of outputs. The amount is quantified by epsilon, where a smaller epsilon means stronger privacy. It is achieved by calibrating noise to a query's sensitivity, how much one person can move the answer. This is the rigorous core, introduced in Dwork et al. (2006). Any specific epsilon value belongs to the study that reports it, presented as the field's figure, never as a property of any system a reader is evaluating here.
- Neighboring datasets
- Differ in the contribution of one protected individual, under the stated definition.
- Epsilon and delta
- Quantify the allowed difference in output distributions.
- Utility
- Measure the accuracy retained for the intended task.
4DP for deep learning
Bound each example, add measured noise.
DP-SGD brings differential privacy into model training. Clip each example's gradient to bound its influence, add Gaussian noise to the aggregate, and track the cumulative privacy loss with a moments accountant. The result is a trained model with a stated privacy budget, so the model learns the pattern rather than the person. The method is set out in Abadi et al. (2016).
- Per-example gradients
Compute each example’s contribution.
- Clip
Bound the contribution’s norm.
- Aggregate with noise
Add calibrated noise to the aggregate.
- Account
Track the privacy cost across training steps.
5Federated learning
Move the model, not the data.
Instead of collecting raw data centrally, send the model to where the data lives, train locally, and send back only model updates that are averaged together. The raw examples stay put. The approach is introduced in McMahan et al. (2017), and it pairs naturally with differential privacy and secure aggregation so the updates themselves do not leak.
- Distribute the model
Send the current parameters to participating devices.
- Train locally
Each device computes an update from its own data.
- Aggregate
Combine updates to produce the next shared model.
6Synthetic replacement
Swap in plausible stand-ins.
For text that must stay readable, detected PII can be replaced with plausible fake values, a well-formed name and phone that are not real, so downstream tasks still work without the real identity. One honest caveat to carry: synthetic stand-ins help readability and reduce exposure, but they are not, by themselves, a formal privacy guarantee. The survey by Lison et al. (2021) sets this in the wider context of measuring disclosure risk rather than only listing entity types.
Dana Ruiz will call Acme tomorrow.
Jordan Avery will call Northstar tomorrow.
Keep a consistent mapping when references recur within the permitted scope.
Work with sensitive data responsibly.
At the field level: build on sensitive text while reducing re-identification risk, state a privacy budget you can reason about, and learn without centralizing.
Sales conversations are full of personal details: customer names, contact information, account numbers, and what was actually said. Anyone evaluating a system that touches those conversations wants to understand how a responsible team thinks about the personal parts. This page exists so a technical reader can understand that field on its own terms.
Use sensitive data responsibly.
Build and evaluate models on transcripts, notes, and tickets while reducing the chance that any individual can be picked back out.
Measurable privacy, not vibes.
Differential privacy lets a team state a budget and reason about it, instead of asserting "anonymous" with no way to check.
Learn without centralizing.
Federated approaches let a model improve from data that, for policy or practical reasons, cannot be pooled into one place.
Cleaner sharing and logging.
Redaction lets teams share examples, debug, and analyze patterns with the identifying parts removed up front.
For AI REV Voice's data protection and access controls, see Security & privacy. This explainer covers the underlying research.
The direction the field is heading.
We describe the direction, not a roadmap of our own. One honest line to close on: every method is a trade-off, and "anonymized" is a risk estimate, not a guarantee.
Estimate disclosure risk directly.
Text-anonymization research is moving from listing entity types toward models that estimate disclosure risk, accounting for quasi-identifiers and linkage, as argued in Lison et al. (2021).
Train and serve LLMs with privacy in mind.
Active work on training and serving language models with differential privacy, on measuring and reducing memorization, and on protecting user prompts at inference time.
Guarantees across the whole pipeline.
Combining federated learning, secure aggregation, and differential privacy so guarantees hold across the whole pipeline, not just one stage.
Better accountants, shared evaluation.
Improved privacy accountants, shared evaluation for re-identification risk, and open tooling such as Presidio lowering the bar to do this well.
State the limits plainly.
No detector has perfect recall, and "anonymized" means low re-identification risk, not zero. Naming that honestly is what reads as credibility.
The papers behind this page.
- Dwork, McSherry, Nissim, Smith (2006). Calibrating Noise to Sensitivity in Private Data Analysis. TCC 2006.
- Abadi et al. (2016). Deep Learning with Differential Privacy (DP-SGD). arXiv:1607.00133.
- McMahan et al. (2017). Communication-Efficient Learning of Deep Networks from Decentralized Data. arXiv:1602.05629.
- Dernoncourt et al. (2017). De-identification of Patient Notes with Recurrent Neural Networks. arXiv:1606.03475.
- Lison, Pilán, Sánchez, Batet, Øvrelid (2021). Anonymisation Models for Text Data: State of the Art, Challenges and Future Directions. ACL 2021.
- Microsoft Presidio. Open-source PII detection and anonymization framework (docs). Cited as a representative tool, not an endorsement.
Keep reading.
Sibling explainers in the Research section.
Talk. Done.
Personal details in sales conversations deserve to be handled to a high standard, with the methods and their trade-offs clearly understood. See it on a real call.
No app required. Just your phone number.