Noise suppression and echo cancellation.
A call from a cafe, a fan running, the echo of your own voice coming back. Here is the field that keeps the voice and drops the rest, in plain language.
Real conversations rarely happen in a quiet room. Noise suppression removes the unwanted background sound while keeping the voice. Echo cancellation removes the part of the signal that is just a returning copy of what the other side already said. We walk it the way the field is built: what it is, why it is hard, how a system does it, what it makes possible, and where it is heading. The papers behind each idea are linked, so you can read the source yourself.
The three ideas to carry: keep the voice and drop the rest, the time-frequency mask, and the echo reference signal.
Keep the voice. Drop the rest.
Two related jobs on the input audio, before recognition: remove unwanted background sound, and remove the returning copy of the far-end voice.
Noise suppression, also called speech enhancement, is the work of removing background sound while keeping the voice intact. Echo cancellation, often written acoustic echo cancellation or AEC, is the related work of removing the part of the signal that is only a reflection of what the other side already said: their voice plays out of your speaker, your microphone picks it back up, and it returns to them as a delayed copy of their own words.
The two look similar but rest on a real difference. Noise has no clean reference to compare against, you only have the mixture. Echo has a reference: you know exactly what played out of the speaker, and that known signal is what makes echo cancellation a different problem with a different toolkit. Both sit early in many voice pipelines, before recognition, because cleaner input audio leads to better transcripts. That hand-off to speech-to-text is why this cleanup step comes first.
- Microphone mixture
- Near-end speech + background noise + speaker echo.
- Cleanup
- Estimate the unwanted components and suppress them.
- Downstream input
- Pass the enhanced signal to recognition or communication.
The unwanted sound hides inside the voice.
If noise and echo sat neatly to one side, you could just cut them out. They do not. They overlap the voice in time and frequency, and they keep changing.
The noise will not hold still.
A steady hum is easy to estimate once and subtract. Real noise is bursty and unpredictable, a sudden bark, a clatter, a door, so a profile measured a second ago is already wrong.
Voice and noise share the same frequencies.
Because they sit in the same place, naive subtraction also removes voice and leaves artifacts: the watery, warbling residue that early methods were known for, called musical noise.
The route from speaker to microphone keeps shifting.
It changes with volume, with the room, and with any movement, so the canceler has to keep re-learning it rather than solving it once.
Both sides speaking at once.
The classic failure point for echo cancellation. The canceler must not mistake the near-end voice for echo and delete it, even while the far-end voice is still coming back.
It has to be cheap as well as accurate.
This stage runs frame by frame at low latency, often on modest hardware, so a method that is accurate but slow does not help. Part of the latency budget is spent right here. More in Low-latency voice.
What sounds clean is a judgment.
Quality is perceptual, so the field leans on subjective listening tests, crowdsourced mean opinion scores under ITU-T P.808, not signal-to-noise math alone.
From hand-tuned filters to learned models.
The field grew out of classic signal processing and moved toward neural networks. The clearest way to follow it is to watch the same job get done in better ways.
1The classic baseline
Measure the noise, then subtract it.
The original approach is spectral subtraction: estimate the noise spectrum during the quiet stretches when no one is speaking, then subtract its magnitude from the mixture and reconstruct the audio. It is cheap and intuitive, and it works well for steady noise. It is also the reference everyone improved on, set out in Boll (1979).
Its weakness is exactly the hard case above. When the noise is non-stationary or overlaps the voice, subtracting a stale estimate removes real speech and leaves the warbling musical-noise artifact. That gap is what later methods set out to close.
- Analyze frames
Convert short audio windows into spectral information.
- Estimate noise
Use a noise model or suitable reference frames.
- Suppress the estimate
Reduce the estimated noise contribution.
- Reconstruct
Convert the enhanced representation back to audio.
2Adaptive filtering for echo
Predict the echo, then remove the copy.
Echo cancellation has an advantage noise does not: a reference signal. You know what played out of the speaker. A classic adaptive filter, from the least-mean-squares family, models the path from speaker to microphone and uses the reference to predict the echo, then subtracts that predicted copy from the microphone signal.
Because the path keeps moving, the filter keeps adapting. A double-talk detector pauses that adaptation when the near-end speaks, so the filter does not mistake a live voice for echo and remove it. This is the linear, signal-processing core that learned methods later built on.
- Known reference
Audio being played through the local speaker.
- Room path
The speaker output returns through reflections to the microphone.
- Echo estimate
Model that path using the reference.
- Cancel
Subtract the predicted echo while preserving the local speaker’s voice.
3The modern core idea
A mask that keeps or drops each point.
Treat the audio as a spectrogram, a grid of time-frequency points, and have a model predict a gain, or mask, for each point: near 1 where there is voice, near 0 where there is only noise. Multiply the mask back into the mixture and the voice stays while the wash drops away. This masking idea is the shape many learned enhancers take.
Unlike a single noise estimate, the mask is predicted fresh for every point in time, so it can track noise that changes from moment to moment. That is the direct answer to the non-stationary problem the classic baseline struggled with.
Mixture
Retained speech regions
4The hybrid approach
A small network, helped by cheap signal processing.
A full neural enhancer can be expensive. A practical line of work keeps the network small by having it predict gains over a handful of perceptual frequency bands, while classic signal processing handles the rest, so the whole thing runs full-band in real time on a low-power CPU. That hybrid pattern is set out in Valin (2018), known as RNNoise.
The thread continues in low-complexity, perceptually motivated designs such as DeepFilterNet (Schröter et al., 2022), a representative modern enhancer that runs full-band while keeping the compute budget small. The goal is the same throughout: accurate enough to sound clean, cheap enough to run live.
- Features
Compute compact spectral and pitch features.
- Small network
Estimate gains or other enhancement parameters.
- Signal processing
Apply the estimates to the full audio signal.
5Learned echo cancellation
A model that uses the reference too.
Newer echo cancellation adds or replaces the adaptive filter with a neural model that takes the far-end reference signal as an extra input. Because it is learned rather than purely linear, it can handle the nonlinear distortion that linear filters miss: the clipping and coloring of cheap speakers driven loud.
It also helps with double-talk, the case where both sides speak at once, by learning to tell the near-end voice apart from the returning echo instead of relying on a hard detector to pause adaptation.
Reference input
Remote participant’s audio sent to the speaker.
Microphone input
Local voice plus echo and noise.
Use the relationship between the reference and microphone signals.
6What made it move
Shared data and shared listening tests.
Method aside, a large part of the recent progress came from common ground. Shared training data plus a common listening-test framework let everyone compare methods fairly. The Deep Noise Suppression Challenge (Reddy et al., 2020) provided the data and a crowdsourced subjective test under ITU-T P.808 for noise suppression.
The Acoustic Echo Cancellation Challenge (Sridhar, Cutler et al., 2020) did the same for echo, with real-device recordings for single-talk and double-talk and a standard way to score them. Open benchmarks like these are a clear entry point if you want to see how the field measures itself.
| Check | What to listen or measure for |
|---|---|
| Speech clarity | Are the words still intelligible? |
| Residual noise | How much unwanted sound remains? |
| Artifacts | Does the enhancement introduce warbling or distortion? |
| Double-talk | Are both intended speech turns preserved? |
Clean input is the quiet foundation for everything downstream.
At the field level: more accurate recognition, less listener fatigue on long calls, and systems that hold up in the places real conversations happen.
The through-line is simple: the cleaner the input, the better everything after it works. The hard cases from the section above, the bursty noise, the moving echo path, the double-talk, are exactly the ones that decide whether a call from a noisy room ends up as a usable transcript or a frustrating one.
Cleaner input, better transcripts.
Cleaner input audio raises transcription accuracy, especially in noisy real-world settings. That is why this stage sits early, ahead of speech-to-text.
Less listener fatigue on long calls.
On long calls, removing background noise and echo lowers the effort of listening for everyone on the line.
Holds up where conversations happen.
A system that works in a cafe, a car, or an open office is usable where conversations actually happen, not only in a quiet booth.
This explainer covers published research. To see how AI REV Voice helps with sales conversations, explore Prism.
The direction the field is heading.
We describe the direction, not a roadmap of our own.
Real time on commodity hardware.
Full-band models that run in real time on ordinary CPUs rather than needing a GPU, so good enhancement is not reserved for heavy machines.
Lock onto one voice.
Models that learn the target speaker's voice and suppress everything else, including other voices. A thread running through the later challenge tracks.
Fold cleanup into recognition.
End-to-end designs that treat enhancement and recognition together rather than as separate boxes, with tighter algorithmic and buffering latency budgets.
Smaller budgets for the cleanup stage.
As an attributed industry figure, the Acoustic Echo Cancellation Challenge has driven the target latency down toward roughly 20 ms. Figures like that belong to the systems and benchmarks that publish them.
Train for how people rate quality.
Training objectives aligned with subjective listening scores, so models optimize for what sounds clean to a person, not only for signal math.
The papers behind this page.
- Boll (1979). Suppression of Acoustic Noise in Speech Using Spectral Subtraction. IEEE Trans. on Acoustics, Speech, and Signal Processing.
- Valin (2018). A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement (RNNoise). arXiv:1709.08243.
- Reddy et al. (2020). The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results. arXiv:2005.13981.
- Sridhar, Cutler et al. (2020). ICASSP 2021 Acoustic Echo Cancellation Challenge: Datasets, Testing Framework, and Results. arXiv:2009.04972.
- Schröter et al. (2022). DeepFilterNet: A Low Complexity Speech Enhancement Framework for Full-Band Audio based on Deep Filtering. arXiv:2110.05588.
Keep reading.
Sibling explainers in the Research section.
Talk. Done.
The cafe, the car, the open office. You talk, and clean, complete notes are waiting. See it on a real call.
No app required. Just your phone number.