A speech enhancement model built on purpose to exploit PESQ scored 3.82 on the VoiceBank-DEMAND test set, above the 3.69 that the best legitimate system in this roundup reports, while its POLQA score collapsed to 1.46 against 3.11 for the untouched noisy input (de Oliveira et al., PESQetarian paper, Interspeech 2024). The legitimate race is real too: PESQ on the same benchmark climbed from 2.16 for SEGAN in 2017 to 3.61 for PrimeK-Net in 2025. Yet STOI, the intelligibility metric often printed next to PESQ, averaged a correlation of only 0.62 with measured intelligibility across 13 listening datasets (Edraki et al., IEEE/ACM TASLP 2020). We aggregated data from ITU-T, peer-reviewed papers at Interspeech, ICASSP, IEEE SLT and WASPAA, IEEE/ACM Transactions on Audio, Speech, and Language Processing, the Microsoft Deep Noise Suppression (DNS) Challenge, and the URGENT and CCF AATC challenge reports. For the latency and product side of the same models, see our neural noise suppression latency statistics; this page is about the scores themselves and how far to trust them.
TL;DR
- PESQ on VoiceBank-DEMAND rose from 2.16 (SEGAN, 2017) to 3.41 (CMGAN, 2022) (Cao et al., CMGAN paper, Interspeech 2022).
- SEMamba reached 3.55 PESQ, and 3.69 with perceptual contrast stretching added (Chao et al., IEEE SLT 2024).
- PrimeK-Net reported 3.61 PESQ with only 1.41M parameters (PrimeK-Net paper, ICASSP 2025).
- The PESQetarian hit 3.82 PESQ with an SI-SDR of -19.8 dB, against 8.4 dB for the noisy input (de Oliveira et al., Interspeech 2024).
- STOI averaged a Pearson correlation of 0.62 with measured intelligibility across 13 datasets, eSTOI 0.88 (Edraki et al., IEEE/ACM TASLP 2020).
- DNSMOS P.835 speech-quality correlation fell from 0.94 per model to 0.71 per clip (Reddy et al., Microsoft, DNSMOS P.835 paper 2021).
- Conv-TasNet’s ESTOI generalization gap on an unseen speech corpus was -68% against a matched reference model (Gonzalez et al., IEEE/ACM TASLP 2023).
- ROSE-CD’s PESQ dropped from 3.49 in-domain to 2.62 out-of-domain (Xu et al., WASPAA 2025).
- Reference-free metrics showed a rank correlation of -0.8 with listener MOS for hybrid systems (Zhang et al., CCF AATC 2025 retrospective).
- ITU-T deleted PESQ (P.862) on January 5, 2024 (ITU-T P.862 recommendation page).
1. The PESQ Race on VoiceBank-DEMAND, 2017 to 2025
PESQ (Perceptual Evaluation of Speech Quality) compares an enhanced recording with its clean original and predicts the score a listener would give. VoiceBank-DEMAND, a small English test set of read sentences mixed with recorded everyday noise, became the default place to report it. PESQ climbed from 2.16 for SEGAN in 2017 to 3.69 for SEMamba with perceptual contrast stretching in 2024, and the pace tells its own story.
From SEGAN (2.16) to MetricGAN+ (3.15) the field added 0.99 PESQ points in four years. From CMGAN (3.41, 2022) to PrimeK-Net (3.61, 2025) it added 0.20 in three (our subtraction from the published figures). The benchmark is close to saturation, so the remaining gains are small enough that add-on techniques and metric-aware training can decide the leaderboard (see section 5). Sources: CMGAN paper, SEMamba paper, PrimeK-Net paper.
| Metric | Value | Source |
|---|---|---|
| PESQ: unprocessed noisy test set vs SEGAN, a GAN-based enhancer (2017) | 1.97 vs 2.16 | Cao et al., CMGAN paper, Interspeech 2022 |
| PESQ of MetricGAN+ (2021) and its gain over MetricGAN | 3.15; about +0.3 | MetricGAN+ paper, Interspeech 2021 |
| PESQ of CMGAN (2022) | 3.41 | Cao et al., CMGAN paper, Interspeech 2022 |
| PESQ of MP-SENet, parallel magnitude and phase denoising (2023) | 3.50 | MP-SENet paper, Interspeech 2023 |
| PESQ of SEMamba, non-causal: without vs with perceptual contrast stretching | 3.55 vs 3.69 | Chao et al., SEMamba paper, IEEE SLT 2024 |
| PESQ of PrimeK-Net (2025) and its model size | 3.61; 1.41M parameters | PrimeK-Net paper, ICASSP 2025 |
Context note: these are each paper’s own reported numbers on the same public test set, not results from one independent re-run of every system.
2. What the Benchmark Actually Measures
The leaderboard rests on a narrow base. The VoiceBank-DEMAND test set has 824 utterances from just 2 unseen speakers, mixed with 5 noise types at fairly gentle signal-to-noise ratios (SNR, how much louder the speech is than the noise) of 2.5 to 17.5 dB. No test clip is mixed below 2.5 dB, so the hardest end of real calls is simply not sampled.
Intelligibility leaves almost no headroom. The noisy input already scores 0.91 on STOI (Short-Time Objective Intelligibility, a predictor of how much of the speech a listener would understand), and CMGAN reaches 0.96. Model size, meanwhile, fell by more than an order of magnitude: SEGAN used 97.47M parameters, CMGAN 1.83M. Overlapping-talker benchmarks work differently, as our speech separation statistics show. Source: CMGAN paper, dataset and results sections.
| Metric | Value | Source |
|---|---|---|
| Training set: size and noise conditions | 11,572 utterances from 28 speakers; 8 DEMAND noise types plus 2 artificial noise types, at SNRs of 0, 5, 10 and 15 dB | Cao et al., CMGAN paper, Interspeech 2022 |
| Test set: size and noise conditions | 824 utterances from 2 unseen speakers; 5 unseen DEMAND noise types, at SNRs of 2.5, 7.5, 12.5 and 17.5 dB | Cao et al., CMGAN paper, Interspeech 2022 |
| STOI: noisy input vs CMGAN output | 0.91 vs 0.96 | Cao et al., CMGAN paper, Interspeech 2022 |
| Composite speech-signal score (CSIG) and overall score (COVL): noisy input vs CMGAN output | 3.35 and 2.63 vs 4.63 and 4.12 | Cao et al., CMGAN paper, Interspeech 2022 |
| Model size: SEGAN vs DEMUCS vs CMGAN | 97.47M vs 128M vs 1.83M parameters | Cao et al., CMGAN paper, Interspeech 2022 |
Context note: CSIG and COVL are composite measures built from several objective metrics to predict listener ratings of the speech signal and of overall quality. They inherit the blind spots of the metrics they combine.
3. Do Quality Metrics Agree With Human Listeners?
The answer depends on whether you average. Correlation here is measured with PCC (Pearson correlation, agreement in value), SRCC (Spearman rank correlation, agreement in order) or KRCC (Kendall rank correlation, another test of order); 1.0 is perfect agreement. DNSMOS P.835, a neural network that predicts listener ratings without a clean reference, reached a PCC of 0.94 with listener speech ratings when averaged per model, but 0.71 on individual clips.
The human side is noisy too. Two separate P.808 rating runs on the same clips agreed at a PCC of only about 0.5 per clip, so a single-clip MOS (mean opinion score, the average of listener ratings) is itself a rough number. It is one more reason to compare systems, not files, and to treat one leaderboard flip as weak evidence. The same per-clip caution applies to synthetic voices, covered in our synthetic speech naturalness MOS statistics. Sources: DNSMOS P.835 paper, DNSMOS paper, ICASSP 2022 DNS Challenge paper, NISQA paper.
| Metric | Value | Source |
|---|---|---|
| DNSMOS P.835 speech score (SIG) vs listener ratings: PCC per model vs per clip | 0.94 vs 0.71 | Reddy et al., Microsoft, DNSMOS P.835 paper 2021 |
| DNSMOS P.835 background-noise score (BAK) vs listener ratings: PCC per model vs per clip | 0.98 vs 0.83 | Reddy et al., Microsoft, DNSMOS P.835 paper 2021 |
| Agreement between two separate human P.808 rating runs on the same clips (PCC per clip) | about 0.5 | Reddy et al., Microsoft, DNSMOS paper 2020 |
| DNSMOS P.835 vs listener ratings in ICASSP 2022 DNS Challenge Track 1, speech score: PCC vs SRCC | 0.93 vs 0.78 | Microsoft, ICASSP 2022 DNS Challenge paper |
| Original DNSMOS rank correlation (SRCC) across categories of the DNS Challenge 2 blind set: single-stage vs multi-stage model | 0.65 vs 0.90 | Reddy et al., Microsoft, DNSMOS paper 2020 |
| NISQA, another neural quality predictor: new training and validation speech files, and datasets used for training and evaluation | over 13,000 files; 81 datasets | Mittag et al., NISQA paper, Interspeech 2021 |
| URGENT 2024: KRCC between the challenge’s overall ranking score and the listener MOS rank | 0.73 | Zhang, Saijo et al., URGENT 2024 lessons paper, Interspeech 2025 |
Context note: DNSMOS P.835 was trained on about 30,000 rated clips (75 hours), the original DNSMOS on about 120,000 clips. Per-model correlations were computed with 5 ratings per clip, and the clip-level analysis used 30 ratings per clip, so the two figures in each row are not from identical setups.
4. Intelligibility Metrics: STOI and Its Successors
STOI was built to answer a narrower question than PESQ: not how pleasant the speech sounds, but how many words survive. Across 13 listening datasets, STOI averaged a Pearson correlation of 0.62 with measured intelligibility, while the extended version eSTOI averaged 0.88. On one dataset of speech in strongly fluctuating noise, STOI’s correlation was negative (-0.77): the metric moved the wrong way.
The practical reading is that STOI works best for the steady noise and time-frequency masking it was designed around, and least for speech in bursty, modulated noise, the kind that competing talkers and crowded rooms produce. Most recent available data: Edraki et al., IEEE/ACM TASLP 2020. Sources: Edraki et al. on PubMed Central, Taal et al., IEEE TASLP 2011.
| Metric | Value | Source |
|---|---|---|
| STOI analysis segment length | 386 ms | Taal et al., IEEE TASLP 2011 |
| STOI validation in its original paper | 3 listening experiments; better correlation than 5 reference intelligibility models | Taal et al., IEEE TASLP 2011 |
| Mean Pearson correlation with measured intelligibility across 13 datasets, after logistic mapping: STOI vs eSTOI vs HASPI vs wSTMI | 0.62 vs 0.88 vs 0.82 vs 0.93 | Edraki et al., IEEE/ACM TASLP 2020 |
| Mean Spearman rank correlation across the same 13 datasets: STOI vs eSTOI | 0.59 vs 0.82 | Edraki et al., IEEE/ACM TASLP 2020 |
| STOI Pearson correlation on a dataset of speech in modulated noise (ModN-Fogerty) | -0.77 | Edraki et al., IEEE/ACM TASLP 2020 |
| STOI Pearson correlation on a near-end listening enhancement dataset (NELE-Taal) | 0.90 | Edraki et al., IEEE/ACM TASLP 2020 |
| Citations of the original STOI paper | 2,265 | OpenAlex record, accessed October 31, 2026 |
Outlier note: wSTMI, the best performer, was trained and validated on two of the 13 datasets, so its average is not fully held-out. HASPI is a metric designed with hearing-impaired listeners in mind, and eSTOI drops STOI’s assumption that frequency bands are independent.
5. Goodhart’s Law: When PESQ Stops Measuring Quality
Goodhart’s law says a measure that becomes a target stops being a good measure. Speech enhancement now has the data to prove it. The PESQetarian, a model trained to maximize PESQ, scored 3.82 on VoiceBank-DEMAND while its SI-SDR (scale-invariant signal-to-distortion ratio, how close the waveform is to the clean original) fell to -19.8 dB, far below the 8.4 dB of the unprocessed noisy input.
The effect is not limited to one paper. A 2025 one-step diffusion model trained with a PESQ loss reached 3.99 PESQ but only 0.40 dB SI-SDR, while the authors’ balanced version scored 3.49 PESQ and 17.8 dB SI-SDR. Even legitimate techniques move the number: perceptual contrast stretching, which boosts the contrast of target features by perceptual importance, added 0.14 PESQ to SEMamba (3.55 to 3.69, our subtraction). The lesson for buyers and reviewers is simple: a PESQ record with no matching listening test is a warning sign, not a headline. Sources: PESQetarian paper, ROSE-CD paper.
| Metric | Value | Source |
|---|---|---|
| PESQ: noisy input vs plain MSE-trained model vs PESQetarian | 1.97 vs 2.78 vs 3.82 | de Oliveira et al., PESQetarian paper, Interspeech 2024 |
| POLQA (the ITU-T successor to PESQ): noisy input vs MSE model vs PESQetarian | 3.11 vs 3.87 vs 1.46 | de Oliveira et al., PESQetarian paper, Interspeech 2024 |
| DNSMOS overall score: noisy input vs MSE model vs PESQetarian | 2.69 vs 3.13 vs 2.06 | de Oliveira et al., PESQetarian paper, Interspeech 2024 |
| PESQ after adding one fixed click to every noisy test file, with no denoising | 3.46 | de Oliveira et al., PESQetarian paper, Interspeech 2024 |
| Direct optimization on six files: PESQ before vs after, and POLQA before vs after | 1.91 to 3.76; 3.46 to 2.73 | de Oliveira et al., PESQetarian paper, Interspeech 2024 |
| ROSE-CD trained with a PESQ loss vs final ROSE-CD: PESQ, SI-SDR and WV-MOS (a neural listener-score predictor) | 3.99, 0.40 dB, 2.31 vs 3.49, 17.8 dB, 4.41 | Xu et al., ROSE-CD paper, WASPAA 2025 |
| PESQ gain from perceptual contrast stretching on SEMamba | +0.14 (3.55 to 3.69) | Chao et al., SEMamba paper, IEEE SLT 2024 |
Context note: the PESQetarian listening test used 10 audio experts rating 10 random test samples on a 0 to 100 scale, and the paper reports the per-system results only as a figure. The PESQetarian is a deliberate stress test, not a product.
6. From Benchmark to Real Recordings: The Generalization Gap
A model that wins on 2 test speakers has not proven it can handle yours. A 2023 study trained 40 models per architecture across five speech, five noise and five room-acoustics databases, then measured each model against a reference model trained on the test condition itself. When the test speech came from an unseen corpus, Conv-TasNet showed an ESTOI generalization gap of -68% against that reference, and speech mismatch hurt more than noise or room mismatch for every model.
The encouraging finding is that noise generalization is solvable with data: training on four noise databases kept the SNR-improvement gap at -5% or better. An out-of-domain test in a 2025 paper shows newer generative models losing ground too. Sources: Gonzalez et al., generalization gap paper, ROSE-CD paper, ICASSP 2021 DNS Challenge paper.
| Metric | Value | Source |
|---|---|---|
| ESTOI generalization gap under speech-corpus mismatch, Conv-TasNet trained on 1 speech database | -68% | Gonzalez et al., IEEE/ACM TASLP 2023 |
| PESQ generalization gap under speech-corpus mismatch: MANNER trained on 1 database vs Conv-TasNet trained on 4 | -50% vs -35% | Gonzalez et al., IEEE/ACM TASLP 2023 |
| SNR-improvement gap under noise mismatch with 4 training noise databases (Conv-TasNet, DCCRN, MANNER) | -5% or better | Gonzalez et al., IEEE/ACM TASLP 2023 |
| Room-acoustics mismatch with 1 training database: PESQ gap for a feedforward network vs ESTOI gap for Conv-TasNet | -10% vs -17% | Gonzalez et al., IEEE/ACM TASLP 2023 |
| ROSE-CD PESQ on VoiceBank-DEMAND vs on out-of-domain TIMIT+NOISE92 | 3.49 vs 2.62 | Xu et al., ROSE-CD paper, WASPAA 2025 |
| SGMSE+ (diffusion model) PESQ on VoiceBank-DEMAND vs on out-of-domain TIMIT+NOISE92 | 2.93 vs 2.08 | Xu et al., ROSE-CD paper, WASPAA 2025 |
| ICASSP 2021 DNS Challenge Track 1 blind test: real recordings vs synthetic clips | 650 real vs 50 synthetic (700 clips) | Microsoft, ICASSP 2021 DNS Challenge paper |
Context note: the generalization gap is measured against a reference model trained on the test condition, so it is a relative figure, not an absolute drop in the ESTOI or PESQ score. The raw ROSE-CD and SGMSE+ drops, by contrast, mix unseen data with task difficulty, the two effects that reference-model method is designed to separate. The DNS organizers built their blind test mostly from real recordings and judged it with listeners, noting that objective measures such as PESQ, SDR and POLQA have been shown not to correlate well with subjective quality in the presence of background noise.
7. How Challenges and Standards Are Changing Evaluation
The current answer to metric gaming is a wider scorecard. In URGENT 2024, the team ranked first on DNSMOS (3.26) finished 18th on listener MOS (3.05), with a word accuracy of only 40.73%, while the top team combined a MOS of 3.52 with 73.57% word accuracy. Word accuracy here means how many words a speech recognizer still gets right after enhancement, a direct check that the model did not invent or erase speech; our speech recognition hallucination statistics cover that failure from the recognizer side.
Hybrid and generative enhancers make the problem sharper. A 2025 speech restoration challenge measured a rank correlation of -0.8 between widely used reference-free metrics and listener MOS for hybrid systems, meaning the metric ranking ran largely opposite to the human one. Standards bodies have moved too: PESQ is no longer an ITU-T recommendation. Sources: CCF AATC 2025 retrospective, URGENT 2025 overview, ICASSP 2026 URGENT overview, ITU-T P.862 page, ITU-T P.863 page.
| Metric | Value | Source |
|---|---|---|
| URGENT 2024 blind test and listening subset | 500 simulated plus 500 real recordings; 300-sample MOS subset, 8 listeners per sample; 21 teams | Zhang, Saijo et al., URGENT 2024 lessons paper, Interspeech 2025 |
| URGENT 2024 training data with mismatched declared vs effective bandwidth | about 25% of LibriTTS; about 100% of DNS5 LibriVox and CommonVoice 11.0 | Zhang, Saijo et al., URGENT 2024 lessons paper, Interspeech 2025 |
| CCF AATC 2025: rank correlation between reference-free metrics and listener MOS for hybrid systems | -0.8 (25 systems analysed in the challenge) | Zhang et al., CCF AATC 2025 retrospective, arXiv 2025 |
| URGENT 2025 evaluation suite | 14 metrics in 5 categories; 7 distortion types; 5 training languages | Saijo et al., URGENT 2025 overview, Interspeech 2025 |
| ICASSP 2026 URGENT Track 1 training data | 700-hour curated subset of the 2,500-hour URGENT 2025 set | ICASSP 2026 URGENT Challenge overview |
| PESQ status at ITU-T | P.862 and parts P.862.1 to P.862.3 deleted on January 5, 2024 | ITU-T P.862 recommendation page |
| POLQA status at ITU-T | P.863 (March 2018) in force, with Amendment 1 (April 2020) | ITU-T P.863 recommendation page |
Context note: URGENT 2024 also flagged about 2,000 speech labels in its “high-quality” training corpora with negative WADA-SNR values, a blind SNR estimate, so label noise sits inside the clean references that intrusive metrics such as PESQ compare against.
Summary: Speech Enhancement PESQ and STOI Benchmarks by the Numbers
| Metric | Value | Source |
|---|---|---|
| VoiceBank-DEMAND PESQ: SEGAN (2017) vs CMGAN (2022) | 2.16 vs 3.41 | Cao et al., CMGAN paper, Interspeech 2022 |
| VoiceBank-DEMAND PESQ: SEMamba without vs with perceptual contrast stretching | 3.55 vs 3.69 | Chao et al., IEEE SLT 2024 |
| VoiceBank-DEMAND PESQ and size of PrimeK-Net | 3.61; 1.41M parameters | PrimeK-Net paper, ICASSP 2025 |
| STOI: noisy input vs CMGAN output | 0.91 vs 0.96 | Cao et al., CMGAN paper, Interspeech 2022 |
| PESQetarian: PESQ, SI-SDR and POLQA (noisy input in brackets) | 3.82, -19.8 dB, 1.46 (1.97, 8.4 dB, 3.11) | de Oliveira et al., Interspeech 2024 |
| PESQ-loss variant of ROSE-CD: PESQ vs SI-SDR | 3.99 vs 0.40 dB | Xu et al., WASPAA 2025 |
| STOI correlation with measured intelligibility: mean across 13 datasets (eSTOI mean in brackets) vs modulated-noise dataset | 0.62 (0.88) vs -0.77 | Edraki et al., IEEE/ACM TASLP 2020 |
| DNSMOS P.835 speech score PCC: per model vs per clip | 0.94 vs 0.71 | Reddy et al., Microsoft, DNSMOS P.835 paper 2021 |
| ESTOI generalization gap under unseen speech corpus (Conv-TasNet) | -68% | Gonzalez et al., IEEE/ACM TASLP 2023 |
| PESQ generalization gap under unseen speech corpus (MANNER, 1 training database) | -50% | Gonzalez et al., IEEE/ACM TASLP 2023 |
| ROSE-CD PESQ in-domain vs out-of-domain | 3.49 vs 2.62 | Xu et al., WASPAA 2025 |
| URGENT 2024: DNSMOS rank vs listener MOS rank of the same team | 1st vs 18th | Zhang, Saijo et al., Interspeech 2025 |
| PESQ after adding one fixed click to every noisy test file, with no denoising | 3.46 | de Oliveira et al., Interspeech 2024 |
| Reference-free metrics vs listener MOS for hybrid systems (rank correlation) | -0.8 | Zhang et al., CCF AATC 2025 retrospective |
| PESQ deleted by ITU-T | January 5, 2024 | ITU-T P.862 recommendation page |
Methodology and Sources
- ITU-T, P.862 (PESQ) recommendation page, status updated January 9, 2024, and P.863 (POLQA) recommendation page, accessed October 2026.
- Taal et al., An Algorithm for Intelligibility Prediction of Time-Frequency Weighted Noisy Speech, IEEE TASLP 2011 (abstract read via the OpenAlex record; citation count from OpenAlex).
- Edraki, Chan, Jensen and Fogerty, Speech Intelligibility Prediction using Spectro-Temporal Modulation Analysis, IEEE/ACM TASLP 2020.
- Cao, Abdulatif and Yang, CMGAN: Conformer-based Metric GAN for Speech Enhancement, Interspeech 2022 (source of the SEGAN, DEMUCS and dataset figures).
- MetricGAN+, Interspeech 2021; MP-SENet, Interspeech 2023; Chao et al., SEMamba, IEEE SLT 2024; PrimeK-Net, ICASSP 2025.
- de Oliveira, Welker, Richter and Gerkmann, The PESQetarian: On the Relevance of Goodhart’s Law for Speech Enhancement, Interspeech 2024.
- Xu, Yan and Kleijn, Robust One-step Speech Enhancement via Consistency Distillation (ROSE-CD), WASPAA 2025.
- Gonzalez, Alstrøm and May, Assessing the Generalization Gap of Learning-Based Speech Enhancement Systems, IEEE/ACM TASLP 2023.
- Microsoft research papers: Reddy, Gopal and Cutler, DNSMOS (2020) and DNSMOS P.835 (2021); ICASSP 2021 DNS Challenge and ICASSP 2022 DNS Challenge papers.
- Mittag, Naderi, Chehadi and Möller, NISQA, Interspeech 2021.
- Zhang, Saijo et al., Lessons Learned from the URGENT 2024 Speech Enhancement Challenge, Interspeech 2025; Saijo et al., Interspeech 2025 URGENT Challenge; ICASSP 2026 URGENT Challenge overview (2026).
- Zhang, Zhu, Xu, Bu, Ling and Wu, The CCF AATC 2025 Speech Restoration Challenge: A Retrospective, arXiv 2025.
- Data watch: VoiceBank-DEMAND scores here are each paper’s own reports, and figures for one system can differ between papers (MetricGAN+ is 3.15 in its own paper and 3.13 in the ROSE-CD table). The ROSE-CD paper is internally inconsistent: its results table gives the final model a PESQ of 3.49, while its conclusion states 3.99, which matches the PESQ-loss-only variant; we report 3.49 as the final model. Citation counts for the STOI paper differ by source (2,265 in OpenAlex, 2,499 in Semantic Scholar, both read October 31, 2026). Intelligibility-metric correlations come from a 2020 study and are the most recent multi-dataset comparison we found. ITU-T published a new edition of P.835, the listening-test method behind DNSMOS P.835, in July 2026, and the ICASSP 2026 URGENT paper does not yet print its quality-predictor correlations in the text.
Last updated: October 31, 2026. We update this roundup quarterly, and the next refresh is expected when the ICASSP 2026 URGENT Challenge proceedings publish their Track 2 quality-predictor correlations.