The best equal error rate (EER) in ASVspoof went from 0.22% in 2019 to 15.64% on the 2021 deepfake task, and then to 8.61% in the closed condition of ASVspoof 5 (Todisco et al., Interspeech 2019; Yamagishi et al., ASVspoof 2021 Workshop; Wang et al., ASVspoof Workshop 2024). EER is the point where the share of fakes accepted equals the share of real speech rejected, so lower is better, and that rising line records a harder exam rather than weaker detectors. Spoofing here means feeding a voice system fake or replayed speech so that it passes as a real person, and the threat is measurable: replay attacks alone pushed a baseline speaker verification system from 1.76% to 31.46% EER (Kinnunen et al., Interspeech 2017). We aggregated data from the ISCA Interspeech and Odyssey archives, IEEE/ACM TASLP, IEEE T-BIOM, IEEE JSTSP, Computer Speech and Language, and the organisers’ own evaluation reports. For the commercial side of the same problem, see our voice biometrics statistics.
TL;DR
- 63 research teams took part in ASVspoof 2019, the participation peak of the series (Todisco et al., Interspeech 2019).
- ASVspoof 5 used 32 attack algorithms and about 2,000 speakers, against about 100 speakers in earlier editions (Wang et al., Computer Speech and Language 2025).
- The best ASVspoof 2015 system averaged 1.211% EER, but 8.49% on the single hardest attack, S10 (Wu et al., IEEE JSTSP 2017).
- Replay attacks pushed a baseline speaker verification system from 1.76% to 31.46% EER in 2017 (Kinnunen et al., Interspeech 2017).
- ASVspoof 2021 physical access stayed hard: the best system reached 24.25% EER and 0.6824 min t-DCF (Yamagishi et al., ASVspoof 2021 Workshop).
- The ASVspoof 5 closed condition was topped at 8.61% EER, against 29.12% for the best baseline (Wang et al., ASVspoof Workshop 2024).
- On ASVspoof 2021 deepfakes, 23 of 33 systems scored under 10% EER on the progress data, and all exceeded 15% on the evaluation data (Liu et al., IEEE/ACM TASLP 2023).
- The winning 2019 logical access ensemble cut t-DCF by 56% against the best single system, while physical access gained only about 2% (Nautsch et al., IEEE T-BIOM 2021).
- The first spoofing-aware speaker verification challenge took a conventional system from 23.83% to 0.13% EER (Jung et al., Interspeech 2022).
- Post-challenge work later reached 0.82% EER on ASVspoof 2021 logical access and 2.85% on deepfakes (Tak et al., Odyssey 2022).
1. Challenge Editions and Participation
ASVspoof has run five times in nine years, and each edition has picked a different weak point. The 2015 edition tested text-to-speech and voice conversion, 2017 tested replay, and 2019 split the field into logical access (fake speech injected straight into the system’s audio path) and physical access (fake speech played through a loudspeaker into a microphone). In 2021 a separate deepfake task followed, and ASVspoof 5 split into stand-alone detection and spoofing-robust speaker verification.
Participation peaked at 63 research teams in 2019 (Todisco et al., Interspeech 2019) and has held above 50 in the two editions since. That stability matters: spoofed evaluation trials rose from 184,000 in 2015 to 542,086 in 2024 (see section 2), yet the field kept showing up. Disclosure is a softer spot, since 34 of the 54 teams with valid 2021 scores filed a system description, leaving 20 teams (54 minus 34) without one on file (Liu et al., IEEE/ACM TASLP 2023).
| Metric | Value | Source |
|---|---|---|
| ASVspoof 2015: teams that requested the database | 28 teams from 16 countries | Wu et al., Interspeech 2015 |
| ASVspoof 2015: teams returning primary submissions | 16 | Wu et al., Interspeech 2015 |
| ASVspoof 2017: teams submitting a replay detector | 49 | Kinnunen et al., Interspeech 2017 |
| ASVspoof 2019: research teams overall | 63 (48 on logical access, 50 on physical access) | Todisco et al., Interspeech 2019 |
| ASVspoof 2021: teams with valid evaluation scores | 54 of 72 registered participants | Liu et al., IEEE/ACM TASLP 2023 |
| ASVspoof 2021: teams that filed a system description | 34 | Liu et al., IEEE/ACM TASLP 2023 |
| ASVspoof 5 (2024): participating teams in the results paper | 53 | Wang et al., IEEE TASLP 2026 |
| SASV 2022, the side challenge joining spoof detection and speaker checks: registrations vs submissions | 53 vs 23 | Jung et al., Interspeech 2022 |
Team counts depend on the paper. The 2019 results paper says 62 teams and the 2024 workshop paper says 54 participants, a gap covered in Data watch.
2. From 100 Speakers to 2,000: The Data Behind Each Edition
The most important change in ASVspoof is not a model. It is the data. Earlier databases held about 100 speakers each, and studio-quality recordings supplied the real speech. ASVspoof 5 moved to crowdsourced speech from about 2,000 speakers, recorded in varied rooms on varied devices (Wang et al., Computer Speech and Language 2025).
That one decision helps explain the higher error rates in section 3. A detector can no longer succeed by learning what one recording setup sounds like. The attack side changed as well: ASVspoof 5 added adversarial attacks for the first time, meaning fakes altered specifically to fool a detector, and tested 12 codec conditions, from no compression to neural codecs (Wang et al., ASVspoof Workshop 2024).
| Metric | Value | Source |
|---|---|---|
| ASVspoof 2015 evaluation set: genuine vs spoofed trials | 9,404 vs 184,000 | Wu et al., Interspeech 2015 |
| ASVspoof 2017 evaluation set: unique replay configurations | 110 | Kinnunen et al., Interspeech 2017 |
| ASVspoof 2019 logical access subset: total utterances | 121,461 | Wang et al., Computer Speech and Language 2020 |
| ASVspoof 2019 physical access: simulated evaluation conditions | 243 (27 rooms and distances times 9 replay types) | Wang et al., Computer Speech and Language 2020 |
| ASVspoof 2021 physical access evaluation: real vs spoofed trials | 94,068 vs 627,264 | Liu et al., IEEE/ACM TASLP 2023 |
| ASVspoof 2021 deepfake evaluation: real vs spoofed trials | 14,869 vs 519,059, from more than 100 attack algorithms | Liu et al., IEEE/ACM TASLP 2023 |
| ASVspoof 5 Track 1 evaluation: real vs spoofed trials | 138,688 vs 542,086 | Wang et al., IEEE TASLP 2026 |
| ASVspoof 5 evaluation attacks: total and adversarial | 16 attacks, 7 of them adversarial, from a pool of 32 | Wang et al., Computer Speech and Language 2025 |
3. Headline EER Results by Edition
Reading these rows as one trend line is the classic mistake. The 0.22% winner of 2019 and the 8.61% winner of 2024 sat different exams, and the organisers made each one harder on purpose. The useful comparison sits inside each row, between the best team and the baseline it had to beat.
On that measure the field keeps gaining. In ASVspoof 5, 27 closed-condition teams beat the best baseline (Wang et al., IEEE TASLP 2026), and the top system’s EER was less than a third of that baseline’s. The exception is physical access in 2021, which the journal analysis calls the most challenging of the three tasks, with the best system stuck at 24.25% (Yamagishi et al., ASVspoof 2021 Workshop).
| Metric | Value | Source |
|---|---|---|
| ASVspoof 2015 best system: known attacks, unknown attacks, average | 0.408%, 2.013%, 1.211% EER | Wu et al., Interspeech 2015 |
| ASVspoof 2017 best replay detector vs baseline | 6.73% vs 24.77% EER | Kinnunen et al., Interspeech 2017 |
| ASVspoof 2019 best vs best baseline, logical access and physical access | 0.22% vs 8.09% EER; 0.39% vs 11.04% EER | Todisco et al., Interspeech 2019 |
| ASVspoof 2021 logical access best vs best baseline | 1.32% vs 9.26% EER | Yamagishi et al., ASVspoof 2021 Workshop |
| ASVspoof 2021 physical access best | 24.25% EER | Yamagishi et al., ASVspoof 2021 Workshop |
| ASVspoof 2021 deepfake task best vs best baseline | 15.64% vs 22.38% EER | Yamagishi et al., ASVspoof 2021 Workshop |
| ASVspoof 5 Track 1 best closed condition, best open condition, best baseline | 8.61%, 2.59%, 29.12% EER | Wang et al., ASVspoof Workshop 2024 |
The closed condition bars outside data, while the open condition allows external data and pre-trained models, which is why the same year shows two very different winners.
4. What Spoofing Does to Speaker Verification, and the Hardest Attacks
Without a detector, speaker verification barely resists. Under the 2015 attack S10, a system with 0.42% EER collapsed to 45.79% (Wu et al., Interspeech 2015). Later editions repeat the pattern at every scale, including ASVspoof 5, where 14 of 16 evaluation attacks pushed speaker verification EER past 20% (Wang et al., Computer Speech and Language 2025).
The more surprising finding is who the hardest attacker is. In the closed condition of ASVspoof 5 it was A19, built on MaryTTS, an older concatenative synthesizer that stitches together recorded speech, and the database paper concludes that both legacy and contemporary generative technology still pose a grave threat. Machines and people also fail differently: in 2019 attack A17 fooled the baseline detector yet was easily spotted by human listeners, while A10 was described as perceptually indistinguishable from real speech (Wang et al., Computer Speech and Language 2020). Our synthetic speech MOS statistics cover the listener side of that gap.
| Metric | Value | Source |
|---|---|---|
| Speaker verification EER, ordinary impostors vs attack S10 (2015) | 0.42% vs 45.79% | Wu et al., Interspeech 2015 |
| Speaker verification EER, ordinary impostors vs replay (2017) | 1.76% vs 31.46% | Kinnunen et al., Interspeech 2017 |
| Speaker verification EER, ordinary impostors vs worst 2019 logical access attack | 2.48% vs 64.52% (attack A16) | Wang et al., Computer Speech and Language 2020 |
| Conventional ECAPA-TDNN speaker verification EER, no spoofing vs with spoofed trials added | 1.63% vs 23.83% | Jung et al., Interspeech 2022 |
| ASVspoof 5 speaker verification EER, attack A26 vs the same attack with the Malacopula adversarial step added (A27) | 15.77% vs 34.68% | Wang et al., Computer Speech and Language 2025 |
| Best ASVspoof 2015 detector EER on the hardest attack, S10 | 8.49% | Wu et al., IEEE JSTSP 2017 |
| ASVspoof 5 baseline detector EER on A19, RawNet2 vs AASIST | 49.80% vs 63.75% | Wang et al., Computer Speech and Language 2025 |
S10 stayed stubborn after the 2015 challenge too: the journal review of the 2015 edition found results for it remained poor across most follow-up studies, with one exception at 1.065%.
5. Beyond EER: t-DCF, a-DCF and the Calibration Problem
EER describes a detector alone, at one threshold chosen after the fact. Since 2019 most ASVspoof tasks have been ranked on costs instead. The tandem detection cost function (t-DCF) prices the detector and the speaker check working together, on a scale with an ASV floor: the cost that remains even with a perfect detector, because the speaker check still makes mistakes. In 2019 the best logical access system landed at 0.0692 against a floor of 0.0627, nearly as good as a detector can get (Nautsch et al., IEEE T-BIOM 2021).
ASVspoof 5 exposes the next problem, calibration, which asks whether a system’s scores can be used at a fixed real threshold. minDCF is the cost at the best threshold chosen after the fact, and actDCF is the cost at the threshold a deployed system would actually use. The closed-condition winner on minDCF scored 0.9956 on actDCF, while the third-placed team lost almost nothing between the two. We counted 11 of 31 closed-condition submissions and 14 of 34 open-condition submissions listed with an actDCF of exactly 1.0000 (Wang et al., ASVspoof Workshop 2024). The organisers link actDCF values of 1.0 to probability-like scores, so a low minDCF and a high actDCF can sit side by side in one system, and they raise calibration-sensitive metrics as possible primary metrics in future editions (Wang et al., IEEE TASLP 2026).
| Metric | Value | Source |
|---|---|---|
| ASVspoof 2019: ASV floor vs best min t-DCF, logical access and physical access | 0.0627 vs 0.0692; 0.1354 vs 0.1437 | Nautsch et al., IEEE T-BIOM 2021 |
| ASVspoof 2021 physical access: ASV floor vs best min t-DCF | 0.12 vs 0.6824 | Liu et al., IEEE/ACM TASLP 2023; Yamagishi et al., ASVspoof 2021 Workshop |
| ASVspoof 5 Track 1 closed condition: minDCF vs actDCF, first place and third place | 0.2436 vs 0.9956; 0.2975 vs 0.2976 | Wang et al., ASVspoof Workshop 2024 |
| ASVspoof 5 Track 2: best closed, best open and best closed baseline, min a-DCF | 0.2814, 0.0756, 0.5741 | Wang et al., ASVspoof Workshop 2024 |
The a-DCF (architecture-agnostic detection cost function) is the primary Track 2 metric, and Track 2 combines the detector with the speaker check. t-DCF values are not comparable across papers: see Data watch.
6. What Wins: Neural Models, Ensembles and Pre-Trained Front-Ends
The series tells a clean story about methods. By 2019 neural networks had taken over the top of the table, and ensembles, meaning several sub-systems combined, had become the default way to win. The winning 2019 logical access ensemble was 56% better than the best single system on t-DCF, while physical access gained only about 2% (Nautsch et al., IEEE T-BIOM 2021). Ensembles had little left to gain on physical access, where the best single system already sat near the ASV floor described in section 5.
By 2024 the deciding factor was the pre-trained front-end. All top ASVspoof 5 submissions used wav2vec 2.0 or WavLM (Wang et al., IEEE TASLP 2026), self-supervised models trained first on large amounts of ordinary speech. Much of the closed-to-open gain, from 8.61% to 2.59%, came with those models and the extra data the open condition allows. Those same model families are measured on general speech tasks in our SUPERB leaderboard statistics. The technique was proven soon after the 2021 edition, when one wav2vec 2.0 system beat every challenge result on the same data (Tak et al., Odyssey 2022).
| Metric | Value | Source |
|---|---|---|
| ASVspoof 2019: top systems using neural networks, and systems using an ensemble of classifiers | Top 7 and 9 on logical access; top 6 and 10 on physical access | Todisco et al., Interspeech 2019 |
| ASVspoof 2019: min t-DCF, best single system vs best ensemble, logical access and physical access | 0.1562 vs 0.0692; 0.1470 vs 0.1437 | Nautsch et al., IEEE T-BIOM 2021 |
| ASVspoof 5 Track 1: sub-systems fused, closed vs open condition | 3 to 4 vs 2 to 6 | Wang et al., IEEE TASLP 2026 |
| ASVspoof 5 Track 1: pre-trained front-ends among the five listed top open systems | wav2vec2-large in 3, WavLM-Base in 2 | Wang et al., IEEE TASLP 2026 |
| Post-challenge wav2vec 2.0 system on ASVspoof 2021 logical access, EER and min t-DCF | 0.82% and 0.2066 | Tak et al., Odyssey 2022 |
| Post-challenge wav2vec 2.0 system on ASVspoof 2021 deepfake task, EER | 2.85% | Tak et al., Odyssey 2022 |
7. The Generalization Gap After the Leaderboard
The unsolved issue in anti-spoofing is what happens off the benchmark. The clearest sign came inside the challenge itself: on ASVspoof 2021 deepfakes, 23 of 33 systems scored under 10% EER on progress data, and every one of them exceeded 15% on the evaluation set (Liu et al., IEEE/ACM TASLP 2023). The best progress score was under 1%, so the leaderboard feedback teams trained against was misleading.
The same result holds outside the challenge. Four top-5 ASVspoof 5 open systems were re-tested on other datasets, and with one exception their EERs rose above 10% (Wang et al., IEEE TASLP 2026). On audio collected from the internet, models trained on ASVspoof 2019 did far worse, and more ASVspoof data did not help (Müller et al., Interspeech 2022). Speech enhancement has a comparable problem, documented in our PESQ and STOI benchmark statistics. A 2025 preprint goes further and argues that detection alone should not carry high-stakes decisions, recommending it as one signal in layered defenses (Berisha et al., arXiv 2025).
| Metric | Value | Source |
|---|---|---|
| ASVspoof 5 open-condition systems T36, T27, T23, T43: EER on an ASVspoof 5 Track 1 subset vs on ASVspoof 2019 logical access | 3.37%, 3.30%, 4.23%, 4.33% vs 16.27%, 17.33%, 16.73%, 26.63% | Wang et al., IEEE TASLP 2026 |
| ASVspoof 2019 physical access winner: EER on simulated replay vs hidden real replay | 0.39% vs 19.68% | Nautsch et al., IEEE T-BIOM 2021 |
| Best model on ASVspoof 2019 vs best model on in-the-wild audio, EER | About 1.2% vs about 33.9% | Müller et al., Interspeech 2022 |
| RawNet2 retrained on all of ASVspoof 2019: EER on in-the-wild audio | 33.1% ± 0.2 | Müller et al., Interspeech 2022 |
| EER degradation of 12 reimplemented models on in-the-wild audio | About 200% to 1000% | Müller et al., Interspeech 2022 |
| WildSpoof Challenge team T03, its own reported EER on in-domain spoof detection | 0.16% | Farhadipour et al., arXiv 2026 |
The T03 figure is one team’s own report, and its authors say their results on other datasets, including ASVspoof 5, show cross-domain generalization remains difficult. In-domain numbers keep falling while the gap stays open. WildSpoof, whose evaluation plan lists authors who also organised ASVspoof 5, pairs a text-to-speech track with a spoofing-robust speaker verification track (Farhadipour et al., arXiv 2026).
Summary: Voice Anti-Spoofing ASVspoof Equal Error Rate by the Numbers
| Metric | Value | Source |
|---|---|---|
| ASVspoof 2015 teams with primary submissions | 16 | Wu et al., Interspeech 2015 |
| ASVspoof 2017 teams | 49 | Kinnunen et al., Interspeech 2017 |
| ASVspoof 5 speakers vs earlier editions | About 2,000 vs about 100 | Wang et al., Computer Speech and Language 2025 |
| Speaker verification EER, ordinary impostors vs attack S10 (2015) | 0.42% vs 45.79% | Wu et al., Interspeech 2015 |
| Speaker verification EER, ordinary impostors vs worst 2019 logical access attack | 2.48% vs 64.52% | Wang et al., Computer Speech and Language 2020 |
| ASVspoof 2015 best average EER | 1.211% | Wu et al., IEEE JSTSP 2017 |
| Best ASVspoof 2015 detector EER on the hardest attack, S10 | 8.49% | Wu et al., IEEE JSTSP 2017 |
| ASVspoof 2017 best EER | 6.73% | Kinnunen et al., Interspeech 2017 |
| ASVspoof 2019 logical access best EER | 0.22% | Todisco et al., Interspeech 2019 |
| ASVspoof 2021 deepfake task best EER | 15.64% | Yamagishi et al., ASVspoof 2021 Workshop |
| ASVspoof 5 Track 1 closed condition best EER | 8.61% | Wang et al., ASVspoof Workshop 2024 |
| SASV 2022 best EER with spoofed trials included | 0.13% | Jung et al., Interspeech 2022 |
| Best post-challenge EER on ASVspoof 2021 logical access | 0.82% | Tak et al., Odyssey 2022 |
| Best post-challenge EER on ASVspoof 2021 deepfake task | 2.85% | Tak et al., Odyssey 2022 |
| ASVspoof 2019 physical access winner on hidden real replay | 19.68% EER | Nautsch et al., IEEE T-BIOM 2021 |
| Best model on in-the-wild audio | About 33.9% EER | Müller et al., Interspeech 2022 |
Methodology and Sources
We read the challenge papers directly rather than third-party summaries. EER figures are percentages on evaluation sets unless a row says otherwise. Most data here is older than three years by nature, because the challenge runs roughly every two to three years. Most recent available data: the 2026 journal report on the 2024 edition.
- ASVspoof official site: edition list, schedules and organisers.
- Wu et al., ASVspoof 2015, Interspeech 2015: first edition, teams, trials and results.
- Wu et al., IEEE Journal of Selected Topics in Signal Processing 2017: S10 detail and post-evaluation review.
- Kinnunen et al., ASVspoof 2017, Interspeech 2017: replay detection results.
- Todisco et al., ASVspoof 2019, Interspeech 2019: overview and leaderboard.
- Wang et al., ASVspoof 2019 database, Computer Speech and Language 2020: data sizes, attacks and human assessment.
- Nautsch et al., IEEE Transactions on Biometrics, Behavior, and Identity Science 2021: revised t-DCF, ensembles, hidden real replay.
- Yamagishi et al., ASVspoof 2021 Workshop: headline 2021 results.
- Liu et al., IEEE/ACM Transactions on Audio, Speech, and Language Processing 2023: full 2021 analysis.
- Wang et al., ASVspoof Workshop 2024: ASVspoof 5 result tables.
- Wang et al., ASVspoof 5 database, Computer Speech and Language 2025: data, attacks and baseline effects.
- Wang et al., IEEE Transactions on Audio, Speech and Language Processing 2026: post-challenge ASVspoof 5 analysis.
- Jung et al., SASV 2022, Interspeech 2022: spoofing-aware speaker verification.
- Tak et al., Odyssey 2022: post-challenge wav2vec 2.0 results.
- Müller et al., Interspeech 2022: in-the-wild generalization test.
- Berisha et al., arXiv 2025: preprint, not peer reviewed in the record we read.
- Farhadipour et al., arXiv 2026 and the WildSpoof evaluation plan 2025: team system paper and challenge rules.
- Data watch: (1) Cross-edition EER is not a trend, because data, attacks and codecs changed each time. (2) t-DCF scales differ: the 2019 overview printed 0.0069 for the logical access winner and 0.0096 for the physical access winner, while the later T-BIOM paper revised the formula and printed 0.0692 and 0.1437, keeping rankings essentially intact; we use the revised values. (3) Team counts conflict: 63 vs 62 for 2019 and 53 vs 54 for ASVspoof 5. (4) The 2021 workshop paper’s text gives 42 logical access and 30 deepfake submissions, but its tables list 41 and 33; we quote only the 33 used in the journal paper. The deepfake winner is 15.64% in the workshop paper and 15.63% in the journal paper. (5) The 2017 paper gives the average primary EER as both 26.01% and 25.91%, so we do not use it. (6) ASVspoof 5 best Track 2 open a-DCF is 0.0756 in the 2024 table and printed as 0.07 in the 2026 paper. (7) The actDCF counts in section 5 are our own tally of the 2024 tables, excluding the two baselines in the closed condition. (8) Table values for ASVspoof 5 were read from the 2024 paper, and many teams appear there without a public ID. (9) The Tak et al. figures are the authors’ own post-challenge results, which we did not reproduce. (10) We found no official organiser summary of WildSpoof final results and no ASVspoof 6 announcement on the official site as of October 31, 2026.
Last updated: October 31, 2026. We update this roundup quarterly, and the next refresh is expected when the ASVspoof organisers publish the evaluation plan for the next edition.