Five commercial speech recognition systems averaged a word error rate (WER) of 0.35 for Black speakers against 0.19 for white speakers (Stanford, Koenecke et al., PNAS 2020), and a study published in 2026 still found a mean WER of 20% against 15% across four commercial systems (University of Washington, Scott et al., LREC 2026). Benchmarks hide the gap: Whisper large scores 2.7% on LibriSpeech test-clean and 19.7% on a corpus of international English accents (University of Edinburgh, EdAcc paper 2023). For speakers with Parkinson’s-related dysarthria, a recognizer that makes 3.4% errors on typical speech makes 36.3% (Speech Accessibility Project, JSLHR 2024). We aggregated data from PNAS, the OpenAI Whisper paper, Meta’s Fair-Speech dataset, the Speech Accessibility Project, Mozilla Common Voice, and more than a dozen other peer-reviewed sources, keeping each source’s own notation (some report WER as a fraction such as 0.35, most as a percentage such as 19.7%). The same metric sits underneath live caption accuracy rules, which is why a few points of WER matter far outside the lab.
TL;DR
- Five commercial ASR systems averaged a WER of 0.35 for Black speakers vs 0.19 for white speakers, and 23% vs 1.6% of snippets were unusable at WER above 0.5 (Stanford, Koenecke et al., PNAS 2020).
- Four commercial systems averaged 20% WER for African American speakers vs 15% for Caucasian American speakers, a 33% relative increase (University of Washington, Scott et al., LREC 2026).
- Whisper large scores 2.7% WER on LibriSpeech test-clean and 19.7% on the EdAcc accented-English test set (University of Edinburgh, EdAcc paper 2023).
- Whisper-large-v2 scored 9.52 for Black or African American speakers vs 3.96 for white speakers on voice commands (Meta, Fair-Speech Dataset 2024).
- A Dutch recognizer hit 55.9% WER on non-native read speech vs 26.1% for native speakers (TU Delft, Feng et al., Quantifying Bias in ASR 2021).
- Dysarthric speech took a 3.4% baseline WER to 36.3%, and fine-tuning brought it to 23.7% (Speech Accessibility Project, JSLHR 2024).
- The best system in the Interspeech 2025 Speech Accessibility Project Challenge reached 8.11% WER against a 17.82% whisper-large-v2 baseline (Zheng et al., Interspeech 2025).
- Six ASR systems averaged a WER of .281 on stuttered speech vs .105 once disfluencies were removed (Mujtaba et al., NAACL 2024).
- Common Voice Scripted Speech v27.0 holds 29,295 validated hours in 295 languages, against a world total of over 7,000 languages (Mozilla Common Voice 2026; Pratap et al., MMS paper 2023).
- AWS General Transcribe had a median WER of 50% for Black home healthcare patients vs 33% for white patients, P = 0.016 (Zolnoori et al., JAMIA Open 2024).
- Court reporters certified at 95% accuracy averaged 82.9% word accuracy on African American English (Jones et al., Language 2019).
- 93% of African American participants in a two-week diary study modified their dialect to be understood by voice technology (Mengesha et al., Frontiers in Artificial Intelligence 2021).
1. Race and Dialect Gaps in Commercial and Open ASR
The canonical number is six years old and has not been retired by better evidence. In the PNAS audit of Amazon, Apple, Google, IBM and Microsoft, the average WER was 0.35 for Black speakers and 0.19 for white speakers, a gap of 0.16 (0.35 minus 0.19) measured on 19.8 hours of audio from 73 Black and 42 white speakers matched on age and gender. The authors located the problem in the acoustic models, because the gap persisted on identical phrases: Microsoft, the best system, scored 0.13 against 0.07 on matched utterances. Most recent available data for a five-vendor audit: Koenecke et al., PNAS 2020.
The newer evidence, drawn from different corpora, shows lower error rates overall and a gap that has not closed. A University of Washington team reports a 33% relative increase for African American speakers across four commercial systems, and Meta’s own voice-command dataset shows Whisper-large-v2 at 9.52 for Black or African American speakers against 3.96 for white speakers. The Washington team ties its errors to vowel-quality variation between dialects, which echoes the acoustic-model diagnosis from 2020.
| Metric | Value | Source |
|---|---|---|
| Average WER, Black vs white speakers, five commercial systems | 0.35 vs 0.19 | Stanford (Koenecke et al.), PNAS 2020 |
| Best and worst vendor, Black vs white speakers | Microsoft 0.27 vs 0.15; Apple 0.45 vs 0.23 | Stanford (Koenecke et al.), PNAS 2020 |
| Snippets with WER above 0.5 (unusable transcripts) | 23% of Black speakers’ snippets vs 1.6% of white speakers’ | Stanford (Koenecke et al.), PNAS 2020 |
| Mean WER, African American vs Caucasian American speakers, four commercial systems | 20% vs 15% (33% relative increase) | University of Washington (Scott et al.), LREC 2026 |
| Mean WER, ChicanX and Yakama speakers, same four systems | 23% and 21% | University of Washington (Scott et al.), LREC 2026 |
| WER, Black or African American vs white speakers, Whisper-large-v2 and Whisper-small | 9.52 vs 3.96; 16.99 vs 4.51 | Meta (Veliche et al.), Fair-Speech Dataset 2024 |
| Relative WER gap across ethnicity groups, Whisper-large-v2 vs Meta semi-supervised video model | 61.13% vs 31.04% | Meta (Veliche et al.), Fair-Speech Dataset 2024 |
Context: the Washington evaluation used 16 speakers (four per group) and reflects system performance in 2022, even though the paper appeared in the LREC 2026 proceedings. Fair-Speech covers 593 US speakers and about 26.5K utterances, and its authors state that for Whisper models the relative gap is larger than 40% across all dimensions they measured.
2. Benchmark WER vs Real-World Speech
The headline benchmark is clean read speech, not conversation. The Whisper paper makes the point with its own table: the same model that scores 2.7 on LibriSpeech Clean scores 25.5 on CHiME6 and 16.2 on CORAAL, the corpus of African American Language interviews. Whisper large goes from 2.7% WER on LibriSpeech test-clean to 19.7% on EdAcc, a set of video-call conversations between friends with more than 40 self-reported English accents.
That is still the robust model. A wav2vec 2.0 system with a 2.9% LibriSpeech score reached 36.1% on the same accented conversations. Speaking style alone moves the needle: a Dutch recognizer averaged 36.2% WER on read speech and 47.5% on human-machine interaction speech from the same speaker pool (TU Delft, Feng et al., Quantifying Bias in ASR 2021).
| Metric | Value | Source |
|---|---|---|
| Whisper Large V2 WER, LibriSpeech Clean | 2.7 | OpenAI (Radford et al.), Whisper paper 2022 |
| Whisper Large V2 WER, CHiME6 and CORAAL | 25.5 and 16.2 | OpenAI (Radford et al.), Whisper paper 2022 |
| Average WER across the paper’s Table 2 datasets, wav2vec 2.0 Large vs Whisper Large V2 | 29.3 vs 12.8 (55.2% average relative error reduction) | OpenAI (Radford et al.), Whisper paper 2022 |
| Whisper large WER, EdAcc test vs LibriSpeech test-clean | 19.7% vs 2.7% | University of Edinburgh (Sanabria et al.), EdAcc paper 2023 |
| wav2vec 2.0 and commercial system WER, EdAcc test vs LibriSpeech test-clean | 36.1 vs 2.9; 18.7 vs 3.8 | University of Edinburgh (Sanabria et al.), EdAcc paper 2023 |
| Best average WER, English short-form track; Whisper Large v3 on the same track | 5.42; 7.44 | Open ASR Leaderboard paper (Srivastav et al.), v4 March 2026 |
| Systems benchmarked | 86 systems from 26 organizations across 12 datasets; 74 open-source | Open ASR Leaderboard paper (Srivastav et al.), v4 March 2026 |
Context: the Open ASR Leaderboard reports averages over curated test sets, and the tables we read carry no speaker-demographic breakdown, so a 5.42 average says nothing about who sits in the tail. The CORAAL figure is the closest the Whisper paper comes to a dialect-specific number: Whisper cut the wav2vec 2.0 error on that corpus from 35.6 to 16.2, and it remained six times the LibriSpeech Clean score (16.2 divided by 2.7).
3. Non-Native and Regional Accents
Accent gaps are measured mostly on English, and mostly with small or self-selected corpora. The broadest audit we found, on 2,713 Speech Accent Archive speakers born in 171 countries, does not use WER at all: it reports word information lost (WIL), and finds it 0.14 lower for speakers whose first language is English (P<0.001). On Pan-African accented English, Whisper-large posted a WER of 0.240 on general speech and 0.375 on clinical speech, and all three commercial cloud systems tested were worse.
Regional accents of a first language follow the same pattern. In YouTube’s automatic captions, 80 speakers across five dialect regions (California, Georgia, New England, New Zealand and Scotland) showed reliably worse recognition for Scottish speakers (University of Washington, Tatman, ACL Ethics in NLP Workshop 2017). In Dutch, Flemish teenagers reading aloud were recognized at 40.5% WER by a model trained on Netherlands Dutch. These errors compound downstream, since a misrecognized source sentence is the input to every speech-to-speech translation pipeline.
| Metric | Value | Source |
|---|---|---|
| EdAcc corpus size | Almost 40 hours, more than 40 self-reported accents, 51 first languages | University of Edinburgh (Sanabria et al.), EdAcc paper 2023 |
| WIL gap, English vs non-English first-language speakers, all services | 0.14 lower for English first-language speakers (P<0.001); 2,713 speakers, 212 first languages | DiChristofano et al., Global Performance Disparities Between English-Language Accents (arXiv) 2023 |
| WIL gap by service | Google 0.17; Amazon 0.14; Microsoft 0.11 | DiChristofano et al., Global Performance Disparities Between English-Language Accents (arXiv) 2023 |
| Whisper-large WER on Pan-African English, general vs clinical domain | 0.240 vs 0.375 | Olatunji et al., AfriSpeech-200, TACL 2023 |
| Commercial cloud WER on Pan-African English, both domains | Azure 0.391; AWS 0.442; GCP 0.578 | Olatunji et al., AfriSpeech-200, TACL 2023 |
| Whisper-medium WER on Pan-African English, pre-trained vs fine-tuned on AfriSpeech | 0.332 vs 0.216 | Olatunji et al., AfriSpeech-200, TACL 2023 |
| Dutch ASR WER on read speech, non-native vs native speakers | 55.9% vs 26.1% | TU Delft (Feng et al.), Quantifying Bias in ASR 2021 |
| Dutch ASR WER on read speech, Flemish teenagers vs teenagers in four Netherlands regions | 40.5% vs 19.3%, 22.9%, 16.8% and 17.9% | TU Delft (Feng et al.), Quantifying Bias in ASR 2021 |
Outlier: non-native status does not always predict worse results. On Fair-Speech voice commands, Whisper-large-v2 scored 3.78 for second-language speakers and 6.11 for first-language speakers (Meta, Fair-Speech Dataset 2024), a reminder that group gaps are properties of a dataset and a task, not constants. In the Dutch data, error did not fall as adult learners’ proficiency rose: 57.2, 56.5 and 57.0 at CEF levels A1, A2 and B1. Most recent available data: the accent-archive transcriptions were run in 2021, and AfriSpeech-200 (200.70 hours, 2,463 speakers, 120 accents, 13 countries) was released in 2023.
4. Gender, Age, and Income Gaps
No demographic axis has a fixed direction, and that is the finding. YouTube captions were worse for women in 2017. Commercial systems were worse for men in 2020. Whisper-large-v2 scored 7.91 for male speakers against 3.86 for female speakers on Fair-Speech, roughly double (7.91 divided by 3.86 is 2.05). A system audited on one dataset can look fair on gender and fail on another.
Age behaves the same way. The Fair-Speech group with the highest error was not the oldest but the 31 to 45 band, while in Dutch the worst native group was children. Child speech is a recurring weak spot: Whisper-Small started at 13.93% WER on the MyST test set of children’s speech before any adaptation (Kid-Whisper, 2024).
| Metric | Value | Source |
|---|---|---|
| Average WER across five commercial systems, Black men vs Black women; white men vs white women | 0.41 vs 0.30; 0.21 vs 0.17 | Stanford (Koenecke et al.), PNAS 2020 |
| Whisper-large-v2 WER, male vs female speakers | 7.91 vs 3.86 | Meta (Veliche et al.), Fair-Speech Dataset 2024 |
| Whisper-large-v2 WER by age band, 18-22, 23-30, 31-45, 46-65 | 4.46, 4.62, 7.48, 3.65 | Meta (Veliche et al.), Fair-Speech Dataset 2024 |
| Whisper-large-v2 WER by household income, low, medium, affluent | 5.4, 6.38, 3.62 | Meta (Veliche et al.), Fair-Speech Dataset 2024 |
| YouTube caption WER, women vs men | Higher for women (t(78) = -3.5, p < 0.01), 80 speakers | University of Washington (Tatman), ACL Ethics in NLP Workshop 2017 |
| Dutch ASR WER on read speech, native children, teenagers, older adults | 35.3%, 18.4%, 24.2% | TU Delft (Feng et al.), Quantifying Bias in ASR 2021 |
| Whisper WER on MyST children’s speech, before vs after fine-tuning | Small 13.93% to 9.11%; Medium 13.23% to 8.61% | Attia et al., Kid-Whisper 2024 |
Context: Fair-Speech income bands are low (US $50k or less), medium ($50k to $100k) and affluent ($100k and above), and the affluent band is the smallest group in the dataset (41 speakers in the paper’s table). Most recent available data on caption gender gaps: Tatman 2017, measured on an isolated word-list task that the author describes as particularly difficult for ASR.
5. Atypical, Dysarthric, and Disfluent Speech
Here the gaps are an order of magnitude, not a few points. In the Speech Accessibility Project’s first data package, a baseline recognizer with a 3.4% WER for typical speakers transcribed speakers with Parkinson’s-related dysarthria at 36.3%. Fine-tuning on 190 training speakers cut that to 23.7%, and two of the 42 test speakers remained at 67.10% and 93.08%.
Data, not architecture, is what moves these numbers. The Interspeech 2025 Speech Accessibility Project Challenge gave teams about 415 hours from 524 participants, and 12 of 22 teams beat the whisper-large-v2 baseline; the top five models averaged 10.06% WER for Parkinson’s disease and 7.36% for ALS, against a baseline of 17.09% and 16.88%. Personalization goes further still: Google’s Project Euphonia reported 4.6% WER for personalized models on short phrases. Pauses create a separate failure mode, since Whisper hallucinated text in 1.7% of segments from speakers with aphasia against 1.2% for a control group (Koenecke et al., Careless Whisper, FAccT 2024).
| Metric | Value | Source |
|---|---|---|
| Baseline WER, typical speakers vs Parkinson’s-related dysarthria; after fine-tuning | 3.4% vs 36.3%; 23.7% | Speech Accessibility Project (Hasegawa-Johnson et al.), JSLHR 2024 |
| Whisper-large-v2 baseline vs top team, challenge Test2 set | 17.82% vs 8.11% WER (top SemScore 88.44%) | Zheng et al., Interspeech 2025 Speech Accessibility Project Challenge |
| Challenge data and participation | About 415 hours, 524 participants, five etiologies; 12 of 22 teams beat the baseline on WER | Zheng et al., Interspeech 2025 Speech Accessibility Project Challenge |
| Personalized vs speaker-independent models, 432 speakers with disordered speech | 4.6% vs 31% WER | Google Research (Green et al.), Interspeech 2021 |
| Median WER on home automation phrases, severely impaired speech, before vs after personalization | Around 89% to 13% | Google Research, Project Euphonia blog 2021 |
| Project Euphonia corpus (as of August 2021) | Over 1 million utterances, over 1,400 hours, 1,330 speakers | Google Research, Project Euphonia blog 2021 |
| Mean WER across six ASR systems, stuttered speech vs disfluencies removed | .281 vs .105 | Mujtaba et al., Lost in Transcription, NAACL 2024 |
| Speech recognition WER for people who stutter, before vs after system changes (91 participants recorded) | 25.4% to 9.9%; utterances cut off 79.1% less often | Lea et al., CHI 2023 |
Context: Parkinson’s disease accounts for 73.4% of the challenge training corpus by duration, so the headline 8.11% mostly describes one etiology; Down syndrome, cerebral palsy and stroke were not reported separately because of small test counts. In the stuttering study, Whisper was the most accurate of the six systems and still went from .063 to .121. Most recent available data for personalized-model accuracy: Green et al., Interspeech 2021.
6. Low-Resource Languages and Dataset Coverage
Error tracks training hours almost mechanically. The Whisper authors report a squared correlation of 0.83 between the log of WER and the log of training hours per language, and estimate that WER halves for every 16x increase in training data. Of Whisper’s 680,000 training hours, 117,000 cover 96 languages other than English, and the paper notes that most languages have less than 1,000 hours.
Coverage has expanded faster than accuracy evidence. The MMS paper put the world at over 7,000 languages with speech technology supporting about 100, and Omnilingual ASR now claims over 1,600. The standard multilingual benchmark, FLEURS, covers 102 languages with approximately 12 hours of supervision each (Conneau et al., FLEURS 2022), so most newly added languages have no shared public yardstick. We track model-side coverage in more depth in our multilingual speech foundation model statistics.
| Metric | Value | Source |
|---|---|---|
| Whisper training data | 680,000 hours total; 117,000 hours covering 96 non-English languages | OpenAI (Radford et al.), Whisper paper 2022 |
| Relationship between training hours per language and WER | Squared correlation 0.83; WER halves for every 16x increase in data | OpenAI (Radford et al.), Whisper paper 2022 |
| World languages vs languages supported by speech technology | Over 7,000 vs about 100 | Pratap et al., MMS paper (Scaling Speech Technology to 1,000+ Languages) 2023 |
| MMS coverage and accuracy | ASR for 1,107 languages; WER more than halved vs Whisper on 54 FLEURS languages | Pratap et al., MMS paper (Scaling Speech Technology to 1,000+ Languages) 2023 |
| Omnilingual ASR coverage | Over 1,600 languages, more than 500 never before served by ASR | Omnilingual ASR team, arXiv 2511.09690, 2025 |
| Google USM pre-training data | 12 million hours of unlabeled audio covering over 300 languages; labeled set 1/7 the size of Whisper’s | Google USM paper (Zhang et al.) 2023 |
| Common Voice Scripted Speech v27.0 | 295 languages; 42,593 total hours; 29,295 validated hours | Mozilla Common Voice, cv-dataset v27.0 2026 |
| Common Voice v27.0 gender split, English and German | English 0.4318 male, 0.1776 female, 0.3894 undeclared; German 0.5786 male, 0.0818 female | Mozilla Common Voice, cv-dataset v27.0 2026 |
Outlier: in the Common Voice v27.0 statistics, Catalan has more recorded audio than English (3,940.31 total hours against 3,795.9), while the smallest locale we read holds 25 clips from 2 users. Volunteer effort, not speaker population, decides which languages get data. The Spontaneous Speech dataset, which is closer to real usage, is far smaller: 541 total hours across 80 languages at v5.0.
7. Consequences in Courts, Clinics, Hiring, and Voice Assistants
The stakes rise where a transcript becomes a record. Humans set a poor baseline: court reporters certified at 95% accuracy averaged 82.9% word accuracy and 59.5% sentence-level accuracy on African American English, and their transcriptions changed the who, what, when or where 31% of the time (Jones et al., Language 2019). Our courtroom speech recognition accuracy statistics cover the automated pilots that inherit this problem.
Clinical evidence is recent and small-sample. In recorded home healthcare visits, two Amazon systems each showed a 17-point median WER gap between Black and white patients (50 minus 33, and 54 minus 37), while Whisper was equally poor for both. Hiring is the one setting with a reassuring result: transcription bias appeared in nearly all comparisons, and the authors found it did not carry into the interview scores. The cost users report is behavioral, since nearly every participant in a two-week diary study changed how they spoke.
| Metric | Value | Source |
|---|---|---|
| Court reporter accuracy on African American English, word level vs sentence level (certification standard 95%) | 82.9% vs 59.5% | Jones et al., Testifying While Black, Language 2019 |
| Transcriptions that changed the who, what, when or where | 31%; 27 court reporters, 9 speakers, 83 sentences | Jones et al., Testifying While Black, Language 2019 |
| Median WER, Black vs white home healthcare patients, AWS General and AWS Medical | 50% vs 33% (P = 0.016); 54% vs 37% (P = 0.0012) | Zolnoori et al., JAMIA Open 2024 |
| Median WER, Black vs white patients, Whisper large-v3 | 75% vs 72% (P = 0.66); 35 patients | Zolnoori et al., JAMIA Open 2024 |
| Error rate in dictated clinical notes: speech recognition draft, after transcriptionist, signed note | 7.4%, 0.4%, 0.3%; 217 notes | Zhou et al., JAMA Network Open 2018 |
| Whisper transcriptions with hallucinated text; share of hallucinations with explicit harms | 1.4% of 13,140 segments; 38% | Koenecke et al., Careless Whisper, FAccT 2024 |
| Mock video interviews transcribed by four ASR methods and one human | N = 1,014 and N = 414; significant ASR bias in nearly all comparisons, no meaningful measurement bias in scores | Hickman et al., Journal of Applied Psychology 2025 |
| African American voice technology users who modified their dialect to be understood | 93% (n = 28 of 30); 77% reported frustration with errors | Mengesha et al., Frontiers in Artificial Intelligence 2021 |
Context: the court reporter study measures human transcribers, not software, and the clinical dictation study is the most recent available data on signed-note error rates (Zhou et al., JAMA Network Open 2018). The hallucination experiments ran in 2023; the authors found 19% of the 312 hallucinated transcriptions involved violence, 13% inaccurate associations and 8% false authority, and no comparable hallucinations when the affected segments were re-run through other vendors’ systems (Careless Whisper). The hiring result (Hickman et al.) comes from mock interviews, not live selection decisions.
Summary: Speech Recognition Accent Bias & Word Error Rate by the Numbers
| Metric | Value | Source |
|---|---|---|
| Average WER, Black vs white speakers, five commercial systems | 0.35 vs 0.19 | Stanford (Koenecke et al.), PNAS 2020 |
| Mean WER, African American vs Caucasian American speakers, four commercial systems | 20% vs 15% | University of Washington (Scott et al.), LREC 2026 |
| Whisper-large-v2 WER, Black or African American vs white speakers | 9.52 vs 3.96 | Meta (Veliche et al.), Fair-Speech Dataset 2024 |
| Whisper-large-v2 WER, male vs female speakers | 7.91 vs 3.86 | Meta (Veliche et al.), Fair-Speech Dataset 2024 |
| Whisper large WER, EdAcc accented English vs LibriSpeech test-clean | 19.7% vs 2.7% | University of Edinburgh (Sanabria et al.), EdAcc paper 2023 |
| Whisper Large V2 WER, CORAAL vs LibriSpeech Clean | 16.2 vs 2.7 | OpenAI (Radford et al.), Whisper paper 2022 |
| Best average WER, Open ASR Leaderboard English short-form track | 5.42 across 86 benchmarked systems | Open ASR Leaderboard paper (Srivastav et al.), v4 March 2026 |
| Whisper-large WER on Pan-African English, general vs clinical | 0.240 vs 0.375 | Olatunji et al., AfriSpeech-200, TACL 2023 |
| WIL gap, English vs non-English first-language speakers | 0.14 (P<0.001) | DiChristofano et al., arXiv 2023 |
| Dutch ASR WER on read speech, non-native vs native speakers | 55.9% vs 26.1% | TU Delft (Feng et al.), Quantifying Bias in ASR 2021 |
| Baseline WER, typical vs Parkinson’s-related dysarthric speech | 3.4% vs 36.3% | Speech Accessibility Project (Hasegawa-Johnson et al.), JSLHR 2024 |
| Whisper-large-v2 baseline vs top challenge system on dysarthric speech | 17.82% vs 8.11% | Zheng et al., Interspeech 2025 Speech Accessibility Project Challenge |
| Personalized vs speaker-independent WER, disordered speech | 4.6% vs 31% | Google Research (Green et al.), Interspeech 2021 |
| Mean WER across six systems, stuttered vs fluent versions | .281 vs .105 | Mujtaba et al., Lost in Transcription, NAACL 2024 |
| Whisper training hours, total vs non-English | 680,000 vs 117,000 (96 languages) | OpenAI (Radford et al.), Whisper paper 2022 |
| Common Voice Scripted Speech v27.0 | 295 languages, 29,295 validated hours | Mozilla Common Voice, cv-dataset v27.0 2026 |
| Median WER, Black vs white home healthcare patients, AWS General | 50% vs 33% (P = 0.016) | Zolnoori et al., JAMIA Open 2024 |
| Court reporter word accuracy on African American English vs certification standard | 82.9% vs 95% | Jones et al., Testifying While Black, Language 2019 |
| Voice technology users who modified their dialect | 93% (n = 28 of 30) | Mengesha et al., Frontiers in Artificial Intelligence 2021 |
Methodology and Sources
We ran 23 searches and read the primary paper, dataset page or journal record for every figure above. WER is reported in each source’s own notation: fractions for Koenecke et al., AfriSpeech-200 and Mujtaba et al., percentages elsewhere. Figures older than three years are flagged in the sections as most recent available data.
- Koenecke et al., Racial disparities in automated speech recognition, PNAS 2020
- Scott, Liang, Wassink and Levow, A Sociophonetic Analysis of Racial Bias in Commercial ASR Systems Using the Pacific Northwest English Corpus, LREC 2026
- Veliche et al. (Meta), Towards measuring fairness in speech recognition: Fair-Speech dataset, 2024
- Radford et al. (OpenAI), Robust Speech Recognition via Large-Scale Weak Supervision, 2022
- Sanabria et al. (University of Edinburgh), The Edinburgh International Accents of English Corpus, ICASSP 2023
- Srivastav et al., Open ASR Leaderboard paper, v4 March 2026
- DiChristofano, Shuster, Chandra and Patwari, Global Performance Disparities Between English-Language Accents in Automatic Speech Recognition, 2023
- Olatunji et al., AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASR, TACL 2023
- Feng, Kudina, Halpern and Scharenborg (TU Delft), Quantifying Bias in Automatic Speech Recognition, 2021
- Tatman (University of Washington), Gender and Dialect Bias in YouTube’s Automatic Captions, ACL Ethics in NLP Workshop 2017
- Attia et al., Kid-Whisper, 2024
- Hasegawa-Johnson et al., Community-Supported Shared Infrastructure in Support of Speech Accessibility, JSLHR 2024
- Zheng et al., The Interspeech 2025 Speech Accessibility Project Challenge
- Green et al. (Google Research), Automatic Speech Recognition of Disordered Speech, Interspeech 2021 and the Project Euphonia dataset blog, 2021
- Mujtaba et al., Lost in Transcription, NAACL 2024
- Lea et al., From User Perceptions to Technical Improvement: Enabling People Who Stutter to Better Use Speech Recognition, CHI 2023
- Koenecke et al., Careless Whisper: Speech-to-Text Hallucination Harms, FAccT 2024
- Pratap et al., Scaling Speech Technology to 1,000+ Languages (MMS), 2023; Omnilingual ASR, 2025; Google USM, 2023; FLEURS, 2022
- Mozilla Common Voice, cv-dataset repository and v27.0 statistics, 2026
- Jones, Kalbfeld, Hancock and Clark, Testifying While Black, Language 95(2), 2019
- Zolnoori et al., Decoding disparities, JAMIA Open 2024
- Zhou et al., Analysis of Errors in Dictated Clinical Documents Assisted by Speech Recognition Software and Professional Transcriptionists, JAMA Network Open 2018
- Hickman, Langer, Saef and Tay, Automated speech recognition bias in personnel selection, Journal of Applied Psychology 2025
- Mengesha et al., Impact of Automated Speech Recognition Errors on African Americans, Frontiers in Artificial Intelligence 2021
- Data watch: several sources disagree with themselves or measure different things. The Whisper paper’s text gives a LibriSpeech clean WER of 2.5 while its Table 2 lists 2.7 (we use the table). Careless Whisper’s abstract says roughly 1% of transcriptions were hallucinated and its results section says 1.4% (we use 1.4%), and its authors note the experiments date from 2023. The Speech Accessibility Project abstract’s 36.3% and 23.7% match the paper’s unshared-prompt rows; on all test prompts the fine-tuned figure is 18.72%. The Fair-Speech text mentions 50 speakers in the top income band while its table lists 41. DiChristofano et al. report word information lost, not WER, so their gaps are not comparable with the other rows. The Scott et al. evaluation ran in 2022 on 16 speakers and was published in 2026. The Open ASR Leaderboard changes continuously: the October 2025 version of the paper listed 64 models with a best average WER of 5.63, the March 2026 version 86 systems and 5.42. The Common Voice hour totals come from the repository’s release chart data, and the gender proportions from its v27.0 statistics file. For Testifying While Black the journal page sits behind an access check, so the figures were read from the lead author’s published summary of the paper; that summary gives two different paraphrase-comprehension figures, which we therefore left out. We could not verify a widely repeated claim about a hiring vendor’s WER for non-native speakers and dropped it.
Last updated: October 11, 2026. We update this roundup quarterly, and the next refresh is expected when Mozilla publishes Common Voice Scripted Speech v28.0.