Fed 301,317 clips of pure non-speech audio, Whisper large-v3 wrote text for 121,378 of them (40.3%), and 9.1% of those fabricated outputs were loops of the same phrase repeated (Barański et al., ICASSP 2025). On the UrbanSound8K library of city sounds, a separate team measured text output on 99.97% of clips (Wang et al., Calm-Whisper, Interspeech 2025). On real speech the rate is far lower but not evenly spread: in the AphasiaBank corpus, segments from speakers with aphasia hallucinated more often than control segments, and longer non-vocal stretches predicted hallucination (Koenecke et al., ACM FAccT 2024). Leaderboards do not catch this, because the Open ASR Leaderboard ranks models on word error rate and speed alone (Srivastav et al., Open ASR Leaderboard paper 2026), the same metric behind the accent and dialect gaps we tracked separately. We aggregated data from Cornell, Carnegie Mellon, Hugging Face, Interspeech and ICASSP papers, AssemblyAI, the Associated Press, and 2026 arXiv work on detection and mitigation.
In this roundup, a “hallucination” means text in a transcript that has no basis in the audio: invented words, phrases or whole sentences, as opposed to mishearing a word that was actually said.
TL;DR
- Whisper large-v3 produced text for 40.3% of 301,317 non-speech clips, 9.1% of them looping (Barański et al., ICASSP 2025).
- Whisper large-v3 produced text for 99.97% of UrbanSound8K noise clips before a targeted fine-tune (Wang et al., Calm-Whisper, Interspeech 2025).
- Stuttered clips hallucinated at 14% with Whisper v2 and 2.9% with v3 (Sridhar and Wu, Interspeech 2025).
- Universal-1 reported a 30% relative reduction in its hallucination rate versus Whisper large-v3, a vendor’s own claim (AssemblyAI, Anatomy of Industrial Scale Multilingual ASR 2024).
- Hallucination Error Rate correlates with distribution shift at α = 0.91 (Atwany et al., arXiv 2025).
- SHALLOW hallucination scores track WER at ρs > 0.80 only in the 10-30% WER band (Koudounas et al., arXiv 2025).
- Whisper turbo still gave lexical output on 74.3% of non-speech clips (Borodin et al., The Null Token Knows, arXiv 2026).
- The Open ASR Leaderboard reports only WER and RTFx, with Conformer-based systems 3.77x faster than Whisper-based ones (Srivastav et al., Open ASR Leaderboard paper 2026).
- Calm-Whisper’s fix raised LibriSpeech test-clean WER only from 2.12% to 2.19% (Wang et al., Calm-Whisper, Interspeech 2025).
- A WhisperX VAD baseline averaged 5.24% non-speech hallucination with 0% false rejection (Abbasihafshejani and Jadliwala, arXiv 2026).
1. How Often ASR Invents Text in Real Speech
The controlled numbers on real speech are small, and that is exactly the trap. 187 of 13,140 AphasiaBank segments (1.4%) reliably produced fabricated text in Whisper in the Careless Whisper study, while the same segments sent through Amazon and Google produced zero comparable fabrications. A low rate that lands on a medical history, an interview or a court record is a different risk from a misheard word, because invented sentences read fluently and nobody thinks to check them.
The field reports collected by the Associated Press in October 2024 are much higher, and they are not comparable with the lab numbers: a machine learning engineer told AP he found hallucinations in about half of the over 100 hours of Whisper transcriptions he analyzed. These are inspections by individual users, with no shared definition, so treat them as a signal of exposure, not a rate.
| Metric | Value | Source |
|---|---|---|
| AphasiaBank segments that reliably produced hallucinated text in Whisper (April-May 2023 runs) | 187 of 13,140 segments (1.4%) | Koenecke et al., Careless Whisper, ACM FAccT 2024 |
| Hallucinating segments that still hallucinated when re-run in December 2023 | 12 of 187 | Koenecke et al., Careless Whisper, ACM FAccT 2024 |
| Whisper large-v3 Hallucination Error Rate, LibriSpeech vs LibriSpeech with added noise | 0.3% vs 0.4% | Atwany et al., Lost in Transcription, arXiv 2025 |
| Public-meeting transcripts with hallucinations, inspected by a University of Michigan researcher | 8 of every 10 | Associated Press investigation, October 2024 |
| Whisper transcripts with hallucinations, reported by one developer | Nearly every one of 26,000 | Associated Press investigation, October 2024 |
| Fabrication rate (5+ consecutive inserted or substituted words per hour), Universal-1 vs Whisper large-v3 | 41% relative reduction (vendor’s own claim) | AssemblyAI, Anatomy of Industrial Scale Multilingual ASR 2024 |
| Overall hallucination rate (5+ consecutive errors per hour), Universal-1 vs Whisper large-v3 and vs Canary-1B | 30% and 22% relative reductions (vendor’s own claim) | AssemblyAI, Anatomy of Industrial Scale Multilingual ASR 2024 |
Context: by the December 2023 re-run, most, not all, of the reproducible cases had stopped hallucinating: 12 of 187 is 6.4% (our calculation). The Amazon and Google comparison was run on January 27, 2024. The AssemblyAI figures are relative reductions measured on 146 hours of English test data chosen by the vendor, and absolute per-hour rates appear only in a figure.
2. Silence, Noise and Non-Speech Audio: The Worst Case
On audio with no speech at all, the correct transcript is empty, so every output word is fabricated. That makes non-speech the cleanest stress test, and the results are stark: depending on the dataset and the counting rule, Whisper large-v3 writes text for anywhere from 21.35% to 100% of non-speech clips across the studies below. The spread is not noise in the research. It reflects different sound libraries, clip lengths and definitions, which is why no single “Whisper hallucination rate” exists.
Bigger is not reliably better here. In the August 2026 Null Token audit, Whisper small gave lexical output on more clips than large-v3, while in the June 2026 steering study large-v3 was worse than small on the combined test set. Canary-1B, a non-Whisper model, sat in the same range. Separating voices from background sound is its own research field (see our speech separation statistics); recognizers that skip that step inherit the problem.
| Metric | Value | Source |
|---|---|---|
| Whisper large-v3 inferences on non-speech audio (AudioSet, MUSAN, UrbanSound8K, FSD50K, noise and silence) that produced text | 121,378 of 301,317 (40.3%); 11,049 of them (9.1%) involved looping | Barański et al., ICASSP 2025 |
| Whisper large-v3 hallucination rate on UrbanSound8K urban-sound clips | 99.97% | Wang et al., Calm-Whisper, Interspeech 2025 |
| Whisper large-v3 vs Whisper small, hallucination rate on a combined non-speech test set (WHAM!, FSD50k, UrbanSound8K) | 86.8% vs 72.6% | Aparin et al., Whisper Hallucination Detection and Mitigation, arXiv 2026 |
| Whisper large-v3 hallucination rate on UrbanSound8K, separate 2026 measurement | 95.2% | Aparin et al., Whisper Hallucination Detection and Mitigation, arXiv 2026 |
| Share of non-speech clips with lexical output, Whisper small, large-v3 and turbo | 96.2%, 77.0% and 74.3% | Borodin et al., The Null Token Knows, arXiv 2026 |
| Share of non-speech clips with lexical output, Canary-1B (subset) | 79.3% | Borodin et al., The Null Token Knows, arXiv 2026 |
| Non-blank output on 200 AudioSet ambient-noise samples, Whisper large-v3, Canary-1B and Universal-1 | 100%, 100% and 10.5% (vendor’s own test) | AssemblyAI, Anatomy of Industrial Scale Multilingual ASR 2024 |
Context: a third 2026 paper measured Whisper large-v3 at 76.08% on UrbanSound8K, 44.25% on ESC-50 and 21.35% on a filtered FSD50K subset (Abbasihafshejani and Jadliwala, arXiv 2026). “Lexical output” in the Null Token paper means any letter left after a normalizer strips event tags, punctuation and digits, a broader count than matching whole phrases.
3. What Gets Invented: Phrase Fingerprints and Clip Length
Whisper’s non-speech hallucinations are not random. Two phrases, “thank you” (24.76%) and “thanks for watching” (10.32%), account for about 35% of all hallucinations on non-speech audio in the ICASSP 2025 study. Those are video sign-offs, a pattern consistent with training text drawn from online video captions rather than verified transcripts (our interpretation, not the authors’ measurement).
The repetition is also the practical opening for defense: because a small set of outputs recurs, the authors built a “bag of hallucinations”, a list of frequent fabricated outputs that can be filtered after transcription. Length matters as much as content. Padding 10-second speech clips with 30 seconds of non-speech sound produced detected hallucinations in 44.5% of files, against 0.3% with 1 second of padding.
| Metric | Value | Source |
|---|---|---|
| Share of non-speech hallucinations that were “thank you” and “thanks for watching” | 24.76% and 10.32% | Barański et al., ICASSP 2025 |
| Unique hallucinated outputs; outputs that appeared more than once; share of all hallucinations covered by those repeats | 41,231 unique; 1,270 repeated; 67.08% | Barański et al., ICASSP 2025 |
| Detected hallucinations when 1 s vs 30 s of non-speech sound is added before or after speech | 0.3% vs 44.5% | Barański et al., ICASSP 2025 |
| Whisper large-v3 training data | 1 million hours weakly labeled plus 4 million hours pseudo-labeled audio | OpenAI, Whisper large-v3 model card |
| Original Whisper training data (2022 release) | 680,000 hours of multilingual and multitask supervision | Radford et al., Whisper paper 2022 |
| Hallucinations by models built with 8% injected noise, share of evaluation datasets | 1-2% for most models; 8% for one model | Frieske and Shi, Hallucinations in Neural ASR, arXiv 2024 |
Context: “weakly labeled” means the text paired with each recording was not verified word by word against the audio, and “pseudo-labeled” means the text was produced by an earlier model. The Whisper authors themselves list “complete hallucination where the model will output a transcript entirely unrelated to the actual audio” among their known long-form failure modes (Radford et al., 2022). Most recent available data for the original training set: Whisper paper, 2022.
4. Who Gets Hallucinated: Speech Patterns With Pauses and Repetitions
Hallucination risk follows the shape of the audio, and some speakers produce audio with more silence or more repeated sounds. Whisper v2 hallucinated on 14% of stuttered podcast clips versus 3.3% of fluent clips, according to the Interspeech 2025 stuttering benchmark, which drew 2,621 clips from the SEP-28K podcast dataset. Whisper v3 cut both numbers sharply, but stuttered clips still hallucinated about three times as often as fluent ones (2.9% vs 0.9%).
For aphasia (a disorder that affects language), the Careless Whisper team found the mechanism in the pauses: speakers with aphasia had far longer non-vocal shares of their recordings, and longer non-vocal shares predicted hallucination. For context on how common these speech patterns are, see our stuttering demographics statistics.
| Metric | Value | Source |
|---|---|---|
| Aphasia vs control audio segments that produced hallucinations in Whisper | 1.7% vs 1.2% (regression: p=0.019) | Koenecke et al., Careless Whisper, ACM FAccT 2024 |
| Aphasia vs control hallucination rate, groups matched on participant demographics | 1.82% vs 1.16% (p-value = 0.044) | Koenecke et al., Careless Whisper, ACM FAccT 2024 |
| Share of total recording duration that is non-vocal, aphasia speakers vs control speakers | 41% vs 15% | Koenecke et al., Careless Whisper, ACM FAccT 2024 |
| Stuttered vs fluent clips with hallucinations, Whisper v2 | 14% (293 of 2,086) vs 3.3% (18 of 534) | Sridhar and Wu, Interspeech 2025 |
| Stuttered vs fluent clips with hallucinations, Whisper v3 | 2.9% (61 of 2,086) vs 0.9% (5 of 534) | Sridhar and Wu, Interspeech 2025 |
| Clips with sound repetitions (repeating the first sound of a word) that hallucinated, Whisper v2 vs v3 | 22.6% vs 6.9% | Sridhar and Wu, Interspeech 2025 |
| Clips with prolongations (stretching a sound) that hallucinated, Whisper v2 vs v3 | 15.4% vs 1.47% | Sridhar and Wu, Interspeech 2025 |
Context: the stuttering paper detected candidate hallucinations by running Whisper twice on each clip and flagging large disagreements, then reviewed every flag by hand. Its clip counts differ slightly between sections (2,079 stuttered clips sampled, 2,086 in the hallucination tally), so we quote each figure with its own denominator.
5. Measuring Hallucination: Metrics Proposed Beyond WER
Word error rate counts substitutions, deletions and insertions, but it cannot tell a misheard word from an invented sentence. The Hallucination Error Rate proposed by Carnegie Mellon and MBZUAI researchers correlates with distribution shift at α = 0.91, meaning hallucinations rise in step with how far test audio drifts from training audio (Atwany et al., 2025). Their pipeline uses an LLM as the judge, which makes it cheap enough to run at scale.
The SHALLOW benchmark splits hallucination into four axes: lexical (invented words), phonetic (sound-alike errors), morphological (wrong word forms) and semantic (changed meaning). Its key finding is where WER stops being informative: correlations are strong at low error rates and collapse as WER climbs, which is exactly where hallucination matters most.
| Metric | Value | Source |
|---|---|---|
| Agreement on hallucination labels, human vs human and human vs GPT-4o-mini | 0.71 vs 0.60 | Atwany et al., Lost in Transcription, arXiv 2025 |
| Cost of the LLM labeling pipeline vs human evaluation | $78 per 1M segments, about 375 times cheaper | Atwany et al., Lost in Transcription, arXiv 2025 |
| Correlation between SHALLOW scores and WER, at 10-30% WER vs at 90% WER (lexical-semantic pair) | ρs > 0.80 vs below -0.45 | Koudounas et al., SHALLOW, arXiv 2025 |
| Synthetic test: correlation with WER, lexical fabrication vs semantic errors | ρ = 0.98 vs ρ = 0.15 | Koudounas et al., SHALLOW, arXiv 2025 |
| Models and test datasets in the SHALLOW evaluation | 12 models, 10 datasets | Koudounas et al., SHALLOW, arXiv 2025 |
| Thresholds used to separate hallucinations from phonetic errors | Cosine similarity 0.2; WER 30; perplexity 200 | Frieske and Shi, Hallucinations in Neural ASR, arXiv 2024 |
| Two-run hallucination flag used in the stuttering benchmark | WER between runs above 0.6, BERT F1 below 0.6, or more than 4 inserted words | Sridhar and Wu, Interspeech 2025 |
Context: these metrics are not interchangeable. AssemblyAI counts runs of 5 or more consecutive errors per hour of audio, the Null Token paper counts clips with any letter on empty-reference audio, and Hallucination Error Rate counts examples labeled as hallucinations. A 2% on one scale says nothing about a 2% on another.
6. Leaderboard WER Versus Hallucination: Model Comparisons
The Hugging Face Open ASR Leaderboard paper is the field’s reference for accuracy and speed, and its scope is explicit: it reports WER and RTFx (inverse real-time factor, how many seconds of audio a system processes per second) and nothing else. The paper does not discuss hallucinations or repetition loops as model errors. Its speed data still matter for this topic, because the faster Conformer families (an encoder design that mixes convolution and attention) are the obvious alternatives to Whisper on that leaderboard.
The CMU and SHALLOW tables show why a ranking by WER is not a ranking by safety. Whisper large-v3 posted a WER above 65% on simulated air traffic control speech while its hallucination rate stayed much lower, and the reverse happened on an adversarial set. Accuracy standards in broadcast captioning face a similar split between timing, completeness and correctness (see our caption accuracy standards statistics).
| Metric | Value | Source |
|---|---|---|
| Metrics reported by the Open ASR Leaderboard | WER and RTFx only | Srivastav et al., Open ASR Leaderboard paper, v4 March 2026 |
| Average RTFx, Conformer-based vs Whisper-based systems | 758 vs 201 (3.77x faster) | Srivastav et al., Open ASR Leaderboard paper, v4 March 2026 |
| Whisper Large v3 WER and RTFx on the long-form English track | 11.2 WER; RTFx 68.6 | Srivastav et al., Open ASR Leaderboard paper, v4 March 2026 |
| Rank of the best TDT model and of the best CTC model on English short-form accuracy | 10th and 34th | Srivastav et al., Open ASR Leaderboard paper, v4 March 2026 |
| Whisper large-v3 WER vs Hallucination Error Rate on ATCOsim (simulated air traffic control speech) | 65.3% vs 17.6% | Atwany et al., Lost in Transcription, arXiv 2025 |
| Whisper large-v3 WER vs Hallucination Error Rate on an adversarial test set | 33.3% vs 49.8% | Atwany et al., Lost in Transcription, arXiv 2025 |
| Whisper-Large-v3 vs Canary, average WER; average lexical fabrication score | 14.20 vs 14.26; 6.74 vs 5.43 | Koudounas et al., SHALLOW, arXiv 2025 |
Context: TDT and CTC are decoding methods that emit text aligned to the audio frames, rather than generating free text token by token like Whisper’s decoder. The leaderboard notes they offer substantially better RTFx at lower accuracy. Near-identical average WER for Whisper-Large-v3 and Canary in SHALLOW hides that Canary made fewer lexical fabrications.
7. Mitigation Results: VAD, Decoding Changes and Model Edits
The most reliable fix is still not to send silence to the model. Silero VAD cut WER on speech padded with non-speech sound from 104.8% to 8.0% in the padding experiments described in section 3, and adding de-looping plus the bag-of-hallucinations filter brought it to 6.5%. VAD (voice activity detection) is a separate model that decides whether a stretch of audio contains speech. In Whisper’s own paper, VAD was one of the decoding heuristics on a path from 11.0 average long-form WER with greedy decoding to 10.0 with all heuristics (Radford et al., 2022).
Model-side fixes work, but each one moves the error somewhere else. Calm-Whisper fine-tuned only 3 attention heads (numbers 1, 6 and 11) and kept WER almost flat. Steering and end-of-text interventions also suppressed most fabrication but started deleting real speech or wrecking Chinese recognition. The null-token authors put it plainly: abstention methods should be judged by both suppression and deletion costs, not hallucination reduction alone.
| Metric | Value | Source |
|---|---|---|
| WER on speech padded with non-speech sound: no processing vs Silero VAD vs Silero VAD plus de-looping and bag-of-hallucinations filter | 104.8% vs 8.0% vs 6.5% | Barański et al., ICASSP 2025 |
| Detected hallucinations with beam size 1 vs beam size 5 | 19.5% vs 28.2% | Barański et al., ICASSP 2025 |
| Calm-Whisper: UrbanSound8K hallucination rate before vs after; LibriSpeech test-clean WER before vs after | 99.97% vs 15.51%; 2.12% vs 2.19% | Wang et al., Calm-Whisper, Interspeech 2025 |
| Sparse-autoencoder steering, Whisper small, combined non-speech test set | 72.63% vs 14.11% | Aparin et al., Whisper Hallucination Detection and Mitigation, arXiv 2026 |
| Same steering on Whisper large-v3: AISHELL-1 Chinese character error rate before vs after | 10.54 vs 66.24 | Aparin et al., Whisper Hallucination Detection and Mitigation, arXiv 2026 |
| End-of-text (null token) intervention, Whisper small: lexical-output rate; deleted reference words per minute | 91.7% vs 3.3%; 33.05 vs 1.58 | Borodin et al., The Null Token Knows, arXiv 2026 |
| WhisperX VAD baseline: average non-speech hallucination rate; false rejection of real speech | 5.24%; 0% | Abbasihafshejani and Jadliwala, arXiv 2026 |
| Whisper vs WhisperX on TED-LIUM: insertion error rate; repeated 5-word sequences | 7.7 vs 6.7; 221 vs 189 | Bain et al., WhisperX paper 2023 |
Context: the beam-size result runs against intuition, since the Whisper authors use beam search with 5 beams to reduce repetition looping on speech, while on non-speech padding the wider beam produced more detected hallucinations. Calm-Whisper also cut multi-token hallucinations (more than five tokens) from 742 to 91. The WhisperX data are from 2023, so this is the most recent available data for that benchmark.
Summary: Speech Recognition Hallucination by the Numbers
| Metric | Value | Source |
|---|---|---|
| Whisper large-v3 non-speech clips that produced text | 40.3% of 301,317 | Barański et al., ICASSP 2025 |
| Whisper large-v3 hallucination rate on UrbanSound8K noise | 99.97% | Wang et al., Calm-Whisper, Interspeech 2025 |
| Whisper large-v3 vs Whisper small, combined non-speech test set | 86.8% vs 72.6% | Aparin et al., arXiv 2026 |
| Whisper turbo lexical output on non-speech clips | 74.3% | Borodin et al., The Null Token Knows, arXiv 2026 |
| AphasiaBank segments with reproducible Whisper hallucinations | 187 of 13,140 (1.4%) | Koenecke et al., Careless Whisper, ACM FAccT 2024 |
| Public-meeting transcripts with hallucinations, one Michigan researcher | 8 of every 10 | Associated Press investigation, October 2024 |
| Hallucination rate, Universal-1 vs Whisper large-v3 | 30% relative reduction (vendor’s own claim) | AssemblyAI, 2024 |
| Stuttered vs fluent clips hallucinating, Whisper v2 | 14% vs 3.3% | Sridhar and Wu, Interspeech 2025 |
| Whisper large-v3 WER vs HER on an adversarial set | 33.3% vs 49.8% | Atwany et al., arXiv 2025 |
| SHALLOW vs WER correlation, 10-30% WER band | ρs > 0.80 | Koudounas et al., SHALLOW, arXiv 2025 |
| Conformer-based vs Whisper-based average RTFx | 758 vs 201 | Srivastav et al., v4 March 2026 |
| Calm-Whisper LibriSpeech test-clean WER before vs after the fix | 2.12% vs 2.19% | Wang et al., Calm-Whisper, Interspeech 2025 |
| Null-token intervention: deleted words per minute | 33.05 vs 1.58 | Borodin et al., arXiv 2026 |
| Sparse-autoencoder steering, Whisper small hallucination rate | 72.63% vs 14.11% | Aparin et al., arXiv 2026 |
| WhisperX VAD baseline hallucination rate and false rejection | 5.24%; 0% | Abbasihafshejani and Jadliwala, arXiv 2026 |
| Whisper vs WhisperX TED-LIUM insertion error rate | 7.7 vs 6.7 | Bain et al., WhisperX paper 2023 |
Methodology and Sources
We read the primary paper, model card or news investigation behind every figure above and quote each in the source’s own notation (percentages, fractions, ρ or α). Where a paper’s prose and tables disagree, we used the table value or the explicit sentence and flagged the conflict below.
- Koenecke et al., Careless Whisper: Speech-to-Text Hallucination Harms, ACM FAccT 2024
- Burke and Schellmann, Associated Press investigation of Whisper transcription, October 2024, with the Cornell Chronicle summary of the Careless Whisper study, June 2024
- Barański et al., Investigation of Whisper ASR Hallucinations Induced by Non-Speech Audio, ICASSP 2025
- Wang et al., Calm-Whisper: Reduce Whisper Hallucination On Non-Speech By Calming Crazy Heads Down, Interspeech 2025
- Sridhar and Wu, J-j-j-just Stutter: Benchmarking Whisper’s Performance Disparities on Different Stuttering Patterns, Interspeech 2025
- Atwany et al., Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models, arXiv 2025
- Koudounas et al., Hallucination Benchmark for Speech Foundation Models (SHALLOW), arXiv 2025
- Srivastav et al., Open ASR Leaderboard paper, v4 March 2026
- Borodin et al., The Null Token Knows: Reducing Message-Free Hallucination in ASR and NMT, arXiv 2026
- Aparin et al., Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders, arXiv 2026
- Abbasihafshejani and Jadliwala, Reducing Hallucinated Transcripts in Whisper via Hallucination Space Projection, arXiv 2026
- AssemblyAI (McCann Ramirez, Chkhetiani et al.), Anatomy of Industrial Scale Multilingual ASR, 2024
- Bain et al., WhisperX: Time-Accurate Speech Transcription of Long-Form Audio, 2023
- Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision (Whisper paper), 2022
- OpenAI, Whisper large-v3 model card on Hugging Face
- Frieske and Shi, Hallucinations in Neural Automatic Speech Recognition: Identifying Errors and Hallucinatory Models, arXiv 2024
- Data watch: the Careless Whisper abstract says about 1% of transcriptions hallucinated, while the body reports 1.4% (187 of 13,140 segments) and 312 hallucinated transcriptions across two runs; AP described the same work as 187 hallucinations in more than 13,000 clips. The Hallucination Space Projection paper reports a 31.31% average baseline that does not match its own per-dataset figures (we quote only the per-dataset values and its WhisperX baseline). The steering paper gives 95.2% for Whisper large-v3 on UrbanSound8K in one table and 95.98% in another. The stuttering benchmark uses 2,079 and 2,086 as stuttered-clip totals in different sections. Atwany et al., SHALLOW, the Null Token paper (submitted to AAAI-27), the steering paper and the projection paper are arXiv preprints without a confirmed peer-reviewed venue at the time of reading. AssemblyAI figures are a vendor’s own claims, and the AP figures are individual field reports, not controlled measurements. The Whisper paper (2022), WhisperX (2023) and Frieske and Shi (2024) are the most recent available data for their specific benchmarks.
Last updated: October 24, 2026. We update this roundup quarterly, and the next refresh is expected when the ICASSP 2027 proceedings and the next revision of the Open ASR Leaderboard paper are published.