Adding video of a speaker’s lips cut a speech recognizer’s word error rate from 88.9% to 24.2% in heavy white noise at -7.5 dB (Ma et al., Auto-AVSR, ICASSP 2023). Lips alone remain far weaker than audio: a 2025 lip reader reports 18.7% WER on the LRS3 TED-talk benchmark (Thomas, Fish and Bowden, VALLR 2025), while an LLM-based audio-visual system reaches 0.77% (Cappellazzo et al., Llama-AVSR, ICASSP 2025). Human lip reading is a lower bar still: professional lip readers scored 86.4% to 92.9% WER on YouTube clips (Shillingford et al., DeepMind and Google, Large-Scale Visual Speech Recognition 2018). We aggregated data from the Oxford Visual Geometry Group (VGG) LRS2 and LRS3 datasets, Meta AI’s AV-HuBERT papers, the WildVSR generalization study, the World Health Organization, and peer-reviewed hearing research in JASA, JSLHR and Trends in Hearing. For how audio-only recognizers break down across speakers, see our speech recognition accent bias statistics.
A note on terms used throughout: WER (word error rate) is the share of words a system gets wrong, counting substituted, missing and extra words, so lower is better. VSR (visual speech recognition) means machine lip reading from silent video, ASR means audio-only recognition, and AVSR means audio-visual recognition that uses both. SNR (signal-to-noise ratio) measures how much louder the speech is than the background noise; a negative SNR means the noise is louder than the voice.
TL;DR
- English WER in 0 dB babble noise fell from 13.8% for audio-only Whisper-Large to 7.2% for audio-visual Whisper-Flamingo (Rouditchenko et al., Interspeech 2024).
- An LLM-based system reports 0.77% WER audio-visual and 0.79% audio-only on LRS3 (Cappellazzo et al., Llama-AVSR, ICASSP 2025).
- A Google lip-reading model reports 12.8% WER on LRS3, described as rivaling audio-only models from four years earlier (Chang et al., Google, Conformers paper 2023).
- A phoneme-based lip reader reached 18.7% WER on LRS3 with 99.4% less labelled data than the next best approach (Thomas, Fish and Bowden, VALLR 2025).
- Professional lip readers scored 92.9% WER without context and 86.4% with context, vs 40.9% for a machine system (Shillingford et al., DeepMind and Google 2018).
- 84 normal-hearing college students identified 12.4% of words from silent video of sentences (Altieri, Pisoni and Townsend, JASA 2011).
- Adults with early-onset hearing loss read 43.55% of words from lips vs 18.57% for normal-hearing adults (Auer and Bernstein, JSLHR 2007).
- With only 30 labelled hours, AV-HuBERT reached 32.5% lip-reading WER, beating a 33.6% system trained on 31,000 hours (Shi et al., Meta AI, ICLR 2022).
- State-of-the-art lip-reading models lose 30 WER points on average when moved from LRS3 to the WildVSR test set (Djilali et al., WildVSR 2023).
- Visible faces lowered the noise threshold for understanding speech by 3.74 dB for clear speech (Blackburn et al., Trends in Hearing 2019).
- 430 million people require rehabilitation for disabling hearing loss (World Health Organization, Deafness and hearing loss fact sheet 2026).
1. Lips vs Audio vs Both: The Modality Gap in Word Error Rate
The gap between eyes and ears is still more than an order of magnitude. The lowest lip-reading scores on LRS3 in this roundup are 12.8% and 18.7% WER, while the lowest audio and audio-visual scores are below 1%. Different sounds can produce exactly the same lip movement (‘p’ and ‘b’ are a standard example), so a silent model must infer from context what an audio model simply hears.
The more surprising number is at the top. In clean audio, video adds almost nothing and can even cost a little: Whisper-Flamingo scored 0.76% WER audio-visual against 0.68% audio-only on LRS3 (Rouditchenko et al., Interspeech 2024). The value of lips shows up in noise (see section 2), not on studio-quality TED talks. The Llama-AVSR paper shows the same near-tie, at 0.79% vs 0.77%.
| Metric | Value | Source |
|---|---|---|
| WER on LRS3, lip reading only vs audio only vs audio plus video, model trained on 3,448 hours | 19.1% vs 1.0% vs 0.9% | Ma et al., Auto-AVSR, ICASSP 2023 |
| WER on LRS3, audio only vs audio plus video, LLM-based system | 0.79% vs 0.77% | Cappellazzo et al., Llama-AVSR, ICASSP 2025 |
| WER on LRS3, audio only vs audio plus video, Whisper-based system | 0.68% vs 0.76% | Rouditchenko et al., Whisper-Flamingo, Interspeech 2024 |
| Lip-reading WER on LRS3, Google Conformer model | 12.8% | Chang et al. (Google), Conformers Are All You Need for Visual Speech Recognition, 2023 |
| Lip-reading WER on LRS3, phoneme-based model with a large language model | 18.7% | Thomas, Fish and Bowden, VALLR, arXiv 2025 (revised January 5, 2026) |
| WER on BBC television (LRS), lips only vs clean audio only vs clean audio plus lips | 50.2% vs 17.7% vs 13.9% | Chung et al. (Oxford VGG and Google DeepMind), Lip Reading Sentences in the Wild, CVPR 2017 |
Context: the Auto-AVSR paper reports these figures for a model trained on 3,448 hours, and its lips-only error rate on the BBC-based LRS2 set was lower, at 14.6%, against 1.5% audio-only. These models share a test set but not their training data, so the scores are not like-for-like. Audio-only systems have failure modes of their own beyond plain word errors, covered in our speech recognition hallucination statistics.
2. How Much Video Helps Speech Recognition at Low SNR
This is where lip video earns its place. As noise rises, audio-only error climbs steeply while audio-visual error climbs slowly, so the gap widens exactly where captions and voice interfaces fail most. Meta’s audio-visual model averaged 5.8% WER in noise against 25.8% for the audio-only version, across 4 noise types and 5 SNR levels from -10 to 10 dB (Shi, Hsu and Mohamed, Interspeech 2022).
The type of noise matters. Babble (many people talking in the background) hurt the audio-only model least in the -7.5 dB tests below, but babble was also used during training, so the authors warn that those results are not directly comparable with pink and white noise (broadband hiss). Those two noise types produced the largest gaps.
| Metric | Value | Source |
|---|---|---|
| WER at -7.5 dB SNR, audio only vs audio plus video: white noise; pink noise; babble noise | 88.9% vs 24.2%; 56.8% vs 16.2%; 8.3% vs 5.6% | Ma et al., Auto-AVSR, ICASSP 2023 |
| WER in white noise at 2.5 dB SNR, audio only vs audio plus video | 10.4% vs 4.3% | Ma et al., Auto-AVSR, ICASSP 2023 |
| Average WER across 4 noise types and 5 SNR levels (-10 to 10 dB), audio-only model vs audio-visual model built on AV-HuBERT | 25.8% vs 5.8% | Shi, Hsu and Mohamed (Meta AI), Robust Self-Supervised Audio-Visual Speech Recognition, Interspeech 2022 |
| WER in babble noise, prior state of the art trained on 433 labelled hours vs AV-HuBERT trained on 30 labelled hours | 28.0% vs 14.1% | Shi, Hsu and Mohamed (Meta AI), Interspeech 2022 |
| English WER in babble noise at 0 dB SNR (MuAViC test), audio-only Whisper-Large vs audio-visual Whisper-Flamingo | 13.8% vs 7.2% | Rouditchenko et al., Whisper-Flamingo, Interspeech 2024 |
| WER on BBC television at 10 dB and 0 dB SNR, audio only vs audio plus lips | 37.6% and 62.9% vs 27.6% and 42.0% | Chung et al. (Oxford VGG and Google DeepMind), CVPR 2017 |
Outlier note: the 2017 BBC figures come from a much weaker generation of models, but they show the same pattern at a higher error level. At 0 dB, adding lips cut the error by 20.9 points (62.9 minus 42.0); at -7.5 dB white noise in 2023, the cut was 64.7 points (88.9 minus 24.2). Most recent available data for that BBC comparison: the 2017 study.
3. Human Lip Readers as a Baseline
Human lip reading is far harder than most people assume. Normal-hearing young adults identified just 12.4% of words from silent video of spoken sentences (Altieri, Pisoni and Townsend, JASA 2011). Lifelong reliance on the face raises that substantially: adults who lost their hearing early read more than twice as many words as hearing adults (Auer and Bernstein, JSLHR 2007).
Machines overtook trained professionals years ago on benchmark video, but the comparison has limits. The human baselines used small samples: 300 videos and 3 lip readers for LipNet, 200 videos and one professional for the BBC study. That professional could see which program each clip came from, study sample training videos and replay clips without limit, so the human score is not a cold-read result. For the classroom side of deaf education, see our deaf and hard of hearing student education statistics.
| Metric | Value | Source |
|---|---|---|
| Words correct from silent video of CUNY sentences, 84 normal-hearing college students | 12.4% (SD 6.67%) | Altieri, Pisoni and Townsend (Indiana University), Some normative data on lip-reading skills, JASA 2011 |
| Words correct from silent video of 30 sentences, adults with early-onset hearing loss (n = 112) vs normal hearing (n = 220) | 43.55% (SD 17.48) vs 18.57% (SD 13.18); t(330) = 14.576, p < .01 | Auer and Bernstein, JSLHR 2007 |
| Consonants recognized from silent video with no mask: children with normal hearing vs children with hearing loss vs adults with normal hearing | about 31% vs 33% vs 42% | Lalonde, Buss, Miller and Leibold, Frontiers in Psychology 2022 |
| WER on GRID sentences from unseen speakers, 3 hearing-impaired lip readers vs LipNet | 47.7% vs 11.4% | Assael et al. (University of Oxford and Google DeepMind), LipNet 2016 |
| WER on BBC test clips, one professional lip reader vs lips-only machine model | 73.8% vs 50.2% (87.6% for the human when declined clips are counted) | Chung et al. (Oxford VGG and Google DeepMind), CVPR 2017 |
| WER on YouTube clips, professional lip readers without context vs with context vs machine system | 92.9% vs 86.4% vs 40.9% | Shillingford et al. (DeepMind and Google), Large-Scale Visual Speech Recognition 2018 |
| Visual benefit for normal-hearing adults (drop in the SNR needed to get half the key words right), clear speech vs noise-vocoded speech | 3.74 dB (SD 2.67) vs 4.68 dB (SD 3.28) | Blackburn et al., Trends in Hearing 2019 |
Context: in experiment 1 of the Blackburn study, listeners averaged only 1.56% correct with video alone, yet seeing the talker still improved their thresholds over audio alone (the 3.74 dB and 4.68 dB figures come from experiment 2). Lip reading works best as a supplement to sound, for machines and people alike. Noise-vocoded speech is audio deliberately degraded by removing fine detail from the sound. Most recent available data for normative human scores: Altieri et al., 2011, and Auer and Bernstein, 2007.
4. Lip-Reading Datasets: Hours, Speakers and Vocabulary
Lip-reading research runs on a handful of public datasets built from broadcast TV and TED talks. The LRS3 benchmark holds over 400 hours from 5,594 TED and TEDx talks, yet its test set is just 1 hour (Afouras, Chung and Zisserman, LRS3-TED 2018). A whole field ranks its models on 1,452 test utterances, which is one reason section 6 matters.
The largest training sets belong to industry. The 3,886-hour LSVSR YouTube set is nearly ten times the LRS3 pre-training set (3,886 divided by 407 is 9.5), while public research compensates with machine-generated labels and self-supervised pre-training.
| Metric | Value | Source |
|---|---|---|
| LRS3-TED total video and source talks; test set size | over 400 hours from 5,594 TED and TEDx talks; 1 hour, 1,452 utterances, 451 speakers | Afouras, Chung and Zisserman (Oxford VGG), LRS3-TED 2018 |
| LRS2-BBC pre-train set vs train-val set | 195 hours (96k utterances) vs 29 hours (47k utterances) | Afouras, Chung and Zisserman (Oxford VGG), LRS3-TED 2018, Table 1 |
| LSVSR YouTube dataset: hours, utterances, vocabulary | 3,886 hours; 2,934,899 utterances; 127,055 words | Shillingford et al. (DeepMind and Google), 2018 |
| English VoxCeleb2 video used for pre-training; total AV-HuBERT pre-training data | 1,326 hours; 1,759 hours | Shi et al. (Meta AI), AV-HuBERT, ICLR 2022 |
| MuAViC multilingual audio-visual corpus | 1,200 hours in 9 languages | Anwar et al. (Meta AI), MuAViC 2023 |
| WildVSR test set: hours, utterances, speakers, vocabulary | 4.8 hours; 2,854 utterances; 618 speakers; 6,040 words | Djilali et al. (Technology Innovation Institute), Do VSR Models Generalize Beyond LRS3?, 2023 |
Context: the LRS3 pre-train and train-val splits overlap in content, so their hours should not be added together. Most recent available data for the core benchmarks: LRS3-TED paper, 2018. The MuAViC paper bills itself as the largest open benchmark for multilingual audio-visual speech recognition.
5. Training Data Scale vs Lip-Reading Accuracy
More video still buys accuracy, but smarter training buys more per hour. AV-HuBERT matched a 31,000-hour system with just 30 labelled hours: 32.5% WER vs 33.6% (Shi et al., Meta AI, ICLR 2022). Self-supervised pre-training on unlabelled video did the heavy lifting: the same architecture trained from scratch on those 30 hours scored 92.3%.
Hours still matter, but they do not settle the ranking. Going from 818 to 3,448 training hours cut LRS3 lip-reading error by 13.9 points (33.0 minus 19.1). Two much larger runs in the table below then land 4.2 points apart, 17.0% on about 90,000 hours and 12.8% on 100k hours, which points to architecture as well as volume.
| Metric | Value | Source |
|---|---|---|
| Lip-reading WER on LRS3 with 818 vs 1,902 vs 3,448 training hours (a model trained on about 90,000 non-public hours: 17.0%) | 33.0% vs 23.5% vs 19.1% | Ma et al., Auto-AVSR, ICASSP 2023 |
| Lip-reading WER on LRS3, AV-HuBERT with 30 labelled hours vs all 433 labelled hours plus self-training (previous best: 33.6% on 31,000 hours) | 32.5% vs 26.9% | Shi et al. (Meta AI), AV-HuBERT, ICLR 2022 |
| Lip-reading WER on LRS3, same large architecture trained from scratch with 30 vs 433 labelled hours | 92.3% vs 62.3% | Shi et al. (Meta AI), AV-HuBERT, ICLR 2022, Table 1 |
| Labelled data used by a phoneme-based lip reader compared with the next best approach | 99.4% less (for 18.7% WER on LRS3) | Thomas, Fish and Bowden, VALLR, arXiv 2025 |
| Training video behind Google’s 12.8% lip-reading WER on LRS3 | 100k hours of transcribed YouTube segments, then fine-tuning on LRS3 (over 400 hours) | Chang et al. (Google), Conformers Are All You Need for Visual Speech Recognition, 2023 |
Context: bigger language models are not a shortcut either. Jain and Harte (arXiv, September 2025) conclude that large language model decoders mainly refine contextual reasoning rather than visual features, and that stronger visual encoders are needed for further progress.
6. Beyond TED Talks: In-the-Wild Video and Non-English Lip Reading
Benchmark WER flatters lip reading. When researchers built a new test set from YouTube video by closely following the LRS3 creation process, state-of-the-art lip-reading models lost 30 WER points on average (Djilali et al., WildVSR 2023). Audio moved far less: Whisper went from 1.1% to 4.2%. The rankings held, though, so a better LRS3 score still predicts a better WildVSR score.
Language is the second blind spot. The method that scored 31.5% WER on English LRS3 scored 56.3% or worse on Spanish, Italian and French TED-style talks (Ma, Petridis and Pantic, Nature Machine Intelligence 2022). Most recent available data for non-English lip reading: that 2022 paper.
| Metric | Value | Source |
|---|---|---|
| Average WER increase for state-of-the-art lip-reading models, LRS3 to WildVSR | 30 points | Djilali et al., WildVSR 2023 |
| Lip-reading WER, best model tested (Auto-AVSR, 3,448 hours), LRS3 vs WildVSR | 19.1% vs 38.6% | Djilali et al., WildVSR 2023, Table 2 |
| Audio-only WER, Whisper, LRS3 vs WildVSR | 1.1% vs 4.2% | Djilali et al., WildVSR 2023, Table 2 |
| Fitted relationship between WildVSR WER and LRS3 WER across models | WildVSR WER = about 1.31 x LRS3 WER + 14.05 | Djilali et al., WildVSR 2023 |
| Lip-reading WER, Llama-2-13B decoder model, LRS3 vs WildVSR | 24.7% vs 47.0% | Jain and Harte, From Hype to Insight, arXiv 2025 |
| Lip-reading error by language, best run: Mandarin character error rate (CMLR) vs Spanish, Italian and French word error rate (Multilingual TEDx) | 8.0% vs 56.3%, 57.4% and 66.2% | Ma, Petridis and Pantic, Nature Machine Intelligence 2022 |
Context: the Mandarin figure counts errors per character, not per word, so it is not directly comparable with the WER figures. The WildVSR gap for the best model is 19.5 points (38.6 minus 19.1), below the 30-point average, consistent with the authors’ finding that stronger models degrade less.
7. Privacy, Surveillance and Accessibility
The documented privacy concern is concrete: researchers themselves flag CCTV. Ma, Petridis and Pantic call visual speech recognition “a dual-use technology” that “can be deployed for surveillance via CCTV or even via smartphone cameras, which raises privacy concerns”, and they argue that government regulation is necessary. They also report that a smartphone app to help speech-impaired people communicate was being trialled in UK NHS hospitals at the time. The UK’s Surveillance Camera Commissioner, Tony Porter, warned of lip-reading surveillance technology on the horizon in August 2019 (Big Brother Watch, August 28, 2019).
The counterweight is technical. Oxford VGG researchers judge CCTV eavesdropping “very low risk” because cameras “usually operate at much lower resolution and frame rates and from unusual angles” (Prajwal, Afouras and Zisserman, 2021). The 3,886-hour system was trained only on faces turned within 30 degrees of the camera, and its authors say it does not perform well in other contexts (Shillingford et al., 2018). The accessibility case rests on a large population: the WHO fact sheet lists captioning and speech reading among the supports for hearing loss. Visual language AI beyond the lips is covered in our sign language AI and computer vision statistics.
| Metric | Value | Source |
|---|---|---|
| Training conditions of the 3,886-hour lip-reading system: head angle; minimum face size; minimum frame rate | yaw and pitch within plus or minus 30 degrees; at least 80 pixels between the eyes; clips under 23 frames per second removed | Shillingford et al. (DeepMind and Google), 2018 |
| Minimum height of the closed lips in the video for reliable automatic lip reading | 4 pixels | Bear, Harvey, Theobald and Lan, Resolution limits on visual speech recognition, ICIP 2014 |
| Lowest video resolution tested in the resolution study vs full HD source | 42 x 32 vs 1440 x 1080 pixels (18 resolutions tested) | Bear, Harvey, Theobald and Lan, ICIP 2014 |
| People who require rehabilitation for disabling hearing loss | 430 million (over 5% of the world’s population) | World Health Organization, Deafness and hearing loss fact sheet, March 3, 2026 |
| People projected to have some degree of hearing loss vs disabling hearing loss by 2050 | nearly 2.5 billion vs over 700 million | World Health Organization, Deafness and hearing loss fact sheet, March 3, 2026 |
| Annual global cost of unaddressed hearing loss | almost US$1 trillion | World Health Organization, Deafness and hearing loss fact sheet, March 3, 2026 |
Context: the Bear et al. study used about 30 minutes of video from two talkers, so the 4-pixel floor is an indicative limit, not a guarantee. Most recent available data for the resolution limit: Bear et al., 2014.
Summary: Lip-Reading Visual Speech Recognition WER by the Numbers
| Metric | Value | Source |
|---|---|---|
| WER at -7.5 dB SNR in white noise, audio only vs audio plus video | 88.9% vs 24.2% | Ma et al., Auto-AVSR, ICASSP 2023 |
| Average WER across 4 noise types and 5 SNR levels, audio-only vs audio-visual model | 25.8% vs 5.8% | Shi, Hsu and Mohamed (Meta AI), Interspeech 2022 |
| English WER in 0 dB babble noise, audio-only Whisper-Large vs Whisper-Flamingo | 13.8% vs 7.2% | Rouditchenko et al., Whisper-Flamingo, Interspeech 2024 |
| WER on LRS3, audio only vs audio plus video, LLM-based system | 0.79% vs 0.77% | Cappellazzo et al., Llama-AVSR, ICASSP 2025 |
| Lip-reading WER on LRS3, Google Conformer model | 12.8% | Chang et al. (Google), 2023 |
| Lip-reading WER on LRS3, phoneme-based model with 99.4% less labelled data | 18.7% | Thomas, Fish and Bowden, VALLR, arXiv 2025 |
| Lip-reading WER on LRS3, AV-HuBERT with 30 labelled hours vs previous best on 31,000 hours | 32.5% vs 33.6% | Shi et al. (Meta AI), AV-HuBERT, ICLR 2022 |
| Average WER increase for state-of-the-art lip-reading models, LRS3 to WildVSR | 30 points | Djilali et al., WildVSR 2023 |
| Lip-reading WER, Llama-2-13B decoder model, LRS3 vs WildVSR | 24.7% vs 47.0% | Jain and Harte, arXiv 2025 |
| Words correct from silent sentences, 84 normal-hearing college students | 12.4% | Altieri, Pisoni and Townsend, JASA 2011 |
| Words correct from silent sentences, early-onset hearing loss vs normal hearing | 43.55% vs 18.57% | Auer and Bernstein, JSLHR 2007 |
| WER on YouTube clips, professional lip readers (without vs with context) vs machine | 92.9% and 86.4% vs 40.9% | Shillingford et al. (DeepMind and Google), 2018 |
| WER on BBC clips, professional lip reader vs lips-only model | 73.8% vs 50.2% | Chung et al. (Oxford VGG and Google DeepMind), CVPR 2017 |
| Visual benefit in noise for clear speech, normal-hearing adults | 3.74 dB | Blackburn et al., Trends in Hearing 2019 |
| LRS3-TED video and talks | over 400 hours, 5,594 talks | Afouras, Chung and Zisserman (Oxford VGG), 2018 |
| Minimum height of the closed lips in the video for reliable automatic lip reading | 4 pixels | Bear et al., ICIP 2014 |
| People who require rehabilitation for disabling hearing loss | 430 million | World Health Organization, 2026 |
Methodology and Sources
- Ma et al., Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels, ICASSP 2023: modality gap, noise robustness tables and training-data scaling.
- Cappellazzo et al., Large Language Models are Strong Audio-Visual Speech Recognition Learners (Llama-AVSR), ICASSP 2025: audio and audio-visual WER on LRS3.
- Rouditchenko et al., Whisper-Flamingo, Interspeech 2024: clean and babble-noise WER for audio-only vs audio-visual Whisper.
- Chang et al. (Google), Conformers Are All You Need for Visual Speech Recognition, 2023: 12.8% lip-reading WER on LRS3 and its 100k-hour training set.
- Thomas, Fish and Bowden, VALLR: Visual ASR Language Model for Lip Reading, arXiv 2025: 18.7% WER and labelled-data efficiency.
- Jain and Harte, From Hype to Insight: Rethinking Large Language Model Integration in Visual Speech Recognition, arXiv September 2025: LLM-decoder results on LRS3 and WildVSR.
- Meta AI: Shi et al., AV-HuBERT, ICLR 2022 and Shi, Hsu and Mohamed, Robust Self-Supervised Audio-Visual Speech Recognition, Interspeech 2022: data efficiency and noise robustness.
- Anwar et al. (Meta AI), MuAViC multilingual audio-visual corpus, 2023: corpus size and language coverage.
- Oxford VGG: Afouras, Chung and Zisserman, LRS3-TED, 2018 and Prajwal, Afouras and Zisserman, Sub-word Level Lip Reading With Visual Attention, 2021: dataset sizes and the CCTV risk assessment.
- Chung et al., Lip Reading Sentences in the Wild, CVPR 2017: BBC lips vs audio results, noise results and professional lip-reader baseline.
- Assael et al., LipNet: End-to-End Sentence-level Lipreading, 2016: results on the GRID sentence corpus and hearing-impaired lip-reader baseline.
- Shillingford et al., Large-Scale Visual Speech Recognition, 2018: LSVSR dataset, professional lip-reader WER, face-angle and resolution constraints.
- Djilali et al., Do VSR Models Generalize Beyond LRS3? (WildVSR), 2023: in-the-wild generalization gap.
- Ma, Petridis and Pantic, Visual Speech Recognition for Multiple Languages in the Wild, Nature Machine Intelligence 2022: non-English WER and the dual-use and privacy discussion.
- Altieri, Pisoni and Townsend, Some normative data on lip-reading skills, JASA 2011: human visual-only sentence scores.
- Auer and Bernstein, Enhanced visual speech perception in individuals with early-onset hearing impairment, JSLHR 2007: deaf vs hearing lip-reading scores.
- Blackburn et al., Visual Speech Benefit in Clear and Degraded Speech, Trends in Hearing 2019: visual benefit in decibels.
- Lalonde, Buss, Miller and Leibold, Face Masks Impact Auditory and Audiovisual Consonant Recognition in Children With and Without Hearing Loss, Frontiers in Psychology 2022: visual-only consonant scores for children and adults.
- Bear, Harvey, Theobald and Lan, Resolution limits on visual speech recognition, ICIP 2014: minimum lip resolution.
- World Health Organization, Deafness and hearing loss fact sheet, March 3, 2026: hearing loss prevalence, projections and cost.
- Big Brother Watch, August 28, 2019: report of the UK Surveillance Camera Commissioner’s warning on lip-reading surveillance.
- Data watch: scores on the LRS3 test set are not like-for-like across papers that train on different data, and test sets are tiny (1 hour for LRS3). Several papers claim state-of-the-art status at different dates: Llama-AVSR’s 0.79% and 0.77% sit beside Whisper-Flamingo’s 0.68% and 0.76%, and VALLR’s 18.7% sits beside Google’s lower 12.8%, which used far more labelled data. VALLR and the Jain and Harte study are preprints with no peer-reviewed venue listed. Hours for the same dataset differ by source: the LRS3-TED paper lists 407 pre-train hours and the LRS2-BBC pre-train and train-val sets at 195 and 29 hours, while Ma, Petridis and Pantic count 438 hours for LRS3 and 223 hours for LRS2. The WildVSR paper gives two slopes for its fit (1.31 and 1.30), and its Appendix B says Whisper scores under 4 WER on both test sets, which conflicts with the 4.2 in its Table 2. It also does not state the Whisper model size. Human baselines rely on very small panels (one professional for the BBC study, three hearing-impaired lip readers for LipNet), and the normative human data dates from 2007 and 2011. The Lalonde study simulated masks rather than having talkers wear them. We found no peer-reviewed measurement of lip-reading accuracy from real CCTV footage: the surveillance concerns cited here are documented warnings, not measured deployments.
Last updated: October 31, 2026. We update this roundup quarterly, and the next refresh is expected when the ICASSP 2027 and Interspeech 2027 proceedings publish new LRS3 and WildVSR results.