Across 67 studies and 392 effect sizes, machine scores correlate with human judgments of pronunciation at r = .606, lower than the pooled figures for fluency or delivery (Wang and Min, Language Testing 2026). Benchmark numbers tell a similar story at a finer grain: the best phoneme-level Pearson correlation (PCC) cited on the speechocean762 corpus is 0.693, and a leading 2026 error detector still misses about one real mispronunciation in three. Test vendors report far higher figures for whole tests, up to 0.97, though their own pronunciation subscores sit well below that. Those numbers measure agreement, not sales, which is why this roundup sits apart from our broader language learning statistics. We aggregated data from Language Testing, the speechocean762 corpus, ETS research reports, Pearson validation reports, PLOS One, Language Learning, and a dozen other papers and technical documents.
TL;DR
- Pooled machine-human correlation for pronunciation: r = .606, against r = .654 across all speaking constructs (Wang and Min, Language Testing 2026).
- Highest phone-level PCC cited on speechocean762: 0.693, up from a 0.45 corpus baseline (Cao et al., arXiv 2025; Zhang et al., 2021).
- Versant English Test Pronunciation subscore: 0.88 machine-human correlation, the lowest of its subscores (Pearson, Versant validation summary 2022).
- Human vs human on TOEFL Junior Speaking: r = .89, against r = .81 for human vs machine (ETS, TOEFL Junior automated scoring report 2015).
- Two teachers in one classroom study agreed at only r = 0.583, while another pair reached r = 0.901 (Chen and Sun, PLOS One 2025).
- Best reported L2-ARCTIC mispronunciation detection F1 in our sources: 71.77% (Geng et al., arXiv 2026).
- False acceptance of real mispronunciations in that system: 32.76%, against 3.39% false rejection of correct sounds (Geng et al., arXiv 2026).
- GPT-4o best result on speechocean762: 0.502 at utterance level, against 0.782 for Azure Pronunciation Assessment (Wang et al., arXiv 2025).
- One consumer app correlated with teachers at r = 0.869 yet scored 92.00 on average against 86.41 from humans (Chen and Sun, PLOS One 2025).
- Moving a detector from a general model to a target learner group raised ROC-AUC from 0.72 to 0.85 (Nijat et al., Applied Sciences 2026).
1. The Baseline: What 67 Studies Say About Machine-Human Agreement
Pearson’s r runs from -1 to 1, and it measures whether two sets of scores rise and fall together. The pooled pronunciation figure of r = .606 is strong by that scale, but squaring it shows the limit: .606 x .606 = .367, so about 37% of the variation in human scores is shared with the machine.
The ranking of constructs is the surprise. Delivery (in speaking assessment, how the speech sounds overall rather than what it says) outscores pronunciation by a wide margin, so systems are better at the broad impression than at the specific construct this roundup tracks. Grammar and vocabulary trail everything. The Language Testing meta-analysis also found that ASR accuracy was not a significant moderator, which undercuts the common assumption that a better speech recognizer automatically means a better scorer.
| Metric | Value | Source |
|---|---|---|
| Size of the meta-analysis and overall pooled machine-human correlation | 67 studies, 392 effect sizes, r = .654 (p < .001) | Wang and Min, Language Testing 2026 |
| Pooled correlation for delivery vs pooled correlation for pronunciation | r = .784 vs r = .606 | Wang and Min, Language Testing 2026 |
| Pooled correlation for overall speaking proficiency vs pooled correlation for fluency | r = .686 vs r = .618 | Wang and Min, Language Testing 2026 |
| Pooled correlation for content vs pooled correlation for grammar and vocabulary | r = .574 vs r = .499 | Wang and Min, Language Testing 2026 |
| Moderator variables with a significant effect vs without one | 10 significant (including age group, rater expertise and inter-rater reliability) vs 4 not significant (including ASR accuracy) | Wang and Min, Language Testing 2026 |
| Pearson correlation band that one vendor labels “High” agreement with human judges | 0.5 to 1.0 | Microsoft, Pronunciation Assessment transparency note 2026 (vendor’s own guideline) |
Context note: on Microsoft’s own scale, even the weakest construct in the meta-analysis almost reaches “High”. Labels like that say little about whether a score can be trusted for one learner. Used as a pass mark, the Microsoft guideline would clear almost every pooled figure in the table above.
2. The Human Ceiling: How Much Raters Agree With Each Other
A machine cannot be measured against a perfect standard, because the human reference is noisy too. Pairs of trained teachers in one 2025 study landed anywhere from r = 0.901 down to r = 0.583, and that lower figure is below the pooled pronunciation correlation for machines in section 1. The sample was only 30 students, and the task was overall spoken proficiency, so it illustrates the spread and does not prove machines beat teachers.
The clinicians in the Arabic study below agreed far more closely than the teachers, though on a different language and task, and averaging helps: the PLOS One study moves from 0.713 for a single rater to 0.882 for the mean of three. ICC, the intraclass correlation coefficient, is a different yardstick from Pearson’s r: in its absolute-agreement form it also penalizes raters for being harsher or more lenient overall. The most useful comparison is the ETS TOEFL Junior study, which reports both numbers on the same data. The gap between the two is the honest measure of what automation still lacks. The same problem of listener judgment runs through synthetic speech quality scores.
| Metric | Value | Source |
|---|---|---|
| TOEFL Junior form-level Speaking, five responses: human vs human correlation compared with human vs machine correlation | r = .89 vs r = .81 | ETS, TOEFL Junior automated scoring report 2015 (most recent available data) |
| Three speech-language pathologists scoring Arabic pronunciation: pairwise PCC range | 0.858 to 0.927 | Azad et al., Harf-Speech, arXiv 2026 |
| Same three clinicians: pairwise ICC(2,1) range | 0.846 to 0.922 | Azad et al., Harf-Speech, arXiv 2026 |
| Three English teachers scoring spoken proficiency: agreement of a single rater, ICC(2,1) | 0.713 (95% CI 0.539 to 0.835) | Chen and Sun, PLOS One 2025 |
| Same three teachers: agreement of the averaged score, ICC(2,k) | 0.882 (95% CI 0.778 to 0.938) | Chen and Sun, PLOS One 2025 |
| Same three teachers: strongest pair vs weakest pair | r = 0.901 vs r = 0.583 | Chen and Sun, PLOS One 2025 |
| Reduction in the influence of trait-irrelevant listener variables after rater training: holistic oral proficiency ratings vs comprehensibility ratings | 75% vs 50% | Dalman and Kang, IntechOpen 2019, summarizing Kang et al. 2019 (most recent available data) |
| Agreement statistic published for the five expert annotators of speechocean762 | None reported | Zhang et al., speechocean762 paper 2021 |
Outlier note: the last row matters more than it looks. The main public benchmark for this field publishes no inter-annotator correlation or kappa, so nobody reading its paper can say how close 0.693 sits to the human ceiling.
3. High-Stakes Tests: Vendor Numbers, and the Pronunciation Gap Inside Them
Whole-test figures look nearly perfect, and pronunciation is where they stop looking that way. In Pearson’s own Versant data, Pronunciation is the weakest subscore at 0.88 while the Overall score reaches 0.97. The drop is consistent across Pearson’s documents: 0.81 in a PTE Academic presentation, and 0.85 for Manner of Speaking, the Versant category that contains pronunciation.
These figures are far above the pooled r = .606 from section 1. Part of the difference is sampling: test scores aggregate many items, and validation samples often span a wide ability range (the Versant Speaking and Listening report includes 26 L1 English speakers among its 260 adults), and wide ranges raise correlations. The meta-analysis also lists system developer and sample unit among its significant moderators. Read vendor numbers as upper bounds from favorable conditions. For how much weight these exams carry, see our English test volume statistics.
| Metric | Value | Source |
|---|---|---|
| Versant English Test, machine vs human correlation by score, n = 143 | Overall 0.97, Sentence Mastery 0.97, Vocabulary 0.96, Fluency 0.94, Pronunciation 0.88 | Pearson, Versant validation summary 2022 (vendor’s own claim, most recent available data) |
| Versant English Test split-half reliability of the Pronunciation score: machine scoring vs human scoring, n = 143 | 0.97 vs 0.99 | Pearson, Versant validation summary 2022 (vendor’s own claim) |
| Versant scores correlated with outside exams, n = 130 each: TOEFL iBT Speaking vs IELTS Speaking | 0.75 vs 0.76 | Pearson, Versant validation summary 2022 (vendor’s own claim) |
| Versant English Speaking and Listening Test, machine vs human correlation, n = 260 | Overall .96, Speaking .88, Manner of Speaking .85 | Pearson, Versant Speaking and Listening validation report 2024 (vendor’s own claim) |
| PTE Academic speaking, machine vs human correlation by trait, N = 158 | Pronunciation 0.81, Fluency 0.82, Overall 0.96 | Pearson, MELTA automated scoring presentation 2018 (vendor’s own claim, most recent available data) |
| ETS SpeechRater timeline: year the service was first deployed vs year version 5.0 went operational | 2006 vs 2016 | Chen et al., ETS Research Report 2018 |
Context note: split-half reliability shows how consistent a scorer is with itself across two halves of a test. A machine can reach 0.97 on that measure and still disagree with humans, which is exactly the pattern in the Versant Pronunciation rows.
4. The speechocean762 Benchmark: Progress by Granularity
speechocean762 holds 5,000 English utterances from 250 non-native speakers, half of them children, each scored by five experts. Its scores exist at three levels: phone (a single speech sound, not a telephone), word and utterance. The metrics are PCC, where higher is better, and MSE (mean squared error), where lower is better. GOP, or Goodness of Pronunciation, is the classic statistical score that newer models try to beat.
Phone-level PCC climbed from 0.45 to 0.693 between the 2021 baseline and a 2023 model, and a paper revised on February 5, 2026 still cites 0.693 as the highest known figure. Agreement falls as the unit shrinks: utterance scores reach the 0.80s while phone and word scores stay in the 0.60s. Averaging over a sentence hides errors, so a learner’s overall score can be trustworthy while the feedback on a single sound is not. The corpus paper also notes that most phone scores, mapped to a 0 to 10 scale, fall between 8 and 10, so correct sounds dominate the data.
| Metric | Value | Source |
|---|---|---|
| Original phone-level baseline: raw GOP value vs GOP-based features | MSE 0.69 and PCC 0.25 vs MSE 0.16 and PCC 0.45 | Zhang et al., speechocean762 paper 2021 |
| GOPT transformer model, phone level | MSE 0.085 ± 0.001, PCC 0.612 ± 0.003 | Gong et al., GOPT, ICASSP 2022 |
| Highest phone-level PCC cited for speechocean762 | 0.693 | Chao et al., Interspeech 2023, as cited in Cao et al., arXiv 2025 |
| Segmentation-free GOP features fed into GOPT, phone level | PCC 0.648 ± 0.002 | Cao et al., arXiv 2025 |
| Word-level total PCC vs utterance-level total PCC, GOPT | 0.549 vs 0.742 | Gong et al., GOPT, ICASSP 2022 |
| Word-level total PCC vs utterance-level total PCC, 3MH model | 0.694 vs 0.811 | Wang et al., GPT-4o pronunciation study, arXiv 2025 |
| HMamba model at utterance level: accuracy, fluency and prosody PCC | 0.807, 0.848 and 0.843 | Sara and Chowdhury, arXiv 2026 |
| Pooled machine-human correlation by technology generation: traditional machine learning, deep learning, transformer | r = .597, r = .641, r = .680 | Wang and Min, Language Testing 2026 |
Outlier note: the same 3MH model reaches only 0.361 for word stress and 0.374 for utterance completeness (Wang et al., arXiv 2025). Those two are the hardest labels on the benchmark.
5. Phoneme Error Detection: Precision, Recall, and the Missed One-Third
Scoring asks how good a sound was. Mispronunciation detection asks a yes or no question for each phoneme, and it reports F1, a single score that balances precision (how many flags were right) and recall (how many real errors were caught). The top L2-ARCTIC result in our sources is an F1 of 71.77%, from a 2026 preprint, 8.73 points above (71.77 - 63.04) the best baseline in a 2025 comparison.
The split between the two kinds of error reflects a design priority. False acceptance means a wrong sound passes as correct, and false rejection means a correct sound gets flagged. The numbers are complementary to recall: 100 - 67.24 = 32.76, so one real error in three goes unflagged. The 2025 paper calls false rejection the most critical metric because wrongly correcting learners frustrates them, and the results fit that priority: a learner sees few false alarms while many true errors go unflagged.
| Metric | Value | Source |
|---|---|---|
| L2-ARCTIC benchmark: total speakers vs speakers in the test set | 24 vs 6 | Tu et al., PER-MDD, arXiv 2025 |
| F1 gain of an early end-to-end detector over the GOP method vs over an end-to-end baseline | 27.71% vs 11.05% | Yan et al., Interspeech 2020 |
| L2-ARCTIC F1: best prior baseline (Joint-Align) vs retrieval system needing no training (PER-MDD) | 63.04% vs 69.60% | Tu et al., PER-MDD, arXiv 2025 |
| L2-ARCTIC, CROTTC-IF system: F1, precision and recall | 71.77%, 76.94% and 67.24% | Geng et al., CROTTC-IF, arXiv 2026 |
| L2-ARCTIC, CROTTC-IF system: false rejection vs false acceptance | 3.39% vs 32.76% | Geng et al., CROTTC-IF, arXiv 2026 |
| Phone error rate on L2-ARCTIC: CROTTC-IF vs its acoustic model alone vs PER-MDD | 46.52 vs 17.48 vs 104.08 | Geng et al., CROTTC-IF, arXiv 2026 |
| Detection F1 on speechocean762: CROTTC-IF vs HMamba vs MuFFIN | 57.16% vs 63.85% vs 67.98% | Geng et al., arXiv 2026; Chao and Chen, HMamba, arXiv 2025 |
Outlier note: phone error rates above 100 are not a typo. The authors attribute them to inserted phones, and they show that a system can detect errors well while transcribing poorly. Arabic results look similar: the Iqra’Eval2 leaderboard in the same 2026 paper tops out at an F1 of 72.01%.
6. Commercial APIs, Consumer Apps, and Chatbots
The gap between services is larger than the gap between research models. A 2025 test on speechocean762 put Azure Pronunciation Assessment at 0.782 at utterance level. The best GPT-4o setup reached 0.502, and at phoneme level it reached only 0.249. General-purpose multimodal models can listen, but on this evidence they cannot yet judge a single sound.
Tests outside vendor labs are less flattering to commercial tools. Won’s 2025 study of 190 read-aloud recordings from Korean elementary learners found moderate correlations between Azure phoneme scores and human judgments, while the word error rates of the most accurate systems, near zero, correlated only weakly with human ratings. A recognizer that understands everyone stops being a useful proxy for pronunciation. The SmartSpeech row below makes a second point: a tool can correlate strongly with humans and still inflate the average score by more than five points.
| Metric | Value | Source |
|---|---|---|
| Vendor statement on agreement with human judges | Above 0.5 Pearson correlation | Microsoft, Pronunciation Assessment transparency note 2026 (vendor’s own claim) |
| Azure Pronunciation Assessment on speechocean762: word accuracy PCC vs utterance total PCC | 0.623 vs 0.782 | Wang et al., GPT-4o pronunciation study, arXiv 2025 |
| Best GPT-4o result on speechocean762: phoneme, word total and utterance total PCC | 0.249, 0.277 and 0.502 | Wang et al., GPT-4o pronunciation study, arXiv 2025 |
| Arabic speech scored by clinicians, 40 samples: PCC with the mean clinician score, Harf-Speech vs Azure | 0.791 vs 0.635 | Azad et al., Harf-Speech, arXiv 2026 |
| Same Arabic test: RMSE on a 0 to 5 scale, Harf-Speech vs Azure | 1.05 vs 1.28 | Azad et al., Harf-Speech, arXiv 2026 |
| Three AI speaking apps vs the average of human raters, 30 students: AI Speaking Master, TalkAI Language Practice, SmartSpeech AI Assessment | r = 0.854, 0.873 and 0.869 | Chen and Sun, PLOS One 2025 |
| SmartSpeech mean score vs human mean score | 92.00 vs 86.41, difference 5.589 (95% CI 5.023 to 6.155), Cohen’s d = 3.688 | Chen and Sun, PLOS One 2025 |
| Duolingo pronunciation scorer: Spearman correlation with expert human ratings | 0.82 | Duolingo English Test blog on Cai et al. 2025 (vendor’s own summary) |
Context note: the Harf-Speech figures come from a preprint by the system’s own authors, so treat its win over Azure with care. Duolingo describes 0.82 as nearly matching the agreement between human raters, but the blog post does not print the human figure.
7. Transfer and Fairness: What Happens Outside the Benchmark
Much of the benchmark evidence above comes from one corpus of Mandarin-speaking learners, and it does not travel cleanly. The same system that scored 0.661 for accuracy on speechocean762 fell to 0.506 on L2-ARCTIC, a corpus with speakers of several native languages, and retraining on the new data lifted it only to 0.527. Adapting to the target group works better for detection: one 2026 study raised ROC-AUC from 0.72 to 0.85 by fine-tuning on the learners themselves. ROC-AUC measures how well a system separates right from wrong across all thresholds, where 0.5 is chance.
Bias findings are mostly directional, since few papers publish group-level gaps. Still, the ETS study of test takers with speech impairments and Duolingo’s Language Learning paper both found measurable problems, including in scorers built by large testing organizations. The recognition layer underneath has its own gaps, covered in our accent bias statistics, and children need their own evidence, as our children’s speech recognition statistics show.
| Metric | Value | Source |
|---|---|---|
| Accuracy PCC of one scoring model fitted on speechocean762: tested on speechocean762 vs tested on L2-ARCTIC | 0.661 vs 0.506 | Sara and Chowdhury, arXiv 2026 |
| Same model on L2-ARCTIC after retraining on L2-ARCTIC data | 0.527 | Sara and Chowdhury, arXiv 2026 |
| Mispronunciation detection ROC-AUC: model before vs after fine-tuning on non-native learners | 0.72 vs 0.85 | Nijat et al., Applied Sciences 2026 |
| Detection F1 on the ERJ corpus: no learner data (zero-shot) vs fine-tuned | 78.44% vs 89.27% | Geng et al., CROTTC-IF, arXiv 2026 |
| Children’s speech (CMU Kids), real errors, AUC: older GOP-DNN-Avg vs segmentation-free GOP-CTC-SF-SD | 0.796 vs 0.914 | Cao et al., arXiv 2025 |
| GOP-CTC-SF-SD on children’s speech: simulated errors vs real errors, AUC | 0.986 vs 0.914 | Cao et al., arXiv 2025 |
| Test takers with speech or hearing impairments, six TOEFL iBT speaking items | SpeechRater scores tended to be higher than human scores | Loukina and Buzick, ETS Research Report 2017 (most recent available data) |
| Sources of bias reported in a modern pronunciation scorer | Audio quality and language family groups | Cai et al., Language Learning 2025 |
Context note: the L2-ARCTIC scoring labels in the first two rows are Azure pseudo-labels with expert review of 10 speakers, so those PCCs measure agreement with another machine as much as with humans. Microsoft separately advises 16 kHz or higher audio and a single speaker for its own service.
Summary: AI Pronunciation Assessment Scoring Accuracy by the Numbers
| Metric | Value | Source |
|---|---|---|
| Pooled machine-human correlation: pronunciation vs all speaking constructs | r = .606 vs r = .654 | Wang and Min, Language Testing 2026 |
| TOEFL Junior Speaking: human vs human compared with human vs machine | r = .89 vs r = .81 | ETS, TOEFL Junior automated scoring report 2015 |
| Three speech-language pathologists: pairwise PCC range | 0.858 to 0.927 | Azad et al., Harf-Speech, arXiv 2026 |
| Three English teachers: strongest pair vs weakest pair | r = 0.901 vs r = 0.583 | Chen and Sun, PLOS One 2025 |
| Versant English Test: Overall vs Pronunciation machine-human correlation | 0.97 vs 0.88 | Pearson, Versant validation summary 2022 (vendor’s own claim) |
| PTE Academic speaking: Pronunciation machine-human correlation, N = 158 | 0.81 | Pearson, MELTA presentation 2018 (vendor’s own claim) |
| speechocean762 phone-level baseline PCC | 0.45 | Zhang et al., speechocean762 paper 2021 |
| GOPT phone-level PCC | 0.612 | Gong et al., GOPT, ICASSP 2022 |
| Highest phone-level PCC cited on speechocean762 | 0.693 | Chao et al., Interspeech 2023, as cited in Cao et al., arXiv 2025 |
| 3MH utterance-level total PCC | 0.811 | Wang et al., GPT-4o pronunciation study, arXiv 2025 |
| Top L2-ARCTIC detection F1 in our sources | 71.77% | Geng et al., CROTTC-IF, arXiv 2026 |
| False rejection vs false acceptance in that system | 3.39% vs 32.76% | Geng et al., CROTTC-IF, arXiv 2026 |
| Utterance-level PCC on speechocean762: Azure vs best GPT-4o setup | 0.782 vs 0.502 | Wang et al., GPT-4o pronunciation study, arXiv 2025 |
| Arabic test: PCC with mean clinician score, Harf-Speech vs Azure | 0.791 vs 0.635 | Azad et al., Harf-Speech, arXiv 2026 |
| SmartSpeech mean score vs human mean score | 92.00 vs 86.41 | Chen and Sun, PLOS One 2025 |
| ROC-AUC before vs after fine-tuning on target learners | 0.72 vs 0.85 | Nijat et al., Applied Sciences 2026 |
Methodology and Sources
We read each study, report or technical document directly and kept each source’s own notation, including r values written without a leading zero. Preprints are labelled arXiv and have not necessarily been peer reviewed.
- Wang and Min, The human-machine correlation in automated speech evaluation: A three-level meta-analysis, Language Testing, 2026.
- Zhang et al., speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment, 2021.
- Gong et al., Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment, ICASSP 2022.
- Chao et al., A Hierarchical Context-aware Modeling Approach for Multi-aspect and Multi-granular Pronunciation Assessment, Interspeech 2023.
- Cao et al., Segmentation-free Goodness of Pronunciation, arXiv 2025, revised February 5, 2026.
- Sara and Chowdhury, Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal, arXiv 2026.
- Wang et al., Exploring the Potential of Large Multimodal Models as Effective Alternatives for Pronunciation Assessment, arXiv 2025.
- Chao and Chen, HMamba paper, arXiv 2025.
- Tu et al., Mispronunciation Detection and Diagnosis Without Model Training, arXiv 2025.
- Geng et al., Beyond Acoustic Sparsity and Linguistic Bias, arXiv 2026.
- Yan et al., An End-to-End Mispronunciation Detection System for L2 English Speech, Interspeech 2020.
- Azad et al., Harf-Speech: A Clinically Aligned Framework for Arabic Phoneme-Level Speech Assessment, arXiv 2026.
- Chen and Sun, Evaluating automated evaluation systems for spoken English proficiency, PLOS One, 2025.
- Won, Assessing the efficacy of word error rate as a proxy for pronunciation quality, Journal of Second Language Pronunciation, 2025.
- Nijat et al., Beyond Native Norms: A Perceptually Grounded and Fair Framework for Automatic Speech Assessment, Applied Sciences, 2026.
- Cai et al., Developing an Automatic Pronunciation Scorer, Language Learning, 2025, with the Duolingo English Test blog summary.
- Pearson, Versant English Test Description and Validation Summary, 2022.
- Pearson, Versant English Speaking and Listening Test validation report, 2024.
- Pearson, MELTA automated scoring presentation, 2018.
- ETS, Automated Scoring for the TOEFL Junior Comprehensive Writing and Speaking Test, 2015.
- Loukina and Buzick, Use of Automated Scoring in Spoken Language Assessments for Test Takers With Speech Impairments, ETS Research Report, 2017.
- Chen et al., Automated Scoring of Nonnative Speech Using the SpeechRater v. 5.0 Engine, ETS Research Report, 2018.
- Microsoft, Characteristics and limitations of Pronunciation Assessment, 2026.
- Dalman and Kang, Listener Background in L2 Speech Evaluation, IntechOpen, 2019.
- Data watch: Most recent available data: ETS 2015 and 2017, Pearson 2018 and 2022, and Dalman and Kang 2019, since we found no newer public equivalents, and ETS does not publish operational TOEFL iBT agreement figures in the documents we could read. Benchmarks are not directly comparable across papers because training data, splits and pseudo-labels differ, and the 0.693 record is what Cao et al. state to the best of their knowledge: we could read the HMamba abstract but not its phone-level table, so a newer phone-level figure may exist. CROTTC-IF is not the top detector on speechocean762, where MuFFIN’s 67.98% leads the same table. Vendor figures come from the vendors’ own validation samples. We found no usable Cohen’s kappa for pronunciation in the sources we read, which favor PCC and ICC, and the Wang and Min and Loukina and Buzick figures come from the published abstracts. Chen and Sun’s sample is 30 students, and Harf-Speech uses 40 samples, so both are small.
Last updated: October 31, 2026. We update this roundup quarterly, and the next refresh is expected when the ISCA Archive posts the next Interspeech proceedings.