Voice Conversion Speaker Similarity and Naturalness Benchmark Statistics (2026): 50+ Data Points on Challenge Results, Objective Metrics, and Evaluation Reliability

Voice conversion similarity and naturalness statistics 2026: top VCC 2020 systems had over 90% of samples judged as the target speaker, yet none hit human naturalness.

In the Voice Conversion Challenge 2020, eight intra-lingual systems were statistically indistinguishable from the real target speaker in similarity tests, with over 90% of their converted samples judged to be the same person (Zhao et al., VCC 2020 overview). Four years earlier, the best VCC 2016 system averaged just 3.0 on a 5-point naturalness scale, and the top systems convinced listeners about 70% of the time. Naturalness has not closed the gap as cleanly: no VCC 2020 system reached human-level naturalness, and in the Singing Voice Conversion Challenge 2025 the top systems scored around 3.7 against around 3.9 for real recordings. This roundup looks at how voice conversion is measured rather than how it is sold; for adoption and market data see our voice cloning statistics, and for text-to-speech listening scores see our synthetic speech naturalness MOS statistics. We aggregated data from the Voice Conversion Challenges of 2016, 2018 and 2020, the Singing Voice Conversion Challenges of 2023 and 2025, peer-reviewed papers from Interspeech, ICLR, AAAI and IEEE JSTSP, and recent studies of speaker-similarity metrics and listening-test bias.

TL;DR

  • 8 VCC 2020 intra-lingual systems were not significantly different from the target speaker in similarity (Zhao et al., VCC 2020 overview 2020).
  • A 2025 zero-shot system scored 3.96 naturalness MOS versus 4.33 for ground truth (Yao et al., StableVC, AAAI 2025).
  • A 2026 codec-space system’s speaker similarity fell from 0.63 in English to 0.49 in Chinese-to-English conversion (Zheng et al., X-VC 2026).
  • Similarity for a self-supervised-feature system fell from 59% to 28% when moving from any-to-one to any-to-any conversion (Huang et al., IEEE JSTSP 2022).
  • 20 of 31 VCC 2020 systems fooled a speaker-verification system more than 90% of the time (Das et al., VCC 2020 objective assessment 2020).
  • Speaker-embedding similarity correlated 0.768 with human ratings on real speech but only 0.512 on VCC 2018 converted speech (Ahn et al., VoxSim, Interspeech 2024).
  • About half of VCC 2018 utterances had a standard deviation above 1 point across their four listener ratings (Lo et al., MOSNet 2019).
  • A system’s MOS can drop by up to around an entire point when it is rated only alongside stronger systems (Cooper and Yamagishi, Interspeech 2023).
  • SVCC 2023’s top singing system matched ground-truth naturalness (around 3.9) but reached 65% similarity versus 80% (Violeta et al., SVCC 2025 analysis 2025).
  • Breathy singing was the hardest style to convert in SVCC 2025, at 37.3% style accuracy (Violeta et al., SVCC 2025 analysis 2025).

1. A Decade of Challenge Results: From 70% to Target-Level Similarity

Voice conversion (VC) changes who a recording sounds like while keeping what is said. The field’s shared yardstick is the Voice Conversion Challenge (VCC), where every team converts the same sentences between the same speakers and one large listening test scores them all. Naturalness is rated on a 5-point mean opinion score (MOS) scale; similarity is judged by asking listeners whether a converted clip and a real clip come from the same speaker.

The trajectory is steep. The best VCC 2016 system averaged 3.0 in naturalness, while by VCC 2020 eight systems were statistically tied with the real target speaker in similarity (Zhao et al., VCC 2020 overview). The order of progress is the interesting part: listeners stopped separating converted clips from the target in clean, same-language English before any system reached human-level naturalness.

MetricValueSource
Research groups that submitted to VCC 201617Toda et al., The Voice Conversion Challenge 2016, Interspeech 2016
Listeners in the VCC 2016 test, run in a controlled environment200Toda et al., The Voice Conversion Challenge 2016, Interspeech 2016
VCC 2016 top systems, share of samples judged the same speaker as the targetaround 70%Wester et al., Analysis of the VCC 2016 Evaluation Results, Interspeech 2016
VCC 2016 best system, average naturalness on the 5-point scale3.0Zhao et al., VCC 2020 overview 2020
VCC 2018 best system (N10): naturalness / share judged the same as targetabout 4.1 / about 80%Lorenzo-Trueba et al., VCC 2018 paper
VCC 2020 same-language systems not significantly different from the target speaker in similarity8Zhao et al., VCC 2020 overview 2020

Context note: the VCC 2018 winner relied on a content extractor trained on hundreds of hours of external speech plus a speaker-dependent WaveNet vocoder (Lorenzo-Trueba et al., VCC 2018). Only 3 VCC 2018 teams used the external-data option. Most recent available data for the speech challenges: VCC 2020, as the fourth edition in 2023 shifted to singing and was renamed (see section 6).

2. The Naturalness Gap: How Far the Best Systems Sit From Real Speech

A “same speaker” verdict does not mean the clip sounds human. Every challenge has kept natural recordings inside the test as anchors, and those anchors show a persistent gap. A 2022 comparison lists USTC-2020, the top system of the VCC 2020 intra-lingual task, at 4.41 naturalness MOS against 4.57 for the real target recordings, a gap of 0.16 points (4.57 minus 4.41) (Huang et al., IEEE JSTSP 2022). That system also reached 82% on the similarity test.

The gap is wider for zero-shot systems, which convert to a voice never seen in training from a short reference clip. A 2025 AAAI paper measured 3.96 MOS for its system against 4.33 for ground truth on VCTK speakers, a 0.37-point gap, even though an automatic MOS predictor put the two only 0.12 apart (Yao et al., StableVC, AAAI 2025). In this test the automatic predictor was the kinder judge, one reason to read such scores next to a listening test.

MetricValueSource
Natural recordings in VCC 2016: naturalness / share judged the same speaker4.5 / 90%Violeta et al., SVCC 2025 analysis 2025
VCC 2016 systems rated significantly less natural than the real source and target speakersall systemsWester et al., Analysis of the VCC 2016 Evaluation Results, Interspeech 2016
VCC 2018 best system vs real target speech (Wilcoxon signed-rank, Bonferroni, α = 0.01)significantly different in both naturalness and similarityLorenzo-Trueba et al., VCC 2018 paper
Top VCC 2020 system (USTC-2020) vs target recordings, same-language naturalness MOS4.41 vs 4.57Huang et al., IEEE JSTSP 2022
Best self-supervised-feature system vs plain mel-spectrogram baseline, naturalness MOS3.59 vs 2.61Huang et al., IEEE JSTSP 2022
Zero-shot system vs ground truth, listener naturalness MOS (VCTK speakers)3.96 ±0.04 vs 4.33 ±0.04Yao et al., StableVC, AAAI 2025
Same comparison using UTMOS, an automatic naturalness predictor4.12 vs 4.24Yao et al., StableVC, AAAI 2025
Best of six published zero-shot baselines, listener naturalness MOS3.67 ±0.07Yao et al., StableVC, AAAI 2025

Outlier note: in VCC 2020, Japanese listeners rated the same-language naturalness of leading system T10 as comparable to the target speaker, while English listeners did not (Zhao et al., VCC 2020 overview). A parity verdict can therefore depend on who is listening, so the listener pool belongs in any reported result.

3. Intelligibility and Spectral Metrics: What WER and MCD Can and Cannot Tell You

Objective metrics are cheap and repeatable, so papers lean on them. Two families recur: intelligibility, measured as the word error rate (WER) or character error rate (CER) of an automatic speech recognizer transcribing the converted audio, and spectral distance, usually mel-cepstral distortion (MCD), which measures how far the converted spectrum sits from the target’s. Lower is better for both.

The VCC data shows WER tracks naturalness reasonably well and similarity poorly. In VCC 2018, WER correlated at -0.7127 with naturalness on the non-parallel task but only -0.2272 with speaker similarity (Lorenzo-Trueba et al., VCC 2018). MCD looks even weaker as a naturalness proxy: in the Singing Voice Conversion Challenge 2023 it did not correlate significantly with in-domain naturalness ratings from either listener group (Huang et al., SVCC 2023). Recognizers also have their own failure modes, covered in our speech recognition hallucination statistics, so a low WER says the words survived, not that the voice sounds right.

MetricValueSource
ASR word error rate on natural source speech, VCC 20186.40%Lorenzo-Trueba et al., VCC 2018 paper
Lowest / highest WER among converted VCC 2018 Hub systems8.60% / 35.60%Lorenzo-Trueba et al., VCC 2018 paper
Correlation of WER with naturalness MOS / with similarity, VCC 2018 parallel-data Hub task and non-parallel Spoke taskHub -0.6587 / -0.3218; Spoke -0.7127 / -0.2272Lorenzo-Trueba et al., VCC 2018 paper
VCC 2020 same-language teams with WER below 5% / above 50%7 / 7Das et al., VCC 2020 objective assessment 2020
MCD, plain mel-spectrogram baseline vs top VCC 2020 system (VCC 2020 data)8.47 vs 6.98Huang et al., IEEE JSTSP 2022
Spearman correlation of MCD with in-domain naturalness MOS, SVCC 2023 (Japanese / English listeners)-0.28 / -0.24 (not significant)Huang et al., SVCC 2023 paper
Spearman correlation of Whisper-based CER with in-domain naturalness MOS, SVCC 2023 (Japanese listeners)-0.80 (p < 0.01)Huang et al., SVCC 2023 paper

Context note: the spread in VCC 2020 is wide. The worst same-language entry posted a 96.17% WER, meaning almost nothing survived conversion for the recognizer, while seven teams stayed below 5% (Das et al., VCC 2020 objective assessment).

A standard automatic similarity score is speaker-embedding cosine similarity (often called SECS or SIM): both clips are passed through a speaker-verification network and the angle between the resulting vectors is compared. On challenge data it agrees well with listeners at the system level. In VCC 2020, the speaker-verification error rate correlated at 0.89 and 0.90 with listener similarity in the two tasks (Das et al., VCC 2020 objective assessment), and 20 of 31 same-language systems were accepted as the target by the verifier more than 90% of the time. The “false acceptance” rate here is the share of converted clips the verifier wrongly accepts as the real target speaker, so higher means a more convincing conversion.

Per-utterance agreement is another story. A pre-trained ECAPA-TDNN embedding reached 0.768 linear correlation with human ratings on real voice pairs but dropped to 0.512 on VCC 2018 converted speech (Ahn et al., VoxSim, Interspeech 2024). A 2025 study from Ubisoft La Forge and Stellenbosch University found that verification models separate the same speaker’s short and long clips well above chance, and that noise and equalization shift scores, so a SIM number partly measures recording conditions (Carbonneau et al. 2025). Anyone comparing voice cloning software on published SIM figures should check that the same embedding model, test set and clip lengths were used.

MetricValueSource
VCC 2020 same-language systems accepted as the target by a speaker verifier more than 90% of the time20 of 31Das et al., VCC 2020 objective assessment 2020
VCC 2020 cross-lingual systems above the same 90% false-acceptance level10Das et al., VCC 2020 objective assessment 2020
Pearson correlation of verifier error rate with listener similarity, VCC 2020 Task 1 / Task 20.89 / 0.90Das et al., VCC 2020 objective assessment 2020
Pearson correlation of embedding cosine distance with listener similarity, VCC 2020 Task 1 / Task 20.85 / 0.82Das et al., VCC 2020 objective assessment 2020
Pre-trained ECAPA-TDNN speaker recognizer vs human similarity, real speech pairs (linear correlation)0.768Ahn et al., VoxSim, Interspeech 2024
Same pre-trained model on VCC 2018 converted speech (linear correlation)0.512Ahn et al., VoxSim, Interspeech 2024
Embedding similarity vs human voice-similarity ratings in a zero-shot synthesis study (utterance level)0.55Li et al., DMOSpeech 2025
Verifier equal error rate when one speaker’s own short clips are compared with their long clips (ECAPA-TDNN; 0.50 means indistinguishable)0.34 ±0.04Carbonneau et al. 2025

Outlier note: adding noise at a 20 dB signal-to-noise ratio pushed the same ECAPA-TDNN equal error rate for one speaker’s clean versus noisy clips to 0.34, and at 0 dB to 0.05 (Carbonneau et al. 2025). In plain words, the verifier starts treating a noisy copy of a voice as a different person.

5. In-Domain vs Cross-Lingual and Any-to-Any: Where Scores Fall Apart

Benchmark numbers are usually reported for the friendliest setting: same language, studio audio, and a target speaker the model has trained on (any-to-one conversion). Change any of those and scores drop. Leading VCC 2020 system T10 lost 0.75 MOS points in naturalness when moving from same-language to cross-lingual conversion, and in that task fewer than 80% of samples were judged to be the target speaker (Zhao et al., VCC 2020 overview). Listeners also gave lower similarity scores when the reference clip was in the speaker’s second language rather than English.

The steepest fall comes from unseen target speakers (any-to-any conversion). In the IEEE JSTSP comparison, a self-supervised-feature system kept its naturalness almost unchanged but its similarity halved (Huang et al., IEEE JSTSP 2022). Recent zero-shot systems still show the gap: on accented and emotional speech, ICLR 2025 authors measured 0.762 speaker similarity for ground-truth recordings versus 0.517 for the best converted output (Zhang et al., Vevo, ICLR 2025).

MetricValueSource
USTC-2020 share judged the same as target, same-language vs cross-lingual82% vs 43%Huang et al., IEEE JSTSP 2022
Self-supervised-feature system, known target speaker (any-to-one) vs unseen target speaker (any-to-any): similarity / naturalness MOS59% vs 28% / 3.59 vs 3.58Huang et al., IEEE JSTSP 2022
VCC 2018 Hub systems other than the winner, naturalness MOS for same-gender vs cross-gender pairs3.08 vs 2.78Lorenzo-Trueba et al., VCC 2018 paper
Codec-space zero-shot system speaker similarity, same-language (English / Chinese) vs cross-lingual (English to Chinese / Chinese to English)0.63 / 0.73 vs 0.52 / 0.49Zheng et al., X-VC 2026
Speaker similarity on accented and emotional speech, real recordings vs best converted output0.762 vs 0.517Zhang et al., Vevo, ICLR 2025

Context note: the 2026 codec-space paper evaluated cross-lingual pairs on a different subset (the first 1,000 utterances per language), so its same-language and cross-lingual numbers are indicative rather than a controlled comparison (Zheng et al., X-VC 2026). Its streaming version ran at 240 ms model latency.

6. Singing Voice Conversion: Naturalness Parity, Identity Gap

Singing stresses voice conversion in ways speech does not: wider pitch range, held notes, and techniques such as vibrato. The Singing Voice Conversion Challenge (SVCC) took over the challenge format in 2023. The top SVCC 2023 system matched ground-truth naturalness at around 3.9, yet reached only 65% similarity against 80% for the real singer (Violeta et al., SVCC 2025 analysis). That is the reverse of the speech story, where identity was matched first.

SVCC 2025 added singing-style conversion, and no system reached human-level naturalness. Style was the hard part: breathy, glissando (sliding between notes) and vibrato were the least accurately reproduced techniques. The authors also questioned whether the standard similarity test is adequate for judging style. Singer identity is hard to score too: even ground-truth samples reached only around 70% on that test.

MetricValueSource
SVCC 2023 submissions plus baselines24 + 2Huang et al., SVCC 2023 paper
SVCC 2023 listeners, English / Japanese40 / 319Huang et al., SVCC 2023 paper
SVCC 2023 teams scoring above 3.0 naturalness, in-domain vs cross-domain task14 vs 8Huang et al., SVCC 2023 paper
SVCC 2023 top system vs ground truth: naturalness / similarityaround 3.9 / 65% vs around 3.9 / 80%Violeta et al., SVCC 2025 analysis 2025
SVCC 2025 top systems vs ground truth: naturalness / style similarityaround 3.7 / 70% vs around 3.9 / 90%Violeta et al., SVCC 2025 analysis 2025
SVCC 2025 style accuracy for breathy / glissando / vibrato singing37.3% / 42.6% / 43.9%Violeta et al., SVCC 2025 analysis 2025
Maximum Spearman correlation between any objective metric and subjective scores, SVCC 2025about 0.8Violeta et al., SVCC 2025 analysis 2025

Context note: in SVCC 2023 the target-to-top-system similarity gap was around 0.4 points on the 4-point scale in both tasks (Huang et al., SVCC 2023), and speaker-embedding distance was the objective metric most correlated with similarity (-0.83 Spearman for Japanese listeners, in-domain task).

7. Evaluation Reliability: Listener Counts, Variance and Bias

Every number above rests on human ratings, and those ratings are noisy. For about half of the VCC 2018 utterances, the standard deviation across their four listener ratings was greater than 1 point on the 5-point scale (Lo et al., MOSNet). Averaging fixes much of that at the system level, which is why automatic predictors look strong there (0.957 correlation) and weak per utterance (0.642).

Test design moves scores too. Cooper and Yamagishi showed that rating only the strongest systems together can lower a system’s MOS by up to around an entire point, because listeners spread their scores across the whole scale whatever the quality range (Cooper and Yamagishi, Interspeech 2023). A survey of 80 Interspeech 2022 speech-synthesis papers found that 51 gave no listener instructions (Chiang et al. 2023), and in the same study UK crowd workers rated one system 2.64 while US workers rated it 3.73.

MetricValueSource
VCC 2018 unique listeners / evaluated utterances267 / 113,168Lorenzo-Trueba et al., VCC 2018 paper
VCC 2018 utterances whose four ratings had a standard deviation above 1about halfLo et al., MOSNet 2019
Automatic MOS predictor vs human MOS on VCC 2018, utterance level vs system level (linear correlation)0.642 vs 0.957Lo et al., MOSNet 2019
Cost of the two VCC 2020 listening tests (English and Japanese listeners)738,128 yen (approx. 6,900 USD)Zhao et al., VCC 2020 overview 2020
Agreement between Japanese and English listener scores, VCC 2020correlations of 0.965 to 0.984Zhao et al., VCC 2020 overview 2020
Largest MOS drop for one system when rated only alongside the top 6% of systems1.28 pointsCooper and Yamagishi, Interspeech 2023
Interspeech 2022 speech-synthesis papers (of 80) giving no listener instructions51Chiang et al., Interspeech 2023
Crowd responses invalidated in a 501-worker listening testabout 30%Li et al., DMOSpeech 2025

Context note: individual papers run much smaller tests than the challenges. A 2023 Interspeech paper on nearest-neighbour conversion collected 2,144 naturalness ratings from 61 listeners and 1,636 similarity ratings over 208 listeners (Baas et al., Interspeech 2023). Its reported intervals were around ±0.1 points, so a gap smaller than that between two close systems says little.

Summary: Voice Conversion Speaker Similarity and Naturalness Benchmark by the Numbers

MetricValueSource
VCC 2016 best system naturalness / top similarity3.0 / around 70%Zhao et al., VCC 2020 overview 2020; Wester et al., Interspeech 2016
VCC 2018 best system naturalness / similarityabout 4.1 / about 80%Lorenzo-Trueba et al., VCC 2018 paper
Natural recordings in VCC 2016: naturalness / similarity4.5 / 90%Violeta et al., SVCC 2025 analysis 2025
USTC-2020 vs target recordings, naturalness MOS4.41 vs 4.57Huang et al., IEEE JSTSP 2022
Zero-shot system vs ground truth, listener naturalness MOS3.96 vs 4.33Yao et al., StableVC, AAAI 2025
MCD correlation with in-domain naturalness, SVCC 2023-0.28 / -0.24 (not significant)Huang et al., SVCC 2023 paper
VCC 2020 same-language systems fooling a speaker verifier more than 90% of the time20 of 31Das et al., VCC 2020 objective assessment 2020
Embedding similarity vs human ratings, real speech vs converted speech0.768 vs 0.512Ahn et al., VoxSim, Interspeech 2024
Verifier error rate, same speaker short vs long clips (0.50 = indistinguishable)0.34Carbonneau et al. 2025
Similarity, any-to-one vs any-to-any conversion59% vs 28%Huang et al., IEEE JSTSP 2022
Similarity, same-language vs Chinese-to-English conversion (2026 system)0.63 vs 0.49Zheng et al., X-VC 2026
Speaker similarity, real recordings vs best converted output on accented and emotional speech0.762 vs 0.517Zhang et al., Vevo, ICLR 2025
SVCC 2023 top system vs ground truth similarity65% vs 80%Violeta et al., SVCC 2025 analysis 2025
VCC 2018 utterances with rating standard deviation above 1about halfLo et al., MOSNet 2019
Largest MOS drop from range-equalizing bias1.28 pointsCooper and Yamagishi, Interspeech 2023

Methodology and Sources

Last updated: October 31, 2026. We update this roundup quarterly, and the next refresh is expected when the next Voice Conversion Challenge or Singing Voice Conversion Challenge listening-test results are published.

Try VoxBooster — 3-day free trial.

Real-time voice cloning, soundboard, and effects — wherever you already talk.

  • No credit card
  • ~30ms latency
  • Discord · Teams · OBS
Try free for 3 days