Zero-Shot Voice Cloning Reference Audio Length Statistics (2026): 45+ Data Points on Prompt Length, Speaker Similarity, and Cloning Security

Zero-shot voice cloning statistics 2026: research systems clone from 3-second prompts, and listeners judged AI clones to be the real speaker about 80% of the time.

Three seconds of speech from a stranger is a standard test condition for zero-shot voice cloning research, the setup behind both VALL-E and Voicebox. On that test, Voicebox raised speaker similarity to 0.681 from VALL-E’s 0.580 on LibriSpeech, and NaturalSpeech 3 later reported 0.67 against 0.68 for the real recordings on its own run of the same dataset. Listeners are not much of a backstop: in a 2025 Scientific Reports study, people judged AI clones to be the real speaker about 80% of the time. For the commercial and market picture, see our broader voice cloning statistics; this roundup stays on one question, how much reference audio cloning needs and what each extra second buys. We aggregated data from Microsoft Research, Meta, ByteDance, the Barcelona Supercomputing Center, the U.S. Federal Trade Commission, Consumer Reports, McAfee, Starling Bank, and peer-reviewed and arXiv papers from ICML, NeurIPS, ICLR and Scientific Reports.

TL;DR

  • Voicebox was tested on 3.0 seconds of prompt audio and matched VALL-E’s speaker similarity with roughly two thirds of the input audio (Meta, Voicebox paper, NeurIPS 2023).
  • On VCTK, VALL-E 2 similarity rose from 0.447 at 3 seconds to 0.558 at 10 seconds, while word error rate rose from 0.9% to 3.3% (Microsoft Research, VALL-E 2 paper 2024).
  • Cutting the average prompt from 7.7 to 5.5 seconds dropped F5-TTS similarity from 0.35 to 0.24 on in-the-wild speech (Barcelona Supercomputing Center, WildSpoof submission 2026).
  • XTTS-v2 reached high-fidelity cloning with 6 to 10 seconds of reference audio, with diminishing returns beyond 20 seconds (Zhu, University of Groningen thesis 2025).
  • A zero-shot model with a 300-second prompt scored 0.932 similarity vs 0.934 for a fine-tuned baseline on 300 seconds (Jiang et al., Mega-TTS 2, ICLR 2024).
  • Fine-tuning on 20 seconds of Portuguese speech raised human similarity ratings from 2.77 to 4.43 (Casanova et al., YourTTS, ICML 2022).
  • Listeners judged AI voices to be the same person as the real speaker about 80% of the time (Barrington, Cooper and Farid, Scientific Reports 2025).
  • McAfee researchers reported an estimated 85% voice match from three to four seconds of audio, a vendor measurement (McAfee, Beware the Artificial Impostor 2023).
  • 28% of UK adults say they were targeted by an AI voice cloning scam in the past year (Starling Bank and Mortar Research survey 2024).
  • Four of six voice cloning products failed to take basic steps to stop unauthorized cloning (Consumer Reports, voice cloning assessment 2025).

1. The 3-Second Test: How Much Audio Zero-Shot Systems Are Given

“Zero-shot” means the system was never trained on the target speaker: it hears a short reference clip, called the “prompt”, at generation time and imitates it on the spot. VALL-E and Voicebox were both evaluated on a prompt of just 3 seconds, and that choice says more about ambition than about limits. These papers do not ask whether a voice can be copied from a few seconds. They compete on how well.

Product requirements run longer. OpenAI described its Voice Engine as working from “a single 15-second audio sample” (OpenAI post on synthetic voices 2024), and ElevenLabs documentation recommends one to two minutes. Those are product settings, not measured thresholds.

MetricValueSource
Prompt length and training data for VALL-E on unseen LibriSpeech speakers3 seconds; 60K hours of English speech, over 7,000 speakersMicrosoft Research, VALL-E paper 2023
Prompt audio used for Voicebox English zero-shot tests3.0 secondsMeta, Voicebox paper (NeurIPS 2023)
Reference clip length used to evaluate XTTSOne segment between 3 and 8 secondsCasanova et al., XTTS paper (INTERSPEECH 2024)
Reference utterance length range in the Seed-TTS test set3 to 20 secondsByteDance, Seed-TTS paper 2024
Audio sample OpenAI Voice Engine works fromA single 15-second sampleOpenAI, synthetic voices post 2024
Minimum and recommended audio for ElevenLabs instant cloningAt least 10 seconds in total, 1 to 2 minutes recommended (vendor’s own guidance)ElevenLabs, Instant Voice Cloning documentation 2026

If you are comparing desktop voice cloning software, sections 2 and 3 show what those seconds change.

2. How Speaker Similarity Scales From 3 to 10 Seconds

Speaker similarity (SIM, or SECS) is the cosine similarity between two speaker “fingerprints” computed by a speaker-recognition network, where higher means closer. Two real recordings of the same person score well below 1.0, so the real-speech figures below are the reference point to read against. Going from a 3-second to a 10-second prompt lifted VALL-E 2’s similarity on the VCTK dataset from 0.447 to 0.558, a gain of 0.111 (0.558 minus 0.447), which is the clearest published curve for this question (VALL-E 2 paper 2024).

The curve bends at different points in different systems. Meta’s Voicebox team reported that similarity “grows quickly and flattens with longer audio prompts” (Voicebox paper), and a 2025 thesis on XTTS-v2 put the plateau at around 10 to 20 seconds (Zhu, University of Groningen). Across these three papers, the first ten seconds do most of the work, though VALL-E 2 was still improving at ten.

MetricValueSource
VALL-E similarity on VCTK (108 speakers) with 3, 5 and 10-second prompts0.382, 0.423, 0.484 (real recordings: 0.546, 0.591, 0.620)Microsoft Research, VALL-E paper 2023
VALL-E 2 (group size 1) similarity on VCTK with 3, 5 and 10-second promptsSingle sample: 0.447, 0.487, 0.558; best of five samples: 0.508, 0.552, 0.620 (real recordings: 0.623, 0.679, 0.709)Microsoft Research, VALL-E 2 paper 2024
Share of the input audio Voicebox needs to equal VALL-E speaker similarityRoughly two thirdsMeta, Voicebox paper (NeurIPS 2023)
F5-TTS similarity with long (7.7 s average) vs short (5.5 s average) prompts on in-the-wild speech0.35 vs 0.24Barcelona Supercomputing Center, WildSpoof TTS submission 2026
StyleTTS2 similarity with long (7.7 s average) vs short (5.5 s average) prompts0.19 vs 0.14Barcelona Supercomputing Center, WildSpoof TTS submission 2026
Reference length at which XTTS-v2 reached high-fidelity cloning (1, 3, 6, 10, 20 and 40 seconds tested)6 to 10 seconds, with diminishing returns beyond 20 secondsZhu, University of Groningen master’s thesis 2025

Outlier note: the Barcelona Supercomputing Center submission is the only 2026 result here and uses the TTS In the Wild dataset. A prompt 2.2 seconds shorter on average (7.7 minus 5.5) cost F5-TTS 0.11 of similarity (0.35 minus 0.24). That is the same size of loss VALL-E 2 showed across a 7-second gap on VCTK, a hint that short prompts may hurt more on everyday audio, though the two tests are not directly comparable.

3. Intelligibility Does Not Always Improve With Longer Prompts

Word error rate (WER) measures intelligibility: a speech recognizer transcribes the cloned audio and the share of wrong words is counted, so lower is better. VALL-E 2’s WER on VCTK rose from 0.9% with a 3-second prompt to 3.3% with a 10-second prompt, the opposite direction from its similarity gain. Human raters agreed: in the same test, similarity ratings fell as prompts lengthened, while ratings for the real recordings rose.

Nothing guarantees that more prompt audio helps. Mega-TTS 2, designed for multi-sentence prompts, kept improving all the way to 300 seconds, while the VALL-E run measured in the same paper broke down at 20 seconds (Mega-TTS 2 paper). Very short prompts carry their own intelligibility risk: StyleTTS2 was the system that got worse when its prompt was cut.

MetricValueSource
VALL-E 2 (group size 1) on VCTK with 3, 5 and 10-second prompts: word error rate, and human similarity rating (SMOS, higher is better)WER 0.9%, 1.9%, 3.3%; SMOS 4.42±0.15, 4.28±0.16, 3.95±0.10 (real recordings 4.47±0.13, 4.53±0.14, 4.74±0.17)Microsoft Research, VALL-E 2 paper 2024
VALL-E on LibriSpeech with 3, 10 and 20-second prompts, as measured by the Mega-TTS 2 authorsWER 5.83%, 5.54%, 8.77%; similarity 0.885, 0.893, 0.805Jiang et al., Mega-TTS 2 (ICLR 2024)
Mega-TTS 2 word error rate with 3, 10, 60 and 300-second prompts2.46%, 2.28%, 2.24%, 2.23%Jiang et al., Mega-TTS 2 (ICLR 2024)
StyleTTS2 word error rate with long (7.7 s) vs short (5.5 s) prompts0.21 vs 0.49 (F5-TTS unchanged at 0.07)Barcelona Supercomputing Center, WildSpoof TTS submission 2026
Voicebox trend when the prompt and target language differ, as prompts lengthenSimilarity improves; WER rises for most language directionsMeta, Voicebox paper (NeurIPS 2023)

Context: the VALL-E 2 trend is not perfectly smooth. Its group size 2 setting recorded WER of 1.5%, 0.9% and 2.3% at the same three lengths. Voicebox’s authors attribute the rising cross-language WER to English making up over 90% of the training data.

4. How Close Short-Prompt Clones Get to the Real Speaker

On benchmark speech, the best published systems now sit next to the real recordings. NaturalSpeech 3 scored a speaker similarity of 0.67 against 0.68 for the ground-truth recordings, and its human similarity rating (4.01) was higher than the real recordings’ 3.85 on the same LibriSpeech test (Microsoft Research, NaturalSpeech 3 paper 2024). That is a long way from the 0.337 YourTTS scored in the first row below.

The real-speech scores are the tell that these are lab conditions: they run from 0.68 to 0.8952 across papers, so each system has to be read against its own ground truth, as in the table. The F5-TTS paper lands within 0.03 of real speech on its own test (0.69 minus 0.66).

MetricValueSource
LibriSpeech, 3-second prompt: YourTTS vs VALL-E vs VoiceboxSimilarity 0.337 vs 0.580 vs 0.681 (real speech 0.754); WER 7.7% vs 5.9% vs 1.9% (real speech 2.2%)Meta, Voicebox paper (NeurIPS 2023)
NaturalSpeech 3 vs real recordings on LibriSpeech: similarity and WER0.67 and 1.81% vs 0.68 and 1.94%Microsoft Research, NaturalSpeech 3 paper 2024
Seed-TTS English similarity vs human recordings, and comparative rating (CMOS) vs human0.762 vs 0.730; CMOS -0.07ByteDance, Seed-TTS paper 2024
F5-TTS vs real speech on 1,127 LibriSpeech-PC samples of 4 to 10 seconds: similarity and WER0.66 and 2.42% vs 0.69 and 2.23%Chen et al., F5-TTS paper 2024
MiniMax-Speech English similarity: speaker embedding alone vs prompt with paired text0.692 vs 0.738 (ground truth 0.730)Zhang et al., MiniMax-Speech paper 2025
XTTS English similarity vs real speech0.6423 vs 0.8952Casanova et al., XTTS paper (INTERSPEECH 2024)

Outlier note: Seed-TTS and the paired-text MiniMax-Speech setting score above their real recordings. The Seed-TTS authors explain that ground-truth and reference clips of one speaker can still differ in speaking style and background environment, so a model that follows the reference closely can out-score a second real recording. It is not evidence of a better-than-human clone.

5. Zero-Shot vs Fine-Tuned Cloning: The Gap Measured in Seconds and Minutes

Fine-tuning means updating a model’s weights on the target speaker’s audio, the older and slower route to a custom voice. With 300 seconds of the target voice, a zero-shot prompt reached 0.932 similarity against 0.934 for a fine-tuned baseline, with fewer word errors (2.23% vs 3.11%) (Jiang et al., Mega-TTS 2, ICLR 2024). The caveat matters: both systems are the authors’ own, and the baseline is a FastSpeech 2 build fine-tuned for 2,000 steps, not the strongest fine-tuned system available.

Fine-tuning still earns its cost when a voice sits far from the training data. YourTTS was trained on a single Portuguese speaker, and fine-tuning on 20 to 31 seconds lifted listeners’ similarity ratings for new Portuguese voices, though overall quality ratings dipped (YourTTS paper). XTTS gained 0.1314 of similarity (0.7166 minus 0.5852) from about 10 minutes per speaker in cross-language cloning.

MetricValueSource
Similarity of the fine-tuned baseline with 10, 60 and 300 seconds of target audio0.895, 0.923, 0.934Jiang et al., Mega-TTS 2 (ICLR 2024)
Similarity of zero-shot Mega-TTS 2 with 3, 10, 60 and 300-second prompts0.898, 0.905, 0.926, 0.932Jiang et al., Mega-TTS 2 (ICLR 2024)
YourTTS, Portuguese female speaker with 20 seconds of speech, zero-shot vs fine-tunedSimilarity 0.743 vs 0.930; Sim-MOS 2.77±0.15 vs 4.43±0.06; MOS 3.59±0.13 vs 3.48±0.13Casanova et al., YourTTS (ICML 2022)
YourTTS, English male speaker with 61 seconds of speech, zero-shot vs fine-tunedSimilarity 0.851 vs 0.880 (real speech 0.875)Casanova et al., YourTTS (ICML 2022)
XTTS cross-language similarity before vs after fine-tuning on about 10 minutes per speaker0.5852 vs 0.7166Casanova et al., XTTS paper (INTERSPEECH 2024)
Seed-TTS with a 20-second prompt vs fine-tuned on 1 to 10 hours per speaker: similarity and WER0.779 and 3.15 vs 0.789 and 2.83 (CMOS +0.37 for fine-tuned)ByteDance, Seed-TTS paper 2024
Audio recommended for ElevenLabs Professional Voice Cloning (its fine-tuned tier)30 to 180 minutes (vendor’s own guidance)ElevenLabs, voice cloning documentation 2026

Most recent available data for the YourTTS fine-tuning test: Casanova et al., ICML 2022. It covers only 4 speakers, so treat it as an illustration of the language effect rather than a population estimate.

6. The Security Side: Short Prompts and Fraud

The research numbers have a direct consumer translation: public video is prompt material. Listeners judged an AI voice to be the same identity as its real counterpart about 80% of the time, and identified AI voices only about 60% of the time, across 604 participants in a study whose real clips averaged 5.37 seconds (Barrington, Cooper and Farid, Scientific Reports 2025). The clones came from a commercial instant-cloning service fed the first two recorded prompts of each of 220 speakers.

The “three seconds” line in fraud warnings rests on company statements. McAfee’s figure is its own benchmark and, for that source, the most recent available data (McAfee, 2023). Starling Bank repeated the claim in 2024 alongside its survey (Starling Bank, Safe Phrases campaign 2024). Our post on voice cloning family fraud covers the scams themselves.

MetricValueSource
Share of the time listeners judged an AI voice to be the same person as the real speakerAbout 80%Barrington, Cooper and Farid, Scientific Reports 2025
Share of the time listeners correctly identified a voice as AI generatedAbout 60%Barrington, Cooper and Farid, Scientific Reports 2025
Audio McAfee researchers needed for an estimated 85% voice match with a free toolThree to four seconds (about 95% with more effort; vendor’s own benchmark)McAfee, Beware the Artificial Impostor 2023
Adults who share their voice online at least once a week53% (7,054 adults, 7 countries)McAfee, Beware the Artificial Impostor 2023
Adults not confident they could tell a real voice from a cloned one70%McAfee, Beware the Artificial Impostor 2023
UK adults who say they were targeted by an AI voice cloning scam in the past year28% (3,010 adults)Starling Bank and Mortar Research survey 2024
UK adults who would send the money requested even if the call seemed strange8%Starling Bank and Mortar Research survey 2024
Voice cloning products assessed that failed to take basic steps to stop unauthorized cloning4 of 6Consumer Reports, voice cloning assessment 2025

Context: the Starling 28% measures what people say happened to them, not confirmed incidents. McAfee’s 85% and 95% are the company’s own estimates, and we found no independent replication. The Consumer Reports assessment covered Descript, ElevenLabs, Lovo, PlayHT, Resemble AI and Speechify.

7. Safeguards, Detection and Release Decisions

Release decisions split the field. Microsoft says it has “no plans to incorporate VALL-E 2 into a product or expand access to the public” (VALL-E 2 project page), and OpenAI chose to “preview but not widely release” Voice Engine while urging banks to phase out voice-based authentication. Meanwhile F5-TTS’s authors call for watermarks and detection models for the misuse risks they name. The FTC treated the problem as open enough to fund a challenge (FTC, Voice Cloning Challenge winners 2024).

Detection results depend heavily on what the fake is compared against. Meta’s classifier separated Voicebox output from original audio perfectly, but scored 0.704 against real speech that had been passed through a vocoder (software that rebuilds audio from its spectrum) at the lowest masking level. For a deeper look at detector performance, see our audio deepfake detection accuracy statistics.

MetricValueSource
Prize money split by the three FTC Voice Cloning Challenge winners$35,000 (announced April 8, 2024)U.S. Federal Trade Commission, Voice Cloning Challenge 2024
Audio chunk size checked by the FTC-recognized Pindrop real-time clone detectorTwo-second chunksU.S. Federal Trade Commission, Voice Cloning Challenge 2024
Voicebox detection classifier accuracy vs original audio, and vs vocoded real audio at 30%, 50% and 90% masking of the audio1.000 vs original; 0.704, 0.809, 0.907 vs vocodedMeta, Voicebox paper (NeurIPS 2023)
When OpenAI first developed Voice Engine vs when it published its limited previewLate 2022 vs March 29, 2024OpenAI, synthetic voices post 2024
Teams in the ASVspoof 5 challenge, and the hardest attack in the Track 1 open condition53 teams; pre-trained zero-shot YourTTS (minDCF 0.33)Wang et al., ASVspoof 5 evaluation paper 2026

Context: minDCF is a detection cost score where higher means detectors struggled more. A pre-trained YourTTS model topping the ASVspoof 5 open-condition difficulty list suggests zero-shot attacks do not need a frontier system to cause trouble, though the paper does not break this result down by prompt length.

Summary: Zero-Shot Voice Cloning Reference Audio Length by the Numbers

MetricValueSource
Audio sample OpenAI Voice Engine works from15 secondsOpenAI, synthetic voices post 2024
VALL-E 2 on VCTK, 3 vs 10-second prompt (group size 1): similarity, and WER0.447 vs 0.558; 0.9% vs 3.3%Microsoft Research, VALL-E 2 paper 2024
Share of the input audio Voicebox needs to equal VALL-E similarityRoughly two thirdsMeta, Voicebox paper (NeurIPS 2023)
F5-TTS similarity, long (7.7 s) vs short (5.5 s) prompts on in-the-wild speech0.35 vs 0.24Barcelona Supercomputing Center, WildSpoof TTS submission 2026
StyleTTS2 word error rate, long (7.7 s) vs short (5.5 s) prompts0.21 vs 0.49Barcelona Supercomputing Center, WildSpoof TTS submission 2026
Reference length for high-fidelity XTTS-v2 cloning6 to 10 secondsZhu, University of Groningen thesis 2025
NaturalSpeech 3 similarity vs real recordings0.67 vs 0.68Microsoft Research, NaturalSpeech 3 paper 2024
F5-TTS word error rate vs real speech2.42% vs 2.23%Chen et al., F5-TTS paper 2024
Zero-shot Mega-TTS 2 vs fine-tuned baseline similarity with 300 seconds of audio0.932 vs 0.934Jiang et al., Mega-TTS 2 (ICLR 2024)
YourTTS listener similarity rating, zero-shot vs fine-tuned on 20 seconds2.77 vs 4.43Casanova et al., YourTTS (ICML 2022)
XTTS cross-language similarity after about 10 minutes of fine-tuning0.5852 to 0.7166Casanova et al., XTTS paper (INTERSPEECH 2024)
Listeners judging an AI clone to be the real speakerAbout 80%Barrington, Cooper and Farid, Scientific Reports 2025
Audio for an estimated 85% match (vendor benchmark)Three to four secondsMcAfee, Beware the Artificial Impostor 2023
UK adults who say they were targeted by a voice cloning scam28%Starling Bank and Mortar Research survey 2024
Voice cloning products that failed to take basic steps against unauthorized cloning4 of 6Consumer Reports, voice cloning assessment 2025
Prize money for FTC Voice Cloning Challenge winners$35,000U.S. Federal Trade Commission, Voice Cloning Challenge 2024

Methodology and Sources

Last updated: October 31, 2026. We update this roundup quarterly, and the next refresh is expected when the ICASSP 2027 proceedings are published.

Try VoxBooster — 3-day free trial.

Real-time voice cloning, soundboard, and effects — wherever you already talk.

  • No credit card
  • ~30ms latency
  • Discord · Teams · OBS
Try free for 3 days