Across 10 languages, people answer a question after a mean gap of just 208 ms, yet Twilio’s starting target for a cascaded voice AI agent is a 1,115 ms mouth-to-ear turn gap, roughly 5 times longer (1,115 ÷ 208 ≈ 5.4). The human baseline is fast because listeners predict the end of a turn: producing even a single word takes about 600 ms, so the reply is planned while the other person is still talking (Levinson and Torreira, Frontiers in Psychology 2015). Voice agents cannot predict the same way, so they wait, transcribe, think and synthesize in sequence, and the most recent independent leaderboard shows average time to first audio for speech-to-speech models from 0.44 s to 2.98 s (Artificial Analysis Speech to Speech Index 2026). For contact centers rolling out automation, covered in our voice agent contact center automation statistics, latency is the part of the experience that callers feel before they hear a single useful word. We aggregated data from the PNAS turn-taking studies, the international telephony delay standard, open-source and vendor engineering guides, independent leaderboards, and research benchmarks published on arXiv and at ACM CUI.
TL;DR
- Mean gap before an answer across 10 languages: 208 ms; median 100 ms (Stivers et al., PNAS 2009)
- 70% to 82% of turn transitions in conversation are shorter than 500 ms (Heldner and Edlund 2010, reviewed in Frontiers in Psychology 2015)
- ITU-T G.114 sets 400 ms as the one-way delay ceiling for network planning, with under 150 ms essentially transparent (ITU-T G.114, 2003)
- A telecom research voice agent averaged 678 ms to first audio across 500 recorded questions (Ethiraj et al., NetoAI, arXiv 2025)
- Identical conversation clips felt more connected with a 55.68 ms average gap than with 557.14 ms (Templeton et al., PNAS 2022)
- Deepgram claims about 260 ms end-of-turn detection for its Flux model (Deepgram Flux documentation 2025, vendor claim)
- LiveKit’s updated end-of-turn model cut false interruptions from 18.66% to 11.34% across 14 languages (LiveKit, December 12, 2025)
- ElevenLabs Flash v2.5 measured at 197 ms median latency including network, versus a 75 ms inference claim (Vapi Humanness Index, June 2026)
- OpenAI reported GPT-4o audio responses averaging 320 ms, versus 5.4 s for its older GPT-4 voice pipeline (OpenAI, Hello GPT-4o 2024, vendor claim)
- Gemini Live took 1.301 s to take the turn versus 0.265 s for Moshi on Full-Duplex-Bench (Lin et al., arXiv 2025)
- Users chose the high-latency agent as the one they liked least 61.11% of the time (Maslych et al., ACM CUI 2025)
1. The Human Baseline: How Fast People Take Turns
The benchmark every voice agent is judged against was set long before software existed. In the largest cross-language comparison of turn timing, the most common gap between a question and its answer was 0 ms, and the cross-language median was 100 ms (Stivers et al., Universals and cultural variation in turn-taking in conversation, PNAS 2009). Speakers do not wait for silence and then react; they launch the answer at the moment the question ends.
That speed is a planning feat, not a reflex. Picture-naming experiments put the time to produce one word at about 600 ms and a short sentence at about 1,500 ms, which is why researchers conclude that listeners must predict when a turn will end and prepare the reply in parallel (Levinson and Torreira, Frontiers in Psychology 2015). A cascaded voice agent does the opposite: it waits for the caller to stop, then starts the whole chain.
| Metric | Value | Source |
|---|---|---|
| Mean gap between question and answer, all 10 languages combined | 208 ms | Stivers et al., PNAS 2009 |
| Median gap between question and answer, all 10 languages combined | 100 ms | Stivers et al., PNAS 2009 |
| Fastest and slowest language means (Japanese vs Danish) | 7 ms vs 469 ms | Stivers et al., PNAS 2009 |
| Share of turn transitions shorter than 500 ms | 70% to 82% | Heldner and Edlund 2010, reviewed in Levinson and Torreira, Frontiers in Psychology 2015 |
| Share of floor transfers that overlap the previous speaker (Switchboard telephone corpus, 348 conversations) | 30.1% | Levinson and Torreira, Frontiers in Psychology 2015 |
| Time needed to start saying a single named word (picture naming) | About 600 ms | Indefrey and Levelt 2004, reviewed in Levinson and Torreira, Frontiers in Psychology 2015 |
| Typical (modal) human response time between turns | Around 200 ms | Templeton et al., PNAS 2022 |
Context note: the 10-language sample covers 101 conversations on 5 continents. The paper says language means fall within about 250 ms either side of the cross-language mean, though Danish sits about 261 ms above it (469 minus 208). Most recent available data: the 2009 and 2015 studies above. These baselines change slowly, so their age matters less than for product benchmarks.
2. Network and Telephony Delay Limits
Before any model runs, the phone network spends part of the budget. “One-way delay” here means the time sound takes to travel from the speaker’s mouth to the listener’s ear in one direction, not a round trip. ITU-T Recommendation G.114 says a one-way delay of 400 ms should not be exceeded for general network planning, and that delays kept below 150 ms are essentially transparent for most applications (ITU-T G.114, One-way transmission time).
The voice agent era turns these limits upside down. The standard was written for human-to-human calls where the far end answers in about 200 ms; an agent that needs a second to think uses more than twice the entire 400 ms network allowance before it says a word. Twilio’s own figures show the split: its target for the full mouth-to-ear turn gap is 1,115 ms and its target inside the agent platform is 885 ms, which implies about 230 ms of network and phone-line transit (1,115 minus 885).
| Metric | Value | Source |
|---|---|---|
| Maximum one-way delay recommended for general network planning | 400 ms | ITU-T G.114 (05/2003) |
| One-way delay below which most speech and non-speech applications feel essentially transparent | 150 ms | ITU-T G.114 (05/2003) |
| Expected VoIP mouth-to-ear delay on intra-regional routes of 5,000 km or less | Under 150 ms | ITU-T G.114 Appendix II |
| VoIP mouth-to-ear delay on worst-case inter-regional terrestrial routes (27,500 km) | Just over 300 ms | ITU-T G.114 Appendix II |
| Public-internet round trip to a speech synthesis server, depending on distance | 20 to 200 ms | ElevenLabs latency documentation 2026 (vendor) |
Context note: the May 2003 edition is still listed as in force. Delays below 400 ms are assessed with the E-model of Recommendation G.107, a formula that estimates conversational quality from delay, and the introduction notes that highly interactive tasks may be affected by delays below 100 ms. Most recent available data: the 2003 edition.
3. The Latency Budget of a Cascaded Voice Agent
A cascaded agent chains three models: speech-to-text (STT), a large language model (LLM), and text-to-speech (TTS). Twilio’s component targets add up to 825 ms (350 + 375 + 100), but its upper limits add up to 1,500 ms (500 + 750 + 250), which means a stack that merely stays inside each limit can still blow past the 1,400 ms end-to-end ceiling (Twilio, A Guide to Core Latency in AI Voice Agents). The guide labels these “starting benchmarks, not best-in-class.”
The Pipecat guide’s reference breakdown shows where the time actually goes: transcription with endpointing, LLM time to first byte, and TTS time to first byte together account for 1,070 ms of a 1,293 ms total, about 83% (1,070 ÷ 1,293), while all the audio plumbing (microphone, codecs, jitter buffers, speaker) takes the rest (Voice AI and Voice Agents guide). A “jitter buffer” is a short holding queue that smooths uneven packet arrival; the breakdown spends 40 ms on one in each direction. A research pipeline built for telecom questions on a single H100 GPU landed in the same range (Ethiraj et al., arXiv 2508.04721).
| Metric | Value | Source |
|---|---|---|
| Speech-to-text latency, target vs upper limit | 350 ms vs 500 ms | Twilio, Guide to Core Latency in AI Voice Agents 2025 |
| LLM time to first token, target vs upper limit | 375 ms vs 750 ms | Twilio, Guide to Core Latency in AI Voice Agents 2025 |
| Text-to-speech time to first byte, target vs upper limit | 100 ms vs 250 ms | Twilio, Guide to Core Latency in AI Voice Agents 2025 |
| Mouth-to-ear turn gap as the caller perceives it, target vs upper limit | 1,115 ms vs 1,400 ms | Twilio, Guide to Core Latency in AI Voice Agents 2025 |
| Reference voice-to-voice total, summing 16 stages from microphone to speaker | 1,293 ms (guide text says about 1,200 ms) | Voice AI and Voice Agents guide, Pipecat 2026 |
| Voice-to-voice latency the guide calls an important target | 1,500 ms | Voice AI and Voice Agents guide, Pipecat 2026 |
| Mean time to first audio in a telecom pipeline, 500 recorded questions (min to max) | 678 ms (412 to 1,482 ms) | Ethiraj et al., NetoAI, arXiv 2025 |
| Mean end-to-end processing time per utterance in the same telecom pipeline | 934 ms | Ethiraj et al., NetoAI, arXiv 2025 |
Outlier note: a Sprinklr research team reported a first audio chunk “as low as 640.9 ms” on an NVIDIA L4 GPU, but 1,748.6 ms for the same setting on a CPU, a reminder that hardware alone can more than double first-audio time (Choudhary and Purwar, arXiv 2509.20971).
4. Endpointing and Turn Detection
“Endpointing” is the agent’s decision that the caller has finished speaking. It is the stage most often tuned by hand, because it trades speed against rudeness: wait too little and the agent cuts people off mid-sentence, wait too long and the caller hears dead air. LiveKit’s updated end-of-turn model cut the false-interruption rate across 14 languages from 18.66% to 11.34%, a 39.23% relative reduction, with no increase in response latency (LiveKit, Improved end-of-turn model cuts voice AI interruptions 39%).
The vendor numbers show how much of the budget this one decision controls. Deepgram claims its Flux model can cut agent response latency by 200 to 600 ms compared with pipeline approaches, and that an “eager” early signal arrives 150 to 250 ms sooner at the cost of 50 to 70% more LLM calls (Deepgram, Introducing Flux). Those extra calls are the hidden price of speed: the agent starts drafting answers to sentences the caller may not have finished. Background noise also shifts these decisions, a topic we track in neural noise suppression latency statistics.
| Metric | Value | Source |
|---|---|---|
| End-of-turn detection latency claimed for Flux | About 260 ms | Deepgram Flux documentation 2025 (vendor claim) |
| Flux end-of-turn latency at the 90th and 95th percentile | 1 s and 1.5 s | Deepgram, Introducing Flux, October 1, 2025 (vendor claim) |
| Reduction in false interruptions claimed for Flux | About 30% | Deepgram, Introducing Flux 2025 (vendor claim) |
| False-interruption rate, all 14 languages, old vs new LiveKit model (at a fixed 99.3% true-positive rate) | 18.66% vs 11.34% | LiveKit, December 12, 2025 |
| False-interruption rate, English, old vs new LiveKit model | 16.6% vs 13.0% | LiveKit, December 12, 2025 |
| Silence the example server VAD configuration waits for before ending a turn | 500 ms | OpenAI Realtime VAD guide 2026 |
Context note: VAD (voice activity detection) is the simple detector that hears whether anyone is speaking; the 500 ms value is the sample configuration in OpenAI’s Realtime VAD guide, not a stated default. A fixed 500 ms silence rule alone is already more than double the human mean gap of 208 ms from section 1.
5. LLM Time to First Token and TTS Time to First Audio
Once the turn ends, the model has to start talking. The Pipecat guide’s rule of thumb is that an LLM time to first token of 600 ms or less is fast enough for most voice use cases, and only two of the four hosted models it lists meet that bar at the median, while none does at the 95th percentile (Voice AI and Voice Agents guide). Time to first token (TTFT) is how long the language model takes to emit the first piece of its answer. The tail matters more than the median on a phone line, because a caller remembers the one 1.7 s silence, not the average.
Speech synthesis is now the smallest slice, but vendor and measured figures diverge. ElevenLabs says its Flash models reach about 75 ms of model inference, explicitly excluding network and application overhead (ElevenLabs latency documentation), while a third-party run of 50 live streaming trials from a benchmark machine measured 197 ms median (Vapi Humanness Index). Teams comparing synthesis engines, including for an AI voice generator, should ask which of the two numbers a spec sheet quotes.
| Metric | Value | Source |
|---|---|---|
| GPT-4.1 time to first token, median vs 95th percentile | 536 ms vs 1,771 ms | Voice AI and Voice Agents guide, Pipecat 2026 |
| Gemini 2.5 Flash time to first token, median vs 95th percentile | 597 ms vs 1,137 ms | Voice AI and Voice Agents guide, Pipecat 2026 |
| Claude 4.5 Haiku time to first token, median vs 95th percentile | 637 ms vs 1,615 ms | Voice AI and Voice Agents guide, Pipecat 2026 |
| GPT-5.1 time to first token, median vs 95th percentile | 739 ms vs 1,492 ms | Voice AI and Voice Agents guide, Pipecat 2026 |
| ElevenLabs Flash v2.5, inference claim vs measured streaming latency including network | 75 ms vs 197 ms | ElevenLabs documentation 2026 (vendor claim); Vapi Humanness Index, June 10, 2026 |
| Cartesia Sonic 3.5, measured streaming latency | 128 ms | Vapi Humanness Index 2026 |
| ElevenLabs Multilingual v2, measured streaming latency | 1,006 ms | Vapi Humanness Index 2026 |
Context note: Vapi runs a voice agent platform, not a speech synthesis engine, but its index is still a platform-run benchmark. The page gives the 50-trial method only for Flash v2.5, so the other measured values may not be directly comparable. ElevenLabs also notes that a 500 ms audio player buffer “is common”, which can erase most of a fast model’s advantage on the client side.
6. Measured Latency of Speech-to-Speech and Full-Duplex Models
Speech-to-speech models skip the text hand-offs, and full-duplex models listen and speak at the same time, so in theory they should close the gap to human timing. Kyutai’s Moshi reports a theoretical latency of 160 ms and 200 ms in practice (Kyutai, Moshi, arXiv 2410.00037), and an independent benchmark measured it taking the turn after 0.265 s (Lin et al., Full-Duplex-Bench, arXiv 2503.04721).
Commercial models are far more spread out. In the Artificial Analysis index of June 23, 2026, average time to first audio ranged from 0.44 s to 2.98 s; the two slowest entries were “High” reasoning settings, and one of them, GPT-Realtime-2 (High) at 2.33 s, posted the top index score of 77.2% (Artificial Analysis, Speech to Speech Index). Speed and smarts still pull in opposite directions. Live captioning faces a similar speed-versus-accuracy trade-off, which we track in real-time subtitling latency statistics.
| Metric | Value | Source |
|---|---|---|
| GPT-4o response time to audio input, fastest vs average | 232 ms vs 320 ms | OpenAI, Hello GPT-4o 2024 (vendor claim) |
| Earlier ChatGPT Voice Mode pipeline average latency, GPT-3.5 vs GPT-4 | 2.8 s vs 5.4 s | OpenAI, Hello GPT-4o 2024 (vendor claim) |
| Moshi latency, theoretical vs practical | 160 ms vs 200 ms | Kyutai, Moshi paper, arXiv 2024 |
| Time to take the turn after the user stops speaking (CANDOR conversation dataset), Moshi vs Gemini Live | 0.265 s vs 1.301 s | Lin et al., Full-Duplex-Bench, arXiv 2025 |
| Time to respond after a user interruption, Moshi vs Gemini Live | 0.257 s vs 1.183 s | Lin et al., Full-Duplex-Bench, arXiv 2025 |
| Average time to first audio, fastest model in the index (Deepslate Opal) | 0.44 s | Artificial Analysis Speech to Speech Index, June 23, 2026 |
| Average time to first audio, GPT-Realtime-1.5 | 0.82 s | Artificial Analysis Speech to Speech Index, June 23, 2026 |
| Average time to first audio, Gemini 3.1 Flash Live Preview (High) | 2.98 s | Artificial Analysis Speech to Speech Index, June 23, 2026 |
Outlier note: the OpenAI figures are a launch claim from the model announcement, not an independent measurement, and they use a different metric from the leaderboard rows in the same table, so the two should not be compared directly. Moshi is fast but not a contact-center model: it takes the floor in 98.5% of synthetic pause cases, meaning it jumps in when the user only paused (Full-Duplex-Bench).
7. What Latency Does to Callers and Conversations
Speed is not a vanity metric. In a controlled listening study with 450 listeners, identical conversation clips edited to have an average 55.68 ms gap were rated as more connected than the same clips at 557.14 ms, with connection F(2, 2,344.3) = 28.51, P < 0.001 (Templeton et al., PNAS 2022). Listeners read timing as a social signal, and a slow agent sends the wrong one.
For AI agents specifically, a 54-person study of virtual agents found every delay level hurt perceived speed, and the authors conclude that latency above 4 seconds degrades the quality of experience; natural spoken fillers improved perceived response speed at 4.0 s and 6.5 s, while an artificial loading icon and sound did not (Maslych et al., ACM CUI 2025). Silence also changes meaning: conversation research links gaps of 700 ms or more to dispreferred answers (replies such as declining or disagreeing), so an agent with a 1-second pause risks sounding hesitant even when the answer is yes. A comparison of audio foundation models found silence made up 32.4% of conversation time with a cascaded system versus 11.8% with Moshi (Arora et al., Talking Turns, arXiv 2503.01174).
| Metric | Value | Source |
|---|---|---|
| Users who picked the low-latency (1.5 s) agent as the one they liked most | 57.41%, χ²(2, N = 54) = 14.12, p < 0.0001 | Maslych et al., ACM CUI 2025 |
| Users who picked the high-latency (6.5 s) agent as the one they liked least | 61.11%, χ²(2, N = 54) = 19.0, p < 0.0001 | Maslych et al., ACM CUI 2025 |
| Measured response time of the study’s own cascaded agent before added delay | 1.47 s on average | Maslych et al., ACM CUI 2025 |
| Gap length associated with dispreferred (negative) answers in human conversation | 700 ms or more | Levinson and Torreira, Frontiers in Psychology 2015 (review) |
| Share of conversation time spent in silence, cascaded system vs Moshi | 32.4% vs 11.8% | Arora et al., Talking Turns, arXiv 2025 |
| Added transmission delay at which call partners were rated less attentive, friendly and self-disciplined than with no delay | 1.2 s | Schoenenberg, Raake and Koeppe, International Journal of Human-Computer Studies 2014 |
| Voice AI developers very satisfied with conversation quality (survey of 40) | 2% | Deepgram, Introducing Flux 2025 (vendor survey) |
| Voice AI developers reporting noticeable to major conversation flow issues (survey of 40) | 61% | Deepgram, Introducing Flux 2025 (vendor survey) |
Context note: the 1.2 s delay finding comes from the lead author’s own summary of the study (phys.org, May 2014); most recent available data: Schoenenberg et al., 2014. We found no peer-reviewed 2025 or 2026 dataset that ties measured agent latency to real call abandonment rates, so the satisfaction evidence above comes from controlled studies rather than production call logs.
Summary: Voice AI Agent Latency by the Numbers
| Metric | Value | Source |
|---|---|---|
| Mean gap between question and answer, 10 languages | 208 ms | Stivers et al., PNAS 2009 |
| Turn transitions shorter than 500 ms | 70% to 82% | Heldner and Edlund 2010, via Frontiers in Psychology 2015 |
| Time to start saying a single word | About 600 ms | Levinson and Torreira, Frontiers in Psychology 2015 |
| Average gap in clips rated more connected vs the same clips rated less connected | 55.68 ms vs 557.14 ms | Templeton et al., PNAS 2022 |
| Maximum one-way delay for network planning | 400 ms | ITU-T G.114 (05/2003) |
| Cascaded agent mouth-to-ear target vs upper limit | 1,115 ms vs 1,400 ms | Twilio, Guide to Core Latency in AI Voice Agents 2025 |
| Reference voice-to-voice breakdown total | 1,293 ms | Voice AI and Voice Agents guide, Pipecat 2026 |
| Telecom research pipeline mean time to first audio | 678 ms | Ethiraj et al., NetoAI, arXiv 2025 |
| Flux end-of-turn detection claim | About 260 ms | Deepgram Flux documentation 2025 (vendor claim) |
| False-interruption rate, old vs new end-of-turn model, 14 languages | 18.66% vs 11.34% | LiveKit, December 12, 2025 |
| ElevenLabs Flash v2.5 measured streaming latency | 197 ms | Vapi Humanness Index, June 2026 |
| GPT-4o average audio response time | 320 ms | OpenAI, Hello GPT-4o 2024 (vendor claim) |
| Moshi practical latency | 200 ms | Kyutai, Moshi paper, arXiv 2024 |
| Turn-taking latency, Gemini Live | 1.301 s | Lin et al., Full-Duplex-Bench, arXiv 2025 |
| Average time to first audio range across the index | 0.44 s to 2.98 s | Artificial Analysis Speech to Speech Index, June 23, 2026 |
| Users who liked the 6.5 s agent least | 61.11% | Maslych et al., ACM CUI 2025 |
| Measured response time of a research cascaded agent before added delay | 1.47 s | Maslych et al., ACM CUI 2025 |
| Silence share of conversation time, cascaded system | 32.4% | Arora et al., Talking Turns, arXiv 2025 |
Methodology and Sources
- Stivers et al., Universals and cultural variation in turn-taking in conversation, PNAS 2009.
- Levinson and Torreira, Timing in turn-taking and its implications for processing models of language, Frontiers in Psychology 2015 (also the source for the Heldner and Edlund 2010 and Indefrey and Levelt 2004 figures it reviews).
- Templeton et al., Fast response times signal social connection in conversation, PNAS 2022.
- ITU-T, Recommendation G.114: One-way transmission time, May 2003, in force.
- Twilio, A Guide to Core Latency in AI Voice Agents (Cascaded Edition), November 17, 2025.
- Pipecat, Voice AI and Voice Agents guide, lead author Kwindla Hultman Kramer, June 2026 update.
- Ethiraj et al., NetoAI, Toward Low-Latency End-to-End Voice Agents for Telecommunications, arXiv, August 5, 2025.
- Choudhary and Purwar, Sprinklr, i-LAVA: Insights on Low Latency Voice-2-Voice Architecture for Agents, arXiv, September 25, 2025.
- Deepgram, Introducing Flux: Conversational Speech Recognition, October 1, 2025, and Flux quickstart documentation (vendor claims and a vendor survey).
- LiveKit, Improved end-of-turn model cuts voice AI interruptions 39%, December 12, 2025 (vendor benchmark).
- OpenAI, Realtime API voice activity detection guide and Hello GPT-4o, 2024 (vendor claims).
- ElevenLabs, Latency documentation, 2026 (vendor claim).
- Vapi, Humanness Index, ElevenLabs Flash v2.5 page, latency check June 10, 2026.
- Kyutai, Moshi: a speech-text foundation model for real-time dialogue, arXiv 2024.
- Lin et al., Full-Duplex-Bench, arXiv 2025 (version 3, August 16, 2025).
- Artificial Analysis, Announcing the Speech to Speech Index, June 23, 2026, and the live Speech to Speech leaderboard.
- Maslych et al., Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents, ACM CUI 2025.
- Arora et al., Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics, arXiv 2025.
- Schoenenberg, Raake and Koeppe, International Journal of Human-Computer Studies 2014, as summarized by the lead author on phys.org.
- Data watch: the live leaderboard moves between updates; on October 27, 2026 it listed GPT-Realtime-2 (High) at 1.14 s time to first audio, against 2.33 s in the June 23, 2026 announcement, and its definition (seconds to generate the first audio token on the Big Bench Audio set) differs from a mouth-to-ear turn gap or the Full-Duplex-Bench turn-taking latency, so compare values within one source, not across sources. The Pipecat guide’s table sums to 1,293 ms while its text says about 1,200 ms. The component targets in the cascaded-agent guide from section 3 (825 ms in total) are not reconciled with its 885 ms platform target. Vendor figures (speech recognition, turn detection, speech synthesis and realtime model makers) are labeled as vendor claims or vendor-run benchmarks. The 2009 and 2015 turn-taking studies, the 2003 edition of G.114 and the 2014 delay study are older than three years but remain the most recent available data for those baselines. No public, peer-reviewed dataset yet links measured agent latency to real call abandonment.
- Last updated: October 27, 2026. We update this roundup quarterly, and the next refresh is expected when Artificial Analysis publishes its next Speech to Speech Index update.