Latency in voice AI: why the 800 ms threshold matters and how to measure it

You ask a question on the phone and the other side goes quiet: one second, a second and a half... The moment you start saying "Are you still there?", the natural rhythm of the conversation is broken. In voice AI assistants this waiting time is called turn latency, and it is one of the most important technical criteria deciding whether an assistant feels human or not.

This article covers the research findings on the natural rhythm of human conversation, the latency thresholds set by telecommunication standards, where latency in voice AI comes from and how to measure it; at the end we show how a vendor's figure should be read together with its conditions.

Why does latency matter in voice AI?

Because human conversation runs on a surprisingly fast turn-taking system. In the 2009 PNAS study by Stivers and colleagues covering 10 languages, the densest range of turn transitions in every language was measured at 0–200 ms, and language-specific means stayed within roughly 250 ms of the cross-language mean [1]. The same study showed that avoiding overlapping talk and minimizing silence between turns is a universal independent of language [1].

Levinson and Torreira's 2015 work completes the picture: the gap between turns is on the order of 200 ms and the great majority of transitions fall between −100 and 500 ms, whereas the latency of language production itself is much longer, over 600 ms [2]. In other words, people start planning their reply while the other person is still talking; comprehension is predictive [2]. The consequence for voice AI is clear: a system that waits for the caller to fall silent and only then starts thinking cannot match the human rhythm.

The telecommunications world has been setting rules on this for decades. According to ITU-T Recommendation G.114, if one-way ("mouth-to-ear") delay can be kept below 150 ms, most applications experience "essentially transparent interactivity"; above 400 ms is considered unacceptable for general network planning [3]. The same document stresses that highly interactive tasks such as voice calls can be affected by much lower delays [3]. The E-model in ITU-T G.107 accounts for delay as a separate impairment factor (Id) that lowers the quality rating R; according to the provisional guide table in Annex B of that document (taken from G.109), an R value of 90 and above is read as "very satisfied", the 80s as "satisfied", the 70s as "some users dissatisfied" and the 60s as "many users dissatisfied" [4].

In short: the human rhythm is ~200 ms; the phone network itself takes up to ~400 ms one way, and that stacks on top of turn latency. Voice AI usually lands above both, because "thinking" time is added on top of network delay.

How fast should an AI phone assistant respond?

The definition in Vapi's own engineering write-up is this: latency is the time between the end of the user's sentence and the start of the agent's sentence [5]. According to the same write-up, conversational flow breaks once this exceeds 1,200 ms, which is roughly the time it takes the user to have a tangential thought [5]. As a similar vendor observation, Telnyx writes that above 1,500 ms callers report the conversation feels "broken" [6].

In this article we propose 800 ms as the practical upper bound; it is not an ITU standard or a peer-reviewed finding but an engineering choice. The reasoning: G.114's 400 ms tolerance applies to the network's one-way delay alone [3]; turn latency adds STT, language-model and TTS time on top of it. Approaching the human rhythm is out of reach, but you still need to stay a safe distance below the 1,200 ms breaking point [5]. We propose 800 ms as a working threshold we assume callers tolerate without feeling that "no answer is coming", and one that is achievable in engineering terms; Telnyx's comparison piece likewise describes anything above 800 ms as "noticeably delayed" [6]. Above it the wait becomes audible; far above it comes the "Are you still there?" question.

Latency does not come from a single part; it is the sum of a chain:

• Speech-to-text (STT): for streaming recognition, Deepgram defines "transcript latency" (how far the transcript lags behind the audio) and "end-of-turn (EOT) latency" (the time from when the user stops speaking to when the end-of-turn event is received) as separate metrics, and says the second is often the decisive one for voice agents [7].
• Language model (LLM): according to Vapi's engineers, almost always the bottleneck of the chain; what needs measuring is the "time to first meaningful sentence" [5].
• Speech synthesis (TTS): ElevenLabs states roughly 75 ms inference for its Flash models and notes that streaming endpoints return audio piece by piece as it is generated, reducing time-to-first-byte [8].
• Telephony: the codec, jitter buffer, echo cancellation and the SIP/carrier path's own delay. The ITU's 150/400 ms thresholds [3] measure exactly this layer.

Component figures on vendor pages are not product promises but examples that "every link has its own cost"; the latency the caller feels is the sum of all links.

How is voice AI latency measured?

For a comparable measurement, the metric must first be defined. Retell's documentation defines it the same way: "end-to-end latency" is the total time from the moment the user stops speaking to the moment the agent begins responding [9]. Some sources call this "time-to-first-audio"; it measures the same thing: the gap between the start of silence and the first syllable heard from the speaker.

A single number is misleading. Deepgram notes that latency naturally fluctuates and should be tracked with percentile statistics across representative samples rather than a single measurement [7]. Retell's documentation likewise breaks per-call latency down into components such as STT, LLM, TTS and knowledge base, and reports p50, p90, p95 and p99 values [9]. If the number you get from a vendor is p50, ask separately about the p90 experience, which one in ten calls will have.

A simple protocol for testing a vendor:

• Make at least 20 real calls from the same phone line in the same time window; write your test sentences in advance and use the same sentences with every vendor.
• On each call, record the moment the caller stops and the moment the agent's first syllable is heard (an audio recorder on your own device and a waveform viewer are enough); measure over the real phone network, not a browser demo.
• Compute the median (p50) and p90 instead of the average; the spread between the two shows how stable the system is.
• Note the audio quality and context length you measured with: latency measured with a short system prompt may not match the long prompt of your real scenario.
• Pause for a second in the middle of a long sentence and note whether the agent cuts you off; this is a quality criterion as important as speed.

How many seconds should the AI wait before cutting in on a phone call?

This question is really "after how much silence should the system decide it is its turn?" There is no single correct value; the approach is to start around half a second and tune it to the scenario. In Vapi's documentation this waiting time defaults to 0.4 seconds; it is recommended to increase it if the assistant speaks too soon and decrease it if there is too much delay [10]. The smart endpointing in the same document tries to tell whether the user has truly finished even when they pause mid-thought; for the audio-based provider Krisp a threshold between 0 and 1 (default 0.5) is defined, where a lower value means snappier and a higher value more conservative detection [10].

The dilemma is clear: shortening the wait lowers latency but raises the chance the system interrupts the caller. Pauses within a turn are ordinary in human speech; transitions have a natural distribution reaching up to 500 ms [2]. A fixed, very short silence threshold will repeatedly cut off a caller who is giving an address, reading out an ID number or thinking while speaking; every interruption costs more than the latency it saved.

The other side of the question is barge-in: if the caller cuts in while the agent is talking, the agent should stop, listen and yield the turn to the caller. What gets tuned here is again a duration: Vapi's documentation defines separately how many words the caller must say (default 0) or how many seconds they must be speaking (default 0.2 s) before the assistant stops, and how long the assistant waits before it starts talking again after being interrupted (default 1 s) [10]. The difficulty is telling a genuine interruption from acknowledgments such as "uh-huh" or "yeah" [10]; an agent that goes quiet at every acknowledgment never finishes a sentence, while one that never yields feels as if it is not listening.

A second speed-up trap is shortening the context. Aggressively trimming the conversation history and the system prompt sent to the language model may lower the time to the first sentence, but the agent forgets the name, date or complaint given three turns earlier and the caller has to repeat it. An improvement that looks good on the measurement sheet produces a longer and more irritating call in a real conversation. That is why any latency figure only becomes meaningful together with the question "at which audio quality, and without shortening the context?"

An assistant that is fast but forgetful, or that cuts you off, is worse than a slow one; lowering latency only pays off when context and courtesy are preserved.

AvaritCall's measured turn latency and its conditions

The values below are AvaritCall's own internal measurements and use the same metric as defined above: the time from when the caller finishes speaking until the agent begins its spoken reply. All were taken at Opus audio quality and without shortening the context; they are averages, and the measurement date, number of calls and p50/p95 distribution are not published. The Türkiye value matches the "0.8 s average response time" stated on the Turkish pricing page; the Europe and USA values are internal measurements only.

• Türkiye: ~0.8 s
• Europe: ~0.6 s
• USA: 0.45–0.5 s

AvaritCall develops its telephony layer itself (codec management including Opus, echo cancellation, channel and concurrency management; SIP trunk and DID support). That lets the entire audio path be optimized end to end. Pricing is pay-as-you-go: $0.08 per spoken minute (STT, LLM and TTS included, one line item), $5 credit on sign-up, no commitment. With the $5 sign-up credit you can run the protocol above from your own line and verify these values yourself (avaritcall.com/en/pricing).

Noise and long answers: where the threshold does not hold

• The ~0.8 s in Türkiye (under the same conditions) is still four times the human rhythm of ~200 ms (a cross-language average) [1][2]; the expectation should not be "as fast as a human" but "fast enough not to feel like waiting".
• In very noisy environments (a workshop, a street, a car on speakerphone) recognition accuracy drops; end-of-turn detection can mistake noise for speech and latency becomes unstable too. This applies to AvaritCall as well.
• Long answers: when the agent has to consult a knowledge base or a tool before replying, the time to the first sentence grows; in Retell's breakdown the knowledge base is a separate latency component [9]. When the agent itself talks at length, what matters is not latency but whether the caller can cut in. 800 ms is a meaningful threshold for short, single-sentence turns.
• The latency you measure does not mean the agent said the right thing. Decisions with legal, medical or financial consequences must be handed to a human; AvaritCall is not designed to replace emergency lines.
• Component-level choice of STT/LLM/TTS is more limited in AvaritCall than on developer platforms; for teams that want to pick every link themselves this is a disadvantage.
• Concurrent capacity depends on the plan; very high concurrency requires the enterprise plan. If a queue forms at the capacity limit, that creates a wait independent of turn latency.

In summary: people take turns in ~200 ms [1][2], the network itself takes up to ~400 ms one way and that stacks on top of turn latency [3], and according to Vapi's engineering write-up the conversation in voice AI breaks at 1,200 ms [5]. The 800 ms in between is not a standard or a promise but a measurement discipline: a real line, p50 and p90, the same audio quality, unshortened context. Values taken from vendor documentation belong to pages accessed on 30 August 2026; vendors may change them.