Why Your n8n Voice Agent Feels Slow: Latency Measured, Not Guessed

The short answer: a voice agent reads as a dead line past about 1.2 seconds, and the delay is almost never the AI. In our production build, a 4.5 second reply broke down as 3.29 seconds of Google Sheets lookups and 24 milliseconds of actual model thinking. The three fixes that mattered were: keep audio out of n8n entirely, bake reference data into the workflow instead of reading it per turn, and pick a low latency text to speech model (Deepgram's Aura-1 aura-asteria-en ran 2.0 to 2.6s where the newer Aura-2 aura-2-thalia-en ran 4.6 to 5.2s on identical text). After that, perceived latency beats real latency: pre warming filler audio moved first sound from 1877ms to 708ms without making anything actually faster.

Every number here came off our own build, a working AI receptionist running on n8n, Deepgram and Claude. We are publishing the measurements because the n8n community keeps asking whether n8n is fast enough for real time voice, and nobody has posted real numbers.

The workflow itself is free. You can download the template and skip straight to the working version, no signup, or read on for why it is built this way.

What counts as too slow for a voice agent?

On a phone call, silence has a meaning. It means the line is dead, or the person has stopped listening. People tolerate a pause from a human because they can hear breathing, background noise, an "uh". A voice agent gives you nothing, so the same pause reads as a fault.

Our working threshold is 1.2 seconds to first sound. Not to a complete answer, to any sound. It is the number we hold the build to, because past it a gap stops reading as someone thinking and starts reading as a fault on the line.

That distinction is the single most useful thing in this post. There are two different numbers and they need separate budgets:

Teams optimise the second number and wonder why the agent still feels broken. The first one is what callers actually judge.

Where does the time in an n8n voice workflow actually go?

We instrumented a full turn. Here is what a 4.5 second reply was made of:

The caller was waiting on a spreadsheet. Not on intelligence, not on the model, not on n8n itself. Four sequential lookups against Google Sheets, each a network round trip to Google and back, consumed three quarters of the response.

This is the most common and most invisible failure in n8n voice builds. Sheets is the default place people keep configuration, price lists, opening hours and customer records, because it is easy and free. In an async workflow that runs at 2am, a 3.29 second read costs nothing. In a live conversation it is the whole problem.

What fixed it: we split the data by how often it changes. Reference data (clinic config, treatment names, prices) is not per caller data. It changes when the business changes something, which is roughly never during a call. So we bake it directly into the generated workflow at build time. That read went from 1.63s to 0ms, because there is no read.

The tradeoff is honest and worth stating: a price change now means re running the generator that builds the workflow. We chose explicit over automatic deliberately. A silent background sync that can fail is worse than a deploy step you can see.

Only genuinely per caller data (does this person exist, what were they last asked, what is in the diary) still hits Sheets, and only when the conversation actually needs it.

Which text to speech model should you use for voice agents?

This is where we found the biggest single win, and it is counterintuitive enough that we would not have believed it without measuring.

The newer model is roughly twice as slow. Aura-2 is the higher quality family. Aura-1 is the low latency family. On a marketing page that difference reads as a minor quality tradeoff. On a live call it is four seconds of silence, and four seconds of silence is not worth a slightly warmer voice. Nobody hangs up because the voice was a little synthetic. They hang up because nothing happened.

The general rule this points at: in any voice pipeline, check whether your provider ships a separate low latency model line, and default to it. The flagship model is usually tuned for quality benchmarks, not for conversation.

Why does perceived latency matter more than real latency?

Here is the result that reframed the whole build.

Our end state, after all the work above, measures first sound at about 0.7 seconds and a full answer between 4.9 and 5.7 seconds depending on reply length. Compare that to where we started: a 4.5 second reply.

The total time went up. The experience got dramatically better.

That is not a contradiction. Three things closed the gap between what the clock says and what a caller feels:

• Pre warmed filler audio. Short clips ("let me check that for you") are generated ahead of time, and we fetch them on the first sign of interaction, when the caller presses the mic or focuses the text box. Never on page load, because a visitor who scrolls past should not cost you characters. This moved first audio from 1877ms to 708ms, which is what makes a 700ms filler timer actually work rather than arriving after the gap it was meant to cover.
• Sentence by sentence speech. The agent speaks the first sentence while it is still fetching audio for the later ones. The caller hears a reply beginning while the reply is still being produced.
• Text before audio. The transcript renders as soon as the answer exists, and the audio catches up. This also means a text to speech failure costs you the voice, not the answer.

Audio is cached by the text it speaks, bounded at 24 clips, so a long conversation cannot leak object URLs and repeated phrases cost nothing the second time.

Should audio go through n8n at all?

No. This is the architectural decision that everything else depends on.

Audio never touches our n8n workflow. Speech to text and text to speech both happen in the browser or through a thin dedicated proxy. Only text crosses the wire into the workflow.

Routing raw audio through a workflow engine costs 2 to 3 seconds per turn before any thinking happens, because you are uploading a file, waiting for a node to process it, and downloading a file, all inside an execution model designed for reliable batch work rather than real time streams. n8n is a superb orchestrator and a poor audio pipe. That is not a criticism of n8n, it is the wrong job for it.

There is a related decision worth copying: our text to speech runs as a second, separate webhook, not as part of the main conversation workflow. The conversation returns text, the page renders it, and the audio is fetched independently. If text to speech fails, times out, hits a spent balance or runs in a browser that does not support the fallback, the caller still gets an answer. Every failure path degrades to the browser's built in speechSynthesis rather than to silence. A receptionist that sounds synthetic is survivable. One that goes silent mid call is not.

If you are weighing n8n against other engines for this kind of work, the n8n vs Make vs Zapier comparison covers where each one's execution model helps or hurts.

What broke along the way?

Latency was not the only thing that cost us deploys. These are the traps that are specific to building voice on n8n, and each one cost us at least one wasted deployment.

The credential one bit us four separate times. n8n picks the first Header Auth credential alphabetically when it imports a workflow, so our Claude call kept getting pointed at a Pexels key and then a Deepgram key. It fails in a way that looks like a model problem rather than an auth problem, which is what made it expensive.

The regex one is the nastiest, because the code compiles. A pattern that has quietly lost its backslashes is a valid regex that matches nothing, so the workflow runs green and the feature just does not work.

What did all of this add up to?

The headline is not the raw number. It is that we spent most of the effort on a database problem and a perception problem, and almost none of it on the AI. If your voice agent feels slow, the model is very unlikely to be why.

Every fix above is already applied in the workflow we give away. The free n8n voice receptionist template is this exact build, exported and stripped of our credentials, with the baked config, the separate text to speech workflow and the slot matching all in place. It is the version that produced these measurements, not a simplified one.

How do you measure your own?

Do this before changing anything. Optimising by intuition is how people end up switching models when their real problem is a spreadsheet.

1. Log a timestamp at every node boundary for one full turn, and print the deltas. n8n's execution view gives you per node timings directly.
2. Separate the two metrics. Time to first sound and time to full answer are different budgets with different fixes.
3. Classify every read as reference data (changes rarely, bake it in) or per caller data (changes constantly, fetch it).
4. Test your text to speech model against its own provider's low latency line, on identical text, several times. Do not trust the marketing tier names.
5. Test on a real phone, not a laptop. Mobile networks add a leg that a desktop test hides completely.

If you want to hear where this ended up, our live AI receptionist demo runs the workflow in the open and shows each step the automation takes next to the transcript, so you can watch what it is doing while it answers. The build itself is described on the AI voice agents service page, and if you are costing one out, our pricing shows real build and monthly ranges without a form.

FAQ

Is n8n fast enough to run a real time voice agent?

Yes, provided you do not route audio through it. n8n is well suited to orchestrating the decision layer of a voice agent, where a turn is text in and text out and the workflow itself contributes milliseconds. It is a poor fit for carrying audio, which adds 2 to 3 seconds per turn. Keep speech to text and text to speech outside the workflow and n8n is not your bottleneck.

What is a good latency target for an AI voice agent?

Aim for under 1.2 seconds to first audible sound and under 6 seconds to a complete answer. First sound is the number that decides whether the call feels alive, because silence on a phone line reads as a dropped call. You can hit the first target with filler audio and streaming even when the full answer takes several seconds more.

Why is my n8n voice agent slow when the AI responds quickly?

Almost always database reads. In our build, four Google Sheets lookups took 3.29 seconds while the AI model took 24 milliseconds. Each lookup is a network round trip, and they run in sequence. Audit every read in the turn, then bake anything that is reference data (prices, opening hours, configuration) directly into the workflow so it costs nothing at runtime.

Should I use Deepgram Aura-1 or Aura-2 for a voice agent?

For live conversation, Aura-1. We measured aura-asteria-en at 2.0 to 2.6 seconds per reply against aura-2-thalia-en at 4.6 to 5.2 seconds on identical text. Aura-2 is the higher quality family and Aura-1 is the low latency family, and on a live call the extra four seconds of silence costs far more than the quality difference gains.

Does using Google Sheets as a database slow down a voice agent?

Significantly, yes, if you read from it during the conversation. Sheets is fine for async workflows where a three second read is invisible, but in a live call it is usually the single largest cost. Split your data by how often it changes: bake reference data into the workflow, and only hit Sheets for genuinely per caller state.

Is there a free n8n voice agent template?

Yes. Ours is free to download with no signup, and the source is on GitHub under MIT if you would rather read every node before importing it. Two n8n workflows: the receptionist itself and a Deepgram text to speech proxy. It is the production build these measurements came from, exported and stripped of our credentials rather than cut down for release. You supply your own Anthropic and Deepgram keys and a Google Sheet, and edit two Code nodes for your business details.

What happens if text to speech fails mid call?

It should degrade to a different voice, never to silence. We run text to speech as a separate webhook from the conversation itself, so the answer text arrives and renders independently. Every failure path, including auth errors, timeouts, a spent balance and unsupported browsers, falls back to the browser's built in speech synthesis. A synthetic sounding receptionist is recoverable. A silent one ends the call.