
How to Get AI NPC Voice Replies Under a Second
A two-second pause after the player stops talking is enough to make an AI NPC feel broken, no matter how good the answer is. Latency is the most common complaint about voiced AI characters, and the fix is rarely a faster model. This guide breaks down where the time goes and how to get a voiced NPC reply under about a second.
Where AI NPC latency comes from
A voiced reply passes through four waits, one after another:
| Stage | What happens | Typical lever |
|---|---|---|
| End of turn | The game decides the player has finished speaking | Push-to-talk, or a shorter silence threshold |
| Speech-to-text | The recording becomes text | Short recordings, a nearby service |
| First token | The language model starts writing | Shorter prompts, a smaller model |
| First audio | The voice model starts speaking | Streaming the first sentence |
What players feel is the time from the end of their sentence to the character's first word. Everything after that first word can keep generating while the character talks.
1. Stream, don't wait
The biggest single win. A pipeline that waits for the whole reply, then the whole audio file, adds the full generation time of both before the player hears anything.
Streaming changes the order: as soon as the first sentence of the reply exists, it goes to text-to-speech, and that audio starts playing while the model writes sentence two. A long answer then costs no more waiting time than a short one.
Neocortex streams speech in the Unity and Godot SDKs and the Web SDK: audio plays as it is generated, line by line, so characters start talking before the whole answer has been voiced, without you building the sentence splitting yourself.
2. Tighten turn detection
Before any model runs, the game has to decide the player is done. With voice activity detection, that is a silence threshold: wait too long and every reply feels slow, too short and you cut players off mid-thought.
- Push-to-talk is instant. Releasing the button ends the turn, with no waiting for silence. It suits PC and controller games.
- Voice detection needs tuning. Start with a short threshold and lengthen it if players get cut off. Hands-free VR usually needs voice detection.
3. Keep prompts and replies short
- Prompt length slows the first token. Send the facts that matter for this scene, not the whole lore bible, and put world knowledge in a knowledge base the character draws from.
- Reply length doesn't change time to first word when you stream, but it does change how long players listen. One to three sentences is good dialogue anyway.
4. Cover the gap with the character
Real people pause too. A short, natural reaction while the reply arrives makes a half-second feel like thinking, not lag:
- Listening and thinking animations. A nod when the player stops talking, a glance away while the reply streams in.
- The emotion tag. Each Neocortex line carries an emotion, so the face can react the moment the first line arrives.
- Short authored fillers, used sparingly. "Hmm." works once. Five times in a row it gets noticed.
5. Move inference closer to the player
The network round trip to a cloud service is a fixed cost. Two ways to shrink it:
- Cut the number of hops. Three separate providers for speech-to-text, the model and the voice mean three round trips and three places to wait. Neocortex runs all three behind one character.
- Run the models on the device. This removes the round trip entirely. Our on-device runtime returns its first token in 0.06 seconds and starts speaking in about one second on a laptop RTX 3080, with a 3B model in about 3 GB of VRAM. Studios can arrange it as a Custom plan: contact us.
On-device has trade-offs: players need the hardware, the game ships the models, and smaller models are less capable. For most games, a streamed cloud pipeline is fast enough. Our guide to local and on-premise AI NPC platforms covers when on-device pays off, and how to run an LLM locally in Unity shows the prototyping route.
Measure before you optimize
Log a timestamp at each stage for a few dozen turns: end of speech, transcript received, first reply line, first audio played. The slowest stage is often not the one you expect. Developers regularly find that most of the delay is in audio and animation, not in the language model.
Common mistakes
- Upgrading the model to fix latency. Bigger models are usually slower to start. Fix streaming first.
- Long silence thresholds "to be safe". Every extra 500 ms of waiting for silence is added to every single reply.
- Waiting for the full audio file. Play the stream as it arrives.
Next steps
Latency is one part of the AI NPC voice pipeline. Cost is the other trade-off, covered in how much AI NPCs cost per player. To set up voice in your engine, see voice chat for AI NPCs in Unity, or browse the guide to building AI NPCs.
Frequently Asked Questions
How do I reduce latency for an AI NPC?
Stream every stage into the next: start speaking the first sentence while the rest of the reply is still being written. Then tighten turn detection, keep prompts and replies short, and cover the gap with animation. Neocortex streams speech, so characters start talking before the whole answer has been voiced.
What is a good response time for a voiced NPC?
About one second from the player finishing to the character's first word feels like conversation. Past two seconds players start to wonder if the game froze.
Is a local LLM faster than a cloud LLM for games?
It removes the network round trip, so the first token can come very fast. Our on-device runtime returns its first token in 0.06 seconds and starts speaking in about one second on a laptop RTX 3080. It is available to studios as a Custom plan.
Does a bigger model make NPCs slower?
Usually, yes. Bigger models take longer to produce the first token and each word after it. For most NPC dialogue a smaller, faster model with a well-written character beats a larger, slower one.
Written by
Sercan Altundas
Date
Nov 16, 2026
Read time
8 min






