Answering sooner without being faster
A small decision model picks a waiting phrase while the words are transcribed, and the silence after the user stops shrinks.
An agent can feel quick without its model getting any faster. What the user feels is the silence between their last word and the agent's first sound, and most of that silence is spent waiting in line, not computing.
The silence that counts
A user asks what is on their calendar tomorrow. In a basic chain, each part waits for the one before it: a fixed silence to be sure the user is done, the end of the transcript, the model reading it and asking for a tool, the tool, the model again, the whole answer rendered by the voice, and only then a sound.
None of these steps is slow on its own. Added end to end, they make a silence long enough for the user to wonder whether they were heard.
Deciding the end of the turn early
A fixed silence has to be long, or it cuts users who pause mid-sentence. Cue's end-of-turn head reads each pause and ends the turn when the sentence is complete, so turnend can come early after a finished question and late after a hesitation.
With streaming speech-to-text, fed with the input's audio while the user talks, the words are already there when the turn ends.
Speaking sentence by sentence
The voice does not need the whole answer. Stream the model's text, cut it at the first sentence, send that sentence to the voice and play it while the rest is still being written.
The first sound now waits for one sentence, not for the whole reply.
Starting the obvious tool before the model
Many requests name their tool in the first words: "what's on my calendar", "turn off the lights". A small classifier on the partial transcript starts the lookup while the user is still talking.
When the model asks for the calendar, the result is already there, or on its way. A wrong guess costs one wasted call, never a wrong answer: the model still decides.
A waiting phrase, chosen while the words are transcribed
Some waits cannot be removed: a slow service, a long search. A short phrase at the end of the turn tells the user they were heard: "Let me check."
It has to come at once, so the phrases are synthesized ahead of time. It has to fit the request, so a small decision model picks it from the words as they arrive, and the choice is ready when turnend fires.
When to stay silent
The decision model reads the kind of request (looking something up, doing something, explaining how, small talk), whether the user sounds in a hurry, and the language. A hurried user gets the shortest phrase, a request in French a French one.
It also knows how long each kind of request usually takes, learned from past answers. When the answer comes quickly, a phrase would only delay it: the agent stays silent and answers.
Putting it together
Each line lights its part of the scene.
input.on('audio', (pcm) => stt.send(pcm))stt.on('partial', (text) => { decide.read(text); tools.prefetch(text) })input.on('turnend', async () => { const phrase = decide.pick() // kind, hurry, language, usual wait if (phrase) player.play(cached[phrase]) for await (const s of llm.sentences(stt.text())) player.queue(await tts(s))})Only the turn comes from Alkove here: the decision model, the cached phrases and the prefetch belong to the app. The whole voice agent guide is in the docs.