One of the shortcomings that we have is having the agent announce something before actually making a tool call, we have implemented something for this, but it’s not 100% accurate.
What I am looking for is the following:
Prior to calling a tool, the agent should announce something, like: “Let me check that for you..”, “One moment”
So this should happen prior to the tool call, not when the tool is called and you do generateReply. It’s not possible to do this through prompting and making it 100% accurate for all tool calls as well. The reason behind this is sometimes tool calls get initiated and it might take some latency, so we want the agent to always say something prior to it.
Is there a mechanism in place or are there any ways in achieving and getting the predictability that we are looking for?
Hi @darryncampbell I saw this feature indeed, but it doesn’t address the exact feature though. This feature basically addresses long running tasks, but it doesn’t address the point in always announcing a tool call, prior of doing the tool call.
For example:
User asks: “Do I have access to the premium subscription?”
Agent calls tool “check_subscription_elligibility”
…tool takes long…
Agent says: “I am still looking for your eligibility”
What I am looking for is this:
User asks: “Do I have access to the premium subscription?”
Agent responds: “let me check that for you” ← to have this guaranteed prior to every tool call/
Agent calls tool “check_subscription_elligibility”
…tool takes long…
Agent says: “I am still looking for your eligibility”
This specifically doesn’t seem to be supported by the framework.
@vvgr001, The async-tools route above is the right call for the mid-call updates ("still looking..."), but the guaranteed pre-call line is a separate mechanism: override llm_node, and the moment the model emits a tool call with no text of its own, flush a filler phrase to TTS. That fires on every such tool call, so it’s deterministic rather than prompt-dependent [ docs.livekit.io/agents/logic/nodes ].
from livekit.agents import Agent, FlushSentinel, ModelSettings, llm
async def llm_node(self, chat_ctx, tools, model_settings):
called, has_text = [], False
async for chunk in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
if isinstance(chunk, llm.ChatChunk) and chunk.delta:
has_text |= bool(chunk.delta.content)
if chunk.delta.tool_calls:
called.extend(chunk.delta.tool_calls)
yield chunk
if called and not has_text: # any tool call, no spoken text
yield "Let me check that for you. "
yield FlushSentinel() # push to TTS now, before the tool result
FlushSentinel ends the current segment and sends that text to TTS immediately instead of waiting for the turn to finish, so the line plays before the tool returns. The docs example gates on a single tool name (get_weather); dropping that to called covers all tools.
@vvgr001, Thanks for trying it. The reason that version isn’t guaranteed is the gate I put on it: it only fires when the model emits a tool call with no text of its own, and the FlushSentinel flush races the streamed tool-call chunks. Both of those are decided by the model mid-stream, so you can’t force them to fire every time.
Drop the stream-catching entirely and speak from inside the tool instead. ctx.session.say() works inside a tool body [ examples/voice_agents/async_tool_agent.py ], so put the line as the first statement:
from livekit.agents import llm, RunContext
@llm.function_tool()
async def check_subscription(self, ctx: RunContext):
ctx.session.say("Let me check that for you.") # enqueued before the lookup runs
eligible = await slow_eligibility_call()
return "yes" if eligible else "no"
It’s unconditional code at the function entry, so it runs on every call regardless of what the model decides to stream, and say() enqueues the audio immediately, before the tool’s work and before the result goes back. To avoid repeating the line, wrap your tools in a small decorator that calls ctx.session.say(...) before delegating to the real function.