Scenario:
during the conversation we come to a slow tool call
The user is impatient and asks another question before the tool call produces output
The LLM stops generating responses due to a tool call in the messages not having a response and this is unrecoverable.
I know that having slow tool calls is on us and we should resolve that, but this seems like a design flaw in the agent chatCtxt?
I attached a screenshot with more details (there is no STT because we prompt the agent to regenerate after some time, but user behaviour also triggers this)
The LLM stops generating responses due to a tool call in the messages not having a response and this is unrecoverable.
Tool calls should be interruptible by default: Tool definition and use | LiveKit Documentation
How can you reproduce this? I tried naively adding a delay to a function but the LLM was able to recover after I interrupted it.
I also have a tool call with an artificial delay (5 seconds) to reproduce it and then I interrupted it with session.generateReply() after 1 second of the agent starting to think
long_tool.py (6.1 KB)
generateReply doesn’t interrupt the tool like the user speaking would - attached is my naive attempt to reproduce but in my case the generateReply just tells me the tool output again, so I assume I’m not testing in the same way as you.
I’m not super familiar with the python threading implementation, but the issue occurs when the tool call response is not yet available so if your generateReply can repeat the output, it’s called at the “wrong” time to reproduce
Here is an reproduction based on the quickstart.
example.agent.ts (3.4 KB)
I have found the issue. There is a groupToolCalls util function that is used by the providers to remove these partial calls. We have a custom node so we have to remove the duplicates ourselves 
Would be nice to add this to the documentation as a consideration or a checklist for custom nodes?
You did tag the question as node-js, so that was my bad.
Glad you found the issue - I’ll get that added to the docs.
