Background audio mixer closed in NodeJs

Agents
, ,
1

We have had several issues with adding typing sounds during tool calls in node js. The issue is that we create a single background audio player (with the session) and then on an event start playing a preloaded track. The first time this works great, the second time it crashes the entire process silently as it happens in an async function, which means our entire agent is killed without warning, logging or post processing.

I have been able to narrow down the issue into the closing of the background audio handle. There is a condition that closes the audio mixer (rtc component) of the audio player if there aren’t any other tracks active anymore.
Our code tried to add another track on this player and then failed.

Not sure if this is a real issue in the internals of those components or that we are using it incorrectly.
We now work around it by immediately adding another track on loop with 1% volume (not audible)

2 Likes
2 
Session
backgroundPlayer = new voice.BackgroundPlayer
|
| Tool call 1
|  | backgroundPlayer.play()
|  |  ---> stops after timeout 
|  |  ---> activeTracks.length === 0 
|  |  ---> mixer.close()
|
| Tool call 2
|  | backgroundPlayer.play()
|  |  ---> mixer.isClosed
|  |  ---> Fails
3

I remember conversations about this previously, The ‘official’ stance is you need to recreate the player to start again, as stated here: Agent speech and audio | LiveKit Documentation

You must create a new player instance if you want to start again

but kudos on the workaround.