When connecting to a staff member for the warm intro, it was still quiet at first, with no greeting. The AI only responded once the staff member spoke.
After the staff member accepted and the call was connected, the AI remained in the session and got in the way of the conversation between the 2 humans.
I also got this error in the logs:
[21:15:40.404] DEBUG (19916): Creating WebSocket connection to OpenAI Realtime API
[21:15:40.405] ERROR (19916): Unsupported item type: agent_config_update
[21:15:40.406] ERROR (19916): failed to update the chat context
err: {
"type": "Error",
"message": "Unsupported item type: agent_config_update",
"stack":
Error: Unsupported item type: agent_config_update
at livekitItemToOpenAIItem (C:\Projects\myproject\livekit-agent\node_modules\.pnpm\@livekit+agents-plugin-open_9161abf43acd1ea4b5f4e83f4ca34c1b\node_modules\@livekit\agents-plugin-openai\src\realtime\realtime_model.ts:2246:13)
at createItem (C:\Projects\myproject\livekit-agent\node_modules\.pnpm\@livekit+agents-plugin-open_9161abf43acd1ea4b5f4e83f4ca34c1b\node_modules\@livekit\agents-plugin-openai\src\realtime\realtime_model.ts:750:21)
at RealtimeSession.createChatCtxUpdateEvents (C:\Projects\myproject\livekit-agent\node_modules\.pnpm\@livekit+agents-plugin-open_9161abf43acd1ea4b5f4e83f4ca34c1b\node_modules\@livekit\agents-plugin-openai\src\realtime\realtime_model.ts:762:13)
at RealtimeSession.updateChatCtx (C:\Projects\myproject\livekit-agent\node_modules\.pnpm\@livekit+agents-plugin-open_9161abf43acd1ea4b5f4e83f4ca34c1b\node_modules\@livekit\agents-plugin-openai\src\realtime\realtime_model.ts:619:33)
at async RealtimeSession._updateSession (C:\Projects\myproject\livekit-agent\node_modules\.pnpm\@livekit+agents@1.7.1_@live_d2e2c21cea33b96d302398e4a0e12667\node_modules\@livekit\agents\src\llm\realtime.ts:210:9)
at async AgentActivity._startSession (C:\Projects\myproject\livekit-agent\node_modules\.pnpm\@livekit+agents@1.7.1_@live_d2e2c21cea33b96d302398e4a0e12667\node_modules\@livekit\agents\src\voice\agent_activity.ts:643:9)
at async AgentActivity.start (C:\Projects\myproject\livekit-agent\node_modules\.pnpm\@livekit+agents@1.7.1_@live_d2e2c21cea33b96d302398e4a0e12667\node_modules\@livekit\agents\src\voice\agent_activity.ts:568:7)
at async runWithContext (C:\Projects\myproject\livekit-agent\node_modules\.pnpm\@livekit+agents@1.7.1_@live_d2e2c21cea33b96d302398e4a0e12667\node_modules\@livekit\agents\src\voice\agent_session.ts:1506:11)
at async Promise.allSettled (index 0)
at async AgentSession._startImpl (C:\Projects\myproject\livekit-agent\node_modules\.pnpm\@livekit+agents@1.7.1_@live_d2e2c21cea33b96d302398e4a0e12667\node_modules\@livekit\agents\src\voice\agent_session.ts:950:5)
}
Also, while having a greeting option is useful, it would be even better to have an option for the AI to start the call directly with the briefing, without waiting for the staff member to speak and without requiring a separate greeting. That was actually my original intention in the previous thread.
@royibernthal that traceback is the whole story, and greetingSpeech is not really the problem.
It fails in AgentSession._startImpl, so the transfer agent’s session never finishes starting. That is why nothing is spoken and why the original agent is still sitting in the room afterwards. The greeting call happens after the dial resolves, and it is wrapped in a try that only logs a warning, so a broken session shows up as silence rather than an error.
The cause is upstream of that. In warm_transfer.ts the transfer session is seeded with:
chatCtx: task.chatCtx.copy(),
A plain copy() keeps agent_config_update items, and the OpenAI realtime plugin has no case for that type, so livekitItemToOpenAIItem hits its fallback throw new Error(“Unsupported item type: …”) at realtime_model.ts:2246. The start fails there.
ChatContext.copy() already takes excludeConfigUpdate, which is exactly what this needs, so copy({ excludeConfigUpdate: true }) looks like the one line fix. Worth confirming with a maintainer whether the exclusion belongs there or whether the realtime plugin should skip unknown item types instead, since arguably both are true.
For your other point, you should get the briefing without a separate greeting once the session actually starts, since the problem is the session never getting off the ground rather than the greeting hook itself.
Thanks for the response. Yes, I imagine the error is responsible for at least part of this weird behavior. It seems like a bug that should probably be fixed internally. Let’s see what the maintainers say.
As for my other point - unfortunately, that wouldn’t be the case. It’s worth reading the previous thread I referred to.
Worth splitting these so they don’t both get stuck.
The crash is a real bug and a forum thread probably won’t get it patched. Since the greetingSpeech work is already in flight in PR #2300, the fastest route is to comment there with your traceback. Whoever wrote that PR is the person touching this code right now, and they’ll immediately see that a realtime model on the transfer leg blows up before the greeting ever runs.
In the meantime, one thing you could try to confirm the diagnosis: run the transfer with an STT-LLM-TTS pipeline instead of a realtime model. The failing line is in the OpenAI realtime plugin’s item conversion, so a normal pipeline shouldn’t hit it. If the greeting fires there, that pins it down and gives you something to work with until it’s fixed.
On the proactive briefing, I haven’t read your earlier thread so I can’t say whether that’s the same cause or a separate limitation. Might be worth restating that part here once the crash is out of the way, otherwise it’ll stay buried under the bug.
I’ve tested it with an STT-LLM-TTS pipeline. No silent error, but exact same behavior. In other words, the weird behavior doesn’t seem to be stemming from that error.
Just left a comment on the PR.
Regarding the proactive briefing - it doesn’t proactively brief by design - it waits for the caller to say something. I was initially suggesting a proactive briefing but instead they decided to implement a proactive greeting. So currently the briefing can either be followed by the agent passively waiting for the user to speak, or by the user reacting to the agent’s proactive greeting.
One thing worth raising with the maintainers: the WarmTransferTask docs still list this as Python only. Under “Available in (BETA)” the Node.js box is unchecked and Python is checked, even though the task itself was merged into agents-js back in June. That would explain both of your symptoms at once rather than as two separate bugs, so it is worth asking directly whether the Node port is considered complete.
The second symptom in particular looks wrong against the documented flow. Step 5 of the warm transfer sequence is the agent leaving so the caller and manager continue on their own. The agent staying in the session and getting in the way is not intended behaviour, so that deserves its own report regardless of what happens with the greeting.
Small thing on the PR while you are talking to the author: the branch line shows it merging into chenghao/feat/AJS-473-caller-hangup-speech rather than main. Stacked PRs sometimes reach releases later than expected, so it may be worth confirming with them whether the version you tested actually contains that code.
I’m pretty sure the AI wasn’t staying in the session before this fix, which would leave me to believe this PR has already been released (unless there was some unrelated regression), but I might be misremembering.
I recall a maintainer saying that WarmTransferTask appears in some of the Node docs but not others, they’re not all aligned and up to date.
Would be great to have a maintainer join the conversation.
Fair on the docs, if a maintainer has already said they’re not aligned then that’s a dead end.
Your recollection is worth testing rather than trusting, and it’s cheap to do. You were on v1.6.0 in your first post and moved to 1.7.1. Pin back to 1.6.0, run one transfer, and see whether the agent leaves after the manager is connected. If it leaves on 1.6.0 and stays on 1.7.1, you have a regression with a known range, and that is a far stronger report than “it doesn’t work.” The package CHANGELOG between those two releases will also tell you directly whether the greeting changeset landed, which answers the released-or-not question without guessing.
Worth considering that these may be one bug rather than two. The documented sequence is that the agent introduces the two parties and then leaves. If greeting logic sits in that window and either throws or awaits something that never settles, you would get silence and an agent that never proceeds to the leave step. That fits both symptoms and it fits your sense that the agent used to leave correctly. I haven’t read the implementation so this is a hypothesis, but it points at a specific code path rather than two unrelated ones.
If the version test confirms a regression, I’d add it to your existing PR comment rather than waiting here. The author is actively in that code, and a reproducible before-and-after on two published versions tends to get a response faster than a forum thread does.