@CWilson - this is happening everyday and is affecting 1% of calls everyday. Can you please suggest what we can do here?
Another help needed from you - do you still see workers reached max dispatch count log - any capacity issues that you notice still, we added a fix yesterday, so wanted to check once
@CWilson - any recommendation on what we can do to mitigate this? This started happening after we upgraded our livekit/agents version to 1.4.6 and livekit (which includes livekit-rtc) to 1.1.2.
Appreciate your help.
I saw some PR in the past week or so that may help. Try the latest version and see if that helps.
I would appreciate if the LiveKit team doesn’t hand wave these issues and ask us to upgrade every time we run into network issues. This has happened to us twice now - Participant Migration and State Mismatch issues in certain SIP calls · Issue #4705 · livekit/agents · GitHub. It was mentioned that all network related issues will be resolved in 1.4.2
But now this is being claimed as we need to upgrade to a new latest version, without being specific on what the issue is and what fix has been made to resolve this
The only thing I can say is that we are trying to debug issues and fix what we believe may be wrong in the code side. Not sure how you can get those fixes if you don’t upgrade.
There is another issue I posted last week that I don’t believe has been pushed to the latest release yet but there is a PR for it. It will also require an update once it becomes available.
The source code is available, so you can review it and see why the issues are happening. Feel free to submit PRs if you find an issue.
01:59:39 RM_h6BiM7yDYd2B
16:08:06 RM_YoTqjYV6L2W7
16:08:09 RM_pyfSxbSMUNtr
16:11:38 RM_viqZWyBwprUv
16:12:05 RM_hYLfqi3N2cLz
16:13:26 RM_fmWoHHuddG3Y
16:14:34 RM_Hf4is9dcsqfZ
16:15:28 RM_5w8mMZmsThaV
16:17:55 RM_RZy6ZSd35Ttt
16:17:57 RM_RqxqgzSMGZxX
17:28:51 RM_tMvrdWvsELkC
17:29:05 RM_CUcFx3ZTWDK7
17:29:10 RM_AXFFcirDv3NB
22:57:47 RM_fKDqq555uH9m
@CWilson - these are the rooms where we faced the issue again - does looking through logs on your end help find the root cause. Timestamps in UTC
I have gone through some of the rust code and there was some major overhaul on signal connection piece in 1.1.2 version of the rust package. But unable to quite make sense of why this only happens for certain calls and not others
I still feel the issue is related to the K8s setup. But these are some of the changes since the version you are running. Several of them looks to me like they could be related to the issue you are having:
Covers the livekit-agents framework, the Python SDK (livekit-rtc), and the Rust SDK (livekit-ffi, the FFI that backs livekit-rtc). Sorted newest first. livekit-protocol (Python) and livekit-api (Python) had no fix releases in this window.
| Date | Library | Version | Fix |
|---|---|---|---|
| 2026-06-12 | livekit-agents | 1.6.0 | fix(neuphonic): use configured encoding and lang_code in TTS requests (#6064) |
| 2026-06-11 | livekit-agents | 1.6.0 | fix(google): correct pitch parameter type from int to float in TTS (#6058) |
| 2026-06-11 | livekit-agents | 1.6.0 | fix(bargein): error when no interruption threshold is known (#6034) |
| 2026-06-11 | livekit-agents | 1.6.0 | fix(deepgram): use stored language when validating model in update_options (#6041) |
| 2026-06-11 | livekit-agents | 1.6.0 | fix(voice): route and drain AsyncToolset executors correctly on handoff (#6049) |
| 2026-06-11 | livekit-agents | 1.6.0 | fix(aws): recover Nova Sonic system instability errors (#6042) |
| 2026-06-10 | livekit-agents | 1.6.0 | fix(elevenlabs): end server vad turns (#5872) |
| 2026-06-10 | livekit-agents | 1.6.0 | fix(sarvam): emit speech timing for STT metrics (#5995) |
| 2026-06-09 | Rust SDK (livekit-ffi) | 0.12.63 | Reject oversized data messages before they break the data channel (+dashmap v6, per-publication video encoder backend selection) |
| 2026-06-08 | livekit-agents | 1.6.0 | fix: resolve shutdown futures when IPC reader stops (#6003) |
| 2026-06-08 | livekit-agents | 1.6.0 | fix(google): preserve request timeout with http options (#5976) |
| 2026-06-08 | livekit-agents | 1.6.0 | fix(openai): handle string realtime status details (#5994) |
| 2026-06-07 | livekit-agents | 1.6.0 | fix(google): require compatible google-genai version (#5988) |
| 2026-06-05 | livekit-agents | 1.6.0 | fix quick reconnect participant keyerror (#5979) |
| 2026-06-05 | livekit-agents | 1.6.0 | fix(openai): skip empty messages in realtime chat updates (#5952) |
| 2026-06-04 | livekit-rtc (Python) | 1.1.10 | fix: unsubscribe FFI queue when the room listen task ends (#699) |
| 2026-06-04 | livekit-agents | 1.6.0 | fix(ipc): rate-limit and contextualize high-memory warnings (#5962) |
| 2026-06-03 | livekit-agents | 1.6.0 | fix(job): log unhandled exceptions in fire-and-forget tasks (#5961) |
| 2026-06-03 | Rust SDK (livekit-ffi) | 0.12.62 | Add rpc max_round_trip_latency / builder pattern (#1127); allow(dead_code) cleanup (#1128); send publisher offer with join request to accelerate connection (#996) |
| 2026-06-02 | livekit-agents | 1.5.17 | fix(inference): guard agent sid header against unconnected room (#5947) |
| 2026-06-02 | livekit-agents | 1.5.17 | fix(inference): don’t leak mock participant sid into request headers (#5943) |
| 2026-06-02 | livekit-agents | 1.5.17 | fix(amd): start listening on lost publisher, forward realtime transcripts, harden attribute wait (#5918) |
| 2026-06-02 | livekit-agents | 1.5.17 | fix examples deploy (#5939) |
| 2026-06-02 | livekit-agents | 1.5.17 | fix(recorder): prevent close hang and fix corrupt frame splits (#5926) |
| 2026-06-02 | livekit-agents | 1.5.17 | fix(job): close RecorderIO at session end if session close was interrupted (#5927) |
| 2026-06-01 | livekit-agents | 1.5.16 | fix(ipc): retry proc acquisition when all in-flight spawns fail (#5874) |
| 2026-05-30 | livekit-agents | 1.5.15 | fix(llm): make to_responses_fnc_ctx.provider_tool_type optional (#5892) |
| 2026-05-30 | livekit-agents | 1.5.16 | fix(examples): raise ToolError instead of returning it in realtime_joke_teller (#5904) |
| 2026-05-30 | livekit-agents | 1.5.16 | fix(examples): check anam_avatar_id in ANAM_AVATAR_ID guard (#5901) |
| 2026-05-30 | livekit-agents | 1.5.16 | fix(workflows): bind GetDtmfTask user-state handler to user_state_changed (#5900) |
| 2026-05-29 | livekit-agents | 1.5.15 | fix(openai realtime): honor OPENAI_BASE_URL env var fallback (#5895) |
| 2026-05-29 | livekit-agents | 1.5.15 | fix(llm): serialize all provider tools for the Responses API + log server-side tool execution (#5865) |
| 2026-05-29 | livekit-agents | 1.5.15 | fix(aws): flatten tool blocks when toolConfig is omitted (#5850) |
| 2026-05-29 | livekit-agents | 1.5.15 | fix(deps): relax bithuman pin to <3 for SDK 2.x compatibility (#5882) |
| 2026-05-29 | livekit-agents | 1.5.15 | fix(llm): sort function tools to make order an invariant (#5884) |
| 2026-05-29 | livekit-agents | 1.5.15 | fix(amd): defer no-speech timer until SIP call is answered (#5848) |
| 2026-05-29 | livekit-agents | 1.5.15 | fix(ipc): bound AgentSession.aclose() during job shutdown (#5875) |
| 2026-05-29 | Rust SDK (livekit-ffi) | 0.12.61 | bump protocol to v1.46.4 (#1121) |
| 2026-05-28 | livekit-agents | 1.5.14 | fix(soniox): surface STT server errors (#5864) |
| 2026-05-27 | livekit-agents | 1.5.14 | Fix ElevenLabs websocket context id handling (#5813) |
| 2026-05-27 | livekit-agents | 1.5.14 | fix(llm): preserve Field() constraints on function tool arguments (#5861) |
| 2026-05-27 | livekit-agents | 1.5.14 | fix(elevenlabs): always close TTS stream context on cancellation (#5845) |
| 2026-05-27 | livekit-agents | 1.5.14 | fix(voice): reset user turn tracker on clear_user_turn (#5857) |
| 2026-05-27 | livekit-agents | 1.5.14 | fix(voice): block on_user_turn_exceeded during agent handoff (#5858) |
| 2026-05-26 | livekit-agents | 1.5.14 | fix(voice): restrict stt pipeline reuse to default stt_node (#5803) |
| 2026-05-25 | livekit-agents | 1.5.13 | fix(plugins/sarvam): thread language_probability into SpeechData.confidence (#5830) |
| 2026-05-25 | livekit-agents | 1.5.13 | fix(anthropic): recreate stream on retry (#5820) |
| 2026-05-25 | livekit-agents | 1.5.14 | fix: return function argument errors (ToolError) to LLM (#5846) |
| 2026-05-25 | livekit-agents | 1.5.14 | fix(core): replace frame drop with silence frame AGT-2914 (#5815) |
| 2026-05-24 | livekit-agents | 1.5.13 | fix: 4xx errors should not be retryable (#5831) |
| 2026-05-24 | livekit-agents | 1.5.13 | fix(lemonslice): bind avatar audio output before the upstream session HTTP call (#5837) |
| 2026-05-24 | livekit-agents | 1.5.13 | fix(llm): convert per-turn instructions on the very first turn too (#5828) |
| 2026-05-24 | livekit-agents | 1.5.13 | fix(voice): propagate ChatMessage.interrupted through proto serializer (#5824) |
| 2026-05-22 | livekit-agents | 1.5.13 | fix(voice): wait for end-of-turn task when waiting for user inactivity (#5792) |
| 2026-05-21 | livekit-agents | 1.5.13 | fix(vad): add support for vad reset directly without stream close (#5687) |
| 2026-05-21 | livekit-agents | 1.5.13 | fix(voice): race between flush and clear_buffer on interrupt leaks unplayed transcript (#5798) |
| 2026-05-21 | Rust SDK (livekit-ffi) | 0.12.60 | Filter internal data streams out of livekit-ffi interface (#1112) |
| 2026-05-20 | livekit-agents | 1.5.12 | fix(interruption): incorrect guard skips true interruptions (#5787) |
| 2026-05-20 | Rust SDK (livekit-ffi) | 0.12.59 | Always emit Disconnected on engine close (#1096); FFI room event ready signal after initial connection (#1068); support large RPC messages via data streams (#1013) |
| 2026-05-19 | livekit-rtc (Python) | 1.1.9 | Strict type checking fixes (#670) |
| 2026-05-19 | livekit-rtc (Python) | 1.1.9 | Fix bug for FrameCryptor and E2E test for E2EE. (#656) |
| 2026-05-19 | livekit-agents | 1.5.11 | fix(cerebras): require openai>=2.16.0 for FinalRequestOptions.content (#5773) |
| 2026-05-19 | livekit-agents | 1.5.12 | fix(soniox): plugin reliability fixes (#5770) |
| 2026-05-18 | livekit-agents | 1.5.10 | fix(deepgram): add per-message recv timeout to TTS WebSocket (#5756) |
| 2026-05-18 | Rust SDK (livekit-ffi) | 0.12.58 | Don’t fire local_track_subscribed during reconnect (#1099); fix LocalTrackPublished handle leak (#1065); return EOS event from data track stream read request |
| 2026-05-15 | livekit-agents | 1.5.10 | fix(barge-in): suppress session-level barge-in errors (#5727) |
| 2026-05-15 | livekit-agents | 1.5.10 | fix(dep): pin bithuman (#5743) |
| 2026-05-15 | livekit-agents | 1.5.10 | fix(ipc): run shutdown callbacks when entrypoint raises (#5741) |
| 2026-05-14 | livekit-agents | 1.5.9 | fix: raise on unexpected ElevenLabs websocket close (#5729) |
| 2026-05-14 | livekit-agents | 1.5.10 | fix(voice): cancel realtime generation when speech is interrupted (#5703) |
| 2026-05-14 | livekit-agents | 1.5.10 | fix: surface Deepgram TTS websocket errors (#5728) |
| 2026-05-14 | Rust SDK (livekit-ffi) | 0.12.57 | Add scalability mode for AV1/VP9 (#1076); LIVEKIT_PREFERRED_HW_ENCODER selection; reword audio filter logs (#1092) |
| 2026-05-13 | livekit-rtc (Python) | 1.1.8 | fix: point rust-sdks to livekit-ffi/v0.12.56 (#662) |
| 2026-05-13 | livekit-agents | 1.5.9 | fix: do not republish tracks on reconnect (#5698) |
| 2026-05-13 | livekit-agents | 1.5.9 | fix(agents): clear pending auto tool reply future on timeout (#5725) |
| 2026-05-13 | livekit-agents | 1.5.9 | fix(agents): await realtime auto tool reply in RunResult (#5702) |
| 2026-05-13 | livekit-agents | 1.5.9 | fix(core): surface real http_context error from STT streams (#5709) |
| 2026-05-12 | livekit-agents | 1.5.9 | fix(examples): consistent CSV schema in survey_agent (#5689) |
| 2026-05-12 | livekit-agents | 1.5.9 | fix(anthropic): raise default httpx read timeout for streaming; add configurable timeout param (#5529) |
| 2026-05-11 | livekit-rtc (Python) | 1.1.8 | fix: type errors with republished event (#657) |
| 2026-05-11 | livekit-agents | 1.5.9 | fix(core): make default user span time explicit (#5699) |
| 2026-05-11 | Rust SDK (livekit-ffi) | 0.12.56 | Sync inner.enabled state for E2EE manager (#1073); upgrade protocol to v1.45.8 |
| 2026-05-11 | Rust SDK (livekit-ffi) | 0.12.55 | Add LocalTrackRepublished event for FFI clients (#1072) |
| 2026-05-10 | livekit-agents | 1.5.9 | fix: pass API key via header in Neuphonic and Murf WebSocket TTS (#5691) |
| 2026-05-10 | Rust SDK (livekit-ffi) | 0.12.54 | Fix missing libwebrtc.jar for Android; derive simulcasted from non-deprecated TrackInfo fields (#1052); fix race in download_webrtc (#1047); SimulateScenario through FFI (#1069); TEL-464 reduce redundant resampling in audio filter (#1019) |
| 2026-05-09 | livekit-agents | 1.5.9 | fix(agents): persist _speech_start_time across intra-turn VAD bursts (#5585) |
| 2026-05-08 | livekit-agents | 1.5.9 | fix(core): clean up variables when committing a user turn manually (#5671) |
| 2026-05-07 | livekit-agents | 1.5.9 | fix(voice): prevent scheduling deadlock when pipeline task crashes (#5678) |
| 2026-05-07 | livekit-agents | 1.5.9 | fix(elevenlabs): restore chunk_length_schedule in WS init payload (#5006) |
| 2026-05-07 | livekit-agents | 1.5.9 | fix: wording in missing required argument error (#5659) |
| 2026-05-06 | livekit-agents | 1.5.9 | Fix the memory leak of Google STT when no audio input. (#5591) |
| 2026-05-06 | livekit-agents | 1.5.9 | fix(console): set mock participant state to active (#5645) |
| 2026-05-06 | livekit-agents | 1.5.9 | fix(async-toolset): use underscore in synthetic call_id (#5656) |
| 2026-05-06 | livekit-agents | 1.5.9 | fix(amd): fix negative zero in amd delay calculation (#5650) |
| 2026-05-05 | livekit-agents | 1.5.9 | fix(fishaudio): prevent TTS generation hang (#5649) |
| 2026-05-04 | livekit-agents | 1.5.8 | fix: end Runway realtime sessions on shutdown (#5623) |
| 2026-05-04 | livekit-agents | 1.5.8 | fix(amd): reset timer for late stt transcript (#5637) |
| 2026-05-04 | livekit-agents | 1.5.8 | fix(openai realtime): reject pending response future on error event (#5576) |
| 2026-05-03 | livekit-agents | 1.5.8 | fix(observability): retry session recording upload (#5627) |
| 2026-05-03 | livekit-agents | 1.5.8 | fix: reduce overly eager call ending behavior (#5630) |
| 2026-05-03 | livekit-agents | 1.5.8 | fix(amd): missing stt start (#5633) |
| 2026-05-03 | livekit-agents | 1.5.8 | Fix Missing user message metrics (MetricsReport) due to early returns in _user_turn_completed_task and no initialization in on_end_of_turn (#5437) |
| 2026-05-02 | livekit-agents | 1.5.8 | fix(aws): wait for stream ready before sending audio start event (#5626) |
| 2026-05-01 | livekit-agents | 1.5.8 | fix(warm_transfer): don’t fall back to env var when sip_connection is set (#5619) |
| 2026-05-01 | livekit-agents | 1.5.8 | fix(amd): amd improvement (AGT-2777) (#5584) |
| 2026-04-30 | livekit-agents | 1.5.7 | fix(inference): skip unknown message warning and rename event name (#5614) |
| 2026-04-30 | livekit-agents | 1.5.7 | fix: cancel Runway realtime sessions on shutdown (#5612) |
| 2026-04-30 | livekit-agents | 1.5.7 | fix: clear stale paused speech state across generation steps (#5594) |
| 2026-04-30 | livekit-agents | 1.5.7 | fix(elevenlabs): use audio_format query param for STT realtime (#5574) |
| 2026-04-29 | livekit-agents | 1.5.7 | fix(recorder): use libopus when possible (#5579) |
| 2026-04-28 | livekit-agents | 1.5.7 | fix(mistral): use conversations API statelessly (#5586) |
| 2026-04-28 | livekit-agents | 1.5.7 | fix(async_toolset): respect allow_interruptions when cancelling tool calls (#5570) |
| 2026-04-28 | livekit-agents | 1.5.7 | fix(gemini live): use parameters instead of parameters_json_schema for raw schema function tools (#5560) |
| 2026-04-28 | livekit-agents | 1.5.7 | fix(smallestai): use close_stream signal to properly terminate STT session (#5562) |
| 2026-04-26 | livekit-agents | 1.5.7 | fix: don’t swallow _ExitCli during shutdown (#5519) |
| 2026-04-25 | livekit-agents | 1.5.7 | fix(soniox): emit PREFLIGHT_TRANSCRIPT for preemptive LLM generation (#5553) |
| 2026-04-24 | livekit-agents | 1.5.7 | fix(voice): pause output when user starts speaking during thinking (#5535) |
| 2026-04-23 | livekit-agents | 1.5.7 | Fix realtime reply generation after interruption (#5526) |
| 2026-04-23 | livekit-agents | 1.5.7 | fix(transcription): seed _start_wall_time fallback in aclose (#5532) |
| 2026-04-23 | livekit-agents | 1.5.7 | fix(openai): forward session.update on RealtimeModel.update_options (#5531) |
| 2026-04-23 | Rust SDK (livekit-ffi) | 0.12.53 | Add device-info telemetry (#982); fix data track packet format breaking E2EE; fix unbound send queue latency in data track messages (#1032); fix raw stream drop from non-tokio thread (#1016) |
| 2026-04-22 | livekit-agents | 1.5.6 | fix: clear _hist buffer in MovingAverage.reset() to prevent stale averages (#5522) |
| 2026-04-22 | livekit-agents | 1.5.7 | fix(cartesia): Move API key from Query Params to Headers (#5516) |
| 2026-04-20 | livekit-agents | 1.5.5 | fix(room-io): ownership-aware FrameProcessor lifecycle management (#5467) |
| 2026-04-18 | livekit-agents | 1.5.5 | fix: strip markdown emphasis adjacent to punctuation (#5481) |
| 2026-04-17 | livekit-agents | 1.5.5 | fix(telemetry): bound OTel provider shutdown to avoid watchdog kills (#5471) |
| 2026-04-17 | livekit-agents | 1.5.5 | fix(tests): update preemptive_generation mock to use dict (#5468) |
| 2026-04-16 | livekit-agents | 1.5.4 | fix(voice): add PreemptiveGenerationOptions for fine-grained control (#5428) |
| 2026-04-16 | livekit-agents | 1.5.5 | fix(examples): update e2ee.py to use encryption kwarg and env var (#5469) |
| 2026-04-15 | livekit-rtc (Python) | 1.1.6 | Fix E2EE connection failure caused by missing required protobuf fields key_ring_size and key_derivation_function in KeyProviderOptions. (#628) |
| 2026-04-15 | livekit-agents | 1.5.3 | fix(liveavatar): wait for connected state and chunk audio before sending (#5453) |
| 2026-04-14 | livekit-agents | 1.5.3 | Fix FrameProcessor lifecycle for selector based noise cancellation (#5433) |
| 2026-04-14 | livekit-agents | 1.5.3 | fix: allow multiple AsyncToolsets by deduplicating management tools (#5369) |
| 2026-04-13 | livekit-agents | 1.5.3 | fix: empty transcript blocks commit_user_turn until timeout (#5429) |
| 2026-04-12 | livekit-agents | 1.5.3 | fix(inworld): do not leak connections when when cancelled (#5427) |
| 2026-04-11 | livekit-agents | 1.5.3 | fix(aws): unwrap doubly-encoded JSON tool arguments from Nova Sonic (#5411) |
| 2026-04-11 | livekit-agents | 1.5.3 | fix(tests): update drive thru instructions (#5405) |
| 2026-04-11 | livekit-agents | 1.5.3 | fix(voice): block new user turns immediately on update_agent() to prevent transition delay (#5396) |
| 2026-04-11 | livekit-agents | 1.5.3 | fix(google): capture usage_metadata before early continues in streaming (#5404) |
| 2026-04-10 | livekit-agents | 1.5.3 | fix(llm): handle double-encoded JSON tool arguments from providers (#5409) |
| 2026-04-10 | livekit-agents | 1.5.3 | fix: AgentTask deadlock when on_enter awaits generate_reply that triggers another AgentTask (#5377) |
| 2026-04-09 | livekit-agents | 1.5.3 | fix(openai): prepend session instructions in realtime generate_reply (#5394) |
| 2026-04-08 | livekit-agents | 1.5.2 | fix: minimax optional dep not bumped by update_versions.py (#5392) |
| 2026-04-07 | livekit-agents | 1.5.2 | fix: say() with missing audio file hangs forever and blocks speech queue (#5358) |
| 2026-04-07 | livekit-agents | 1.5.2 | Fix RoomIO teardown listener cleanup (#5357) |
| 2026-04-07 | livekit-agents | 1.5.2 | fix xAI realtime update chat ctx (#5320) |
| 2026-04-06 | livekit-agents | 1.5.2 | fix(smallestai): add lightning-v3.1 endpoint routing (#5330) |
| 2026-04-06 | livekit-agents | 1.5.2 | fix(sarvam): sync missing API params, fix value ranges, and update models (#5347) |
| 2026-04-05 | livekit-agents | 1.5.2 | fix: add PARTICIPANT_KIND_CONNECTOR to default participant kinds (#5339) |
| 2026-04-03 | livekit-agents | 1.5.2 | fix: wait_for_participant waits until participant is fully active (#5271) |
| 2026-04-03 | livekit-agents | 1.5.2 | fix: convert oneOf to anyOf in strict schema for discriminated unions (#5324) |
| 2026-04-03 | livekit-agents | 1.5.2 | fix(voice): make function call history preservation configurable in AgentTask (#5288) |
| 2026-04-03 | livekit-agents | 1.5.2 | fix(cartesia): handle flush_done message in TTS _recv_task (#5321) |
| 2026-04-02 | livekit-agents | 1.5.2 | fix(openai realtime): support per-response tool_choice in realtime sessions (#5211) |
| 2026-04-02 | Rust SDK (livekit-ffi) | 0.12.52 | Use bounded buffer for video stream (#956) |
| 2026-04-01 | livekit-agents | 1.5.2 | fix is_context_type for generic RunContext types (#5307) |
| 2026-04-01 | livekit-agents | 1.5.2 | fix(core): reduce TTS output buffering latency (#5292) |
| 2026-04-01 | livekit-agents | 1.5.2 | fix(cli): prevent api_key/api_secret from leaking in tracebacks (#5300) |
| 2026-04-01 | livekit-agents | 1.5.2 | fix(core): fix BackgroundAudioPlayer.play() hanging indefinitely (#5299) |
| 2026-03-31 | livekit-agents | 1.5.2 | fix(core): reset user state to listening when audio is disabled (#5198) |
| 2026-03-31 | Rust SDK (livekit-ffi) | 0.12.51 | Expose participant active event/state/joined_at; fix Unity android build with ‘livekit’ prefixed jni (#983); upgrade to thiserror 2 |
| 2026-03-30 | livekit-agents | 1.5.2 | fix: catch TimeoutError from drain() so aclose() always runs (#5282) |
| 2026-03-30 | livekit-agents | 1.5.2 | fix(stt): reset VAD when STT sends EOT (#5095) |
| 2026-03-30 | livekit-agents | 1.5.2 | fix(recorder): guard against empty agent speech frames (#5279) |
| 2026-03-30 | livekit-agents | 1.5.2 | fix(openai): close current generation channels on realtime reconnect (#5276) |
| 2026-03-30 | livekit-agents | 1.5.2 | fix: prevent AttributeError in ThreadJobExecutor.logging_extra() (#5277) |
| 2026-03-30 | livekit-agents | 1.5.2 | fix: prevent CancelledError from propagating to unrelated Tee peers (#5273) |
| 2026-03-30 | livekit-agents | 1.5.2 | fix: only start session host when it’s primary session (#5241) |
| 2026-03-29 | livekit-agents | 1.5.2 | fix: prevent TTS retry after partial audio and replay input on retry (#5242) |
| 2026-03-27 | livekit-agents | 1.5.2 | Fix/stt fallback adapter propagate aligned transcript (#5237) |
| 2026-03-27 | livekit-agents | 1.5.2 | fix: expose endpointing_opts in AgentSession.update_options() (#5243) |
| 2026-03-27 | livekit-agents | 1.5.2 | fix(utils): improve type annotation for deprecate_params decorator (#5244) |
| 2026-03-27 | livekit-agents | 1.5.2 | fix: ensure MCP client enter/exit run in the same task (#5223) |
| 2026-03-26 | livekit-agents | 1.5.2 | fix: Nova Sonic interactive context bugs and dynamic tool support (#5220) |
| 2026-03-25 | livekit-agents | 1.5.2 | fix(test): replace oai with deepgram and fix broken tests (#5225) |
| 2026-03-25 | livekit-agents | 1.5.2 | fix(utils): preserve type annotations in deprecate_params (#5200) |
| 2026-03-25 | livekit-agents | 1.5.2 | fix(cli): update log level width in console mode (#5224) |
| 2026-03-25 | livekit-agents | 1.5.2 | fix(llm): surface validation error details to LLM on function call argument failures (#5193) |
| 2026-03-25 | livekit-agents | 1.5.2 | fix: allow codec format specification via the user for Sarvam TTS (#5209) |
| 2026-03-24 | livekit-agents | 1.5.2 | fix: pass prometheus_multiproc_dir in from_server_options initialization (#5195) |
| 2026-03-23 | livekit-agents | 1.5.1 | fix: skip redundant realtime events in OpenAI plugin (#5204) |
| 2026-03-23 | livekit-agents | 1.5.1 | fix: expose Chirp 3 google STT endpoint sensitivity (#5196) |
| 2026-03-22 | Rust SDK (livekit-ffi) | 0.12.50 | Fix H.264 codec matching; bounded buffer for audio_stream; send client os/os_version (#952) |
| 2026-03-20 | livekit-rtc (Python) | 1.1.3 | fix: correct error handling in process_reverse_stream (#588) |
| 2026-03-20 | livekit-agents | 1.5.1 | fix(core): restore chat topic support in room IO (#5181) |
| 2026-03-20 | livekit-agents | 1.5.1 | fix(turn-detector): relax transformers upper bound to allow 5.x (#5174) |
| 2026-03-20 | livekit-agents | 1.5.1 | fix(core): version mismatch due to bad merge (#5176) |
| 2026-03-19 | livekit-agents | 1.5.0 | fix(11labs): add empty keepalive message and remove final duplicates (#5139) |
| 2026-03-19 | livekit-agents | 1.5.1 | fix azure openai realtime support & add realtime models tests (#5168) |
| 2026-03-18 | livekit-agents | 1.5.0 | Fix Phonic generate_reply to resolve with the current GenerationCreatedEvent (#5147) |
| 2026-03-18 | livekit-agents | 1.5.0 | Fix(inworld): “Context not found” errors caused by invalid enum parameter types (#5153) |
| 2026-03-18 | livekit-agents | 1.5.0 | fix(realtime): sync remote items to local chat_ctx with placeholders to prevent in-flight deletion (#5114) |
| 2026-03-18 | livekit-agents | 1.5.0 | fix: update_instructions() now reflected in tool call response generation (#5072) |
| 2026-03-18 | livekit-agents | 1.5.0 | fix: preserve ‘type’ field in TaskGroup JSON schema enum items (#5073) |
| 2026-03-17 | livekit-agents | 1.5.0 | fix: strip empty {} entries from anyOf/oneOf in strict JSON schema (#5137) |
Are you referring to the k8s setup that we have? It is a plain k8s cluster with normal pods on bottlerocket
LiveKit fixes since livekit-agents 1.4.6 (2026-03-16 → 2026-06-15)
Can you point to the PRs that would help resolve the issue we are seeing? Without being specific, I don’t think there is a definitive answer that will solve these network issues.
I don’t know the PR# I am just looking at the change log
RM_inoeUnCnAUAr
RM_e4wjtVmh74PD
RM_jkW89NpKS7NF
RM_CDGWVwZqR62D
RM_fqfjAjff7DMs
RM_eQRcri2MdS8G
RM_PwWZbRuv9T7e
RM_ivzNxeGBPqRb
@CWilson - we aer facing this everyday now. Room Ids above where we faced this today. Will this PR solve this issue too - Server-initiated migration fails to resume on agents 1.4.6 — subscriber + publisher PC fail, no recovery, process killed (expected fixed in >1.4.2 per agents #4705) - #10 by CWilson
I took a look. My understanding is that the issue in this topic relates to a delayed agent join after it has been dispatched. I see this same behaviour in the logs for RM_inoeUnCnAUAr.
The other topic, Server-initiated migration fails to resume on agents 1.4.6 — subscriber + publisher PC fail, no recovery, process killed (expected fixed in >1.4.2 per agents #4705) - #10 by CWilson , when I compare the server logs for those rooms, I see no delay between the agent being dispatched and the agent joining (beyond what would be expected in normal operations). So, I believe these are different issues, but I defer to @CWilson as he has looked at this closer than me.
⏺ ┌────────────┬─────────────────┬─────────────────┬──────────┐
│ time (UTC) │ room_id │ job_id │ v0 fails │
├────────────┼─────────────────┼─────────────────┼──────────┤
│ 23:21:27 │ RM_DKMhvZxqgv5b │ AJ_gL9RTbVnU8ns │ 1 │
├────────────┼─────────────────┼─────────────────┼──────────┤
│ 23:21:33 │ RM_pX4tNGhoeZ5k │ AJ_LnBnd9TQpgoC │ 1 │
├────────────┼─────────────────┼─────────────────┼──────────┤
│ 23:21:47 │ RM_g22kXojvRxM3 │ AJ_KwaBLBrsQ3ac │ 2 │
├────────────┼─────────────────┼─────────────────┼──────────┤
│ 23:21:58 │ RM_b9d7MqDtydVU │ AJ_KfjQaF32iqjn │ 1 │
└────────────┴─────────────────┴─────────────────┴──────────┘
Here are the Room IDs that ran into the issue again with signal connection failed on v0 path: Timeout.
I am sharing these here so that if it helps debug quicker and we can find a better solution.
@CWilson - can you please help escalate this internally and check when this can be resolved, this is causing issues daily now for our calls
Oh, you are still seeing this after you upgraded?
There was a Rust update that improved the connection handling. Let me see where that is.
Looks like it was merged here:
I will check if it is part of an agent release yet.
That agent release that has that Rust fix should be out later today:
and
@CWilson - I believe @darryncampbell mentioned the signal issue won’t be fixed from the full reconnect PR as this might be separate issue. Check here - Signal connection times out on the "v0 path" at agent join, forcing a fallback that adds 0.5–5s of call-setup latency - #31 by darryncampbell
I am sure there must be some confusion. Can you please help clarify the PR - fix: escalate to full reconnect if connection failed during a resume by davidzhao · Pull Request #1175 · livekit/rust-sdks · GitHub once released on agents side and when we upgrade will solve both the problems we are seeing wrt network issues:
signal connection failed issue - this current post - Signal connection times out on the "v0 path" at agent join, forcing a fallback that adds 0.5–5s of call-setup latencySubscriber pc failed - Server-initiated migration fails to resume on agents 1.4.6 — subscriber + publisher PC fail, no recovery, process killed (expected fixed in >1.4.2 per agents #4705)Would appreciate some clarity here.
Can you test and see if it solves your issue? Would appreciate some clarity here if it helps or not.
The problem is I don’t have a way to reproduce this or test this in our test environments or locally.
This occurs only during production and I haven’t found any pattern in our logs. Hence I was sharing the Room Ids daily to see if you can find some pattern on your end. This started occurring when we migrated to 1.4.6 from 1.3.12 version of the livekit/agents sdk
If you can confirm internally both the signal connection issue and the subscriber pc state are interlinked and the PR helps fix the issue, then it becomes an easy decision for us to check and upgrade to latest version
Lets hope the new version solves your issue