Failed to perform clear buffer rpc (Simli Avatar Integration)

Agents
1

{“message”: “failed to perform clear buffer rpc”, “level”: “ERROR”, “name”: “livekit.agents”, “exc_info”: “Traceback (most recent call last):\n File "****\venv\Lib\site-packages\livekit\agents\voice\avatar\_datastream_io.py", line 179, in _clear_buffer_task\n await self._room.local_participant.perform_rpc(\n File "****\venv\Lib\site-packages\livekit\rtc\participant.py", line 328, in perform_rpc\n raise RpcError._from_proto(cb.perform_rpc.error)\nlivekit.rtc.rpc.RpcError: Connection timeout”, “timestamp”: “2026-02-05T16:08:03.478562+00:00”}
{“time”:“2026-02-05 21:38:03,503”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”}
{“message”: “didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”, “level”: “WARNING”, “name”: “livekit.agents”, “timestamp”: “2026-02-05T16:08:03.503887+00:00”}

I am getting this error in Simli Avatar. Does this error belong to Livekit or Simli service?, or am I doing something bad in simli intergrations?

What I am doing for Simli Intergration in agent session>>
avatar = simli.AvatarSession(

        *simli_config*=simli.SimliConfig(

            *api_key*=os.getenv("SIMLI_API_KEY"),

            *face_id*="b9e5fba3-071a-4e35-896e-211c4d6eaa7b",

            *emotion_id*="7a65257c-25b3-4dc1-889d-ff8a3d51ee01",

        ),

        *avatar_participant_name*="Aditi",

    )



    *await* avatar.start(session, *room*=*ctx*.room)

I did this As documented in LK docs.

Thanks a lot in advance

2

At a glance, with the context you provided, you are doing it correctly.

Is that a persistent error, or do you just see it once in a while?

3

Yes, this error is persistent, and I am getting this error after 2-3 min of conversation, and suddenly get this error, and the agent session stops with this error

4

Can you tell me more about your setup:

  • Where are your agents hosted (Cloud or Self)?
  • What version of LiveKit agents are you running?
  • I need to see more of your log from your agent*
5

Hi Jitendra, Are you seeing any slow tts warning messages?

6
  • It is self-hosted on an Azure VM.
    • Name: livekit-agents Version: 1.3.12
    • Here are the detailed logs for a session : {“time”:“2026-02-08 08:28:08,759”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“RoomInputOptions and RoomOutputOptions are deprecated, use RoomOptions instead”}
      {“time”:“2026-02-08 08:28:08,763”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“RoomIO audio output is enabled but output.audio is already set, ignoring..”}
      {“time”:“2026-02-08 08:28:08,759”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“RoomInputOptions and RoomOutputOptions are deprecated, use RoomOptions instead”}
      {“message”: “RoomInputOptions and RoomOutputOptions are deprecated, use RoomOptions instead”, “level”: “WARNING”, “name”: “livekit.agents”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:28:08.759347+00:00”}
      {“time”:“2026-02-08 08:28:08,763”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“RoomIO audio output is enabled but output.audio is already set, ignoring..”}
      {“message”: “RoomIO audio output is enabled but output.audio is already set, ignoring..”, “level”: “WARNING”, “name”: “livekit.agents”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:28:08.763082+00:00”}
      {“time”:“2026-02-08 08:28:08,954”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“resume_false_interruption is enabled but audio output does not support pause, it will be ignored”}
      {“time”:“2026-02-08 08:28:08,954”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“resume_false_interruption is enabled but audio output does not support pause, it will be ignored”}
      {“message”: “resume_false_interruption is enabled but audio output does not support pause, it will be ignored”, “level”: “WARNING”, “name”: “livekit.agents”, “audio_output”: “RecorderIO”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:28:08.954484+00:00”}
      {“time”:“2026-02-08 08:28:19,389”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:28:19,389”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:28:19.389331+00:00”}
      {“time”:“2026-02-08 08:28:47,873”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:28:47,873”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:28:47.873563+00:00”}
      {“time”:“2026-02-08 08:28:56,307”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”}
      {“time”:“2026-02-08 08:28:56,308”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“playback_finished called more times than playback segments were captured”}
      {“time”:“2026-02-08 08:28:56,307”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”}
      {“message”: “didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”, “level”: “WARNING”, “name”: “livekit.agents”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:28:56.307486+00:00”}
      {“time”:“2026-02-08 08:28:56,308”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“playback_finished called more times than playback segments were captured”}
      {“message”: “playback_finished called more times than playback segments were captured”, “level”: “WARNING”, “name”: “livekit.agents”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:28:56.308275+00:00”}
      {“time”:“2026-02-08 08:29:04,007”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:29:04,007”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:29:04.007423+00:00”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:29:15.396444+00:00”}
      {“time”:“2026-02-08 08:29:15,396”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:29:15,396”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:29:18,999”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:29:18,999”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:29:18.999802+00:00”}
      {“time”:“2026-02-08 08:29:32,048”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:29:32,048”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:29:32.048180+00:00”}
      {“time”:“2026-02-08 08:29:50,554”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:29:50,554”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:29:50.554026+00:00”}
      {“time”:“2026-02-08 08:29:59,090”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:29:59,090”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:29:59.090427+00:00”}
      {“time”:“2026-02-08 08:30:33,890”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:30:33,890”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:30:33.890118+00:00”}
      {“time”:“2026-02-08 08:30:55,223”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:30:55,223”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:30:55.223523+00:00”}
      {“time”:“2026-02-08 08:31:14,150”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”}
      {“time”:“2026-02-08 08:31:14,151”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“playback_finished called more times than playback segments were captured”}
      {“time”:“2026-02-08 08:31:14,150”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”}
      {“message”: “didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”, “level”: “WARNING”, “name”: “livekit.agents”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:31:14.150396+00:00”}
      {“time”:“2026-02-08 08:31:14,151”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“playback_finished called more times than playback segments were captured”}
      {“message”: “playback_finished called more times than playback segments were captured”, “level”: “WARNING”, “name”: “livekit.agents”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:31:14.151096+00:00”}
      {“time”:“2026-02-08 08:31:19,449”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“time”:“2026-02-08 08:31:19,449”,“level”:“INFO”,“module”:“livekit.agents”,“event”:“playback finished event received”}
      {“message”: “playback finished event received”, “level”: “INFO”, “name”: “livekit.agents”, “caller_identity”: “simli-avatar-agent”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:31:19.449132+00:00”}
      {“time”:“2026-02-08 08:31:37,518”,“level”:“ERROR”,“module”:“livekit.agents”,“event”:“speech not done in time after interruption, cancelling the speech arbitrarily.”}
      {“time”:“2026-02-08 08:31:37,518”,“level”:“ERROR”,“module”:“livekit.agents”,“event”:“speech not done in time after interruption, cancelling the speech arbitrarily.”}
      {“message”: “speech not done in time after interruption, cancelling the speech arbitrarily.”, “level”: “ERROR”, “name”: “livekit.agents”, “speech_id”: “speech_5b22195e7fff”, “timeout”: 5.0, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:31:37.518244+00:00”}
      {“time”:“2026-02-08 08:31:40,250”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“_SegmentSynchronizerImpl text marked as ended in capture text, rotating segment”}
      {“time”:“2026-02-08 08:31:40,250”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“_SegmentSynchronizerImpl text marked as ended in capture text, rotating segment”}
      {“message”: “_SegmentSynchronizerImpl text marked as ended in capture text, rotating segment”, “level”: “WARNING”, “name”: “livekit.agents”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:31:40.250467+00:00”}
      {“time”:“2026-02-08 08:31:45,267”,“level”:“ERROR”,“module”:“livekit.agents”,“event”:“speech not done in time after interruption, cancelling the speech arbitrarily.”}
      {“time”:“2026-02-08 08:31:45,267”,“level”:“ERROR”,“module”:“livekit.agents”,“event”:“speech not done in time after interruption, cancelling the speech arbitrarily.”}
      {“message”: “speech not done in time after interruption, cancelling the speech arbitrarily.”, “level”: “ERROR”, “name”: “livekit.agents”, “speech_id”: “speech_436cb9264b02”, “timeout”: 5.0, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:31:45.267120+00:00”}
      {“time”:“2026-02-08 08:31:46,162”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“_SegmentSynchronizerImpl text marked as ended in capture text, rotating segment”}
      {“time”:“2026-02-08 08:31:46,162”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“_SegmentSynchronizerImpl text marked as ended in capture text, rotating segment”}
      {“message”: “_SegmentSynchronizerImpl text marked as ended in capture text, rotating segment”, “level”: “WARNING”, “name”: “livekit.agents”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:31:46.162588+00:00”}
      {“time”:“2026-02-08 08:31:47,700”,“level”:“ERROR”,“module”:“livekit.agents”,“event”:“failed to perform clear buffer rpc”}
      Traceback (most recent call last):
      File “/usr/local/lib/python3.13/site-packages/livekit/agents/voice/avatar/_datastream_io.py”, line 179, in _clear_buffer_task
      await self._room.local_participant.perform_rpc(
      …<3 lines>…
      )
      File “/usr/local/lib/python3.13/site-packages/livekit/rtc/participant.py”, line 328, in perform_rpc
      raise RpcError._from_proto(cb.perform_rpc.error)
      livekit.rtc.rpc.RpcError: Response timeout
      {“time”:“2026-02-08 08:31:47,700”,“level”:“ERROR”,“module”:“livekit.agents”,“event”:“failed to perform clear buffer rpc
      Traceback (most recent call last):
      File “/usr/local/lib/python3.13/site-packages/livekit/agents/voice/avatar/_datastream_io.py”, line 179, in _clear_buffer_task
      await self._room.local_participant.perform_rpc(
      …<3 lines>…
      )
      File “/usr/local/lib/python3.13/site-packages/livekit/rtc/participant.py”, line 328, in perform_rpc
      raise RpcError._from_proto(cb.perform_rpc.error)
      livekit.rtc.rpc.RpcError: Response timeout”}
      Traceback (most recent call last):
      File “/usr/local/lib/python3.13/site-packages/livekit/agents/voice/avatar/_datastream_io.py”, line 179, in _clear_buffer_task
      await self._room.local_participant.perform_rpc(
      …<3 lines>…
      )
      File “/usr/local/lib/python3.13/site-packages/livekit/rtc/participant.py”, line 328, in perform_rpc
      raise RpcError._from_proto(cb.perform_rpc.error)
      livekit.rtc.rpc.RpcError: Response timeout
      {“message”: “failed to perform clear buffer rpc\nTraceback (most recent call last):\n File "/usr/local/lib/python3.13/site-packages/livekit/agents/voice/avatar/_datastream_io.py", line 179, in _clear_buffer_task\n await self._room.local_participant.perform_rpc(\n …<3 lines>…\n )\n File "/usr/local/lib/python3.13/site-packages/livekit/rtc/participant.py", line 328, in perform_rpc\n raise RpcError._from_proto(cb.perform_rpc.error)\nlivekit.rtc.rpc.RpcError: Response timeout”, “level”: “ERROR”, “name”: “livekit.agents”, “exc_info”: “Traceback (most recent call last):\n File "/usr/local/lib/python3.13/site-packages/livekit/agents/voice/avatar/_datastream_io.py", line 179, in _clear_buffer_task\n await self._room.local_participant.perform_rpc(\n …<3 lines>…\n )\n File "/usr/local/lib/python3.13/site-packages/livekit/rtc/participant.py", line 328, in perform_rpc\n raise RpcError._from_proto(cb.perform_rpc.error)\nlivekit.rtc.rpc.RpcError: Response timeout”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:31:47.700991+00:00”}
      {“time”:“2026-02-08 08:31:47,703”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”}
      {“time”:“2026-02-08 08:31:47,703”,“level”:“WARNING”,“module”:“livekit.agents”,“event”:“didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”}
      {“message”: “didn’t receive playback finished event after clear buffer, marking playout as done arbitrarily”, “level”: “WARNING”, “name”: “livekit.agents”, “pid”: 1246, “job_id”: “AJ_RRrsuEu7vqzy”, “room_id”: “RM_Fg8U8Jv52GQM”, “timestamp”: “2026-02-08T08:31:47.703810+00:00”}
7

can I see your tts, stt, llm configuration?

I am suspcious that you have some generation of tts, stt, llm somewhere that is causing this.

I think it s worth looking at your traces.zip and see if you have any slow bit in your component

8

Have you tried one of the other avatar providers to see if it works better for you?

9

Here is the configuration:
def prewarm_fnc(proc: JobProcess):

*proc*.userdata\["vad"\] = silero.VAD.load()

*proc*.userdata\["tts"\] = google.TTS(

    *language*="en-IN",

    *voice_name*="en-IN-Chirp3-HD-Callirrhoe",

    *gender*="female",

    *credentials_file*="credentials.json",

    *speaking_rate*=1,

)

async def entrypoint(ctx: JobContext):

*await* *ctx*.connect()

vad: silero.VAD = *ctx*.proc.userdata\["vad"\]




session = AgentSession(

    *vad*=vad,

    *stt*=deepgram.STT(

        *model*="nova-3", *language*="multi", *punctuate*=False, *profanity_filter*=True

    ),

    *tts*=*ctx*.proc.userdata\["tts"\],

    *llm*=google.LLM(*model*="gemini-2.5-flash-lite", *temperature*=0.5),

    *turn_detection*=MultilingualModel(),

    *min_endpointing_delay*=0.5,

    *max_endpointing_delay*=0.8,

)

avatar = simli.AvatarSession(

        *simli_config*=simli.SimliConfig(

            *api_key*=os.getenv("SIMLI_API_KEY"),

            *face_id*="b9e5fba3-071a-4e35-896e-211c4d6eaa7b",

            *emotion_id*="7a65257c-25b3-4dc1-889d-ff8a3d51ee01",

        ),

        *avatar_participant_name*="Aditi",

    )

    *await* avatar.start(session, *room*=*ctx*.room)

     *await* session.start(

    *room*=*ctx*.room,

    *agent*=starting_agent(*chat_ctx*=initial_ctx),

    *room_input_options*=RoomInputOptions(

        *noise_cancellation*=noise_cancellation.BVCTelephony(),

    ),

    *room_output_options*=RoomOutputOptions(

        *transcription_enabled*=True, *sync_transcription*=True

    ),

)

}

if _name_ == “_main_”:

agents.cli.run_app(

    WorkerOptions(

        *entrypoint_fnc*=entrypoint,

        *prewarm_fnc*=prewarm_fnc,

        *agent_name*="test-agent",

        *load_threshold*=0.9,

        *permissions*=WorkerPermissions(

            *can_publish*=True,

            *can_subscribe*=True,

            *can_publish_data*=True,

            *hidden*=False,

        ),

        *worker_type*=WorkerType.ROOM,

    )

)
10

Yes, I have tried with other providers like Tavus and am not getting this type of Error. Ig it is a bug from Simli Side.

11

After contacting Simli Support i found the issue . And the default max_idle_time value is 30 seconds in the Simli configuration, which is why the avatar left with this “clear buffer error”.

But ideally, it should leave silently with a warning or something, and the agent should be switched to voice-only mode for further conversation. And it should be handled by Livekit, isn’t it ?.