"failed to connect to livekit" warnings

I have livekit deployed to k8s and every now and then I see something like this

exc_info: "Traceback (most recent call last):  File "/ai/.venv/lib/python3.12/site-packages/livekit/agents/worker.py", line 1021, in _connection_task    await self._run_ws(ws)  File "/ai/.venv/lib/python3.12/site-packages/livekit/agents/worker.py", line 1103, in _run_ws    await asyncio.gather(*tasks)  File "/ai/.venv/lib/python3.12/site-packages/livekit/agents/worker.py", line 1075, in _recv_task    raise Exception("worker connection closed unexpectedly")Exception: worker connection closed unexpectedly"

level: "WARNING"

message: "failed to connect to livekit, retrying in 0s"

name: "livekit.agents"

followed by “registered worker”.

  • Is there any way to silence these?
  • Maybe only show them if cannot connect in say 5min?

Is there any way to silence these?
Maybe only show them if cannot connect in say 5min?

No, other than raising the log level to ERROR, but I’m worried there is an underlying issue that is causing this. It might be worth looking for patterns for when you see this warning, and whether it always recovers

  1. how can I raise the log level to error for these messages
  2. ok, I’ll keep an eye but its’ been weeks if not months like this

Why are you getting connection issues in the first place. Is there something wrong with the network stability of the cluster?