Interruption inference timeout: ', status_code=408

We recently had this exception on a bunch of calls today.

It seems to be related to “adaptive” interruptionMode. What is the exact root cause and what can we do to remediate such errors?

Error in predict
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 539, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/aiohttp/streams.py", line 703, in read
    await self._waiter
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/livekit/agents/inference/interruption.py", line 766, in predict
    async with self._session.post(
               ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/aiohttp/client.py", line 1510, in __aenter__
    self._resp: _RetType = await self._coro
                           ^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/aiohttp/client.py", line 779, in _request
    resp = await handler(req)
           ^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/aiohttp/client.py", line 757, in _connect_and_send_request
    await resp.start(conn)
  File "/app/.venv/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 534, in start
    with self._timer:
         ^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/aiohttp/helpers.py", line 713, in __exit__
    raise asyncio.TimeoutError from exc_val
TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 17, in async_fn_logs
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/livekit/agents/inference/interruption.py", line 809, in predict
    raise APIStatusError(
livekit.agents._exceptions.APIStatusError: message='interruption inference timeout: ', status_code=408, retryable=False