Agent Observability set to Disabled, causing errors (inconsistent behaviour)

Agents
1

Hi,

On one of our Cloud environments I have disabled Agent Observability on one of the projects in Livekit Cloud.

With some sessions we still get this error:

2026-02-25T15:12:02.774654966Z stdout F {“message”: “Failed to export logs batch code: 401, reason: \b\u0010\u0012+project data recording is disabled by owner\u001a+\n)type.googleapis.com/google.rpc.BadRequest”, “level”: “ERROR”, “name”: “opentelemetry.exporter.otlp.proto.http._log_exporter”, “room”: "call-

2026-02-25T15:11:37.3881419Z stdout F {“message”: “Failed to export logs batch code: 401, reason: \b\u0010\u0012+project data recording is disabled by owner\u001a+\n)type.googleapis.com/google.rpc.BadRequest”, “level”: “ERROR”, “name”: “opentelemetry.exporter.otlp.proto.http._log_exporter”, “room”: "call-room…

2026-02-25T15:12:27.632020767Z stdout F {“message”: “failed to upload the session report to LiveKit Cloud\nTraceback (most recent call last):\n File “/app/.venv/lib/python3.11/site-packages/livekit/agents/job.py”, line 265, in _on_session_end\n await _upload_session_report(\n File “/app/.venv/lib/python3.11/site-packages/livekit/agents/telemetry/traces.py”, line 505, in _upload_session_report\n resp.raise_for_status()\n File “/app/.venv/lib/python3.11/site-packages/aiohttp/client_reqrep.py”, line 636, in raise_for_status\n raise ClientResponseError(\naiohttp.client_exceptions.ClientResponseError: 401, message=‘Unauthorized’, url=‘******.livekit.cloud/observability/recordings/v0’”;, “level”: “ERROR”, “name”: “livekit.agents”, “exc_info”: "Traceback (most recent call last):\n File “/app/.venv/lib/python3.11/site-packages/livekit/agents/job.py”, line 265, in _on_session_end\n await _upload_session_report(\n File “/app/.venv/lib/python3.11/site-packages/livekit/agents/telemetry/traces.py”, line 505, in _upload_session_report\n resp.raise_for_status()\n File “/app/.venv/lib/python3.11/site-packages/aiohttp/client_reqrep.py”, line 636, in raise_for_status\n raise ClientResponseError(\naiohttp.client_exceptions.ClientResponseError: 401, message=‘Unauthorized’,

and with some sessions we don’t get this at all. Is this somehow related that the setting change needs to propagate in all the Livekit cloud nodes by any chance? We also see this behaviour is not consistent across all sessions?

2

From what I understand that in the AgentSession you have to set record=False if Observability is disabled in Livekit cloud.

3

What version of LiveKit agents are you running?

`pip freeze |grep livekit`

What is your project ID?

This looks to me like it is not a LiveKit Insights error but an OLTP issue. Are you using OpenTelemetry in your agent?

4

@CWilson

We are using livekit agents 1.4.3 (latest) and yes we are using OpenTelemetry integration, but we use it with Grafana, Loki, Tempo ourselves. Could it be that we are publishing to both livekit cloud and our own stack?

After setting record=False the errors are gone though.

Good thing to know. On our DEV and Acceptance environment Observability was disabled, on Production it was enabled, and we disabled it. But I thought maybe this had to do with setting propagation to livekit cloud nodes.

Does this info help?

5

Did you ever have Agent Insights enabled in an environment where you expected it not to have insights? It can take some time for changes to propagate. Also, I believe the value is sent when the agent is deployed, but I’m not 100% sure when it loads. I would need to look at the source code.

6

So yes, our Production project on Livekit cloud, there Agent Observability was enabled and after that I disabled it.

The issue was, that we were getting these errors not consistently, with some sessions we were getting them, and with some sessions we weren’t. That tells me that it wasn’t coming from the agents framework or being caused by it. After I set record=False, it completely stopped.

1 Like