New Cloud Agent versions crashloop in eu-central

Agents
, ,
1

Newly created Cloud Agent versions in eu-central are entering CrashLoopBackOff shortly after startup.

We first saw this in development on July 29, 2026, and again in staging and production on August 2. A retained version created on July 24 remains
healthy after rollback, so the issue may have started between July 24–29.

Environment

  • LiveKit Agents 1.6.4
  • LiveKit RTC 1.1.12
  • Python 3.13 with uv
  • Standard command:

uv run “$AGENT_ENTRY” start

The image builds successfully, secrets are present, and the worker registers locally.

Findings

  • sleep 3600 remains healthy, so this is not a universal probe or registration timeout.
  • Starting the agent causes a crashloop, even with all output redirected to a file.
  • lk agent logs returns only the first second of output or nothing.
  • PID 1 is a LiveKit-provided run.sh wrapper with two tee children.
  • The same application boots, initializes, registers, and remains healthy when started two seconds after a helper process with stdout/stderr
    redirected:

helper-process &
sleep 2
uv run “$AGENT_ENTRY” start >/tmp/agent.log 2>&1 &
sleep 3600

We reproduced this twice. A delay without output redirection still fails.

This suggests a startup race or interaction with the Cloud Agents logging wrapper, but we cannot see the container’s termination reason through the
CLI.

Could the LiveKit team check:

  1. The exit reason or signal for the affected containers.
  2. Whether the Cloud Agents runtime changed in eu-central between July 24–29.
  3. Whether there is a safe temporary workaround.

We can provide project, agent and version IDs privately.

2

I see your production agent was created on 24th July, then you pushed an update to it on 2nd August. It looks like you then rolled back to the 24th July deployment shortly after.
My first assumption would be something in your agent between those two versions of the agent code is causing the crash loop.

It might be a good first step to isolate this by redeploying a new agent version whose code matches your 24th July agent - if it’s an issue with your updated code, then I would expect that to work.

Another thing to try is to run the agent in uv run AGENT dev which gives you a lot more logging without having to use the cli, but you would not want to do that with your prod environment.

3

Thanks! We ran your suggested isolation today plus two controls on our staging agent, and it isolated cleanly: it’s not our code delta, it’s the Docker CMD form.

  1. Jul-24 code, exec-form CMD [“uv”,“run”,“demo-agent”,“start”] → healthy, full logs, registered worker in 19s.
  2. Aug-2 code, shell-form CMD [“sh”,“-c”,“exec uv run “$AGENT_ENTRY” start”] → CrashLoop in ~90s, zero retained logs.
  3. Aug-2 code, switched back to exec-form → healthy, still running.

The Jul-24→Aug-2 correlation was a confound: the Aug-2 deploy was also the first time our Dockerfile used a shell-form CMD (introduced for entrypoint indirection). Across all ~19 versions we deployed while debugging, every crashlooper was shell-form and every exec-form survived. Notably, sh -c “exec uv run …” dies even with stdout/stderr fully redirected to a file, and PID 1 in-pod is your run.sh wrapper with two tee children, so our guess is the wrapper mishandles its direct child exec()ing into a different binary. (uv run dev was already clean locally; this only reproduces inside the platform harness.)

We’re switching to exec-form CMD as the fix. Could you confirm whether shell-form CMDs are supported in Cloud Agents (or document that they aren’t), and check the termination events on your side, the crashlooping version was killed repeatedly ~11:18–11:22Z today (eu-central), directly followed by the healthy exec-form deploy of the same code?

4

Glad you found the root cause. At the moment we say this:

Startup command: Provide a fixed ENTRYPOINT/CMD that directly launches the agent using the start command, without backgrounding or wrapper scripts.

At Builds and Dockerfiles | LiveKit Documentation .I’m not a Docker expert, so let me raise this with the docs team.

Incidentally, the docs AI assistant seemed confident that the documentation was sufficient to exclude shell-form CMD - I’m not saying it’s correct, but I would recommend connecting our MCP server if you haven’t already done so, Coding agent support and tools | LiveKit Documentation, since it uses the same source of truth as the docs AI assistant.