Intermittent repeated agent responses with Azure OpenAI in LiveKit Agents

Agents
, , ,
1

Hey LiveKit team, we are seeing an intermittent issue where the voice agent repeats the same sentence within a single response.

This seems to happen only on calls where we are using Azure OpenAI as the LLM provider. We are still not sure whether the issue is coming from Azure streaming, LiveKit Agents response handling, telemetry reconstruction, or something else in the voice pipeline.

Have others reported similar repeated responses when using Azure OpenAI with LiveKit Agents? Are there any known cases where streamed LLM chunks, agent responses, or TTS playout can cause the same sentence to be repeated?

We are using the Python LiveKit Agents stack:

  • livekit-agents==1.5.8
  • livekit-plugins-azure==1.5.8
  • livekit==1.1.7

Any pointers on how to narrow this down would help.

2

@amaan, first clarifier: which Azure OpenAI path?

livekit-plugins-azure is Azure Speech (STT/TTS), not the LLM, the Azure OpenAI LLM goes through either of those two paths. The diagnosis splits by which one.

Enable MetricsCollectedEvent and watch around a repetition. Two ttft records for one user turn = LLM-side stream retry. One ttft but two ttfb for one completion = TTS-side double submission (check preemptive_tts) [ Data hooks | LiveKit Documentation ]. Log the raw delta.content chunks with timestamps to confirm whether Azure is sending duplicate text or the framework is re-submitting.

No existing GH issue matches this symptom on the agents repo.

3

Which LLM model are you using ? I was facing the same double string issue in my agent. It was a model level issue with Gpt 5.4 from my debugging.

4

We are currently using openai.LLM.with_azure(…), but this issue has occurred with azure.responses.LLM(…) as well. Also it has occurred on 5.4-mini and 4.1. Will test MetricsCollectedEvent enabled and get back to you.