I have been using Gemma 4 via LiveKit Inference ever since it launched, but recently, my agents started hitting ambiguous que times during calls resulting in 2000ms+ latency just for the LLM. Has anyone been dealing with this? Is LiveKit planning to address this or expand capacity to fix this?

What is the session ID for a session that had the issue? If you have agent insights enabled, can you share that session with LiveKit staff too? Agent logs would he helpful too.

The agent itself is self-hosted. But everything else is through LiveKit Inference. Best sessions to look at (room names, all 2026-08-17 UTC): lk_SCL_S5N4wGJFwaKB, 14:43:46–14:45:24Z. Five consecutive LLM requests at 1620 / 2320 / 2790 / 2850 / 1640 ms TTFT. lk_SCL_K6N8yAF3r6c4, overlapping the sa…

Looking at your project, it looks like you have agent observability disabled, which makes it more difficult to investigate issues like this. How are you measuring the TTFT? If you could share OTEL traces, that might also help. The best solution would be to enable Agent Observability, reproduce th…

Okay, I enabled observability will provide that data tomorrow. But the rate is holding across multiple days, roughly 4.3% of call turns have that queueing signature and it causes them to have over 2000ms TTFT on Gemma 4.

[image] LiveKit Inference timeouts happening since past 2 weeks Getting Started @CWilson @Adrian_Cowham Thanks, we can mark this as solved! The timeout errors are solved. However, I’m seeing high TTFS and TTFT than before for gemma. Looking at benchmarks, the…

Agreed 100%. The 2000ms TTFT figure stood out to me though since that’s below even the P99 values on that benchmarks page.

I’ve seen similar results when using Gemma 4 31B through LiveKit inference. From the session logs the model seems to fall back to deepinfra as the inference provider, which results in higher latency as well as considerably worse output - there seems to be an issue with the checkpoint that deepinfra …

Gemma 4 Que Times - LiveKit Inference

Agents

Kaushal_Shah (Kaushal Shah) August 19, 2026, 2:24pm 6

i have raised similar issue here.

Its probably that gemma is used more often and is receiving more traffic and latency is increasing.

Topic		Replies	Views
LiveKit Inference timeouts happening since past 2 weeks Getting Started llm , livekit-inference	61	346	August 19, 2026
Gemma-4-31B TTFT on shared gateway (~400ms floor) vs self-hosted (28ms), same checkpoint Getting Started react-components	6	102	July 30, 2026
Introducing Gemma 4 on LiveKit Inference Announcements	4	231	August 3, 2026
Bad LiveKit Inference ttft for gpt-4.1 Agents livekit-inference , livekit-cloud	1	59	April 15, 2026
Gemini 3 Flash Preview via LiveKit Inference has much higher TTFT/jitter than direct Vertex in same Agents workflow Agents llm	1	49	May 15, 2026

Gemma 4 Que Times - LiveKit Inference

Related topics