High latency (5-8 seconds) with Google Gemini Realtime plugin over SIP

LiveKit Community
Wed, 21 Jan 2026 15:52:16 +0000
Self Hosting

This question originally came up in our Slack community and the thread has been consolidated here for long-term reference.

I’m working on a voice assistant using LiveKit SIP (with Twilio) and the Google Gemini Realtime plugin. Calls connect successfully and conversation flows, but I’m facing 5-8 second latency for agent replies.

I’ve tested both self-hosted (Docker Compose) and LiveKit Cloud - latency is the same on both.

My stack:

Model: gemini-2.5-flash-native-audio-preview via google.realtime

SIP: Twilio Elastic SIP Trunk

Optimizations tried:

Added silero.VAD for faster speech-end detection

Added turn_detector.EnglishModel() for turn-taking

Reduced min_silence_duration to 0.4

Has anyone else experienced this delay with the Gemini Realtime plugin?

4 posts - 3 participants