Can google/gemma-4-31b-it on LiveKit Inference be pinned to GPU inference infrastructure physically located in the EU, including prompt processing, KV/cache handling and model output generation, and is there a dedicated EU inference base URL or contractual data-residency commitment?
Can we consume that EU inference service using inference.LLM from our existing proprietary voice-agent runtime, without using LiveKit Cloud for realtime transport or deploying the agent on LiveKit?
Thanks
Second question is a clear yes. inference.LLM takes base_url, api_key and api_secret directly, falling back to LIVEKIT_INFERENCE_API_KEY / LIVEKIT_INFERENCE_API_SECRET (then LIVEKIT_API_KEY / LIVEKIT_API_SECRET). No room, job or LiveKit transport is required the job-context headers it tries to attach are wrapped in try/except RuntimeError: pass and are simply omitted outside a job. Construct it in your own runtime and call it like any other LLM.
On the EU endpoint, the SDK resolves to exactly two constants:
DEFAULT_INFERENCE_URL = "https://agent-gateway.livekit.cloud/v1"
STAGING_INFERENCE_URL = "https://agent-gateway.staging.livekit.cloud/v1"
No region-specific inference endpoints exist anywhere in that module. The override mechanism does base_url, or LIVEKIT_INFERENCE_URL, which takes priority over both so if LiveKit provisions an EU gateway you can point at it without code changes.
Whether that gateway exists, where the GPUs for google/gemma-4-31b-it sit, and what residency commitments are on offer are all LiveKit-side answers.
Specifically for Gemma4 served through LiveKit Inference, the official answer is here:
Is Gemma 4 31B on LiveKit Inference available in EU?
We are working hard on it, it’s great to see so many requests for EU hosting following the positive feedback we have had with our US hosted instance.
Another +1 on EU deployments.
This is especially relevant to us, since gemini flash 2.5 is going to be deprecated in October and i.m.o. their 3.1 lite is not a proper replacement. I have higher hopes in Gemma 4