Is Gemma 4 31B on LiveKit Inference available in EU?

Agents
1

Love the new LLM inference offering and we can’t wait to move our workloads to it. We have Livekit Cloud agents deployed in both EU and US; can you confirm whether this managed LLM inference is available in both locations? If so do we need to specify which region we’re targeting or is it auto-routing?

2

AWS Bedrock Mantle also supports in within an EU region. You can use it there as well.

3

Thanks, I’m assuming you’re referring to Gemma 4 31B - Amazon Bedrock ?

It looks like the selling point of Livekit’s offering is that they’ve tuned its inference stack to minimize time-to-first-answer latency over throughput. Assuming latency is the primary target concern, it’s natural to ensure multi-region deployments to minimize network hops.

I guess now I’m also curious whether this specific model is fine-tuned at all for voice use cases or simply optimized for voice inference?

4

Yes, but AWS has Bedrock Mantle, which hosts several models that usual Bedrock doesn’t have, like GPT 5.4.

We are using AWS Bedrock for multiple models and it’s quite fast. We have tried Gemma 4 31B and it’s tool calling accuracy is 0%, we haven’t been able to make it do tool calling correctly.

5

Hi, to answer your specific question, Gemma 4 31B is available on LiveKit Inference in the US right now. That won’t be the case forever, but I don’t have any specific information on the timeline for availability outside of the US I’m afraid.

6

Hey @D3x_Ops yes it will be in the EU. We’re tuning it in the US to make sure we have things working correctly. But EU will be next. I hesitate to put a specific timeline on it but ~1-2 months feels about right.

Re: latency. Yeah we are hosting to make sure TTFT is fast but also TTFS (time-to-first-sentence) because this is ultimately what drives the latency overall (TTS systems need more than just the first token to generate audio). Gemma 4 we found is a great model intelligence-wise that can also be extremely fast (more on that soon) and when you really need SOTA intelligence, other models can be used in tandem with async tool calls.