Introducing Gemma 4 on LiveKit Inference

Announcements
1

The smartest models are too slow to feel natural in a live conversation, and production voice prompts are huge and expensive before the user even says hello.

What we’re launching today is the best answer to the question of how to balance speed, cost, and intelligence for voice: Gemma 4 31B, now live on LiveKit Inference.

→ 88% task completion on a 100-conversation hotel-reception eval—winning or matching much larger models
→ 381ms time-to-first-sentence—more than 2x faster than the next-closest model
→ ~100ms warm first-token latency, full-precision weights, generous context
→ $1.20 / 1M output tokens, on your existing LiveKit key

This is the fastest, highest-quality voice-optimized LLM available. Try it and feel the difference for yourself!

Announcement blog:

Resources

3 Likes
2

Since Gemma is multimodal, is it also possible to feed audio in and use it in half cascade mode?

3

@Phanos_Anastasiou It’s not possible to use the model in half cascade mode. We serve google/gemma-4-31b-it through LiveKit Inference as an OpenAI-compatible Chat Completions LLM, which is a text interface registered as an llm with no audio capabilities. Half cascade mode would require a realtime model.