Gemma-4-31B TTFT on shared gateway (~400ms floor) vs self-hosted (28ms), same checkpoint

Hey @AI_Tools thanks for the thorough write-up. Right now we do our routing based on the room id (proper prefix trie routing will follow). So right now, the speed benefits will be limited to live agent sessions. Were your tests running in a livekit room or done by directly calling the completions endpoint?