I’m Patrick, building GRAI Labs. We compared Gemini 3.1 Flash Live and GPT-Live 1 on 60 phone calls using LiveKit + Twilio and mock business tools.
I preferred Gemini’s pacing in the calls I’ve reviewed. But the part I want to keep working on is how we evaluate and debug speech-to-speech agents before production: tool behavior, state changes and what actually happened on the call.
I’ve open-sourced the simulations, order sandbox and trace inspector. The scripted demo runs without keys; new AI/phone tests use your own paid accounts. Connecting another agent takes adapter work.
Report: GPT-Live vs Gemini: 60 real phone calls | GRAI Labs Code: GitHub - patrick25076/grai-voice-bench: GRAI Labs: reusable real-phone tests for Gemini Live and GPT-Live, with synthetic tasks, tool checks and observable evidence. · GitHub
I’d like to connect with other LiveKit builders doing evals. What do you wish you could see when a call fails, and would this sandbox fit your tool setup?