Issue: RealtimeModelMetrics.ttft is always logged as 0.0 for gemini 3.8 live on vertex ai

Agents
, ,
1

@livekit @LiveKit-Community @livekitteams @darryncampbell @CWilson @Muhammad_Usman_Bashir, @Akiniesta
Hi LiveKit Team,
We are observing that Time To First Token (ttft) is consistently logging as 0.0 ms when using the gemini-3.8-live model (via Vertex AI, location: ‘us’).
Here is a sample metric log for reference (reproduced on livekit-agents versions 1.5.7 and 1.8.5):
JSON

{“message”: “RealtimeModel metrics”, “level”: “INFO”, “model_name”: “gemini-3.8-live”, “model_provider”: “Vertex AI”, “ttft”: 0.0, “input_tokens”: 2020, “output_tokens”: 309, “tokens_per_second”: 29.53}

Based on manual testing, the actual end-to-end response latency feels under
<500 ms, so the functionality itself is fast, but the metric collection seems bugged or unhandled for this realtime audio stream.
Could you please clarify:
Is there an additional configuration needed in the agent/plugin options to capture ttft correctly for Gemini Live models on Vertex AI?
Is this a known logging issue with bidirectional/realtime audio streams in livekit-agents?
How can we accurately measure first-byte/first-token latency for this model in the meantime?

2

Thanks for flagging. I will let the team know.

3

@Harshita_Sukumar_Patil, This is a plugin instrumentation issue in the Gemini realtime path, not your config; no agent or plugin option changes it. ttft is first_token_timestamp - created_timestamp (realtime_api.py#L1611-L1613), and both anchors come from the server response stream: _created_timestamp is set when the plugin detects a new generation in the server messages (#L1227), and _first_token_timestamp on the first audio chunk (#L1420-L1421). Those two arrive together, so the difference collapses to ~0. The real gap, from end-of-user-turn to first token, is never measured, because the start anchor is the server response, not the request.

It is not a tracked issue (no matching one on the repo), so a focused bug report with your repro and both versions is the fix path (new issue). Add that _first_token_timestamp is set only on audio inline_data, not on text or output transcription (#L1408-L1421), so the anchor is doubly narrow.

The other fields (tokens, tokens_per_second) are unaffected; only ttft is wrong here. For a real number meanwhile, timestamp it yourself: record when the user turn ends and when the first agent audio frame arrives, then subtract. That bypasses the plugin anchor entirely.