# Realtime metric calculation issue

**URL:** <https://community.livekit.io/t/realtime-metric-calculation-issue/2333>\
**Category:** Agents\
**Tags:** agent-development, livekit-cloud\
**Created:** [October 22, 2026, 10:31am UTC](https://community.livekit.io/t/realtime-metric-calculation-issue/2333 "2026-10-22T10:31:39Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sumanth\_thumati](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/sumanth_thumati/32/2889_2.png) [@sumanth\_thumati](https://community.livekit.io/u/sumanth_thumati)\
**Post date:** [October 22, 2026, 10:31am UTC](https://community.livekit.io/t/realtime-metric-calculation-issue/2333/1 "2026-10-22T10:31:39Z")

</div>

> ### **Approach 1 :**

`user_state_changed`: We used the `speaking → listening` state transition as T0 (`created_at`) and the assistant’s `started_speaking_at` as T1, calculating E2E = T1 − T0 for `user_reply` turns.

Why it failed: With Gemini Live without session VAD, `listening` was tied to the plugin’s `input_speech_stopped` event at turn completion, which could fire after the bot had already replied rather than when the user actually stopped speaking. This made T0 late or stale across turns, sometimes pairing the wrong start timestamp with the next T1. The calculation was correct, but the start-time anchor was wrong.

Observed values: E2E was inflated, often around 6–15 seconds per turn in `latency_ms_turns` (occasionally spanning roughly 1–15 seconds), rather than the expected ~0.5–1.5 seconds. Meanwhile, `RealtimeModelMetrics` TTFT remained very low—often around 1–6 ms median—because it reflects the Google plugin’s generation/transcript-related metric, not actual user-perceived response latency.

### **Approach 2**

`user_input_transcribed`: We set T0 to the `created_at` of the latest non-final `user_input_transcribed` event at or before T1, kept the assistant’s `started_speaking_at` as T1, and calculated E2E = T1 − T0 for `user_reply` turns.

Why it failed: On these Gemini Live calls, input text typically arrived as a single chunk just before the first agent audio, rather than as partial transcripts while the user was still speaking. Consequently, T0 landed close to T1 instead of marking the acoustic end of the user’s speech. The subtraction was correct, but the T0 proxy was wrong: we measured transcript arrival to playout, not user speech-end to playout.

Observed values: E2E was deflated, often around 10–300 ms per turn; one session included values such as 28, 11, 10, 44, 187, and 137 ms, while another had a median of approximately 32 ms, with many turns between 5 and 350 ms. `RealtimeModelMetrics` TTFT was still misleadingly low, with medians around 2.8–6.4 ms, because the Google plugin’s metric is tied to its transcript/generation pipeline and does not represent true model response latency or user-perceived delay.

i have tried two apporaches for the Gemini 3.8 live model. i have got the metrics wrong which has show the ttft around 10ms to 300ms and similar end to end latency. i think might also happen with openai aslo.

---

<div class="post-metadata">

**Author:** ![darryncampbell](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/darryncampbell/32/20_2.png) [@darryncampbell](https://community.livekit.io/u/darryncampbell)\
**Post date:** [October 26, 2026, 9:40am UTC](https://community.livekit.io/t/realtime-metric-calculation-issue/2333/2 "2026-10-26T09:40:18Z")

</div>

Hi, the best way to calculate realtime metrics would be to use the session reports:

> **[Session reports - Data hooks | LiveKit Documentation](https://docs.livekit.io/testing/observability/data/#session-reports)**
>
> The LiveKit Agents SDK includes access to extensive detail about each session, which you can collect locally and integrate with other systems. For information about data collected in LiveKit Cloud, see the Agent insights overview. To choose which...

which provide you with all the required metrics, such as TTFT for each turn.

If you are comparing TTFT with published figures, make sure you are comparing like-for-like, e.g. often published figures will state the P50 figure. The session reports, passed through AI, can also help you understand any anomalies, for example perhaps your sessions have a high number of tool calls which are affecting your metrics.

---

<div class="post-metadata">

**Author:** ![Abdul\_Rasheed](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/abdul_rasheed/32/2855_2.png) [@Abdul\_Rasheed](https://community.livekit.io/u/Abdul_Rasheed)\
**Post date:** [October 26, 2026, 5:14pm UTC](https://community.livekit.io/t/realtime-metric-calculation-issue/2333/3 "2026-10-26T17:14:13Z")

</div>

Your first anchor is late because, without a session VAD, the user’s “listening” state just follows Gemini’s input\_speech\_stopped event, timed when the event arrives.

Pass the session its own VAD:

```python
session = AgentSession(llm=google.realtime.RealtimeModel(...), vad=silero.VAD.load())

```

Gemini still decides when to reply. But the end-of-speech time now comes from Silero, back-dated by the silence it waited for. Heads up: Silero can now also interrupt the agent when the caller talks over it, so check barge-in still feels right.

Then you don’t have to compute anything. The assistant messages carry it:

```python
@session.on("conversation_item_added")
def _(ev):
    if ev.item.role == "assistant" and "e2e_latency" in ev.item.metrics:
        print(ev.item.metrics["e2e_latency"])

```

That’s first audio frame out minus when the user stopped speaking. It’s missing on turns the agent starts without new user speech, like the greeting or a generate\_reply. The plugin’s TTFT is its own internal timing, not what the caller hears.

This is from reading the livekit-agents 1.8.9 source (pip show livekit-agents to check yours). I haven’t run it against Gemini Live, so compare a few turns with a recording first.