# Does LiveKit Inference apply xai/tts-1 extra fields such as optimize\_streaming\_latency?

**URL:** <https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108>\
**Category:** Agents\
**Tags:** tts, xai\
**Created:** [September 23, 2026, 12:06pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108 "2026-09-23T12:06:21Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Benjamin\_Lowe](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/benjamin_lowe/32/314_2.png) [@Benjamin\_Lowe](https://community.livekit.io/u/Benjamin_Lowe)\
**Post date:** [September 23, 2026, 12:06pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/1 "2026-09-23T12:06:21Z")

</div>

I am using `livekit.agents.inference.TTS` with `model="xai/tts-1"`, not `livekit.plugins.xai.TTS`. I would like to know which xAI TTS parameters the inference gateway actually forwards.

xAI documents these as WebSocket query parameters on `wss://api.x.ai/v1/tts`:

- `optimize_streaming_latency`: `0`, `1`, or `2`. `0` is the default and keeps the largest first chunk. `2` is the smallest first chunk.
- `speed`: speaking-rate multiplier, about `0.7` to `1.5`.
- `text_normalization`: bool.

The direct plugin sends those as query parameters. That was [livekit/agents#6095](https://github.com/livekit/agents/pull/6095), which closed [livekit/agents#6077](https://github.com/livekit/agents/issues/6077). The Inference docs for `xai/tts-1` only list `model`, `voice`, and `language`. They do not mention those three.

The Inference client still accepts them. [livekit/agents#6469](https://github.com/livekit/agents/pull/6469) added `optimize_streaming_latency: Literal[0, 1, 2]` to `XaiOptions`. `speed`, `bit_rate`, and `text_normalization` are on that TypedDict too. The SDK does not turn them into query parameters. It copies `extra_kwargs` into the JSON `extra` field on `session.create` and on each `input_transcript` packet to the inference gateway.

So this type-checks and is sent:

```python
inference.TTS(
    model="xai/tts-1",
    voice="carina",
    language="ja",
    extra_kwargs={
        "text_normalization": True,
        "optimize_streaming_latency": 2,
    },
)

```

Questions:

1. Does the inference gateway read `extra.optimize_streaming_latency`, `extra.speed`, `extra.text_normalization`, and `extra.bit_rate` and pass them to xAI?
2. If yes, which packet matters, `session.create` or `input_transcript`? Is `2` accepted, or only `0` and `1`?
3. If no, please treat this as a feature request. Inference should accept the same xAI streaming options the direct plugin already forwards, and the Inference docs should list them. `optimize_streaming_latency` is the one I was lookign to try to avoid stuttering in TTS. With the default, generated audio falls behind realtime and LiveKit logs `flush audio emitter due to slow audio generation`, which is an audible cut in playback.

---

<div class="post-metadata">

**Author:** ![Benjamin\_Lowe](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/benjamin_lowe/32/314_2.png) [@Benjamin\_Lowe](https://community.livekit.io/u/Benjamin_Lowe)\
**Post date:** [September 24, 2026, 1:28pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/2 "2026-09-24T13:28:05Z")

</div>

For context: When using Livekit Inference with Japanese

livekit-agents 1.8.2 Python on Linux:

- model=“xai/tts-1”,
- voice=“carina”,
- language=“ja”
- _extra\_kwargs_={“text\_normalization”: False, “optimize\_streaming\_latency”: 1},

Regardless of the `optimize_streaming_latency` flag we often get DEBUG message like

> flush audio emitter due to slow audio generation

corresponding to tiny (millisecond) gap in the TTS playback. Sometimes that is not logged but we get it anyway like the attached clip:

```auto
lk.pii.response.text: "承知いたしました。"

```

 ![image](https://us1.discourse-cdn.com/flex001/uploads/livekit2/original/2X/a/a348928b55619aa9363f09637c29937521c969f2.png)

[https://us1.discourse-cdn.com/flex001/uploads/livekit2/original/2X/9/9f8ef1e9d6c95cacad6817447e8323d9b6a499af.wav](https://us1.discourse-cdn.com/flex001/uploads/livekit2/original/2X/9/9f8ef1e9d6c95cacad6817447e8323d9b6a499af.wav)

I suspect its due to STT playback being slower than it can be streamed, and does not occur when I use the same string on [https://console.x.ai/team/5d42776b-1d07-4093-9b76-2f8523a0b8c2/voice/text-to-speech](https://console.x.ai/team/5d42776b-1d07-4093-9b76-2f8523a0b8c2/voice/text-to-speech) (which is not streamed presumably). So that is why I am keen to enable `optimize_streaming_latency` but I suspect it currently is not enable-able.

---

<div class="post-metadata">

**Author:** ![CWilson](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/cwilson/32/135_2.png) [@CWilson](https://community.livekit.io/u/CWilson)\
**Post date:** [September 24, 2026, 2:48pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/3 "2026-09-24T14:48:25Z")

</div>

@Benjamin_Lowe I am not sure offhand, but opening up the code now to take a look. Will let you know soon.

---

<div class="post-metadata">

**Author:** ![Muhammad\_Usman\_Bashir](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/muhammad_usman_bashir/32/1149_2.png) [@Muhammad\_Usman\_Bashir](https://community.livekit.io/u/Muhammad_Usman_Bashir)\
**Post date:** [September 24, 2026, 2:49pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/4 "2026-09-24T14:49:00Z")

</div>

@Benjamin_Lowe, Your client-side trace is exactly right. `XaiOptions.optimize_streaming_latency` is `Literal[0, 1, 2]`, and the inference SDK copies `extra_kwargs` into the `extra` field on both the `session.create` packet ([inference/tts.py#L538-L541](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/inference/tts.py#L538-L541)) and every `input_transcript` packet ([#L688-L691](https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/inference/tts.py#L688-L691)). So the fields do leave the SDK on both.

What happens after that is not answerable from open source. The inference gateway is Cloud-side, not in `livekit/agents`, so whether it reads `extra.optimize_streaming_latency`, `speed`, `text_normalization`, or `bit_rate` and forwards them to xAI, which packet wins, and whether `2` is accepted are all gateway-internal. No issue tracks this yet, so filing the feature request you drafted is the move, and it gets the authoritative answer from the people who own the gateway ([new issue](https://github.com/livekit/agents/issues/new)).

To actually exercise the flag now, run this one agent through the direct plugin instead of Inference. `livekit.plugins.xai.TTS` takes these and puts them on the xAI WebSocket URL as real query parameters ([xai/tts.py#L142-L149](https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-xai/livekit/plugins/xai/tts.py#L142-L149)):

```python
from livekit.plugins import xai
# reads XAI_API_KEY from env (its own key, not the gateway's); same voice/language you already use
tts = xai.TTS(voice="carina", language="ja",
              optimize_streaming_latency=2, text_normalization=False)

```

Set one expectation before pinning hopes on it. By xAI’s own definition (0 largest first chunk, 2 smallest), `optimize_streaming_latency` tunes first-chunk size, not sustained generation rate. `flush audio emitter due to slow audio generation` is a mid-utterance underrun, generation falling behind realtime playout, so if xAI’s sustained rate for that Japanese text is below realtime, a smaller first chunk starts sooner but may not close the gap. The direct-plugin run will tell you quickly whether it helps.

---

<div class="post-metadata">

**Author:** ![CWilson](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/cwilson/32/135_2.png) [@CWilson](https://community.livekit.io/u/CWilson)\
**Post date:** [September 24, 2026, 2:51pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/5 "2026-09-24T14:51:19Z")

</div>

Oh, maybe @Muhammad_Usman_Bashir answered your question? I assumed you meant off the backend of the inference gateway not from agent to gateway. I will still validate if those get forwarded to the peer service.

---

<div class="post-metadata">

**Author:** ![CWilson](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/cwilson/32/135_2.png) [@CWilson](https://community.livekit.io/u/CWilson)\
**Post date:** [September 24, 2026, 2:57pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/6 "2026-09-24T14:57:23Z")

</div>

The Inference gateway reads all four xAI extras and sends them to xAI as query parameters on the WebSocket URL, the same way the direct plugin does.

Do you happen to have the session ID for the session where that stutter happened? I can try and look through the logs.

---

<div class="post-metadata">

**Author:** ![CWilson](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/cwilson/32/135_2.png) [@CWilson](https://community.livekit.io/u/CWilson)\
**Post date:** [September 24, 2026, 2:59pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/7 "2026-09-24T14:59:34Z")

</div>

> [@CWilson](#):
>
> Do you happen to have the session ID for the session where that stutter happened? I can try and look through the logs.

Nevermind. I see it is in the filename.

---

<div class="post-metadata">

**Author:** ![CWilson](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/cwilson/32/135_2.png) [@CWilson](https://community.livekit.io/u/CWilson)\
**Post date:** [September 24, 2026, 3:18pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/8 "2026-09-24T15:18:45Z")

</div>

It would help to narrow it down with a test like this:

1. Set `optimize_streaming_latency` to 0, or leave it out, and see whether the early-utterance gaps go away.
2. Run the same test on a wired connection, or from a small cloud VM near Frankfurt (your room is hosted there). If the gaps disappear, the delay is on the way to your machine.
3. Try the same Japanese text with livekit.plugins.xai.TTS directly. If it gaps there too, it’s xAI’s streaming throughput for Japanese and we’ll raise it with them.

---

<div class="post-metadata">

**Author:** ![Benjamin\_Lowe](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/benjamin_lowe/32/314_2.png) [@Benjamin\_Lowe](https://community.livekit.io/u/Benjamin_Lowe)\
**Post date:** [September 25, 2026, 12:38pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/9 "2026-09-25T12:38:05Z")

</div>

@CWilson and @Muhammad_Usman_Bashir thank you for your response. I tried setting optimize\_streaming\_latency to absent, 0, 1 and 2 and the issue persists with Livekit Inference XAI TTS.

Ok so I have tried now with the XAI plugin directly ([https://cloud.livekit.io/projects/p\_1f33ky4suyp/sessions/RM\_bkdtpHRUEbq9](https://cloud.livekit.io/projects/p_1f33ky4suyp/sessions/RM_bkdtpHRUEbq9), [https://cloud.livekit.io/projects/p\_1f33ky4suyp/sessions/RM\_JDbaxzEkczS7](https://cloud.livekit.io/projects/p_1f33ky4suyp/sessions/RM_JDbaxzEkczS7) ) with optimize\_streaming\_latency 0 and 2 and it seems to work fine in either case without these stutters. A longer example with the flag omitted: [https://cloud.livekit.io/projects/p\_1f33ky4suyp/sessions/RM\_6awkG45G7hMR/observability](https://cloud.livekit.io/projects/p_1f33ky4suyp/sessions/RM_6awkG45G7hMR/observability)

I should note that “flush audio emitter due to slow audio generation” I think may have beed a “red herring”, as most of the time this issue occurs without that issue. I also tried setting `os.environ.setdefault("LIVEKIT_AGENTS_LOOP_BLOCK_WARN_MS", "20")` so see if its an asycio block related, but no warnings are logged.

You suggested trying this on a cloud VM near Frankfurt - which seems challenging as it would need to be able to route microphone traffic etc. but I can rule our my internet connection being too slow (wired connection) given if that was the problem then XAI plugin directly would surely have the same issue.

In this case, I guess the conclusion may be just that livekit inference is unable to offer the same streaming bandwidth to my location as XAI?

I attach another example sessions from my local machine in UK using the livekit inference with optimize\_streaming\_latency omitted. In this example, almost everytime the agent says the first word in an utterance (e.g. 承知いたしました in the screenshot) it gets a small gap between the first and second character.

[https://cloud.livekit.io/projects/p\_1f33ky4suyp/sessions/RM\_zCATXCZ8Ymki/observability?mode=transcript&t=1790338829009](https://cloud.livekit.io/projects/p_1f33ky4suyp/sessions/RM_zCATXCZ8Ymki/observability?mode=transcript&t=1790338829009)

 ![image](https://us1.discourse-cdn.com/flex001/uploads/livekit2/original/2X/6/6d448490256641baebce724064347af248ebf031.png)

If there is no workaround, it may prevent us adopting the Expressive mode from Livekit (at least with this TTS provider) which is a shame, but it does seem to be specific to Livekit Inference itself. The odd thing to me is that other than these tiny glitches, everything else seems healthy (no high latency to respond etc.) : if it was related to Livekit Inference having slow bandwidth to my particular machine’s ip, I would have thought it would cause havoc in other parts of the pipeline.

i’ll check with my colleagues in Japan if this issue is specific to me in the UK (and Frankfurt based Livekit Room). How does the regionality of livekit inference work - is it always the same region as the room? I wonder which region is used by Livekit Inference when the room is hosted in Tokyo.

---

<div class="post-metadata">

**Author:** ![Benjamin\_Lowe](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/benjamin_lowe/32/314_2.png) [@Benjamin\_Lowe](https://community.livekit.io/u/Benjamin_Lowe)\
**Post date:** [September 25, 2026, 12:45pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/10 "2026-09-25T12:45:06Z")

</div>

> os.environ.setdefault(“LIVEKIT\_AGENTS\_LOOP\_BLOCK\_WARN\_MS”, “20”)

I notice these gaps in speech like I screenshotted and the other I shared above are often just under 20ms actually, so this doesn’t prove much, and 20ms is the lowest this env var goes

I wonder if when running livekit inference it’s possible something is blocking event loop for 19ms every now and then? But if that was the case I can’t imagine I would be the only one reporting it!

---

<div class="post-metadata">

**Author:** ![CWilson](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/cwilson/32/135_2.png) [@CWilson](https://community.livekit.io/u/CWilson)\
**Post date:** [September 25, 2026, 4:34pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/11 "2026-09-25T16:34:03Z")

</div>

Thanks for digging into this. I am going to work on reproducing this and see if I can find the root cause. I will let you know.

---

<div class="post-metadata">

**Author:** ![CWilson](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/cwilson/32/135_2.png) [@CWilson](https://community.livekit.io/u/CWilson)\
**Post date:** [September 25, 2026, 5:17pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/12 "2026-09-25T17:17:11Z")

</div>

> [@Benjamin\_Lowe](#):
>
> I wonder if, when running LiveKit inference, it’s possible something is blocking the event loop for 19ms every now and then? But if that was the case, I can’t imagine I would be the only one reporting it!

@Benjamin_Lowe I dug into this and created tests that reproduce it. I looked through the agent code and believe I’ve found the root cause. I am going to pass this off to our agents team to validate my results.

At a high level, I ran the test directly against the inference service (no agent framework) and didn’t see the issue. I did the same test with the agents framework and was able to reproduce the issue you exposed here.

I believe the issue originates from here but will need experts to verify it:  
AudioByteStream, [livekit-agents/livekit/agents/utils/audio.py#L62-L88](https://github.com/livekit/agents/blob/1257fe0f987b53054e1d31b98e514991776f85d4/livekit-agents/livekit/agents/utils/audio.py#L62-L88)

I will keep you posted.

---

<div class="post-metadata">

**Author:** ![Benjamin\_Lowe](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/benjamin_lowe/32/314_2.png) [@Benjamin\_Lowe](https://community.livekit.io/u/Benjamin_Lowe)\
**Post date:** [September 25, 2026, 6:09pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/13 "2026-09-25T18:09:42Z")

</div>

Wow thanks for looking into this! That sounds very promising

---

<div class="post-metadata">

**Author:** ![Benjamin\_Lowe](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/benjamin_lowe/32/314_2.png) [@Benjamin\_Lowe](https://community.livekit.io/u/Benjamin_Lowe)\
**Post date:** [September 29, 2026, 2:49pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/14 "2026-09-29T14:49:20Z")

</div>

Was it confirmed as a livekit-agentgs SDK side issue by the agents team?

---

<div class="post-metadata">

**Author:** ![Benjamin\_Lowe](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/benjamin_lowe/32/314_2.png) [@Benjamin\_Lowe](https://community.livekit.io/u/Benjamin_Lowe)\
**Post date:** [October 5, 2026, 12:08pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/15 "2026-10-05T12:08:35Z")

</div>

@CWilson sorry to bother you, just wanted to check if you had any updates on this one?

---

<div class="post-metadata">

**Author:** ![CWilson](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/cwilson/32/135_2.png) [@CWilson](https://community.livekit.io/u/CWilson)\
**Post date:** [October 5, 2026, 7:16pm UTC](https://community.livekit.io/t/does-livekit-inference-apply-xai-tts-1-extra-fields-such-as-optimize-streaming-latency/2108/16 "2026-10-05T19:16:10Z")

</div>

I’ve not heard back. I will check in with the team on this. We are all returning from an all-hands offsite, so we’re catching up now.