xAI TTS returns 403 and no audio frames pushed; plugin appears to use different WebSocket endpoint than xAI docs

Agents
1

I’m testing the new xAI TTS integration in LiveKit Agents and consistently getting TTS failures in a normal voice-agent flow.

What I’m seeing

The agent generates text, but TTS fails with:

failed to synthesize speech: message=‘Invalid response status’, status_code=403, retryable=False

and then:

livekit.agents._exceptions.APIError: no audio frames were pushed for text: …

The runtime log identifies the TTS implementation as:

“tts”: “livekit.plugins.xai.tts.TTS”

Why I’m posting

I noticed a mismatch between the LiveKit xAI plugin source and xAI’s current public TTS streaming docs.

LiveKit plugin source

The xAI TTS plugin source uses:

XAI_WEBSOCKET_URL = “wss://api.x.ai/v1/realtime/audio/speech”

Source:
https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-xai/livekit/plugins/xai/tts.py

xAI public docs

xAI’s public TTS docs currently document streaming TTS at:

wss://api.x.ai/v1/tts

Docs:
https://docs.x.ai/developers/model-capabilities/audio/text-to-speech

Important detail

Unary TTS works with the same xAI API key outside LiveKit:

curl -X POST https://api.x.ai/v1/tts
-H “Authorization: Bearer $XAI_API_KEY”
-H “Content-Type: application/json”
-d ‘{“text”:“Hello world”,“voice_id”:“ara”,“language”:“en”}’

This returns 200, so the key is valid for xAI TTS at least on the REST path.

Questions

  1. Is livekit.plugins.xai.tts.TTS intentionally supposed to use wss://api.x.ai/v1/realtime/audio/speech?

  2. If yes, is that endpoint officially supported by xAI today, and where is it documented?

  3. If no, should the plugin be using wss://api.x.ai/v1/tts instead?

  4. Does the 403 + no audio frames were pushed pattern match a known issue with the new xAI TTS plugin?

Context

I’m likely running livekit-agents 1.4.6+ because the runtime is instantiating livekit.plugins.xai.tts.TTS, and xAI TTS appears to have been added in the March 16 release.

Relevant PRs:
https://github.com/livekit/agents/pull/5120
https://github.com/livekit/agents/pull/5122

2

What version of LiveKit and plugins are you running pip freeze |grep livekit

Also can you share the code where you set up the LLM,TTS, and STT?

3

I face the same issue with grok TTS.

Unrecoverable error from TTS: type=‘tts_error’
timestamp=1775022717.850421
label=‘livekit.plugins.xai.tts.TTS’
error=APIError('no audio frames were pushed for
text

4

@Rishiikesh_Balaji to avoid potential issues with your API key validity, can you try xAI with Inference? If that works, it’s a strong indication there is an issue with the key. 403 usually indicates an error like this.

5

@darryncampbell Thanks for the response. We currently have an xAI api key with a $5 in credits.
we were able to hit all the REST API xAI apis through postman and python as well. But we were unable to use the same XAI_API_KEY to get response from the X_AI model through livekit plugin. We are using self hosted livekit server and agent.
But still couldn’t resolve this issue.
current livekit version: livekit-plugins-xai versions 1.5.1

6

It’s working through Inference for me, but I’m unable to test with the plugin until I have a key. Let me get a key and then I can try with the Plugin.

8

Sure.
Let me know if you find anything

9

I think there are different issues being discussed in this thread.

Status code 403 typically indicates an issue with the key

I am seeing a status code 400 when I try using the plugin, failed to synthesize speech. To me it looks like an issue with the plugin, but since the plugin was only recently modified, and Inference still works, I have raised with the team for more clarity.

10

The team are working on a fix for this, (xai tts): update fields and ws setup by tinalenguyen · Pull Request #5350 · livekit/agents · GitHub.

Thank you for raising this @Rishiikesh_Balaji . Also @Vrishabh_Kumar , re-reading your original post, you will also be affected by this update

11

The fix mentioned above has been included in the latest Agents release: