Grok RealtimeModel not accepting custom xAI voice clone (only default voices work)

Agents
1

Hi guys,

I’m using the latest xAI Grok RealtimeModel via the LiveKit xAI plugin:

session = AgentSession(

llm=xai.realtime.RealtimeModel(

    model=settings.livekit_agent_model,

    voice=settings.livekit_agent_voice,

    api_key=settings.xai_api_key,

    base_url=settings.xai_api_base_url,

    turn_detection=\_build_turn_detection(),

),

)

I cloned a custom voice in Grok/xAI, and it works correctly when using the standard xAI TTS endpoint.

However, when using RealtimeModel, only the default voices (e.g., ara, leo) work. Passing my custom voice ID via environment variable does not work.

Questions:

  1. Does Grok Voice Agent API currently support custom voice clones?

  2. If yes, is there a specific format or additional configuration required?

  3. If not, is support planned?

Thanks!

2

@smartchat, the plugin side isn’t where it’s getting blocked. xai.realtime.RealtimeModel’s voice arg accepts GrokVoices | str | None (source
(agents/livekit-plugins/livekit-plugins-xai/livekit/plugins/xai/realtime/realtime_model.py at main · livekit/agents · GitHub)), so a custom string passes straight through to xAI without transformation. Default voices working while your clone doesn’t means xAI's Realtime backend is rejecting the ID, not the LiveKit plugin filtering it.

That’s consistent with how other realtime providers handle voice catalogs: their Realtime API generally maintains a fixed voice list separate from their broader TTS endpoint. OpenAI Realtime and Gemini Live both work this way; clones registered for standard TTS aren’t automatically available on the realtime session.

For confirmation and roadmap on Grok Realtime supporting custom clones, the answer needs to come from xAI directly. The LiveKit plugin already accepts arbitrary strings, so once xAI exposes clone IDs on their Realtime endpoint you won’t need to change anything on your end.

3

Hi @smartchat , the team are aware of the feature and there are plans to add it, but I don’t have any dates to share I’m afraid.

For now, the official answer is we only support the default voices, as linked to from the docs at xAI Grok Voice Agent API plugin | LiveKit Documentation

Custom voices were only recently added to xAI: Custom Voices | xAI Docs