Facing errors while calling update_chat_ctx when using azure open ai realtime llm

Agents
, ,
1

Hi we are using azure open ai realtime llm and our Agent Session configuration is:

return AgentSession(
llm=openai.realtime.RealtimeModel.with_azure(
azure_deployment=os.getenv(“ARE”),
azure_endpoint=os.getenv(“ARD”),
api_key=os.getenv(“KEY”),
api_version=os.getenv(“API”),
input_audio_transcription=InputAudioTranscription(
model=os.getenv(“TM”),
language=“en”,
),
),
userdata=userdata,
vad=vad,
tts=tts,
user_away_timeout=user_away_timeout,
turn_detection=turn_detection,
max_endpointing_delay=max_endpointing_delay,
preemptive_generation=preemptive_generation,
)

We call update_chat_ctx on every agent switch and before we call this we manually add some system instructions to the chat ctx using

chat_ctx.add_message(
role=“system”,
content=(
“TEST”
),
)

We are facing two errors

  1. On calling update_chat_ctx for first time we get

type=‘error’
error=RealtimeModelError(type='rea…
timestamp=1773645063.741534,
label='livekit.plugins.openai.real…
error=APIError(‘OpenAI Realtime API
returned an error’,
body=Error(message=“Missing
required parameter: ‘item.type’.”,
type=‘invalid_request_error’,
code=‘missing_required_parameter’,
event_id='chat_ctx_create_0304ac1b…
param=‘item.type’),
retryable=True), recoverable=True)
source=<livekit.plugins.openai.rea…
object at 0x7f50b0694c20>
created_at=1773645063.74164

  1. On subsequent calls to update_chat_ctx we get

type=‘error’ error=RealtimeModelError(type=‘realtime_model_error’, timestamp=1773645063.7664914, label=‘livekit.plugins.openai.realtime.realtime_model_beta.RealtimeModelBeta’, error=APIError(‘OpenAI Realtime API returned an error’, body=Error(message=“Error adding item: the previous item with id ‘item_d4fc3f27baf6’ does not exist.”, type=‘invalid_request_error’, code=‘item_create_invalid_previous_item_id’, event_id=‘chat_ctx_create_e307d32c164d’, param=None), retryable=True), recoverable=True) source=<livekit.plugins.openai.realtime.realtime_model_beta.RealtimeModelBeta object at 0x7f50b0694c20>

2

Hi @livekitteams,

This looks like a known gap in the RealtimeModelBeta code path, which is what RealtimeModel.with_azure(…) uses since Azure doesn’t yet support the GA Realtime API.

The beta model’s update_chat_ctx method does not filter out internal chat item types (like agent_handoff and agent_config_update) before sending them to Azure. The GA model does this filtering correctly, but the beta model does not. When these internal items are serialized, they produce conversation items without a type field, and Azure rejects them with “Missing required parameter: ‘item.type’”.

The second error (“the previous item with id ‘…’ does not exist”) is a cascading failure – since the first item creation was rejected, any subsequent items referencing it also fail.

Workaround: Before calling update_chat_ctx, manually filter out the internal items from your chat context:

chat_ctx = session.current_agent.chat_ctx.copy(
    exclude_handoff=True,
    exclude_instructions=True,
    exclude_config_update=True,
)
chat_ctx.add_message(role="system", content="Your content here")
await session.current_agent.update_chat_ctx(chat_ctx)

This mimics what the GA model does internally and should prevent the malformed items from being sent to Azure.

Longer term, open issues are tracking Azure GA Realtime API support ( feat: Support GPT Realtime GA API on Azure · Issue #5017 · livekit/agents · GitHub , Gpt-realtime Model with_azure function - API version 2025-08-08 · Issue #4719 · livekit/agents · GitHub ), which would move Azure users to the GA code path that handles this correctly.

I’d recommend opening a GitHub issue for this specific bug if you’d like to track the fix. Let me know if this helps or if you need more details!

1 Like
3

Hi @Ahmed We tried with the Workaround you suggested but still we are facing the same issue.