Terry_So
July 27, 2026, 2:40am
1
Hi @Long Chen
We have report a bug and proposed enhancements in the livekit agent repo, which are related to the livekit.agents.llm.RealtimeModelFallbackAdapter introduced in the latest released version v1.6.6 of livekit agent.
opened 01:46AM - 27 Jul 26 UTC
enhancement
### Feature Type
Would make my life easier
### Feature Description
In recent … released version v1.6.6, a new class `livekit.agents.llm._FallbackRealtimeSession` is the subclass of `livekit.agents.llm.RealtimeSession`. Also, a existing class `livekit.plugins.openai.realtime.realtime_model.RealtimeModel.RealtimeSession` is the subclass of `livekit.agents.llm.RealtimeSession` as well.
Since **openai specific** realtime session event (eg. "openai_server_event_received", "openai_client_event_queued" etc. ) handlers are only allowed to be attached to `livekit.plugins.openai.realtime.realtime_model.RealtimeModel.RealtimeSession`, we have to check whether the realtime session object of agent is the instance of `livekit.plugins.openai.realtime.realtime_model.RealtimeModel.RealtimeSession` or `livekit.agents.llm._FallbackRealtimeSession`
If it is `livekit.plugins.openai.realtime.realtime_model.RealtimeModel.RealtimeSession`, we attach those openai specific** realtime session event handlers to the realtime session directly.
If it is `livekit.agents.llm._FallbackRealtimeSession` and its **active child realtime session** is the instance of `livekit.plugins.openai.realtime.realtime_model.RealtimeModel.RealtimeSession`, we attach those openai specific** realtime session event handlers to its **active child realtime session** instead. Like below:
```python
rt_session = agent.realtime_llm_session
if isinstance(rt_session, livekit.agents.llm._FallbackRealtimeSession) and isinstance(rt_session._active, `livekit.plugins.openai.realtime.realtime_model.RealtimeModel.RealtimeSession`):
@rt_session._active.on("openai_server_event_received"):
def on_openai_server_event_received(event: dict[str, Any]) -> None:
....
```
However, since `livekit.agents.llm._FallbackRealtimeSession` is a private class and thus cannot import it for instance check directly.
We want `livekit.agents.llm._FallbackRealtimeSession` become a public class `livekit.agents.llm.FallbackRealtimeSession` so that we can easily import it for instance check directly.
### Workarounds / Alternatives
We temporarily implement the instance check through
```python
rt_session = agent.realtime_llm_session
if type(rt_session).__name__ == "_FallbackRealtimeSession" and isinstance(rt_session._active, `livekit.plugins.openai.realtime.realtime_model.RealtimeModel.RealtimeSession`):
@rt_session._active.on("openai_server_event_received"):
def on_openai_server_event_received(event: dict[str, Any]) -> None:
....
```
### Additional Context
_No response_
opened 02:32AM - 27 Jul 26 UTC
enhancement
### Feature Type
I cannot use LiveKit without it
### Feature Description
Acco… rding to deepwiki [https://deepwiki.com/search/can-fallbackrealtimesession-im_15e6ca5e-6280-4091-b3d6-65ddfb72aedf?mode=deep](https://deepwiki.com/search/can-fallbackrealtimesession-im_15e6ca5e-6280-4091-b3d6-65ddfb72aedf?mode=deep),
for `livekit.agents.llm.RealtimeModelAapteri`, `openai specific` realtime session event handlers are attached to its **child** realtime session and handlers for **general** reatime session event in [https://github.com/livekit/agents/blob/87ef1b15f11239fa1e4cec9f6eace72b71904830/livekit-agents/livekit/agents/llm/realtime_fallback_adapter.py#L58](https://github.com/livekit/agents/blob/87ef1b15f11239fa1e4cec9f6eace72b71904830/livekit-agents/livekit/agents/llm/realtime_fallback_adapter.py#L58) ) are forwarded to attached to its **parent** realtime session.
When model fallback happens, its **old** child realtime session are closed and discarded immediately before the **new** one is created and its **parent** realtime session sill keep those handlers of those **general** realtime session event. As a result, those **openai specific** realtime session event handlers original attached to the child session will **lose**. Therefore, we need to **resubscribe** those **openai specific** realtime session event handlers to the new child realtime session. It has to be done **manually** every time when we have **openai specific** realtime session event handlers to attach.
Can you modify the source codes so that the **resubscribe** is implemented **automatically** when realtime model is fallbacked to the same plugin-type model (eg. openai -> openai, google -> google)?
### Workarounds / Alternatives
Since we have set realtime model fallback from openai to Azure openai as below:
```python
llm=RealtimeModelFallbackAdapter(
openai.realtime.RealtimeModel(),
openai.realtime.RealtimeModel.with_azure()
)
```
,
we need to manually **resubscribe** those **openai specific** realtime session event handlers to the new child realtime session through the codes like below:
```python
def _attach_oai_realtime_event_handlers(self, rt_session: RealtimeSession) -> None:
if not isinstance(rt_session, livekit.plugins.openai.realtime.realtime_model.RealtimeModel.RealtimeSession:
raise TypeError(f"Expected OpenAI RealtimeSession, got {type(rt_session).__name__}")
@rt_session.on("openai_server_event_received")
def on_openai_server_event_received(event: dict[str, Any]) -> None:
pass
if type(rt_session).__name__ == "_FallbackRealtimeSession":
current_child_session = rt_session._active
@rt_session.on("session_reconnected")
def on_session_reconnected(_: RealtimeSessionReconnectedEvent) -> None:
nonlocal current_child_session
new_child_session = rt_session._active # type: ignore[attr-defined]
# check whether the new child session is different from the current child session
# to make sure the received 'session_reconnected' event is emitted by model fallback
if new_child_session is not current_child_session:
LOGGER.info(
"Resubscribe OpenAI realtime event handlers when model fallback happens "
"for RealtimeFallbackAdapter"
)
attach_oai_realtime_event_handlers(new_child_session)
current_child_session = new_child_session
```
### Additional Context
_No response_
opened 07:13AM - 27 Jul 26 UTC
bug
### Bug Description
When we use`RealtimeModelFallbackAdapter` as the llm, acco… rding to [source codes](https://github.com/livekit/agents/blob/48c179329522a44a979c97f58865773badf5241f/livekit-agents/livekit/agents/llm/realtime_fallback_adapter.py#L135), implementing **llm.restart_session(switch_model=False)** will close the old child realtime session and **all attached plugin-specific realtime session event handlers (eg. openai specific realtime event handlers) will lose**. Even though we use the same model, the new child realtime session is completely different from the old one. We think this is not a right behavior.
Please correct the source code so that **all attached plugin-specific realtime session event handlers should be automatically forwarded to the new child realtime session if llm.restart_session(switch_model=False) is implemented.**
### Expected Behavior
all attached plugin-specific realtime session event handlers should be automatically forwarded to the new child realtime session if `llm.restart_session(switch_model=False)` is implemented.
### Reproduction Steps
1. set `
```python
llm=RealtimeModelFallbackAdapter(
[openai.realtime.RealtimeModel(),
openai.realtime.RealtimeModel.with_azure()]
)
```
2. attach openai-specific realtime session event handlers:
```python
rt_session = agent.realtime_llm_session
@rt_session._active.on("openai_server_event_received")
def on_openai_server_event_received(event: dict[str, Any]) -> None:
logger.info("openai_server_event_received handling"
```
3. implement
```python
await agent.llm.restart_session(switch_model=False)
```
5. try to simulate to emit "openai_server_event_received" realtime event
6. check whether the attached openai-specific realtime session event handlers still work
### Operating System
MacOS, linux
### Models Used
Openai gpt-realtime, Azure gpt-realtime
### Package Versions
```bash
livekit=1.1.13
livekit-agents=1.6.6
livekit-api=1.2.0
```
### Session/Room/Call IDs
_No response_
### Proposed Solution
```python
```
### Additional Context
_No response_
### Screenshots and Recordings
_No response_
Please tackle them one by one and keep me posted
Terry So
1 Like
system
August 11, 2026, 2:41am
2
This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.
