How to retain system instructions in update_chat_ctx?

Agents
,
1

Quick question about the OpenAI realtime plugin — is there any workaround to keep system messages?

In the recent PR (https://github.com/livekit/agents/pull/4310/files#diff-ff18b8060d7fec5a4f3840b596c32f3c8d4a4331794ce9909e0e685dd76ada0c), the implementation for:

async def update_chat_ctx(self, chat_ctx: llm.ChatContext) -> None:
    async with self._update_chat_ctx_lock:
        chat_ctx = chat_ctx.copy(exclude_handoff=True, exclude_instructions=True)

now hardcodes exclude_instructions=True, which overrides a user call like:

chat_ctx = session.current_agent.chat_ctx.copy(exclude_instructions=False)

Is there any recommended way to preserve system instructions, or a planned change for this?

Thanks

2

For realtime models, I didn’t think the instructions were part of the chat context (as the first item), but they’re treated as part of the overall session state.

So, adding an item to the chat context, `update_chat_ctx` shouldn’t remove the instruction behaviour.

3

I’m running into an issue with a specific use case and would appreciate guidance.

I’ve built an education agent where users can ask generic questions. On an RPC call, we dynamically inject activity_prompts to switch the interaction mode (e.g., start an MCQ activity for science).

instructions = f"[type=activity_start|activity_code={activity_code}]"
interjection_handling_prompt = get_interjection_handling_prompt(instructions)

chat_ctx = session.current_agent.chat_ctx.copy()
chat_ctx.add_message(role="system", content=interjection_handling_prompt)

await session.current_agent.update_chat_ctx(chat_ctx)

session.generate_reply(instructions="Handle the interjection gracefully")

This allowed me to dynamically modify agent behavior mid-session.

Importantly, the activity prompt needs to remain in context, because the LLM must continue asking a predefined sequence of questions (e.g., step-by-step MCQ flow). So it’s not just a one-time instruction — it needs to persist across turns.

After PR #4310 (with exclude_instructions=True hardcoded in update_chat_ctx), this flow no longer works, since system messages injected via chat_ctx are stripped.

What is the recommended way to dynamically inject behavior-changing instructions (like activity switches) mid-session?

4

I haven’t set up an agent to test this, but would this work (using update_instructions)?

# Preserve existing instructions and append activity context  
current = session.current_agent.instructions  
activity_prompt = get_interjection_handling_prompt(  
    f"[type=activity_start|activity_code={activity_code}]"  
)  
new_instructions = f"{current}\n\n{activity_prompt}"  
await session.current_agent.update_instructions(new_instructions)  
session.generate_reply(instructions="Handle the interjection gracefully")
5

Thanks the suggested change works.

That said, I have a broader architectural question.

Wouldn’t it make more sense for update_chat_ctx to respect the chat_ctx as provided by the developer? Currently, with exclude_instructions=True hardcoded, it removes flexibility from the framework.

As a developer, I may intentionally want to include system messages in the chat context. By forcing exclusion inside the plugin, that decision is no longer configurable at the application layer.

Also, in many cases, session-level instructions are set once during agent initialization (e.g., personality, characteristics, base behavior). Dynamic system messages injected into chat_ctx often serve a different purpose — such as temporary behavior shifts or structured flows.

Is there a reason this behavior was made non-configurable? Would it make sense to expose this as a flag (e.g., allow instructions in chat_ctx optionally), so developers can decide based on their use case?

Would love to understand the design reasoning here.

6

I’m afraid I can’t speak authoritatively to the design reasoning, my guess would be that since instructions are handled differently for the realtime model, it was done deliberately, but I’m unsure why it was only introduced in that recent PR.

7

I have to agree with @Anmol_Gaud. I don’t see a good reason for Realtime’s update_chat_ctx to use exclude_instructions=True.

Here is OpenAI’s cookbook for doing context summarisation: openai-cookbook/examples/Context_summarization_with_realtime_api.ipynb at main · openai/openai-cookbook · GitHub

They specifically recommend adding a context item with role system:

The summary is appended as a SYSTEM message rather than an ASSISTANT message. Testing revealed that, during extended conversations, using ASSISTANT messages for summaries can cause the model to mistakenly switch from audio responses to text responses. By using SYSTEM messages for summaries (which can also include additional custom instructions), we clearly signal to the model that these are context-setting instructions, preventing it from incorrectly adopting the modality of the ongoing user-assistant interaction.

The way you would manage context in LiveKit is via update_chat_ctx, but if that function fully disregards the system role, you can’t even follow the recommended approach for context management with OpenAI realtime models.

I would be interested to know if there is a design decision in LiveKit that forces this?

1 Like
8

For additional context, this was also raised here: [BUG] update_chat_ctx strips system instructions and prevents dynamic behavior injection in realtime agents · Issue #4875 · livekit/agents · GitHub with a response from engineering.

9

There is a misunderstanding there:

For OpenAI realtime, you can also have system messages in chat context. The OpenAI cookbook I linked is a good example:
You have:

  • Your actual instructions which generally tell the agent how to behave. (You would use update_instructions)

  • You have chat context items that are generated by system. (you would use update_chat_ctx)

    • For example, for summarising chat context. You delete a bunch of items from the chat context and replace it will a single summarisation item. For that single summarisation item they suggest to set the role as system rather than assistant :

Important implementation detail:

  • The summary is appended as a SYSTEM message rather than an ASSISTANT message. Testing revealed that, during extended conversations, using ASSISTANT messages for summaries can cause the model to mistakenly switch from audio responses to text responses. By using SYSTEM messages for summaries (which can also include additional custom instructions), we clearly signal to the model that these are context-setting instructions, preventing it from incorrectly adopting the modality of the ongoing user-assistant interaction.

I will write a reply to the engineer.

10

Thanks, and that’s probably the best move. FYI davidzhao is our CTO :slight_smile:

11

Looks like there will be a fix out soon :slight_smile:

1 Like
12

Awsome, thanks for following up