What is the best/efficient way to to have an external platform update/inform a livekit room when an event happens

[Kamal_Moha]

I’m searching to know the most efficient/reliable way to send external events to a livekit room/agent.

Use case

My use case is that tool calls in Livekit are initiated as background tasks. I’m using Inngest to handle background. This is how

Here is the link to my livekit agent.py file Livkeit & Inngest · GitHub . It shows how I’m initiating tool calls in Livekit as background jobs done outside of livekit.

I want when my backend server Inngest completes or reaches the desired step (completes task), then it should inform livekit room/agent that tool call is done and then provide the tool result.

How I have navigated this so far

Whenever I need to inform the livekit agent/room, I have been using SendData in Livekit & using session.generate_reply to inject the update into the conversation.

This is my code that does SendData whenever an external event happens

from pydantic import BaseModel

class SendData(BaseModel):

  room_name: str

  message_type: str

  message_text: str

def create_room_admin_token(room: str) -> str:

  token = api.AccessToken(os.getenv('LIVEKIT_API_KEY'),

                          os.getenv('LIVEKIT_API_SECRET')) \

      .with_identity("Token-Generator") \

      .with_name("Kamal") \

      .with_grants(api.VideoGrants(

          room_join=True,

          room_admin=True,

          can_publish=True,

          can_subscribe=True,

          room=room)).to_jwt()

return token

async def send_data_to_agent(content: SendData):

    token = create_room_admin_token(content.room_name)

print(f"TOKEN:{token}")


# Your business event payload

    payload = content.model_dump()

print(f"PAYLOAD: {payload}")

    encoded_data = base64.b64encode(

        json.dumps(payload).encode("utf-8")

).decode("utf-8")

print(f"Encoded Data: {encoded_data}")

# Convert WebSocket URL to HTTP API URL

    api_url = os.environ.get('LIVEKIT_URL').replace('wss://', 'https://').replace('ws://', 'http://')

if not api_url.endswith('/'):

        api_url += '/'




print(f"API URL: {api_url}")




    headers={

"Authorization": f"Bearer {token}",

"Content-Type": "application/json"

}

    payload={

"room": content.room_name,

"data": encoded_data,

"kind": "reliable",

# "destination_identities": [AGENT_IDENTITY],

"destination_identities": None,

# "topic": "backend-event",

}




# Send data message via REST API

    url = f"{api_url}twirp/livekit.RoomService/SendData"




    response = requests.post(url, json=payload, headers=headers, timeout=10)

if response.status_code == 200:

    print(f"Successfully sent message to room")

    return f"Successfully sent message to room"

else:

    print(f"Failed to send message. Status: {response.status_code}, Response: {response.text}")

    raise Exception(f"LiveKit API error: {response.status_code} - {response.text}")

As you can see, I have set kind to reliable

What I’m trying to do

I’m building an agent using Whatsapp connector. Here is how flow is & the steps taken:

1. User calls the whatsapp number
2. Agent answers the call & has a normal conversation with the user
3. User sends a whatsapp message. Our backend receives a webhook event from Meta
4. We now need to inform the livekit agent about this new message. The livekit agent can use this message to execute a tool.
5. Livekit agent executes a tool. I’m using inngest to run the tool as a background task
6. When tool is successful, inform the livekit agent that the tool is successful and provide it the result.
7. Then message the user on Whatsapp a summary of tool result.

Every step is orchestrated by inngest because my use case depends on external platforms (Meta, Livekit, etc) exchanging events/data/information.

How I have navigated this

For steps 4 & 6 where I need to inform the livekit agent, I have been using SendData in Livekit & using session.generate_reply to inject the update into the conversation.

Problems I have faced

For steps 4 & 6 where I need I have noticed that SendData is not a very reliable method of updating the agent of what has been done in the backend. In my testing, I have noticed several times that even if when I do SendData, the agent is not even aware of what it has been sent and continues to say I’m still working on executing the tool, yet the tool has already been executed and results were shared to it.

Watch this quick video I have recorded > https://youtu.be/smxC5W2GQaY

By about 2:17 in the above video, the Livekit agent/room was sent data from my backend. But the agent is not aware of anything and doesn’t know tool result throughout the entire session.

Watch the above video.

I also get these errors in the logs

2026-08-31 05:05:09,544 - ERROR livekit - livekit::rtc_engine::rtc_session:679:livekit::rtc_engine::rtc_session - publisher data channel '_reliable' closed unexpectedly {"pid": 1948, "job_id": "AJ_ZTfqJHAgEVKd", "room": "whatsapp-room"}

2026-08-31 05:05:09,545 - ERROR livekit - livekit::rtc_engine::rtc_session:679:livekit::rtc_engine::rtc_session - publisher data channel '_lossy' closed unexpectedly {"pid": 1948, "job_id": "AJ_ZTfqJHAgEVKd", "room": "whatsapp-room"}

2026-08-31 05:05:09,547 - ERROR livekit - livekit::rtc_engine::rtc_session:679:livekit::rtc_engine::rtc_session - publisher data channel '_data_track' closed unexpectedly {"pid": 1948, "job_id": "AJ_ZTfqJHAgEVKd", "room": "whatsapp-room"}

Reason I’m picking inngest

• Durable execution
• It’s event driven and provides reliable runs of several separate steps & agent actions
• Idempotency
• Does retries if tool execution fails
• State persistence

For example, this is what happens when I execute/trigger a livekit tool. This is how the backend orchestrates multiple steps.

For my use case, multiple steps need to take place and that’s why execution needs to be outside of livekit. But it’s VERY important I inform the active livekit room/agent about what’s happening on the backend.

Questions

1. What is the most efficient way to send external events/webhooks/data to a livekit room/agent?
2. When sending external data to a room, how can you inform that this is tool response/result.

[abidullahcs.uk (Abid)]

@Kamal_Moha Your tool returns before the work finishes, so from the LLM’s point of view the call already completed successfully with “I’ve started tracking your shipment”. Anything you push in later has no relationship to that tool call, which is why the agent keeps saying it is still working.

with_filler is built for exactly this. Its docstring is “schedule filler speech while a long-running step blocks the tool”, so the intended shape is to keep the tool awaiting and let the filler cover the wait:

  async with ctx.with_filler("Still checking, hang on a sec.", delay=5, interval=15):
      result = await wait_for_inngest_result(...)   # block here
  return f"Your package is {result.status}, expected {result.eta}."

The return value is then a real tool result in the right position and you do not need to inject anything or label it.

If you do need genuinely out of band events, SendData is the correct transport since RPC is participant to participant and your backend is not in the room. But sending is only half of it, and your post only shows the send side. The agent needs a handler registered, roughly:

  @ctx.room.on("data_received")
  def _on_data(packet: rtc.DataPacket):

Worth checking those data channel errors first though. publisher data channel ‘_reliable’ closed unexpectedly means the agent’s connection dropped, and nothing you send lands after that regardless of method. If those timestamps line up with 2:17 in your video, that is the actual failure rather than SendData being unreliable.

[Muhammad_Usman_Bashir]

@Kamal_Moha, Your SendData approach fights the framework. When the tool returns “I’ve started tracking”, the LLM thinks the tool is done, and a later SendData is just bytes to the room that nothing turns into a new turn. Your data channel closed unexpectedly logs also mean the session was already gone when you pushed.

The reliable way is what the framework is built for: await the result inside the tool and return it. A long-running tool keeps the agent talking while it waits (async tools). You already use ctx.update and ctx.with_filler, so just await the Inngest run instead of returning early. The job is its own async process, so this does not block other jobs.

@function_tool(on_duplicate="reject")
async def check_shipping_status(self, ctx: RunContext, tracking_num: str) -> str:
    await ctx.update(f"Checking shipping status for {tracking_num}.")
    async with ctx.with_filler("Still checking, hang on a sec.", delay=5):
        result = await track_and_wait(tracking_num, self.phone_number)  # await Inngest completion
    return f"Tracking {tracking_num}: {result}"   # real result; the LLM voices it

That also answers your second question: you do not tag a SendData as a tool result. Returning it from the tool makes it literally the result the model consumes. Only use out-of-band SendData for genuinely fire-and-forget events, and even then the agent must receive it and call session.generate_reply, with the session still live.

[Kamal_Moha]

@abidullahcs.uk The whole point is to execute every tool as a background job. This will mean that the livekit agent.py just initiates the tool. But the actual execution of the tool is done on my backend server, inngest (outside of livekit).

The livekit agent tool is executed on my backend server and when the tool is completed, the backend server updates the livekit room/agent & provides the tool result. This is where I’m using SendData to update the livekit room/agent that the tool execution is done and here is the tool result.

I was thinking the cause of those errorrs were because data wasn’t being sent to the livekit room.

---

Yes my livekit agent has the necessary code to handle receiving data. The complete livekit agent.py file is here Livkeit & Inngest · GitHub

[abidullahcs.uk (Abid)]

@Kamal_Moha thanks, and your handler registration is fine. I checked the event against rtc/room.py and both the name and the DataPacket signature match what you have.

The thing I would look at is the last line:

asyncio.create_task(handle())

Nothing holds a reference to that task. Python’s own docs warn about this on asyncio.create_task: “Save a reference to the result of this function, to avoid a task disappearing mid-execution”, because the event loop only keeps weak references. An unreferenced task can be garbage collected before it finishes, which fits an update that sometimes lands and sometimes does not.

  _bg: set[asyncio.Task] = set()

  @ctx.room.on("data_received")
  def on_data(packet: rtc.DataPacket):
      t = asyncio.create_task(handle())
      _bg.add(t)
      t.add_done_callback(_bg.discard)

To split this quickly, put a print at the top of on_data before the task is created. If it never fires, the packets are not reaching the agent and the problem is upstream of your handler. If it fires but nothing is spoken, it is the task or the generate_reply call, and those are very different fixes.

One thing worth knowing either way: data sent through RoomService.SendData has no sending participant, so packet.participant will be None. That does not affect you now since you do not filter on it, but it will bite if you add sender filtering later.

[Kamal_Moha]

You already use ctx.update and ctx.with_filler, so just await the Inngest run instead of returning early. The job is its own async process, so this does not block other jobs.

@Muhammad_Usman_Bashir I cannot simply do;

result = await track_and_wait(tracking_num, self.phone_number)

return result

This is because inngest just returns the background job id. The REAL execution of the tool check_shipping_status() is done on the backend server, inngest (outside of Livekit).

The reliable way is what the framework is built for: await the result inside the tool and return it. A long-running tool keeps the agent talking while it waits (async tools)

Livekit AsyncTools assumes/expects that the execution of the tool is done within the livekit agent. It doesn’t support executing tools as background jobs that can be returned back after several minutes from a third party platform.

Only use out-of-band SendData for genuinely fire-and-forget events, and even then the agent must receive it and call session.generate_reply, with the session still live.

For my case, it’s really fire-and-forget. The job of livekit is to just initiate the tool and send an event to the backend, then the actual tool execution is done on the backend. The backend server is REQUIRED to inform the livekit room/agent about the tool result while the session is still alive. That’s why I’m using SendData.

@Muhammad_Usman_Bashir I have edited my post with extra info on what I’m trying to do & the several steps I have to go through. I have also included relevant screenshots. Please reread the post to get some context.

[Kamal_Moha]

@abidullahcs.uk Thanks for the guidance on asyncio.create_test. I will incorporate the suggested code

_bg: set[asyncio.Task] = set()

@ctx.room.on("data_received")
def on_data(packet: rtc.DataPacket):
t = asyncio.create_task(handle())
_bg.add(t)
t.add_done_callback(_bg.discard)

I have kept the event in SendData to be sent to everyone in the room. Ideally, the event should be sent to the agent and agent informs the human about the received data.

@abidullahcs.uk I have edited my post with extra info on what I’m trying to do & the several steps I have to go through. I have also included relevant screenshots. Please reread the post to get some context.

[abidullahcs.uk (Abid)]

@Kamal_Moha read the update, and your reasoning for keeping execution in Inngest makes sense. Step 4 has no tool call in flight at all, so the “await it inside the tool” shape does not apply there either way.

One correction worth making, to my own earlier hint as much as anything. Those data channel errors do not mean the session had gone. In rtc_session.rs that log is guarded:

  if !inner.closed.load(Ordering::Acquire)
      && !inner.disconnecting.load(Ordering::Acquire)
      && inner.publisher_pc.is_connected()
  {
      log::error!("publisher data channel '{}' closed unexpectedly", label);
  }

It only fires when the session is not closed, not disconnecting, and the publisher peer connection is still up. The guard exists precisely to stay quiet during normal teardown, so seeing it means a channel dropped mid-session while everything else was still live.

Two things follow. Those are publisher channels, so they cover the agent sending rather than receiving, and they may not explain the missed inbound update at all. And whatever drops them is a real transport issue worth chasing on its own.

[Kamal_Moha]

Step 4 has no tool call in flight at all, so the “await it inside the tool” shape does not apply there either way.

@abidullahcs.uk Step 4 & 6 are about informing/updating the livekit room/agent about an external event from a third party platform/API.

Those data channel errors do not mean the session had gone. In rtc_session.rs that log is guarded:

What do those errors mean?

On my side, the livekit room/session is kept alive before SendData is attempted. I’m not sure why that error is happening.

[abidullahcs.uk (Abid)]

@Kamal_Moha the comment sitting above that handler in rtc_session.rs says it plainly:

  // Log when a publisher data channel closes without the engine or peer
  // connection tearing it down

So it means one of the publisher data channels went to Closed while the engine was not shutting down and the publisher peer connection was still connected. Nothing more is inferred from it, and the handler only logs. There is no recovery or error propagation attached to it, so it is diagnostic rather than something failing your call.

@Kamal_Moha

What do those errors mean?

On my side, the livekit room/session is kept alive before SendData is attempted. I’m not sure why that error is happening.

The useful part for you is which channels those are. The session holds two sets: reliable_dc, lossy_dc and data_track_dc on the publisher side, and separately sub_reliable_dc, sub_lossy_dc and sub_data_track_dc for the subscriber side. Your three errors are all publisher ones, which carry data the agent sends out. Inbound SendData reaches the agent over the subscriber channels, so these errors do not explain a missed inbound update. They are a real problem, just not that problem.

As for why they close mid session while the connection is still up, I cannot tell from the client source, and I would rather not guess. That is worth putting to LiveKit directly with your job and room IDs, since it needs server side visibility.

[Kamal_Moha]

@abidullahcs.uk I have changed my workflow on how the Livekit agent will get updates when backend server (inngest) completes the background job. I’m no longer using SendData to update the agent when background job is completed. Instead, I’m using Redis pub/sub to handle this. Now, this is how the flow is when running a tool;

1. Livekit agent subscribes to a Redis channel
2. The livekit agent then initiates the tool by triggering an inngest event that is run as a background job.
3. Inngest executes the tool, acquires the result.
4. Then inngest publishes the result into the redis channel that the livekit agent is subscribed to.
5. Then the livekit agent that’s subscribed to the channel gets the tool result immediately and it’s returned as a normal tool return. Then agent speaks with the user about the result.

This is how things will work now. This flow looks to be much more efficient than forcing a SendData.

I’m facing issues with how the Livekit agent is handling the result it’s receiving when the background job is completed. Here is the link to the agent.py Livekit Agent that runs tools as background jobs. Subscribes to a Redis channel to get updates when its published · GitHub . Here are the two issues I have experienced with it.

1. I’m seeing that the background job result is being returned as as the tool output which is correct. But then now, the livekit agent is NOT verbally acknowledging that and it’s not saying this result to the user. It even hallucinates yet the tool result has been returned to it.

As you can see on the above image from the logs. The tool is returned with the correct output, but the agent doesn’t verbally speak the result. Even when I ask the agent, it still thinks that it hasn’t completed the tool. I’m not sure why this is happening.

2. My tool is an AsyncTool, but when I’m interacting with the agent, the agent doesn’t keep updating me on where it’s with things. I’m using ctx.update, but agent doesn’t update. I see that the agent just keeps quiet when it’s executing the tool and it shouldn’t be doing that.

Is there anything wrong with my agent.py that’s causing the issues above?

[abidullahcs.uk (Abid)]

@Kamal_Moha Both of those are the same mechanism, and your screenshot already shows it.

The _final suffix on that call_id is the tell. In voice/tool_executor.py:

entry_id = call_id + “_final” if run_ctx._updates else call_id

That suffix is only appended when run_ctx._updates is non-empty, so your ctx.update() calls are firing and being recorded. That is not the broken part.

What happens next is what bites. The final result goes through _enqueue_reply, which inserts the pair into the chat context immediately:

  chat_ctx.insert(items)
  await target.update_chat_ctx(chat_ctx)
  ctx.session.history.insert(items)
  self._pending_updates.append(...)

but speaking it is a separate step, _deliver_reply, and that one opens with:

target_activity = await session.wait_for_idle()

wait_for_idle is documented as “wait until this activity has no in-flight agent or user work”, and it waits on both by default (wait_for_agent=True, wait_for_user=True). So the result lands in context right away while the spoken reply parks until the session goes idle.

Now look at your timeline: agent speaks at 01:27.62, then user input at 01:28.25 and again at 01:29.07. If user turns keep arriving, the activity never reaches idle and the deferred reply stays queued. Your progress updates go through the same gate, which is why they are silent too.

So it is one root cause rather than two bugs, and it is not the Redis part of your redesign. Worth logging when wait_for_idle actually resolves in one of those runs to confirm it on your side.

I have not read your gist, so this is from the SDK and your screenshot rather than your code.

[Kamal_Moha]

@abidullahcs.uk Here is an overview of the timeline of crucial events.

01:20.19 - User input

01:20.25 - Agent start running the tool

01:27.10 - Tool output is returned

01:27.49 - User asks whether tool is completed

01:27.62 - Agent says, I’m still running the tool.

As you can see here, the agent should have known that the tool output has been returned, so it should have update the user & continue. But instead the agent says, I’m still running the tool which is wrong. And other times it hallucinates.

Your progress updates go through the same gate, which is why they are silent too.

How can I fix this? What is the right way to fix this? I want the progress updates to be communicated with the user as agent is waiting for the tool result to be returned.

So it is one root cause rather than two bugs, and it is not the Redis part of your redesign.

Yes, Redis is not the issue. It’s that the livekit agent is not behaving as it’s expected.

I have not read your gist, so this is from the SDK and your screenshot rather than your code.

If the issue is with the SDK (I’m using python), I will have to open an issue in livekit/agents github, but I’m still unclear & not understanding the SDK issue.

Anyway, please go through the gist and tell me if there is anything wrong with my code.

[abidullahcs.uk (Abid)]

Read the gist. The cause is your two ctx.update() calls, specifically the first one.

In voice/events.py, the first update short-circuits:

  if not self._first_update_fut.done():
      self._first_update_fut.set_result(message)
      self._function_call.extra["__livekit_agents_tool_non_blocking"] = True
      return

with the comment above it: # first update keeps the original call_id. So the first ctx.update() ends the blocking call and its message becomes the output for the original call_id. It also gets wrapped in UPDATE_TEMPLATE before the LLM sees it:

The tool {function_name} has updated, message: {message} The task is still running, so DON’T make up or give information not included in the message above.

At 01:27.49 your model was reading “Checking the shipping status for tracking number 1Z81…” plus a literal instruction not to invent anything beyond it. Saying “I’m still checking” is it obeying that. Not a hallucination, and nothing to file.

Your real return then goes out as the _final pair through _deliver_reply, which opens with await session.wait_for_idle(), so it waits its turn to be spoken.

Fix: delete both ctx.update() calls and keep ctx.with_filler(). Its docstring is explicit: “Schedule filler speech while a long-running step blocks the tool.” Fillers go through session.say, not the update pair, so they keep the user informed without hijacking your return value. The tool then blocks on the Redis push and the return becomes a normal tool output the agent speaks immediately. Your delay=5, interval=10, max_steps=3 is already right for that,
and the finally cleanup is fine.

Correction to my last post: I put the weight on user turns keeping the session non-idle. That gate is real, but your timeline showing the output at 01:27.10 before the 01:27.49 user turn is what pointed at the actual cause.

[Kamal_Moha]

I have updated the agent.py as you have suggested. Check the last comment on the gist

I have deleted both ctx.update() calls and kept the ctx.with_filter(). Redis does its thing and tool output is returned. But I still see that;

• The user is not continuously updated as the tool is awaiting output
• When tool output is returned, the agent doesn’t speak this output to the user.

Here is the the timeline:

1. 00:44.10 - Agent receives the tracking number.
2. 01:21.90 - Starts executing tool
3. 01:39.40 - Tool output is returned

4. 01:41.32 - User asks whether tool execution is done
5. 01:41.45 - Agent says, it’s still executing tool.

I’m not sure why Livekit is behaving like this. Weird

[abidullahcs.uk (Abid)]

@Kamal_Moha You removed two of the three. There is still a ctx.update() in the updated version, the one just before the return:

await ctx.update(f"Tracking complete for {tracking_num}. Preparing response for the customer.")

That one is now the first update, so it does exactly what the other two were doing: resolves first_update_fut, flags the call non-blocking, and its
message becomes the output for the original call_id wrapped in UPDATE_TEMPLATE.

Your screenshot confirms it. You have two track_package entries and the first one reads:

"The tooltrack_package has updated, message: Tr…"

That is the template, not your return. And at 01:41.45 the agent says “I have successfully tracked your order using the number 1Z81R3Y36707933065. I’m now processing the details and will share the current status with you shortly.” That is your “Tracking complete for 1Z81… Preparing response for the customer” plus the template’s trailing line: “The task is still running, so DON’T make up or give information not included in the message above.” The model is quoting you and obeying the instruction. Delete that third call and the tool blocks properly, and your return becomes the normal output.

Separately, on the fillers being silent: with_filler waits for the session to be continuously idle for delay seconds before each fire. You have delay=5, and your timeline shows user input at 01:39.48, 01:41.32, 01:42.59 and 01:44.52. Each one breaks the continuous-idle window, so it never reaches 5 seconds of quiet and never fires. Drop delay to 1 or 2 and you should start hearing them.

[Kamal_Moha]

@abidullahcs.uk I have implemented your suggestions and updated the gist.

I can now see that the tool output is being returned as a normal output and the agent is immediately acknowledging that and communicating the tool output with the user.

But still throughout the tool execution while the agent waits for the output, the user is NOT being updated with new updates. The agent is not using ctx.with_filler and is just quiet throughout the period of tool execution. It talks when it receives the output.

[Kamal_Moha]

@abidullahcs.uk I have noticed two scenarios I would like to share

First scenario

await ctx.update(

                f"I'm tracking package {tracking_num} now. This may take a moment."

)

# Rotating fillers, up to 3 plays with 10s between them.

            followups = [

"Almost there, tracking the package.",
"Still working on it, won't be long.",
"Hang tight, almost done.",

]

async with ctx.with_filler(
lambda step: followups[step], delay=2, interval=10, max_steps=len(followups)
): 
....

I have noticed when I have a single ctx.update() before ctx.with_filler()like above, I see that the agent is able to continuously keep updating the user as the tool is being executed.

But when the tool output is returned, in this scenario that output is not used as a normal output and agent doesn’t talk back with the user.

Second Scenario

When I just have ctx.with_filler() and no ctx.update() before it like this;

followups = [

                "Almost there, tracking the package.",

"Still working on it, won't be long.",

"Hang tight, almost done.",

]

async with ctx.with_filler(

lambda step: followups[step], delay=2, interval=10, max_steps=len(followups)

): 

...

In this scenario, I see that when the tool execution is going through, the agent doesn’t update the user. But when the tool output is returned, it uses the tool output as a normal output and speaks with the user about the output.

---

What I need is to have both to work. I want the agent to continuously keep updating the user as the tool is being executed. And then once the tool output is returned, it should use it as a normal output and actually speak with ther user about the output.

[abidullahcs.uk (Abid)]

@Kamal_Moha Your two scenarios are the same thing seen from both ends, and right now you can’t have both.

with_filler opens its loop with await self._session.wait_for_idle() (voice/filler_scheduler.py), and wait_for_idle counts the agent as busy while a speech handle is still current. A blocking tool keeps that handle alive. The SDK admits as much in an unrelated guard message: “the speech handle is waiting for the function tool to complete”.

So Scenario 2, the tool blocks, session never goes idle, filler loop never gets past line one. Scenario 1, your ctx.update() flags the call non-blocking, the handle lets go, session goes idle, fillers fire, but the return value was already handed over so the real result comes back deferred.

Fillers need idle. Blocking tools prevent idle. That’s it.

I’d file this one. Your two scenarios are a tidy repro, and with_filler’s own docstring says it’s for “filler speech while a long-running step blocks the tool”, which is exactly when it can’t fire.

For now just drive them yourself and keep the tool blocking:

  async def _fillers(session, lines, first=2.0, gap=10.0):
      await asyncio.sleep(first)
      for line in lines:
          session.say(line)
          await asyncio.sleep(gap)

  filler_task = asyncio.create_task(_fillers(ctx.session, followups))
  try:
      # wait on redis exactly as you do now
      ...
  finally:
      filler_task.cancel()

Same thing _FillerScheduler does internally, minus the idle gate. You do lose the guard that keeps a filler from landing while the user is mid-sentence, so keep gap generous.

[Kamal_Moha]

@abidullahcs.uk

and right now you can’t have both.

Does this mean that with the current version of the SDK AsyncTool, we can’t have it continuously update the user and aslo use tool output as a normal output. Are having these two things mutually exclusive?

@abidullahcs.uk

I’d file this one.

Will I have to file an issue in livekit/agents github?

Isn’t AsyncTool designed to have ctx.update(), ctx.with_filler() and tool output to work alongside each other?