How to handle long-running async tool calls with OpenAI Realtime API

Agents
,
1

This question originally came up in our Slack community and the thread has been consolidated here for long-term reference.

When using async function calls with the OpenAI Realtime API, particularly for long tool calls (>10 seconds), how do you manage running the tool call in the background while keeping the conversation going, then having the AI notify the user when the result is ready?

We have delays with tools that require starting services with slow external API discovery.

2

You can use Python’s asyncio fire-and-forget pattern. In the function tool call:

  1. Create an asyncio task that executes in the background
  2. Return a response immediately (e.g., “I’ve initiated X, do something else meanwhile”)
  3. When the task completes, update the chat context with the result for the LLM to take action

Example approach:

async def my_slow_tool(...):
    # Create background task
    asyncio.create_task(do_slow_work(...))
    # Return immediately
    return "I've started the process. I'll let you know when it's ready."

async def do_slow_work(...):
    result = await slow_api_call()
    # Update chat context with result
    await update_chat_context(result)

The background task won’t interfere with the ongoing conversation. The tool output won’t correspond to the original tool call unless you manually link them - you’ll need to update the chat context when the result is ready for the LLM to respond.