Runner initialization timed out

Agents
1

Hello guys.

I’m trying run my agent, was working, but now, every time that I try run it, show the error message bellow.

Some one know mean what is happen in my code? Or possible motive?

Error: runner initialization timed out
at Timeout._onTimeout (C:\alfred_system\back-end\node_modules@livekit\agents\src\ipc\supervised_proc.ts:176:24)
at listOnTimeout (node:internal/timers:581:17)
at process.processTimers (node:internal/timers:519:7)

Node.js v20.19.5

——-

Bellow, is my initialization:

export default defineAgent({

prewarm: async (proc: JobProcess) => {

if (!proc.userData.vad) {

  proc.userData.vad = await silero.VAD.load();

}

},

entry: async (ctx: JobContext) => {

try {

  await ctx.connect();



  const vad = ctx.proc.userData.vad! as silero.VAD;



  const participants = Array.from(ctx.room.remoteParticipants.values());

  const user = participants\[0\];

….

….

await session.start({

    agent: assistant,

    room: ctx.room,

    inputOptions: {

      noiseCancellation: BackgroundVoiceCancellation(),

    },

  });



  const handle = session.generateReply({

    instructions: “…“  });

  await handle.waitForPlayout();



  ctx.room.once('disconnected', () => {

    Logger.log(\`Cleaning up session for user\`);

    *// Nullify references to help Garbage Collection*

    (tools as any) = null;

    (authService as any) = null;

  });

} catch (error) {

  Logger.error(error);

  return;

}

},

});

cli.runApp(

new ServerOptions({

agent: path.resolve(process.cwd(), 'src/ai/agent.ts'),

}),

);

2

This error usually means the agent worker process didn’t finish booting in time, so the parent “runner” killed it. In Node.js, each job runs in its own process, and heavy or blocking code during module load or prewarm can cause this.

From the quickstart example, the expected pattern is:

  • export default defineAgent({ ... })

  • cli.runApp(new ServerOptions({ agent: fileURLToPath(import.meta.url), agentName }))

  • No long synchronous work at the top-level

  • await ctx.connect() inside entry (after session setup is fine, as shown in the quickstart)

Common causes of runner initialization timed out:

  1. A top-level await, infinite loop, or long sync operation before defineAgent is exported.

  2. silero.VAD.load() hanging (model files not downloaded). Make sure you ran:

    pnpm download-files

    as described in the Voice AI quickstart (/agents/start/voice-ai-quickstart/).

  3. Wrong agent path in ServerOptions (must point to the built JS file in production).

First step: run with debug logs:

pnpm start -- --log-level=debug

(as shown in Server options → Log levels: /agents/server/options/)

Can you confirm:

  • Are you running dev or start?

  • Did you recently change Node version or rebuild?

  • Does it fail before any of your own logs print?

3

For this error, I updated my agents and fix some tools that was without required atributes. But now, I have other error: Is starting my agent, saying the first phrase, but when I speak back, not understand…
[22:24:36.542] DEBUG (5804): Task.runTask: task AgentActivity.pipelineReply done
[22:24:38.334] DEBUG (5804): VAD task: START_OF_SPEECH
[22:24:41.534] DEBUG (5804): VAD task: END_OF_SPEECH
[22:24:41.535] DEBUG (5804): running EOU detection
audioTranscript: “”
[22:24:41.536] DEBUG (5804): skipping EOU detection
[22:24:47.806] DEBUG (5804): VAD task: START_OF_SPEECH
[22:24:55.355] DEBUG (5804): VAD task: END_OF_SPEECH
[22:24:55.356] DEBUG (5804): running EOU detection
audioTranscript: “”
[22:24:55.356] DEBUG (5804): skipping EOU detection

What can be?

Other thing that I want know, is about ServerOptions and WorkerOptions, now I’m using ServerOptions, but before I was using worker. What is the difference between both? When is better use server or worker?

And thanks so much for help me bro.

4

Try this agent and see if it works for you:

5

fileURLToPath not work with my configs, I’m using nestJs.

About the error, I found: The STT is not working. I tried change from assembly to cartesia, but not work too. The apy key is correct: “ASSEMBLYAI_API_KEY=”, stt: new inference.STT({

      model: 'assemblyai/universal-streaming-multilingual',

      language: 'pt-BR',

    }),

What can be?

6

jobID: “AJ_QaqtSSU8UhHU”
error: {
“type”: “Error”,
“message”: “Unhandled error. ({\n type: ‘error’,\n error: APIConnectionError: Error connecting to LiveKit WebSocket\n at WebSocket.onError (C:\alfred_system\back-end\node_modules\@livekit\agents\src\inference\utils.ts:48:16)\n at Object.onceWrapper (node:events:639:26)\n at WebSocket.emit (node:events:524:28)\n at emitErrorAndClose (C:\alfred_system\back-end\node_modules\ws\lib\websocket.js:1046:13)\n at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {\n body: null,\n retryable: true\n },\n source: STT {\n _events: [Object: null prototype] {\n metrics_collected: [Function: onMetricsCollected],\n error: [Function: onModelError]\n },\n _eventsCount: 2,\n _maxListeners: undefined,\n opts: {\n model: ‘assemblyai/universal-streaming-multilingual’,\n language: ‘pt-BR’,\n encoding: ‘pcm_s16le’,\n sampleRate: 16000,\n baseURL: ‘wss://agent-gateway.livekit.cloud/v1’,\n apiKey: ‘,\n apiSecret: ’,\n modelOptions: {},\n fallback: undefined,\n connOptions: [Object]\n },\n streams: Set(1) { [SpeechStream] },\n [Symbol(shapeMode)]: false,\n [Symbol(kCapture)]: false\n },\n createdAt: 1772157680332\n})”,
“stack”:
Error [ERR_UNHANDLED_ERROR]: Unhandled error. ({
type: ‘error’,
error: APIConnectionError: Error connecting to LiveKit WebSocket
at WebSocket.onError (C:\alfred_system\back-end\node_modules@livekit\agents\src\inference\utils.ts:48:16)
at Object.onceWrapper (node:events:639:26)
at WebSocket.emit (node:events:524:28)
at emitErrorAndClose (C:\alfred_system\back-end\node_modules\ws\lib\websocket.js:1046:13)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
body: null,
retryable: true
},
source: STT {
_events: [Object: null prototype] {
metrics_collected: [Function: onMetricsCollected],
error: [Function: onModelError]
},
_eventsCount: 2,
_maxListeners: undefined,
opts: {
model: ‘assemblyai/universal-streaming-multilingual’,
language: ‘pt-BR’,
encoding: ‘pcm_s16le’,
sampleRate: 16000,
baseURL: ‘wss://agent-gateway.livekit.cloud/v1’,
apiKey: ‘…’,
apiSecret: ‘…’,
modelOptions: {},
fallback: undefined,
connOptions: [Object]
},
streams: Set(1) { [SpeechStream] },
[Symbol(shapeMode)]: false,
[Symbol(kCapture)]: false
},
createdAt: 1772157680332
})
at AgentSession.emit (node:events:513:17)
at AgentSession.emit (C:\alfred_system\back-end\node_modules@livekit\agents\src\voice\agent_session.ts:255:18)
at AgentActivity.onError (C:\alfred_system\back-end\node_modules@livekit\agents\src\voice\agent_activity.ts:598:25)
at STT.onModelError (C:\alfred_system\back-end\node_modules@livekit\agents\src\voice\agent_activity.ts:138:10)
at STT.emit (node:events:524:28)
at SpeechStream.emitError (C:\alfred_system\back-end\node_modules@livekit\agents\src\stt\stt.ts:256:15)
at SpeechStream.mainTask (C:\alfred_system\back-end\node_modules@livekit\agents\src\stt\stt.ts:230:18)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
“code”: “ERR_UNHANDLED_ERROR”,
“context”: {
“type”: “error”,
“error”: {
“body”: null,
“retryable”: true,
“name”: “APIConnectionError”
},
“source”: {
“_events”: {},
“_eventsCount”: 2,
“opts”: {
“model”: “assemblyai/universal-streaming-multilingual”,
“language”: “pt-BR”,
“encoding”: “pcm_s16le”,
“sampleRate”: 16000,
“baseURL”: “wss://agent-gateway.livekit.cloud/v1”,
“apiKey”: “…”,
“apiSecret”: “…”,
“modelOptions”: {},
“connOptions”: {
“maxRetry”: 3,
“retryIntervalMs”: 2000,
“timeoutMs”: 10000
}
},
“streams”: {}
},
“createdAt”: 1772157680332
}
}

7

That line implies you are conflating STT with LiveKit inference and STT with Plugins. Only the latter requires an API key whereas the former is specified by inference.STT'(not new inference.STT)

Inference docs for AssemblyAI: AssemblyAI STT | LiveKit Documentation

Plugin docs for AssemblyAI: AssemblyAI plugin guide | LiveKit Documentation

Your error is probably related to that. I recommend you first try with the basic_agent.ts that @CWilson pointed to above, since that is a known-working sample. That will use LiveKit inference, so can be run on the free tier without additional API keys.

8

I found the problem: I was using assemblyAI with the inference… how can I run using an plugin in node? Is possible? Because assembly is very better then openai to STT… and the price is the same. Can I run e.g. other plugin with assembly credentials to work?

9

Unfortunately, the assemblyAI STT plugin is only available in Python: AssemblyAI plugin guide | LiveKit Documentation (see the ‘Available In’, towards the top of the page).

As you say, Inference interface for AssemblyAI is available in both Node.js and Python.