inference.TTS doesn't work with FallbackAdapter in node sdk

Agents
1

I’m using a FallbackAdapter and saw these logs in my agent container:

{"level":20,"time":1778729514757,"pid":253,"hostname":"deployment-p-68r506qo8mu-ca-h5rmcco2rxoq-55775bf7b9-wt77v","tts":"inference.TTS","error":{"type":"Error","message":"ChunkedStream is not implemented","stack":"Error: ChunkedStream is not implemented\n    at TTS.synthesize (/app/node_modules/.pnpm/@livekit+agents@1.3.0_@livekit+rtc-node@0.13.27_typescript@6.0.3_zod@3.25.76/node_modules/@livekit/agents/dist/inference/tts.cjs:158:11)\n    at Task.fn (/app/node_modules/.pnpm/@livekit+agents@1.3.0_@livekit+rtc-node@0.13.27_typescript@6.0.3_zod@3.25.76/node_modules/@livekit/agents/dist/tts/fallback_adapter.cjs:130:32)\n    at run (/app/node_modules/.pnpm/@livekit+agents@1.3.0_@livekit+rtc-node@0.13.27_typescript@6.0.3_zod@3.25.76/node_modules/@livekit/agents/dist/utils.cjs:409:25)\n    at AsyncLocalStorage.run (node:internal/async_local_storage/async_hooks:91:14)\n    at Task.runTask (/app/node_modules/.pnpm/@livekit+agents@1.3.0_@livekit+rtc-node@0.13.27_typescript@6.0.3_zod@3.25.76/node_modules/@livekit/agents/dist/utils.cjs:411:36)\n    at new Task (/app/node_modules/.pnpm/@livekit+agents@1.3.0_@livekit+rtc-node@0.13.27_typescript@6.0.3_zod@3.25.76/node_modules/@livekit/agents/dist/utils.cjs:381:10)\n    at Task.from (/app/node_modules/.pnpm/@livekit+agents@1.3.0_@livekit+rtc-node@0.13.27_typescript@6.0.3_zod@3.25.76/node_modules/@livekit/agents/dist/utils.cjs:396:12)\n    at FallbackAdapter.tryRecovery (/app/node_modules/.pnpm/@livekit+agents@1.3.0_@livekit+rtc-node@0.13.27_typescript@6.0.3_zod@3.25.76/node_modules/@livekit/agents/dist/tts/fallback_adapter.cjs:128:47)\n    at Timeout._onTimeout (/app/node_modules/.pnpm/@livekit+agents@1.3.0_@livekit+rtc-node@0.13.27_typescript@6.0.3_zod@3.25.76/node_modules/@livekit/agents/dist/tts/fallback_adapter.cjs:158:16)\n    at listOnTimeout (node:internal/timers:585:17)\n    at process.processTimers (node:internal/timers:521:7)"},"msg":"TTS recovery failed, will retry"}

After some Claude research it seems like the issue is that inference.TTS does not support synthesize, which is what FallbackAdapter uses for it’s health checks, which both pollutes logs + makes it so the backup is not actually working as expected. Just thought I would raise this here as I would like to use FallbackAdapter for reliability.

2

@marc, Your diagnosis is right. On main of agents-js, agents/src/inference/tts.ts:413 has

synthesize(_: string): ChunkedStream { throw new Error('ChunkedStream is not implemented'); }

FallbackAdapter’s tryRecovery (line 130 of fallback_adapter.cjs in your stack) calls synthesize for health checks, so the recovery loop is structurally broken whenever one wrapped TTS is inference.TTS. Upgrading to @livekit/agents@1.4.2 won’t fix it because the stub is still on main.

Two paths until a fix lands:

  • Replace inference.TTS inside the FallbackAdapter with a provider plugin that implements synthesize (cartesia.TTS, elevenlabs.TTS, etc.). You lose Inference routing on that arm but keep working failover.

  • File a bug on livekit/agents-js. No matching issue exists today, so it becomes the canonical tracker. The minimal fix is either implementing synthesize on inference.TTS by buffering the streaming output, or having FallbackAdapter use the streaming path for recovery instead of synthesize.

3

I wasn’t sure when the Fallback adapter for Inference was implemented as I noticed you were using 1.3.0 but it looks like it’s been there since 1.0.43, 1029 · Releases · livekit/agents-js · GitHub. I would still recommend using the latest version of Agents however (but I’m not aware of any fixes that might have resolved this specific issue).

Can you please share your AgentSession configuration, and FallbackAdapter config.

4

Worth flagging for @darryncampbell: I checked the FallbackAdapter constructor on main and there’s no recovery-disable option. FallbackAdapterOptions only exposes ttsInstances, maxRetryPerTTS (default 2), and recoveryDelayMs (default 1000). tryRecovery() runs unconditionally via markUnAvailable(), so no AgentSession or FallbackAdapter config will silence the error as long as inference.

TTS is one of the wrapped instances. The synthesize stub at agents/src/inference/tts.ts:413 is the structural root cause; config can’t route around it. OP’s two viable paths remain swapping inference TTS for a provider plugin (cartesia.TTS, elevenlabs.TTS, etc.) on the fallback arm, or filing the bug.