Auto STT model errors leads to stalled agent

Getting Started
,
1

Over the last week, I’ve noticed a spike in errors during the STT phase when using the ‘auto’ model. Specifically, most of the errors match the one attached below. I wanted to report this and ask for a solution or workaround, as it is affecting roughly one out of every three calls.

The typical behavior is that I say something and the agent doesn’t respond; however, after I repeat it a couple of times, it suddenly recovers.

[21:34:10.249] WARN (41350): Failed to parse STT server event
    rawData: {
      "type": "final_transcript",
      "session_id": "09c99ccd-c000-477d-b52b-0bc9f9867e9e",
      "transcript": "",
      "start": 34.844,
      "duration": 7.7,
      "confidence": 1,
      "words": null,
      "language": "es",
      "extra": null
    }
    error: {
      "type": "ZodError",
      "message": "[\n  {\n    \"expected\": \"array\",\n    \"code\": \"invalid_type\",\n    \"path\": [\n      \"words\"\n    ],\n    \"message\": \"Invalid input: expected array, received null\"\n  }\n]",
      "stack":
          ZodError: [
            {
              "expected": "array",
              "code": "invalid_type",
              "path": [
                "words"
              ],
              "message": "Invalid input: expected array, received null"
            }
          ]
              at new ZodError (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/core.js:39:39)
              at Module.<anonymous> (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/parse.js:53:20)
              at inst.safeParseAsync (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/classic/schemas.js:42:57)
              at recv (/Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/@livekit+agents@1.2.0_@livekit+rtc-node@0.13.24_zod@4.3.6/node_modules/@livekit/agents/src/inference/stt.ts:502:60)
              at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
              at async run (/Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/@livekit+agents@1.2.0_@livekit+rtc-node@0.13.24_zod@4.3.6/node_modules/@livekit/agents/src/utils.ts:501:14)
      "name": "ZodError"
    }
[21:34:10.574] DEBUG (41350): VAD task: END_OF_SPEECH
[21:34:10.574] DEBUG (41350): running EOU detection
    audioTranscript: ""
[21:34:10.574] DEBUG (41350): skipping EOU detection
[21:34:10.574] DEBUG (41350): overlap speech ended
[21:34:20.191] WARN (41350): Failed to parse STT server event
    rawData: {
      "type": "final_transcript",
      "session_id": "09c99ccd-c000-477d-b52b-0bc9f9867e9e",
      "transcript": "",
      "start": 42.544000000000004,
      "duration": 9.95,
      "confidence": 1,
      "words": null,
      "language": "es",
      "extra": null
    }
    error: {
      "type": "ZodError",
      "message": "[\n  {\n    \"expected\": \"array\",\n    \"code\": \"invalid_type\",\n    \"path\": [\n      \"words\"\n    ],\n    \"message\": \"Invalid input: expected array, received null\"\n  }\n]",
      "stack":
          ZodError: [
            {
              "expected": "array",
              "code": "invalid_type",
              "path": [
                "words"
              ],
              "message": "Invalid input: expected array, received null"
            }
          ]
              at new ZodError (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/core.js:39:39)
              at Module.<anonymous> (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/parse.js:53:20)
              at inst.safeParseAsync (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/classic/schemas.js:42:57)
              at recv (/Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/@livekit+agents@1.2.0_@livekit+rtc-node@0.13.24_zod@4.3.6/node_modules/@livekit/agents/src/inference/stt.ts:502:60)
              at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
              at async run (/Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/@livekit+agents@1.2.0_@livekit+rtc-node@0.13.24_zod@4.3.6/node_modules/@livekit/agents/src/utils.ts:501:14)
      "name": "ZodError"
    }
[21:34:25.574] DEBUG (41350): User away timeout triggered
[21:34:26.169] WARN (41350): Failed to parse STT server event
    rawData: {
      "type": "final_transcript",
      "session_id": "09c99ccd-c000-477d-b52b-0bc9f9867e9e",
      "transcript": "",
      "start": 52.494,
      "duration": 5.95,
      "confidence": 1,
      "words": null,
      "language": "es",
      "extra": null
    }
    error: {
      "type": "ZodError",
      "message": "[\n  {\n    \"expected\": \"array\",\n    \"code\": \"invalid_type\",\n    \"path\": [\n      \"words\"\n    ],\n    \"message\": \"Invalid input: expected array, received null\"\n  }\n]",
      "stack":
          ZodError: [
            {
              "expected": "array",
              "code": "invalid_type",
              "path": [
                "words"
              ],
              "message": "Invalid input: expected array, received null"
            }
          ]
              at new ZodError (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/core.js:39:39)
              at Module.<anonymous> (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/parse.js:53:20)
              at inst.safeParseAsync (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/classic/schemas.js:42:57)
              at recv (/Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/@livekit+agents@1.2.0_@livekit+rtc-node@0.13.24_zod@4.3.6/node_modules/@livekit/agents/src/inference/stt.ts:502:60)
              at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
              at async run (/Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/@livekit+agents@1.2.0_@livekit+rtc-node@0.13.24_zod@4.3.6/node_modules/@livekit/agents/src/utils.ts:501:14)
      "name": "ZodError"
    }
[21:34:27.450] WARN (41350): Failed to parse STT server event
    rawData: {
      "type": "final_transcript",
      "session_id": "09c99ccd-c000-477d-b52b-0bc9f9867e9e",
      "transcript": "",
      "start": 58.444,
      "duration": 1.3,
      "confidence": 1,
      "words": null,
      "language": "es",
      "extra": null
    }
    error: {
      "type": "ZodError",
      "message": "[\n  {\n    \"expected\": \"array\",\n    \"code\": \"invalid_type\",\n    \"path\": [\n      \"words\"\n    ],\n    \"message\": \"Invalid input: expected array, received null\"\n  }\n]",
      "stack":
          ZodError: [
            {
              "expected": "array",
              "code": "invalid_type",
              "path": [
                "words"
              ],
              "message": "Invalid input: expected array, received null"
            }
          ]
              at new ZodError (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/core.js:39:39)
              at Module.<anonymous> (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/parse.js:53:20)
              at inst.safeParseAsync (file:///Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/classic/schemas.js:42:57)
              at recv (/Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/@livekit+agents@1.2.0_@livekit+rtc-node@0.13.24_zod@4.3.6/node_modules/@livekit/agents/src/inference/stt.ts:502:60)
              at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
              at async run (/Users/carloslostao/Eighteen/realtie/node_modules/.pnpm/@livekit+agents@1.2.0_@livekit+rtc-node@0.13.24_zod@4.3.6/node_modules/@livekit/agents/src/utils.ts:501:14)
      "name": "ZodError"
    }
2

Are you just defining auto per https://docs.livekit.io/agents/models/stt/#automatic-model-selection ? As a workaround you can specify the STT provider directly with LiveKit Inference, e.g. Deepgram would be: Deepgram STT | LiveKit Documentation

Can you share your STT config so I can reproduce this?

3

Sorry for the delay:

    stt: new inference.STT({
      model: 'auto',
      language: 'es',
      modelOptions: { keyterms: KEYTERMS },
    }),

This is the configuration I have

4

Thanks for getting back to me.

I notice the docs say to use stt: "auto:es", but I still see the same ZodError if I follow the docs exactly, even with the latest Agents JS release, 1.2.8.

It looks like a bug to me, can I please ask you to raise an issue in the Agents JS repo at Issues · livekit/agents-js · GitHub, that way you will receive updates.

If you let me know the ID of the issue, I’ll also raise awareness internally.

FYI, obviously ZodError is coming from JavaScript, but I did try this setting with a Python agent in case there was some common error or root cause, but this works fine in Python.