Deepgram Aura-2 TTS drops the word after a spaced "..." when streamed through livekit-plugins-deepgram — anyone else seeing this?

Self Hosting
,
1

Setup: livekit-agents 1.8.x, livekit-plugins-deepgram 1.8.1, deepgram.TTS(model=“aura-2-theia-en”) in a standard STT → LLM → TTS pipeline (Bedrock Claude Haiku 4.5 as the LLM, Deepgram Flux STT, SIP phone
calls, self-hosted LiveKit).

Symptom: the agent reads a phone number back to the caller and one digit is silently missing from the audio, while the transcript shows the full number. The LLM had written:

▎ Thank you. I received zero, four, zero, three … one, two, three … six, four, eight. Is that correct?

and the caller heard “zero four zero three … two three six four eight” — the “one” after the first … was never spoken. The caller insisted “there’s a one missing” four times; the agent kept reading the same
text and the voice kept skipping it.

What I think is happening: SynthesizeStream._run_ws.send_task sends one {“type”: “Speak”, “text”: " "} message per word token, and the default basic.WordTokenizer(ignore_punctuation=False) splits on
whitespace only. So a spaced … (or -) reaches Aura-2 as a standalone punctuation message, and Aura-2 then sometimes drops the next word. It is intermittent — the same text plays fine on some runs.

Reproduction (real plugin, pushing text word-by-word with ~60 ms gaps to mimic LLM streaming, then transcribing the audio with nova-3 word timestamps and checking for ten digit words; a commas-only control
was clean):

image
image1150×415 13.4 KB

It is not digit-specific: “Let me check that for you … okay, I found it” lost “okay” too. aura-2-andromeda-en showed the same class of failure on different words.

Sending the whole sentence in one Speak message (non-streaming / REST path) does not reproduce it, which is why tts.synthesize() tests look fine.

Workaround I’m running: a custom word_tokenizer= passed to deepgram.TTS(…) whose stream holds one token and, when the next token is …/…, drops it and appends , to the previous word (“three” + “…” →
“three,”). For a pure-digit previous token (0403) it drops the ellipsis without the comma, since 0403, 123 is read as whole numbers but 0403 123 digit-by-digit. With that in place: 10 of 10 intact on every
line, both on aura-2-* and the Flux (TTSv2) voices. Side effect worth knowing: Deepgram starts synthesising at a … or . but waits for the sentence end at a , so first audio on those sentences arrives
~100–150 ms later at a normal LLM token rate.

Questions

  1. Has anyone else seen Aura-2 swallow the word after a standalone … (or other punctuation-only Speak messages)? Any other tokens that trigger it?
  2. Would a plugin-level change be welcome — e.g. merging punctuation-only tokens into the preceding word before sending, or sending on sentence/clause boundaries rather than per word? Happy to open an
    issue/PR with the repro harness if so.
  3. Is the per-word Speak cadence something Deepgram recommends, or was it chosen for TTFB? Curious whether clause-level chunking has been measured against it.
2

@troy Your read of the plugin is correct. tts.py:89 sets basic.WordTokenizer(ignore_punctuation=False) as the default, and tts.py:348 sends {“type”: “Speak”, “text”: f"{word.token} "} per token. In _basic_word.split_words, ignore_punctuation=False leaves translation_table = None and the loop only breaks on
char.isspace(), so a spaced … does become its own Speak message. Confirmed.

The part worth a second look is in your own table. The attached cases still fail:

  - three... one (attached) -> 1 of 7
  - three. One, -> 1 of 5

Neither produces a standalone punctuation token, so tokenization can’t be the whole story. Meanwhile commas are clean at 0 of 17 and bare numerals at 0 of 7.

That matters for your proposed plugin change. Merging a punctuation-only token into the preceding word produces exactly three…, which is your 1-in-7 row. So that change would reduce the failure rate without eliminating it.

Your workaround does better, and I think for a different reason than the framing suggests: it isn’t the merge that saves it, it’s substituting a comma. three + … → three, lands on the 0-of-17 case. That’s the behaviour worth proposing, and it’s a stronger pitch as “map pause punctuation to commas in digit
contexts” than as “merge punctuation tokens”.

On your third question, Deepgram’s text chunking guide (Text Chunking for TTS REST Optimization | Deepgram's Docs) offers max-character, clause/sentence, and dynamic chunking, and says to chunk on clause and sentence boundaries because “When people speak, they tend to pause at the end of a clause or a sentence.” Per-word isn’t among the strategies, and their dynamic example won’t emit subchunks under three characters, which a lone …
violates. So there’s documented support for clause-level chunking, though the latency tradeoff you measured is real.