FallbackAdapter with aligned_transcript

Getting Started
1

Hi, I’m running into issues using FallbackAdapter for STT returning false for aligned_transcript even if the underlying models support aligned transcript.

The models I had wrapped in the FallbackAdapter were:

  • Deepgram’s nova-3
  • AssemblyAI’s default model

Is there something I’m doing wrong here? Or is FallbackAdapter just not propagating correctly? Thanks!

2

FallbackAdapter does not override or strip STT capabilities. It wraps multiple STT instances and forwards their capabilities, including aligned_transcript, based on the active provider. If aligned_transcript is returning False, it means the adapter’s effective capabilities (as reported by the wrapped STT instances) do not indicate support.

Aligned transcripts are required for adaptive interruption handling and are determined by the STT plugin’s capabilities.aligned_transcript property, not by the adapter itself. See the STT reference and fallback behavior in Events and error handling and the STT module reference at [/reference/python/livekit/agents/stt/].

3

This is actually an issue with FallbackAdapter. I ran into the same thing and confirmed it with LiveKit support. They’ve already fixed it on their end, but I don’t think the fix has been released yet.

For now, you can use a temporary workaround by manually overriding the capability:

stt_adapter = stt.FallbackAdapter(
            [
                inference.STT(model="deepgram/flux", language="en"),
                inference.STT(model="assemblyai/universal-streaming", language="en"),
            ]
        )
# manual overriding to enable adaptive interruption handling
stt_adapter.capabilities.aligned_transcript = "word"```


Just make sure your underlying STT providers support aligned-transcripts, otherwise fallback behavior could be inconsistent.
4

Thanks @Tejas_Jadhav ,

Looks like that fix was included in Fix/stt fallback adapter propagate aligned transcript by miladmnasr · Pull Request #5237 · livekit/agents · GitHub which was part of the Agents 1.5.2 release:

5

Thanks for the update