Can I dynamically inject keyterms for STT?

Agents
, ,
1

For example if we have a user lookup tool during the call, I’d want to inject the first and last name as keyterms for just that call to improve STT accuracy.

2

You can use

@server.rtc_session(agent_name="dynamic-keyterms-agent")

async def entrypoint(ctx: JobContext):

# ---- Example: result from your user lookup tool ----

    first_name = "John"

    last_name = "Doe"

    keyterms = [first_name, last_name]

# -----------------------------------------------------

    session = AgentSession(

        stt=inference.STT(

            model="deepgram/nova-3",

            language="en",

            extra_kwargs={

# Deepgram keyword boosting example

# Format depends on provider API; adjust boost value as needed

"keywords": [{"keyword": term, "boost": 10} for term in keyterms]

},

),

        llm=inference.LLM(model="openai/gpt-4.1-mini"),

        tts=inference.TTS(model="cartesia/sonic-3"),

        turn_detection=MultilingualModel(),

        vad=ctx.proc.userdata["vad"],

) 
    :
    :

see code here

3

Hey @CWilson thanks for responding. Does this solution work mid call? The case I’m looking at is when we execute a tool mid call and want to dynamically turn some of that response into keywords

4

Short answer: no — not automatically.

The stt instance is created when the AgentSession starts, so parameters like extra_kwargs (including keyword boosts) are fixed for the lifetime of that session. See the STT overview and configuration model here:
https://docs.livekit.io/agents/models/stt.md
and Deepgram parameters here:
https://docs.livekit.io/agents/models/stt/inference/deepgram.md

If you need to change keywords mid-call, you have two supported approaches:

  1. Recreate or swap the STT instance (advanced).

  2. Override the stt_node and manage the STT stream yourself, so you can stop the current stream and start a new one with updated parameters after your tool runs.

The stt_node is explicitly designed for this kind of customization:
https://docs.livekit.io/agents/logic/nodes.md

In practice, you’d:

  • Detect when your tool returns new entities (e.g., name).

  • Close the current STT stream.

  • Instantiate a new STT with updated extra_kwargs.

  • Continue transcription with the new stream.