Clarification on usage of vad/turn detector about whether only Agentsession supports or Agent also supports it?

Getting Started
,
1

Looking for clarification on VAD / turn detection usage in LiveKit Agents.

I understand that AgentSession accepts a vad parameter:

session = AgentSession(
    userdata=userdata,
    user_away_timeout=settings.MAX_INACTIVITY_DURATION,
    vad=ctx.proc.userdata["vad"],
)

However, I also see types of Agent class where VAD and turn detection are supported directly on the Agent:

super().__init__(
    stt=stt,
    tts=tts,
    llm=llm,
    vad=vad,
    turn_handling=TurnHandlingOptions(
        turn_detection=build_turn_detection(),
        endpointing={...},
        interruption={...},
    ),
)

My question is:

  • Does VAD work when configured only on the Agent, or is it only supported/configured at the AgentSession level?

  • If both AgentSession(vad=...) and Agent(vad=...) are provided, which one takes precedence?

  • Similarly, is turn_detection supported only through Agent.turn_handling, or can it also be configured at the session level?

  • What is the recommended approach for current LiveKit Agents versions?

Trying to understand the separation of responsibilities between AgentSession and Agent for VAD, endpointing, interruptions, and turn detection.

Related github issue created: Bug: preemptive_generation={"enabled": False} and endpointing={ "min_delay": 1 } is ignored — session report always shows defalt values · Issue #6112 · livekit/agents · GitHub

2

The recommended approach is to define both VAD and turn detection at the AgentSession level, Turns overview | LiveKit Documentation . In general, the techniques and code samples documented at docs.livekit.io are considered authoritative and correct.

Are you trying to have two different agents in a single AgentSession with different options? In that case, I think, the settings defined at the agent-level would apply, but I’m not sure. At least, that’s what it looks like from the code.

In general, I would expect these settings to be applied at the AgentSession as that is responsible for orchestrating VAD / turn detection within the room.

3

Yes, i have two seperate agents in same session that will be changed after phase change i.e agent handoff and both needs to have bit different settings, If livekit can confirm the proper usage of it agent class vs agent session with proper description on what will be used and when, it would be very helpful.

Needed clarification for both turn_detection as well as vad.

But Thanks @darryncampbell , i will have turn detector settings at agent session as of now and vad at both agent as looks like it will use from agent if passed.

4

Yes, i have two seperate agents in same session that will be changed after phase change i.e agent handoff and both needs to have bit different settings, If livekit can confirm the proper usage of it agent class vs agent session with proper description on what will be used and when, it would be very helpful.

This would be a good, we don’t have anything documented for this use case today so I’ll add it to my backlog of technical articles to create.