Hi, I don’t know why livekit don’t allow use my own agent framework ( like pydantic ai ) . There are a lot of custom configuration, like tool calling . Why not be more framework agnostic ?
LiveKit Agents is opinionated because it tightly integrates realtime media (WebRTC), turn detection, tool calling, and model streaming into a single session pipeline. That’s what enables low‑latency voice, interruptions, and telephony support out of the box.
That said, it isn’t locked to a single LLM or tool system. You can plug in different LLM providers, define your own tools, or even forward tool calls to your frontend or external services, as described in Tool definition & use and the overall Agents overview.
@garbancito, Pydantic AI and LiveKit Agents aren’t competitors; they sit at different layers. LiveKit owns the realtime pipeline (WebRTC, VAD, STT/TTS, turn detection, interruption handling); Pydantic AI owns the reasoning layer. You use both.
The concrete extension point @CWilson hinted at: subclass Agent and override llm_node(). Per the Pipeline nodes and hooks | LiveKit Documentation, every stage is an overridable method: stt_node, llm_node, tts_node, transcription_node, plus hooks like on_user_turn_completed.
Put your Pydantic AI graph inside llm_node(), take chat context in, and yield generated content out. LiveKit doesn’t care whether the tokens came from openai.chat.completions, Pydantic AI, LangGraph, or a hand-rolled state machine. You can also skip LiveKit’s function_tool system and emit tool decisions directly from Pydantic AI.
The framework is agnostic at the llm_node boundary, not at AgentSession.