Has anyone built a voice agent using Agentic RAG? Would love to hear about your approach and experience.
@Mahendra worth knowing the repo already ships both shapes, in examples/voice_agents/llamaindex-rag.
query_engine.py is the agentic one: retrieval is a @llm.function_tool, so the model decides when to look something up. retrieval.py takes the other approach, overriding llm_node to retrieve and inject context before every generation. chat_engine.py is a third variant that hands the whole turn to
LlamaIndex’s chat engine.
The tradeoff between the first two matters much more in voice than in chat. The tool version costs an extra model round trip before anything is spoken, since the model has to decide to call the tool, wait for the result, then generate. The llm_node version retrieves unconditionally but keeps the turn to a single round trip. In text that difference is invisible, on a call it is dead air.
If you go the agentic route, ctx.with_filler is worth looking at, since it is built to speak while a slow tool blocks and stops the silence being the thing the user notices.