Gemma (via LiveKit Inference) narrates "let me check" then emits no tool call stalls the turn

Agents
1

Environment

LiveKit Agents (Python), LLM via LiveKit Inference gateway, model google/gemma-4-31b-it. streaming llm_node pass-through.

Problem

On turns that clearly warrant a tool, Gemma frequently emits a terminal text turn that only promises the action, for example “Olá! Com certeza, vou já verificar.” or “Sure, let me look for the reservation right now.”, with zero tool_calls. The turn ends, so the agent waits for the user. The tool only fires on the next user utterance, leaving a roughly 20 second dead-air stall in between.

Reference IDs

Observability session: RM_5JMY2MnP65yZ (agent says “I’ll look for the reservation right now” at 00:23.88, find_reservation does not fire until the caller speaks again at 00:44.07).

tool_choice=“auto” (default): 0/10 clean tool call, 10/10 narration-only stall, 0 errors.

reasoning_effort=“none”: 0/10 clean tool call, 10/10 narration-only stall, 0 errors.

tool_choice=“required”: 10/10 clean tool call, 0/10 narration-only stall, 0 errors.

Prompt hardening (“call the tool silently and immediately, do NOT narrate”), the same rule that works for other tools, only moved it to about 5/15 under the full prompt. Gemma treats tool use as optional and ignores prose rules buried in a large prompt.

What works: tool_choice=“required” forces the call and suppresses content, 10/10, and the Inference gateway accepts it (no 400). But it cannot be a global default: it would force a tool on pure-chat and KB-prose turns too.

2

Hi, there is a conversation about Gemma tool calling here, but without a clear resolution:

Can you share the Observability data for RM_5JMY2MnP65yZ as detailed here:

3

https://cloud.livekit.io/projects/p_lmrgd1emfni/sessions/RM_5JMY2MnP65yZ/observability?mode=metrics

4

Thanks for sharing.

I see the turn in question:

Observability session: RM_5JMY2MnP65yZ (agent says “I’ll look for the reservation right now” at 00:23.88, find_reservation does not fire until the caller speaks again at 00:44.07).

The model’s reply didn’t contain any function_calls, it just contained the “Com certeza, deixa-me ver se é possível. Vou procurar a reserva agora mesmo.” text.

So, the question is why didn’t the model decide to return any function_calls.

I honestly think it comes down to your prompt, it’s very large and frankly probably too large for a model of this size.

I think it’s telling that the model DID respond with a tool call in the next turn, after they had been provided with a reservation name. Your instructions say “If user asks to change details directly, call modify_reservation immediately.” and earlier instructions say “First locate reservation using find_reservation (caller phone is used automatically).” (although that was for a different tool call) — so it looks like the model was confused as the detail WAS in the instructions.

The model may also have been confused because for some tools you say “Do NOT narrate (“I’m booking now”, “one moment”, etc.)” whereas for this modify_reservation you allow “An occasional natural hesitation is fine where a person would pause to think (“hmm”, “deixe-me ver”, “let me see”),”. That nuance may have confused the model.

My recommendation would be to move a lot of this instruction logic into tasks, Tasks and task groups | LiveKit Documentation and keep the instructions simpler.

I don’t think the multiple languages were a primary root cause, but I would suggest when you define examples, to use both languages. I know you do that in a lot of places, but for example here: “If caller says “find my reservation”, “check my reservation”, or similar, call find_reservation immediately before follow-up questions.”, you only give the examples in English

5

thanks for the answer i will try this