We have an issue with inbound calls. If for some reason workers are not available when we receive a lot of concurrent inbound calls, these calls never get answered. We use dispatch rules.
What is the best approach to make sure all inbound calls are picked up by Livekit Agents? I want to retry a few times before abandoning the call.
The recommended approach would be to ensure there is sufficient capacity to handle inbound calls. There is no built-in retry mechanism for agent dispatch, and although you could build one yourself in theory by monitoring whether agents join calls, I wouldn’t recommend it.
Is this for self-hosted agents? I would like to understand the reason why workers aren’t available.
@darryncampbell yes it’s for self hosted agents. The reason for workers not being available is we have a surge of inbound calls and it takes time to scale up the pods in our Kubernetes cluster. We don’t want to waste compute by keeping a lot of agent workers idle.
Got it, we have some documentation for autoscaling for self-hosted agents here:
Did you see that? I would still recommend keeping sufficient spare capacity to scale up in line with your projected load. We also have a load testing guide that my colleague wrote: How to load test voice agents and get meaningful results | LiveKit that explains some of the tools available.
The other thing to recommend would be to use LiveKit cloud hosting, which manages the scaling for you.
How are you currently scaling your workers?
I am very hesitant to recommend an agent dispatching strategy based on monitoring rooms for agent presence, and retrying the dispatch until an agent joins - that will be unreliable.