Setup: voice agents on livekit-agents 1.4.6 (Python 3.11), inbound SIP via Twilio Elastic SIP Trunking → LiveKit Cloud SIP (project p_3tqm7ro6kbs),
agent dispatch via inbound trunk + dispatch rule. Each call = one agent worker process.
What we saw (2026-06-08, UTC):
┌────────────┬────────────────────────────────────────────────────────────────────────────────────────────┐
│ Time │ Event │
├────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 14:29:23 │ Twilio sends the INVITE toward LiveKit (Twilio CallSid CA4083e5e125b6065fd9db3941e2b022a4) │
├────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 14:29:29 │ Caller abandons after ~6s of ringing — Twilio final status no-answer, 0s │
├────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ 14:29:31.5 │ Agent job AJ_NWYstzRMnfLS dispatched — our entrypoint fires (session RM_jusR5FARB6hz) │
├────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
So the job was dispatched ~8s after the INVITE — and ~2.5s after the caller had already hung up.
Why we think the delay is dispatch-side: this was 1 of 208 inbound calls on this trunk that day; the other 207 dispatched within 0.5–2.5s of ring start. The entrypoint timestamp is the first line of the job (no work before it), and the same warm workers dispatched neighboring calls within ~1s.
We have autoscaling and we spin up resources at 50% CPU / memory capacity. So that we can handle the burst of calls quickly.
The default capacity for the worker is 80% I believe. Do you see any logs around why the worker mentioned no worker available? We are on 1.4.6 version of the livekit/agents sdk
What do you have num_idle_processes set to? How do you see the work distributed through out your cluster during that time? How long had the “new” instances been added before the dispatch happened. “Cold” instances can take time to warm up.
What was the load before you hit the high load moment?
num_idle_processes - set to default, 4 cores CPU - so 4 processes
Work distribution - 3 worker pods at the time, on AWS EKS (us-east-2).
Load at the timeframe - steady ~300 job dispatches per 5 minutes (~1 new call/sec) across the 3 pods from 14:15–14:40 UTC — no spike at 14:29
Autoscaling - we scale using CPU and Memory HPAs on Keda when either reach 50% and the pod comes up within 1 min, but in this case there were already 3 pods running ready to serve requests.
I dug into the logs for (RM_cHCojGmsaoQw and the worker logs). Quick summary of what I’m seeing:
This isn’t a dispatch problem, jobs are getting assigned in ~0.2s. The delay is entirely in the agent joining the room after assignment. In the worst case the agent took ~13s to join, and cloud-sip holds the caller in “180 Ringing” until the agent is in, so the caller just hears extended ringing (and some hang up).
The cause is the worker failing to open new outbound connections to LiveKit Cloud during short bursts. The timing matches the SDK’s own timeouts exactly: the signal WebSocket connect hangs and hits the 5s timeout (“v0 path Timeout”), then the region fetch (an HTTPS GET to your project host) hangs and hits its 3s timeout (“region fetch timed out”), then a retry to the same host succeeds a few seconds later. Two separate connections to the same *.livekit.cloud host stalling back-to-back, in bursts, across 5 different pods.
That points to egress/connection-establishment contention on your side rather than Cloud or the carrier. It’s also rare, 13 slow joins out of ~8,000 calls over 3 hours, clustered in a couple of bursts.
Worth checking for the 15:07–15:16 window:
AWS NAT Gateway ErrorPortAllocation / ActiveConnectionCount (SNAT port exhaustion is my top suspect, lots of new sockets per call toward a few destinations)
node conntrack usage (nf_conntrack_count vs max)
CoreDNS latency
The CPU/autoscaler fix you’re already making should also help here, since a pegged pod can’t service the connect within the timeout. And since you’re on agents 1.4.6 / an older rtc core, an upgrade is worth testing, newer versions improve the region/connect path.