Real room/SIP webhooks never delivered, but dashboard "test event" reaches the same URL fine

Cloud Dashboard
1

Summary

On my LiveKit Cloud project, real project webhooks are never delivered to my configured webhook URL. This is not limited to room lifecycle events, it appears that all real project events are affected. For example, room_started, participant_joined, and room_finished are never received.

However, the dashboard Send a test event button successfully delivers to the exact same webhook URL.

This indicates that the webhook endpoint is correctly configured and publicly reachable. The issue is that webhook deliveries for actual project activity are not reaching the endpoint.

Environment

  • LiveKit Cloud project (single webhook URL configured)
  • Inbound SIP calls via a SIP trunk + dispatch rule (individual-per-caller, room prefix)
  • Webhook receiver: Python (livekit.api.WebhookReceiver) running on localhost:8090
  • Receiver exposed publicly via Cloudflare Tunnel:
cloudflared tunnel --url http://localhost:8090

which exposes:

https://<subdomain>.trycloudflare.com/webhook

What works

  • SIP → trunk → dispatch rule → agent job dispatch works correctly.
  • The agent is dispatched and joins the room.
  • lk room list and lk room participants list show the room live with both the SIP caller and the agent as ACTIVE participants.
  • The dashboard Send a test event button successfully delivers to the configured webhook URL.
  • The webhook is visible in my server logs.

What fails

For real rooms and SIP calls, my webhook server receives nothing at all.

This is not limited to specific webhook types. For example, I never receive room_started, participant_joined, or room_finished, nor any other real project webhook events.

There are no signature-verification failures, no filtered events, and no rejected requests because the server never receives an HTTP request in the first place.

The room demonstrably exists (verified with the LiveKit CLI), both participants are active, the room starts and finishes normally, yet no webhook deliveries are made to the configured endpoint.

What I’ve already ruled out

Endpoint reachability

The endpoint is reachable. The dashboard Send a test event successfully reaches the webhook receiver.

Signature / API key mismatch

A signature mismatch would still result in the webhook request reaching my server, where it would fail verification (for example with HTTP 401). Instead, no HTTP request arrives at all.

Multiple webhook URLs

I reduced the configuration to a single webhook URL, and the behavior remains unchanged.

Regression

This setup previously worked correctly. No changes were made to the webhook receiver or SIP flow before this issue appeared. It suddenly stopped delivering real project webhooks while dashboard test events continued to work.

Project details

  • Project ID: p_43z8x0uyw44
  • Project subdomain: dev1-app-489epzkw
  • LiveKit URL: wss://dev1-app-489epzkw.livekit.cloud
  • Configured webhook URL: https://<subdomain>.trycloudflare.com/webhook
2

Hi, you didn’t specify a session so I just chose the most recent session under that project, RM_pvbvQ2JnDguv. I see server logs that we attempted to send the webhook, but the attempt failed with a no such host error. I’ll post the redacted error below:

POST https://<subdomain>.trycloudflare.com/webhook giving up after 3 attempt(s): Post "https://<subdomain>.trycloudflare.com/webhook": dial tcp4: lookup <subdomain>.trycloudflare.com on 10.96.5.5:53: no such host

This is confusing, since as you say the ‘send test event works’

3

I could see my webhook server is receiving event logs again now. I haven’t changed anything on my side, so it seems someone on your side may have fixed the issue.

4

I am not going to argue if it is now working for you, but I am not aware that there have been any changes on our side in the past hour or so :slight_smile:

5

We’re hitting this exact issue as well.

Real room events (room_started, participant_joined, track_published, egress_started/egress_ended, room_finished) are never delivered, but “Send a test event” from the dashboard reaches the same webhook URL within seconds and passes signature verification fine.

To rule out our side we tested two independent receivers: our own endpoint behind a Cloudflare tunnel (confirmed reachable — test events and a self-signed synthetic webhook both land with 200s, and the tunnel’s request counter shows zero other inbound attempts), and a plain webhook.site URL. During multiple full room lifecycles (room created, participants joined, tracks published, egresses started and stopped), neither receiver got a single event — the only deliveries ever received are the manual test events from the web UI.

We also created a brand-new project to rule out per-project state, and it reproduces there too. That project ID is p_4w4zeza05bo if the team wants logs to look through — room activity from today (2026-07-23, ~13:30–17:00 UTC) should show egresses running with no webhook deliveries going out.

EDIT: After a couple hours of it not working at all, it has now randomly started working (no changes, exact same webhook.site URL). It would be nice if this were a little more reliable.

6

@Edward_Partenie_D_Albon I see server logs indicating that the webhook WAS successfully sent, contrary to the report earlier in this thread where there was an error during send.

Typically this is because some logic is happening in the webhook that prevents reception:

Does that apply to you? It looks like you’re using a test site, so maybe that’s not the case.

If you give me a specific session, I can dig into when the webhooks were sent.

7

I have seen this as well over the last few days. The “test webhook” seems to fire and I can see it clearly on my server logs, but getting them from production usage seems to be inconsistent or absent. Some kind of webhook history that shows the hook was sent and it’s response code from my system would be a huge help, that way if it IS my code I can tell why… but I’m not seeing it on my internal logs.

I seem to be seeing this mainly when the call ends and everyone leaves, I expect a participant left or room finished that never arrives.

Sessions seeing this: RM_yi5QFqm7Dy55, RM_3zPXHqWaEmME

8

I am experienceing a simlar issue.

In my case as far as I know I was only receiving room_finished webhook event for real events this morning and the test events were being received as expected.

Now I am getting real events normally.

9

@Edward_Partenie_D_Albon @David_Hanson @dansoonie

It is very strange to get so many reports of failing webhooks in the past 24 hours (I also see another one raised in the Slack community)

I have looked at the sessions I have IDs for, and our logs show that webhooks were successfully sent.

I have also looked at our overall trends for the past 48 hours, and I don’t see an elevated rate of failed webhooks

There are no ongoing incidents or internal changes that could have caused this.

I don’t see an elevated number of similar reports on our paid support channels.

Let me keep investigating this, and please let me know if you see this moving forward.

10

for me I’m facing the same issue , from the cloud test I’m receiving the events , but on real room events it’s not coming to my end

11

@adham.ismail can you share a session ID where you saw this this happen please? I.e. a session which did not send you events as you expected?

12

session id: RM_vKE2BLEew4n6
project ID : p_3redsn0jkck

13

i just tested again , it start working, i received the event at my end, is that a confirmation that the is issue s resolved ??

14

Thanks @adham.ismail , still investigating. For RM_vKE2BLEew4n6, was it all webhooks, that did not fire, or just some of them?

15

i started a new one and it works but for the RM_vKE2BLEew4n6 to be honest i didn’t try with it again

16

Thank you, I have escalated this internally

17

@darryncampbell — noticed a similar-shaped issue in the webhook thread (signaling/
connectivity succeeds, but real live-room data doesn’t arrive, while test/simpler
paths work fine). Not sure if related, but flagging in case it’s useful context for
triaging. Happy to share full session data if it helps.

18

@adham.ismail @Edward_Partenie_D_Albon @David_Hanson @dansoonie

Was this affecting webhooks that already existed? Or were these new projects set up with new webhooks?

19

This seems to happen quite often with newly created webhooks. The pattern I’ve observed is:

  1. You create the webhook.
  2. You click “Send a test event”, and your server successfully receives the event.
  3. You make a real call, but no webhook events are delivered.
  4. If you leave it for a while and then make another real call, your server may start receiving the events correctly.

I’ve repeated this test several times, and the behavior has been consistent every time. I hope this observation helps you diagnose the issue more effectively.

20

@darryncampbell Thank you for your assitance. Here is your answer to your following question.

Was this affecting webhooks that already existed? Or were these new projects set up with new webhooks?

Existing projects with existing webhook settings were affected. I even tried removing the existing webhook and creating a new webhook configuration(with the same endpoint and API key) and the problem was still not solved. After about an hour it started to work noramlly again.