Drift between audio and video in the participant recording egress livekit cloud

Egress / Ingress
1

I am trying to record egresss for participant recordings. no matter what i try, there is always drift between audio and video on the livekit side for the participant recording.i am using the giiven instruction but there is always a drift. any fixes or suggestions? it is impacting our prod pdocasting capabilities

2

I’m not sure about your setup, but perhaps moving to track composite egress would help? Participant & TrackComposite egress | LiveKit Documentation

3

I’m using TrackComposite and having this same issue. Logs show drift. I’m pushing via whip ingress key and url and use egress TrackComposite to grab the audio and video tracks. But, I’m getting a lot of drift.

It was working great, then started getting it. All 3 are using the latest versions

4

Do you know which version it stopped working in?

5

@arslan.javaid @Donald_DOC_OConnor Just checking - do you use LK Cloud services or host egress yourself?

In order to be able to help we would need some info:

  • if you use LK cloud - egressIDs of affected recordings would be good to have so we can check, ideally a sample of recording exhibiting the problem if you are able to share it.
  • if you self host - debug service logs, service configuration (stripped of sensitive info) and exact version of the egress image used

Thanks,
Milos

6

While the team gathers the data they asked for, two characterization questions often narrow A/V drift quickly:

  • Is the drift a fixed offset (audio and video start out misaligned but stay that way), or progressive (the gap grows over the recording’s duration)? Fixed offset points at session-start timestamp handling; progressive points at sample-rate mismatch or accumulated jitter compensation. Worth measuring on one affected recording.
  • @Donald_DOC_OConnor: does the drift show up only on WHIP-ingested tracks, or also when the audio/video is published directly from a client SDK (browser/mobile)? If WHIP-only, the issue is scoped to the WHIP >> SFU >> egress timestamp path, which is much easier to triage.

Both answers are something you can produce without Cloud access, and they sharpen what the team is looking for.

7

The drift was in the hardware encoder clock of the LiveKit egress service, not specific to WHIP. Any track source (WHIP, RTMP, browser SDK) going through egress would exhibit the same drift. I fixed it by enabling audio_tempo_controller in my egress.yaml, with: adjustment_rate: 0.08, which imperceptibly adjusts the audio playback speed to keep it locked to the video timestamp. I started with: 0.02, and adjusted. So, this is something others may need to play with, to get theirs corrected.

8

@DoC_DiD_iT, Nice, that lines up with the progressive-drift branch from above: an encoder-clock offset that grows is the sample-rate/clock case, and audio_tempo_controller is the intended fix for it. It’s the documented “audio tempo adjustments for compensating PTS drift” knob, with adjustment_rate as the dial you tuned [ livekit/egress pkg/config/base.go ]. Same block has a sibling, enable_sync_engine (“Chrome-inspired sync engine for improved cross-participant alignment and A/V sync”), worth a try if you’d rather use the newer sync path than tempo nudging.

One caveat for anyone landing here from the title: both are egress.yaml settings, so they only apply if you self-host egress. On LiveKit Cloud you can’t set them, so the original Cloud report still needs the egressIDs the team asked for, not this knob.

9

thank you. I checked 3 or 4 times, and didn’t see a reply. And, I thought I checked regularly. My last comment was what fixed it for me, and I’m going to go back into my code and check what you mentioned at the end. I am sure you correct, as my guest was most likely on an iPhone, causing issues with Turn and Ice Connections. Though, after my fix, it work great.

It ran smoothly ever since. I’ve successfully ran 5 interviews, and going to push it to the max, when I multicast my next interviews to my social platforms, in a couple days. I’m waiting on my Fiber to be connected. A must for any media. And, what I use fiber in my main office. Though, my home office is where I work from mostly. Which, was att air, and running OBS studio, on a Laptop. :slight_smile: Amazingly, all works well on it, but I keep it small.

Now, it is self hosted on my Server in a datacenter. All, but OBS Studio. Maybe too much info for prying eyes, but we all know the feeling of having everything working correctly.

I will add: Not sponsored by, but Powered By LiveKit & DOC’s media. :wink: as all frontend, and backend code is mine. Though, the engine sitting in that shinny car, is LiveKit. Thank you guys for providing this.

This is closed, and I highly recommend LiveKit. Let them run this in their Cloud for you, as they know the best solutions for their users. Self hosting takes a lot of time, and they will cut that out for you. :wink: