Hey everyone, I’m facing an issue in production with a SIP phone call.
Sometimes, when the user has speakerphone enabled, the agent’s audio gets picked up again through the user’s microphone. This causes the agent’s own speech to be detected as user speech and triggers an unwanted interruption.
For example, the actual conversation was:
Agent:
“Sure, what can I help you with exactly—scheduling an appointment, a…”
Human (detected):
“Sure. What can I help you with exactly? Scheduling an appointment?” (This is actually the agent’s audio being picked up through the user’s microphone.)
Human (actual user speech):
“Medical records.”
So the agent’s own audio is being transcribed as a Human turn, which causes the agent to interrupt itself.
Has anyone faced a similar issue with LiveKit SIP?
Is there any LiveKit-supported way to handle acoustic echo cancellation (AEC) or speakerphone feedback, or any SIP/room/audio configuration that we should enable to prevent the agent’s audio from being picked up by the user’s microphone?
Any suggestions or recommended approach would be appreciated. Thanks!
I do have some sort of solution that I applied, which is that I change the way of speaking by holding the button to speak, so once the “Human” finishes, I release the button to speak; only after that will it be sent to the agent to be picked up and translated(if needed). I also use an echo noise cancellation implementation, just like the Teams or Google Meet style concept.
@Pawel_Lach are you asking me? If you’re asking me, this is how I implement it:
How we do it — the simple version:
We don’t use any special library or AI tool for this. We use what’s already built into the phone and WebRTC (WebRTC is the technology that powers video/audio calls).
Think of it like this: your phone already knows how to filter out background noise — it does it every time you make a regular phone call. We just tell it “hey, turn that on.”
How it works in our code:
When the worker opens the pre-join screen (before entering the interview), we create a microphone track and tell it:
“Please remove echo” (echoCancellation: true) — this stops the speaker sound from feeding back into the mic
“Please adjust volume automatically” (autoGainControl: true) — so quiet voices get louder and loud voices don’t clip
The worker sees a toggle button (a chip) on the pre-join screen. They can tap it to turn noise suppression on or off. When they tap it, we destroy the old microphone track and create a new one with the updated setting. We have to recreate it because at this point the worker hasn’t joined the call yet, so we can’t just update the existing track.
There’s also a speaker test feature where the worker records their voice and plays it back to check how they sound. This recording doesn’t go through WebRTC, so we separately ask the phone’s own audio system to apply noise filtering:
On iPhone: Apple has a built-in audio engine called “Voice Processing I/O” that handles it
On Android: there’s a built-in effect called “NoiseSuppressor” that does the same job
In short: We’re not doing any fancy noise removal ourselves. We’re just flipping switches that tell the phone and WebRTC “please clean up the audio for us” — and they handle all the hard work.
Thanks @Hidayat_Noradzman_Hisham yes I was asking you I thought that maybe you have some model to cancel the echo, as the built-in solution doesn’t work me all the time.
@sahil.tulani what model are you using? Is it STS model by chance? Such as GPT Live? I have encountered this issue multiple times, and when I used cascaded systems it’s kind of easier to control it, when the model is duplex it’s often gets back to the microphone. You could technically try some voice isolation model - in my experience it helps sometimes.
We’re currently using ElevenLabs Scribe v2 for STT. Previously, we were using Deepgram, and I’ve seen this echo issue there as well. I’ve applied Krisp noise cancellation, but it hasn’t helped with the echo issue so far.
I thought that maybe you have some model to cancel the echo, as the built-in solution doesn’t work me all the time.