Built-in laptop mic causing incorrect Whisper transcriptions

Agents
,
21

Hi @Pawel_Lach

We have just released Quail Voice Focus 2.2 that also works in the far-field mode, so it should work now correctly also in cases where the foreground speaker is a bit further away from the microphone. However, it’s not available on LiveKit yet, it should be there hopefully in a matter of days.

Any idea by when this model will be live as we need to test it?

22

It’s not yet officially integrated in the LiveKit ai-coustics plugin, but we are happy to help you with a custom patch that should make the model work with LiveKit. Feel free to send me a DM, happy to chat, we can also arrange some call and have a look into it together.

23

That explanation from @Pawel_Lach hits the exact root cause!

Because QUAIL_VF_S is tuned as a near-field model (expecting a headset mic 1–2 inches from the mouth), placing a user 20–30 inches away on a laptop mic makes the model treat the reverberant, lower-amplitude voice as background noise. It aggressively suppresses the speech formants, leaving behind phase artifacts.

When Whisper receives those low-SNR phase artifacts instead of clear audio, its auto-detect panics and hallucinates random Chinese/foreign characters to fill the gap.

While waiting for Quail Voice Focus 2.2 (or the custom LiveKit plugin patch Pawel offered above), here are 3 things you can do right now in your code to fix the laptop mic issue:

  1. Force English on Whisper STT: If your users speak English, explicitly set language="en" in your Whisper STT config instead of using auto-detect. This stops Whisper from outputting Chinese characters when it encounters phase-cancelled or quiet audio.

  2. Ensure WebRTC Browser Echo Cancellation (AEC) is Active: Laptop speakers bleed directly into laptop mics. Make sure your frontend requests WebRTC constraints with AEC enabled:

    //JavaScript //
    navigator.mediaDevices.getUserMedia({
    audio: { echoCancellation: true, autoGainControl: true }
    });

  3. Lower enhancement_level or Switch to Quail L: Drop enhancement_level from 0.8 down to 0.3 – 0.4 for laptop users, or test standard QUAIL_L as Pawel suggested so the model doesn’t over-suppress far-field speech.

Having filmed live broadcasts and managed remote multi-guest streams for years, the single hardest variable to control is guest user behavior. Non-technical users genuinely don’t realize that their laptop speakers bleed directly back into their built-in microphone, creating an invisible acoustic feedback loop that ruins STT and audio processing downstream.

How we handle this in production: We built a mandatory “Greenroom” check in our Next.js frontend before guests can join a broadcast session.

  1. It forces a media device check (verifying input/output selection).

  2. It requires them to check a toggle confirming “Headphones / Earbuds in Use” (with a clear visual warning explaining why built-in laptop speakers cause echo and audio corruption).

  3. The “Enter Studio” button stays disabled until those checks pass.

Offloading the problem to a pre-flight user check prevents 90% of far-field audio, acoustic feedback, and Whisper STT hallucination issues before a single packet ever hits your processing pipeline or LiveKit room!

24

Thanks for mentioning me @DoC_DiD_iT and good news is that we have published our custom plugin just Today! :slight_smile:

More info here: Quail Voice Focus 2.2 L - #8 by Pawel_Lach

25

Most welcome, and I gotta check it out