Whats the observed latency of Turn detection, i am observing latency greater than 600ms which is making my entire stack slower
Most of the 600ms you’re seeing for turn detection would be coming from the default minimum endpointing delay, which is 0.5s: Turn-taking tuning | LiveKit Documentation. You could try reducing this value in your turn detection settings, but it might lead to more aggressive agent interruption.
Based on your other recent question (here), I assume you are using Deepgram Flux, or at least testing with it, in which case be aware that this model has its own turn detection that can be used and configured: Deepgram STT | LiveKit Documentation