OpenAI recently released a websocket API (For their regular models not only realtime)
This should reduce latency since we don’t have to establish a new connection per generation.
Are there plans to adapt this? I imagine is going to require significant redesign of some core sections of the livekit openai plugin.
Is this a good first PR to work on?
But I believe the team already has one in flight.