Is it possible to use Livekit Inference models on a self hosted deployment? (with a Livekit cloud Api Key and a paid plan)
I’m developing a self hosted application and I noticed that TTFT are lower when using models through Livekit Cloud Inference vs using the providers directly, so I’d like to know if is it possible to use this API in an agent that is not running on your cloud
Just to clarify, I’m interpreting your question as self-hosted LiveKit server. You can call Inference models through self-hosted Agents using LiveKit cloud as your transport without issue.
Technically, yes, it does work, but it’s not officially recommended or ‘supported’. There was an interesting thread recently with a user who had done just this, but they were seeing lower TTFT and missed caching compared with agents running on cloud, so you may find you wouldn’t get the benefits you’re looking for anyway: