Sharing: a New real-time 3DGS avatar plugin for LiveKit Agents by Spatius.ai (PR #5821)

Agents
1

Hey LiveKit community :waving_hand:

Sharing what we’ve been building — livekit-plugins-spatius, currently up at PR #5821.

Quick context: we’re Spatius, a real-time avatar layer for voice agents. Since opening early access a few weeks ago we’ve had 50+ developer signups, and about 70% of them are building on LiveKit Agents. That ratio is what pushed us to write the plugin — the easier we make this for LiveKit devs, the more it helps both sides.

What the plugin does

Lets a LiveKit Agent drive a photoreal, 3DGS-rendered avatar in a room. Audio in (the agent’s audio track) → real-time lip-sync + microexpressions, ~<300ms rendering latency, ~<1s end-to-end with low-latency TTS. Drops into an existing agent workflow without rewriting your pipeline.

Why we think it fits LiveKit’s bandwidth-first ethos

Cloud-rendered avatars are usually the cost bottleneck — running full render in the cloud and streaming 1–5 Mbps of pixels per session adds up fast once you scale past demos. We took a different cut: lightweight audio feature extraction in the cloud, native Gaussian Splatting rendering on the edge (Web / iOS / Android / embedded).

What that gets you:

  • ~10–20 KB/s wire bandwidth — friendly to flaky / mobile networks
  • 1080p 25fps on entry-level SoCs — we have had clients deploying our avatars on G88, RK3576, etc. with stable performance
  • ~$0.007 / min rendering cost (~$1 per 142 min) — economics that scale past a demo
  • Long-session visual stability — the avatar doesn’t drift across extended interactions, which matters for AI tutors, interviewers, customer service agents

Architecturally the plugin extends BaseAvatarSession and follows the standard architecture of adding an avatar to livekit agents — we don’t reinvent transport.

Demo

Live playground: Spatius — pick any character, start chatting. These calls route through LiveKit rooms!

PR Request

PR: feat: Spatius avatar plugin by 3DRX · Pull Request #5821 · livekit/agents · GitHub

About 70% of our early users are building on LiveKit Agents, so shipping this cleanly really matters to us. Happy to iterate on whatever feedback comes back :folded_hands:

— Jingyang @ Spatius

2

Hey Livekit team,

Developer working with spatius here. Any update on this?

3

No update. I will flag it to the agents team

4

We’ve been reviewing the issues flagged with our last PR submission and will share a response shortly.

5

To answer questions in our eariler submission #5014

Also, is there a reason why livekit_egress doesn’t accept a JWT token instead of LiveKit API secret and key? It would be good to scope credentials to each room that way. I also don’t think you need to set audio_output to False in the RoomOptions with that approach

The initial implementation of our LiveKit integration generates livekit token in Spatius’ service, which is not good for security. We’ve changed it since, now the token generation is in this livekit agents plugin.

I tested it out and I wasn’t able to see the avatar join the room at all, but audio was published. Could you pull from main and bump the versions to see if that helps?

We’ve rebased again, pull from main and bump the versions. For my testing it works. LiveKit Agents Integration - Spatius Is the documentation we write to help with the setup (a bit more complicated than video streaming avatar, since we do the rendering on client side).

6

Hi Livekit team, it’s been a while since we submitted the PR request in May. Just wondering if there is anything still blocking the review that we can further clarify, thanks.

7

Been running this in an agent and it drops onto BaseAvatarSession cleanly. The client-side Gaussian-splat rendering is a genuinely different take from the usual cloud-pixel-streaming avatars, and it is already installable as livekit-plugins-spatius on PyPI, so others can try it while #5821 is in review.

8

thanks man, I will try