Full-stack LiveKit voice agent starter: worker + token server + React, web and SIP on one worker [open source]

Agents
, ,
1

I kept rebuilding the same scaffolding around LiveKit Agents for client projects, so I cleaned it up and open-sourced it (MIT). Sharing here because most of it maps directly to questions that come up in this category.

livekit-starter
livekit-starter1774×887 230 KB

It’s three pieces, each runnable and deployable on its own:

  • agent: a LiveKit voice worker (Deepgram STT, an OpenAI model, Cartesia TTS, Silero VAD, multilingual turn detector)
  • backend: a FastAPI service that mints room tokens
  • frontend: React + Vite + Tailwind on the Agents UI components (audio visualizer, live transcript, text chat)

The LiveKit-specific decisions, since those are the interesting part here:

  • Explicit dispatch via agent_name. The worker registers under a name; the web client requests it through the token’s room_config (the backend passes room_config straight through), and SIP comes in through a dispatch rule. One worker, one dispatch path you can reason about.

  • Web and SIP on the same worker. A single identify() step classifies the participant (SIP participant kind, or a sip.phoneNumber attribute) and the conversation after that is identical. No second code path for “browser” vs “phone number.”

  • Standard token endpoint. POST /token follows LiveKit’s documented token-endpoint schema, so the client SDKs connect with TokenSource.endpoint(…) and no glue. Backend and agent share the same API key/secret, so a backend-minted token is valid for the room the agent joins.

  • Prewarm loads Silero VAD once per process and shares it across sessions, and the server uses a drain timeout so deploys don’t cut active calls.

  • Turn detection uses the multilingual model with VAD-based interruption, set through TurnHandlingOptions.

Providers are single-line swaps, and it points at self-hosted LiveKit or Cloud by changing the URL. Docker Compose brings the whole stack up with one command, or you can run each service in its own terminal. Typed, tested, CI across all three.

Repo: GitHub - mahimairaja/livekit-starter: Full-stack LiveKit voice agent starter: a Python voice worker, FastAPI token server, and React frontend. Run, deploy, and swap each piece on its own. · GitHub

Built on livekit-agents 1.5.x. I’d value a second opinion on two things from people who’ve shipped more of these than I have: the dispatch setup (is room_config passthrough the cleanest way to trigger a named agent from the web client?), and the web-vs-SIP identification (anything I’m missing for edge cases like transferred calls?).

Happy to answer anything about the structure.

2

Thanks for sharing :lk-party: , all of the LiveKit-specific decisions align with the official recommendations. The one caveat being interruption - you have set it to VAD which is appropriate for self-hosted agents, but agents hosted in LiveKit cloud would be better to use adaptive.

There is something to be said to having this all in the same project for new users, it is arguably easier to understand than the official docs, which advise piecing the solution together based on:

  • Agent: Use one of our two agent starters (Python / Node)
  • Backend: Use the Token Server you get as part of your cloud project
  • Frontend: Use our React starter with agents UI

There’s no right or wrong way however: by presenting each of the pieces separately we make it clearer that you can swap bits out (exchange the React starter for an Android starter for example)

3

Thanks for the comment @darryncampbell

4

@Mahimai_Raja, Nice, clean separation. On your two:

Dispatch: yes, room_config in the token is the canonical client trigger, nothing cleaner [ docs.livekit.io/agents/worker/agent-dispatch ]. One constraint: token dispatch only fires when the room is first created; if the room already exists it’s silently ignored, so a unique room name per session has to hold, or only the first client triggers the agent.

Identify and transfers: branching on participant kind (SIP vs not) is the solid part. The edge case is the sip.* attributes, not the kind. Before server 1.9.1, sip.phoneNumber could land in a follow-up update rather than at join (tightened in 1.9.1, #3693), so an identify() that reads it synchronously on the join event can race on older builds. Gate on the attribute being present. And on a transfer, kind stays SIP but sip.phoneNumber is the current leg (the transfer target), so treat it as informational, not identity.