# Per-session CPU burst from a plugin's native model being built lazily on the first audio frame (AI\_Cousitcs) — any pattern for prewarmable per-session init?

**URL:** <https://community.livekit.io/t/per-session-cpu-burst-from-a-plugins-native-model-being-built-lazily-on-the-first-audio-frame-ai-cousitcs-any-pattern-for-prewarmable-per-session-init/2180>\
**Category:** Agents\
**Tags:** realtime, noise-cancellation, livekit-cloud\
**Created:** [October 2, 2026, 8:08pm UTC](https://community.livekit.io/t/per-session-cpu-burst-from-a-plugins-native-model-being-built-lazily-on-the-first-audio-frame-ai-cousitcs-any-pattern-for-prewarmable-per-session-init/2180 "2026-10-02T20:08:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Paras\_Agarwal](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/paras_agarwal/32/2691_2.png) [@Paras\_Agarwal](https://community.livekit.io/u/Paras_Agarwal)\
**Post date:** [October 2, 2026, 8:08pm UTC](https://community.livekit.io/t/per-session-cpu-burst-from-a-plugins-native-model-being-built-lazily-on-the-first-audio-frame-ai-cousitcs-any-pattern-for-prewarmable-per-session-init/2180/1 "2026-10-02T20:08:53Z")

</div>

**Description:**  
A noise-cancellation plugin we use builds its native model lazily on each session’s first audio frame rather than ahead of time, causing a short but real CPU burst at the start of every session. Looking for whether there’s an existing pattern/hook in the agents framework (or plugin ecosystem generally) for doing this kind of per-session-but-heavy initialization earlier — ideally during prewarm — instead of on the first real frame.

**Problem text:**

We run `livekit-agents` (Python) as a voice agent using `ai-coustics` for noise cancellation (`ai_coustics.audio_enhancement(model=EnhancerModel.QUAIL_VF_S)`). We noticed a consistent, short CPU spike at the start of every new session and traced it to the plugin’s lazy initialization.

Looking at `AICousticsAudioEnhancer._process()`:

```python
if self._enhancer is None or (
    # implicitly recreate on sample rate/channel change
    ...
):
    self._settings = EnhancerSettings(
        sample_rate=frame.sample_rate,
        num_channels=frame.num_channels,
        samples_per_channel=frame.samples_per_channel,
        model=self._model,
        ...
    )
    self._enhancer = Enhancer(auth_mode, self._settings) # native model construction happens here

```

The actual native model (a compiled Rust core, constructed via FFI) is only built on the **first real audio frame** of a session — not when the `AICousticsAudioEnhancer`/`AudioInputOptions` object is constructed, and not at room-join time. In our setup this lines up exactly with a CPU spike each time a new session starts.

We prewarm other per-process resources ahead of time via `WorkerOptions.prewarm_fnc` (e.g. warming our own service clients), but we can’t do the same for this model, because its settings (`sample_rate`, `num_channels`, `samples_per_channel`) are derived from the actual incoming audio frame — they aren’t known until a real session’s real track is live. So today the only two options we can see are: (a) eat the cost on the first frame, as it does now, or (b) build some kind of throwaway “warm-up” instance during prewarm, which only helps if there’s shared/cacheable state at the process level (model weights loaded into page cache, a shared native runtime, etc.) rather than purely per-instance cost — something we have no visibility into since the plugin is a closed-source compiled component.

We did notice the plugin exposes `update_stream_info()` and `update_credentials()` methods for re-pointing an already-constructed `Enhancer` at a different stream/credentials, which suggests the native core might be _designed_ to support reuse across streams rather than one-construction-per-session — but we haven’t found guidance on whether that’s an intended/supported usage pattern, or whether it’s safe to do from application code (thread-safety, state leakage between sessions, etc.).

**Questions for the community:**

- Is there an existing pattern or hook — in `livekit-agents` itself, or in how plugin authors structure things — for per-session initialization that still needs to run ahead of time (e.g., during prewarm) rather than lazily on the first frame?
- Has anyone tried pooling/reusing a plugin’s native processing instance across sessions via re-pointing APIs like `update_stream_info()`/`update_credentials()`, instead of constructing fresh per session? Any gotchas?
- More generally for plugin authors doing native/FFI-backed per-session setup: is “lazy construction on first frame” the recommended approach, or is this specific to how this plugin happens to be built?

Environment: `livekit-agents` (Python), one `ai-coustics` enhancer constructed per session via `AudioInputOptions(noise_cancellation=...)`.

---

<div class="post-metadata">

**Author:** ![Rehan\_Sanjay\_Venkatesan](https://sea1.discourse-cdn.com/flex001/user_avatar/community.livekit.io/rehan_sanjay_venkatesan/32/2739_2.png) [@Rehan\_Sanjay\_Venkatesan](https://community.livekit.io/u/Rehan_Sanjay_Venkatesan)\
**Post date:** [October 9, 2026, 8:00am UTC](https://community.livekit.io/t/per-session-cpu-burst-from-a-plugins-native-model-being-built-lazily-on-the-first-audio-frame-ai-cousitcs-any-pattern-for-prewarmable-per-session-init/2180/2 "2026-10-09T08:00:45Z")

</div>

The part you can’t see into the plugin for, you can settle from the framework side — process state is shared across sessions, so a throwaway warm-up is worth testing.

prewarm\_fnc runs inside \_JobProc.initialize(), which creates the JobProcess once and then hands control to entrypoint(). That entrypoint loops on StartJobRequest messages, so one process serves many jobs sequentially and the same JobProcess — and its userdata — persists across all of them. Prewarm is per-process, not per-session.

So the question isn’t whether warming can be shared. It’s whether the cost you’re paying is process-level (loading the native library and model weights into the process) or per-instance (allocating a new enhancer around already-loaded weights). You can find that out without any visibility into the plugin:

Build two enhancers back to back inside prewarm\_fnc and time both. If the second is much faster than the first, the expensive part is process-level and a throwaway warm-up during prewarm will kill your per-session spike. If both cost the same, it’s per-instance and warming buys you nothing — at which point pooling via update\_stream\_info() is the only route left.

Five minutes, and it tells you which of your two options is actually available.

On whether lazy-on-first-frame is the recommended FFI pattern — I’d say the framework’s shape argues against it. prewarm\_fnc plus JobProcess.userdata exists precisely so heavy init lands at process start rather than in the audio path, and initialize() even imports a \_preload module described in the source as the framework’s own warm-up. Deriving settings from the first frame is a real constraint, but it’s a plugin design choice rather than a platform one.