What is the reason for not allowing plugins to be imported in worker threads?

Client SDKs
,
1

TLDR;

Since we’re importing all submodules and dependencies of the plugin by running it in the worker process anyway, why not also the top-level name. Why is the threading check important?

(edit → moved TLDR to the top)

In the plugins.py code, there is a classmethod which restricts the imports of all plugins to the main thread and if not then raises a runtime error:

@classmethod
def register_plugin(cls, plugin: Plugin) → None:
if threading.current_thread() != threading.main_thread():
raise RuntimeError(“Plugins must be registered on the main thread”)

    cls.registered_plugins.append(plugin)
    cls.emitter.emit("plugin_registered", plugin)

I want to know whether this is a necessary check, and whether a RuntimeError is the right response to it. Because in an alternate world where this didn’t exist, I could do something like this for telephony inbounds for example:

  1. Run all livekit heavy imports in a worker thread as async
  2. Python will populate sys.modules with this top level name (say livekit.plugins.inworld) and start reading the __init__.py file
  3. Every submodule/dependencies will get imported into sys.modules and this happens async in a single thread which is not main.

Meanwhile, the worker would be dispatched and the call would be picked up. So, the goal of this excerise is to reduce the number of rings that the user experiences. If we move heavy imports to a worker thread and we ALWAYS expect to play a greeting at call pickup, then this is a win win, cause there is enough time for sys.modules to be populated.

And once sys.modules is populated, the main thread can just use that.

Today, doing this results in a race condition because the RuntimeError will cause only the top level import to be removed (that’s what python eagerly adds, so once there is exception, python will remove it), BUT the dependencies and submodules will populate the sys.modules anyway.

For example, this is the inworld plugin imports which WILL still be there in sys.modules:

from .stt import STT, SpeechStream
from .tts import TTS, ChunkedStream, Encoding, SynthesizeStream, TextNormalization, TimestampType
from .version import __version__

This means I anyways have all the core of the module, only the top level name is missing. So, now if we are catching the RuntimeError and the call is picked up, we’ll be loading these again and from the code’s perspective inworld is not present because the top level name is not there.
Now the problem is, if a worker thread again tries to import it, the calls will fail. If the main thread imports, calls will pass.

I hope the question makes!

2

I am not asking if we should remove it but, it would be cool if we could have an env var, like an additional check for the if condition which would change the class to:

@classmethod
def register_plugin(cls, plugin: Plugin) → None:
    if threading.current_thread() != threading.main_thread() and not 
os.environ.get("LIVEKIT_ALLOW_THREADED_PLUGIN_REGISTRATION"):
    raise RuntimeError(“Plugins must be registered on the main thread”)
cls.registered_plugins.append(plugin)
cls.emitter.emit("plugin_registered", plugin)
3

Did we make any progress on this?

4

I’ve passed it on to the agents architecture team to take a look.