Using the MCP Gateway Pattern to support many MCP servers

Agents
1

Does anyone have experience using the MCP Gateway pattern with LiveKit agents to support dynamic injection of skills as a way to provide access to a large set of MCP servers without burning through context? Any tips for doing that before I dive in? So far what I’ve got is something like this:

  1. At agent start time, inject static instructions in how to use a single gateway MCP server.
  2. Also query for a concise list of descriptions of available MCP servers so that agent knows what is available should it need it later
  3. When the agent wants to use an MCP server, it goes through the Gateway MCP to get the skill text to inject into the context just in time.
  4. Agent can now go through Gateway MCP to a dynamically added MCP
2

LiveKit agents have native MCP (Model Context Protocol) support. You can configure MCP servers at both the Agent and AgentSession levels: agent.py:47 agent_session.py:146

How MCP Tools Are Currently Loaded

When an agent starts, all MCP tools are loaded upfront: agent_activity.py:526-552

The tools from all configured MCP servers are fetched during initialization and added to the agent’s tool list: agent_activity.py:293-296

Your Gateway Pattern Approach

Your proposed gateway pattern isn’t directly supported by the current architecture. The framework loads all MCP tools at agent startup and includes them in every LLM context. There’s no built-in mechanism for:

  1. Lazy/dynamic tool loading - Tools are fetched once at initialization

  2. Just-in-time tool injection - The tool list is static after _start_session() completes

  3. Tool filtering based on context - All tools from all servers are always available

Implementation Suggestions

To implement your gateway pattern, you’d need to:

  1. Create a custom gateway MCP server that exposes meta-tools like list_available_servers and load_server_tools

  2. Configure only the gateway server in your agent: agent.py:47

  3. Implement gateway tools that dynamically call other MCP servers and return their results, rather than trying to inject tools into the context mid-conversation

  4. Use the allowed_tools parameter on MCPServerHTTP to filter which tools are exposed: mcp.py:188-189

Notes

The current architecture assumes a relatively static set of tools. Dynamic tool injection would require modifications to how AgentActivity manages its tool list and how it syncs with realtime LLM sessions. The _mcp_tools list is populated once and not updated during the agent’s lifetime.

Your gateway approach is creative, but you’ll likely need to implement it as a proxy MCP server that handles routing rather than trying to dynamically modify the agent’s tool context at runtime.

3

Thanks for that detailed and quick response. I like your idea of having the MCP server handle routing. Looking forward to testing this out.

4

Interesting thread. One pattern I’ve been seeing emerge in multi-agent systems is what some call “Skills”, where instead of loading all tools upfront, the agent gets a lightweight catalog of available capabilities, and only pulls in the full tool definitions and context just-in-time when the conversation actually needs them

It’s conceptually similar to your step 3 (get skill text to inject into context), and it plays well with CWilson’s suggestion of a proxy MCP server. The proxy exposes a small set of meta-tools (list_skills, load_skill), and each “skill” bundles the instructions + tool definitions for a specific domain. This keeps the per-turn tool count low while still giving the agent access to a wide surface area

Curious how your testing goes with the routing approach