Livekit failing to use an MCP server that's working on other clients

Hi all, I’m trying to use Livekit as a client to consume an MCP server. Here is the starter code;

from dotenv import load_dotenv

from livekit import agents, rtc

from livekit.agents import AgentServer, AgentSession, Agent, room_io, mcp

from livekit.plugins import (

    google,

    cartesia,

    groq,

    silero,

    noise_cancellation

)

from livekit.plugins.turn_detector.multilingual import MultilingualModel

from livekit.plugins.turn_detector.english import EnglishModel



load_dotenv()




class Assistant(Agent):

    def __init__(self) -> None:

        super().__init__(instructions="""

        You are a helpful assistant who is an expert working with Google Calendar.

        The email is '**********@gmail.com'

        

        """)




server = AgentServer()




@server.rtc_session(agent_name="test-mcp")

async def my_agent(ctx: agents.JobContext):




  session = AgentSession(

      stt = cartesia.STT(

        model="ink-whisper"

      ),

      llm=groq.LLM(

        model="llama-3.1-8b-instant"

      ),

      tts=cartesia.TTS(

        model="sonic-3",

        # voice="f786b574-daa5-4673-aa0c-cbe3e8534c02",

        voice="228fca29-3a0a-435c-8728-5cb483251068"

      ),

      turn_detection=EnglishModel(),

      vad=silero.VAD.load(),

      mcp_servers=[

        mcp.MCPServerHTTP(

            "https://03f0-102-209-109-234.ngrok-free.app/mcp"

        )

    ]

  )




  await session.start(

      room=ctx.room,

      agent=Assistant(),

      room_options=room_io.RoomOptions(

          audio_input=room_io.AudioInputOptions(

              noise_cancellation=lambda params: noise_cancellation.BVCTelephony() if params.participant.kind == rtc.ParticipantKind.PARTICIPANT_KIND_SIP else noise_cancellation.BVC(),

          ),

      ),

  )




  await session.generate_reply(

      instructions="Greet the user and offer your assistance. You should start by speaking in English."

  )

if __name__ == "__main__":

    agents.cli.run_app(server)

I get this error when the agent runs;

File "/usr/local/lib/python3.12/dist-packages/livekit/agents/utils/log.py",
line 17, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/lib/python3.12/dist-packages/livekit/agents/voice/agent_activity.py"
, line 534, in _list_mcp_tools_task
await mcp_server.initialize()
File "/usr/local/lib/python3.12/dist-packages/livekit/agents/llm/mcp.py", line
70, in initialize
await self._client.initialize() # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/mcp/client/session.py", line
171, in initialize
result = await self.send_request(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/mcp/shared/session.py", line
306, in send_request
raise McpError(response_or_error.error)
mcp.shared.exceptions.McpError: Session terminated

I can confirm that the MCP server is usable on other clients like PydanticAI.

Yesterday this MCP server worked fine with Livekit agent. But I don’t why this is happening today. Am I doing anything wrong?

Yesterday this MCP server worked fine with Livekit agent. But I don’t why this is happening today. Am I doing anything wrong?

I am unable to connect to your MCP server using MCP Inspector - Model Context Protocol, I get connection failed. Is the server definitely up?

@darryncampbell This server was running locally on my computer and I just exposed it through ngrok. After several tests, I see that the MCP server occasionally works with Livekit. Though its not consistent.

@darryncampbell As I develop this locally and inspect the MCP Server using MCP Inspector, what should I pay close attention to? When I connect using the MCP Inspector, I see all the tools, etc the server has. Is that what I need to check?

It’s the LLM that ends up calling your MCP server endpoint, so as long as your MCP endpoints are accessible externally, and callable by the LLM (which will use the same endpoints as exposed through MCP inspector) I would expect this to work. My point was I could not access your MCP server, so that is probably why it was failing at the time.