How to configure serviceTier for AWS Bedrock voice agents

Agents
, , ,
1

I’m currently using AWS Bedrock LLMs for my voice agent, and I’ve been facing latency issues during peak hours in certain regions. In some cases, even fallback LLMs from nearby regions also experience slower response times.

To address this, I’m exploring the use of the priority service tier to ensure faster and more consistent responses for real-time voice interactions.

However, I’ve run into an issue: while initializing the agent, I don’t see any option to configure a serviceTier. From the AWS documentation, it appears that this option is available in the converse_stream API:
https://docs.aws.amazon.com/boto3/latest/reference/services/bedrock-runtime/client/converse_stream.html

Has anyone here successfully configured or used serviceTier with Bedrock in a real-time voice setup (especially with LiveKit)? Am I missing something in the agent initialization, or is this not yet supported in the current integration?

2

I don’t have any experience with service tier, and interested if others feel this is the correct approach.

If you did go this route, serviceTier is not exposed publicly. I think your best bet would be to add it yourself (looks like the source is here: agents/livekit-plugins/livekit-plugins-aws/livekit/plugins/aws/llm.py at main · livekit/agents · GitHub) and submit back as a pull request.

3

Okay will do, Thanks! Since this is affecting our agent latency, I tried having Fallback LLM to other region in case there’s high traffic, still it’s not as effective as switching to priority tier.