Hi team,
We run voice agents for K-12 oral assessments, so sessions contain student audio.
Docs say Build-plan projects are in the model improvement program, and that the program covers agent observability data only. We have Agent observability set to Disabled at the project level.
- With observability disabled, does any data from our project enter the model improvement program?
- If not, is any other session data (media server telemetry, turn detection, connection metadata) retained past 30 days or used for training on Build?
Hi, the way observability works is your data is captured in the agent, then uploaded to observability at the conclusion of the session.
If you disable observability, then data would not be captured and (even if it were), data would not be uploaded to our Observability service, so there is no technical way that your data could be used for model improvement.
Quite separately though, running a production solution that students depend on for their assessments using a free account is NOT recommended for a variety of reasons.
The assessments are not used towards grading students. They have the form of formative assessments, for in-class assessment purposes. Could you pls elaborate more on the risks of using a free account?
Sure, I’m not in sales, but if you are using a free account you risk:
- Running out of credits mid-assessment
- Depending on how many simultaneous users you have, you risk coming up against concurrency limits
- With only 1 deployable agent on LiveKit cloud you are not able to run sandboxed tests
- You are subject to agent cold starts.
It depends on your architecture, and those won’t apply if you are self-hosting your agents and going directly to model providers using plugins, but I’d still be wary since a failure during an in-class assessment would cause your users to lose confidence in your product. That’s just my 2c 
Appreciate the elaboration, Darryn. We are at the PMF stage and the plan is to upgrade plans once the solution is validated.