Online workshop: Reduce LLM costs without sacrificing agent quality

​Token cost is reaching unsustainable levels, and it’s difficult to know if a cheaper model is actually viable for your agent. After the switch, token costs may drop, but failures increase, retries pile up, and human cleanup can erase the savings. In production, cost depends on the system around the model as much as the model itself.

​Evals can help solve this problem, giving teams the ability to test models with real production data to monitor cost and performance side-by-side. One Braintrust customer recently saved 80% on LLM costs for a production pipeline without hindering agent quality.

​In this hands-on workshop, you'll learn how to:

• ​Run eval experiments to do objective, comparative analysis between two models

• ​Monitor and estimate spend changes by model and prompt

• ​Reliably switch to a cheaper model without sacrificing agent quality