Observability and Evals Masterclass
Shipping an agent is just the start. The real work is building a system that gets better with every prompt change, model swap, tool update, and production failure.
That system is the eval loop.
In this workshop, you’ll build the complete loop in Mastra. You’ll use traces to understand what happened, turn real failures into datasets, compare changes with experiments and scorers, and automate regression evals so improvement becomes systematic instead of accidental.
No observability 101. No vibe-checking. Just the practical workflow for making your agent better release after release.
What you’ll learn:
Understand every run: Trace quality, latency, and cost down to the exact span, model call, or tool call responsible.
Build datasets from real failures: Capture production mistakes and edge cases as test cases, so every issue you fix improves your coverage.
Run the complete eval loop: Use datasets, experiments, and scorers to compare prompts, models, and tools with evidence you can trust.
Stop regressions before they ship: Add evals to CI/CD so new changes are tested against failures you have already fixed.
Build a self-improving agent: Give a coding agent CLI access to traces so it can investigate a failure, change the code, rerun the evals, and check whether its own fix worked.
Expect real code, a live end-to-end demo, and plenty of time for Q&A.
Hosted by
Alex Booker, Head of Observability and Evals, Mastra
Yujohn Nattrass, Engineering Lead, Observability and Evals, Mastra
Recording and code examples will be available to everyone who registers.