Master Production-Ready AI Agents: Evaluate & Ship With Confidence

​Shipping an AI agent is easy. Trusting it in production is where most teams struggle.

​Demos are deceptive. An agent that performs flawlessly in a controlled environment can quietly break the moment a tool errors, a prompt shifts, or a subagent delegates incorrectly. Most teams still ship on intuition — and pay the price.

​In this session, Kwasi Ankomah — Lead AI Architect at SambaNova Systems with 15 years of cross-industry experience — walks you through the end-to-end evaluation discipline that separates demos from production-ready AI agents, built live on a real system using traces and telemetry.

​What You'll Learn

• ​Why traditional software tests fail for multi-step, non-deterministic agents — and what to use instead

• ​The four evaluator types every production agent needs: rule-based, LLM-as-a-judge, trajectory, and recovery-from-failure

• ​How to build a scorecard and regression gate that runs automatically in CI on every prompt, model, tool, or architecture change

• ​LangSmith workflows: datasets, experiments, and trace-based evaluation that surfaces failure-and-retry sequences inside subagents

• ​LangFuse + OpenTelemetry as a full open-source observability alternative

• ​Online evaluation and pass^k reliability — how to score live traffic and build statistical confidence your agent performs correctly every time