The state of harness engineering
Models are getting better fast. But when an agent fails in production, the model often isn’t the problem these days. Instead, problems increasingly start and stop with the agent harness.
The failure might be in the context it received, the tool it called, the path it took through a workflow, the state it carried forward, or the recovery logic that kicked in when something went wrong. Together, those systems make up the agent harness, and they increasingly determine whether an agent actually works.
Join Arize and Google DeepMind in London for a practical evening on how to find and fix failures across the systems surrounding the model.
We’ll look at where agent harnesses break in production, how teams use traces and evals to understand what went wrong, and the patterns developers are using to make agents more reliable as models become more capable.
You’ll hear perspectives from Arize, Google DeepMind, and another team building agents in production, followed by drinks, food, and time to compare notes with fellow AI builders.
Who this is for
AI engineers, agent builders, technical leads, and PMs building LLM-based products, especially anyone working on tools, context, evals, tracing, orchestration, or agent reliability.
What you'll leave with
A practical model for thinking about the systems that make up an agent harness
A clearer picture of where agents fail beyond the model itself
Techniques for using traces and evals to find the source of failures
Patterns for debugging and improving tools, context, workflows, and recovery logic
Perspectives from Google DeepMind and teams shipping agents in production
New connections with fellow AI builders in London
Format
Three talks plus networking, approximately 2.5 hours.
Level
Beginner to advanced. No specific platform experience required.
Agenda (TBC)
6:00 - 6:45 PM | Opening: Check-in, welcome drinks & snacks, opening remarks
6:45 - 7:15 PM | Session 1: Building AI at scale: Typeform's Approach to Evaluation and Agent Quality - Lora Kiosseva (AI Eng) and Mohammad Afsharmoqaddam (Senior ML Eng), Typeform
7:15 - 7:45 PM | Session 2: Everything that breaks around the model: How to diagnose and improve the agent harness - Dat Ngo (VP of Strategy), Arize AI
7:45 - 8:00 PM | Break
8:00 - 8:30 PM | Session 3: The benchmark is the bottleneck, Ivan Leo (Developer Experience), Google DeepMind
8:30 - 9:00 PM | Networking & catering