Production AI Needs Adults in the Room: The Evals Playbook (w/ Austen Allred CEO @ Gauntlet AI)

​CEO Austen Allred and CTO Ash Tilawat from Gauntlet will share how AI systems move from interesting demos to dependable production systems. Austie will cover how AI-first teams think about reliability, measurement, and human review when the stakes are high.

​Then Ash will live-build a real workflow, showing how evals, scoring, and subject matter review turn AI from a one-off demo into something that can be trusted over time.

​This session is built for people who want to understand what it actually takes to run AI in production at scale, without removing humans from the loop.

​It's going to be TV entertainment mixed with overwhelming value.

​More on Gauntlet:

​Gauntlet runs a 10-week, in-person program in Austin, Texas for experienced software engineers who want to build real AI systems. Hiring partners pay for the program, not the engineers. Travel, food, laundry, and compute are covered, so participants can go all in on AI (literally for 1,000 hours). When engineers graduate, they are placed into roles at partner companies. Salaries start at $200k, with some engineers earning closer to seven figures. Cohort 4 starts in February.