Your Agent Hasn't Faced Adversity Yet!

​Please join us as we host FJ Collins coming in from NYC to discuss how the team at Intuit prepares their agents for the real world.

​Abstract

​Every coach knows a play that looks perfect on a whiteboard can fall apart the moment a defender brings the heat. So why do we expect AI agents to be any different? In the lab, your agent is playing against air; in the real world, it's facing a defense it hasn't practiced for. This talk breaks down three field-tested strategies to move your AI from "meeting room ready" to "game day proven."

First, we'll look at the Scout Team approach: deploying adversarial "confused user" agents to throw every unexpected look at your product before your customers do. Second, we'll discuss how to "watch the film"—systematically calibrating AI judges against human ground truth (using Cohen's κ) until you can trust the machine to make the right call unsupervised. Finally, we'll explore the role of the Head Coach: ensuring that while agents execute the plays, a human always retains the final say on the high-stakes decisions.

These aren't drills run in a sandbox — they're the same practices behind the agents powering Intuit's Expert Network of 31,000 experts. The takeaway is simple: an AI doesn't earn trust by working once in practice. It earns it by surviving the defenses it was never designed to face.

​Location

​This event is hosted in the Teamworking by TechNexus space. Upon arrival, guests can usually come directly to the 12th floor. If we have a large turnout and you arrive after 6pm, you may need to check-in with the security desk before heading up. The reception team will be there to greet you and help guide you to your room.

Parking & Transportation
The Civic Opera Building does not have a dedicated parking lot. There is limited metered parking nearby. We recommend using iParkit or SpotHero to find discounted parking options. The CME Building (10 S. Wacker) is the nearest parking garage, with several additional garages located within a two-block radius.

​The venue location is highly walkable and conveniently situated near multiple CTA train lines and bus stops, making it easy to access via public transportation.

​Agenda

​6:00 - 6:30 Networking and Bites

​6:30 - 6:45 Rising Star Speaker

​6:50 - 7:30 Presentation

​7:30 - 7:40 Q&A

​7:40 - 8:00+ Networking and Wrap Up

​Sponsors:
​​Food and Beverage: Focused an exclusive LangChain partner and boutique consultancy dedicated to building production-ready AI systems that work in the real world.

​Venue: Teamworking by TechNexus

​Continue the conversation between gatherings. Join our discord (Join Discord Server)