Beyond the leaderboard: AI agents that learn in simulated worlds
Who it’s for
AI engineers, agent builders, and AI/ML leads taking agents to production.
Benchmarks say your agent is ready. Production disagrees.
AI agents pass the leaderboard, then fail in front of real users who change their minds, push back, and ask for things no test covered. In this live session, we’ll show how simulation closes that gap with a digital twin of your agent that learns from every mistake. Then we’ll take you inside how frontier labs evaluate and improve their models.
What we’ll cover
Part 1 · Research talk (20 min)
How simulated worlds catch the failures static tests miss
with Dr. Elad Levi
The leaderboard vs. production gap
Building a digital twin of your agent: tool mocks, personas, scenarios and synthetic artifacts
Putting the twin to work: evaluation, optimization and failure replay
Part 2 · Fireside chat (25 min)
An inside look at how frontier labs evaluate and improve models
with Reah Miyara and Dr. Ilan Kadar
Model evals at OpenAI and Google
Simulated environments for self-improving agents and models
Where the shift in models is heading, from frontier closed models to open source
Part 3 · Live Q&A (15 min)
Your questions on simulation, evals and self-improving agents. Bring the edge cases your agent keeps tripping on.
Speakers
Reah Miyara, Senior Director, Gemini Models, Google Cloud
Leads work on Gemini models at Google Cloud. Previously led the Reinforcement Learning Platform and Model Evaluation teams at OpenAI.
Dr. Elad Levi, Co-founder & CTO, Plurai
Creator of AutoPrompt (2,700+ GitHub stars) and co-creator of IntellAgent. Former Principal Scientist at Sightful and AI Architect at Nexar. PhD in Mathematics.
Dr. Ilan Kadar, Co-founder & CEO, Plurai
A decade of deploying AI at scale, from autonomous driving to GenAI. Former EVP of AI & Automotive at Nexar, working with Waymo. Co-creator of IntellAgent. PhD in Machine Learning and Computer Vision.
Can’t make it live? Register anyway and we’ll send you the recording.
Join our community on Discord: Join Discord Server