Simulation Fidelity for AI Agents
Research-focused social event hosted by CollinearAI.
If you’re doing RL, evals, or post‑training for AI agents, you’re doing it in a simulated environment.
But how faithful does that simulation actually need to be? How important is reproducibility? What if the reward model is not deterministic? And what shortcuts hold up in internal evals but quietly break at test time?
These are decisions every team building serious agents is making right now, usually without much data to back them up.
We've been digging into this with folks across frontier labs and enterprise verticals, the same questions keep coming up. At this meetup, we’ll walk through concrete experiments and share which approaches have held up in practice and which ones haven’t.
We're bringing together a small group of senior researchers and builders from GDM, Anthropic, Cognition, Harvey, OpenAI, xAI, and others for an evening at the Collinear office in Sunnyvale to talk through them honestly.
Format
Short panel with key researchers who've built and broken simulation environments at scale
Open discussion
Food, drinks, and time to swap ideas
Who this is for
Researchers and engineers working on RL, post-training, evals, or agent infrastructure.
Apply to attend.
Nazneen Rajani — CEO & Founder, Collinear AI
linkedin.com/in/nazneenrajani
Nazneen has led post-training and evaluation work across Salesforce Research and Hugging Face, and has co-authored well over 100 papers and artifacts on LLMs, interpretability, and alignment, with work covered by the New York Times, Quanta, Science, and MIT Technology Review. She now leads Collinear AI, where she focuses on making frontier models safer and more reliable in production through high-fidelity simulations and evals.
Mikhail Yurochkin — Staff AI Scientist, LLM Data Team Lead, MBZUAI Institute of Foundation Models
linkedin.com/in/mikhail-yurochkin-a45659114
Mikhail leads LLM data mixing for pre-training at the MBZUAI Silicon Valley Lab and previously led the Statistical Large Language Modeling group at the MIT-IBM Watson AI Lab. His research spans LLM evaluation, post-training, data quality, efficient inference, and model fusion — including tinyBenchmarks, downloaded over 250,000 times on Hugging Face, and scaling laws for LLM skills that predict benchmark performance across model families. PhD in Statistics from the University of Michigan.
When & where
Date: Thursday, April 16
Location: Collinear office - 530 Lakeside Dr #100, Sunnyvale, CA 94085
Schedule
5:00 pm — Doors open, early arrivals
6:00 – 7:00 pm — Food + drinks
7:00 – 7:30 pm — Panel: Simulation Fidelity for Agent Training
7:30 – 8:30 pm — Open Q&A
8:30 – 10:00 pm — Networking, food + drinks
About Collinear
Collinear is building SimLab, a simulation platform that frontier labs and enterprises use to create complex, high-fidelity environments with realistic users, long-horizon tasks, and automated verifiers, for agent baselining, hill climbing, and user research. SimLab has delivered 18–60% improvements on long-horizon tasks while saving over $100M in GPU costs.