RL Environments: Building the Arenas Where Frontier Models Learn

​RL environments have quietly become one of the most consequential pieces of infrastructure in modern AI.

​This talk traces the evolution of what the field started with, what worked, what broke down at scale, and what's changing now. Anshuman and Naman will draw on their experience building AI systems at scale to unpack the craft behind great RL environments:

​What will speakers cover: The arc from simple RL benchmarks to today's fine-tuning environments for LLMs. Lessons from what didn't work: reward hacking, brittle rubrics, environments that optimise for the wrong thing.

​What makes a great environment: task design, reward shaping, and evaluation rubrics that actually correlate with capability. The shift toward multi-agent RL and what it demands from environment design.

​And finally where this is all heading, emerging patterns, open problems, and bets on what the next generation of RL environments will look like.

​Speakers

​Anshuman Singh, Founder of Scaler and Scaler AI Labs. Previously at Facebook.

​Naman Bhalla, Founder, Scaler AI Labs. He led Ads P&L Infra at Google. Built and scaled AI products reaching 10M+ users across Scaler and NPTEL within 12 months. One of the early engineers at Shipsy and CureFit.

​
Look forward to seeing you at the event.