What does it take to find scaling laws for AI R&D?

​AI systems are becoming increasingly helpful for conducting AI research, from generating code for experiments to discussing new research ideas and writing papers. But just how good are AIs at doing AI R&D autonomously, and why should we care? 

​Lawrence will talk about METR’s recent efforts at making realistic AI R&D evaluations and the results on frontier models such as Anthropic’s Claude 3.5 Sonnet and OpenAI’s o1.

Using the results from METR’s human baseline studies and autonomous capability evaluations, he’ll discuss how far current AIs are from catching up to human ML research engineers, and how we might create “scaling laws” for powerful real-world capabilities like AI R&D.

Join BuzzRobot Slack to connect with the community