Reading Group (+🧋): Continual Learning Bench
Join the Snorkel AI Reading Group, a recurring forum to explore the latest frontier developments in AI while building meaningful connections within the community.
In this afternoon's session, UC Berkeley's Parth Asawa will present his recent paper, Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments, a collaboration with Snorkel AI and the University of Wisconsin-Madison.
Agenda:
4 pm - doors open
4:30 pm - talk begins
🧋🧋🧋 Boba tea and other refreshments will be provided ! 🧋🧋🧋
Among other things, you'll learn:
What separates an agent that truly learns on the job from one that just looks smart on a single task.
How Continual Learning Bench tests 6 real domains, from coding to poker to epidemiology, where agents must adapt across sequences, not solve problems in isolation.
Why a new "gain" metric is the only fair way to measure learning, stripped of raw model skill.
The upset: simple context memory beats expensive, dedicated memory systems like Mem0 and ACE.
Why even the best AI system today only captures a quarter of the learning that's possible.
Continual Learning Bench is a collaboration between UC Berkeley, Snorkel AI, and the University of Wisconsin-Madison, supported by Snorkel AI's Open Benchmarks Grants program and the Laude Institute's Laude Slingshots program.