Raymond Chua - Predictive Representations and Memory for Continual Reinforcement Learning

​Biological agents, such as humans and animals, are capable of continuous adaptation and learning in complex, dynamic environments. Despite recent advancements in Artificial Intelligence (AI), enabling continual learning remains a major challenge for deep Reinforcement Learning (RL) agents. Understanding the computational principles that support lifelong learning therefore represents both a neuroscience question and an AI challenge.

In this talk, I will present two pieces of work from my PhD that address this question. First, I will introduce predictive state representations through Simple Successor Features (SFs), which is an efficient approach inspired by Successor Representation, that is a computational model linked with hippocampal cognitive maps. I will show how these predictive representations improve continual RL while preserving the theoretical properties of the underlying SRs.

I will then discuss biologically-inspired synaptic consolidation mechanisms that interact with these predictive representations to improve continual learning in dynamically changing environments. Our results suggest that consolidation is particularly effective when it stabilises predictive representations rather than action-value mappings alone, supporting the idea that structured internal models may provide a more robust computational substrate for lifelong learning. Together, these studies illustrate how computational principles inspired by neuroscience can improve continual learning in artificial agents while offering new hypotheses about how biological systems acquire, adapt, and preserve knowledge over time.

Bio:
Raymond Chua recently completed his Ph.D. in Computer Science at McGill University and Mila – Quebec AI Institute, where he was co-supervised by Prof. Doina Precup and Prof. Blake A. Richards. His research lies at the intersection of neuroscience and artificial intelligence, with a focus on continual learning, reinforcement learning, and biologically inspired algorithms. During his Ph.D., he developed methods based on predictive representations and synaptic consolidation to improve continual adaptation in reinforcement learning agents, drawing inspiration from computational neuroscience and hippocampal theories of learning and memory.

He will soon be joining the Center for Theoretical Neuroscience at Columbia University as a postdoctoral researcher, where he will work with Prof. Ken Miller and Prof. Kim Stachenfeld on computational neuroscience and NeuroAI. His long-term goal is to understand the computational principles underlying biological intelligence and leverage them to build more adaptive AI systems while advancing our understanding of brain function.