MLn Club (ML Reading Group) #8: Nested Learning: The Illusion of Deep Learning Architectures

​Welcome to Week 8: Nested Learning: The Illusion of Deep Learning Architectures

​How might nested optimization structures allow models to generalize more efficiently by reusing global representations while adapting locally to new tasks?

Could the hierarchical nature of nested learning make large models inherently more resilient to poisoning or backdoor attempts by isolating adversarial influence within inner-loop adaptations?

​The Paper Link Here

A recent study by Google Research introduces Nested Learning (NL): a new paradigm that redefines deep learning as a hierarchy of nested optimization problems rather than stacked layers. The authors argue that modern neural networks, including Transformers, effectively compress their own context flow, and that in-context learning emerges naturally from this internal compression process.

​Through this framework, common algorithms such as SGD and Adam are reinterpreted as associative memory modules that store and adapt to gradient information. Building on these insights, the paper presents HOPE, a self-referential model that learns to modify its own update rules using a continuum memory system inspired by human neuroplasticity.

​Empirical results across language modeling and reasoning benchmarks show that HOPE outperforms recent architectures like DeltaNet and Titans, particularly in long-context reasoning and continual learning. These findings suggest a new direction for AI model design, toward systems that learn across multiple timescales and self-optimize beyond traditional backpropagation.

​Join us at CASI for discussion at 8 pm, and (optional) quiet reading from 7 pm.

​📖 Reading Recommendations, Questions, or Comments? Contact us here!
🔎 View past meeting notes here.

​What's this?

• ​A super warm group of folks discussing their favorite topics!

• ​In the first half, we host an optional quiet reading space

• ​In the second half, we have a discussion where people can talk about what they found interesting about the reading and ask questions about things they didn't understand

​When/Where:

• ​CMU AI Safety Initiative's Office, 201 Craig Street, right across the PNC bank. Look for the open door up the stairs.

• ​8pm discussion, 7pm optional quiet reading time.

​Here's how it usually goes:

​7:00 PM — arrival and settling in
8:00 PM — introductions
8:10 PM — discussion time
9:00 PM — wrap up then open discussion

​Who's it for?

​People who've been wanting to read up on the latest papers in ML and other fields but just haven't been able to find the time/motivation.

​Why:

• ​We've been procrastinating too much on our readings, even though we have so much fun doing them. We know we're not alone in this and want to keep others accountable for learning more about what they're passionate about!

• ​We've also met a ton of really fun friends by discussing what we care about!

​Rules/guidelines on how to act:

• ​Act like a host, include people in conversations, talk to people even if they're strangers, offer to explain what you know, and keep an open mind! come to read stuff and find super fun friends :)

• ​Bring snacks if you're feeling kind!