Name: MLn Club (ML Reading Group) #8: Nested Learning: The Illusion of Deep Learning Architectures
Start: 2026-07-06T19:00:00.000-04:00
End: 2026-07-06T21:00:00.000-04:00
Location: Pittsburgh, PA

We meet weekly on Sundays in person to talk about ML papers! Optional quiet reading from 7 pm, discussion starts 8 pm. Come and join the discussion! Open to all. Website: https://czhs.github.io/mln/

MLn Reading Club

Welcome to Week 8: Nested Learning: The Illusion of Deep Learning Architectures

How might nested optimization structures allow models to generalize more efficiently by reusing global representations while adapting locally to new tasks?

Could the hierarchical nature of nested learning make large models inherently more resilient to poisoning or backdoor attempts by isolating adversarial influence within inner-loop adaptations?

: a new paradigm that redefines deep learning as a hierarchy of nested optimization problems rather than stacked layers. The authors argue that modern neural networks, including Transformers, effectively 

, and that in-context learning emerges naturally from this internal compression process.

Through this framework, common algorithms such as SGD and Adam are reinterpreted as 

that store and adapt to gradient information. Building on these insights, the paper presents 

, a self-referential model that learns to modify its own update rules using a 

Empirical results across language modeling and reasoning benchmarks show that HOPE outperforms recent architectures like DeltaNet and Titans, particularly in 

. These findings suggest a new direction for AI model design, toward systems that learn across multiple timescales and self-optimize beyond traditional backpropagation.

 for discussion at 8 pm, and (optional) quiet reading from 7 pm.

📖 Reading Recommendations, Questions, or Comments? Contact us 

People who've been wanting to read up on the latest papers in ML and other fields but just haven't been able to find the time/motivation.