VAM! AI Reading Group - Paper Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

​1) 📄 Paper: Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

​https://arxiv.org/pdf/2601.07372

​2) Presenter: Safak Atakan Celik

​3) Paper 3-line Summary: Experimenting with a modification to the transformer block, the paper introduces the Engram module. The module aims to, with the combination of static n-gram lookups and context-aware gating, relieving the early layers from static reconstruction, freeing up more resources for complex reasoning and deepening the network. Let's get together and discuss how and if and why and what.

​Recommended Action Item:

• ​Reading the paper for increased engagement and more fruitful discussions. Even a quick skim goes a long way.

• ​It would be nice to have some questions in mind. No answers are guaranteed, but surely there will be some grounded brainstorming!

​

​Want to present?

• ​please message Issam Laradji if you would like to present an AI paper

​The list of papers will be available here: https://docs.google.com/spreadsheets/d/1HET5sjnHjwiF3IaCTipR_ZWspfgglqwdBFRWAfKBhp8/edit?usp=sharing

​To connect with the group, join the Discord: https://discord.gg/teJvEejs94

​Timeline:
🕠 7:00 PM – Arrival & Networking.

​🗣️ 7:10 PM ~ 7:55 – Paper Presentation & Discussions

​About the Facilitator

​Issam Laradji is a Research Scientist at ServiceNow and an Adjunct Professor at University of British Columbia. He holds a PhD in Computer Science and a PhD from the University of British Columbia, and his research interests include natural language processing, computer vision, and large-scale optimization.

​Looking forward to discussing the latest AI Papers!