Name: MLn Club (ML Reading Group) #12: Towards Monosemanticity
Start: 2026-08-10T19:00:00.000-04:00
End: 2026-08-10T21:00:00.000-04:00
Location: Pittsburgh, PA

We meet weekly on Sundays in person to talk about ML papers! Optional quiet reading from 7 pm, discussion starts 8 pm. Come and join the discussion! Open to all. Website: https://czhs.github.io/mln/

MLn Reading Club

Welcome to Week 12: Towards Monosemanticity — Decomposing Language Models With Dictionary Learning

Can we identify meaningful, human-interpretable features inside a language model when individual neurons are polysemantic and represent many unrelated concepts at once?

Can sparse autoencoders recover the hidden features represented in superposition—and give us a better unit for mechanistic interpretability than neurons themselves?

 investigates whether sparse autoencoders can decompose a language model’s activations into interpretable features hidden by 

. Rather than treating individual neurons as the fundamental units of computation, the authors train overcomplete sparse autoencoders to reconstruct a transformer’s MLP activations as sparse combinations of learned feature directions—revealing concepts such as 

Arabic script, DNA sequences, Base64, and Hebrew

 that may be distributed across many neurons.

The resulting features are substantially more interpretable than neurons and can also behave as 

: artificially activating features can steer the model toward the corresponding behavior, such as generating Base64 or Arabic text. The paper also uncovers deeper structure, including 

 as dictionary size increases, similar features appearing across independently trained models, and groups of features interacting in ways resembling 

. Together, these results provide an early proof of concept for 

sparse autoencoders as a tool for mechanistic interpretability

, suggesting that models may be better understood in terms of learned features rather than their raw neuron basis.

 for discussion at 8 pm, and (optional) quiet reading from 7 pm.

📖 Reading Recommendations, Questions, or Comments? Contact us 

People who've been wanting to read up on the latest papers in ML and other fields but just haven't been able to find the time/motivation.