MLn Club (ML Reading Group) #22: Neural Networks Are Already Clean
Welcome to Week 22: Neural Networks Are Already Clean
We will read this quite recent blog post on interpretability claiming (preposterously 🧐) that the cause for polysemanticity in early models was not superposition, and was instead due to mostly fixable issues. Further, it claims that modern neural networks are much more interpretable via normal science (simply performing experiments on the neurons) than many people expected and currently believe. The article gives some interesting examples of neural networks with almost all neurons assigned some explanation.
In "Neural Networks Are Already Clean," Nick Cammarata argues that neural networks are far more readable and interpretable than AI safety researchers previously believed, asserting that the field misread early evidence of "polysemanticity" (neurons responding to unrelated concepts). Rather than models arbitrarily packing concepts together via "superposition" to save space—a theory that led the field to rely on complex secondary models like sparse autoencoders—Cammarata demonstrates that neural networks primarily rely on recursive generalization, sprouting related concept sub-branches as they scale. By using modernized tools and architectures, the authors found that 90–97% of neurons in large modern language and vision models (including models up to trillions of parameters) have coherent, primary roles rather than arbitrary mixtures, suggesting that networks can be directly reverse-engineered without untangling complex internal knots first.
Join us at CASI for discussion at 8 pm, and (optional) quiet reading from 7 pm.
📖 Reading Recommendations, Questions, or Comments? Contact us here!
🔎 View past meeting notes here.
What's this?
A super warm group of folks discussing their favorite topics!
In the first half, we host an optional quiet reading space
In the second half, we have a discussion where people can talk about what they found interesting about the reading and ask questions about things they didn't understand
When/Where:
CMU AI Safety Initiative's Office, 201 Craig Street, right across the PNC bank. Look for the open door up the stairs.
8pm discussion, 7pm optional quiet reading time.
Here's how it usually goes:
7:00 PM — arrival and settling in
8:00 PM — introductions
8:10 PM — discussion time
9:00 PM — wrap up then open discussion
Who's it for?
People who've been wanting to read up on the latest papers in ML and other fields but just haven't been able to find the time/motivation.
Why:
We've been procrastinating too much on our readings, even though we have so much fun doing them. We know we're not alone in this and want to keep others accountable for learning more about what they're passionate about!
We've also met a ton of really fun friends by discussing what we care about!
Rules/guidelines on how to act:
Act like a host, include people in conversations, talk to people even if they're strangers, offer to explain what you know, and keep an open mind! come to read stuff and find super fun friends :)
Bring snacks if you're feeling kind!