90/30 Club Reading: Discovering Interpretable Algorithms by Decompiling Transformers to RASP

​Can we decompile a transformer to find out how it works? Join us to discuss Discovering Interpretable Algorithms by Decompiling Transformers to RASP: In this paper, we present a general method to extract such programs from trained Transformers. The idea is to faithfully re-parameterize a Trans- former as a RASP program and then apply causal interventions to discover a small sufficient sub- program. In experiments on small Transformers trained on algorithmic and formal language tasks, we show that our method often recovers simple and interpretable RASP programs from length- generalizing transformers.

​Paper link: https://proceedings.mlr.press/v306/huang26b.html

​Event Schedule:

​7pm-8pm: Quiet reading time, grab a snack and read! (optional)

​8pm-9pm: Group discussion about the paper 📝

​9pm-10pm: We have our space for a bit longer, stay to socialize or network!

​Thank you Mox SF for hosting us! Mox is a coworking & community space for do-gooders in AI Safety. Check out their memberships here.