AI Safety Poland Talks #13

​Welcome to AI Safety Poland Talks!

​​A biweekly series where researchers, professionals, and enthusiasts from Poland or connected to the Polish AI community share their work on AI Safety.

​💁 Topic: Predicting the Predictor: Computational Mechanics for Transformer Interpretability
📣 Speaker: Mateusz Piotrowski
🇬🇧 Language: English
🗓️ Date: 30.04.2026, 18:00
📍 Location: Online

​Speaker Bio
Mateusz Piotrowski works on interpretability of large language models, primarily attribution methods, and is a co-creator of the open-source circuit-tracer library. He has also co-authored work applying computational mechanics to transformers, showing that attention implements constrained Bayesian belief updating.

​Abstract
Computational mechanics asks: given a stochastic process, what is the minimal structure an optimal predictor of it must maintain? It turns out what you need are the process's causal states and a belief distribution over them — and these depend only on the data, not on the architecture doing the predicting. This gives an alternative route into interpretability. Instead of taking apart a large model trained on data we don't understand, we train transformers on data from processes we do understand, derive what an optimal predictor has to represent, and check how well that matches what the model actually learned. I'll introduce the framework, walk through recent results from this line of work, and make the case for it as a path toward understanding computation in models we actually care about.