AI Safety Poland Talks #2

​Welcome to AI Safety Poland Talks!

​​A biweekly series where researchers, professionals, and enthusiasts from Poland or connected to the Polish AI community share their work on AI Safety.

​💁 Topic: Making LLM Unlearning More Selective with Collapse of Irrelevant Representations
📣 Speaker: Filip Sondej
🇬🇧 Language: English
🗓️ Date: 20.11.2025, 18:00
📍 Location: Online

​Speaker Bio
Filip Sondej currently work on removing dangerous knowledge and tendencies from LLMs. In the past he has worked on mitigating AI conflict and on multi-agent LLM security.

​Abstract
Current unlearning and safety training methods consistently fail to remove dangerous knowledge from language models. We identify the root cause - unlearning targets representations which are too general - and develop a highly selective technique that unlearns robustly while preserving general performance. Our method performs PCA on activations and module-output gradients to identify subspaces containing common representations, then collapses these subspaces before computing unlearning updates, a technique we term Collapse of Irrelevant Representations (CIR). This avoids unlearning general knowledge and targets only representations specific to the facts being unlearned. When unlearning bio- and cyber-hazardous facts from Llama-3.1-8B, we achieve over 30× greater reduction in post-attack accuracy than the best baseline (Circuit Breakers), while disrupting general performance 30× less, and using less than 3~GPU-seconds per fact.

​Preprint: https://arxiv.org/abs/2509.11816