Kyle O'Brien on how filtering pretraining data can make open weight AI models safer

​Can we stop open-weight LLMs from ever picking up dangerous capabilities - by never teaching them those capabilities in the first place?

​Kyle O’Brien, co-author of Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards, will walk through new results from training multiple 6.9B-parameter models from scratch on datasets with and without biorisk content. The team (with collaborators across EleutherAI, UK AISI, and Oxford) asks a simple but underexplored question: If you filter enough of the relevant pretraining data, do models stay ignorant of harmful know-how - and does utility suffer?

​Key findings we’ll discuss

• ​Knowledge prevention: Filtered models perform near chance on biorisk evaluations without notable regressions on general knowledge.

• ​Tamper-resistance: Even after fine-tuning on ~300M tokens of targeted biorisk data, filtered models remain well below the no-filter baseline.

• ​Defense-in-depth: Filtering can’t stop in-context injection of harmful knowledge, but it pairs well with circuit-breaking techniques; staged attacks still challenge today’s defenses.

​If robust, pretraining-time filtering could offer a simple, scalable way to reduce dangerous capabilities in open-weight models - while preserving usefulness for benign tasks.

​Format
30 minute talk → 30-minute Q&A. So bring questions.

​Pre-read (optional)
Paper, artifacts, and models: deepignorance.ai