AI Safety Reading Group - author Jonathan Bostock: Stress-testing Alignment Midtraining

BlueDot Impact

Zoom

Past Event

Welcome! To join the event, please register below.

About Event

Hi all! Join us for an author presentation by Jonathan Bostock of Stress-testing Alignment Midtraining in the AI Safety Reading Group!

Can we trust AI models to follow alignment principles in situations they were never explicitly trained for?

This paper investigates whether Alignment Midtraining (AMT) can produce models that remain reliably aligned in unfamiliar deployment settings. Testing models up to 110B parameters, the authors identify important limitations:

Fragile Motivations: Midtraining can steer a model toward a desired motivation, but even a tiny fraction of fine-tuning data suggesting a competing motivation can erase that effect.
Limits of Generalization: Models struggle to robustly follow rules without demonstrations during training, raising concerns about their reliability when encountering unfamiliar situations.

Presentation then active discussion with ~100 participants usually. All are welcome (do read the paper/summary).

📄Paper: Stress-testing Alignment Midtraining
Authors: Sid Baines, Jonathan Bostock, Maria Angelica Martinez, Andrew Draganov, David Africa, Daniel Tan
(2026-09-17 arXiv:2609.20412)

Links (full URLs below): Paper, LessWrong, Blog

Host: Andreas Turanski (linkedin)
Co-hosts: Nange Sam (linkedin)

☑️RSVP here on Luma to join
🔗Also: AI Safety Reading Group https://evalsreadinggroup.com/📢 sign up to present (or via website), or contact us at 📧 evalsreadinggroup@gmail.com with questions. Everyone is welcome!