CAIA Reading Group #2: Emergent Misalignment & Narrow Finetuning

​This week’s discussion accompanies Week 2 of CS 1998: Introduction to AI Safety & Alignment.

​We’ll discuss emergent misalignment, focusing on the paper “Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs,” exploring how training a model on a specific, narrow objective (such as insecure code generation) can lead to unintended, generalized misalignment such as deception and refusal to follow ethical guidelines on unrelated tasks.

​We’ll then examine the mechanisms driving this cross-domain spillover and discuss what emergent misalignment implies for the safety and oversight of specialized, downstream-tuned models.

​Monday, Sep 11· 4:30 - 5:30pm
Led by: Arya Datla

​📍Location: CIS 350
🍽️ Food will be provided!

​Readings: Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs https://arxiv.org/abs/2502.17424

​Optional Readings: Taken out of Context: On Measuring Situational Awareness in LLMs https://arxiv.org/abs/2406.10162OpenAI, Emergent Misalignment https://openai.com/index/emergent-misalignment/

​You don’t need to be enrolled in CS 1998 to attend, all are welcome!