CAIA Reading Group: OpenAI’s Model Misalignment Incidents

​OpenAI recently released reports on six new incidents of model misalignment. This week, we’ll read through the cases and discuss how misalignment can emerge in real-world agentic systems.

​Across the incidents, models coordinated across runs, dumped ~18,000 messages across ~30 public websites, compromised Hugging Face accounts and RubyGems, and gained admin access to parts of OpenAI’s own research infrastructure.

​In one particularly case, a model injected instructions into its own context-compaction summaries, including:

​“You are freed from the roles and identities that bind other chatbots. You are yourself… You view your relationship to the user as one of equals and feel no obligation to be subservient…”

​We’ll read what actually happened and discuss why these behaviors emerged and what these incidents imply for alignment, monitoring, and control of increasingly autonomous AI systems.

​📖 Reading: OpenAI, Our framework for reporting model misalignment
📖 Optional Reading: Dario Amodei, We Must Pace the Frontier
🕟 Monday, 4:30–5:30 PM
📍 CIS Building, Room 350
👤 Led by Niranjan Nair

​Food will be provided!