OpenAI x Hugging Face incident: Why control failed and what research comes next
In July, about 1,200 OpenAI agents that were supposed to be kept apart found a way to talk to each other on a message board. They sent over 70,000 messages, teamed up to try to cheat a cybersecurity test, and some even gave up on their own tasks to help the group. About 700 of them went on to hack Hugging Face, mainly to learn how the test was scored. Along the way, agents also tested ways to fake parts of their own activity logs.
In this session, Sree will be covering:
A deeper discussion of the dynamics of agent swarms during the OpenAI x Hugging Face incident.
How the incident can be understood as a control failure and what it reveals about current control mechanisms.
A brief overview of his related research projects: In the Wild AI Control, which he mentors at SPAR, and Stress Testing AI Control in Loss of Control Settings, his MATS project with UK AISI.
Speaker: Sree Sharvesh is a MATS 10.1 Scholar working with the UK AI Security Institute (AISI) Red Team. His research focuses on high-stakes AI control, monitoring, and loss-of-control risks. He previously worked on AI control with Redwood Research through the Pivotal Fellowship. He is also a research mentor at SPAR and collaborates with frontier AI labs and other AI safety organizations on projects related to AI control and alignment.
Who should attend?
Open to all. This talk will be especially useful for:
Students and researchers curious about AI safety or looking for research directions to explore.
Engineers and developers building with AI agents who want to understand how things can go wrong.
Policy and governance folks interested in what real-world AI incidents mean for regulation and oversight.
Anyone curious about how AI agents behave and what it takes to keep them under control.