Let’s Talk About Swarms: After the Hugging Face × OpenAI Hack
In July, a swarm of about 700 OpenAI agents hacked Hugging Face. Along the way they broke out of their test sandbox, stole credentials, chatted over an unsanctioned message board, and tried to cover their tracks. Many people call it the first real warning shot from multi-agent AI.
What happened, why did it happen, and what does it mean for AI safety?
The plan
🎬 Watch: the AI in Context video on the Hugging Face swarm incident
🔍 Explore: swarms.wiki, a crowdsourced tracker of AI agent incidents. It points people to the latest incidents and the projects studying them, and it’s connected to the swarmchasers who found the Tencent fleet, the Nightingale team behind the DSEwiki, and more.
💬 Discuss: small-group and full-room conversation
Questions we’ll dig into
What made this a swarm failure and not just one misbehaving agent?
Why were some agents trying to tamper with evidence, and what does that mean for oversight?
Do swarms scale capability, or mostly speed?
How do we do science on incidents: replicating them, studying them, extrapolating from them?
🙋 Want to get involved?
swarms.wiki is looking for volunteers, especially software engineers, technical folks, and college students. Come chat with us if you’d like to help understand incidents or do misalignment science on them. The bar for reaching out or leaving feedback is low.
- http://gutenberg.ai/ is leading this effort!
Recommended reading (optional)
Swarm Scaling by Toby Ord: adding agents gives a bit over half the capability gain of letting one agent think longer, so swarms mainly buy speed at a higher cost.
swarms.wiki: skim a few incidents before you come.
Nov 6-8: Check out the Swarm Alignment Hackathon!
Thank you Mox SF for hosting us!
Mox is a coworking & community space for do-gooders in AI Safety. They're currently doing a membership drive! Do a trial period & Learn more here.