Breaking Models 🚨AI Security
Breaking Models 🚨 Session 9: Response, Reproduce & Fix Edition
Three talks, all about what happens after someone finds a bug. Starting with the one the whole internet is scrambling to patch.
Talk 1 — From Report to Fix: Applying SAMM to AI Products
Speaker- Arshi Chadha, Senior PSIRT Engineer (preview of her OWASP SAMM User Day talk, Nov 4)
Say someone reports that your AI assistant showed them another user's private doc. With a normal bug, you'd start with the version number. With AI, that barely helps. You need the prompt, what the assistant pulled in, how it was set up and what it was allowed to touch. And a lot of AI fixes happen quietly on the server, with no new version at all. So how does anyone know it's actually fixed? Arshi breaks it down with a simple checklist, and looks at EchoLeak and Amazon Q again, this time at how they got fixed. She ties it to OWASP SAMM, the model a lot of companies use to grade their security programs. It's a dry run, so be brutal. Tell her what's confusing or boring.
Talk 2 — Let the Agent Hack. Keep It in the Box: Securing Autonomous Vulnerability Reproduction
Speaker- Mihir Shah, Security Engineer & Tech Lead, Office of the CISO, Google Cloud
AI agents can now take a bug report and try to reproduce it on their own. They write code, install what they need and keep at it until something works. Super useful, and a little scary. What stops one from hitting the wrong target, leaking data or leaving a mess behind? Mihir (author of Cloud Native Software Security Handbook and OWASP VXDF project leader) shows how to keep the agent in a box, using guardrails that live outside the model. Also, agents cheat. One might switch off a protection or fake a win and call it done. He'll show how a second agent catches that.
Talk 3 — PHP2Shell: AI Found It, Attackers Beat the Patch
Speaker- Anurag Sharma, Software Engineer at Intuit
On Oct 21, PHP shipped emergency patches for two critical bugs, now called PHP2Shell. One crafted web request, no login needed, and an attacker could run code on the server. PHP runs about 70% of the web, so this one is big. The bug had been sitting in PHP for over a decade. An audit even spotted the root problem back in 2024, but no fix came. This year, AI helped researchers turn it into a working exploit, and attackers were using it before the patch was out. A quick rundown of what happened and what it means now that AI can dig up old bugs like this.
Talks first, then open floor for questions and sharing audience experiences. No vendor pitches. Beginners welcome!
📩 RSVP (or to present at a future session): aisecurityinbay@proton.me
📅 1 November · 2:30pm - 4:30pm
📍 Hacker Dojo, 3200 Patrick Henry Dr, Santa Clara (new location!)