Prod is DOWN!!!
Prod is DOWN!! is an event inspired by F*ckup Nights but for Devs!
Have you ever:
Accidentally deleted a production database on a Friday afternoon?
Exposed your LLM endpoints without rate limits and face a huge bill shock?
Forgotten guardrails and watched your AI agent get prompt-injected into exposing your secret keys?
From the same team that bought you RoastMyTechStack, Prod is DOWN!!! is a fun and casual way to learn from mistakes of your fellow devs'.
The Format:
1. Problem Framing: "Oh Sh*t" Moments in Production
We will kick off with a featured speaker sharing their firsthand experience of a catastrophic production outage, that exact moment everything went down and panic set in.
2. Interactive Breakout Sessions
Following the talk, we will split into smaller discussion groups to dive into three core pillars of incident management:
Damage Control: How to quickly isolate the blast radius and minimize further impact.
Disaster Recovery: Best practices for recovering from downtime and effectively managing stakeholders during a crisis.
Guardrails: Practical strategies and engineering practices to ensure a mistake like this never happens again.
We will regroup and share insights with the broader team afterwards
3. Foreshadowing Problems in the Agentic World.
To wrap up, we will contextualize these lessons for the next wave of technology. We will discuss how production failures might evolve in the near future as we transition to agentic AI systems, and what unique risks we need to anticipate.