EvalOps unfiltered #2: Evaluating LLM-based applications
LLM applications & agents behave differently with the slightest prompt tweak, context change, or input variation. If you're building anything real with LLMs, you already know the outputs can surprise you: and not in a good way. That's why you test. Full stop.
EvalOps Unfiltered is a practical event series for AI teams tackling the real-world challenges of evaluating LLM & agentic applications. Focused on the emerging field of EvalOps, it goes beyond benchmarks to address unpredictable model behavior, adversarial risks, and production readiness.
Sessions feature speakers sharing what worked for them and what they learned the hard way, followed by breakout discussions and honest conversations about what truly works when deploying LLM apps and agents.
What to expect (on 17. June 2026):
Doors open at 17:30. Event starts at 18:00.
🔧 Lightning talks from three teams presenting their evaluation & testing challenges:
Giulia van den Winkel, AI Conversation Designer @ GetYourGuide
Rouven Glauert, prev. Senior Applied Scientist @ Parloa; now Founder @ Lelia: "You rerun your simulations - your results don't hold. Now what?"
tbd.
🧠 Breakout sessions where you'll dig deep into one challenge, discuss solutions, and share experiences with fellow builders.
🍺 Drinks while the conversations continue!
No panels, no pitches: just builders sharing what's actually broken and collaborating on what might work. This isn't about theory. It's about the unglamorous, critical work of making LLM & agentic applications reliable for the real world.
Location: Berlin, Germany; more details upon registration.
Target Audience:
AI engineers wrestling with evaluation pre-release
Technical leads managing LLM-powered products
Data scientists designing and fine-tuning LLM-based applications
Product owners responsible for delivering reliable LLM-driven features
Please note: Attending the event is only possible upon confirmed registration.
Rhesis AI (www.rhesis.ai) proudly hosts this event in collaboration with AI NATION (https://www.ai-nation.de)