SAIC #040: From prototype to production: knowing when your AI agent is ready
Building an AI agent that impresses in a demo can take an afternoon. Knowing whether it is ready for real users is much harder, and it is where most teams find themselves relying on intuition, without the data to back it up.
This session walks through the practices that help close that gap, from first evaluation to running safely in production. To keep things concrete, we use an example agent throughout: a multilingual (FR/DE/EN/IT) support agent built with LangGraph, MCP tools and RAG.
We will look at
Evaluation: defining quantifiable metrics for how the agent performs, and using a golden dataset to assess the impact of every change
Evolution: comparing models and prompts on the same dataset to catch regressions or validate hypotheses for improvement
Production: tracing an agent end to end and noticing problems before your users do
Value: connecting the agent's performance to business metrics, as a basis for discussing ROI
Presenter
Michele Ricciardi, Senior Sales Engineer at Datadog
Michele Ricciardi is a technologist with over ten years’ experience across software engineering, cloud architecture and applied generative AI.
He currently works at Datadog, where he helps enterprise teams adopt observability.
Previously at AWS, he helped organisations across Europe design and build generative AI applications.
Presentation language: English
Agenda
18:30: Doors open, welcome
19:00–20:00: Presentation
20:00–22:00: Networking, plus test drives of Tesla cars around the city
Venue
Tesla, Centre Confédération, Geneva. Thanks to Tesla for hosting us.
After the presentation, participants can join Tesla test drives and form small groups of up to 4 people to connect, exchange, and network.
Bring your driving licence if you want to take a car out. No licence, no drive, and there's no way around it on the night.
Registration
Free entry. Registration is required and confirmation is automatic. Capacity: 60 participants.