Hands-on workshop: Build, trace, and eval
How do you know a customer-support agent retrieved the right policy, escalated the risky case, and did not invent a refund?
This is a hands-on workshop that takes a LangGraph customer-support agent through the core Arize AX dev loop: build → trace → offline-eval → experiment → chat live & watch traces.
You build "Sunny", a LangGraph customer-support assistant for a fictional outdoor retailer. The agent does real retrieval over a small policy knowledge base and escalates hard or high-risk requests to a person. The workhorse model is gpt-4o-mini. You then trace the run in Arize, score it offline against a curated golden dataset, and compare gpt-3.5-turbo with gpt-4o-mini.
What you will do
Build a tool-using agent: orders, refunds, knowledge-base retrieval, and escalation
Trace it into Arize with two lines of auto-instrumentation, including retrieval spans
Run an offline eval against a curated golden dataset, with a code check and an LLM judge
Compare gpt-3.5-turbo and gpt-4o-mini side by side
Chat live in a Gradio UI and watch your own traces appear in Arize
Who this is for
AI engineers, software engineers, ML engineers, and data scientists who are building agents.
What you will leave with
A working support agent, traces in Arize, an offline eval, and a model comparison you can rerun.
Format
Hands-on workshop of about two hours, then optional networking drinks. Bring a laptop and charger. Have an OpenAI API key and a free Arize account (see prerequisites below).
Agenda
1:45 PM | Arrival
2:00 - 4:00 PM | Hands-on workshop
4:00 - 5:30 PM | Networking drinks
Prerequisites
Please have these ready before the session:
An OpenAI API key (OPENAI_API_KEY) — get one at platform.openai.com/api-keys (Arize can provide this on the day if you don’t have one).
A free Arize account at app.arize.com, and your Space ID + API key from Settings → Space API Keys.
A Laptop
That's the only input you need. The notebook prompts for all three keys at the top.