Chicago Workshop: Building Legal AI Agents You Can Trust
A hands-on lab for handling the most common failure-modes for agents in legal.
Context
A legal AI answer can sound right, cite sources, and still miss the thing that matters. In this hands-on workshop, you'll investigate realistic failures: missing authority, outdated sources, weak citations, conflicting evidence, and answers that are more confident than the underlying material allows.
Your job is to figure out what went wrong, trace the answer back to the evidence, and decide whether you'd actually trust it.
Program
We are joined by speakers from Qdrant and Arize, and together they'll demonstrate how the most advanced teams build and deploy agents that can be trusted in high-stakes domains.
5:30pm - Doors open | Food, Drinks & Networking
6:00pm - Introductory statements
6:10pm - Voice Agents in Legal Demo
6:15pm - Evals in Practice Presentation [Talk by Arize]
6:30pm - The Reliability Lab Workshop [Run by Qdrant]
8:00pm - Breakout for Networking
9:00pm - Doors close
What You'll Learn
Arize will open with how teams building AI for high-stakes domains actually evaluate it: what to measure, what "good" looks like when there's no single right answer, and how to tell whether your system is improving or just changing.
With that in hand, you'll go looking for failures yourself, run by Qdrant. You'll investigate realistic breakdowns in a legal AI system: missing authority, outdated sources, weak citations, conflicting evidence, and answers more confident than the underlying material supports. For each one, your job is to figure out what went wrong, and trace the answer back to the evidence.
Most of these failures aren't the model's fault. They happen upstream, in retrieval — the right authority was never pulled, or a superseded version outranked the current one, or the search returned five documents that agreed and missed the one that didn't. You'll work in a real retrieval stack and see how what you feed a model determines what it can possibly get right.
By the end you'll have a working mental model for where legal AI breaks, language for describing it to your team, and a starting point for evaluating systems you're building or buying.
What you need
A laptop. That's it. We provide the environment, the data, and the exercises — no setup, no coding environment, no prior ML experience. If you'd rather watch than work through it yourself, that's completely fine.
Who this is for
Legal technologists, innovation and KM teams, engineers building legal products, and anyone responsible for deciding whether an AI system is good enough to put in front of lawyers.
Space is extremely limited — register early.