How to turn agent traces and evals into EU AI Act evidence
Engineering and product teams shipping AI agents in the EU need to connect responsible AI policies to the systems they build and operate. That means preserving evidence of how an agent behaves, how quality is measured, when people intervene, and what changed between releases.
In this live technical session, we will instrument and evaluate an agent in Phoenix using OpenTelemetry and OpenInference. We will then carry the resulting traces, eval results, annotations, and human-review decisions into Arize AX for production monitoring, CI release gates, retention, EU data residency, and audit history.
You will see how to translate requirements around recordkeeping, human oversight, robustness, and monitoring into artifacts that fit an engineering workflow. We will also distinguish between the evidence observability systems can provide and the decisions that still require legal or governance judgment.
Who this is for
This session is for AI engineers, product managers, and technical leads building or operating AI agents used in the EU. It will be especially useful if you are defining acceptance criteria or release gates for an agent, moving from Phoenix-based development into production, or connecting a responsible AI policy to a running system.
What you will learn
How to instrument an agent with OpenTelemetry and OpenInference and capture evidence for later review
How to map AI Act requirements to traces, evals, annotations, human reviews, release gates, monitoring, and audit logs
How product and engineering teams can turn quality metrics and human feedback into shared release criteria
Where Phoenix and Arize AX fit as an agent moves from local development into production
How to validate judge-to-human agreement before using an eval score as a governance signal
Which questions technical evidence can answer and which require legal or governance judgment
Format
30 min content + 15 min Q&A
Level
Intermediate