AWS + Arize Workshop: Evaluating and Observing Agentic Workflows

​Join Arize AI and AWS for a hands-on for a hands-on advanced workshop on building and evaluating AI agents in production. In this two-hour session, you’ll go beyond simple agent demos and learn how to design, instrument, and optimize complex agentic workflows using AWS Strands Agents and the Arize AX observability platform.

​Participants will build a complete end-to-end pipeline:

• ​Agent Development: Create a Strands agent with multiple tools powered by Amazon Bedrock

• ​Instrumentation: Add tracing and observability using OpenTelemetry and Arize AX

• ​Evaluation: Automate quality assessments with LLM-as-a-Judge techniques

• ​Optimization: Experiment with prompts and regression datasets to improve performance

• ​Monitoring: Deploy production-ready monitoring, dashboards, and alerts

​Expect to walk away with a deeper understanding of how to manage nondeterministic AI workflows, detect hidden failures, optimize agent decisions, and monitor costs and performance at scale.

​Who should attend:
This workshop is designed for AI engineers, ML practitioners, and developers with experience in LLMs who want to advance their skills in building production-ready agentic systems.

​Details:

• ​Date/Time: Doors open at 4 PM; content runs 4:30 – 7:30 PM

• ​Format: Lecture + hands-on labs (bring your laptop)

• ​Level: Advanced

• ​Food & beverages provided

​Agenda Overview

• ​4:00 PM – Doors open, food & beverages

• ​4:30 PM – Welcome & introduction to agentic AI systems

• ​5:00 PM – Hands-on lab: Build a Strands agent on AWS Bedrock

• ​5:30 PM – Add tracing & observability with Arize AX

• ​6:00 PM – Automate evaluations and optimize prompts

• ​6:30 PM – Monitoring, dashboards, and production best practices

• ​7:00 PM – Wrap-up, resources, and next steps
  

​**We have limited capacity — please apply only if you plan to join us in person.