Session 09: Making AI Trustworthy
Testing, Evaluating, and Securing Intelligent Systems
About the Event
AI applications can produce impressive results, but how do we know when they are actually working well?
Traditional software can be tested with predictable inputs and expected outputs. AI systems are different: their responses can change, information can be misunderstood, and seemingly small changes can affect the quality of an entire system.
As AI becomes responsible for searching information, using tools, and carrying out tasks, simply asking a few questions and deciding that "it looks good" is no longer enough.
In Session 09 of this series, we focus on making intelligent systems measurable, testable, and secure.
Building on the retrieval systems, AI tools, and workflows developed throughout the series, we will explore how to evaluate AI responses, test systems against unexpected or malicious inputs, add safeguards, and automatically detect when changes negatively affect system quality.
What We'll Cover
Testing AI Systems: Understanding how testing AI differs from traditional software and why both quality and behaviour need to be measured.
Measuring AI Quality: Learning how to evaluate whether AI found the right information, used it correctly, and produced a useful response.
Testing for Unexpected Behaviour: Exploring how AI systems can be tested against misleading, malicious, or unexpected inputs.
Building Guardrails: Adding protections that help prevent sensitive information leaks and unauthorized actions.
Continuous Testing: Understanding how automated evaluations can run alongside regular software tests to catch problems before they reach production.
The Hands-On Codelab: Putting AI to the Test
In this practical workshop, participants will:
Create a test set for evaluating their AI retrieval and response quality.
Build automated checks to measure the quality of AI-generated results.
Test their AI tools and workflows against challenging and potentially malicious inputs.
Add safeguards around AI inputs, outputs, and tool actions.
Connect automated AI evaluations to the development workflow so changes can be tested continuously.
By the end of the session, participants will have a system that does more than work—it will have a way to measure its quality, identify weaknesses, and prevent regressions as it evolves.
The Big Idea
If we can't measure whether an AI system is working well, we can't reliably improve it.