Evaluating LLMs and Engineering Resilient Test Automation | The Test Tribe Huddle (North America Edition)
Building AI applications has become easier than ever. Building AI applications that are reliable, trustworthy, and maintainable is the real challenge.
As organizations adopt Large Language Models and AI-powered automation, new questions are emerging. How do you evaluate an AI system when the same input can produce different outputs?
How do you know if your RAG application is retrieving the right information? And how can test automation become resilient enough to adapt as applications constantly evolve?
The Test Tribe's North America Virtual Huddle brings together QA professionals, SDETs, automation engineers, AI practitioners, and engineering leaders to explore these challenges through two practical sessions.
Event Details
Date: 30 July 2026
Time: 12:00 PM – 02:00 PM ET
Platform: Zoom ( Joining details will be shared closer to the event )
Agenda (All the timings are in ET)
12:00 PM - 12:15 PM: Welcome and introductions
12:15 PM - 1:00PM PM - Session 1 – AI Evaluation Frameworks: Measuring Quality, Reliability, and Trust in LLM Applications by Sangeetha Kumari Gopal Prasad
1:00 PM - 1:10 PM - Solution Spotlight Demo by BrowserStack
1:10PM - 1:55PM - Session 2 – AI-Powered: Combining Visual Intelligence with Self-Healing Tests by Sathish Kumar
1:55 PM - 2:00 PM - Closing
Session 1 – AI Evaluation Frameworks: Measuring Quality, Reliability, and Trust in LLM Applications
Speaker: Sangeetha Kumari Gopal Prasad (AI Engineer)
Why This Session Matters
Deploying an LLM application is only the beginning. The bigger challenge is ensuring that it consistently produces responses that are accurate, reliable, and trustworthy. Unlike traditional software, AI systems generate non-deterministic outputs, making conventional testing approaches insufficient.
In this session, Sangeetha Kumari Gopal Prasad will explore how AI evaluation frameworks (Evals) help engineering teams validate LLM-powered applications before and after deployment. You'll learn practical approaches to designing evaluation datasets, defining success criteria, and measuring the quality of Retrieval-Augmented Generation (RAG) systems using frameworks like Ragas to assess retrieval quality, faithfulness, context relevance, and answer quality.
Session 2 – AI-Powered: Combining Visual Intelligence with Self-Healing Tests
Speaker: Sathish Kumar E (Principal SDET with more than a decade of experience)
Why This Session Matters
Traditional automation frameworks often struggle when applications evolve. Minor UI updates, locator changes, and visual inconsistencies can create flaky tests that demand constant maintenance.
Sathish Kumar E will demonstrate how AI-powered visual testing and self-healing automation are changing that reality. Through practical demonstrations and architectural insights, you'll see how intelligent automation can recover from UI changes, detect visual defects that functional assertions miss, and build resilient test suites that require significantly less maintenance.
Why Show Up on 30 July?
Because you'll learn practical techniques for evaluating LLM applications and discover how AI is making test automation more resilient and maintainable.
Because you'll connect with fellow quality engineers, automation architects, and AI practitioners who are solving these challenges in real production environments.
Because your perspective belongs in the conversation.
Become a Speaker at The Test Tribe Events
Wish to share your insights with the community at a future event?
Submit your talk idea here, and if it resonates with the perspective we are trying to build at our events, our team will connect with you.
About The Test Tribe
The Test Tribe is the world's largest software testing community, empowering testers globally since 2018. With 925+ events and 170K+ members across 130+ countries, we bring software professionals together to learn, grow, and connect through meetups, conferences, cohorts, masterclasses, and more.
By RSVPing, you agree to our Terms & Conditions and Privacy Policy and consent to be contacted by The Test Tribe and our event partners.