BrowserStack Meetup Birmingham: Testing QA for LLM & RAG Features

​Topic: Testing the Untestable: QA for LLM and RAG Features

More QA teams are being asked to test AI features, and the usual approach of asserting on exact output stops working. Ask a chatbot the same question twice and you may get two differently worded answers that are both correct, or one confident answer that is simply made up.

Join us online for the Birmingham Chapter meetup as Srinivas Santhanathan demonstrates how testers can apply their existing skills to LLM and RAG applications. Through a live demo of a customer support chatbot, discover how to build golden datasets, write pytest evaluations with DeepEval to verify data grounding, catch hallucination regressions from prompt tweaks, and run automated AI quality gates in CI.

🔍 What We’ll Explore
Applying Core QA Skills to AI: Leveraging familiar techniques like boundary value analysis and negative testing to construct robust golden datasets for LLMs.

Groundedness vs. Exact Matching: Writing pytest-based evaluations with DeepEval to verify that answers are grounded in source data, allowing reworded responses to pass reliably.

Catching Prompt Regressions: Detecting hallucinations triggered by prompt changes and establishing automated evaluation suites.

Quality Gates in CI: Running LLM evaluation suites within continuous integration pipelines to prevent flawed models from reaching production.

Starter Framework: Attendees will walk away with a practical starter framework ready to implement in their own projects.

🎙️ Featured Speaker
Srinivas Santhanathan — Web Developer & AI Integration Specialist at Design Factory Birmingham, Aston University

Srinivas focuses on LLM integrations, RAG pipelines, and agentic AI tools. Prior to AI integration, he spent over a decade in QA and test automation spanning healthcare, IoT, e-commerce, and audit technology. He holds an MSc in Artificial Intelligence from Aston University.

📍 Event Details
Topic: Testing the Untestable: QA for LLM and RAG Features

Date: Wednesday, November 11th, 2026

Time: 4:30 PM GMT (Birmingham Time)

Format: Online via Zoom (Join link shared upon RSVP)

Who Should Attend: QA Engineers, SDETs, Test Automation Specialists, and Developers working on or transitioning to AI and LLM feature validation.