​An AI agent can pass every scripted test and still take an unexpected action once it encounters a situation the test suite did not anticipate.

​That creates a different kind of quality problem. Traditional automation remains useful, but deterministic checks alone may not be enough for systems that make decisions, use tools, and produce non-deterministic outputs.

​At The Test Tribe’s 28th Porto Meetup, Ana Ramos and Paulo Oliveira will explore two sides of this challenge: evaluating AI agents more effectively and turning QA knowledge into a reusable AI-assisted workflow.

​Event Details

​Date & Time: 12th August, 18:30 (Portugal Time)
Venue: Capital on Tap, Via do Castelo do Queijo N.º 395 Lojas 22 e 23, 4100-429 Porto
Language: Portuguese

​What's on the Agenda

​Session 1: QA for AI Agents: New Challenges, New Strategies

​Speaker: Ana Ramos, Quality Assurance Engineer at WIT Software

​Tools such as Playwright, Selenium, and Cypress are designed around predictable actions and expected results. AI agents introduce additional challenges because their responses and decisions may vary even when the underlying input appears similar.

​Ana will explain where conventional automation approaches may be insufficient when testing autonomous, non-deterministic agents. She will also cover agent-aware evaluation techniques tested against an LLM-powered API.

​The session will help attendees understand how the testing approach needs to change when the system under test does more than follow a fixed sequence of steps.

​Session 2: Stop Just Asking: Turn Your QA Knowledge into an AI Tool

​Speaker: Paulo Oliveira, Quality Assurance Engineer at Mindera

​Prompting an AI tool for individual tasks can be useful, but it often depends on repeatedly providing the same context, instructions, and judgement.

​Paulo will show how QA professionals can turn their own heuristics and decision-making into a reusable, context-aware workflow built on an AI CLI.

​The session will focus on moving from one-off prompts to a more repeatable way of applying QA knowledge through an AI-assisted tool.

​How This Meetup Solves Your Problem

​Teams testing AI-powered products often face questions that traditional pass or fail checks do not fully answer:

• ​How should an AI agent be evaluated when its behaviour is not completely deterministic?

• ​Where do existing UI automation tools fit, and where are additional evaluation methods needed?

• ​How can QA knowledge be captured in a reusable workflow instead of being rewritten in every prompt?

​The two sessions approach these challenges from different directions. Ana will focus on evaluating agent-based systems, while Paulo will explore how QA expertise can be turned into a repeatable AI-assisted workflow.

​Together, they offer a grounded view of how testing practices may need to adapt when AI becomes part of both the product and the testing process.

​Why You Should Show Up

​This meetup is an opportunity to learn directly from QA professionals working with these problems, ask questions about their approaches, and compare them with the challenges you are seeing in your own work.

​You will also have time to meet other testers, automation engineers, developers, and engineering professionals from Porto, continue the discussion after the sessions, and exchange ideas that may be useful for your team.

​Become a Speaker at The Test Tribe events:

​Wish to share your insights with the community at a future event? Submit your talk idea here and our team will get back to you, if the idea resonates with them too.

​About The Test Tribe

​The Test Tribe is the world's largest software testing community, empowering testers globally since 2018. With 700+ events and 150K+ members across 130+ countries, we bring testers together to learn, grow, and connect through meetups, conferences, cohorts, masterclasses, and more.

​By RSVPing, you agree to our Terms and Conditions and Privacy Policy and consent to be contacted by The Test Tribe and our event partners