The QA Sprint: Automating AI Agent Tests with Google ADK
It works on your machine. But what happens when it's live? Stop flying blind. Join this hands-on engineering lab to automate your agent evaluation, fix prompt logic, and ship to production with confidence. Execution > Consumption.
Unlike traditional software, generative AI is non-deterministic. You cannot write a simple if statement to check correctness, and you cannot fix what you cannot measure. We believe in building true developer intelligence, which means moving past the hype and focusing on verified, production-ready systems.
Join us for an intensive, hands-on engineering sprint where we will walk through the professional lifecycle of agent testing using the Google Agent Development Kit (ADK). You will learn how to evaluate both the trajectory of your agent and its final response, ensuring your AI systems are built on solid engineering rather than guesswork.
If you are ready to transition your agents from a brittle prototype to reliable enterprise software, this code lab is for you.
What We Are Evaluating
Agent evaluation is more complex than standard LLM evaluation. We aren't just grading the text output; we are grading the logic. You will learn to measure:
Trajectory Evaluation (The Process): Ensuring the agent follows deterministic business logic (e.g., Did it call
check_inventorybeforeplace_order? Did it authenticate the user before checking a balance?). This is your regression testing layer.Response Evaluation (The Output): Validating language quality, safety boundaries, and checking for hallucinations. This is your automated quality assurance layer.
The Core Development Lifecycle
We will code our way through the industry-standard agent testing lifecycle, focusing on tangible proof of work:
Phase 1: Local Visual Inspection | Utilizing the ADK Web UI to manually trace agent logic, inspect token usage, and observe tool calling execution.
Phase 2: Unit & Regression Testing | Running automated test cases locally using the ADK CLI to catch quick logic regressions instantly.
Phase 3: Deep Debugging | Analyzing failed execution traces, isolating edge cases, and fixing prompt/tool routing logic.
Phase 4: CI/CD Pipeline Integration | Automating your agent evaluations using
pytestinside a continuous integration pipeline, ensuring no broken agents ever get shipped to production.
Who Should Attend?
This is a technical, code-first environment designed for:
AI Engineers and Backend Developers tasked with productionizing LLM systems.
QA/Test Engineers moving into AI quality engineering and validation.
Tech Leads looking to establish strict technical telemetry for internal AI projects.
Prerequisites (What to bring)
To follow along with the live code architectures, please bring:
A laptop and charger.
Your preferred IDE (VS Code, Cursor, etc.).
Strong basic proficiency in Python (specifically writing tests with
pytest).An active Google AI Studio account for API keys.
Event Schedule
10:00 AM: Doors Open, Coffee, & Environment Setup
10:30 AM: The Trust Gap: Trajectory vs. Response Evaluation
11:15 PM: Lunch Break 🍕
11:45 PM: Hands-on Lab 1: Visual Tracing with ADK Web UI & CLI Testing
1:00 PM: Hands-on Lab 2: Writing Custom Evaluation Metrics & Prompt Debugging
2:00 PM: CI/CD Integration: Automating Agent Tests via Pytest
2:45 PM: Architecture Q&A & Wrap-up
3:00 PM: Event Concludes
Hosted By
AIHouse is the community layer powering this ecosystem. Built for founders, researchers, engineers, students, and AI enthusiasts, AI House creates a space where people consistently learn, share, collaborate, and grow together. Through initiatives like 100 Days of AI House, community gatherings, founder conversations, knowledge-sharing sessions, and industry collaborations, AI House has built a thriving network of 15,000+ community members who are actively shaping the future of AI. More than a community, AI House is where meaningful relationships, opportunities, and ideas are created every day.
HiDevs is a Builder Intelligence Platform for the Agentic Era We help developers transform ideas into production-grade AI agents through personalized guidance, intelligent evaluation, continuous feedback, and contextual memory. By combining specialized AI agents with developer intelligence, HiDevs understands each builder's goals, tracks their progress, and provides the insights needed to accelerate their journey. Through 100+ community experiences, builder initiatives, and ecosystem collaborations, HiDevs has engaged 15,000+ developers and is building the infrastructure that powers the next generation of AI builders.
Together, HiDevs and AI House are building one of the fastest-growing AI ecosystems where ideas turn into products, builders become founders, and collaborations turn into impact.