Building Agents and Eval Harnesses with Local LLMs with Ravin Kumar (Google DeepMind)
The industry is moving rapidly beyond simple chatbots toward autonomous agents: systems that don’t just talk, but actually act. But building an agent that reliably triggers the right tool at the right time is a massive engineering challenge. It requires moving past the "vibe check" era of AI and into a world of rigorous evaluation, an understanding of the subtle gradient between a single function call and a multi-step agent, and sometimes even specialized fine-tuning for on-device applications.
Join host Hugo Bowne-Anderson for an inside look at the frontier of agents, evals, tool-use, and local LLMs with Ravin Kumar, a Researcher at Google DeepMind. Ravin has been on the front lines of developing the Gemma model family, including the newly released FunctionGemma. Whether you are building massive agentic loops with frontier models like Gemini 3 Flash or deploying specialized, on-device models to mobile hardware, this session provides the technical blueprint for making agents production-ready.
In this live, code-forward workshop, we will dive into the practicalities of evaluation and fine-tuning for tool-calling models. We’ll move from basic prompts to building specialized agents capable of executing complex mobile actions.
In this session, you will learn:
The Function-to-Agent Gradient: Navigate the technical spectrum between single-step tool calling and multi-hop autonomous agents, and learn how "FunctionGemma" helps unlock this progression.
Beyond the "Vibe Check": Move past natural language benchmarks to implement rigorous evaluation frameworks. Learn to measure function selection accuracy, parameter integrity, and how to conduct component-wise failure analysis.
Fine-Tuning for High-Stakes Actions: Discover how to take a base model and fine-tune it for specific, task-oriented schemas—specifically focusing on on-device actions like system settings, messaging, and hardware control.
Frontier vs. Local Models: Understand the cost-performance Pareto frontier. We’ll discuss when to reach for a massive model like Gemini 3 Flash and when a specialized, 270M parameter local model is the superior choice for latency and privacy.
The Path to the Edge: Explore the workflow for deploying agentic capabilities directly to mobile devices using frameworks like Google AI Edge, turning a model into a truly integrated assistant.
Prerequisites: This workshop is designed for developers and data scientists. Familiarity with Python and basic LLM concepts is recommended. We will be using Google Colab and Hugging Face for the hands-on portions.
Register to join live or get the recording and notebooks afterwards.