Building the Agent Factory: Build, Test and Improve Agents as a Team

​A hands-on masterclass for AI product teams

​Getting an agent to work once is a milestone. Making it work after the next change, and giving your team a repeatable way to build the next one, is a different challenge.

​Software factories are attracting attention because they ask how much of development can become repeatable or autonomous. In this masterclass, we bring that question to AI agents: what should your team standardise, what can agents help automate, and how do you know whether the result is ready to go live?

​Working in small groups, you’ll build a customer-service voice AI agent in Rasa, connect it to a simulated business system, and test what happens when the conversation or the integration goes wrong. Then you’ll introduce a change and check whether it improves the agent without breaking existing behaviour.

​You’ll leave with a working sandbox example, test results, and a checklist you can use with your own team.

​Who this is for:

​This session is for leaders moving from AI experiments into working applications, adding agents to customer workflows, or improving systems already deployed.

​You do not need prior Rasa or agentic experience. Bring a laptop and familiarity with a customer or operational workflow. The technical build uses a prepared starter project; groups will include someone comfortable reading configuration or code. Setup instructions and a short readiness check will be sent in advance.

​You’ll learn how to:

​- Turn a business request into an agent specification with observable success criteria.

​- Connect an agent to a business system and handle unsuccessful or ambiguous outcomes.

​- Identify which parts of agent development can be automated next—and which need stronger checks first.

​The build:

​Your group will build an agent that helps a customer check an order and request a delivery-address change.

​The starter project includes synthetic orders and a simulated order API. Your job is to make the agent understand the request, apply the eligibility rule, confirm the intended change and report the actual result.

​Then we’ll challenge it: an ambiguous request, an order that has already shipped, and an API failure. Finally, you’ll introduce a policy change and rerun your checks.

​The goal is a small, inspectable system with evidence of what works and what still needs attention.