Build Reliable AI Agents with Open Source | Hands-on Workshop, SF

​A 2-hour hands-on session for AI developers building real AI Agents or Applications. Learn how to build accurate, production-ready AI agents using simulation, evaluation, and optimization with open-source libraries.

Most AI agents work in demos. They break in production.

​​The gap between "it works on my laptop" and "it works for 10,000 users" is reliability. And right now, most teams are shipping agents with no systematic way to test, evaluate, or improve them before they go live.

​​This workshop changes that.

​​What you will build (hands-on)

​​You will go from a raw AI agent to a production-grade one in 2 hours, using fully open-source tools. The session walks you through three stages:

​​Simulate -- Generate hundreds of realistic user conversations with your agent before real users ever touch it. Catch failures that unit tests miss.

​​Evaluate -- Run your agent through structured evals that go beyond "does it respond." Measure accuracy, hallucination rates, tool-calling correctness, and task completion across scenarios.

​​Optimize -- Use the eval results to systematically improve your agent's prompts, tool configs, and guardrails. No more guesswork tuning.

​​By the end, you will have a repeatable workflow for shipping agents that actually hold up in production.

​​What to bring

• ​​A laptop with Python installed

• ​​Familiarity with building LLM-based agents (any framework: LangChain, CrewAI, LlamaIndex, OpenAI SDK, etc.)

• ​​A willingness to break things and fix them

​​Who is this for

​​AI engineers, backend developers working with LLMs, and anyone shipping (or about to ship) AI agents into production. Whether you are building customer support bots, coding assistants, RAG pipelines, or multi-agent workflows, this applies to your stack.

​​The tools

​​Everything in this workshop runs on Future AGI's open-source libraries, which cover the full agent reliability lifecycle: simulation, evaluation, optimization, observability, and guardrails. You will leave with the code, the workflow, and the tools installed on your machine.

​​Details

• ​​Date: May 13, 2025

• ​​Duration: 2 hours

• ​​Location: San Francisco (venue TBA)

• ​​Cost: Free

​​Spots are limited. Register to confirm your seat.