Las Vegas | Claude Code Workshop
Join a Claude Community Event, organized by local enthusiasts for everyone who loves building with Claude!
----
Anyone can build an AI agent that works in a demo. The hard part is knowing it still behaves when a real customer is vague, angry, or changes their mind halfway through.
In this hands-on workshop you'll learn eval-driven development: turning business rules and observed failures into tests that tell you how often your agent gets it right, and whether your fixes actually helped.
You'll work on a customer refund agent for a fictional retailer, built with Claude Code on the Anthropic API. It runs, and it has flaws. Your job is to find them, measure them, and fix them.
What you'll do
Turn a vague business policy into a testable eval spec
Write test cases in plain English and let Claude Code turn them into a dataset
Run the agent, read its transcripts, and group its failures
Build graders: code checks for actions, plus an LLM judge you calibrate against your own labels
Fix the agent's prompt, tools, and design, then rerun and watch pass rates move
You'll leave with a working eval suite and a workflow you can apply to your own agents.
Who it's for
Engineers and engineering students. Mixed experience is fine: checkpoint branches catch you up if you fall behind, and stretch goals keep fast finishers busy. All data is synthetic.
What to bring
A laptop with Claude Code and Python 3 (detailed instructions to follow before the event)/
Basic git (clone and switch branches)
API credits provided for approved attendees. Register with the email you'll use for your Anthropic Console account.
You'll write and run Python from the terminal. Setup steps will be sent before the event.
Agenda
5:15 PM Doors open
5:30 PM Setup check
5:45 PM Why agents need evals, not just unit tests
5:55 PM From business policy to an eval spec
6:20 PM Building the test dataset, in plain English
6:45 PM Break
6:55 PM Baseline run and error analysis
7:25 PM Graders: code checks and an LLM judge
7:50 PM Fix, rerun, and harden
8:15 PM Wrap-up: shipping with online evals
8:30 PM Event ends. Hang around for Q&A until 9:00.
Times are approximate. Checkpoint branches keep everyone on track if you fall behind.
About Claude Community
Claude Community events are hosted by community members worldwide. Meet other people building with Claude and learn something hands-on. Hosted by Dom Roselli, Claude Community Leader, Las Vegas.