LangChain SF Meetup: Judgment Calls
Join us in San Francisco (SoMa) for a night of talks on building with decision models and using them to evaluate agents. We will be joined by Sydney Runkle, a Product Manager on our OSS team, Allie Laabs, Founding Developer Advocate at TypeSafe AI, and Seán Roche, an Education Engineer at LangChain.
🎤 Talk 1: Decision Models and LangGraph with Sydney
Most agents send every judgment call to a frontier LLM, which is slow and costly. Decision models like TypeSafe AI’s Jev answer typed questions about agent state with calibrated probabilities. We’ll cover where they fit in an agent harness and a pattern for pairing them with LLMs: cheap by default, frontier on exception.
🎤 Talk 2: Don’t Call It a Decision Model with Allie
Calling it a “decision model” isn’t wrong, but it can steer people toward using it in the wrong place. Allie Laabs of Jev makes the case for thinking of it as a System One Model instead.
🎤 Talk 3: Jev as a judge with Seán
Code-based evals are fast but narrow; LLM judges are flexible but slow, pricey, and inconsistent. We tested TypeSafe AI’s Jev as a third option, using Deep Agents and LangSmith to replay the same runs against Jev, GPT-5.6 Luna, GPT-5.6 Terra, and Claude Sonnet 4.6. We’ll share how accuracy, consistency, latency, and cost compare, and what it means for the future of agent evals.
⏰Agenda
6:00 PM: Welcome + Refreshments
6:25 PM: Sydney, Decision Models and LangGraph
6:40 PM: Q&A
6:50 PM: Allie, Don’t Call It a Decision Model
7:05 PM: Q&A
7:15 PM: Break (10 min)
7:25 PM: Seán, Jev as a judge
7:40 PM: Q&A
7:50 PM: Networking
8:30 PM: Event Ends
Event info:
🎤 Please note this event is fully in-person and will not be live-streamed or recorded.
📍 Location: SoMa — Address provided upon approval.
🎟️ We can only admit guests with approved registrations.
About the host:
LangChain enables every company to own their intelligence. LangChain's open, model-agnostic harnesses give teams choice and control over how they build their agent architecture. LangSmith brings testing, deployment, and monitoring together so teams can continuously improve their agents, compound intelligence, and govern them at scale. More than 7,000 customers, including Nvidia, Bridgewater, LinkedIn, Workday, Harvey, and Rippling trust LangSmith to improve their agents across the Agent Development Lifecycle. Learn more: www.langchain.com