Lightning Lab: Ship Your First Eval.
You're Already Running Agents in Prod. Ship Your First Eval.
Your agent already runs on Inngest; every step is durable, and its state is recorded. Rich evaluation data already exists as a byproduct.
In this session, we’ll show you how to turn runs you’re already producing into continuous online scores without adding a second SDK or system.
In 20 minutes, with code and live demo footage, we'll cover:
Determining a first scoring metric
Attaching a scorer to a function you've already deployed
Deferred scoring, so a real product outcome that lands hours or days later becomes the score
Weighted experiments across models and parameters on live traffic
We'll also be direct about where this doesn't fit: if you need versioned offline datasets and offline experiment tracking, online scoring is a complement, not a replacement.
For: teams already running functions or agents on Inngest who haven't turned on Agent Evals yet.