Decision models: cheaper judges and faster guardrails
What does it cost to run an LLM judge on every span?
Decision models are small, fast models built to pick from a set of choices. You pass in the input and the choices. They return each choice with a probability that it is the right one. They do not write text. They run faster than an LLM, and they are cheap enough to run on every span.
In this session we will cover:
What decision models are, and how the input and output work
A short demo of a decision model in action
Using them as eval judges
How they compare with an LLM on cost and speed
Using them as real-time guardrails
Where they fit, and where an LLM is still the right call
Who this is for
AI engineers building and evaluating AI agents.
What you'll leave with
How to read a decision model's probabilities, and how to use one as an eval judge and a real-time guardrail.
Format
30 min live session with demos + 15 Q&A
Level
Intermediate. Assumes knowledge of LLMs, observability, and evals.