framework session: safe to infer?

​safe to infer:
open, closed and what’s actually in production

​a voice engineer worries about time-to-first-token. a content engineer worries about cost-per-output. same word, inference, completely different math behind it.

​this is a small, technical afternoon for engineers and researchers actually running inference systems in production. ten to twelve people, operators feeling the tradeoffs day to day, plus researchers working on scheduling and cost-latency problems who can connect what you're dealing with to what's actually being worked on right now.

​we start from different priority metrics, latency, cost, utilisation, compute, and build a shared framework live from wherever the insights lead.

​saturday, august 29th, 2:30pm