SGLang Summit Workshop by Modal: Inference from First Principles

​A SGLang Summit ticket is required to attend the event and get approved on Luma. Both Full and Expo passes are accepted.

​Ticket 🎫 https://luma.com/sglang-summit

​Modern LLMs are increasingly co-designed with the systems that serve them, with architectures built from the ground up to be efficient for inference. But the reasoning behind these design choices is often reduced to graphs and benchmarks showing numbers going up, with little explanation of why they work. In this talk, we'll start from first principles; using simple analytical models, we'll derive key properties of today's models and serving systems, and explore what they suggest about the next generation of inference.

​Session Details

• ​Duration: 60 minutes

• ​Level: Moderate experience in LLM inference serving. Familiarity with concepts like sharding and hybrid attention will be helpful.

• ​What to bring: No laptop needed

​Speaker

• ​Gilford Ting, Member of Technical Staff, Modal