GPU Engineering Meetups: Fundamentals of GPU Orchestration

​When you train a large AI model across dozens, hundreds, or thousands of GPUs, you don’t just throw code at them. You need orchestration frameworks that handle parallelism, scheduling, and fault tolerance.

​In this talk, we’ll look at -

• ​Why GPU Orchestration Matters

• ​How Ray, Horovod and DeepSpeed work

• ​Consistency Models in These Frameworks

• ​Real-World Trade-Off

​Material Notes from the event are now available at: https://modelcraft.substack.com - Post: From TensorOps to TeraFlops

​Next event in the series will be announced soon too