Schedule Heavy and Long-Living Workloads in Highly Scalable Kubernetes Clusters

​Kubernetes was designed for short-lived and fast stoppable workloads, but batch jobs and ML training runs don't care about that. In this workshop you'll see how Apache Kafka becomes the backbone for orchestrating long-running workloads at scale, and how a message re-queuing mechanism keeps every job alive even when a runner crashes mid-execution. You leave with a concrete architectural pattern for your own cluster.