90/30 Club (ML reading) #55: Kernel Krazy
Week 55: Cublass and Kernel Review
Additional Review
This week, we'll be discussing How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklog. The paper provides a practical walkthrough of the iterative process behind high-performance GPU kernel optimization, covering techniques such as tiling, memory hierarchy utilization, vectorization, and workload scheduling. It offers valuable insight into how modern AI workloads achieve efficient execution on GPUs and serves as a useful foundation for next week's hands-on optimization project. We do not mandate the additional readings, but it will be beneficial for the following week, in which we will issue a small project to be completed before the reading group (1 to 3 hours). We will release the project shortly.
Join us at Mox to explore:
What were the most impactful optimizations in closing the gap between a naive matrix multiplication kernel and cuBLAS-level performance?
How much of GPU performance comes from algorithmic improvements versus hardware-aware engineering decisions such as memory access patterns, occupancy, and data movement?
As AI workloads continue to scale, where are the biggest opportunities for custom kernels to outperform general-purpose libraries like cuBLAS?
Discussion at 20:00, (optional) quiet reading from 19:00.