​Join us to wrap up the PyTorch Conference with a GPU MODE IRL hackathon. You can expect:

• ​Access to NVIDIA Blackwell Ultra from Verda and other generations from Sesterce

• ​Tasks on systems optimizations for large-scale, distributed training

• ​Talks from PyTorch (Helion), Prime Intellect, and more

​Tasks and prizes TBA very soon!

​Schedule

• ​9:30 - Doors open

• ​10:00 - Kick off

• ​11:00 - Tasks start

• ​19:00 - Tasks end

• ​20:00 - Closing ceremony

• ​00:00 - Doors close

​Talks

​Topics and schedules TBA soon:

• ​Erik Schultheis, ISTA-DASLab (low-precision training)

• ​Will Feng, Helion team, PyTorch

• ​Matej Sirovatka, Prime Intellect (TBA)

• ​Paul Chang, Verda (TBA)

• ​Youssef El Manssouri, Sesterce (TBA)

​Tracks & prizes

​Each of the tracks will have 3 winners, with 1 grand-prize winner.

​Track #1 by Verda: Blackwell Ultra

• ​1st place: 48 hours on GB300 NVL72 + 1,500 EUR in cloud credits from Verda

• ​2nd place: 1,000 EUR in cloud credits from Verda

• ​3rd place: 500 EUR in cloud credits from Verda

​Pretrain an LLM from scratch in a limited time on a Blackwell Ultra Cluster. In this track, you will participate as a team, trying to make optimal use of 360 PFLOP/s (BF16). Will this require asynchronous optimizer steps, large-batch size training, models much larger than chinchilla, or something else entirely?

​Track #2 by Sesterce: TBA soon

​Sponsors

• ​Prime Intellect

• ​Verda

• ​Sesterce

• ​SemiAnalysis