Spark Connect: NVIDIA Accelerator for Spark SQL and MLlib

​Please join us 🤝 to learn more about Apache Spark™, Spark Connect, and Spark ML at NVIDIA.

​📅 Date: October 29, 2025
⏰ Time: 9:30 AM - 10:30 AM PST (45min talk, then Q&A)
📍 Location: online (live streaming to LinkedIn, X & YouTube)

​Agenda:

• ​Welcome and Introductions

• ​Talk: GPU Accelerated Apache Spark™ Connect: NVIDIA Accelerator for Spark SQL and MLlib

• ​Q&A

​Talk: GPU Accelerated Apache Spark™ Connect: NVIDIA Accelerator for Spark SQL and MLlib

​Abstract:
Spark Connect, first included in Apache Spark™ 3.4 and recently extended to MLlib in Spark 4.0+, introduced a new way to run Spark applications over a gRPC protocol. This has many benefits, including easier adoption for non-JVM clients, version independence from applications, and increased stability and security of the associated Spark clusters.

​In this talk, we shall demonstrate how the recent Spark Connect extension for ML, together with Spark SQL’s existing plugin interface, can be used with NVIDIA GPU-accelerated open source plugins for ML and SQL to enable no-code change, end-to-end GPU acceleration of Spark applications over Spark Connect, with performance up to 9x at 80% cost reduction.

​We will introduce a working pattern for Spark Connect with accelerated ETL and ML for use in lakehouses. We will discuss how such an architecture can be used in practice and provide a few industry use cases.