DataFusion Community Showcase Vol. 5

​We're excited to present the fifth DataFusion Community Showcase!

​This virtual session will highlight two projects building on DataFusion:

​1) Jayendra Parmar, Zarr DataFusion

​"Querying Scientific Data with a Relational Model"

​Zarr DataFusion is a Rust library that integrates Zarr (v2 and v3) array storage with Apache DataFusion for querying multidimensional scientific data using SQL.

The library allows you to query Zarr stores (commonly used for weather, climate, and scientific datasets) using DataFusion's SQL engine. It flattens multidimensional arrays into a tabular format, enabling SQL queries over gridded data

This talk reviews how Zarr DataFusion applies the relational model to large scale scientific data.

​2) Timothy Saucer, Rerun.io

​"Building a Data Layer for Physical AI"

​Rerun.io is building an opinionated data layer for physical AI. We use DataFusion heavily in multiple aspects for data storage, indexing, querying, and visualizing. I will demonstrate how queries written with datafusion-python can find interesting robotics cases at scale and transition this directly to our visualization system.

​If you are interested in presenting at a future Community Showcase, please sign up here. And for more info, check out the DataFusion repo.