DataFusion Community Showcase Vol. 7: Bauplan & Phoebe Genomics

​We're excited to present the seventh DataFusion Community Showcase!

​This virtual session will highlight two projects building on DataFusion:

​1) Jacopo Tagliabue, Bauplan

​"DF makes plans and AI laughs: LLM-based query optimization"

​We explore how large language models can improve DataFusion’s query plans through targeted edits and evolutionary search, building a custom plan serializer and a new evaluation harness for correctness and performance. On workloads modeled on TPC-H and TPC-DS, we achieve speedups of up to 4.78× over baseline plans on individual queries, with selected improvements transferring to larger datasets.

​2) Tim Poterba, Phoebe Genomics

​"Phoebe: a domain-optimized database built on DAVE (Datafusion, Arrow, Vortex, Embedded)"

​Analytical databases optimized for weird domain data and weird domain queries outperform general solutions. Domain-optimized databases used to be huge projects, but with Datafusion Arrow / Vortex, you can spend all your time on the domain optimizations extending these fantastic general storage/computequery engines.

​If you are interested in presenting at a future Community Showcase, please sign up here. And for more info, check out the DataFusion repo.