DataFusion Community Showcase Vol. 3

​We're excited to present the third DataFusion Community Showcase!

​This virtual session will highlight two projects building on DataFusion:

​Milind Srivastava, Carnegie Mellon University

​"ASAPQuery: A Drop-in Sketch-based Query Accelerator for Metrics Observability"

​ASAPQuery is a drop-in query accelerator that fits in your existing metrics observability stack like Prometheus-Grafana or Clickhouse-Grafana. ASAPQuery uses sketch-based approximation to reduce query latency by 2 orders of magnitude while simultaneously reducing query cost. Internally, ASAPQuery converts existing PromQL/SQL queries to Datafusion-based plans (with custom nodes) and executes them using sketches.

​

​Yaroslav Tkachenko, Streamling

​"Streamling: Building a Lightweight Data Streaming Runtime on Apache DataFusion"

​Many analytical databases and query engines use DataFusion as a foundation. But can you extend it to build a data streaming framework? In this talk, I'll cover the lessons learned from extending DataFusion to support streaming. We'll look into how well most of the existing components can be leveraged with little or no modifications. We'll also see how modern features, like implementing TypeScript support via WebAssembly, can be integrated with the Arrow format.