Mihai Budiu on Automatic Incremental View Maintenance
Main: Mihai Budiu on DBSP: Automatic Incremental View Maintenance for Rich Query Languages
Abstract: Incremental computations repeatedly evaluate a function on some input values that are changing. The goal of an efficient implementation is to reuse previously computed results: when presented with a new change to the input, an incremental computation should only perform work proportional to the size of the change, rather than to the size of the entire dataset. In databases, incremental computation is known as Incremental View Maintenance (IVM), and has long been a central problem of database theory and practice.
We describe a set of simple ideas which combined solve the IVM problem for arbitrary queries (including recursive queries and nearly all SQL queries):
representing changes as a first-class object
treating all computations as (stateful) stream computations
a trivial algorithm for converting any standard stream computation into an incremental computation
This work has received the 2023 VLDB best paper award, and the 2024 ACM SIGMOD research highlights award.
This works! These ideas are not just a pretty theory: they are very practical. Feldera is a startup which has built an incremental query engine which maintains incrementally arbitrary collections of views described in SQL; the incremental maintenance produces many orders of magnitude reduction in query latency and computational resource usage compared with traditional batch SQL query engines.
Bio: Mihai Budiu is chief scientist at Feldera, an early-stage startup, and was previously employed at VMware Research, Barefoot Networks, and Microsoft Research. He has a Ph.D. in computer science from Carnegie Mellon University. Five of his papers have received "test of time" awards!
Mini: Vitali Lovich on SLOG: Serializable, Low-latency, Geo-replicated Transactions
Abstract: For decades, applications deployed on a world-wide scale have been forced to give up at least one of (1) strict serializability (2) low latency writes (3) high transactional throughput. In this paper we discuss SLOG: a system that avoids this tradeoff for workloads which contain physical region locality in data access. SLOG achieves high-throughput, strictly serializable ACID transactions at geo-replicated distance and scale for all transactions submitted across the world, all the while achieving low latency for transactions that initiate from a location close to the home region for data they access.
Bio: Vitali’s passion is all things compute architecture and engineering. From embedded to distributed systems and from mobile to server compute, he’s worked for 20 years across companies like Apple, Google, Meta, and Cloudflare. He specializes in applying deep research to 0-1 products, serving as technical lead of Oculus Link, Cloudflare's R2 storage, and now leading strategic projects at Archer Aviation.