SF Systems: The Future of Data

​The field of data management is being reshaped to fit fundamentally new workloads through autonomous engineering at scale. The SF Systems Club is back with a pair of exciting talks exploring what is possible in this new era!

​This meetup is generously hosted by Recall at their San Francisco office. Food and drinks will be provided!

​Agenda

• ​5:30 PM: Doors Open, Food and Socializing

• ​6:30 - 7:30: Introductions and Talks

• ​7:30-: Community Socializing!

​Talks

​Robert Escriva (Chroma) - Tally: An agents-only version control system in which patches are morphisms in a groupoid

​This talk introduces Tally, a project seeking to replace git with a version control system that's agents-only.  Once you adopt that world-view, the natural thing to do is speak in patches.  Tally encodes the data structure of a patch twice:  Once as content suitable for applying the patch, and once as a checksum that encodes the behavior of a patch.  A commit is encoded likewise.  The payoff comes when a commit plus a patch equals a commit. This talk walks through Tally's motivation and a worked example.

​Michael Malis (pgrust) - How we rewrote Postgres in Rust and made it 900x faster for analytics

​Postgres is the default database for applications today, but analytics on it is slow enough that many teams copy their data into a separate system, and running it means hand-tuning memory and hoping one runaway query doesn't get the whole server OOM-killed. We made pgrust, Postgres rewritten in Rust, to fix these problems and more, aiming to be a drop-in replacement for Postgres. We'll talk about how we did the rewrite and how we check that it behaves exactly like Postgres, including a compiler we built that translates our code to Lean so we can formally verify the C and Rust implementations behave the same, and deterministic simulation testing. Then we'll cover some of what makes it fast and easier to run: morsel-driven parallelism, our own columnar format and JIT compiler, a memory governor that decides which queries and caches get more memory and which have to give it back, and more.