Beyond Data Ingestion: What Data Engineering Design Patterns for Open Lakehouse
Every lakehouse project starts with data ingestion. However, true operational maturity, scale, and maintainability are achieved only by moving beyond ingestion and adopting structured architectural solutions to complex post-ingestion problems.
In this session we'll be going through different data engineering design patterns you can apply to your workloads to make your engineering life easier and your consumers happier!
Speaker
Bartosz Konieczny, Freelance Data Engineer, waitingforcode.com
Bio: Bartosz Konieczny is a freelance Data Engineer and author with 15 years of experience architecting enterprise-scale data ecosystems. Specializing in the modern data stack, including Apache Spark, Databricks, Delta Lake, and Apache Kafka, he helps organizations navigate complex challenges in data ingestion, sessionization, and real-time processing across AWS and GCP.
Bartosz is the author of Data Engineering Design Patterns at O'Reilly, and the the voice behind waitingforcode.com. A regular speaker, he bridges the gap between complex open source internals and practical, production-ready engineering.
Host
Robert Pack, Staff Developer Advocate, Databricks
Bio: Robert Pack is a data and AI systems expert with foundations in process systems engineering and numerical optimization, holding degrees from RWTH Aachen University and Imperial College London. He began his career in process R&D within a multinational chemical company, where he designed and led transformative cloud and data initiatives that shaped the organization's digital landscape. As Chief Technology Lead, he was responsible for global data and AI platform engineering.
Robert’s open-source journey began with deep contributions to Delta Lake and Delta-RS and later expanded into the broader Rust and Apache Arrow data ecosystems. Today, as part of Databricks, he serves as a Developer Advocate dedicated to fostering a modern, high-performance data ecosystem for all. Through his work on projects such as Delta Kernel, Delta Lake, Unity Catalog, and Apache Iceberg, Robert remains committed to advancing the Open Lakehouse vision and facilitating the next generation of data infrastructure.