The Agentic Shift in Data Engineering: From Pipeline Builders to Architects of Controlled Chaos
AI agents are changing data engineering on two fronts at once.
In development, they compress the engineering loop: helping write SQL, generate transformations, debug pipelines, create tests, document systems, and reason about legacy codebases.
In production, they become active participants in the data ecosystem: consuming context, calling tools, triggering workflows, creating side effects, and introducing new forms of operational chaos.
This changes the role of the data engineer. It is no longer enough to build pipelines that move data from A to B. Data engineers now need to design the control layer for agentic systems: trusted context, data contracts, permissions, observability, audit trails, and closed feedback loops.
The future data engineer is not just a pipeline builder. They are the architect of controlled autonomy.
About the Speaker:
Nik Bakanchev is a hands-on Data Engineer and Platform Architect specializing in enterprise-scale data infrastructure. His current focus lies at the intersection of data engineering and AI - specifically, designing, observing, and securing autonomous agentic workflows in production. He joins us today as an independent industry expert to share practical patterns and field observations on the future of data platforms.
DataTalks.Club is the place to talk about data. Join our Slack community!