Workshop: dlt Go-to-production

​What to expect:

• ​In this 2-hour workshop, we'll dive into transferring a pipeline from local development to production and solving various use cases with dlt.

  • ​Learn to create sources using a REST API client.

  • ​Reverse ETL. Implement custom destinations for data pipelines.

  • ​Inspecting loaded packages and tracing.

  • ​Utilize last_value_func, lookback, and backfill strategies for custom incremental loading.

  • ​Schema configuration. Set data typing, autodetection, and establish data contracts.

  • ​Performance optimization. Explore parallelization, memory management, and separate control over extraction, normalization, and loading. Speed up processes with different file loader formats and chunking.

  • ​Configure logging with Sentry, manage logging levels, and customize retries using a requests wrapper.

  • ​Understand the importance of staging and how to use it effectively.

  • ​Deploy pipelines using Lambda, Airflow, and Dagster.

  • ​dbt-runner. Integrate and run dbt models within your dlt workflows.

• ​Live demonstrations:

  • ​Watch live demos showcasing advanced dlt use cases and deployment scenarios.

• ​Practical exercises:

  • ​Participate in guided exercises to practice advanced dlt functionalities, including performance optimization and deployment strategies.

  • ​Homework assignment that will be reviewed and feedback provided.

• ​Q&A session:

  • ​Each workshop includes a separate channel in our Slack Community for support and close communication.

  • ​Get your questions answered by our dlt experts and share insights with fellow participants.

​Who should attend:

• ​Data professionals with prior experience in dlt, including Data Engineers, Data Analysts and Python Developers looking to advance their skills in data pipeline deployment and management.

​Event details:

• ​Location: Online (Google Meeting link will be provided later)

• ​Duration: 2 hours