Module 11
Data engineering
Pipelines, warehouse modelling, orchestration and data quality — how raw events become tables a model can trust.
Outcome
What you will be able to do
You can design and schedule an idempotent ELT pipeline with tested data-quality contracts.
Lessons
Work through these in order
- 01Pipelines and idempotencyBatch versus streaming, ELT over ETL, and why every task you write must be safe to run twice. 27 min
- 02Warehouse modellingStar schemas, slowly changing dimensions, surrogate keys, and how to model events so metrics stay comparable. 30 min
- 03Orchestration and schedulingDAGs, dependencies, retries, backfills and alerting — running pipelines on a schedule without babysitting them. 24 min
- 04Data quality and contractsTests on data, not just code: uniqueness, not-null, referential integrity, distribution drift, and how to fail loudly. 22 min
Assessment
Module quiz — 70% to pass
8 questions mixing concept checks and short code-output problems. Graded on the server with per-question explanations afterwards, unlimited retakes, and a badge with a verification code the moment you pass.