Data Pipelines, Orchestration & Streaming
Not Started0 / 10 complete
Open a video or resource link, watch it, then check its box to track your progress and unlock what comes next.
10 lessons3 levels1 capstone project
Build reliable batch and streaming data flows with orchestration, scheduling, event streams, retries, idempotency, backfills, observability and data-contract thinking.
BASIC
- Data Workflow Orchestration — Mastering Apache Airflow (Long)
Covers DAGs, workflow logic and orchestration patterns. Shows how - Realtime Logs Processing with Airflow and Kafka (Long)
orchestration and streaming fit together in a production-style pipeline. - End-to-End Data Engineering Project — Realtime Data Streaming (Long)
Builds a real-time pipeline using Airflow, Kafka and Python.
INTERMEDIATE
- Full Walkthrough — GCP, Kafka, PySpark, Airflow
A deeper treatment of the same skill; use it to move from concept to working patterns. - Star Schema / Dimensional Modeling
Apply Data Pipelines, Orchestration & Streaming through the adjacent capability: Data Engineering. - Data Engineer Roadmap 2026 — Codebasics
Apply Data Pipelines, Orchestration & Streaming through the adjacent capability: Data Engineering.
ADVANCED
- End-to-End Databricks Data Engineering Project
Extend Data Pipelines, Orchestration & Streaming into senior/production practice through Data Engineering. - Quality, Lineage, Privacy and Trust — Data Governance
Extend Data Pipelines, Orchestration & Streaming into senior/production practice through Data Quality, Governance & Lineage. - Databricks ? How to Build a Cloud Data Platform Modern Data Stack references Modern Data Stack references
Extend Data Pipelines, Orchestration & Streaming into senior/production practice through Data Engineering.