Data Engineering Bootcamp
A 16-week, live, project-led course that takes you from messy raw data sources to reliable production pipelines.
What you will be able to do
Build and deploy a scheduled, monitored pipeline that loads clean tables into a warehouse
Judge whether a schema or DAG design is sound, not just whether a job runs
Defend pipeline design reasoning in a design review
Produce a documented end-to-end capstone pipeline to walk through with an interviewer
Who it is for
- Can write basic code in any language, or will put in extra hours on Module 01
- Knows basic SQL, or can learn it alongside Python in the first two weeks
- Can commit about 8 hours a week for 16 weeks, including live sessions
- Wants to understand what happens inside a pipeline, not just wire up managed services
Curriculum
12 modules, 128 hours
- STR-01Foundations of Data EngineeringData pipelines · OLTP vs OLAP · Batch vs streaming · Data team roles · Tooling overview · Environment setup8 h
- STR-02SQL for Data EngineeringWindow functions · CTEs · Query optimization · Partitioning · Indexes · Database design patterns8 h
- STR-03Python for Data Pipelinespandas · Pydantic · API ingestion · File I/O · Error handling · Pipeline design patterns8 h
- STR-04Apache Spark & PySparkDataFrames · Lazy evaluation · Partitioning · Joins · Performance tuning · UDFs on Databricks16 h
- STR-05Data Ingestion & SourcesREST APIs · JDBC · Change data capture with Debezium · Parquet, Avro and JSON formats · Schema registry8 h
- STR-06Cloud Storage & Data LakeAWS S3 · Lake design and partitioning · AWS Glue · Athena · IAM basics · Cost management8 h
- STR-07Lakehouse with Delta LakeMedallion architecture · ACID transactions · Schema evolution · Time travel · Merge operations12 h
- STR-08Transformation with dbtModels and sources · Tests · Jinja macros · Incremental models · dbt Cloud with Snowflake12 h
- STR-09Pipeline OrchestrationDAGs and dependencies · Triggers and retries · Sensors · XCom · Airflow vs Dagster8 h
- STR-10Streaming Data PipelinesKafka producers and consumers · Consumer groups · Spark Structured Streaming · Windowing12 h
- STR-11Data Quality & ObservabilityData contracts · Great Expectations · Lineage · Alerting and SLAs · Observability concepts8 h
- STR-12Capstone: End-to-End ProjectIngestion to lakehouse · dbt transformations · Airflow orchestration · BI layer · Weekly mentor review20 h