Data Foundations & SQL

The data lifecycle and its language. What data engineers do, the lifecycle and modern data stack, set theory behind relations, and SQL from joins to window functions.

• Introduction to Data Engineering & the Data Lifecycle — The plumbing behind every dashboard and AI model. Learn what data engineers do and how they differ from data scientists and analysts, the data lifecycle (generation, ingestion, storage, transformation, serving), batch vs streaming, OLTP vs OLAP, structured/semi-structured/unstructured data, the modern data stack (sources, ingestion tools, lakes and warehouses, transformation, orchestration, BI and ML), data volume/velocity/variety, and sizing a platform — with Python estimating ingest volume, storage and throughput.
• Sets, Relations & Functions — The language of collections and mappings. Learn set notation, operations (union, intersection, difference, complement), Venn diagrams, power sets and Cartesian products, relations and their properties (reflexive, symmetric, transitive), equivalence relations and partial orders, and injective, surjective and bijective functions.
• SQL for Data Engineering: Joins, Aggregations & Window Functions — The universal language of data. Learn SELECT, WHERE, ORDER BY, LIMIT; joins (INNER, LEFT, RIGHT, FULL, CROSS, self-joins) and join pitfalls (duplicates, NULLs); GROUP BY with aggregates and HAVING; subqueries and CTEs; window functions (ROW_NUMBER, RANK, running totals, LAG/LEAD, moving averages); CASE expressions; NULL semantics; logical query order; deduplication and upserts (MERGE); indexes and query plans — with runnable SQL on SQLite computing revenue by city, running totals and ranks.