DP-601T00 - Implementing a Lakehouse with Microsoft Fabric
Course Description
This course is designed to build your foundational skills in data engineering on Microsoft Fabric, focusing on the Lakehouse concept. This course will explore the powerful capabilities of Apache Spark for distributed data processing and the essential techniques for efficient data management, versioning and reliability by working with Delta Lake tables. This course will also explore data ingestion and orchestration using Dataflows Gen2 and Data Factory pipelines. This course includes a combination of lectures and hands-on exercises that will prepare you to work with lakehouses in Microsoft Fabric.
Duration: 1 days
Who should attend
The primary audience for this course is data professionals who are familiar with data modeling, extraction and analytics. It is designed for professionals who are interested in gaining knowledge about Lakehouse architecture, the Microsoft Fabric platform and how to enable end-to-end analytics using these technologies.
Module 1 - Introduction to end-to-end analytics using Microsoft Fabric
Microsoft Fabric components and workloads OneLake and unified data storage Data engineer, analyst and scientist responsibilities Fabric capacities, workspaces and items Typical end-to-end analytics workflow Instructor demonstration: Navigate a Fabric workspace and show how lakehouses, notebooks, pipelines, semantic models and reports relate to one another.
Module 2 - Get started with lakehouses in Microsoft Fabric
Lakehouse concepts and use cases Files area versus managed tables Delta tables and the SQL analytics endpoint Data ingestion options Querying lakehouse data with T-SQL Guided lab: Create a workspace and lakehouse, ingest a sample dataset, create a Delta table, explore it in Lakehouse Explorer and query it through the SQL analytics endpoint.
Module 3 - Use Apache Spark in Microsoft Fabric
Spark pools and session configuration Fabric notebooks Loading data into a DataFrame Selecting, filtering, grouping and transforming data Spark SQL and temporary views Notebook visualisations Choosing between Spark and SQL Guided lab: Use a Fabric notebook to clean and analyse raw data, create a temporary SQL view and visualise a summary result.
Module 4 - Work with Delta Lake tables in Microsoft Fabric
Delta Lake transaction architecture Managed and external tables Creating and writing Delta tables Append, overwrite and merge patterns Table optimisation and maintenance Table history and versioning Structured streaming concepts Guided lab: Create a Delta table, load incremental data, perform an upsert and inspect the table transaction history.
Module 5 - Ingest data with Dataflows Gen2 in Microsoft Fabric
Dataflows Gen2 architecture Power Query transformations Sources, credentials and destinations Data type and data-quality handling Loading data into a lakehouse Refresh and pipeline integration Guided lab: Create a Dataflow Gen2 that imports, cleans and loads source data into a lakehouse table.
Module 6 - Orchestrate processes and data movement with Microsoft Fabric
Fabric Data Factory pipelines Copy Data assistant and Copy Data activity Notebook and Dataflow activities Parameters and dynamic expressions Dependencies and failure paths Scheduling, running and monitoring pipelines Guided lab: Build a pipeline that copies source data, runs the Dataflow Gen2 transformation and monitors the completed execution.
Module 7 - Organize a Fabric lakehouse using medallion architecture design
Purpose of Bronze, Silver and Gold layers Raw-data preservation Cleansing, deduplication and conformance Business-ready and dimensional data Partitioning and performance Security and governance considerations Serving data to Power BI Capstone activity: Design and, where time permits, implement a solution that ingests raw data into a Bronze layer, transforms it into cleaned Silver Delta tables and produces a business-ready Gold table for the SQL endpoint or Power BI.