Overview
This course is designed to build your foundational skills in data engineering on Microsoft Fabric, focusing on the Lakehouse concept. This course will explore the powerful capabilities of Apache Spark for distributed data processing and the essential techniques for efficient data management, versioning, and reliability by working with Delta Lake tables. This course will also explore data ingestion and orchestration using Dataflows Gen2 and Data Factory pipelines. This course includes a combination of lectures and hands-on exercises that will prepare you to work with lakehouses in Microsoft Fabric.
Audience Profile
The primary audience for this course is data professionals who are familiar with data modeling, extraction, and analytics. It is designed for professionals who are interested in gaining knowledge about Lakehouse architecture, the Microsoft Fabric platform, and how to enable end-to-end analytics using these technologies.
Job role: Data Analyst, Data Engineer, Data Scientist
Prerequisites
You should be familiar with basic data concepts and terminology
Outline
Introduction to end-to-end analytics using Microsoft Fabric
Discover how Microsoft Fabric can meet your enterprise's analytics needs in one platform. Learn about Microsoft Fabric, how it works, and identify how you can use it for your analytics needs
- Introduction
- Explore end-to-end analytics with Microsoft Fabric
- Data teams and Microsoft Fabric
- Enable and use Microsoft Fabric
- Knowledge Check
- Summary
Get started with lakehouses in Microsoft Fabric
Lakehouses merge data lake storage flexibility with data warehouse analytics. Microsoft Fabric offers a lakehouse solution for comprehensive analytics on a single SaaS platform.
- Introduction
- Explore the Microsoft Fabric Lakehouse
- Work with Microsoft Fabric Lakehouses
- Exercise - Create and ingest data with a Microsoft Fabric Lakehouse
- Knowledge check
- Summary
Use Apache Spark in Microsoft Fabric
Apache Spark is a core technology for large-scale data analytics. Microsoft Fabric provides support for Spark clusters, enabling you to analyze and process data in a Lakehouse at scale.
- Introduction
- Prepare to use Apache Spark
- Run Spark code
- Work with data in a Spark dataframe
- Work with data using Spark SQL
- Visualize data in a Spark notebook
- Exercise - Analyze data with Apache Spark
- Knowledge check
- Summary
Work with Delta Lake tables in Microsoft Fabric
Tables in a Microsoft Fabric lakehouse are based on the Delta Lake storage format commonly used in Apache Spark. By using the enhanced capabilities of delta tables, you can create advanced analytics solutions.
- Introduction
- Understand Delta Lake
- Create delta tables
- Work with delta tables in Spark
- Use delta tables with streaming data
- Exercise - Use delta tables in Apache Spark
- Knowledge check
- Summary
Ingest Data with Dataflows Gen2 in Microsoft Fabric
Data ingestion is crucial in analytics. Microsoft Fabric's Data Factory offers Dataflows (Gen2) for visually creating multi-step data ingestion and transformation using Power Query Online.
- Introduction
- Understand Dataflows (Gen2) in Microsoft Fabric
- Explore Dataflows (Gen2) in Microsoft Fabric
- Integrate Dataflows (Gen2) and Pipelines in Microsoft Fabric
- Exercise - Create and use a Dataflow (Gen2) in Microsoft Fabric
- Knowledge check
- Summary
Frequently asked questions
See all of our FAQsHow can I create an account on myQA.com?
There are a number of ways to create an account. If you are a self-funder, simply select the "Create account" option on the login page.
If you have been booked onto a course by your company, you will receive a confirmation email. From this email, select "Sign into myQA" and you will be taken to the "Create account" page. Complete all of the details and select "Create account".
If you have the booking number you can also go here and select the "I have a booking number" option. Enter the booking reference and your surname. If the details match, you will be taken to the "Create account" page from where you can enter your details and confirm your account.
Find more answers to frequently asked questions in our FAQs: Bookings & Cancellations page.
How do QA’s virtual classroom courses work?
Our virtual classroom courses allow you to access award-winning classroom training, without leaving your home or office. Our learning professionals are specially trained on how to interact with remote attendees and our remote labs ensure all participants can take part in hands-on exercises wherever they are.
We use the WebEx video conferencing platform by Cisco. Before you book, check that you meet the WebEx system requirements and run a test meeting (more details in the link below) to ensure the software is compatible with your firewall settings. If it doesn’t work, try adjusting your settings or contact your IT department about permitting the website.
Learn more about our Virtual Classrooms.
How do QA’s online courses work?
QA online courses, also commonly known as distance learning courses or elearning courses, take the form of interactive software designed for individual learning, but you will also have access to full support from our subject-matter experts for the duration of your course. When you book a QA online learning course you will receive immediate access to it through our e-learning platform and you can start to learn straight away, from any compatible device. Access to the online learning platform is valid for one year from the booking date.
All courses are built around case studies and presented in an engaging format, which includes storytelling elements, video, audio and humour. Every case study is supported by sample documents and a collection of Knowledge Nuggets that provide more in-depth detail on the wider processes.
Learn more about QA’s online courses.
When will I receive my joining instructions?
Joining instructions for QA courses are sent two weeks prior to the course start date, or immediately if the booking is confirmed within this timeframe. For course bookings made via QA but delivered by a third-party supplier, joining instructions are sent to attendees prior to the training course, but timescales vary depending on each supplier’s terms. Read more FAQs.
When will I receive my certificate?
Certificates of Achievement are issued at the end the course, either as a hard copy or via email. Read more here.