Book Image

Data Cleaning with Power BI

By : Gus Frazer
Book Image

Data Cleaning with Power BI

By: Gus Frazer

Overview of this book

Microsoft Power BI offers a range of powerful data cleaning and preparation options through tools such as DAX, Power Query, and the M language. However, despite its user-friendly interface, mastering it can be challenging. Whether you're a seasoned analyst or a novice exploring the potential of Power BI, this comprehensive guide equips you with techniques to transform raw data into a reliable foundation for insightful analysis and visualization. This book serves as a comprehensive guide to data cleaning, starting with data quality, common data challenges, and best practices for handling data. You’ll learn how to import and clean data with Query Editor and transform data using the M query language. As you advance, you’ll explore Power BI’s data modeling capabilities for efficient cleaning and establishing relationships. Later chapters cover best practices for using Power Automate for data cleaning and task automation. Finally, you’ll discover how OpenAI and ChatGPT can make data cleaning in Power BI easier. By the end of the book, you will have a comprehensive understanding of data cleaning concepts, techniques, and how to use Power BI and its tools for effective data preparation.

Related Content you might be interested in

Current Title:

Small book image
Data Cleaning with Power BI
Gus Frazer
Feb, 2024 | 340 pages
Small book image
Power Query Cookbook
Andrea Janicijevic
Oct, 2021 | 412 pages
Small book image
The Definitive Guide to Power Query (M)
Rick de Groot | Greg Deckler | Melissa de Korte
Mar, 2024 | 758 pages
Small book image
Learn Power Query.
Warren Sparrow | Linda Foulkes
Jul, 2020 | 428 pages
Table of Contents (23 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Conventions used
Get in touch
Share Your Thoughts
Download a free PDF copy of this book
Free Chapter
1
Part 1 – Introduction and Fundamentals
Part 1 – Introduction and Fundamentals
2
Chapter 1: Introduction to Power BI Data Cleaning
Chapter 1: Introduction to Power BI Data Cleaning
Technical requirements
Cleaning your data in Power BI
Understanding Power Query
Understanding DAX
Where do we begin with data?
Summary
Questions
3
Chapter 2: Understanding Data Quality and Why Data Cleaning is Important
Chapter 2: Understanding Data Quality and Why Data Cleaning is Important
What is data quality?
Where do data quality issues come from?
The role of data cleaning in improving data quality
Best practices for data quality overall
Summary
Questions
4
Chapter 3: Data Cleaning Fundamentals and Principles
Chapter 3: Data Cleaning Fundamentals and Principles
Defining data cleaning
Who’s responsible for cleaning data?
Building a process for cleaning data
Understanding quality over quantity in data cleaning
Summary
Questions
5
Chapter 4: The Most Common Data Cleaning Operations
Chapter 4: The Most Common Data Cleaning Operations
Technical requirements
Removing duplicates
Removing missing data
Splitting columns
Merging columns
Replacing values
Creating calculated columns versus measures
Summary
Questions
6
Part 2 – Data Import and Query Editor
Part 2 – Data Import and Query Editor
7
Chapter 5: Importing Data into Power BI
Chapter 5: Importing Data into Power BI
Technical requirements
Understanding data completeness
Understanding data accuracy
Understanding data consistency
Assessing data relevance
Assessing data formatting
Assessing data normalization, denormalization, and star schemas
Summary
Questions
8
Chapter 6: Cleaning Data with Query Editor
Chapter 6: Cleaning Data with Query Editor
Technical requirements
Understanding the Query Editor interface
Data cleaning techniques and functions
Using Query Editor versus DAX for transformation
Summary
Questions
Further reading
9
Chapter 7: Transforming Data with the M Language
Chapter 7: Transforming Data with the M Language
Technical requirements
Understanding the M language
Filtering and sorting data with M
Transforming data with M
Working with data sources in M
Summary
Questions
10
Chapter 8: Using Data Profiling for Exploratory Data Analysis (EDA)
Chapter 8: Using Data Profiling for Exploratory Data Analysis (EDA)
Understanding EDA
Exploring data profiling features in Power BI
Reviewing column quality, distribution, and profile
Turning data profiles into high-quality data
Summary
Questions
11
Part 3 – Advanced Data Cleaning and Optimizations
Part 3 – Advanced Data Cleaning and Optimizations
12
Chapter 9: Advanced Data Cleaning Techniques
Chapter 9: Advanced Data Cleaning Techniques
Technical requirements
Using Power Query Editor from within Dataflow Gen1 – fuzzy matching and fill down
Using R and Python scripts
Using ML to clean data
Summary
Questions
13
Chapter 10: Creating Custom Functions in Power Query
Chapter 10: Creating Custom Functions in Power Query
Planning for your custom function
Using parameters
Creating custom functions
Summary
Questions
14
Chapter 11: M Query Optimization
Chapter 11: M Query Optimization
Technical requirements
Creating custom functions
Filtering and reducing data
Using native M functions
Optimizing memory usage
Parallel query execution
Summary
Questions
Further reading
15
Chapter 12: Data Modeling and Managing Relationships
Chapter 12: Data Modeling and Managing Relationships
Understanding the basics of data modeling
Using bidirectional cross-filtering
Understanding what’s the right cardinality
Handling large and complex datasets
Avoiding circular references
Summary
Questions
Further reading
16
Part 4 – Paginated Reports, Automations, and OpenAI
Part 4 – Paginated Reports, Automations, and OpenAI
17
Chapter 13: Preparing Data for Paginated Reporting
Chapter 13: Preparing Data for Paginated Reporting
Technical requirements
Understanding the importance of paginated reports
Connecting to data sources within Power BI Report Builder
Data preparation
Using filters and parameters
Using row groups/column groups
Summary
Questions
18
Chapter 14: Automating Data Cleaning Tasks with Power Automate
Chapter 14: Automating Data Cleaning Tasks with Power Automate
Technical requirements
Handling triggers for automation
Automating notifications
Automating refreshing of data
Best practices with Power Automate
Summary
Questions
Further reading
19
Chapter 15: Making Life Easier with OpenAI
Chapter 15: Making Life Easier with OpenAI
Optimizing efficiency with OpenAI, ChatGPT, and DAX
Using OpenAI for M queries
Using Microsoft Copilot
Tackling challenges with AI
Summary
Questions
Further reading
Putting it together
20
Assessments
Chapter 1 – Introduction to Power BI Data Cleaning
Chapter 2 – Understanding Data Quality and Why Data Cleaning is Important
Chapter 3 – Data Cleaning Fundamentals and Principles
Chapter 4 – The Most Common Data Cleaning Operations
Chapter 5 – Importing Data into Power BI
Chapter 6 – Cleaning Data with Query Editor
Chapter 7 – Transforming Data with the M Language
Chapter 8 – Using Data Profiling for Exploratory Data Analysis (EDA)
Chapter 9 – Advanced Data Cleaning Techniques
Chapter 10 – Creating Custom Functions in Power Query
Chapter 11 – M Query Optimization
Chapter 12 – Data Modeling and Managing Relationships
Chapter 13 – Preparing Data for Paginated Reporting
Chapter 14 – Automating Data Cleaning Tasks with Power Automate
Chapter 15 – Making Life Easier with OpenAI
21
Index
Index
Why subscribe?
22
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts
Download a free PDF copy of this book

Removing duplicates

In many cases, as we start working with data, there will often be duplicates within the data. As we discussed in Chapter 2, Understanding Data Quality and Why Data Cleaning is Important, there are a number of reasons why the values in your data may have been duplicated. For example, say we're a retailer and we accidentally entered two product items for the same product. We don’t want to have inaccurate numbers for that product by leaving the duplicate data in, so it’s key that we remove it before we get started with our analysis.

So, let’s get started. In the following example, we will find, select, and remove the duplicate in the data:

  1. Download the Products.xlsx dataset from the given GitHub repository.
  2. Connect to this CSV using Power BI Desktop by selecting Get data in the toolbar (as shown) and then selecting Excel workbook:
Figure 4.1 – The Get data menu within Power BI Desktop

Figure 4.1 – The Get data menu within Power BI Desktop

...
Previous Section
End of Section 3
Next Section