Matthew Roche

Senior Program Manager

Microsoft
@SQLAllFather
https://ssbipolar.com

Introducing Power BI dataflows

Please Thank & Support our Sponsors
SQL Saturday is made possible with the generous support of these sponsors.
You can support them by opting-in and visiting them in the sponsor area.

Global Alliance Partner: Venue & Internet Sponsor: Silver Sponsor:

Operations Support:
Endless Support from:

OGMA Consulting Corp.

Bronze Sponsor:
Introducing Power BI dataflows,
Three Ways
 Power BI dataflows are part of the evolution of BI

Self-Service BI

OLTP systems Data Warehouse Reports and dashboards

Data Lake
Data Preparation / ETL OLAP / Analytics Models
 Power BI dataflows are another object/artifact type

Reports & dashboards

Datasets

Dataflows

Azure Data Lake Storage Gen2

CDM folder CDM folder CDM folder

Business analysts
Low/no code
 Power BI dataflows are like Excel
Sources

20
Ingest from Clean and
Dynamics Sales 8 enrich sales
CRM – Production data
Dynamics 365 10
22 entities 10 entities Final Business
View
4 1
1
11 entities
IoT Signal Product Add Telemetry
Azure Data Lake Storage Telemetry in 4 Customer
1 Product Usage
Azure Attributes
Dataset
5 entities 6 entities

Sales/Telemetry
Reference Data
External Dataflow
Demo: Power BI dataflows
end to end
 Self-Service BI

OLTP systems Data Warehouse Reports and dashboards

Data Lake
Data Preparation / ETL OLAP / Analytics Models
Power BI dataflows sort-of-FAQ

 A new capability for self-service data  NOT a replacement for datasets

preparation in Power BI
 NOT a replacement for a data
 Delivered in a familiar Power Query
experience warehouse
 Built on the foundation of Azure Data Lake  NOT a replacement for Azure Data
Storage gen2 Factory or SSIS
 Utilize the CDM folder format for data
 NOT a Premium-only feature
storage
 A tool for business users to drive data reuse  NOT an additional cost or fee
without requiring IT involvement
 NOT spelled with a space or any
 Enable Excel-like data lineage and
capital letters
orchestration
Integrating Power BI dataflows
with
Azure Data Lake Storage gen2
 Using an organizational ADLSg2 resource with Power BI

Reports & dashboards

Datasets

Azure Data Azure Azure SQL Azure ML

Dataflows Factory Databricks DW

Azure Data Lake Storage Gen2

CDM folder CDM folder CDM folder

Data scientists
Business analysts
Data engineers
Low/no code
Low to high code
 BYOSA E2E – 3 steps
What “done” means:
Workspace Admins can now configure
Create and configure Attach storage Enable people workspace to store its dataflows in ADLS
account to Power BI to use it Done
Storage account account Global Admin configured

Global Admin

In Power BI Dataflow CDM folder is stored in ADLS

and is accessible by its creator ONLY.
Assign workspace Create and
Create v2 workspace Done
to ADLS account refresh dataflow
Workspace Admins Other people in workspace can get data if
they are authorized to the CDM folder
outside of Power BI

Find dataflow
Attach to CDM folder Get authorized to
CDM folder Done
from “Azure Data services" storage location Now data can be read from CDM Folder
storage location
via Azure Data Services or LOB solutions
Developers
and
In Azure that are CDM folder aware

Data scientists
Bring your own storage account
• Use your organizational ADLSg2 resource and attach it to Power BI
• A Global Administrator account and an Azure subscription is required
• Once a dataflow storage location is configured, it cannot be changed
• By default, only owners of a dataflow stored in Azure Data Lake Storage
Gen2 can access its data
• To authorize additional users to the dataflows stored in Azure, you must
add them to the dataflow’s CDM folder
• Creating dataflows with linked entities is only possible when they are
stored in the same storage account
Deliver ready-made insights to Power BI users from Azure

Self service
customizations
Dynamics 365
in Power BI

Azure Data Azure Azure SQL Azure ML

Dataflow Dynamics 365 data Factory Databricks DW

Azure Data Lake Storage Gen2

CDM folder
Demo: Azure Integration,
Two Ways
Wrapping Things Up
Positioning Power BI dataflows
• Dataflows are for Power Query users
• Easy to build reusable data entities
• Easier to compose and orchestrate
• Multi-stage, multi-user data prep workflows
• Dataflows fill a self-service gap in the end-to-end story
• Without dataflows, users will
• Be blocked on IT involvement
• Use Excel and manual processes
• Require 3rd party data preparation tools like Alteryx, Datameer, Trifacta, etc.
• Dataflows bridge Power BI and Azure
• CDM folders and BYOSA enable simple collaboration between business and IT
• CDM and CDM folders are strategic technologies beyond Power BI
Production Customer Scenario
Sources Dataflows Datasets
Metric Base Tables
Ingest

PostgreSQL

Link

Ingest Metric

Spark (ODBC) Final calculated

cleansed data
Workspace
Link
Metric – specific
business line
Metric – Metric – Metric –
specific specific specific Filtered to the specific
business line business line business line line of business

Filtered to
Workspace Workspace Workspace Workspace
Session resources
Dataflows on public Microsoft sites:
• Dataflows documentation: https://docs.microsoft.com/en-us/power-bi/service-dataflows-overview

• Dataflows roadmap / release notes:

https://docs.microsoft.com/en-us/business-applications-release-notes/April19/business-intelligence/power-bi-service/power-bi-dataflows/self-service-data-
prep-with-dataflows

• Dataflows on Power BI Ideas: https://ideas.powerbi.com/forums/265200-power-bi-ideas?category_id=341638

• Dataflows on Power BI community forum: link

Common Data Model on public Microsoft sites: :

• Common Data Model documentation: https://docs.microsoft.com/en-us/common-data-model/

• CDM Folder model metadata: https://docs.microsoft.com/en-us/common-data-model/model-json

• Common Data Model on GitHub: https://github.com/Microsoft/CDM

• End to end CDM partner sample: https://aka.ms/cdmadsblog

Matthew’s blog:
• Dataflows landing page: https://ssbipolar.com/2018/10/23/dataflows-in-power-bi/

• Dataflows FAQ: https://ssbipolar.com/2018/11/27/power-bi-dataflows-faq/

Thank you!