Microsoft 365 – Workflow

Automation
- Ken Ding
What is Microsoft 365?

April 21st 2020

“Our cloud productivity offering has grown well beyond what people traditionally think of
as ‘Office.’ […] we’re breathing new life into these apps with the help of the cloud and AI, and
we’re adding new, born-in-the-cloud experiences like Teams, Stream, Forms, and Planner. […] It’s
Office and a whole lot more.”

--- the Microsoft team

 What is Microsoft 365?

With SharePoint Online, users can:

• Build sites, pages, document libraries, and lists

• Add web parts to customize their pages
• Share important visuals, news, and updates with a team
• ……
 What is Microsoft 365?

With Microsoft Teams, you can:

• Communicate through chat, meetings, and

calls
• Collaborate with integrated Microsoft 365 apps
• Connect across devices
• ……
 What is Microsoft 365?

With Microsoft Forms, you can get feedback

effortlessly with surveys, polls, and quizzes
 What is Microsoft 365?

With Microsoft Automate, you can:

• Robotic Process Automation (RPA) => Boost

efficiency
• Automate quickly and more securely
• Enhance workflows with AI
 Agenda

• Use case 1 – Neighborhood Rating Tool with Power BI

• Airbnb analysis
• Toronto Neighborhood Profile Analysis
• Toronto Neighborhood Rating Tool

• Use case 2 – Neighborhood Rating Report with Power Automate and Forms

• Use case 3 – Power Automate Desktop

• Excel automation
• RPA
 Use Case 1:
Neighborhood Rating Tool with Power BI
• Airbnb analysis
 Background of Airbnb in Toronto
• In Sep, stricter
bylaws for
short-term
rentals came
• Airbnb into effect for
was Toronto Airbnb
founded hosts
in 2008
2008 2015 2017 2019 2020
• Airbnb became • The city • Airbnb names
more and more started to Toronto as one
popular in Toronto consider short- of its 10
term biggest
rental(less revenue cities
than 28 days) in the world
regulations
• It was estimated 74% of
revenue would disappear and
7,300 homes returned to
housing market

Source: https://precondo.ca/toronto-airbnb-condos-rules/
 Learning Outcome
• Tool: Power BI
• Data: Airbnb - Inside Airbnb
• Business:
• Analyze Toronto Airbnb market size:
• Average price
• Number of listings
• Number of reviews over time
• How is Airbnb really being used in your neighborhoods?
• Does Airbnb really affect Toronto’s long-term rental supply?
• Availability analysis
• Listings per host

• Data Manipulation and Visualization:

• Fetch data from web source to Power BI
• Clean dirty data
• Handle GeoJSON data and create shape map in Power BI
 Clean the Dirty Data

A data analyst/data scientist usually spends 80% of time in cleaning data

Structure Dirty

Dirty Data

• Wrong
format
• Missing
Content Dirty • Invalid value
• Duplicated
• ……..

Data must be cleaned and preprocessed before doing any analysis!

Clean the Dirty Data

When there is no standard databases (SQL), Power Query Editor in

Power BI is powerful to complete ETL, data cleaning and preprocess

Power Query is a data transformation and data preparation engine

Understand Relationships in Power BI
• Relationships between tables are useful to accurately calculate results and
display the correct information in reports

• 4 types of relationships (Cardinality):

• Many to one (*:1): A many-to-one relationship is the most common, default type of
relationship
• One to one (1:1): a one-to-one relationship; it means both columns contain unique
values
• One to many (1:*): a one-to-many relationship
• Many to many (*:*): a many-to-many relationship between tables; it means both
columns can contain duplicate values
Understand Relationships in Power BI

• Power BI relationships propagate filters applied on the columns of

model tables to other model tables

• Single: single direction

Cross filter
directions
(direction of a
relationship
means the way • Both (bi-directional): both directions, for filtering purposes, both tables are
that filter treated as if they're a single table.
Cardinality type Cross filter options
propagates)
One-to-many (or Many- • Single
to-one) • Both
One-to-one • Both
Many-to-many • Single (Table1 to Table2)
• Single (Table2 to Table1)
• Both
Create Measures in Power BI

• Measures are defined calculations on data

that are performed at the time of your
query
• Simple summarizations such as sums,
averages, minimum, maximum and counts
can be set through the Fields
• Use Data Analysis Expressions (DAX)* to
create your own measure

*DAX includes a library of over 200 functions, operators, and constructs

Create Calculated Columns in Power BI
• Sometimes the data you’re analyzing doesn’t contain a particular field you need
to get the results you’re after. This is where calculated columns come in
• Calculated columns use DAX formulas to define a column’s values
• Calculated columns are similar to measures in that both are based on DAX
formulas, but they differ in how they are used
Practice 1

Create one page dashboard to answer the following questions:

• What is the average listing price per night? Total number of listings in Toronto?
Average listings per Host?
• Line chart showing the count of reviews over time => How was the Airbnb business
affected by Covid-19?
• Average price and number of listings by room type
• What is the overall annual availability rate?
• The majority of hosts has multiple listings or single listing?
• Create a map showing each listing’s location
• Calculate number of listings and average price for each neighborhood

Please refer to Practice 1 note for detailed instruction

 Create Shape Map

In the case, we can use Mapshaper

to convert neighbourhoods.geojson
to TopoJson and add to Power BI as
Custom map
• Create a Shape Map visual to
compare regions on a map using color
• Unlike the Map visual, Shape Map
can't show precise geographical
locations of data points on a map

Map data editor: https://mapshaper.org/

We can use this tool to edit other geojson files
 Use Case 1:
Neighborhood Rating Tool with Power BI
• Airbnb analysis
• Toronto Neighborhood Profile Analysis
Learning Outcomes

• Tool: Power BI
• Data: Toronto Open - https://open.toronto.ca/
• Business – deal with an ambiguous problem:
• Analyze Toronto 140 neighborhoods’ profile
• Rank neighborhood based on different criteria
• Safety (based on crimes)
• Income
• Diversity (based on the percentage of visible minorities)
• Airbnb activities
• Data manipulation and visualization:
• Fetch data from web source to Power BI
• Advanced data cleaning
Practice 2

1. Get the data:

• Neighbourhood Crime Rates: this dataset includes the Crime Data by Neighborhood
• Direct download link: https://ckan0.cf.opendata.inter.prod-
toronto.ca/download_resource/4754baf5-3715-4166-a347-
eda9813521de?format=csv&projection=4326
2. Complete transform in Query Editor to prepare data Neighborhood Crime
Rate

Please refer to Practice 2 note for detailed instruction.

Custom Column vs. Calculated Column vs. Measure

Custom Column Calculated Measure

Column
• Created in Query • Created in Report • Created in Report
Editor view or Data view view or Data view
 Custom Column vs. Calculated Column vs. Measure
• M formula language (Power Query):
Query-Time Transformations to shape
the data while you are extracting it
Creation • DAX: In-Memory Transformations to
happens analyze data after you've extracted it

before data
loaded into Custom Column Calculated Measure
model
Column
Creation
I. Created in Query Editor I. Created in Report I. Created in Report happens
II. Use Power Query (M formula view or Data view view or Data view after data
language) to create II. Use DAX to create II. Use DAX to create loaded into
III. Appears in Fields list without III. Appears in Fields III. Appears in Fields model
special icon list with a special list with a special
icon icon
IV. Calculated for each row IV. Calculated for each row IV. Calculated only when it is used in a
V. Saved before data loaded V. Saved back into the visual
into model model => take up space V. Not saved into the model
• Use a Calculated Column / Custom Column when you want to evaluate each row
• If Custom Column can do the job, use it because Power Query can help ‘prune’ data
• Use a Measure when you need an aggregate
 Deal with Structure Dirty Data
• Most of open datasets are disorganized
Data Source Characteristic City of Toronto Agincourt North Agincourt South-Malvern West Alderwood …..
City of Toronto Neighbourhood Number 129 128 20…..
City of Toronto TSNS2020 Designation No Designation No Designation No Designation …..

Census Profile 98-316-X2016001 Population, 2016 2,731,571 29,113 23,757 12,054…..

Census Profile 98-316-X2016001 Population, 2011 2,615,060 30,279 21,988 11,904…..

Census Profile 98-316-X2016001 Population Change 2011-2016 4.50% -3.90% 8.00% 1.30% …..

Census Profile 98-316-X2016001 Total private dwellings 1,179,057 9,371 8,535 4,732…..

Census Profile 98-316-X2016001 Private dwellings occupied by usual residents 1,112,929 9,120 8,136 4,616…..

Census Profile 98-316-X2016001 Population density per square kilometre 4,334 3,929 3,034 2,435…..

Census Profile 98-316-X2016001 Land area in square kilometres 630.2 7.41 7.83 4.95 …..

Census Profile 98-316-X2016001 Children (0-14 years) 398,135 3,840 3,075 1,760…..

Census Profile 98-316-X2016001 Youth (15-24 years) 340,270 3,705 3,360 1,235…..

Census Profile 98-316-X2016001 Working Age (25-54 years) 1,229,555 11,305 9,965 5,220…..

Census Profile 98-316-X2016001 Pre-retirement (55-64 years) 336,670 4,230 3,265 1,825…..

Census Profile 98-316-X2016001 Seniors (65+ years) 426,945 6,045 4,105 2,015…..

Census Profile 98-316-X2016001 Older Seniors (85+ years) 66,000 925 555 320…..
 Pivot vs. Unpivot vs. Transpose
Reshape data by using Pivot, Unpivot and Transpose:
• Transformation engine in Power Query has many prebuilt functions used for cleaning data

Pivot Column Turn rows to columns

 Pivot vs. Unpivot vs. Transpose
Reshape data by using Pivot, Unpivot and Transpose:
Pivot Column Turn rows to columns

• It can also group each unique value from the

column, do an aggregate calculation for each
value, and pivots the column into a new table
 Pivot vs. Unpivot vs. Transpose
Reshape data by using Pivot, Unpivot and Transpose:

Unpivot Column Turn columns to rows

Power Query will always create the attribute-value pair by using two columns:
• Attribute: The name of the column headings that were unpivoted
• Value: The values that were underneath each of the unpivoted column headings
 Pivot vs. Unpivot vs. Transpose
Reshape data by using Pivot, Unpivot and Transpose:

Transpose Table Rotate table 90 degrees => turning row to columns and columns to
rows

• Only the contents of the table will be transposed during the transpose
operation; the column headers of the initial table will be lost =>
When you need to use Transpose, you should delete any
Promoted Header step
Practice 3

1. Get the data:

• Neighborhood Profiles: people and household profile data based on Statistics Canada
for the 2016 Census
• Direct download link: https://ckan0.cf.opendata.inter.prod-
toronto.ca/download_resource/ef0239b1-832b-4d0b-a1f3-4153e53b189e?format=csv
2. Complete transform in Query Editor to prepare Neighborhood Income
and Neighborhood Diversity

Please refer to Practice 3 note for detailed instruction

Grouping or Summarizing Rows in Power Query

• In Power Query, with Group by:

• You can group by one or more columns and aggregate a column by using an aggregate
function
• Perform a row operation

Similar to
Group By
clause in SQL
Merge Queries in Power Query

• A merge queries operation joins two existing tables together based on matching values from
one or multiple columns
• When you have one or more columns that you’d like to add to another query, you merge the
queries.

Similar to Join
in SQL
Merge Queries in Power Query
• A join kind specifies how a merge operation will be performed:

Join kind Description Icon

Left outer All rows from the left table, matching rows
from the right table
Right outer All rows from the right table, matching rows
from the left table
Full outer All rows from both tables

Inner Only matching rows from both tables

Left anti Only rows from the left table

Right anti Only rows from the right table

Practice 4.1

1. Transform in Query Editor

• Use Group by to create a new query – Neighborhood Airbnb
• Join Neighborhood Income, Neighborhood Diversity, Neighborhood Airbnb, and
Neighborhood Crime Rate all together

Please refer to Practice 4 note for detailed instruction

 Relationships vs. Join Tables
• Having extra relationships, doesn’t help with the performance
Performance • Depends on situations => if data is not huge, you can’t tell the
differences
• Separating dimension and fact tables is a good database design
user-friendliness
• Having a few tables and relationships make the model simple to
understand

Complexity • Relationships come with a cost of complexity => different directions

To avoid:
• Create too complex relationships
• Create a big data including
everything (losing flexibility) • Relationship and Merge Queries (Join) can be used together
 Use Case 1:
Neighborhood Rating Tool with Power BI
• Airbnb analysis
• Toronto Neighborhood Profile Analysis
• Toronto Neighborhood Rating Tool
 Decision Matrix Analysis

Decision Matrix Analysis is a useful technique for making a decision by weighting different
factors based on their importance

• It is very useful when:

▪ You have a number of good alternative to choose from
▪ You have many different factors to consider, but there isn’t a clear or obvious preferred
option
▪ You need to assess a situation from a logical viewpoint
• A decision matrix analysis can not only help you make a decision, but it can also aid you in prioritizing tasks,
problem-solving and crafting arguments to defend a decision you've already made
Example of using Decision Matrix analysis to decide restaurant location:
Build a Rating Tool based on Decision Matrix Technique
Methodology of building a Decision Matrix:
What to do? How to do?

Develop and define metrics/factors

• Market research
• VOC (Voice of clients)
• Brainstorming

Calculate score value for each metric • Collect and analyze data
• Define methodologies to
calculate score based on
analytics

Define Weighting • Brainstorming & Analytics

• Preference-based weighting Toronto Neighborhood Rating Tool
• Survey Toronto Neighborhood Rating Report

• Dashboard design and visualization

Calculate composite score and rank
• Storytelling
Toronto Neighborhood Rating Tool
Based on Decision Matrix analysis, you can build a rating tool to get your top preferences of
neighborhoods:
• Crime rate
Choose criteria • Diversity
• Income
• Airbnb listings

• Each criterion may have different

Calculate score for each criterion methods to calculate the score
• Calculate the score for each
neighborhood

Define Weighting • Users decide the weighting

(importance) for each criterion
based on their own preferences

• Create the tool in Power BI

Calculate composite score and rank • List the top 10 neighborhoods
 DAX Functions – PERCENTILE.INC
• PERCENTILE.INC - returns the k-th percentile of values in a range, where k is in the range 0..1,
inclusive
• DAX Syntax:
PERCENTILE.INC(<column>, <k>)
• column: a column containing the values that define relative standing
• k: the percentile value in the range 0..1, inclusive

Example:
▪ Income 25th percentile =
PERCENTILE.INC ( 'Neighborhood Rating'[% of $100K+Income], 0.25 )

▪ Income 50th percentile =

PERCENTILE.INC ( 'Neighborhood Rating'[% of $100K+Income], 0.50 )

▪ Income 75th percentile =

PERCENTILE.INC ( 'Neighborhood Rating'[% of $100K+Income], 0.75 )
DAX Functions – VAR and Return
• VAR – stores the result of an expression as a named variable, which can then be passed as an
argument to other measure expressions
• DAX Syntax:
VAR <name> = <expression>
• name: the name of the variable (identifier)
• expression: a DAX expression which returns a scalar or table value.

• RETURN – can be used together with VAR to consume variables defined in previous VAR
statements.
• DAX Syntax:
VAR <name> = <expression>
[VAR <name2> = <expression2> [...]]
RETURN <result_expression>
DAX Functions – VAR and Return
Use variables (VAR) in DAX formulas can:
• Improve performance
• Improve readability
• Simplify debugging
• Reduce complexity
Example:
▪ Airbnb Score =
VAR high =
PERCENTILE.INC ( 'Neighborhood Rating'[Tot_listings], 0.75 )
VAR mid =
PERCENTILE.INC ( 'Neighborhood Rating'[Tot_listings], 0.50 )
VAR low =
PERCENTILE.INC ( 'Neighborhood Rating'[Tot_listings], 0.25 )
RETURN
IF (
'Neighborhood Rating'[Tot_listings] <= low,
1,
IF (
'Neighborhood Rating'[Tot_listings] <= mid,
2,
IF ( 'Neighborhood Rating'[Tot_listings] <= high, 3, 4 )
)
)
Practice 4.2

• Create a Quadrant Chart by importing from Custom Visuals

• Calculate the score for each metric

Please refer to Practice 4 note for detailed instruction

Use What-if Parameters

• What-if parameters give you the flexibility to dynamically change the output of your queries
depending on their value
 Use What-if Parameters
Increment is how much • In addition to creating the
the parameter will adjust parameter, creating a what-if
when interacted with in a parameter also creates a measure
report • both the parameter and the
measure become part of your
model

Add slicer to this page checkbox

automatically puts a slicer with what-if
parameter onto the current report page
Use What-if Parameters

• Pass the What-if parameters into a DAX expression to create a Measure,

achieving dynamic calculation

• Power BI does not support to create dynamic calculated

column based on slicer or filter
• Use Measure instead
Calculate the Weighted Average Score

• To calculate composite score, we can assign a weight of W1 to metric M1, and

assign a weight of W2 to metric M2……
• Then, the following formula can be used to calculate the weighted average score
(composite score):

➢ (M1 * W1 + M2 * W2) / (W1 + W2)

Pass the parameter into a DAX expression to create

Example:
measure
▪ Weighted Average Score =
(
[Airbnb Weight Value] * SUM ( 'Neighborhood Rating'[Airbnb Score] )
+ [Crime Rate Weight Value] * SUM ( 'Neighborhood Rating'[Crime Score] )
+ [Diversity Weight Value] * SUM ( 'Neighborhood Rating'[Diversity Score] )
+ [Income Weight Value] * SUM ( 'Neighborhood Rating'[Income score] )
) / ( [Income Weight Value] + [Airbnb Weight Value] + [Crime Rate Weight Value] + [Div
ersity Weight Value] )
DAX Functions – RANKX

• RANKX – Returns the ranking of a number in a list of numbers for each row in the table
argument
• DAX Syntax:
RANKX(<table>, <expression>[, <value>[, <order>[, <ties>]]])
• table: returns a table of data over which the expression is evaluated
• expression: returns a single scalar value
• value(optional): returns a single scalar value whose rank is to be found
• order: 0(default) - the highest number in expression gets rank 1; 1 - the lowest number gets rank 1
• Ties: skip(default); dense - the next rank value, after a tie, is the next rank value
Example:
▪ Rank =
RANKX (
ALL ( 'Neighborhood Rating'[Neighborhood] ),
[Weighted Average Score],
,
0,
DENSE
)
DAX Functions – ALL

• ALL – Returns all the rows in a table, or all the values in a column, ignoring any filters that might
have been applied
• It is useful for clearing filters and creating calculations on all the rows in a table
• DAX Syntax:
ALL( [<table> | <column>[, <column>[, <column>[,…]]]] )
• table: The table that you want to clear filters on
• Column: the column that you want to clear filters on.
This function is not used by itself, but serves as an intermediate function that can be used to change the set of
results over which some other calculation is performed
Example:
▪ Rank =
RANKX (
ALL ( 'Neighborhood Rating'[Neighbourhood] ),
[Weighted Average Score],
,
0,
DENSE
)
 Use Case 2:
Neighborhood Rating Report with Power Automate and Forms
Use Case 3:
Power Automate Desktop
Thank You
- Ken Ding

500-80 Bloor Street West, Toronto

ON
www.weclouddata.com