Automated Machine Learning

with Microsoft Azure

Getting the best model for YOUR data

Axel Sirota
Introduction
What are we going to do?
 Introduction
AutoML in the SDK

• Who am I?
• SDK vs Studio
• What is AutoML?
• Feature Engineering in the SDK
• What does it solve?
• Interpreting results

AutoML in the Studio

• Triggering an AutoML run

• Validating an AutoML model
• Deploying an endpoint
 Introduction
AutoML in the SDK

• Who am I?
• SDK vs Studio
• What is AutoML?
• Feature Engineering in the SDK
• What does it solve?
• Interpreting results

AutoML in the Studio

• Triggering an AutoML run

• Validating an AutoML model
• Deploying an endpoint
Introduction
Who am I?
Who am I?

– Microsoft Certified Trainer

Who am I?

QR to my
Pluralsight courses
– Microsoft Certified Trainer

– Author, Instructor, and Editor at Pluralsight,

Develop Intelligence, and O’Reilly Media QR to my O’Reilly

trainings
Who am I?

QR to my
– Microsoft Certified Trainer Pluralsight courses

– Author, Instructor, and Editor at Pluralsight,

Develop Intelligence, and O’Reilly Media

QR to my O’Reilly
–AI and Cloud Consultant trainings
Introduction
What Is AutoML and Why it will revolutionize the
way Data Science works today?
Poll: What is you previous experience with AutoML

- None
- Heard of it in conferences but no usage
- POCs but no production experience
- Production experience
 A Case Study: Creating an ML
model
Which features to
engineer?
 A Case Study: Creating an ML
model
Which model/
Which features to
architecture to
engineer?
use?
 A Case Study: Creating an ML
model
Which model/
Which features to Which parameters
architecture to
engineer? to use?
use?
 A Case Study: Creating an ML
model
Which model/ Which
Which features to Which parameters
architecture to hyperparameters
engineer? to use?
use? to use?
 A Case Study: Creating an ML
model
Which model/ Which
Which features to Which parameters
architecture to hyperparameters
engineer? to use?
use? to use?

Each of these decisions add up possibilities

A Case Study: Creating an ML
model

AutoML comes to solve this!

What is AutoML?

• Use ML to find the best ML

model
• Smart optimisation
• Gets the best model for OUR
data
A Case Study: Creating an ML
model

Basic Neural
Network
A Case Study: Creating an ML
model

Optimal
Basic Neural
AutoML architecture and
Network
hyperparameters
A Case Study: Creating an ML
model

Optimal
Basic Neural
AutoML architecture and
Network
hyperparameters

AutoML reduces the complexity of making new Machine

Learning Models
AutoML Use Cases
• Productivity
• Focus on the important design decisions
• Focus on iteration
• And many more…
Azure AutoML
Azure provides a one-stop solution for
AutoML
Azure AutoML
Azure provides a one-stop solution for
AutoML
It provides
• Easy computation of experiments
Azure AutoML
Azure provides a one-stop solution for
AutoML
It provides
• Easy computation of experiments
• Easy iteration and avoids biases
Azure AutoML
Azure provides a one-stop solution for
AutoML
It provides
• Easy computation of experiments
• Easy iteration and avoids biases
• Easy deployment options
Azure AutoML
Azure provides a one-stop solution for
AutoML
It provides
• Easy computation of experiments
• Easy iteration and avoids biases
• Easy deployment options
• Easy interpretation of results
Summary

• Automated Machine Learning simplifies the search

for an optimal ML pipeline
Summary

• Automated Machine Learning simplifies the search

for an optimal ML pipeline
• It finds the best features, model architecture, and
hyperparameters based on the dataset you present
Summary

• Automated Machine Learning simplifies the search

for an optimal ML pipeline
• It finds the best features, model architecture, and
hyperparameters based on the dataset you present
• Azure enables AutoML easily for all within Azure ML.
AutoML in the Studio
Triggering an AutoML run
 Introduction
AutoML in the SDK

• Who am I?
• SDK vs Studio
• What is AutoML?
• Feature Engineering in the SDK
• What does it solve?
• Interpreting results

AutoML in the Studio

• Triggering an AutoML run

• Validating an AutoML model
• Deploying an endpoint
Exercise Time
Live Demo: Triggering an Automated ML Run

Time allocated: 20 minutes

Pulse Check Time
How confident do you feel about everything
we saw up to now?
AutoML in the Studio
What is AutoML saving us from doing?
 Machine Learning Workflow

Raw Data

Data
Cleaning
 Machine Learning Workflow

Feature
Raw Data
Selection

Data Feature
Cleaning Construction

Feature
Preprocessing
 Machine Learning Workflow

Feature
Raw Data
Selection

Data Feature Model

Cleaning Construction Selection

Feature
Preprocessing
 Machine Learning Workflow

Feature
Raw Data
Selection

Data Feature Model Hyperparameter Model

Cleaning Construction Selection tuning Validation

Feature
Preprocessing
 Machine Learning Workflow

Feature
Raw Data
Selection

Data Feature Model Hyperparameter Model

Cleaning Construction Selection tuning Validation

Feature
Preprocessing
Trained
Model
 Machine Learning Workflow

Feature
Raw Data
Selection

Data Feature Model Hyperparameter Model

Cleaning Construction Selection tuning Validation

Feature
Preprocessing
Trained
AutoML Model
Why AutoML?

Default
parameters are
“always” bad
Why AutoML?

Default
“Best” model
parameters are
is a surprise
“always” bad
Why AutoML?

Data cleaning
Default
“Best” model and feature
parameters are
is a surprise extraction time
“always” bad
Default
Parameters

Image Source: “Data-driven advice for applying machine learning to bioinformatics problems”
Best
model

Image Source: “Data-driven advice for applying machine learning to bioinformatics problems”
Data cleaning time
AutoML in the Studio
Validating our model
Cross Validation

This happens on EACH “model” tested in Azure

The Pipeline
Model Scores Model 1

Model Scores Model 2

Model Scores Model 3

Model Scores Model 4

Cross Validation
The Pipeline
Model Scores Model 1

Model Scores Model 2

Best model

Model Scores Model 3

Model Scores Model 4

Cross Validation Model Selection

The Pipeline
Model Scores Model 1

Test set
Model Scores Model 2
True
Best model performance
Model Scores Model 3 best model

Model Scores Model 4

Cross Validation Model Selection Model Validation

Exercise Time
Live Demo: Deploying and Validating an
AutoML model output

Time allocated: 20 minutes

Pulse Check Time
How confident do you feel deployment and
validation of AutoML runs?
Break
Time Allocated: 10 minutes
AutoML in the SDK
Triggering from the SDK
Exercise Time
Live Demo: Triggering an AutoML Experiment
from the SDK

Time allocated: 20 minutes

AutoML in the SDK
Feature Engineering
Engineered Features

Scaling / Normalization Rolling Features

Inputting Dropping
Scaling and Normalisation

Some algorithms converge faster

 Rolling Windows

Source: “Minute-Scale Wind Speed Forecasting Using Scanning Lidar Inflow Measurements” Simon et. at. 2018
Rolling Windows
Date Load

2021-01-01 4500

2021-01-02 4650

2021-01-03 4400

2021-01-04 4100
Rolling Windows
Date Load Load + 2

2021-01-01 4500 4400

2021-01-02 4650 4100

2021-01-03 4400

2021-01-04 4100
Rolling Windows
Date Load

2021-01-01 4500

2021-01-02 None

2021-01-03 4400

2021-01-04 4100
Rolling Windows
Date Load

2021-01-01 4500

2021-01-02 4333 (mean of Load)

2021-01-03 4400

2021-01-04 4100
Exercise Time
Live Demo: Adding Feature Engineering to an
AutoML Experiment from the SDK

Time allocated: 10 minutes

Pulse Check Time
How confident do you feel about using the
SDK?
Exercise Time

Live Demo: Interpreting Results

Time allocated: 10 minutes

Exercise Time
Live Demo: Deploying to Kubernetes and
consuming the model

Time allocated: 10 minutes

Further Resources
AutoML Org: https://www.automl.org/

AutoML in Azure: https://learning.oreilly.com/library/

view/practical-automated-machine/9781492055587/
Final Q&A
Time Allocated: 10 minutes
Thank you

Axel Sirota