Chapter 4 ML PDF

Chapter 4 – Introduction to
Machine Learning
Dr. Mohamad Hafis Izran Bin Ishak
Department of Control & Mechatronics (CMED)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia.
SKEM 4173 – Artificial Intelligence

www.utm.my
Dr. Hafis
Herman SKEM4173
innovative innovative
• entrepreneurial
MKEM 1713 • entrepreneurial • global
• global
Chapter Outline
Introduction to Machine Learning

• Definition of Machine Learning
• Types of Learning Algorithm
• Examples of Learning Techniques
• Decision tree
• K-Nearest Neighbors
• Linear regression
• Logistic regression
Dr. Hafis
Herman SKEM4173
MKEM 1713 innovative • entrepreneurial • global
Introduction to Machine
Learning
 Definition of Machine Learning
 Types of Learning Algorithm
 Examples of Learning Techniques
Dr. Hafis
Herman SKEM4173
MKEM 1713 innovative • entrepreneurial • global 3
Machine Learning is…
Machine learning, a branch of artificial intelligence, concerns

the construction and study of systems that can learn from data.
Dr. Hafis
Herman SKEM4173
Machine learning is programming computers to optimize a

performance criterion using example data or past experience.
-- Ethem Alpaydin
The goal of machine learning is to develop methods that can

automatically detect patterns in data, and then to use the
uncovered patterns to predict future data or other outcomes of
interest.
-- Kevin P. Murphy
The field of pattern recognition is concerned with the automatic

discovery of regularities in data through the use of computer
algorithms and with the use of these regularities to take actions.
-- Christopher M. Bishop
Dr. Hafis
Herman SKEM4173
Machine learning is about predicting the future, based on the

past.
-- Hal Daume III
past future
Training model/ Testing model/

Data predictor Data predictor
Dr. Hafis
Herman SKEM4173
Show Time
• Machine Learning (ML)
• Introduction
• What is ML?
Dr. Hafis
Herman SKEM4173
Algorithms
• The success of machine learning system depends on the

algorithms.
• The algorithms control the search to find and build the

knowledge structures.
• The learning algorithms should extract useful information

from training examples.
Dr. Hafis
Herman SKEM4173
Types of Learning Algorithm
• Supervised (inductive) learning

– Training data includes desired outputs
• Unsupervised learning
– Training data does not include desired outputs
• Semi-supervised learning
– Training data includes a few desired outputs
• Reinforcement learning
– Rewards from sequence of actions
Dr. Hafis
Herman SKEM4173
Algorithms
Supervised learning Unsupervised learning
Semi-supervised learning
10
Dr. Hafis
Herman SKEM4173
Show Time
• Supervised Learning
Dr. Hafis
Herman SKEM4173
Learning techniques
- Supervised Learning
• Supervised learning categories and techniques:
– Linear classifier (numerical functions)

– Parametric (Probabilistic functions)
• Naïve Bayes, Gaussian discriminant analysis (GDA), Hidden Markov
models (HMM), Probabilistic graphical models
– Non-parametric (Instance-based functions)
• K-nearest neighbors, Kernel regression, Kernel density estimation,
Local regression
– Non-metric (Symbolic functions)
• Classification and regression tree (CART), decision tree
– Aggregation
• Bagging (bootstrap + aggregation), Adaboost, Random forest
Dr. Hafis
Herman SKEM4173
Learning techniques
- Supervised Learning
• Linear classifier
• Techniques:
– Perceptron 
– Logistic regression 
– Support vector machine (SVM)
– Ada-line 
– Multi-layer perceptron (MLP) 
Dr. Hafis
Herman SKEM4173
Data
examples
Data
Dr. Hafis
Herman SKEM4173
Supervised learning
examples
label
label1
label2
labeled examples
label3
label4
Supervised learning: given labeled examples

Dr. Hafis
Herman SKEM4173
Supervised learning
label
label1
model/
label2 predictor
label3
label4
Supervised learning: given labeled examples
Dr. Hafis
Herman SKEM4173
Supervised learning
model/ predicted label

predictor
Supervised learning: learn to predict new example
Dr. Hafis
Herman SKEM4173
Classification Example
Differentiate
between low-risk
and high-risk
customers from
their income and
savings
Discriminant: IF income > θ1 AND savings > θ2

THEN low-risk ELSE high-risk
Dr. Hafis
Herman SKEM4173
Classification Applications
Face recognition
Character recognition
Spam detection
Medical diagnosis: From symptoms to illnesses
Biometrics: Recognition/authentication using physical

and/or behavioral characteristics: Face, iris, signature, etc
...
Dr. Hafis
Herman SKEM4173
Face Recognition
Training examples of a person
Test images
AT&T Laboratories, Cambridge UK

http://www.uk.research.att.com/facedatabase.html
Dr. Hafis
Herman SKEM4173
Regression Example
Price of a used car
x : car attributes y = wx+w0

(e.g. mileage)
y : price
Dr. Hafis
Herman SKEM4173
Regression Applications
Economics/Finance: predict the value of a stock
Epidemiology
Car/plane navigation: angle of the steering wheel,

acceleration, …
Temporal trends: weather over time
Dr. Hafis
Herman SKEM4173
Show Time
• Unsupervised Learning
Dr. Hafis
Herman SKEM4173
Learning techniques
- Unsupervised Learning
• Unsupervised learning categories and techniques
– Clustering
• K-means clustering
• Spectral clustering
– Density Estimation
• Gaussian mixture model (GMM)
• Graphical models
– Dimensionality reduction
• Principal component analysis (PCA)
• Factor analysis
Dr. Hafis
Herman SKEM4173
Unsupervised learning
Unupervised learning: given data, i.e. examples, but no labels

Dr. Hafis
Herman SKEM4173
Unsupervised learning
applications
learn clusters/groups without any label
customer segmentation (i.e. grouping)
image compression
bioinformatics: learn motifs
…
Dr. Hafis
Herman SKEM4173
Learning techniques
- Reinforcement Learning
left, right, straight, left, left, left, straight GOOD
left, straight, straight, left, right, straight, straight BAD
left, right, straight, left, left, left, straight 18.5
left, straight, straight, left, right, straight, straight -3
Given a sequence of examples/states and a reward after

completing that sequence, learn to predict the action to take
in for an individual example/state
Dr. Hafis
Herman SKEM4173
Reinforcement learning
example
Backgammon
… WIN!
… LOSE!
Given sequences of moves and whether or not the

player won at the end, learn to make good moves
Dr. Hafis
Herman SKEM4173
Other learning variations
What data is available:

• Supervised, unsupervised, reinforcement learning
• semi-supervised, active learning, …
How are we getting the data:

• online vs. offline learning
Type of model:
• generative vs. discriminative
• parametric vs. non-parametric
Dr. Hafis
Herman SKEM4173
Homework (2 marks each)
Please find more details about:

1. Decision tree
2. K-Nearest Neighbors
3. Linear regression
4. Logistic regression
– What is it?
– Advantages & Disadvantages
– Example Applications
Dr. Hafis
Herman SKEM4173

Chapter 4 ML PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 4 ML PDF

Uploaded by

Copyright:

Available Formats

Chapter 4 – Introduction to

SKEM 4173 – Artificial Intelligence

Introduction to Machine Learning

Machine learning, a branch of artificial intelligence, concerns

Machine learning is programming computers to optimize a

The goal of machine learning is to develop methods that can

The field of pattern recognition is concerned with the automatic

Machine learning is about predicting the future, based on the

Training model/ Testing model/

• The success of machine learning system depends on the

• The algorithms control the search to find and build the

• The learning algorithms should extract useful information

• Supervised (inductive) learning

Supervised learning Unsupervised learning

– Linear classifier (numerical functions)

Supervised learning: given labeled examples

Supervised learning: given labeled examples

model/ predicted label

Supervised learning: learn to predict new example

Discriminant: IF income > θ1 AND savings > θ2

Medical diagnosis: From symptoms to illnesses

Biometrics: Recognition/authentication using physical

Training examples of a person

AT&T Laboratories, Cambridge UK

Price of a used car

x : car attributes y = wx+w0

Economics/Finance: predict the value of a stock

Car/plane navigation: angle of the steering wheel,

Temporal trends: weather over time

Unupervised learning: given data, i.e. examples, but no labels

customer segmentation (i.e. grouping)

bioinformatics: learn motifs

left, right, straight, left, left, left, straight GOOD

left, straight, straight, left, right, straight, straight BAD

left, right, straight, left, left, left, straight 18.5

left, straight, straight, left, right, straight, straight -3

Given a sequence of examples/states and a reward after

Given sequences of moves and whether or not the

What data is available:

How are we getting the data:

Please find more details about:

You might also like