Professional Documents
Culture Documents
1a Introduction To ML
1a Introduction To ML
LEARNING
INTRODUCTION
LEC-1A
1 Hammad Afzal
Department of CSE
hammad.afzal@mcs.edu.pk
COURSE INTRO
Pre-requisite
Introductory knowledge of Probability, Statistics and Linear
Algebra
Course Resources
Lectures slides, assignments (computer/written), solutions to
problems, research papers, projects, and announcements will be
uploaded on LMS page.
2
WHAT IS MACHINE LEARNING?
3
MACHINE LEARNING
4
MACHINE LEARNING
5
LEARNING PROBLEMS – EXAMPLES
Learning = Improving with experience over some
task
Improve over task T,
With respect to performance measure P,
Based on experience E.
Example
T= Play checkers
P = % of games won in a tournament
E = opportunity to play
6
LEARNING PROBLEMS – EXAMPLES
Handwriting recognition learning
problem
Task T: recognizing handwritten
words within images
Performance measure P: percent of
words correctly recognized
Training experience E: a database of
handwritten words with given
classifications
8
LEARNING PROBLEMS – EXAMPLES
A robot driving learning problem
Task T: driving on public four-lane
highways using vision sensors
Performance measure P: average
distance traveled before an error (as
judged by human overseer)
Training experience E: a sequence of
images and steering commands
recorded while observing a human
driver
9
MACHINE LEARNING
Nicolas learns about trucks and dumpers
10
MACHINE LEARNING
But will he recognize others?
12
MACHINE LEARNING
13
Applications
CREDIT SCORING
Differentiating between
low-risk and high-risk
customers from their
income and savings
AUTONOMOUS DRIVING
ALVINN* – Drives 70mph on highways
*Autonomous Land Vehicle In 15
a Neural Network
OCR & HANDWRITING RECOGNITION
17
TEMPLATE MATCHING
Problem: Recognize letters A to Z
Image is converted into 12x12 bitmap.
18
TEMPLATE MATCHING
Bitmap is represented by 12x12-matrix or by 144-vector
with 0 and 1 coordinates.
0 0 0 0 0 0 1 1 0 0 0 0
0 0 0 0 0 1 1 1 0 0 0 0
0 0 0 0 0 1 0 1 1 0 0 0
0 0 0 0 1 1 0 1 1 0 0 0
0 0 0 0 1 0 0 0 1 0 0 0
0 0 0 1 1 0 0 0 1 1 0 0
0 0 0 1 1 0 0 0 1 1 0 0
0 0 1 1 1 1 1 1 1 1 1 0
0 0 1 1 0 0 0 0 0 1 1 0
0 1 1 0 0 0 0 0 0 1 1 0
0 1 1 0 0 0 0 0 0 0 1 1
1 1 0 0 0 0 0 0 0 0 1 1
TEMPLATE MATCHING
Training samples – templates with corresponding class:
t1 { (0,0,0,0,1,1,...,0), ' A '}
t 2 { (0,0,0,0,0,1,...,0), ' A '}
.........
t k { (0,0,1,1,1,1,...,0), ' B'}
..........
Template of the image to be recognized:
T { (0,0,0,0,1,1,...,0), ' A? '}
Algorithm:
1. Find ti , so that ti T . 20
2. Assign image to the same class as ti .
FEATURES
Features are the individual measurable properties of the
signal being observed.
The set of features used for learning/recognition is called
feature vector.
The number of used features is the dimensionality of the
feature vector.
n-dimensional feature vectors can be represented as points
in n-dimensional feature space 22
FEATURES
height
x1
x x
weight 2
Class 1 Class 1
Class 2 Class 2
23
FEATURE EXTRACTION
Feature extraction aims to create discriminative features
good for learning
Good Features
Objects from the same class have similar feature values.
Objects from different classes have different values.
24
25
Supervised learning
Classification
Regression
Unsupervised learning
Reinforcement learning
26
CLASSIFICATION
27
SUPERVISED LEARNING -
CLASSIFICATION
Objective
Make Nicolas recognize what is an apple and what is an
orange
28
CLASSIFICATION
Apples Oranges
29
CLASSIFICATION
You had some training example or
‘training data’
What is this???
The examples were ‘labeled’
30
Its an
apple!!!
CLASSIFICATION
Apple
Pear
Tomato
Cow
Dog
Horse
32
CLASSIFIER
K-Nearest Neighbor Classifier
Use k nearest neighbors instead of 1 to classify pattern.
Class 1
Class 2
33
CLASSIFICATION
Cancer Diagnosis – Tumor size for prediction
Malignant
Benign
Tumor Size
35
CLASSIFICATION
Cancer Diagnosis – Generally more than one variables
Malignant
Benign
Age
Tumor Size
Why supervised – The algorithm is given a number of patients
with the RIGHT ANSWER and we want the algorithm to learn 36
to predict for new patients
CLASSIFICATION
Cancer Diagnosis – Generally more than one variables
Predict for this
patient
Malignant
Benign
Age
Tumor Size
Malignant or Benign
SUPERVISED LEARNING - EXAMPLE
Cancer diagnosis – Many more features
Patient ID # of Tumors Avg Area Avg Density Diagnosis
1 5 20 118 Malignant
2 3 15 130 Benign
3 7 10 52 Benign
4 2 30 100 Malignant
Supervised learning
Classification
Regression
Unsupervised learning
Reinforcement learning
39
COURSE OUTLINE
Machine Learning: Theory and Applications
Introduction to probability theory and Linear Algebra
Parametric Methods
Dimensionality Reduction
Clustering
Decision Trees