Professional Documents
Culture Documents
Lecture 3 Supervised
Lecture 3 Supervised
Lecture 3 Supervised
• CRISP-DM / CRISP-ML
• Visualisation
• Preprocessing
Machine Learning
Machine Learning
• Classical problems
• Predict category Classification
Computer Vision
Robotics
Speech
Sub-fields of AI
Supervised Learning
Machine Learning Unsupervised Learning
Computer Vision
Robotics
Speech
Machine Learning
Classification
Supervised
Learning
Regression
Clustering
Unsupervised
Machine Learning Association
Learning
Dimensionality reduction
Game AI
Reinforcement
Learning
Robot navigation
Machine Learning
Classification
Supervised
Learning
Regression
Clustering
Unsupervised
Machine Learning Association Lecture 5
Learning
Dimensionality reduction
Game AI
Reinforcement
Learning
Robot navigation
Machine Learning
Classification
Supervised
Learning
Regression
Clustering
Unsupervised
Machine Learning Association
Learning
Dimensionality reduction
Game AI
Reinforcement
Learning
Robot navigation
Supervised Machine Learning
Supervised Machine Learning
Object recognition
Supervised Learning
Forecasting
Regression Predicting
Preprocessed Labeled
Model training Output
Data Data
• Human annotator
• Intensive
Preprocessed Labeled
Model training Output
Data Data
• Binary target
4
• Two variables 3
y
2
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
Decision rules
5
• IF _ THEN _ (ELSE _) 3
y
2
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
Decision Tree
5
y
2
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
Decision Tree
5
x<3
●
3
y
2
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
Decision Tree
5
x<3
y > 3.5 ●
3
y
2
●
1
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
Decision Tree
5
x<3
y > 3.5 ●
3
y
2
● x < 1.5
1
● 0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
Decision Tree
5
x<3
y > 3.5 ●
3
y
2
● x < 1.5
1
● y < 2.5 0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
● ●
Decision Tree
5
Hyperparameters
4
• Split method
3
• Max splits 1
•…
Decision Tree
Pros Cons
• Average
Random Forest (Majority vote)
y y y y
y
x x x x x
x
Random Forest (Averaged)
y y y y
y
x x x x x
x
Random Forests
Hyperparameters
• Number of trees
• Sampling strategy
• Combination strategy
Random Forests
Pros Cons
• Parameter complexity
Neural Networks
Binary classification
y
2
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
2
½ 4
-1
½ 3
1
y
1 2
-2
1 1
-¼
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
2
½ 4
-1
2 2
½ 3
1
y
1 2
4 4 -2
1 1
-¼
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
0
2
½ 4
-1
2 2
½ 3
1
y
1 2
4 4 -2
1 1
-¼
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
0
2
½ 4
-1
2 2
½ 3
1
5 y
1 2
4 4 -2
1 1
-¼
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
0
2
½ 4
-1
2 2
½ 3
1
5 y
1 2
4 4 -2
1 1
-¼
-5
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
0
2
½ 4
-1
2 2
½ 3
1
5 0 y
1 2
4 4 -2
1 1
-¼
-5
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
0
2
½ 4
-1
2 2
½ 3
1
5 0 ≠ 1 y
1 2
4 4 -2
1 1
-¼
-5
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
0
2
½ 4
-1
2 2
½ 3
1.2
5 0 ≠ 1 y
1 2
4 4 -2
0.8 1
-¼
-5
0
0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
Binary classification
Multiclass classification
Hyperparameters
• Full architecture
• Hidden layers, neurons per layer
• Activation functions
• Learning rate
Neural Networks
Pros Cons
Preprocessed Labeled
Model training Output
Data Data
• Straight line
• Finding trends
Linear Regression
actual predicted
18
16
14
12
10
0
0 1 2 3 4 5
Linear Regression
Pros Cons
• AutoRegressive:
• Moving Average:
• ARIMA(p, d, q)
ARIMA
ARIMA
Pros Cons
• Explainability
Adapting classification methods
• Regression trees
0.75 0.5 ● ●
Regression trees
x<3
0.75 0.5
x
Neural Networks
• Classification / Regression
• Random Forest
• Neural Networks
Next week
Next week
• Overfitting / Underfitting