Professional Documents
Culture Documents
13 - 1bias Variance Tradeoff
13 - 1bias Variance Tradeoff
Models Diagnostics
Bias and Variance
Thanks to Drs. Sridhar Pappu and Anand Jayaraman for the material
The best place for students to learn Applied Engineering http://www.insofe.edu.in
The Ultimate Test of Model Accuracy
• Holdout set: Split data into train, validation and test sets (in 70:20:10 or
60:20:20, etc. ratios), and ensure model performance is similar.
• Training Set: For fitting a model
• Validation Set: For selecting a model based on estimated prediction errors
• Test Set: For assessing selected model’s performance on “new” data
e5
e4
e3
e2
e1
• a small standard deviation of the errors (low variance, i.e., error does
not vary much based on the choice of the dataset)
• MAE (Mean Absolute Error): Mean of the absolute value of the difference between the
predicted and actual values.
• MAPE (Mean Absolute Percentage Error): Same as above but converted into
percentages to allow for comparison across different scales (e.g., comparing accuracies
of forecasts on BSE vs NSE).
• RMSE (Root Mean Square Error): Accounts for infrequent large errors, whose impact
may be understated by the mean-based error measures.
• If the model performance on training and testing data sets is inconsistent, it indicates a
problem either with Bias or Variance.
• Variance arises due to the model being overly sensitive to small fluctuations in the
training data. Such a model overfits the data, including the random noise rather than
just the actual behaviour.
7 7 7
6 6 6
5 5 5
4 4 4
3 3 3
2 2 2
1 1 1
0 0 0
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8
10
9 y = 0.6071x + 2.5714
R² = 0.46916
Training Error 10
9
y = 0.0292x6 - 0.6958x5 + 6.6042x4 - 31.687x3 + 80.367x2 - 100.62x + 51
R² = 1
8 8
7 7
6 6
5 5
3
Underfitting Model Complexity Overfitting 4
2 2
1 1
0 0
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8
Underfitting (Bias)
Validation/Test Error • High Bias problem
• High training error
• Low difference between
training and test/validation
Error
errors
10
9 y = 0.6071x + 2.5714
R² = 0.46916
Training Error 10
9
y = 0.0292x6 - 0.6958x5 + 6.6042x4 - 31.687x3 + 80.367x2 - 100.62x + 51
R² = 1
8 8
7 7
6 6
5 5
2 2
1 1
0 0
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8
Overfitting (Variance)
Validation/Test Error • High Variance problem
• Low training error
• High difference between
training and test/validation
Error
errors
10
9 y = 0.6071x + 2.5714
R² = 0.46916
Training Error 10
9
y = 0.0292x6 - 0.6958x5 + 6.6042x4 - 31.687x3 + 80.367x2 - 100.62x + 51
R² = 1
8 8
7 7
6 6
5 5
2 2
1 1
0 0
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8
• Regularization decreases variance at the cost of increasing bias. It can be applied to a variety of
techniques (not just linear models).
• In Artificial Neural Networks, bias decreases at the cost of increasing variance with addition of hidden
layers.
• In Decision Trees, increasing the length of the tree increases variance. Pruning is used to control
variance.
Ref: https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff
Last accessed: July 08, 2017
• Boosting methods combine many “weak” (high bias) models in an ensemble that lowers bias compared to
individual models.
Ref: https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff
Last accessed: July 08, 2017
Social Media
Web: http://www.insofe.edu.in
Facebook: https://www.facebook.com/insofe
Twitter: https://twitter.com/Insofeedu
YouTube: http://www.youtube.com/InsofeVideos
SlideShare: http://www.slideshare.net/INSOFE
LinkedIn: http://www.linkedin.com/company/international-school-of-engineering
This presentation may contain references to findings of various reports available in the public domain. INSOFE makes no representation as to their accuracy or that the organization
subscribes to those findings.