Bias & Variance

• Bias means error in the training data.

• Bias represents the inability of a machine learning model to truly
capture the relationship in the training data.
 Bias & Variance

• Variance means an error in the testing data.

• Variance is basically difference of fit on different dataset.
 Bias & Variance
When the error on the training data set is very high, then this scenario is
called underfitting.

The leftmost figure shows this case.

The error will also be high on the test data.

 Bias & Variance
When the error on the training data is very low but on test data, accuracy goes
down, then this scenario is called overfitting.

The rightmost figure shows this case.

 Bias & Variance

The middle figure shows the scenario when neither the issue of
overfitting nor the issue of underfitting exists.
 Bias & Variance

Underfitting implies high bias and high variance.

Overfitting implies low bias and high variance.

We always want to design a model with low bias and low variance, that is, which
neither has an issue of underfitting nor an issue of overfitting.
Bias & Variance
 Bias & Variance

To handle the issue of underfitting and overfitting, three techniques are available
in machine learning:
1. Regularization
2. Bagging
3. Boosting