Bias and Variance

Before beginning, define the following:

1.
Generalization - The ability of a model to make accurate predictions on independent test data.
Model complexity Test error - The test error over an independent test sample.
Consider the following:
Note: Figure 7.1 from ESL shows the effect of increasing model complexity on the training error and the
test error: The training error falls as we increase the model complexity while the test error imitates a parabola.
Increasing the factors of our model is a good thing to a point at which we begin to ascribe meaning to noise.