Suppose we want to best-fit a quadratic function y(x) = ax?

+ b to three data points as shown

below. This means we want to find the values of constants a and b such that the best-fit curve

is “closest” to the three points given.

(a) Given that the error, d i , between each data point ( x i , y i ) and the best-fit curve is
d i= y ( x i )− y i=( a x 2i +b ) − y i
show that the sum of squared errors (SSE) is
3
S SE ( a , b )=∑ d 2i =( b−1 ) + ( 4 a+ b−2 ) + ( 9 a+b−4 )
2 2 2

i=1
Solution:

The points are:

( x 1 , y 1 ) =(0 , 1)
( x 2 , y 2 ) =(2 , 2)
( x 3 , y 3 ) =(3 , 4)
Then,
d 1=( a ( 0 )2 +b )−1=b−1
d 2=( a ( 2 )2 +b ) −2=4 a+ b−2
d 3=( a ( 3 )2 +b )−4=9 a+b−4
Hence,
3
SSE ( a , b )=∑ d 2i =d 21 +d 22 +d 33= ( b−1 ) + ( 4 a+b−2 ) + ( 9 a+ b−4 )
2 2 2

i=1
Therefore,
2 2 2
SSE ( a , b )= ( b−1 ) + ( 4 a+ b−2 ) + ( 9 a+b−4 )
(b) The best-fit curve occurs when the SSE is at a minimum. Solve for the constants a and
b of the best-fit curve and apply derivative tests to show that both the local & global
minimum value of the SSE occurs at these constant values.

Solution:
First, we find the partial derivatives,
∂
SSE ( a , b )=2 ( 4 a+ b−2 )( 4 ) +2 ( 9 a+ b−4 )( 9 )
∂a
¿ 32 a+8 b−16+162 a+18 b−72
¿ 194 a +26 b−88

∂
SSE ( a , b ) =2 ( b−1 ) +2 ( 4 a+b−2 ) +2 ( 9 a+b−4 )
∂b
¿ 2 b−2+ 8 a+2 b−4 +18 a+2 b−8
¿ 26 a+ 6 b−14
Partial derivatives equal to 0

{
194 a+26 b−88=0 ⟹ 97 a+13 b=44
26 a+6 b−14=0 {
13 a+3 b=7

Solving the system we get, ( a , b )= ( 41 107

,
122 122 ).

Now, we find the second order partial derivatives,

2
∂
2
SSE ( a , b ) =194
∂a
2
∂
SSE ( a , b )=26
∂a∂b
2
∂
2
SSE ( a , b )=6
∂b
We see that,

( )
2 2 2 2
∂ ∂ ∂ 2
2
SSE ( a , b ) ∙ 2 SSE ( a , b )− SSE ( a , b ) =194 ( 6 )−26 =488>0
∂a ∂b ∂ a ∂b
and,
2
∂
2
SSE ( a , b ) =194>0
∂a

Therefore, the function is convex and ( a , b )= ( 122 ,

122 )
41 107
is a global minimum.