Professional Documents
Culture Documents
3
3
3
• In the first category the number of data points are exactly equal
to (n+1), that is, one more than the degree of polynomial to be
fitted
1
Introduction
• It is imperative in this procedure that there is no error in the given data points,
hence we wish them to be on the curve “exactly” , i.e. exact polynomial fitting
• The second category is such that the number of data points which are to be
represented by the curve is greater than (n+1). Then it is impossible to pass a
single polynomial of degree n through all of the data points “exactly.”
• There are two alternatives: one is the so-called regression analysis which
minimizes some norm of the error vector, ei, where the error is defined as the
difference between the data points and the corresponding point on the curve
• The second alternative is to pass more than one polynomial through the subsets
of the given data points by imposing additional conditions at the points joining
subsequent curves. (For example, the derivative of the two adjacent polynomials
can be set equal to each other at the common point.) This concept leads to the
so-called spline curve fitting.
2
Exact Polynomial Fitting
Given (n+1) points: (x0, y0), (x1, y1), (x2, y2), … (xn, yn), there is a unique polynomial of
the nth degree which passes through these points.
The coefficients b0, b1, b2, … bn can be determined by solving the following linear
system of equations.
b0 + b1x0 + b2x02 + … +bnx0n = y0
b0 + b1x1 + b2x12 + … +bnx1n = y1
b0 + b1x2 + b2x22 + … +bnx2n = y2
. . . .
(4.2.2)
. . . … . .
. . . . .
b0 + b1xn + b2xn2 + … +bnxnn = yn
In matrix form these equations can be written as
[A]{B} = {Y}
(4.2.3)
Where the elements of [A] are given by: 3
Example 1
Determine the 3rd degree polynomial which passes through the following points:
x 0 1 2 3
y 1 0 1 0
Solution:
Substitute these values in Eq. 4.2.2 to obtain:
1 0 0 0 b0 1
b0 + b1(0) + b2(0) + b3(0) = 1 1
b0 + b1(1) + b2(1) + b3(1) = 0 1 1 1 b
1 =
0
1 2 4 8
b0 + b1(2) + b2(4) + b3(8) = 1 b2 1
b0 + b1(3) + b2(9) + b3(27) = 0 1 3 9 27 b3
0
Let n (x x )
j
L n
i (x ) (x x )
(4.2.5)
j0 i j
ji
Note that:
1 x xi
L (x)
n
i
0 x xj ji
( x x1 ) ( x x2 )
L20 ( x ) (4.2.6a)
( x0 x1 ) ( x0 x 2 )
(x x0 ) (x x2 ) (4.2.6b)
L (x )
2
1
( x1 x 0 ) ( x1 x 2 )
( x x 0 ) ( x x1 ) (4.2.6c)
L (x )
2
2
( x 2 x 0 ) ( x 2 x1 ) 5
Lagrange Polynomials
The nth degree Lagrange polynomial that passes through (n+1) points is then given by:
n (4.2.7)
L ( x ) y L ( x ) y L ( x ) y L ( x ) ... y L ( x )
n n
i i
n
0 0
n
1 1
n
n n
Where Lin(x) isi 0given by Eq. 4.2.5 and yi = f(xi) are the given function values
corresponding to x = xi.
6
Example 2
Determine the third degree Lagrange polynomial that passes through the points (0, 1);
(1, 0); (2, 1); (3, 0). That is, the same points as in Ex. E4.2.1:
Solution:
( x x1 )( x x2 )( x x3 ) ( x 1)( x 2)( x 3) 1
L03(x) = = = - ( x-1)(x-2)(x-3)
( x0 x1 )( x0 x2 )( x0 x3 ) (0 1)(0 2)(0 3) 6
( x x0 )( x x2 )( x x3 ) ( x 0)( x 2)( x 3) 1
L13(x) = = = ( x)(x-2)(x-3)
( x1 x0 )( x1 x2 )( x1 x3 ) (1 0)(1 2)(1 3)
2
( x x0 )( x x1 )( x x3 ) ( x 0)( x 1)( x 3) 1
L2 (x) =
3
= = - ( x)(x-1)(x-3)
( x2 x0 )( x2 x1 )( x2 x3 ) (2 0)(2 1)(2 3) 2
or
It is left to the reader to show that this polynomial is the same as that found in
Example E4.2.1 using the direct method.
8
Newton`s Polynomials
Another way of constructing simple polynomials for a small number of data points is
to use Newton’s divided difference polynomials (also called Newton’s interpolating
polynomials). These have the following form:
y = f(x) = b0 + b1(x - x0) + b2(x - x0)(x - x1) +…+ bn(x - x0)(x - x1)…(x - xn-1)
(4.2.8)
with the given (n+1) data points (x0, y0), (x1, y1)…(xn, yn). The coefficients bi (i = 0, 1,
2, … , n) are calculated using the finite divided difference formulae.
, x ] =y(y y2x0
x2y1)/(x
x12, – – ,xx1)
yx0 , xy[x
,
1 2 1 2
x 1
x 2 x 0
yx2 , x3 y(4.2.9b)
x1, x2
yx1, x2 , x3
x3 x1 9
y x1 , x2 , x3 y x0 , x1 , x2
y x0 , x1 , x2 , x3
(4.2.9c) x3 x0
y[x1, x2, x3, x4] = similarly, and so on . . .
b0 = y0
b1 = y[x0, x1]
b2 = y[x0, x1, x2]
(4.2.10)
..
. .
. .
bn = y[x0, x1,…, xn]
Solution:
Construct the table given below using Eq. 4.2.9. The coefficients are equal to the
divided differences given at the top row. Hence,
or
f(x) = 1-x + x(x - 1) - 2/3x(x - 1)(x - 2)
Interested readers may show that this is the same as the polynomial found in
Example E4.1.2
11
Cont`d
12
4.3 Least Squares Regression
Linear Regression
Consider, for example, the data given in Table 4.3.1 which is collected from a survey
of randomly selected people between the ages of 10 - 50 in a small town.
13
Cont`d
h = a + bw w = c + dh
85 2
80 1.9
Weight (kg)
75
Height (m)
1.8
70
65 1.7
60 1.6
55
50 1.5
1.5 1.7 1.9 50 60 70 80
Height (m) Weight (kg)
Least squares regression is a method that calls for minimization of the sum of the squares of
the components of the error vector Ei.
14
Cont`d
(i) Assume that there is no error in w, then the function to be minimized is defined
by:
N
S h ( a , b) [h i (a bw i )] 2
i 1
(4.3.3a)
(4.3.4)
where the error in w at each point is given by:
(iii) Assume that both variables may involve error. In this case one can minimize the
sum of the combined error in h and w simultaneously. That is, if we solve Eq. 4.3.1
for w, we obtain:
w = -(a/b) +(1/b)h
(4.3.5)
If the error in h and w were given equal weights (or importance) then it follows
intuitively that the function that should be minimized should be:
N
a 1
S hw (a , b) {[hi (a bwi )]2 [ wi ( hi )]2 }
i 1 b b
(4.3.6)
y = a0 + a1x
(4.3.7)
N
and
S ( a 0 , a1 ) [ y i ( a 0 a1 xi )]2
i 1
(4.3.8)
To find the minimum of S we take the derivative of it with respect to first a 0, then with
S N
respect to a1 { set
, and i (a
[ ythem 0 a
equal 2)} 0
x i )](Thus
to1zero.
a0 i 1
S N
(4.3.9a) { [ y i ( a 0 a 1 x i )]( 2 x i )} 0
a1 i 1
(4.3.9b)
17
Comment:It can be shown that due to the nature of the function S at the point where
Eqs. 4.2.9a & b are satisfied S has a minimum (not a maximum). This can also be
Cont`d
By rearranging 4.3.9a & b and simplifying, we arrive at:
where all summations are implied to be from i = 1 to N. These two equations are
sufficient for determining the coefficients a 0 and a1.
To compare the results from the two cases we solve h = a + bw for w which is:
w = -(a/b) + (1/b)h
If there were no errors in w we must have c = -(a/b) = -57.74 and d = (1/b) = 75.19,
which are significantly different than what we found by assuming that w values had
errors but h values did not. The true answer must lie somewhere between these.
20
Cont`d
Here we suggest that equal weight is given to both sets of values (a, b) and (c, d).
Adding Eqs. 4.2.1 & 4.2.2 side by side and solving for h yields:
h = [(a-c)/(1+d)] + [(1+b)/(1+d)]w
This yields:
h = 0.456 + 0.018w or w = -25.33 + 55.56h
(Interested readers may solve the problem using non-linear analysis which requires
minimization of the sum of the squares of the error E i2 = [Ehi2 + Ewi2]1/2.
21
Pseudo Non-Linear Regression
Exponential and power law functions of the form
y = a ebx ; y = a xb
(4.3.11)
S(a,b) = [ yi -( a ebxi) ] 2
(4.3.13)
Taking the derivatives of S with respect to a and b and setting them equal to zero yield
the following non linear equations to be solved to determine the two parameters a and
b.
[ yi -( a ebxi) ] (-2 ebxi) = 0
(4.3.14a)
[ yi -( a ebxi) ] (-2axiebxi ) = 0
(4.3.14b)
We like to simplify the problem observing that the exponential function given by
Equation (4.3.11) can be linearized by taking the natural logarithm of both sides, that
is
ln(y) = ln(a) +b x
(4.3.16a)
Hence
y* = a* + b* x*
(4.3.16b)
i: 1 2 3 4 5 6 7
------------------------------------------------------
x: 0.05 0.40 0.80 1.20 1.60 2.00 2.40
------------------------------------------------------
y: 0.55 0.75 1.00 1.40 2.00 2.70 3.75
Taking the natural logarithm of both sides of the above equations yields
ln(y) = ln (a) + b x
24
N = 7; x*i = 8.45; (x*i)2 = 14.5625; y*i = 2.4580; (x*i y*i ) = 6.5266
Cont`d
The equations to be solved are
7 a* + 8.45 b* = 2.4580
a = 0.5335; b=0.81284 25
Cont`d
Table 4.3.2 compares the two solutions.
* Note ln(A + B) lnA + lnB, but ln(AB) = lnA + lnB; ln(Ab) = b lnA
27
Iterative Regression
Consider
y = a + bxm
(4.3.18)
Here “a” is the intercept of the function to be found (i.e. y = a at x = 0 for m > 0) This
situation is best handled by the following iterative procedure: First make a wise
(educated) guess for the value of “a”, then determine b & m. Later check if indeed
the sum of the square of the errors is minimal. If not change the value of “a” and
repeat the process until the error function is minimized.
28
4.4 Polynomial Regression
In this section we generalize the procedure presented in Section 4.3 for linear
regression to any polynomial of degree m given by
y a0 a1 x a2 x 2 ... am x m (4.4.1)
For brevity let us derive the equations for a 2nd order polynomial and then extend
them to the general case.
A second order polynomial is given by
y a0 a1 x a2 x 2
(4.4.2)
(4.4.3)
29
where N is the total number of data points.
Cont`d
The minimum of this error function S is determined by solving the following equations.
S
yi a 0 a1 xi a 2 xi2 2 0
N
(4.4.4a)
a 0 i 1
S
yi a0 a1 xi a 2 xi2 2 xi 0
N
(4.4.4b)
a1 i 1
S
yi a0 a1 xi a 2 xi2 2 xi2 0
N
a 2 i 1
(4.4.4c)
30
Cont`d
For a polynomial of degree m we have (m+1) of such equations and they can be
written in a compact matrix form as
Na 0 xi a1 xi2 a 2 ... xim a m y i
xi a0 xi2 a1 xi3 a2 ... xim 1 am xi yi
x a x a x a
2
i 0
3
i 1
4
i 2 ... xim 2 am xi2 yi
. . . . .
. . . . . (4.4.5)
. . . . .
xim a0 xim 1 a1 xim 2 a2 ... xim m am xim y
In Equation (4.4.5) all summations are performed from i=1 to N. It should be noted
that the coefficient matrix is a symmetric matrix.
31
Example E4.4.1
Fit a cubic polynomial to the data from Ex. E4.3.2.
Solution:
Using the program POLYREG the following set of equations are obtained
7.000000 a0 + 8.450001 a1 + 14.562500 a2 + 28.224130 a3 = 12.150000
The results from the cubic polynomial are compared with the actual data below. It is
seen that the agreement is very good. The sum of the squares of the error is
significantly less than those of exponential fits (see Example E4.3.2). Hence a cubic 32
polynomial would be a good choice for this problem.
Cont`d
33
4.5 Multidimensional Linear
Regression
To fit a least squares function of the form (4.5.1) to a given set of data points we need
to minimize the error function given by
Fi a bx i cy i
N 2
S(a,b,c)
i 1 =
(4.5.2)
34
Cont`d
We set the derivative of S with respect to a, b and c to zero to obtain the following
equations:
S
Fi a bxi cy i 2 0
a
S (4.5.3)
Fi a bxi cy i 2 xi 0
b
S
Fi a bxi cy i 2 yi 0
c
After rearranging and simplifying we obtain:
Na x i b y i c Fi
(4.5.4) x a x b x y c x F
i
2
i i i i i
y a x y b y c y F
i i i
2
i i i
F = axbyc (4.5.5)
36
Example E4.5.1
The following data is obtained from measurements of pressure, P, Temperature, T, and
density, , of an unknown gas. Find out by using multidimensional regression analysis of
the state equation P = T to determine what this species could be and whether it is an
ideal gas.
Solution
The power law expression given above can be linearized by taking the logarithm of both
sides of the equation to yield
ln P ln ln ln T
Let
F = ln P, x = ln , y = ln T, a = ln , b = , c = .
F = a + bx + cy
which is the same as Equation (4.5.1). The set of equations to be solved are given by
Equation (4.5.4). Inserting N = 20 and the values of the sums given in Tables E4.5.1 into
Equation (4.5.4) gives
Hence
P T
MRu
= 8314
= = 43.90
R 189.39
This gas is most probably CO2 which has a molecular weight of 44 and it is an ideal
gas because the equation of state closely approximates P = RT, .
1.0, 1.0
38
Cont`d
39
Cont`d
40
4.6 Spline Curve Fitting and
Interpolation
Imagine a given data set containing 50 points, how would a polynomial of 49 th
degree look like and how difficult would it be to determine the coefficients. An
alternative method is to fit piece-wise polynomials to a sub set of the whole data (say
two, three or four points at a time), then patch these at the point that is common to
two adjacent intervals as illustrated in Figure 4.6.1. The constraints at the points
joining the two adjacent intervals can be selected as desired but our purpose is to
obtain “smooth” curves.
41
Cont`d
42
Quadratic Splines
Here, the polynomials to be determined for each interval are 2 nd degree and given by
First we require that each polynomial must pass through the two end points of that
interval, that is, for each ith interval.
For i = 1, 2, 3, …, n
The two end points of the ith interval are (xi-1, yi-1) and (xi, yi) as seen in Figure 4.6.1
43
.
Cont`d
Next to make the combined curve smooth we require the first derivative of two
neighboring polynomials be equal at the connecting points that is.
bi +2cixi = bi+1+2ci+1xi+1
(4.5.2c)
This gives us additional n-1 equations. The last equation needed is usually derived by
requiring the 2nd derivative of either the first polynomial or the last polynomial be zero
at the first or last point respectively. This means of course either the first or the last
curve is a straight line. Hence we take either c 1 = 0 or cN = 0 whichever is more
appropriate for our problem
44
Example E4.6.1
Fit a quadratic spline curve to the 3 intervals of the data given below which is
generated from f(x) = 1/(1+x2).
i 0 1 2 3
x 0.0 0.5 1.0 1.5
y 1.0 0.8 0.5 4/13
Solution:
The Equations 4.6.2 a and 4.6.2b yield (note that for the first polynomial, the
second derivative is taken to be zero. That is c =0 )
1
i=1 a1 + b1x0 = y0
a1 + b1x1 = y1
i=1 b1 + 0 = b2 + 2c2x1
i=2 b2 + 2c2x2 = b3 + 2c3x2
These are the 8-equations to be solved to determine the remaining 8-unknown
coefficients.
Substituting the x and the y values from the table into the above equations and
rearranging we obtain the following matrix equation
46
Cont`d
C a1 , b1 , a2 , b2 , c2 , a3 , b3 , c3
T
47
References
• Celik, Ismail, B., “Introductory Numerical Methods for Engineering Applications”,
Ararat Books & Publishing, LCC., Morgantown, 2001
• Fausett, Laurene, V. “Numerical Methods, Algorithms and Applications”, Prentice Hall,
2003 by Pearson Education, Inc., Upper Saddle River, NJ 07458
• Rao, Singiresu, S., “Applied Numerical Methods for Engineers and Scientists, 2002
Prentice Hall, Upper Saddle River, NJ 07458
• Mathews, John, H.; Fink, Kurtis, D., “Numerical Methods Using MATLAB” Fourth
Edition, 2004 Prentice Hall, Upper Saddle River, NJ 07458
• Varol, A., “Sayisal Analiz (Numerical Analysis), in Turkish, Course notes, Firat
University, 2001
• http://math.uww.edu/faculty/mcfarlat/inverse.htm
48