Professional Documents
Culture Documents
Introduction To Curve Fitting PDF
Introduction To Curve Fitting PDF
• In the first category the number of data points are exactly equal
to (n+1), that is, one more than the degree of polynomial to be
fitted
2
Introduction
• It is imperative in this procedure that there is no error in the given data points,
hence we wish them to be on the curve “exactly” , i.e. exact polynomial fitting
• The second category is such that the number of data points which are to be
represented by the curve is greater than (n+1). Then it is impossible to pass a
single polynomial of degree n through all of the data points “exactly.”
• There are two alternatives: one is the so-called regression analysis which
minimizes some norm of the error vector, ei, where the error is defined as the
difference between the data points and the corresponding point on the curve
• The second alternative is to pass more than one polynomial through the subsets
of the given data points by imposing additional conditions at the points joining
subsequent curves. (For example, the derivative of the two adjacent polynomials
can be set equal to each other at the common point.) This concept leads to the
so-called spline curve fitting.
3
Exact Polynomial Fitting
Given (n+1) points: (x0, y0), (x1, y1), (x2, y2), … (xn, yn), there is a unique polynomial of
the nth degree which passes through these points.
The coefficients b0, b1, b2, … bn can be determined by solving the following linear
system of equations.
b0 + b1x0 + b2x02 + … +bnx0n = y0
b0 + b1x1 + b2x12 + … +bnx1n = y1
b0 + b1x2 + b2x22 + … +bnx2n = y2
. . . . (4.2.2)
. . . … . .
. . . . .
b0 + b1xn + b2xn2 + … +bnxnn = yn
In matrix form these equations can be written as
[A]{B} = {Y} (4.2.3)
Where the elements of [A] are given by:
x 0 1 2 3
y 1 0 1 0
Solution:
Substitute these values in Eq. 4.2.2 to obtain:
1 0 0 0 b0 1
b0 + b1(0) + b2(0) + b3(0) = 1 1 1 1 1 b 0
b0 + b1(1) + b2(1) + b3(1) = 0 1 =
1 2 4 8
b0 + b1(2) + b2(4) + b3(8) = 1 b2 1
b0 + b1(3) + b2(9) + b3(27) = 0 1 3 9 27 b3
0
Let n (x − x )
(x
j
L n
i (x )= (4.2.5)
j =0 −x )
i j
ji
Note that:
1 x = xi
L (x) =
n
i
0 x = xj ji
( x − x 0 ) ( x − x1 ) (4.2.6c)
L22 ( x ) =
( x 2 − x 0 ) ( x 2 − x1 ) 6
Lagrange Polynomials
The nth degree Lagrange polynomial that passes through (n+1) points is then given
by:
n
L ( x) = yi Lni ( x ) = y0 Ln0 ( x) + y1L1n ( x) + ... + yn Lnn ( x)
n
(4.2.7)
i =0
Where Lin(x) is given by Eq. 4.2.5 and yi = f(xi) are the given function values
corresponding to x = xi.
7
Example 2
Determine the third degree Lagrange polynomial that passes through the points (0, 1);
(1, 0); (2, 1); (3, 0). That is, the same points as in Ex. E4.2.1:
Solution:
( x − x1 )( x − x2 )( x − x3 ) ( x − 1)( x − 2)( x − 3) 1
L03(x) = = = - (x-1)(x-2)(x-3)
( x0 − x1 )( x0 − x2 )( x0 − x3 ) (0 − 1)(0 − 2)(0 − 3) 6
( x − x0 )( x − x2 )( x − x3 ) ( x − 0)( x − 2)( x − 3) 1
L13(x) = =
( x1 − x0 )( x1 − x2 )( x1 − x3 ) (1 − 0)(1 − 2)(1 − 3) = (x)(x-2)(x-3)
2
( x − x0 )( x − x1 )( x − x3 ) ( x − 0)( x − 1)( x − 3) 1
L2 3(x) = = = - (x)(x-1)(x-3)
( x2 − x0 )( x2 − x1 )( x2 − x3 ) (2 − 0)(2 − 1)(2 − 3) 2
3(x)
( x − x0 )( x − x1 )( x − x2 ) ( x − 0)( x − 1)( x − 2) 1
L3 = = = (x)(x-1)(x-2)
( x3 − x0 )( x3 − x1 )( x3 − x2 ) (3 − 0)(3 − 1)(3 − 2) 6 8
The third degree polynomial is then found from,
or
It is left to the reader to show that this polynomial is the same as that found in
Example E4.2.1 using the direct method.
9
Newton`s Polynomials
Another way of constructing simple polynomials for a small number of data points is
to use Newton’s divided difference polynomials (also called Newton’s interpolating
polynomials). These have the following form:
y = f(x) = b0 + b1(x - x0) + b2(x - x0)(x - x1) +…+ bn(x - x0)(x - x1)…(x - xn-1) (4.2.8)
with the given (n+1) data points (x0, y0), (x1, y1)…(xn, yn). The coefficients bi (i = 0, 1,
2, … , n) are calculated using the finite divided difference formulae.
b0 = y0
b1 = y[x0, x1]
b2 = y[x0, x1, x2] (4.2.10)
. .
. .
. .
bn = y[x0, x1,…, xn]
Solution:
Construct the table given below using Eq. 4.2.9. The coefficients are equal to the
divided differences given at the top row. Hence,
or
f(x) = 1-x + x(x - 1) - 2/3x(x - 1)(x - 2)
Interested readers may show that this is the same as the polynomial found in
Example E4.1.2
12
Cont`d
13
4.3 Least Squares Regression
Linear Regression
Consider, for example, the data given in Table 4.3.1 which is collected from a survey
of randomly selected people between the ages of 10 - 50 in a small town.
14
Cont`d
h = a + bw w = c + dh
85 2
80 1.9
Height (m)
Weight (kg)
75
1.8
70
65 1.7
60 1.6
55
50 1.5
1.5 1.7 1.9 50 60 70 80
Height (m) Weight (kg)
Least squares regression is a method that calls for minimization of the sum of the squares of
the components of the error vector Ei.
15
Cont`d
(i) Assume that there is no error in w, then the function to be minimized is defined
by:
N
S h ( a , b) = [h i − (a + bw i )]2 (4.3.3a)
i =1
(ii) Assume that there is no error in h, then the function to be minimized is defined by
N
S w (a , b) = [ w i − (c + dh i )]2 (4.3.4)
i =1
where the error in w at each point is given by:
16
Cont`d
(iii) Assume that both variables may involve error. In this case one can minimize the
sum of the combined error in h and w simultaneously. That is, if we solve Eq. 4.3.1
for w, we obtain:
If the error in h and w were given equal weights (or importance) then it follows
intuitively that the function that should be minimized should be:
N
a 1
Shw (a, b) = {[hi − (a + bwi )]2 + [ wi − ( − + hi )]2 } (4.3.6)
i =1 b b
The minimization of Sh or Sw given by Eqs. 4.3.3 & 4.3.4, respectively, is
straightforward as will be shown shortly. The minimization of S hw on the other hand,
is not trivial and left as an advance topic.
17
Cont`d
But first, we formulate the problem of linear regression in terms of generic variables, x
and y. That is,
y = a0 + a1x (4.3.7)
and
N
S (a0 , a1 ) = [ yi − (a0 + a1 xi )]2 (4.3.8)
i =1
To find the minimum of S we take the derivative of it with respect to first a0, then with
respect to a1, and set them equal to zero. Thus
S N
= { [ y i − (a 0 + a 1 x i )]( −2)} = 0 (4.3.9a)
a0 i =1
S N
(4.3.9b)
= { [ y i − (a 0 + a 1 x i )]( −2 x i )} = 0
a1 i =1
Comment:It can be shown that due to the nature of the function S at the point where
Eqs. 4.2.9a & b are satisfied S has a minimum (not a maximum). This can also be
shown a posteri by substituting the values of a0 and a1 in Eq. 4.3.8 and observing that
indeed the solution set corresponds to the minimum of S.
18
Cont`d
By rearranging 4.3.9a & b and simplifying, we arrive at:
where all summations are implied to be from i = 1 to N. These two equations are
sufficient for determining the coefficients a0 and a1.
To compare the results from the two cases we solve h = a + bw for w which is:
w = -(a/b) + (1/b)h
If there were no errors in w we must have c = -(a/b) = -57.74 and d = (1/b) = 75.19,
which are significantly different than what we found by assuming that w values had
errors but h values did not. The true answer must lie somewhere between these.
21
Cont`d
Here we suggest that equal weight is given to both sets of values (a, b) and (c, d).
Adding Eqs. 4.2.1 & 4.2.2 side by side and solving for h yields:
h = [(a-c)/(1+d)] + [(1+b)/(1+d)]w
This yields:
h = 0.456 + 0.018w or w = -25.33 + 55.56h
(Interested readers may solve the problem using non-linear analysis which requires
minimization of the sum of the squares of the error Ei2 = [Ehi2 + Ewi2]1/2.
22
Pseudo Non-Linear Regression
Exponential and power law functions of the form
y = a ebx ; y = a xb (4.3.11)
We like to simplify the problem observing that the exponential function given by
Equation (4.3.11) can be linearized by taking the natural logarithm of both sides, that
is
Hence
y* = a* + b* x* (4.3.16b)
24
Example E4.3.2
Fit an exponential function of the form of Equation (4.3.11) to the data given below using
the pseudo non-linear method. Compare the results for the direct non-linear method which
obtains the parameters a and b from Equations 4.2.15 a&b.
y = a ebx
i: 1 2 3 4 5 6 7
------------------------------------------------------
x: 0.05 0.40 0.80 1.20 1.60 2.00 2.40
------------------------------------------------------
y: 0.55 0.75 1.00 1.40 2.00 2.70 3.75
Taking the natural logarithm of both sides of the above equations yields
ln(y) = ln (a) + b x
Let y* = ln(y); a* = ln(a); b* = b, and x* = x, then prepare a modified table involving the
transformed variables
i: 1 2 3 4 5 6 7
-----------------------------------------------------------------
x*: 0.05 0.40 0.80 1.20 1.60 2.00 2.40
-----------------------------------------------------------------
y*: -0.598 -0.288 0.00 0.336 0.693 0.993 1.322
25
N = 7; x*i = 8.45; (x*i)2 = 14.5625; y*i = 2.4580; (x*i y*i ) = 6.5266
Cont`d
The equations to be solved are
7 a* + 8.45 b* = 2.4580
a = 0.5335; b=0.81284 26
Cont`d
Table 4.3.2 compares the two solutions.
* Note ln(A + B) lnA + lnB, but ln(AB) = lnA + lnB; ln(Ab) = b lnA
28
Iterative Regression
Consider
y = a + bxm (4.3.18)
Here “a” is the intercept of the function to be found (i.e. y = a at x = 0 for m > 0) This
situation is best handled by the following iterative procedure: First make a wise
(educated) guess for the value of “a”, then determine b & m. Later check if indeed
the sum of the square of the errors is minimal. If not change the value of “a” and
repeat the process until the error function is minimized.
29
4.4 Polynomial Regression
In this section we generalize the procedure presented in Section 4.3 for linear
regression to any polynomial of degree m given by
y = a0 + a1 x + a2 x 2 +...+am x m (4.4.1)
For brevity let us derive the equations for a 2nd order polynomial and then extend
them to the general case.
A second order polynomial is given by
y = a0 + a1 x + a2 x 2 (4.4.2)
The minimum of this error function S is determined by solving the following equations.
S
= yi − (a0 + a1 xi + a 2 xi2 ) (− 2 ) = 0
N
(4.4.4a)
a0 i =1
S
= yi − (a0 + a1 xi + a 2 xi2 ) (− 2 xi ) = 0
N
(4.4.4b)
a1 i =1
S
= yi − (a0 + a1 xi + a2 xi2 ) (− 2 xi2 ) = 0
N
(4.4.4c)
a2 i =1
31
Cont`d
For a polynomial of degree m we have (m+1) of such equations and they can be
written in a compact matrix form as
Na 0 + xi a1 + xi2 a 2 +... + xim a m = yi
xi a0 + xi2 a1 + xi3 a2 +...+ xim+1 am = xi yi
( x )a + ( x )a + ( x )a
2
i 0
3
i 1
4
i 2 + ... + ( xim+2 )am = xi2 yi
. . . . .
. . . . . (4.4.5)
. . . . .
xim a0 + xim+1 a1 + xim+2 a2 +...+ xim+m am = xim y
In Equation (4.4.5) all summations are performed from i=1 to N. It should be noted
that the coefficient matrix is a symmetric matrix. 32
Example E4.4.1
Fit a cubic polynomial to the data from Ex. E4.3.2.
Solution:
Using the program POLYREG the following set of equations are obtained
7.000000 a0 + 8.450001 a1 + 14.562500 a2 + 28.224130 a3 = 12.150000
The results from the cubic polynomial are compared with the actual data below. It is
seen that the agreement is very good. The sum of the squares of the error is
significantly less than those of exponential fits (see Example E4.3.2). Hence a cubic 33
polynomial would be a good choice for this problem.
Cont`d
34
4.5 Multidimensional Linear
Regression
To fit a least squares function of the form (4.5.1) to a given set of data points we need
to minimize the error function given by
Fi − (a + bx i + cy i )
N 2
S(a,b,c) = (4.5.2)
i =1
35
Cont`d
We set the derivative of S with respect to a, b and c to zero to obtain the following
equations:
S
= Fi − (a + bxi + cy i )(− 2 ) = 0
a
S
= Fi − (a + bxi + cy i )(− 2 xi ) = 0
(4.5.3)
b
S
= Fi − (a + bxi + cy i )(− 2 yi ) = 0
c
After rearranging and simplifying we obtain:
Na + ( x i )b + ( yi )c = Fi
( x )a + ( x ) b + ( x y )c = x F
i
2
i i i i i
(4.5.4)
( y )a + ( x y ) b + ( y )c = y F
i i i
2
i i i
F = axbyc (4.5.5)
37
Example E4.5.1
The following data is obtained from measurements of pressure, P, Temperature, T, and
density, , of an unknown gas. Find out by using multidimensional regression analysis of
the state equation P = T to determine what this species could be and whether it is an
ideal gas.
Solution
The power law expression given above can be linearized by taking the logarithm of both
sides of the equation to yield
ln P = ln + ln + ln T
Let
F = ln P, x = ln , y = ln T, a = ln , b = , c = .
F = a + bx + cy
which is the same as Equation (4.5.1). The set of equations to be solved are given by
Equation (4.5.4). Inserting N = 20 and the values of the sums given in Tables E4.5.1 into
Equation (4.5.4) gives
Hence
P T
40
Cont`d
41
4.6 Spline Curve Fitting and
Interpolation
Imagine a given data set containing 50 points, how would a polynomial of 49 th
degree look like and how difficult would it be to determine the coefficients. An
alternative method is to fit piece-wise polynomials to a sub set of the whole data (say
two, three or four points at a time), then patch these at the point that is common to
two adjacent intervals as illustrated in Figure 4.6.1. The constraints at the points
joining the two adjacent intervals can be selected as desired but our purpose is to
obtain “smooth” curves.
42
Cont`d
43
Quadratic Splines
Here, the polynomials to be determined for each interval are 2nd degree and given by
First we require that each polynomial must pass through the two end points of that
interval, that is, for each ith interval.
For i = 1, 2, 3, …, n
The two end points of the ith interval are (xi-1, yi-1) and (xi, yi) as seen in Figure 4.6.1
44
Cont`d
Next to make the combined curve smooth we require the first derivative of two
neighboring polynomials be equal at the connecting points that is.
This gives us additional n-1 equations. The last equation needed is usually derived by
requiring the 2nd derivative of either the first polynomial or the last polynomial be zero
at the first or last point respectively. This means of course either the first or the last
curve is a straight line. Hence we take either c 1 = 0 or cN = 0 whichever is more
appropriate for our problem
45
Example E4.6.1
Fit a quadratic spline curve to the 3 intervals of the data given below which is
generated from f(x) = 1/(1+x2).
i 0 1 2 3
x 0.0 0.5 1.0 1.5
y 1.0 0.8 0.5 4/13
Solution:
The Equations 4.6.2 a and 4.6.2b yield (note that for the first polynomial, the
second derivative is taken to be zero. That is c =0 )
1
i=1 a1 + b1x0 = y0
a1 + b1x1 = y1
i=1 b1 + 0 = b2 + 2c2x1
i=2 b2 + 2c2x2 = b3 + 2c3x2
These are the 8-equations to be solved to determine the remaining 8-unknown
coefficients.
Substituting the x and the y values from the table into the above equations and
rearranging we obtain the following matrix equation
47
Cont`d
C = a1 , b1 , a2 , b2 , c2 , a3 , b3 , c3
T
48
References
• Celik, Ismail, B., “Introductory Numerical Methods for Engineering Applications”,
Ararat Books & Publishing, LCC., Morgantown, 2001
• Fausett, Laurene, V. “Numerical Methods, Algorithms and Applications”, Prentice Hall,
2003 by Pearson Education, Inc., Upper Saddle River, NJ 07458
• Rao, Singiresu, S., “Applied Numerical Methods for Engineers and Scientists, 2002
Prentice Hall, Upper Saddle River, NJ 07458
• Mathews, John, H.; Fink, Kurtis, D., “Numerical Methods Using MATLAB” Fourth
Edition, 2004 Prentice Hall, Upper Saddle River, NJ 07458
• Varol, A., “Sayisal Analiz (Numerical Analysis), in Turkish, Course notes, Firat
University, 2001
• http://math.uww.edu/faculty/mcfarlat/inverse.htm
49