Lecture Notes 6

CHAPTER 4
Interpolation and
Approximation
CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Curve Fitting
• Curve fitting is an important and useful statistical
measurement
• Why do we need to to know curve-fitting
1) It is a method for mathematically describing the
relationship between variables.
2) The equations of a line provides a value tool in predicting
or forecasting the outcome of a certain value.
• Variable Relationship:
The relationship between data points determines the curve
fitting properties for each set of data.
 When two variables are linearly related, the
mathematical expression showing their relationship is:
Curve Fitting
y  mx  n
• For this type of relationship, a line best fits the data points
100
Math Score
90
80
70
60
50
85 105 125 145
IQ

Curve Fitting
• Relationships between some data sets are nonlinear, e.g.,
quadratic or exponential. For this type of relationship, a
curve best fits the data points
100
Fuel Consumption
80
60
40
20
25 50 75 100
Velocity

Curve Fitting
• Linear Curve Fitting (Linear Regression)
f(x)
• Given the general form of a straight line x
f ( x)  ax  b
• How can we pick the coefficient that best fits the line to the data?
• What makes a particular straight line a good fit?
• Why does the red line appear to us to fit the trend better?

Curve Fitting
• Consider the distance between the data and points on the line.
f(x)
Error in the case red fitting func.
e4.
Sum(error)=(e1)^2+(e2)^2
e1.
Error=true – our approximation
x
• Add up the length of the all red and blue vertical lines.
• This is an expression of the “error” between data and fitted line.
• The one line that provides a minimum error is said the “best”
straight line

Curve Fitting
• Quantifying Error in a Curve Fit:
• Assumptions:
1) Positive or negative error have the same value (data point is
above or below the line)
2) Weight greater errors more heavily
• We can do both of these things by squaring the distance

f(x)
10 10-1=9
1
(10/11)*100%=91%
x
LEAST SQUARE
102 + (-1)2=101
(100/101)*100%=99%
100
1

Curve Fitting
f(x) (x4,f(x4)) f ( x)  ax  b
Given data
points (x3,y3)
x y e4
e3
x1 y1 (x2,f(x2))
(x4,y4)
(x3,f(x3))
x2 y2 (x1,y1)
y1 e2
x3 y3 e1 (x2,y2)
x4 y4 f ( x1 )  ax1  b (x1,f(x1))
• Blue spots denote data values (x,y)

• Black spots denote points on the fitted line as (x, f(x)).
• Calculate errors (ei=f(xi)-yi) and sum the errors at the four data points

Given data
points
x y
x1 y1
x2 y2
x3 y3
x4 y4
y1
f ( x1 ) CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Curve Fitting
• Summation of square of error:
n
error   (ei ) 2  ( y1  f ( x1 )) 2  ( y2  f ( x2 )) 2  ( y3  f ( x3 )) 2  ( y 4  f ( x4 )) 2
i 1
• Our fit is a straight line, so now substitute,

f ( x)  ax  b
• Then
n n
error   ( yi  f ( xi ))   ( yi  (axi  b)) 2
2
(n: number of data points)
i 1 i 1
• The “best” line has minimum error between the line and data points.
• This is called the least squares approach, since we minimize the square
of the error.

Curve Fitting
n
 
minimize  error= ( yi  ( axi  b)) 2 
 i 1 
• Take the derivative of the error with respect to a and b , set each to
zero
 (error ) n
 2 xi ( yi  axi  b)  0
a i 1
 (error ) n
 2 ( yi  axi  b)  0
b i 1
• Solve for the a and b so that the previous two equations both are equal
to zero. Re-write these two equations:
n n n
a  xi  b xi   xi yi
2
i 1 i 1 i 1
n n
a  xi  b  n   yi
i 1 i 1
Curve Fitting
• If we put these into matrix form
 n
  n 
 n  x i 
 b   i 1
 i 
y
 i 1
    
 n 2  a   
n n
  xi  ( xi )   i i 
( x y )
 i 1 i 1   i 1 
• Example: Fit a straight line to the x and y values
i 1 2 3 4 5 6
x 0 0.5 1.0 1.5 2.0 2.5
y 0 1.5 3.0 4.5 6.0 7.5
• Solution:
n6
6 6
x i  x1  x2  x3  x4  x5  x6  7.5  (x )
i 1
i
2
 ( x1 ) 2  ( x2 ) 2  ( x3 )2  ( x4 )2  ( x5 ) 2  ( x6 )2  13.75
i 1
6
6
y i  y1  y2  y3  y4  y5  y6  22.5 x y
i 1
i i  x1 y1  x2 y2  x3 y3  x4 y4  x5 y5  x6 y6  41.5
i 1
Curve Fitting
• Substituting the values into equation
 n
  n 
 n  x i 
 b   i 1
 i 
y
 6 7.5   b   22.5
a3
 i 1
    7.5 13.5  a    41.5 b0
 n 2  a   
n n
    
  xi   i i 
( x ) ( x y ) f ( x)  3 x  0
i 
 i 1 i 1   i 1 
• Example: Fit a straight line to the x and y values
i 1 2 3 4 5 6
x 0 0.5 1.0 1.5 2.0 2.5
y -0.4326 -0.1656 3.1253 4.7877 4.8535 8.6909

Curve Fitting
• Following Table displays the summations
x y x2 x.y
0 -0.4326 0 0
0.5 -0.1656 0.25 -0.0828
1 3.1253 1 3.1253
1.5 4.7877 2.25 7.18155
2 4.8535 4 9.707
2.5 8.6909 6.25 21.72725
= 7.5 20.8592 13.75 41.6583
• Substituting the values:
 n
  n 
 n  xi 
 b   i 1
 yi 
6 7.5   b   20.86 
a  3.56
 i 1
     7.5 13.75  a    41.66  b  0.98
 n 2  a   
n n
    
  xi  ( xi )   i i 
( x y ) f ( x)  3.56 x  0.98
 i 1 i 1   i 1 

Curve Fitting
• Home Exercise: Fit a straight line to the x and y values
i 1 2 3 4 5 6 7
x 1 2 3 4 5 6 7
y 0.5 2.5 2.0 4.0 3.5 6.0 5.5 Answer: y  0.0714  0.839 x

Curve Fitting
• Quantification of Error of Linear Regression (Curve fitting):
• In linear regression, sum of the squares is defined as:
n n
Sr   (ei )   ( yi  a0  a1 xi ) 2
2
i 1 i 1
• Total sum of the squares around the mean of dependent variable is represented S t .
n y i
St   ( yi  y ) 2
where y i 1
(mean)
i 1 n
• Then, (r2) (Coefficient of determination) is calculated as follows:
St  S r
r2 
St
• For perfect fit Sr=0 and r2=1
• If Sr= St and r2=0 (Poor fit)

Curve Fitting
• Example: It is known the tensile strength of a plastic increases as a
function of the time it is heat treated. The following data is collected
through experiment.
i 1 2 3 4 5 6 7 8 9
Time (min) 10 15 20 25 40 50 55 60 75
Tensile strength (MPa) 5 20 18 40 33 54 70 60 78
• a) Fit a straight line to this data.
• b) Find out whether this fit expresses the data well.
• c) Determine the tensile strength at a time of 32 min.

Curve Fitting
* 2
xi yi xi2 xiyi y i* S t =(y i -y)2 Sr =(y i -y i ) y*  a0  a1 x
10 5 100 50 11.4 1369 41.06
 n
  n 
15 20 225 300 16.7 484 10.87  n  xi  
 a0   i 1 
yi
 i 1
    
20 18 400 360 22 576 15.98  n n
2   a1   n

25 40 625 1000 27.3 4 161.47
  xi  ( xi )   i i 
y x
 i 1 i 1   i 1 
40 33 1600 1320 43.2 81 103.59
 9 350   a0   378 
50 54 2500 2700 53.8 144 0.054 350 17700   a   19030 
  1  
55 70 3025 3850 59.1 784 119.62
a0  0.8179
60 60 3600 3600 64.4 324 18.99 a1  1.059
75 78 5625 5850 80.2 1296 5.03
yi *  0.8179  1.059 xi
350 378 17700 19030 5062 476.7
378
y  42
9

Curve Fitting
• b)
5062  476.67
r2   0.9058 Close to 1, good fit
5062
• c)
Tensile Strength=0.8179+1.059×(Time)
For time=32
Tensile Strength=0.8179+1.059×32=34.7 MPa

Curve Fitting
• Matlab Code for Linear Regression
Example: x=[0 1 2 3 4 5]
Fit a straight line to given data points y=[2.5 6 5 5.5 8 12]
Calculate r2 A(1,1)=length(x)
Plot the given data points and A(1,2)=sum(x)
A(2,1)=sum(x)
developed straight line together
A(2,2)=sum(x.^2)
B(1)=sum(y)
x 0 1 2 3 4 5 B(2)=sum(y.*x)
C=inv(A)*B'
y 2.5 6 5 5.5 8 12
b=C(1)
a=C(2)
yfit=a*x+b
ybar=mean(y)
Sr=sum((y-yfit).^2)
St=sum((y-ybar).^2)
r2=1-Sr/St
plot(x,y,'o',x,yfit)
grid on

Curve Fitting
• Solution
a b r2
1.5429 2.6429 0.8011
12
11
y=1.5429+2.6429x 10
2
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

Curve Fitting
• Linearization of Nonlinear Relationships
y y
x x
• Linear regression provides a powerful technique for fitting a “best” line
to data if the relationship between the dependent and independent
variables is linear. However, this is not always the case.
• Figures show some data is obviously curvilinear. For this cases,

techniques such as polynomial regression are appropriate (This
technique will be described in further sections). For others,
transformations can be used to express the data in a form that is
compatible with linear regression.
Curve Fitting
• One example is the exponential model.
y  a1eb1x
• Another example of a nonlinear model is the simple power equation.
y  a2 xb2
• The third example is hyperbolic equation
x
y  a3
b3  x
• Nonlinear regression techniques are available to fit these equations to
experimental data directly. However, a simpler alternative is to use
mathematical manipulations to transform the equations into a linear
form.
form Then simple linear regression can be employed to fit the
equations to data.
Curve Fitting
• Linearization of Equations
1) y  a1eb1x
taking the natural logarithm, the equation yields;
ln y  ln a1  b1 x ln e ln y  c  b1 x
c 1
2) y  a2 x b2
is linearized by taking its natural logarithm to give.
ln y  ln a2  b2 ln x ln y  c  b2 ln x
c

Curve Fitting
3) x
y  a3
b3  x
is linearized by inverting it to give;
1 b3  x 1 b3 1 1 1 1
   c d
y a3 x y a3 x a3 y x
(c) (d)
•Example: Fit a power equations ( y  ax)b to given data points
x 1 2 3 4 5
y 0.5 1.7 3.4 5.7 8.4

Curve Fitting
• Solution: First linearize the power equation
y  axb ln y  ln a  b ln x ln y  a0  a1 ln x
 a0  a1
n
error= (ln yi  (a0  a1 ln xi )) 2
i 1  a0 n 
n n n
 (error)
a  a1  ln xi   ln yi
n
 2 (ln yi  a0  a1 ln xi )  0 0
a0 i 1 i 1 i 1 i 1
n n n
 (error) n
 2 ln xi (ln yi  a0  a1 ln xi )  0 a0  ln xi  a1  (ln xi )   (ln yi )(ln xi )
2
a1 i 1 i 1 i 1 i 1
 n
  n

 n  ln x i 
 a0  
 ln yi 
 i 1
   i 1

 n 2   a1   
n n
  ln xi  (ln xi )   (ln xi )(ln y )

i 
 i 1 i 1   i 1 
Curve Fitting
• We can calculate the needed values as follows:

n n
xi yi lnxi lnyi (lnxi)2 (lnxi)(lnyi)    
 n  ln xi 
a  
 ln y i 
1 0.5 0 -0.693 0 0  i 1
 0   i 1

n 2   a1   
n n
2 1.7 0.693 0.531 0.480 0.368  ln xi  (ln x )

i   (ln xi )(ln y )
i 
 i 1 i 1   i 1 
3 3.4 1.099 1.224 1.207 1.342
4 5.7 1.386 1.741 1.923 2.413
5 8.4 1.609 2.128 2.590 3.425  5 4.787   a0   4.931

 4.787 6.200   a    7.548
Σ= 4.787 4.931 6.200 7.548   1  
a0  0.69 a0  ln a a  e a0  e 0.96  0.5016

a1  1.75 b  a1  1.75 y  0.5016 x1.75

Curve Fitting
• Example: The yield stress of most metals is sensitive to the rate at

which the material is loaded. The data below gives the yield stress of a
certain steel at various strain rates.
Strain rate (s-1) 0.00007 0.0002 0.05 0.8 4.2 215 3500
Yield stress (MPa) 345 362 419 454 485 633 831
• The yield stress as a function of strain rate ca be modeled as:
 y  a0  a1 t 0.16
• where σy is the yield stress in (MPa) and εt is the strain in s-1 .

Determine the constant a0 and a1.

Curve Fitting
y  a0  a1 x m m  0.16
n
error= ( yi  (a0  a1 xi m )) 2
i 1  a0 n 
n n n
 (error ) n
a0
 2 ( yi  a0  a1 xi m )  0 a
i 1
0  a1  xi   yi
i 1
m
i 1
i 1
n n n
 (error ) n
 2 xi m ( yi  a0  a1 xi m )  0 a0  xi  a1  ( xi )   yi xi m
m m 2
a1 i 1 i 1 i 1 i 1
n
   n 
 n  xm
i 
 a    i  y
 i 1
  0    i 1 
 n m 2 m   a1   
n n
  xi   i i 
m
xi  x y
 i 1 i 1   i 1 

Curve Fitting
xi yi xi0.16 xi0.32 xi0.16yi yi*
0.00007 345 0.2164 0.0468 74.7 351.7  7 9.3663   a0   3529 

9.3663 22.204   a   6036.4 
0.0002 362 0.2560 0.0655 92.7 357.1   1  
0.005 419 0.6192 0.3834 259.4 406.4 a0  321.3
0.8 454 0.9648 0.9311 438.1 453.4 a1  136.3
4.2 485 1.2581 1.5829 610.2 493.3 y*  321.3  136.6 x 0.16

215 633 2.3615 5.5767 1494.8 643.2 n
S r   ( yi  yi *) 2  451.96
3500 831 3.6902 13.618 3066.6 823.8 i 1
n
Σ= 3720.1 3529 9.3663 22.204 6036.4 St   ( yi  y )  179100
i 1
St  S r 179100  451.96
r2    0.9975 Good Fit !!!
St 179100

Curve Fitting
• Polynomial Regression: The least squares procedure can
be extended to fit the data to a higher-order polynomial.
polynomial For example,
suppose that we fit a second-order polynomial or quadratic:
quadratic
y  a0  a1 x  a2 x 2  error
error (ei )= y  a0  a1 x  a2 x 2
• Sum of the squares of the residuals is;
n n
Sr   ei   ( yi  a0  a1 xi  a2 xi 2 ) 2
2
i 1 i 1
n n n n
Sr n
a0
 2 ( yi  a0  a1 xi  a2 xi 2 )  0 a
i 1
0  a1  xi  a2  xi   yi
i 1 i 1
2
i 1
i 1
n n n n
Sr n
a0  xi  a1  xi  a2  xi   yi xi
2 3
 2 xi ( yi  a0  a1 xi  a2 xi 2 )  0
a1 i 1 i 1 i 1 i 1 i 1
n n n n
Sr n
a0  xi  a1  xi  a2  xi   yi xi 2
2 3 4
 2 xi 2 ( yi  a0  a1 xi  a2 xi 2 )  0
a2 i 1
i 1 i 1 i 1 i 1

Curve Fitting
• In matrix form:
n n
   n 
 n x i  x 2
i    i y
 i 1 i 1
  a0   i 1 
 n n n
3    n

  xi  i  xi   a1     yi xi 
2
x
 i 1 i 1 i 1
  a2   i 1 
 n 2 n n
4  n 2
  xi x    yi xi 
3
i xi 
 i 1 i 1 i 1   i 1 
• Example: Fit a quadratic least-squares (2nd order polynomial

function) to given data
xi 0 1 2 3 4 5
yi 2.1 7.7 13.6 27.2 40.9 61.1

Curve Fitting
xi yi xi2 xi3 xi4 yixi yixi2
0 2.1 0 0 0 0 0  6 15 55   a0   152.6 
15 55 225  a    585.6 
1 7.7 1 1 1 7.7 7.7   1  
55 225 979   a2   2488.8
2 13.6 4 8 16 27.2 54.4
3 27.2 9 27 81 81.6 244.8 a0  2.4786

4 40.9 16 64 256 163.6 654.4 a1  2.3593
5 61.1 25 25 625 305.5 1527.5 a2  1.8607
Σ= 15 152.6 55 225 979 585.6 2488.8
y  2.4786  2.3593x  1.8607 x 2

Curve Fitting
• y=[2.1 7.7 13.6 27.2 40.9 61.1];
• x=[0 1 2 3 4 5];
• n=length(x);
• sx=sum(x);
• sx2=sum(x.^2);
• sx3=sum(x.^3);
• sx4=sum(x.^4);
• sy=sum(y);
• sxy=sum(x.*y);
• sx2y=sum((x.^2).*y);
• A=[n sx sx2; sx sx2 sx3; sx2 sx3 sx4];
• f=[sy; sxy; sx2y];
• c=inv(A)*f;
• a0=c(1)
• a1=c(2)
• a2=c(3)
• ffit=a0+a1*x+a2*x.^2
• plot(x,y,'o',x,ffit)
• grid on
• Sr=sum((y-ffit).^2)
• ybar=mean(y) a0  2.4786
• St=sum((y-ybar).^2)
• r2=1-Sr/St a1  2.3593 r2=0.9985
a2  1.8607

Lecture Notes 6

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture Notes 6

Uploaded by

Copyright:

Available Formats

CHAPTER 4

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Given the general form of a straight line x

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Error=true – our approximation

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Blue spots denote data values (x,y)

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

f ( x1 ) CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Our fit is a straight line, so now substitute,

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Example: Fit a straight line to the x and y values

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• a) Fit a straight line to this data.

• b) Find out whether this fit expresses the data well.

• c) Determine the tensile strength at a time of 32 min.

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Tensile Strength=0.8179+1.059×32=34.7 MPa

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Figures show some data is obviously curvilinear. For this cases,

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

•Example: Fit a power equations ( y  ax)b to given data points

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

  ln xi  (ln xi )   (ln xi )(ln y )

• We can calculate the needed values as follows:

2 1.7 0.693 0.531 0.480 0.368  ln xi  (ln x )

4 5.7 1.386 1.741 1.923 2.413

5 8.4 1.609 2.128 2.590 3.425  5 4.787   a0   4.931

a0  0.69 a0  ln a a  e a0  e 0.96  0.5016

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Example: The yield stress of most metals is sensitive to the rate at

• The yield stress as a function of strain rate ca be modeled as:

• where σy is the yield stress in (MPa) and εt is the strain in s-1 .

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

0.00007 345 0.2164 0.0468 74.7 351.7  7 9.3663   a0   3529 

4.2 485 1.2581 1.5829 610.2 493.3 y*  321.3  136.6 x 0.16

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Example: Fit a quadratic least-squares (2nd order polynomial

yi 2.1 7.7 13.6 27.2 40.9 61.1

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

3 27.2 9 27 81 81.6 244.8 a0  2.4786

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

You might also like