Computational Techniques Computational Techniques

Module 5: Module 5: Regression and Interpolation Regression and Interpolation Module 5: Module 5: Regression and Interpolation Regression and Interpolation
Dr. Niket Kaisare
Department of Chemical Engineering Department of Chemical Engineering
Indian Institute of Technology - Madras
Regression Example Regression Example Regression Example Regression Example
Given the following data:
x 0.8 1.4 2.7 3.8 4.8 4.9
y 0.69 1.00 2.02 2.39 2.34 2.83 y 0.69 1.00 2.02 2.39 2.34 2.83
Regression:
Ob h l Obtain a straight line
that best fits the data
Interpolation Example Interpolation Example Interpolation Example Interpolation Example
Given the following data:
x 0.8 1.4 2.7 3.8 4.8 4.9
y 0.69 1.00 2.02 2.39 2.34 2.83 y 0.69 1.00 2.02 2.39 2.34 2.83
Interpolation:
J h d d Join the dots and
find a curve passing
through the data.
Regression vs. Interpolation Regression vs. Interpolation Regression vs. Interpolation Regression vs. Interpolation
x 0.8 1.4 2.7 3.8 4.8 4.9
In regression we are interested in fitting a
y 0.69 1.00 2.02 2.39 2.34 2.83
In regression, we are interested in fitting a
chosen function to data
y =045+047x
In interpolation, given finite amount of data, we
y = 0.45 + 0.47x
p , g ,
are interested in obtaining new data-points
within this range.
At x = 2.0, y = 1.87
Example: Kinetic Rate Constants Example: Kinetic Rate Constants
((R i ) R i ) ((Regression) Regression)
Experiments with conversion measured at
various temperatures
E
6
a
c e k r
RT
E
) (
0
=
2
4
r
)
) ln(
1
) ln( ) ln(
0 a
c
T R
E
k r +
|
.
|

\
|

+ =
-2
0
l
o
g
(
r
Slope =E/R
T R
. \
-3.5 -3 -2.5 -2 -1.5
x 10
-3
-4
-1/T
Intercept =ln(k
0
)
Example: Viscosity of Oil Example: Viscosity of Oil
((I l i ) I l i )
Viscosity of lubricant oil was measured between
((Interpolation) Interpolation)
-20 to 200 degrees Celcius in steps of 20
o
C.
Interpolation is used if viscosity is desired at an
intermediate temperature
General Setup General Setup General Setup General Setup
Let x be an independent variable
and y be a dependent variable
Given the data:
( ) ( ) ( )
N N
y x y x y x , , , , , ,
2 2 1 1

Find parameters togetabestfitcurve
( ) u ; x f y = ( ) ; f y
Regression vs. Interpolation Regression vs. Interpolation Regression vs. Interpolation Regression vs. Interpolation
Regression Interpolation
Choose a function
form for f(x;)
Various standard
function forms exist
Regression Interpolation
form for f(x;)
For a given , obtain
function forms exist
The interpolating

the values from

the model
function passes
through all the points
i
y
The best minimizes
the error
Can be used to fill-
in the data at new
i i
y y
points
i i
y y
Regression vs. Interpolation Regression vs. Interpolation Regression vs. Interpolation Regression vs. Interpolation
Curve Fitting Joining the dots g J g
Obtain a functional
form to fit the data
Obtain the value of y
at intermediate point p
Outline: Regression Outline: Regression Outline: Regression Outline: Regression
Linear Regression in One Variable Linear Regression in One Variable
Linear Regression in Multiple Variables
Polynomial Regression
A l i d E i Analysis and Extension
Non-Linear Regression g
Linear Regression: One Variable Linear Regression: One Variable Linear Regression: One Variable Linear Regression: One Variable
Model:
( ) u ; x f y =
Actual Data:
P d
( ) ( ) ( )
N N
y x y x y x , , , , , ,
2 2 1 1

( ) u f
Prediction:
Errors:
( ) u ;
i i
x f y =
i i i
y y e =
( )
i i i
e x f y + =

;u
Mean / Variance:
y
i

( )
2

N
x
x
i
=
( )
) 1 (
2

N
x x
s
i
x
x
s = o
Extension to Multi Extension to Multi--Variables Variables Extension to Multi Extension to Multi Variables Variables
Let x
1
, x
2
, , x
n
be n variables.
Let there be N data points for each:
( )
n
y x x x , ; , , ,
1 1 21 11
( )
( )
n
n
y x x x
y x x x
, ; , , ,
, ; , , ,
2 2 22 12
1 1 21 11

( )
N nN N N
y x x x ; , , ,
2 1

Obtain for
( ) x; f y =
Linear Linear Regression (multi Regression (multi--variable) variable) Linear Linear Regression (multi Regression (multi variable) variable)
Data
( )
i i i i
y w u x ; , ,
Data
Model
( )
i i i i
y ; , ,
w a u a x a a y
3 2 1 0
+ + + =
Error
( )
i i i i i
w a u a x a a y e
3 2 1 0
+ + + =
Least Squares Criterion
2
( )

|
|
|
|

|
+ + +
N
i i i i
w a u a x a a y
2
3 2 1 0
min ( )

=
|
|
.

\
i
e
i i i i
a a a a
i
1
3 2 1 0
, , ,
3 2 1 0

Linear Regression (alternate) Linear Regression (alternate) Linear Regression (alternate) Linear Regression (alternate)
y a w u x
(
(

(
(

(
(

1 0 1 1 1
1

y
a
a w u x
(
(
(

=
(
(
(

(
(
(

2
2
1 2 2 2
1

Y
N
X
N N N
y a
a
w u x
(
(

(
(

(
(

u
3
2
1
Y X
u
Least Squares
( ) Y X X X
T T
1
= u ( ) Y X X X = u
Polynomial / Polynomial / Functional Functional Regression Regression Polynomial / Polynomial / Functional Functional Regression Regression
Example: Specific heat as a function of T p p
Methane: c
p
= 85.8 + 1.126e-2 T 2.1141e-6 T
2
Example: Antoines vapor pressure relationship
b

( )
c T
b
a p
sat
+
= ln
Polynomial / Functional Regression Polynomial / Functional Regression Polynomial / Functional Regression Polynomial / Functional Regression
(
(

2
1
2
1 1
1 x x x
3
3
2
2 1 0
x a x a x a a y + + + =
Model:
(
(
(
(

=
3
2
2
2 2
1 x x x
X

3 2 1 0
x a x a x a a y + + +
(
(

3 2
1
N N N
x x x
M d l
( )
( )
(
(
(

x
x
x x
x x
2
1
2
1
1
1
ln 1
ln 1
1
x a
a
x a a y
3
2
1 0
) ln( + + + =
Model:
( )
( )
(
(
(
(

=
N N
x
x x
X
1
2 2
ln 1
2

x
( )
(

N
x
N
x x
N
1
ln 1
Outline: Interpolation Outline: Interpolation Outline: Interpolation Outline: Interpolation
Polynomial fitting and limitations Polynomial fitting and limitations
Lagrange interpolating polynomials
Newtons methods
S li i l i Spline interpolation
Lagrange Polynomials Lagrange Polynomials Lagrange Polynomials Lagrange Polynomials
( )( ) ( ) ( ) ( )
( )( ) ( ) ( ) ( )
+

=
N i i
i
x x x x x x x x x x
P
... ...
1 1 2 1
( )( ) ( ) ( ) ( )
[
+
|
|
|

|

=

j
N i i i i i i i
x x
x x x x x x x x x x ... ...
1 1 2 1
[
=
|
.

\

i j
j i
x x
The interpolating
polynomial becomes
N N
P y P y P y x f + + + = ... ) (
2 2 1 1
Newtons Divided Differences Newtons Divided Differences Newtons Divided Differences Newtons Divided Differences
| |
i i
y y
i i y

+
+1
1 | |
| |
i i
i i
i i y i i y
i i i y
x x
y y
i i y
+ + +
= + +

= +
+
+
1
1
] , 1 [ ] 1 , 2 [
1 2
, 1
| |
| |
i i
i i i y i i i y
i i i i y
x x
i i i y
+ + + + +
= + + +

= + +
+2
] , 1 , 2 [ ] 1 , 2 , 3 [
, 1 , 2 , 3
, 1 , 2
| |

i i
x x
y

+3
, , ,
| | | | | | ... , 1 , 2 , 3 , 4 , 1 , 2 , 3 , 1 , 2 ,
3 2 1 1 0
= = = = y c y c y c y c
... ) )( ( ) ( ) (
2 1 2 1 1 0
+ + + = x x x x c x x c c x f