Professional Documents
Culture Documents
1 - Linear Regression
1 - Linear Regression
by
CSE, UAP
10/11/22 1
What is Regression?
What is regression? Given n data points ( x1, y1), ( x 2, y 2), ... , ( xn, yn )
best fit y f (x) to the data. The best fit is generally based on
minimizing the sum of the square of the residuals, Sr .
y f (x)
Sum of the square of the residuals
n
( x1, y1)
Sr ( yi f ( xi )) 2
i 1
Figure. Basic model for regression
2
Linear Regression-Criterion#1
Given n data points ( x1, y1), ( x 2, y 2), ... , ( xn, yn ) best fit y a 0 a1 x
to the data.
y
xi , yi
i yi a0 a1 xi x ,y
n n
x ,y
2 2
x ,y
3 3
x1 , y1 i yi a0 a1 xi
x
Figure. Linear regression of y vs. x data showing residuals at a typical point, xi .
n
Does minimizing
i 1
i work as a criterion, where i yi ( a 0 a1 xi )
Observed value Predicted value
3
Example for Criterion#1
Example: Given the data points (2,4), (3,6), (2,6) and (3,8), best fit
the data to a straight line using Criterion#1
8
x y
6
2.0 4.0
y
4
3.0 6.0
2
2.0 6.0
3.0 8.0 0
0 1 2 3 4
x
4
Linear Regression-Criteria#1
This is calculated by the equation of line from two points
x y ypredicted ε = y - ypredicted 8
y
3.0 6.0 8.0 -2.0 4
5
Linear Regression-Criteria#1
Using y=6 as a regression curve
6
Linear Regression – Criterion #1
i 1
i 0 for both regression models of y=4x-4 and y=6.
7
Linear Regression-Criterion#2
n
Will minimizing
i 1
i work any better?
y
x,y
i i
i yi a0 a1 xi x ,y
n n
x2 , y 2
x ,y
3 3
x1 , y1 i yi a0 a1 xi
x
Figure. Linear regression of y vs. x data showing residuals at a typical point, xi .
8
Linear Regression-Criteria 2
Using y=4x-4 as the regression curve
y
2.0 4.0 4.0 0.0 4
3.0 6.0 8.0 2.0
2
2.0 6.0 4.0 2.0
0
3.0 8.0 8.0 0.0 0 1 2 3 4
4 x
i 1
i
4
Figure. Regression curve for y=4x-4, y vs. x
data
9
Linear Regression-Criteria#2
Using y=6 as a regression curve
10
Linear Regression-Criterion#2
4
i 1
i 4 for both regression models of y=4x-4 and y=6.
11
Least Squares Criterion
The least squares criterion minimizes the sum of the square of the
residuals in the model, and also produces a unique line.
n n 2
S r i yi a0 a1 xi
2
i 1 i 1
y
xi , yi
i yi a0 a1 xi xn , y n
x2 , y 2
x3 , y3
x1 , y1
i yi a0 a1 xi
x
Figure. Linear regression of y vs. x data showing residuals at a typical point, xi .
12
Finding Constants of Linear Model
n n 2
i 1 i 1
To find a 0 and a1 we minimize Sr with respect to a1 and a 0 .
S r n
2 y i a0 a1 xi 1 0
a 0 i 1
S r n
2 y i a0 a1 xi xi 0
a1 i 1
giving
n n n
a a x y
i 1
0
i 1
1 i
i 1
i
n n n
a x a x yi xi
2
0 i 1 i
i 1 i 1 i 1
13
Finding Constants of Linear Model
Solving for a 0 and a1 directly yields,
n n n
n x i y i x i y i
i 1 i 1 i 1
a1 2
2
n n
n x i x i
i 1 i 1
and
n n n n
x y x x y
i 1
2
i
i 1
i
i 1
i
i 1
i i
a0 2
n
n
n x x i
2
i
i 1 i 1
Average value of y
Average value of x
a 0 y a1 x
14
Example 1
The torque, T needed to turn the torsion spring of a mousetrap through
an angle, is given below. Find the constants for the model given by
T k1 k 2
Angle, θ Torque, T
Torque (N-m)
0.3
Radians N-m
0.2
0.698132 0.188224
0.959931 0.209138
0.1
1.134464 0.230052
0.5 1 1.5 2
1.570796 0.250965 θ (radians)
1.919862 0.313707
Figure. Data points for Torque vs Angle data
15
Example 1 cont.
The following table shows the summations needed for the calculations of
the constants in the regression model.
Table. Tabulation of data for calculation of important
summations
Using equations described for
T 2 T
a 0 and a1 with n 5
Radians N-m Radians2 N-m-Radians 5 5 5
0.698132 0.188224 0.487388 0.131405 n i Ti i Ti
0.959931 0.209138 0.921468 0.200758 k2 i 1 i 1 i 1
2
5 5
2
1.134464 0.230052 1.2870 0.260986
n i i
1.570796 0.250965 2.4674 0.394215 i 1 i 1
51.5896 6.28311.1921
1.919862 0.313707 3.6859 0.602274
5
i 1
6.2831 1.1921 8.8491 1.5896 58.8491 6.2831
2
9.6091 10 2 N-m/rad
16
Example 1 cont.
Use the average torque and average angle to calculate k1
5 5
_ T i _ i
T i 1
i 1
n n
1.1921 6.2831
5 5
2.3842 10 1 1.2566
Using,
_ _
k1 T k 2
2.3842 10 1 (9.6091 10 2 )(1.2566)
1.1767 10 1 N-m
17
Example 1 Results
Using linear regression, a trend line is found from the data
Can you find the energy in the spring if it is twisted from 0 to 180 degrees?
18
Linear Regression with zero
intercept (special case)
Given
best fit
y a1 x
to the data.
19
Linear Regression (special case cont.)
y a1 x
n n n
n x i y i x i y i
i 1 i 1 i 1
a1 2
n
n
n x i2 x i
i 1 i 1
Is this correct?
20
Linear Regression (special case cont.)
y
xi , yi
i yi a1 xi x ,y
n n
xi , a1 xi
x1 , y1
21
Linear Regression (special case cont.)
i yi a1 xi
Sum of square of residuals
n
Sr i
2
i 1
n 2
yi a1 xi
i 1
22
Linear Regression (special case cont.)
Differentiate with respect to a1
dS r n
2 y i a1 xi xi
da1 i 1
n
2 yi xi 2a1 xi
2
i 1
dS r
0
da1
gives
n
x y i i
a1 i 1
n
i
x 2
i 1
23
Linear Regression (special case cont.)
Does this value of a1 correspond to a local minima or local
maxima? n
x y i i
a1 i 1
n
i
x 2
i 1
dS r
n
2 y i xi 2a1 xi
2
da1 i 1
d 2Sr n
2 xi 0
2
2
da1 i 1
x y i i
a1 i 1
n
x
i 1
2
i
24
Example 2
To find the longitudinal modulus of composite, the following data is
collected. Find the longitudinal modulus, E using the regression model
Table. Stress vs. Strain data E and the sum of the square of the
Strain Stress
(%) (MPa)
residuals.
0 0 3.0E+09
0.183 306
0.36 612 S tress, σ (P a)
2.0E+09
0.5324 917
0.702 1223
0.867 1529 1.0E+09
1.0244 1835
1.1774 2140
0.0E+00
1.329 2446
0 0.005 0.01 0.015 0.02
1.479 2752
Strain, ε (m/m)
1.5 2767
1.56 2896 Figure. Data points for Stress vs. Strain data
25
Example 2 cont.
n
Table. Summation data for regression model i 1
i i
i ε σ ε2 εσ E n
2
1 0.0000 0.0000 0.0000 0.0000 i
2 1.8300×10−3 3.0600×108 3.3489×10−6 5.5998×105 i 1
12
3
4
3.6000×10−3
5.3240×10−3
6.1200×108
9.1700×108
1.2960×10−5
2.8345×10−5
2.2032×106
4.8821×106
i 1
i
2
1.2764 10 3
5 7.0200×10−3 1.2230×109 4.9280×10−5 8.5855×106 12
i
11 1.5000×10−2 2.7670×109 2.2500×10−4 4.1505×107 i 1
12 1.5600×10 −2
2.8960×10 9
2.4336×10 −4
4.5178×10 7
2.3337 108
12
1.2764 10 3
1.2764×10−3 2.3337×108
i 1 182.84 GPa
26
Example 2 Results
The equation 182.84 10 9 describes the data.
27
THE END