Multiple Regression

The Multiple Regression Equation:

ŷ  a  b1 x1  b2 x 2

ŷ : dependent var.
x1, x2 : independent var.
To find constants a, b1, b2 :

Find “ b1, b2 ” by solve 2 equations:

(y-y) (x1-x1) = b1(x1-x1)2 + b2(x1-x1) (x2-x2)
(y-y) (x2-x2) = b1(x1-x1) (x2-x2) + b2(x2-x2)2

Find “a” by equation.

a  y  b1 x1  b2 x 2
y  x1  x2
n n n
Test the significant of the Model “F” test:

(1) SST =  (y - y)2

(2) SSR = b1 [ (y – y) (x1 – x1)] + b2 [(y – y) (x2 – x2)]
(3) SSE = SST – SSR
(4) ANOVA Table:
Source of Sum of Degrees of Mean Sum F-ratio
variation Squares Freedom of Squares Fc
S.S D.F M.S.S

Regression (R) SSR 2 MSSR =SSR

2 MSSR

MSSE = SSE MSSE
Error (E) SSE n-3
n-3
Total (T) SST n–1

(5) Ftable (Critical value) = F (2, n-3, )

(6) Compare between Fc and Ft :

Fc > Ft Fc < Ft
Model is valid for Model is not valid for
predicting (y) predicting (y)
• The standard error of estimate : S yx x= MSSE
1 2

• The determination coefficient R2 = SSR

SST
means : The independent variables x1 and x2 explain
% of the variability in the dependent variable (y).

• The coefficient of correlation (r) = R2

 Example
Y 1 1 2 2 3 3
X1 10 8 6 5 4 3
X2 2 4 6 5 7 6
 4X5 (5)2 5X6 4X6 (6)2 (4)2
1 2 3 4 5 6 7 8 9 10 11 12

Y-Y X1-X1 X2-X2 (Y-Y) (X1-X1) (Y-Y) (X2-X2)2 (Y-Y)2

Y X1 X2 Y-2 X1-6 X2-5 (X1-X1)
(X1-X1)2
(X2-X2) (X2-X2)

1 10 2 -1 4 -3 -4 16 -12 3 9 1
1 8 4 -1 2 -1 -2 4 -2 1 1 1
2 6 6 0 0 1 0 0 0 0 1 0
2 5 5 0 -1 0 0 1 0 0 0 0
3 4 7 1 -2 2 -2 4 -4 2 4 1
3 3 6 1 -3 1 -3 9 -3 1 1 1
12 36 30 Zero Zero Zero -11 34 -21 7 16 4
Y= = 2 x 1= = 6
x 2= = 5
Find b1, b2 by solve 2 equations
(Y-Y) (X1-X1)= b1 (X1-X1)2+ b2 (X1-X1) (X2-X2)
(Y-Y) (X2-X2)= b1 (X1-X1) (X2-X2) + b2 (X2-X2)2
-11= 34 b1 – 21 b2 1 X21
7= -21 b1 + 16 b2 2 X34
-231= 714 b1 – 441 b2
238= -714 b1 + 544 b2
7= 103 b2
b2 = = 0.07
7 = -21 b1 + 1.12
7 – 1.12 = -21 b1
5.88 = -21 b1
b1= = -0.28
Find “a”
a = y – b 1 X 1 – b2 X 2
= 2 – (- 0.28) (6) – (0.07) (5)
= 2 + 1.68 – 0.35 = 3.33
The Regression Equation:
^
Y = a + b 1 X 1 + b2 X 2

^
Y= 3.33 – 0.28 X1 + 0.07 X2
“F” Test:

1) S.S.R= b1 (Y –Y ) (X1-X1) + b2 (Y-Y) (X2-X2)

= -0.28 (-11) + 0.07 (7)
= 3.57
2) S.S.T= (Y-Y)2 = 4
3) S.S.E= S.S.T – S.S.R
= 4 – 3.57= 0.43
4) ANOVA Table:

analysis of variance
1 2 3 4 5
Source of S.S D.F M.S.S Fc
variation
R 3.57 2 ÷
1.79
n-3 12.45
E 0.43 6-3 ÷ 0.14 ÷
3
T 4
5) Ft (2 ,3) = 9.55

Column Row
6) Fc > Ft
12.45 9.55
The Regression equation is Significant
can be used for Prediction.
Estimate Y when X1=5 & X2 = 10

Y = 3.33 – 0.28 (5) + 0.07 (10) = 2.63

R2123= == 0.89
89% of The Total Variation in (Y) is explained by
The Variation in X1 & X2 and 11% by The other
Wise.

r123= = = 0.94 Strong

Standard error of estimate:
SY X X =
1 2

= = 0.378
 Exercises
a- Find Multiple Regression equation
b- Test of significance of Regression equation.
c- Estimate (y) when x1 = 10 & x2 = 15
d- Find coefficient of determination (R2) and
coefficient of Correlation (R) .
1 48 40 31 22 19 Y

9 8 6 5 2 X1

12 14 16 18 20 X2

2 5 4 3 2 1 Y

8 7 5 4 2 X1

3 4 5 7 10 X2