Solving

MULTIPLE
REGRESSION
using Matrix
 2

Multiple Linear Regression Models

In general, the dependent variable or response Y may be related to k
independent or regressor variables. The model

Eq. (1)

Is called a multiple linear regression model with k regressor variables.

The parameters 𝛽𝑗, 𝑗 = 0, 1, … , 𝑘, are called the regression coefficients.
This model describes a hyperplane in the k-dimensional space of the
regressor variables {𝑋𝑗 }. The parameter 𝛽𝑗 represents the expected
change in response Y per unit change in 𝑥𝑗 when all the remaining
regressors 𝑥𝑖 (𝑖 ≠ 𝑗)are held constant.
 3

Multiple Linear Regression Models

Matrix Approach to Multiple Linear Regression
Suppose the model relating the regressors to the response is

In matrix notation this model can be written as

 4

Multiple Linear Regression Models

Matrix Approach to Multiple Linear Regression

where
 5

Multiple Linear Regression Models

Matrix Approach to Multiple Linear Regression

We wish to find the vector of least squares estimators that minimizes:

The resulting least squares estimate is

 6

Multiple Linear Regression Models

Matrix Approach to Multiple Linear Regression
The fitted regression model is

In matrix notation, the fitted model is

The difference between the observation yi and the fitted value 𝑦ො𝑖 is
a residual, say, 𝑒𝑖 = 𝑦𝑖 − 𝑦ො𝑖 .The (n x 1) vector of residuals is denoted
by
 7

Multiple Linear Regression Models

Example 2

In example 1, we illustrated fitting the multiple regression model

Where y is the observed pull strength fro a wire bond, x 1 is the wire
length, and x2 is the die height. The 25 observations are in Table 2.
Use the matrix approach to fit the regression model above to these
data. The model matrix X and y vector for this model are
 8

Example 2
 9

Multiple Linear Regression Models

Example 2
 10

Multiple Linear Regression Models

Example 2

Therefore, the fitted regression model with the regression

coefficients rounded to five decimal places is

This is identical to the results obtained in Example 1.

 11

Multiple Linear Regression Models

Example 2
This regression model can be used to predict values of pull strength for various
values of wire length (x1) and die height (x2). Fitted values 𝑦ො𝑖 can be obtained by
substituting each observation ( 𝑥𝑖1 , 𝑥𝑖2 , I = 1, 2, 3…, n into the equation. For
example, the firts observation has x11 = 2 and x12 = 50, and teh fitted value is

The corresponding observed values is 𝑦𝑖 = 9.95. 𝑇ℎ𝑒 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 corresponding to

the first observation is
 12

Multiple Linear Regression Models

Example 2
Table 3. Observations, Fitted Values, and Residuals for Example 2
 13

Table 4. Minitab Multiple Regression Output for the Wire Bond Pull Strength Data