REVIEW PAPER REPORT

Multicollinearity and Regression Analysis

1
Abstract:

There is compulsory to have an interdependence in between the predictors and

response in the Regression analysis, b ut having interrelation in between predictors is

.

obviously disagreeable. There are many predictors involved in the model of regression that

relies on some factors like experience we got and historical data from our previous

figures , considered like wise . The decision we should be choosing of which predictors

depends on the researcher. We can define Multiicollinearity as in the situation in which we

are having 2 variables or more than 2 variable which are explanatory in model of

regression
i and linearly strongly high related. But this increases standard error of

coefficient that means the coefficient of some or all independent variables may be differ

from value Zero. on the other hand, By increasing Standard errors, due to which some

variables are negligible by Multi collinearity when these variable must be considered
i

important. So, for this research, we will discusse Multiicollinearity, and its reasons and it’s

consequence on validation of the model of Regression

2
1 Introductiio n
I .

In model of analysis of the regression , presumptions for model of regression

comes, that is Multiicollinearity, uncertain, Heterogeneity, Linearity and autocovariance. If

any of the presumption is break then the model then the model of regression is not acceptable

and will not be more validated in estimation of parameters of the given population.

We will be studying In the conducted research, we will be focusing on the

Multiicollinearity, that is one of the essential reason for the violation of basic presumption of

the successful model of regression. Multiicollinearity is considered as the situation having 2

ore more than 2 variables which are explanatory in the model of regression are linearly high

co-related. Lower Multiicollinearity causes a lot of problem but medium or high level

Multiicollinearity is a problem that can be solved. focusing on Multiicollinearity as a

breaking of any of the essential asumption for the sucessful regression model ,supposition of

sucessful regression of model. Multiicollinearity comes when 2 or more than 2 variable

which are independent in the model of regression are co-r lated. sometimes creates a problem
e

but when there is highly fluctuation with in corelation then it will be a problem to be solved.

Multiicollinearity take place when variables which are independent  in model of

regression are inter related . If there is no static relationship among the variables then, So, it is

called Orthog onal.

o

When predicted variable s are not orthogonal then the Multiicollinearity will be
e

3
 Noticed in cases mentioned below of regressio n. .

1) when the variable is added or deleted then the lot of changes happens in the estimated

Interdependence

2) when the data point is added or dropped or removed then the lot of changes appears

in the Interdependence.

Multiicollinearity may appear if:

1) The prior expectation do not follow to the signs in algebraic for the estim ated .

Interdependence.

2) There is large standard errors in the Interdependence of the variables that are

considered to be important.

The researcher doesn’t know about the Multiicollinearity except collection of data has not

done

There are 2 types of Multii collinearity has been found.

i

1) . When the collected data is purely observed and the experiment is poorly designed,

due to negligence of the researcher then it is called Data- based Multi collinearity.
i

2) When the new independent variables are generated from the one or more existing

variables then it is called structural Multiicollinearity. For example, y3 from y. It is in-fact

mathematical tool artiifact that leads to Multiicollinearity.

Focusing on the reasons of the Multiicollinearity exist between chosen variablees on

hypothesis testing taken decision. In analysis of regression there are many suppositions

about the model, named, linearity, autocorrelation. And Multiicollinearity), If 1 or more

than it supposition is over ruled, then reliability of model not more exist & not acceptable

in measuring the parameter of the population.

.

4
 2 LITERATURE REVIEW

Co-relation of predictors & effect on model of regression

.

What is the effect of correlation that is among the predictors have in the subsequent

conclusions and model of regression? To show the consequences of the co-relation in btw

the predictors on validity of obtained model. The Correlation can be low or high, we are
.

using 2 sets of data 1 of the set with high correlation. between predictors & other set of the
.

data with low correlation between predictors. The first set of data results the below

mentioned regression information for the analysis of regression

We have taken the two samples. One is multiple fitting model containing both of the

predictor variables then simple fitting models with one of variable predictor each time,

the coefficients of correlation among two predictors was very low (-0.038) that is found in

the multiple model of regression include both variables . Results can be seen in Table#1

mentioned below.

above table ( 1 ) shows. that the value of ( T )for (X1 ) when considered individually itself

in model coming not far away and is very close to T Value coming when both of the

predictors variables were included, same for ( X2 ) the T-value if their values did not
i

come unique/separate from its value when both of the values of included predictors . Then

5
parameter on decision test will be same for both. On other side , the standrd error of

Interdependence not been changed drastically, for x1 ( 3.3 - 3.46 ) from model of simple

fitting and the model of multiple fitting and for ( X2 ) variables from ( O.638 - O.662.) All

this can be viewed as a result of low interdependence among variables. Coming towards set

of data we had taken on second basis that is representing very h igh interdependence among
i

variables (O.996) that is seen in below Table (2). Coefficients of first data set of model.
s

table 2 showing there is large change in values coefficient for X-11 ranging between

(-0.309 - 2.71) in multiiple fitting models & the simple fitting model . Further to that
s

addition, we can notice that there is a large increase in the standard error of

Interdependence. For x1 it is from (O.279 - 2.96) and for x2 from (O.579 - 6.24) when we

compared the simple & multiple models. Then the bond in-between predictor variable for

both of 1st & the 2nd data set can be seen below.

6
 Fitted Line Plot
x1 = 1.542 - 0.00734 x2

2.0 S0.558606
R-Sq0.1%
R-Sq(adj)0.0%
1.8

1.6

1.4

1.2

1.0

345678910

x2

Graph#1: we are having correlation low Graph#2 we are having correlation high

This comparing of predictors corelation with low value and predictors with high corelation

value is the proof of the effect of interdependence on standrd . error of Interdependence was

highly changing ( S.D.E ) standard error of Interdependence for the 2 nd set of data, that

leads automatically the analyst for the conclusions which are wrong on model. All .or some

predictors. will become unidentifiable when they must be identifiable because of inflation

in standrd error for variables Interdependence. As a summary having the high correlation

between variables will avoid the researchers analyzing most important variables for the

involvement in model.

Analyzing of Multiicollinearity .

Most of indicators are involved in analyzing the Multi collinearity between which i

 Of The interdependence between variables is big

 In case if interdependence is not calculated then these are the symptoms of having the
.

Multiicollinearity:

a.When the variable’s interdependence differ from one model with another.

b. When we are applying t-test, the interdependence will not be considered

important but putting all together that is F test for complete model is

7
 important.

Depending only on the interdependence among combination or pair of variables has

y

limitation, the small or large value of interdependence is like something subjectiive that is

relying on the individual and on field for exploring & research reason is why it tooks

most of the time to research the Multiicollinearity we use some of indicator known as

variance Rising factor (VIF ).

. e

3 METHADOLOGY

Factors of Variance Inflation (VIF )

It is known that correlation exists among predictor’s when there is some predictors

coefficients standard error exist of will inflate and unfortunately value of variance of

predictor coefficients were increased . there is a tool for measuring and calculating how

much variance is increased known as VIF. These are generally intended by the software

as part of analysis of regression and will be shown in VIF column as part of result

outcome. For explaining and justifying interpreting variance inflation factor value on

basis of parameters below mentioned rule is used in the table below:

In inclusion to the meaning of variance inflation factor itself in representing whether the

predictors are correlated, the square root of the variance inflation factor tells how much

8
bigger value of standard error is coming , let suppose if value of variance inflation factor

= 9 this means that standard error on the predictor coeffic ient . that is Three times
i

lareger as it would be if that predictor is uncorrelated with all of other predictors. variance

inflation factor can measured or calculated by using the given formula:

it can be measured for every single of predictor including in model & the way is to give
i
the assuming variable.

The Ith variable against all of other value of predictor s. We get Ri Square which we can
.

be use to find variance inflation factor, and the thing which is same can applied on all

of the other predictors.

p

Getting back on the results for our data analysis, as shown in Table#1, we get to know

about that the value of VIF can be seen clearly X1 was 1 for both of the models is

named simple and multiple model, same in the case of X 2 remains unchanged and it was

taken 1, this is all reason for having a very low correlation among the variable taken for

the very first data set , going towards second data set , variance inflation factor for both of

the variables results in changed from value 1 for simple model to value 113.67 for

multiple model. at the conclusion we are unable to begin with regression analysis until this

problem is solved [3].

.

9
 4 Problem solving

The relationship among variables which are dependent and variables which are

independent is deform by the very tight and highly strong bond among the Variables .

which are independent when two of them or more predictors are highly correlated, that
e. .

leads to the likeness of our explanation of relationships will be incorrect. If case is worst,
.

and the variables are strong perfectly correlated then cannot calculate regression. [4]

Resilience is known as the measure of fluctuation in one of free factor that is not clarified

by the other factors which are independent. Multiicollinearity is distinguished by

inspecting the resistance for every autonomous variable. what's more, it is in actuality

1 – R2. Tolerance esteems under 0.10 show Multiicollinearity

If co-lineari ty in regression outcome develops then we should must eliminate the

.

prediction of the relationships as it is false until the problem or query is solved.[3]

i

Multiicollinearity can be settled by precluding a variable from the investigation that related

with other variable or by joining the profoundly associated factors through analysis of

principal component
.

5 Conclusions

1. In conclusion we get to know about Multiicollinearity is one of significant

problems that must be removed and solved while initiating the modeling process

of data,

2. At the End highly. suggested that all of regression analysis supposition should

meet as they helps making assumption on the population which are contributing

to the accurate end result.

1
 3. We will say that Multiicollinearity bring to light after discovery of the model

specially with correlation high among the independent and dependent variable as

the model cannot be interpreted then Ignored and dismiss the model .

References

[1] Carl F. Mela* and Praveen K. Kopalle. The impact of collinearity on regression

analysis: the asymmetric effect of negative and positive correlations. J. of Applied

Economics, 2002, 43,667-677.

[2] D.R.Jensen and D.E. Ramirez. Variance Inflation in Regression, Advances in Decision

Sciences, 2012, 2013, 1-15.

[3] Debbie J. Dupuis1 and Maria-Pia Victoria-Feser. Robust VIF regression with

application to variable Selection in large data sets, The Annals of Applied Statistics,

2013, 7,319-341.

[4] George A. Milliken, Dallas E. Johnson (2002). Analysis of Messy data, Vol.3, Chapman

& Hall/CRC

[5] Golberg, M. Introduction to regression analysis, Billerica, MA:

Computational Mechanics Publications, 2004, 436.

[6] Jason W. Osborne and Elaine Waters (2002). Four Assumptions of Multiple

Regressions that Researchers should always Test. J. of Practical Assessment, Research,

and Evaluation. Vol.8, No.2, PP1-5.

[7] Kleinbaum, David G. Applied regression analysis and other multivariable methods,

Australia;Belmont, CA: Brooks/Cole, 2008,906.

[8] McClendon, McKee J. Multiple regression and causal analysis, Prospects

1
Heights, Ill.: Waveland Press 2002,358.

[9] Seber, G. A. F. (George Arthur Frederick). Linear regression analysis,

Hoboken, N.J.: Wiley- Interscience, 2003,557.