Heteroskedasticity

Heteroskedasticity:
Nature and Detection

Aims and Learning Objectives
Learning Outcomes: Students should be able to
• Explain the nature of heteroskedasticity
• Understand the causes and consequences of

heteroskedasticity
• Perform tests to determine whether a regression

model has heteroskedastic errors
Nature of Heteroskedasticity
Heteroskedasticity is a systematic pattern in

the errors where the variances of the errors
are not constant.
Ordinary least squares assumes that all
observations are equally reliable.
Kuadrat terkecil biasa mengasumsikan bahwa
semua pengamatan sama-sama handal
Regression Model
Yi = 1 + 2Xi + Ui
Homoskedasticity: Var(Ui) = 2

Or E(Ui2) = 2
Heteroskedasticity: Var(Ui) = i2

Or E(Ui2) = i2
Homoskedastic pattern of errors
Consumption
Yi
.
. . . .
.. . . . .
. . . ...
... .. .
. . .. . .
. .. . . . .
..
. .
.
Income Xi
The Homoskedastic Case
f(Yi) Yi
n
p ti o
s um
Co
n
.
.
.
.
X1 X2 X3 X4 Xi
Income
Heteroskedastic pattern of errors
Consumption
.
Yi .
. .
. . . .
. .
. . . .
. . . . .
. . . .. . . .
. . . . . . .
. . . . .
. . . . .
. .
Income Xi
The Heteroskedastic Case
f(Yi)
Y
i
o n
p ti
s um
n
Co
.
. rich people
.
poor people
X1 X2 X3 Income Xi
EXAMPLE
Household’s consumption and income:

• Household with a low income do not have much flexibility in spending. Most of
the income will go for basic necessities such as food, shelter, clothing and
transportation.
• Rich families have a great deal of flexibility in spending. Some might be large
consumers, others might be large savers and investors in real estate, stock
market. This implies that actual consumption might be quite different from
average consumption. Higher income households have a larger dispersion
around mean consumption than lower income households.
Why (other reasons)----------
Var(Ui) = i2
• Error learning model…learn more less errors
• Discretionary income…only for higher income earners---
gives more choices in spending and saving
• Data collecting techniques-better techniques better
results and less errors
• Existence of outliers
• Due to specification errors-important variables omitted
• Skewness-eg. Income, only top few owns larger sum of
income
• Incorrect data transformation
• Incorrect functional form
Consequences of Heteroskedasticity
1. Ordinary least squares estimators still linear, unbiased and

consistent. {Unbiasedness and consistency requires that  has
zero mean and is uncorrelated with X.}
2. Ordinary least squares estimators not efficient. The estimated
variances and covariances of the ’s are biased and
inconsistent. Tests of hypotheses are no longer valid.
3. Confidence intervals based on usual standard errors are wrong.
4. The OLS estimators are no longer BLUE and will be
inefficient. Forecasts will also be inefficient.
^ ^
Yi = 1 + 2Xi + ei
heteroskedasticity: Var(ei) = i2
Formula for ordinary least squares variance
(homoskedastic disturbances):  2
Var ( ˆ ) 
x
2 2
i
Formula for ordinary least squares variance

(heteroskedastic disturbances): ˆ
Var (  2 ) 
 xi  i
2 2
 x 
2 2
i
Therefore when errors are heteroskedastic ordinary

least squares estimators are inefficient (i.e. not “best”)
Detecting Heteroskedasticity
Y  ˆ  ˆ X  ˆ X  e
i 1 2 2i 3 3i i
ei2 : squared residuals provide proxies for U i2

(variance of the population)
Preliminary Analysis
• Data - Heteroskedasticity often occurs in cross
sectional data (exceptions: ARCH, panel data)
• Graphical examination of residuals - plot e i or ei2

against each explanatory variable or against predicted
Y
Residual Plots
Plot residuals against one variable at a time after

sorting the data by that variable to try to find a
heteroskedastic pattern in the data. Whether there
is relationship or not, no relationship no. hetero.
.
. . .
ei2
.
.. . .
. .
. .. .
0 . . . . ..
. .
. . Ŷ
. . .
.
.
Residual Plots
Relationship could be linear, quadratic or others
. .
. . .
e 2
.
.. .
i
. . .
. . . . ..
0 . .. . . .
. . Xi
. . .
.
.
Formal Tests for Heteroskedasticity
The Goldfeld-Quandt Test
1. Identify a variable (say Z) to which the error variance is related.

Sort data set according to increasing values of Z, (smallest to
largest). fF
FFFFFFFFff
ff
2. Divide the sample into the first n1 and the last n2, thus omit the
middle c observations. The number of observations to be omitted is
arbitrary and usually between one-sixth and one-third.
3. Estimate separate regressions on first n1 observations and last n2

observations. Obtain the error sum of squares – SSE1 and SSE2.
SSE 2 /( n 2  k  1)
• 4. Compute F 
SSE1 /( n 1  k  1)
If disturbances are homoskedastic then Var(Ui) should be the

same for both samples
Jika gangguan yang homoskedastic kemudian Var (Ui) harus
sama untuk kedua sampel
5. Specify null and alternative hypothesis

There is no heteroscedasticity
Ho: 1 = 2 2 2
H1: 22 > 12 There is heteroscedasticity
6. Test statistic
2 2
GQ  ~ F( n  k 1, n  k 1)
1 2 1 2
Compare test statistic value with critical value from

F-distribution table
Dependent Variable: EXPTR
Method: Least Squares
Sample: 1 20
Included observations: 20
Variable Coefficient Std. Error t-Statistic Prob.
INC 0.094943 0.055501 1.710669 0.1043
C 0.196430 1.396950 0.140614 0.8897
R-squared 0.139842 Mean dependent var 2.377750
Adjusted R-squared 0.092055 S.D. dependent var 2.677782
S.E. of regression 2.551554 Akaike info criterion 4.805922
Sum squared resid 117.1877 Schwarz criterion 4.905495
Log likelihood -46.05922 F-statistic 2.926389
Durbin-Watson stat 2.338219 Prob(F-statistic) 0.104321
Dependent Variable: EXPTR
Sample: 32 51
INC 0.060328 0.005976 10.09445 0.0000
C -1.021211 1.516486 -0.673406 0.5092
Adjusted R-
squared 0.841532 S.D. dependent var 9.615438
GQ = SSE2/SSE1 =263.7257/117.1877 = 2.34
At =0.05, F0.05,18,18= 2.222. Decision: Reject H0.

There is heteroscedasticity
Lagrange Multiplier (LM) Tests
Let the model be

Y = 0 + 1X1 + 2X2 + ….+ kXk + 
Where the error variance is 2 = E(2)
Form the auxiliary equations:
2 = 0 + 1Z1 + 2Z2 + …..+ pZp – BREUSCH-PAGAN
 = 0 + 1Z1 + 2Z2 + …..+ pZp - GLESJER
ln 2 = 0 + 1Z1 + 2Z2 + …..+ pZp – HARVEY-GODFREY
Zs are variables with known values (some or all of the Zs might be
the Xs in the model)
Lagrange Multiplier (LM) Tests
• H0 : 1= 2= …..= p= 0. (homoscedasticity)
• H1 : at least one of the   0. (heteroscedasticity)
• Because we do not know  , we use estimate obtained by

applying OLS to the model, i.e we would use 2 for 2 .
BREUSCH-PAGAN TEST
1. Estimate model using OLS – obtained the residuals

m = Y - 0 - 1X1 - 2X2 - ….- kXk
2. Square the residuals and form the auxiliary regression
2 = 0 + 1Z1 + 2Z2 + …..+ pZp
3. Compute test statistic LM=nR2, where n is the number of
observations used in estimating the auxiliary regression.
4. Obtain critical value from 2 distribution
(df = no. of explanatory variables in auxiliary regression)
5. Decision rule: if test statistic > critical 2 value
then reject null hypothesis of no heteroskedasticity
BREUSCH-PAGAN TEST
Dependent Variable: UHATSQ

Sample: 1 51
C 2.192914 4.636127 0.473006 0.6383
INC 0.056935 0.028510 1.997036 0.0514
LM = nR2 = 51 X 0.075265 = 3.838515

At =0.10,2 (1) = 2.70554. Decision: Reject H0.
GLESJER TEST
m = Y - 0 - 1X1 - 2X2 - ….- kXk
2. Compute the absolute value of residuals and form the auxiliary
regression  = 0 + 1Z1 + 2Z2 + …..+ pZp
GLESJER TEST
Dependent Variable: UHATABS
Sample: 1 51
C 0.759643 0.396291 1.916882 0.0611

INC 0.008259 0.002437 3.388887 0.0014
LM = nR2 = 51 X 0.189876 = 9.683676

HARVEY-GODFREY TEST

m = Y - 0 - 1X1 - 2X2 - ….- kXk
2. Take logarithm of squared residuals and form the auxiliary
regression ln 2 = 0 + 1Z1 + 2Z2 + …..+ pZp
HARVEY – GODFREY TEST
Dependent Variable: LUHATSQ

Sample: 1 51
C -1.628557 0.447713 -3.637502 0.0007
INC 0.010068 0.002753 3.656767 0.0006
R-squared 0.214390 Mean dependent var -0.570103
LM = nR2 = 51 X 0. 21439 = 10.93389

PARK TEST – special case of the Harvey-Godfrey Test

m = Y - 0 - 1X1 - 2X2 - ….- kXk
2. Take logarithm of Squared residuals and form the
auxiliary regression ln 2 = 0 + 1lnX1
3. If there is more than one explanatory variable, run the
regression against each X variable
4. Test on 1 by the usual t test. A significant t statistic is a rejection
of no heteroskedasticity.
PARK TEST
Dependent Variable: LUHATSQ

Sample: 1 51
C -5.941761 1.339337 -4.436346 0.0001
LINC 1.305717 0.315417 4.139647 0.0001
R-squared 0.259110 Mean dependent var -0.570103
Since Prob(t-stat)=0.0001  =0.05, Reject H0.

White’s Test
1. Estimate Yˆi  ˆ1  ˆ 2 X 2i  ˆ3 X 3i

And obtain the residuals and square it
2. Run the following auxiliary regression:
e  A0  A1 X 2i  A2 X 3i  A3 X  A4 X  A5 X 2i X 3i  Vi
2
i
2
2i
2
3i
3. Calculate White test statistic from auxiliary

regression
nR ~  d . f .
2 2


WHITE TEST [WITH CROSS TERM]
Dependent Variable: UHAT2SQ
C -1.325156 6.550338 -0.202304 0.8406
INC 0.314301 0.469365 0.669629 0.5065
POP -3.298486 10.38129 -0.317734 0.7522
INCSQ -0.010942 0.008184 -1.337009 0.1879
POPSQ -4.798280 4.068829 -1.179278 0.2445
INC_POP 0.454811 0.362085 1.256089 0.2156
LM = nR2 = 51 X 0.115692 = 5.90029.
At =0.10,2 (5) =11.0705. Decision: Do not Reject H0.

There is no evidence of heteroscedasticity
WHITE TEST [WITH CROSS TERM]
White Heteroskedasticity Test:

F-statistic 1.177448 Prob. F(5,45) 0.335158
Obs*R-squared 5.900287 Prob. Chi-Square(5) 0.316043
Test Equation:
Dependent Variable: RESID^2
C -1.325156 6.550338 -0.202304 0.8406
INC 0.314301 0.469365 0.669629 0.5065
INC^2 -0.010942 0.008184 -1.337009 0.1879
INC*POP 0.454811 0.362085 1.256089 0.2156
POP -3.298486 10.38129 -0.317734 0.7522
POP^2 -4.798280 4.068829 -1.179278 0.2445
WHITE TEST [WITH NO CROSS TERM]
Dependent Variable: UHAT2SQ

C 0.532684 6.421115 0.082958 0.9342
INC 0.324953 0.472227 0.688129 0.4948
POP -4.838522 10.37317 -0.466446 0.6431
INCSQ -0.000712 0.000813 -0.876456 0.3853
POPSQ 0.289100 0.391617 0.738220 0.4641
LM = nR2 = 51 X 0.084687 = 4.319037

At =0.10,2 (4) =9.48773. Decision: Do not Reject H0.
There is no evidence of heteroscedasticity
WHITE TEST [WITH NO CROSS TERM]
White Heteroskedasticity Test:

F-statistic 1.064006 Prob. F(4,46) 0.385124
Obs*R-squared 4.319031 Prob. Chi-Square(4) 0.364548
Test Equation:
Dependent Variable: RESID^2
C 0.532684 6.421115 0.082958 0.9342
INC 0.324953 0.472227 0.688129 0.4948
INC^2 -0.000712 0.000813 -0.876456 0.3853
POP -4.838522 10.37317 -0.466446 0.6431
POP^2 0.289100 0.391617 0.738220 0.4641
Heteroskedasticity:
Remedies
Normality
Assumption
Aims and Learning Objectives
By the end of this session students should be able to:
• Use the weighted least squares procedure to deal

with heteroskedasticity
• How reformulating the model may help remove apparent

heteroskedasticity problems
• Understand what the normality assumption means and why it is

important
Remedies for Heteroskedasticity
• Weighted Least Squares
• Redefine the variables

Weighted Least Squares
Intuitively, we want to explicitly take account of the variability of

consumption for different income groups
As mentioned earlier, that OLS in the presence of

heteroskedasticity is LUE but not BLUE
OLS assigns equal weights (importance) to each observation
We would like to use an estimation method such that observations

with greater variability are given less weight than those with
smaller variability
Generalized (or Weighted) Least Squares
Case 1: Disturbance term is known
Yi  1   2 X 2i   3 X 3i  U i
Var(Ui) = i2
Divide by i to obtain:
Yi 1 X 2i X 3i Ui
 1  2  3 
i i i i i
The variance of the disturbance term in the transformed
regression is
2
 Ui 
E  
 i 
Therefore 1
 E (U ) 2
 2 i
i
1
 ( )  1
2
 2 i
i
The variance of the transformed disturbance term is now
homoskedastic (and therefore BLUE)
Generalized (or Weighted) Least Squares
Case 2: Multiplicative Heteroscedasticity with known
Proportional Factor
Yi  1   2 X 2i   3 X 3i  U i
Var(Ui) = i2
The variance is
assumed to be
proportional to
the value of di2
where σ i
2
= σ 2
d i
2
Standard deviation proportional to d i
Yi  1   2 X 2i   3 X 3i  U i
variance: var(Ui) = i2 i2 = 2 di2
standard deviation: σi = σdi
To correct for heteroskedasticity divide the model by d i
Yi 1 X 2i X 3i U i
 1   2  3 
di di di di di
Yi 1 X 2i X 3i U i
 1   2  3 
di di di di di
Y
*
i =  1
*
+  2 Xi2
*
+  3 i3 + Vi
X *
2
 Ui  2 di 2
Var (Vi )  E      2
 di  di 2
Ui is heteroskedastic, but Vi is homoskedastic

Weighted Least Squares
Recap:
1. Decide which variable is proportional to the heteroskedasticity
(di could be one of the X variables or predicted Y).
2. Divide all terms in the original model by the square root of that
variable (divide by di).
3. Run least squares on the transformed model which has new Yi,
Xi2 and Xi3 variables
Feasible Generalized Least Squares
The GLS procedure consists of dividing each variable by σ and then

applying OLS to the resulting transformed model.
As the structure of the heteroscedasticity is generally unknown (i.e σ
is unknown), a researcher must first obtain estimates of σ by
some means and then use the WLS procedure – this method is
known Feasible Generalized Least Squares (FGLS).
The actual procedure for estimating  has, however, varied widely in
practice.
STEP I . Estimate equation by OLS and obtain residuals 

and their squares 2
STEP II. Estimate the auxiliary equation by regressing 2
against a constant, X1 , X2 and X12 , X22 (squares term) and
X1X2 (cross term)
STEP III. Use the estimates of the auxiliary regression and
obtain the predicted variance.
STEP IV. Set the weight wt = 1/σt and use weighted least
squares.
• The WLS estimates obtained in this way are consistent

and so are the estimated variances and covariances of
the estimates. Also, the estimates are asymptotically likely
to be more efficient than OLS estimators.
• PROBLEM – there is no guarantee that the predicted
variances in steps III will be positive for all t [Breusch-
Pagan and Glesjer]. If any of them is exactly zero, then
corresponding weight is undefined. If any of them is
negative, we cannot take the square root.
• If this situation arises for some observations, then we can
used the original 2 [Breusch-Pagan] or  [Glesjer]
Hetero-corrected standard error
estimators (unknown variance)
• An option after OLS – to produce robust standard
errors
• Three possible options
– The error variance is propotional to Xi2
– The error variance is propotional to Xi
– The error variance is propotional to square of the
mean of Y (or estimated Y as proxy)
• This will involve transformation
• Another alternative – use difference functional form

Heteroskedasticity

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Heteroskedasticity

Uploaded by

Copyright:

Available Formats

Heteroskedasticity:

Nature and Detection

Learning Outcomes: Students should be able to

• Explain the nature of heteroskedasticity

• Understand the causes and consequences of

• Perform tests to determine whether a regression

Heteroskedasticity is a systematic pattern in

Homoskedasticity: Var(Ui) = 2

Heteroskedasticity: Var(Ui) = i2

Household’s consumption and income:

1. Ordinary least squares estimators still linear, unbiased and

Formula for ordinary least squares variance

Therefore when errors are heteroskedastic ordinary

ei2 : squared residuals provide proxies for U i2

• Graphical examination of residuals - plot e i or ei2

Plot residuals against one variable at a time after

Relationship could be linear, quadratic or others

1. Identify a variable (say Z) to which the error variance is related.

3. Estimate separate regressions on first n1 observations and last n2

If disturbances are homoskedastic then Var(Ui) should be the

5. Specify null and alternative hypothesis

H1: 22 > 12 There is heteroscedasticity

Compare test statistic value with critical value from

Method: Least Squares

GQ = SSE2/SSE1 =263.7257/117.1877 = 2.34

At =0.05, F0.05,18,18= 2.222. Decision: Reject H0.

Let the model be

• Because we do not know  , we use estimate obtained by

1. Estimate model using OLS – obtained the residuals

Dependent Variable: UHATSQ

LM = nR2 = 51 X 0.075265 = 3.838515

C 0.759643 0.396291 1.916882 0.0611

LM = nR2 = 51 X 0.189876 = 9.683676

1. Estimate model using OLS – obtained the residuals

Dependent Variable: LUHATSQ

LM = nR2 = 51 X 0. 21439 = 10.93389

At =0.10,2 (1) = 2.70554. Decision: Reject H0.

1. Estimate model using OLS – obtained the residuals

Dependent Variable: LUHATSQ

Since Prob(t-stat)=0.0001  =0.05, Reject H0.

1. Estimate Yˆi  ˆ1  ˆ 2 X 2i  ˆ3 X 3i

3. Calculate White test statistic from auxiliary

4. Obtain critical value from 2 distribution

5. Decision rule: if test statistic > critical 2 value

LM = nR2 = 51 X 0.115692 = 5.90029.

At =0.10,2 (5) =11.0705. Decision: Do not Reject H0.

White Heteroskedasticity Test:

Dependent Variable: UHAT2SQ

LM = nR2 = 51 X 0.084687 = 4.319037

White Heteroskedasticity Test:

By the end of this session students should be able to:

• Use the weighted least squares procedure to deal

• How reformulating the model may help remove apparent

• Understand what the normality assumption means and why it is

• Weighted Least Squares

• Redefine the variables

Intuitively, we want to explicitly take account of the variability of

As mentioned earlier, that OLS in the presence of

OLS assigns equal weights (importance) to each observation

We would like to use an estimation method such that observations

Case 1: Disturbance term is known

To correct for heteroskedasticity divide the model by d i

Ui is heteroskedastic, but Vi is homoskedastic