Professional Documents
Culture Documents
Heteroscedasticity
Heteroscedasticity
Because heteroscedasticity concerns expectations of the second moment of the errors, its presence is
referred to as misspecification of the second order.[4]
Contents
Definition
Consequences
Detection
Fixes
Examples
Multivariate case
Notes
References
Further reading
External links
Definition
Suppose there is a sequence of random variables and a sequence of vectors of random variables,
. In dealing with conditional expectations of Yt given Xt, the sequence {Yt}t=1n is said to be
heteroscedastic if the conditional variance of Yt given Xt, changes with t. Some authors refer to this as
conditional heteroscedasticity to emphasize the fact that it is the sequence of conditional variances that
changes and not the unconditional variance. In fact, it is possible to observe conditional
heteroscedasticity even when dealing with a sequence of unconditional homoscedastic random variables;
however, the opposite does not hold. If the variance changes only because of changes in value of X and
not because of a dependence on the index t, the changing variance might be described using a scedastic
function.
When using some statistical techniques, such as ordinary least squares (OLS), a number of assumptions
are typically made. One of these is that the error term has a constant variance. This might not be true
even if the error term is assumed to be drawn from identical distributions.
For example, the error term could vary or increase with each observation, something that is often the case
with cross-sectional or time series measurements. Heteroscedasticity is often studied as part of
econometrics, which frequently deals with data exhibiting it. While the influential 1980 paper by Halbert
White used the term "heteroskedasticity" rather than "heteroscedasticity",[5] the latter spelling has been
employed more frequently in later works.[6]
The econometrician Robert Engle won the 2003 Nobel Memorial Prize for Economics for his studies on
regression analysis in the presence of heteroscedasticity, which led to his formulation of the
autoregressive conditional heteroscedasticity (ARCH) modeling technique.[7]
Consequences
One of the assumptions of the classical linear regression model is that there is no heteroscedasticity.
Breaking this assumption means that the Gauss–Markov theorem does not apply, meaning that OLS
estimators are not the Best Linear Unbiased Estimators (BLUE) and their variance is not the lowest of all
other unbiased estimators. Heteroscedasticity does not cause ordinary least squares coefficient estimates
to be biased, although it can cause ordinary least squares estimates of the variance (and, thus, standard
errors) of the coefficients to be biased, possibly above or below the true or population variance. Thus,
regression analysis using heteroscedastic data will still provide an unbiased estimate for the relationship
between the predictor variable and the outcome, but standard errors and therefore inferences obtained
from data analysis are suspect. Biased standard errors lead to biased inference, so results of hypothesis
tests are possibly wrong. For example, if OLS is performed on a heteroscedastic data set, yielding biased
standard error estimation, a researcher might fail to reject a null hypothesis at a given significance level,
when that null hypothesis was actually uncharacteristic of the actual population (making a type II error).
Under certain assumptions, the OLS estimator has a normal asymptotic distribution when properly
normalized and centered (even when the data does not come from a normal distribution). This result is
used to justify using a normal distribution, or a chi square distribution (depending on how the test
statistic is calculated), when conducting a hypothesis test. This holds even under heteroscedasticity. More
precisely, the OLS estimator in the presence of heteroscedasticity is asymptotically normal, when
properly normalized and centered, with a variance-covariance matrix that differs from the case of
homoscedasticity. In 1980, White proposed a consistent estimator for the variance-covariance matrix of
the asymptotic distribution of the OLS estimator.[5] This validates the use of hypothesis testing using
OLS estimators and White's variance-covariance estimator under heteroscedasticity.
Heteroscedasticity is also a major practical issue encountered in ANOVA problems.[8] The F test can still
be used in some circumstances.[9]
However, it has been said that students in econometrics should not overreact to heteroscedasticity.[6] One
author wrote, "unequal error variance is worth correcting only when the problem is severe."[10] In
addition, another word of caution was in the form, "heteroscedasticity has never been a reason to throw
out an otherwise good model."[6][11] With the advent of heteroscedasticity-consistent standard errors
allowing for inference without specifying the conditional second moment of error term, testing
conditional homoscedasticity is not as important as in the past.
For any non-linear model (for instance Logit and Probit models), however, heteroscedasticity has more
severe consequences: the maximum likelihood estimates (MLE) of the parameters will be biased, as well
as inconsistent (unless the likelihood function is modified to correctly take into account the precise form
of heteroscedasticity).[12] Yet, in the context of binary choice models (Logit or Probit), heteroscedasticity
will only result in a positive scaling effect on the asymptotic mean of the misspecified MLE (i.e. the
model that ignores heteroscedasticity).[13] As a result, the predictions which are based on the
misspecified MLE will remain correct. In addition, the misspecified Probit and Logit MLE will be
asymptotically normally distributed which allows performing the usual significance tests (with the
appropriate variance-covariance matrix). However, regarding the general hypothesis testing, as pointed
out by Greene, “simply computing a robust covariance matrix for an otherwise inconsistent estimator
does not give it redemption. Consequently, the virtue of a robust covariance matrix in this setting is
unclear.”[14]
Detection
There are several methods to test for the presence of
heteroscedasticity. Although tests for heteroscedasticity between
groups can formally be considered as a special case of testing
within regression models, some tests have structures specific to
this case.
Tests in regression
Levene's test
Goldfeld–Quandt test
Park test[15]
Glejser test[16][17] Absolute value of residuals for
Brown–Forsythe test simulated first order heteroscedastic
Harrison–McCabe test data
Breusch–Pagan test
White test[5]
Cook–Weisberg test
Fixes
There are four common corrections for heteroscedasticity. They are:
View logarithmized data. Non-logarithmized series that are growing exponentially often
appear to have increasing variability as the series rises over time. The variability in
percentage terms may, however, be rather stable.
Use a different specification for the model (different X variables, or perhaps non-linear
transformations of the X variables).
Apply a weighted least squares estimation method, in which OLS is applied to transformed
or weighted values of X and Y. The weights vary over observations, usually depending on
the changing error variances. In one variation the weights are directly related to the
magnitude of the dependent variable, and this corresponds to least squares percentage
regression.[18]
Heteroscedasticity-consistent standard errors (HCSE), while still biased, improve upon OLS
estimates.[5] HCSE is a consistent estimator of standard errors in regression models with
heteroscedasticity. This method corrects for heteroscedasticity without altering the values of
the coefficients. This method may be superior to regular OLS because if heteroscedasticity
is present it corrects for it, however, if the data is homoscedastic, the standard errors are
equivalent to conventional standard errors estimated by OLS. Several modifications of the
White method of computing heteroscedasticity-consistent standard errors have been
proposed as corrections with superior finite sample properties.
Use MINQUE or even the customary estimators (for
Examples
Heteroscedasticity often occurs when there is a large difference among the sizes of the observations.
Notes
a. The spellings homoskedasticity and heteroskedasticity are also frequently used. Karl
Pearson first used the word in 1905 with a c spelling.[1] J. Huston McCulloch argued that
there should be a ‘k’ in the middle of the word and not a ‘c’. His argument was that the word
had been constructed in English directly from Greek roots rather than coming into the
English language indirectly via the French. See McCulloch, J. Huston (March 1985).
"Miscellanea: On Heteros*edasticity". Econometrica. 53 (2): 483. JSTOR 1911250 (https://w
ww.jstor.org/stable/1911250).
References
1. Pearson, Karl (1905). "Mathematical Contributions to the Theory of Evolution. XIV. On the
General Theory of Skew Correlation and Non-linear Regression". Draper's Company
Research Memoirs: Biometric Series. II.
2. Goldberger, Arthur S. (1964). Econometric Theory (https://archive.org/details/econometricth
eor0000gold). New York: John Wiley & Sons. pp. 238–243 (https://archive.org/details/econo
metrictheor0000gold/page/238).
3. Johnston, J. (1972). Econometric Methods. New York: McGraw-Hill. pp. 214–221.
4. Long, J. Scott; Trivedi, Pravin K. (1993). "Some Specification Tests for the Linear
Regression Model". In Bollen, Kenneth A.; Long, J. Scott (eds.). Testing Structural Equation
Models. London: Sage. pp. 66–110. ISBN 978-0-8039-4506-7.
5. White, Halbert (1980). "A heteroskedasticity-consistent covariance matrix estimator and a
direct test for heteroskedasticity". Econometrica. 48 (4): 817–838.
CiteSeerX 10.1.1.11.7646 (https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.11.76
46). doi:10.2307/1912934 (https://doi.org/10.2307%2F1912934). JSTOR 1912934 (https://w
ww.jstor.org/stable/1912934).
6. Gujarati, D. N.; Porter, D. C. (2009). Basic Econometrics (Fifth ed.). Boston: McGraw-Hill
Irwin. p. 400. ISBN 9780073375779.
7. Engle, Robert F. (July 1982). "Autoregressive Conditional Heteroscedasticity with Estimates
of the Variance of United Kingdom Inflation". Econometrica. 50 (4): 987–1007.
doi:10.2307/1912773 (https://doi.org/10.2307%2F1912773). ISSN 0012-9682 (https://www.
worldcat.org/issn/0012-9682). JSTOR 1912773 (https://www.jstor.org/stable/1912773).
8. Jinadasa, Gamage; Weerahandi, Sam (1998). "Size performance of some tests in one-way
anova". Communications in Statistics - Simulation and Computation. 27 (3): 625.
doi:10.1080/03610919808813500 (https://doi.org/10.1080%2F03610919808813500).
9. Bathke, A (2004). "The ANOVA F test can still be used in some balanced designs with
unequal variances and nonnormal data". Journal of Statistical Planning and Inference. 126
(2): 413–422. doi:10.1016/j.jspi.2003.09.010 (https://doi.org/10.1016%2Fj.jspi.2003.09.010).
10. Fox, J. (1997). Applied Regression Analysis, Linear Models, and Related Methods.
California: Sage Publications. p. 306. (Cited in Gujarati et al. 2009, p. 400)
11. Mankiw, N. G. (1990). "A Quick Refresher Course in Macroeconomics". Journal of
Economic Literature. 28 (4): 1645–1660 [p. 1648]. doi:10.3386/w3256 (https://doi.org/10.33
86%2Fw3256). JSTOR 2727441 (https://www.jstor.org/stable/2727441).
12. Giles, Dave (May 8, 2013). "Robust Standard Errors for Nonlinear Models" (http://davegiles.
blogspot.com/2013/05/robust-standard-errors-for-nonlinear.html). Econometrics Beat.
13. Ginker, T.; Lieberman, O. (2017). "Robustness of binary choice models to conditional
heteroscedasticity". Economics Letters. 150: 130–134. doi:10.1016/j.econlet.2016.11.024 (h
ttps://doi.org/10.1016%2Fj.econlet.2016.11.024).
14. Greene, William H. (2012). "Estimation and Inference in Binary Choice Models" (https://boo
ks.google.com/books?id=-WFPYgEACAAJ&pg=PA733). Econometric Analysis (Seventh
ed.). Boston: Pearson Education. pp. 730–755 [p. 733]. ISBN 978-0-273-75356-8.
15. R. E. Park (1966). "Estimation with Heteroscedastic Error Terms". Econometrica. 34 (4):
888. doi:10.2307/1910108 (https://doi.org/10.2307%2F1910108). JSTOR 1910108 (https://
www.jstor.org/stable/1910108).
16. Glejser, H. (1969). "A new test for heteroscedasticity". Journal of the American Statistical
Association. 64 (325): 316–323. doi:10.1080/01621459.1969.10500976 (https://doi.org/10.1
080%2F01621459.1969.10500976).
17. Machado, José A. F.; Silva, J. M. C. Santos (2000). "Glejser's test revisited". Journal of
Econometrics. 97 (1): 189–202. doi:10.1016/S0304-4076(00)00016-6 (https://doi.org/10.101
6%2FS0304-4076%2800%2900016-6).
18. Tofallis, C (2008). "Least Squares Percentage Regression". Journal of Modern Applied
Statistical Methods. 7: 526–534. doi:10.2139/ssrn.1406472 (https://doi.org/10.2139%2Fssr
n.1406472). SSRN 1406472 (https://ssrn.com/abstract=1406472).
19. J. N. K. Rao (March 1973). "On the Estimation of Heteroscedastic Variances". Biometrics.
29 (1): 11–24. doi:10.2307/2529672 (https://doi.org/10.2307%2F2529672).
JSTOR 2529672 (https://www.jstor.org/stable/2529672).
20. Holgersson, H. E. T.; Shukur, G. (2004). "Testing for multivariate heteroscedasticity".
Journal of Statistical Computation and Simulation. 74 (12): 879.
doi:10.1080/00949650410001646979
(https://doi.org/10.1080%2F00949650410001646979). hdl:2077/24416 (https://hdl.handle.n
et/2077%2F24416).
21. Gupta, A. K.; Tang, J. (1984). "Distribution of likelihood ratio statistic for testing equality of
covariance matrices of multivariate Gaussian models". Biometrika. 71 (3): 555–559.
doi:10.1093/biomet/71.3.555 (https://doi.org/10.1093%2Fbiomet%2F71.3.555).
JSTOR 2336564 (https://www.jstor.org/stable/2336564).
22. d'Agostino, R. B.; Russell, H. K. (2005). "Multivariate Bartlett Test". Encyclopedia of
Biostatistics. doi:10.1002/0470011815.b2a13048 (https://doi.org/10.1002%2F0470011815.b
2a13048). ISBN 978-0470849071.
Further reading
Most statistics textbooks will include at least some material on heteroscedasticity. Some examples are:
External links
Econometrics lecture (topic: heteroscedasticity) (https://www.youtube.com/watch?v=TfuqBx
RgRTU&list=PLD15D38DC7AA3B737&index=3#t=47s) on YouTube by Mark Thoma
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using
this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia
Foundation, Inc., a non-profit organization.