Professional Documents
Culture Documents
RJournal 2011-2 Arnold+Emerson PDF
RJournal 2011-2 Arnold+Emerson PDF
the discrete form of the tests involves calculating the cal instability. First, consider the following inequality:
percentiles of the weighted sum of chi-squares,
p
!
p
!
p P i 21 x P max 21 x (10)
Q= i 2i,1d f (8) i =1
i =1
i =1 x
= P 2p (11)
p max
where p is the number of elements in the support of
the hypothesized distribution. Imhof (1961) provides The values for the weighted sum can be bounded
a method for obtaining the distribution of Q, easily using a simple transformation and a chi-squared dis-
adapted for our case because the chi-squared vari- tribution of a higher degree of freedom. Second, con-
ables have only one degree of freedom. The exact sider the Markov inequality:
formula given for the distribution function of Q is p
!
given by P i 21 x
i =1
1 1 sin [ (u, x )]
Z
p
P{ Q x } =
+ du (9)
2 0 u(u) E exp t i Zi2 exp(tx ) (12)
i =1
for continuous functions (, x ) and () depending exp(tx )
=q (13)
on the weights i . p
i=1 (1 2ti )
There is no analytic solution to the integral in (9),
so the integration is accomplished numerically. This where the bound can be minimized over t
seems fine in most situations we considered, but nu- (0, 1/2max ). The upper bounds for the p-value given
merical issues appear in the regime of large test statis- by (11) and (13) are both calculated and the smaller
tics x (or, equivalently, small p-values). The function is used in cases where the numerical instability of (9)
(, x ) is linear in x; as the test statistic grows the cor- may be a concern.
responding periodicity of the integrand decreases and The original formulation, numerical integration of
the approximation becomes unstable. As an example (9), is preferable for most p-values, while the upper
of this numerical instability, the red plotted in Figure bound described above is used for smaller p-values
1 shows the non-monotonicity of the numerical eval- (smaller than 0.001, based on our observations of the
uation of equation (9) for a null distribution that is numerical instability of the original formulation). Fig-
uniform on the set {1, 2, 3}. ure 1 shows the bounds with the blue and green
dashed lines; values in red exceeding the bounds
are a result of the numerical instability. Although
Unstable asymptotic pvalue from (9) it would be preferable to determine the use of the
Bound given by (11)
bound based on values of the test statistic rather than
1e03
provided (a warning is issued when the sample size and show usage of the new ks.test() implementa-
is greater than 30 due to possible numerical instabili- tion. The first is the default two-sided test, where the
ties). When exact = FALSE (or when exact is unspec- exact p-value is obtained using the methods of Gleser
ified and the sample size is greater than 30) the classi- (1985).
cal Kolmogorov-Smirnov null distribution of the test
> set.seed(1)
statistic is used and resulting p-values are known to
> x <- sample(1:10, 25, replace = TRUE)
be conservative though imprecise (see Conover (1972) > x
for details). In such cases, simulated p-values may
be desired, produced by the simulate.p.value=TRUE [1] 3 4 6 10 3 9 10 7 7 1 3 2 7
option. [14] 4 8 5 8 10 4 8 10 3 7 2 3
The function cvm.test() is similar in design to
> dgof::ks.test(x, ecdf(1:10))
ks.test(). Its first two arguments specify the data
and null distribution; the only extra option, type, One-sample Kolmogorov-Smirnov test
specifies the variant of the Cramr-von Mises test:
data: x
x a numerical vector of data values. D = 0.08, p-value = 0.9354
alternative hypothesis: two-sided
y an ecdf or step-function (stepfun) for specifying
the null model Next, we conduct the default one-sided test, where
Conovers method provides the exact p-value (up to
type the variant of the Cramr-von Mises test; W2 is
the numerical precision of the implementation):
the default and most common method, U2 is for
cyclical data, and A2 is the Anderson-Darling > dgof::ks.test(x, ecdf(1:10),
alternative. + alternative = "g")
Discussion
Bibliography
This paper presents the implementation of several
nonparametric goodness-of-fit tests for discrete null T. W. Anderson and D. A. Darling. Asymptotic the-
distributions. In some cases the p-values are known to ory of certain goodness of fit criteria based on
stochastic processes. Annals of Mathematical Statis- R. Lockhart, J. Spinelli, and M. Stephens. Cramr-von
tics, 23:193212, 1952. mises statistics for discrete distributions with un-
known parameters. Canadian Journal of Statistics, 35
C. J. G. Bellosta. ADGofTest: Anderson-Darling GoF test, (1):125133, 2007.
2009. URL http://CRAN.R-project.org/package=
ADGofTest. R package version 0.1. F. Massey. The Kolmogorov-Smirnov test for good-
ness of fit. Journal of the American Statistical Associa-
V. Choulakian, R. A. Lockhart, and M. A. Stephens.
tion, 46:6878, 1951.
Cramr-von Mises statistics for discrete distribu-
tions. The Canadian Journal of Statistics, 22(1):125 H. Niederhausen. Sheffer polynomials for computing
137, 1994. exact Kolmogorov-Smirnov and Rnyi type distri-
W. J. Conover. A Kolmogorov goodness-of-fit test for butions. The Annals of Statistics, 9(5):923944, 1981.
discontinuous distributions. Journal of the American ISSN 0090-5364.
Statistical Association, 67(339):591596, 1972.
M. J. Slakter. A comparison of the Pearson chi-square
H. Cramr. On the composition of elementary errors. and Kolmogorov goodness-of-fit tests with respect
Skand. Akt., 11:141180, 1928. to validity. Journal of the American Statistical Associa-
tion, 60(311):854858, 1965.
T. de Wet and J. Venter. Asymptotic distributions
for quadratic forms with applications to tests of fit. M. A. Stephens. Edf statistics for goodness of fit and
Annals of Statistics, 2:380387, 1994. some comparisons. Journal of the American Statistical
Association, 69(347):730737, 1974.
J.-M. Dufour and A. Farhat. Exact nonparametric
two-sample homogeneity tests for possibly discrete R. E. von Mises. Wahrscheinlichkeit, Statistik und
distributions. Unpublished manuscript, October Wahrheit. Julius Springer, Vienna, Austria, 1928.
2001. URL http://hdl.handle.net/1866/362.
J. E. Walsh. Bounded probability properties for
J. W. Emerson and T. B. Arnold. Statistical sleuthing Kolmogorov-Smirnov and similar statistics for dis-
by leveraging human nature: A study of olympic crete data. Annals of the Institute of Statistical Mathe-
figure skating. The American Statistician, 2011. To matics, 15:153158, 1963.
appear.
G. S. Watson. Goodness of fit tests on the circle.
C. Franz. cramer: Multivariate nonparametric Cramer-
Biometrika, 48:109114, 1961.
Test for the two-sample-problem, 2006. R package ver-
sion 0.8-1. Y. Xiao and Y. Cui. CvM2SL1Test: L1-version of Cramer-
L. J. Gleser. Exact power of goodness-of-fit tests of von Mises Two Sample Tests, 2009a. R package ver-
kolmogorov type for discontinuous distributions. sion 0.0-2.
Journal of American Statistical Association, 80(392):
Y. Xiao and Y. Cui. CvM2SL2Test: Cramer-von Mises
954958, 1985.
Two Sample Tests, 2009b. R package version 0.0-2.
L. A. Goodman. Kolmogorov-Smirnov tests for psy-
chological research. Psychological Bulletin, 51:160
168, 1954. Taylor B. Arnold
Yale University
J. Gross. nortest: Tests for Normality, 2006. R package 24 Hillhouse Ave.
version 1.0. New Haven, CT 06511 USA
J. P. Imhof. Computing the distribution of quadratic taylor.arnold@yale.edu
forms in normal variables. Biometrika, 48:419426,
1961. John W. Emerson
Yale University
A. Kolmogorov. Sulla determinazione empirica di una 24 Hillhouse Ave.
legge di distribuzione. Giornale dell Instuto Italiano New Haven, CT 06511 USA
degli Attuari, 4:8391, 1933. john.emerson@yale.edu