Professional Documents
Culture Documents
Graduate PDF
Graduate PDF
Graduate PDF
Introduction
Parametric statistical procedures are based on certain assumptions. For example, the
assumption that samples have been drawn from normally distributed populations. Since real
life populations do not always meet the assumptions underlying parametric tests, we need
some procedures whose validity do not depend on rigid assumptions. In this way,
nonparametric statistical procedures satisfy this need because they are valid under very
general assumptions.
The first use of what we would now call a nonparametric statistical procedure was in
1710 by John Arbuthnot (Danial, 1990, Page 18). The rudiments of nonparametric statistics
originate in the latter part of the nineteenth century. In the 1940s, the interests in both the
theory and application of nonparametric statistics grew rapidly. Wolfowitz introduced the
word nonparametric statistics in 1942 (Danial, 1990, Page 18). Nonparametric statistics is
currently one of the most important branches of statistics and it is widely used in many fields,
such as physics, biology, social sciences and so on. In this paper we provide the theory
behind some frequently used nonparametric statistical procedures for location parameters
based on ranks.
Making inferences about a location parameter or a measure of central tendency is very
common is everyday statistical practice. More often the data do not satisfy the assumptions
of parameter procedures like one sample t -test, two sample t -test, paired t -test, one-way
analysis of variance and two-way analysis of variance.
Our objective is to present the nonparametric analogues of the above procedures in
detail. We have organized the presentation as follows: general introduction, hypotheses, test
statistic, decision rules, small sample theory, an example of creating critical value tables and
large sample theory.
Sign Test
Nonparametric methods are used to make inferences about a location parameter or
population measure of central tendency when the population from which the sample data
have been drawn is no t normally distributed. In these nonparametric procedures, the median
rather than the mean is the location parameter. The sign test is perhaps the oldest of all the
nonparametric procedures. Arbuthnott used it in 1710. The sign test gets its name from
the fact that it uses plus and minus signs rather than quantitative measures as its data.
Testing whether a given population has a known mean is very common. In the
parametric setting, one sample t -test is the appropriate method. This test assumes that the
data comes from a normal distribution. More often, this assumption is not true. Then the
sign test is used to find whether the population has a known median M .
Assumptions :
1. It is a random sample from a population with unknown median M .
2. The variable is measured on at least an ordinal scale.
H0 : M = M0,
H1 : M M 0
B. (One-sided):
H0 : M M0,
H1 : M > M 0
C. (One-sided):
H0 : M M0,
H1 : M < M 0
Test Statistic:
Record the sign of the difference obtained by subtracting the hypothesized median M 0
from each sample value. If the difference is zero, the sample value is dropped from the
analysis and the sample size is thereby reduced.
A.The number of plus signs or the number of minus signs, whichever is smaller.
B. The number of minus signs.
C. The number of plus signs.
Decision Rules:
The probability associated with the occurrence of a particular number of + ' s and
' s can be determined by reference to the Binomial distribution with p = 0.5 , where n is
the reduced sample size.
Suppose the level of significance is for each hypothesis. Then the decision rules
are as follows:
A.Reject H 0 if the probability of observing as few (or fewer) as the test statistic is less
than or equal to /2.
B & C. Reject H 0 if the probability of observing as few (or fewer) as the test statistic
is less than or equal to .
Theory:
If the null hypothesis is true, we will have about as many plus signs as minus signs
when the n differences X i M 0 have been computed. In other words, the probabilities that
a difference X i M 0 yields a plus sign and a minus sign are equal to 0.5. We will discard
observations that lead to zero differences and reduce n accordingly. Let K be the
random variable that is the number of signs of interest under the given hypothesis and k is
the observed value of the test statistic. Find the value P ( K k n,0.5 ) from the
appropriate Binomial table and make the decision by using the decision rules. Since the
method of how to generate Binomial tables is well known, it is not provided in this paper.
Large-Sample Approximation:
For samples of size 25 or larger, we may use the normal approximation to the binomial
distribution. Since K has a Binomial distribution with parameters n and p = 0.5 , one
can show that E ( K ) = 0.5n and Var ( K ) = 0.25n . By using the Central Limit
Theorem, Z =
K E(K)
Var ( K )
K 0.5n
0.5 n
( K + 0.5) 0.5n .
0.5 n
( K 0.5) 0.5n .
0.5 n
Z for significance with values of the standard normal distribution for the chosen level of
significance.
The Wilconxon Signed-Ranks Test is used for the same purpose as the sign test. Sign
test does not use a measure of the magnitude of the data. Wilconxon Signed-Ranks test is
designed to consider the sign and the magnitude of the differences. First, we rank the
differences are ranked in order of absolute size and assign the original signs of the differences
to the ranks. Second, the ranks of positive signs and negative signs are added seperately.
Assumptions:
1. It is a random sample from a symmetric population with unknown median M .
2. The variable is continuous and the n sample measurements are noted
by x1 , x2 ,L , xn .
3. The variable is measured on at least an interval scale.
Hypotheses:
A. (Tow-sided):
H0 : M = M0,
H1 : M M 0
B. (One-sided):
H0 : M M0,
H1 : M > M 0
C. (One-sided):
H0 : M M0,
H1 : M < M 0
Test Statistic:
Let Di = X i M 0 , i = 1,2,L , n . We will discard observations that lead to zero
differences and reduce n accordingly.
difference to each rank. Let T+ be the sum of the ranks with positive signs and T be the
sum of the ranks of negative signs. The test statistic for each hypothesis is:
A: Either T+ or T , whichever is smaller.
B: T .
C: T+ .
Decision Rules:
Probability levels for the Wilcoxon Signed Ranks test are available in any
standard nonparametric test book. An example of how to generate these tables is given in
Appendix A2.
Suppose the level of significance is for each hypothesis. Then the decision rules
are as follows:
A. Reject H 0 if the probability of observing as few (or fewer) as the test statistic is
less than or equal to /2.
B & C. Reject H 0 if the probability of observing as few (or fewer) as the test statistic
is less than or equal to .
Theory:
If the null hypothesis is true, the probability of observing a positive difference Di of a
given magnitude is equal to the probability of observing a negative difference of the same
magnitude. That means T+ and T should be equal. But this seldom happens, we
always find T+ is not equal to T . However, we hope there will be a little difference in
their values.
We can easily show that:
T+ + T =
n ( n + 1)
=1+ 2 +L + n
2
and the probability distribution of T+ and T are the same. The proof is given in
Appendix A1. We can find the value P (test statistic T | n ) from the table and make the
conclusion according to the decision rules.
Large-Sample Approximation:
When n is larger than 30, we cannot use Wilcoxon signed rank table to determine the
significance of a computed value of the test statistic. We can find T * , where
T =
*
n ( n + 1)
4
, ( T is the test statistic). According to the Central Limit Theorem,
n ( n + 1)( 2 n + 1)
24
T * has approximately the standard normal distribution. The proof is given in Appendix A3.
Mann-Whitney Test
Mann-Whitney test is used to test whether two independent samples have been drawn
from identical populations. The test is sometimes also referred to as the
Mann-Whitney-Wilcoxon test. The independence is within and between samples. This is
one of the most powerful of the nonparametric tests, and it is a most useful alternative to the
parametric t -test when the researcher wishes to avoid the assumptions of the t -test, or when
the measurement in the research is weaker than interval scaling.
Assumptions:
1. The data consist of two random samples x1 , x2 ,L , xn1 and y1 , y2 ,L , yn2 from
population 1 and population 2, respectively.
2. The variable of interest is continuous.
3. The two samples are independent.
4. The data are measured on at least an ordinal scale.
5. The two populations are identical except for a possible difference in location.
Hypotheses:
A. (Tow-sided):
H0 : Mx = M y
H1 : M x M y
B. (One-sided):
H0 : Mx M y
H1 : M x < M y
C. (One-sided):
H0 : Mx M y
H1 : M x > M y
Test Statistic:
We combine the two samples and rank all sample observations from smallest to largest.
Then the ranks of the observations from population 1 are added. Suppose S is the sum.
Then the test statistic is defined as T = S
n1 ( n1 + 1)
.
2
1
2
1
2
= n1n2 w .
2
n1n2
2
, which has an approximately the standard
n1n2 ( n1 + n2 + 1)
12
T
Kruskal-Wallis Test
Kruskal-Wallis test is the most widely used nonparametric technique for testing
whether several independent samples have been drawn from identical populations. Sample
values almost invariably differ somewhat, and the question is whether the differences among
the samples signify genuine populations differences or whether they represent merely chance
variations such as are to be expected among several random samples from identical
populations. When only two samples are being considered, Kruskal-Wallis test is equivalent
to the Mann-Whitney test. The proof is given in Appendix C1.
Assumptions:
1. The data consist of k random samples of sizes n1 , n2 ,L , nk .
2. The variable of interest is continuous.
3. The observations are independent both within and among samples.
4. The data are measured on at least an ordinal scale.
5. The populations are identical except for a possible difference in location.
Hypotheses:
H 0 : M 1 = M 2 = L = M k where M i is the median of the ith sample.
The
i =1
Where Ri is the sum of the ranks assigned to observations in the ith sample, and
ni ( N + 1)
is the expected sum of ranks for the ith sample under H 0 . Derivation of the
2
test statistic H is given in Appendix C2.
Decision Rules:
Probability levels for the Kruskal-Wallis test are available in any standard
nonparametric test book. An example of how to generate these tables is given in Appendix
C3.
Suppose the level of significance is a for each hypothesis. When there are only three
samples and each sample has 5 or fewer observations, we compare H for significance with
the Kruskal-Wallis test statistic table and reject H 0 if H exceeds the critical value.
When we cant use the Kruskal-Wallis table, we compare H for significance with the
Chi-Square distribution table and reject H 0 if H exceeds the Chi-Square value for a and
10
Friedman Test
When the subjects show a great deal of variability and we want to know whether there
is a difference among treatment effects, we divide subjects into homogeneous subgroups, call
blocks, and then making comparisons among subjects within the subgroups. The Friedman
test is the nonparametric analogue of the parametric two-way analysis of variance and is
useful for testing whether the samples have been drawn from the identical populations.
Assumptions:
Consider the following data layout.
Block
1
2
3
M
i
M
b
Treatment
1
2
3 L
j
x11 x12 x13 L x1 j
x21 x22 x23 L x2 j
x31 x32 x33 L x3 j
M M M L
M
xi1 xi2 xi3 L xij
M M M L
M
xb1 xb2 xb3 L xbj
L
L
L
L
L
L
L
L
k
x1 k
x2 k
x3 k
M
xik
M
xbk
11
k
k
b ( k + 1)
12
12
2
=
R
=
Rg j 3b ( k + 1)
gj
bk ( k + 1) j=1
2
bk ( k + 1) j =1
2
or
W=
r
=
b ( k 1)
2
12 Rg j 3b 2 k( k + 1)
j =1
b k ( k 1)
2
where Rg j is the sum of the ranks assigned to observations in the jth column, and
b ( k + 1)
is the expected sum of ranks for the jth column under H 0 . Derivation of the
2
test statistic is give n in Appendix D1.
Decision Rules:
Probability levels for the Friedman test are available in any standard nonparametric test
book. An example of how to generate these tables is given in Appendix D2.
Suppose the level of significance is a for each hypothesis. When b and k are
small, we compare W for significance with the Friedman table and reject H 0 if W
exceeds the critical value for b , k and a.
compare r2 for significance with the Chi-Square distribution table and reject H 0 if r2
exceeds the Chi-Square value for a and k 1 degrees of freedom. It can be proven that
r is distributed approximately as a Chi-Square distribution with k 1 degrees of freedom.
2
12
magnitudes of the sums of the ranks by column. If H 0 is true, we expect the sums will be
very close to each other. When H 0 is false, there must be at least one sum to be
sufficiently different from the other sums. Differences among rank sums of sufficient
magnitude will give a sufficiently large test statistic value and cause us to reject H 0 .
13
Bibliography
Wayne W. Daniel, (1990). Applied Nonparametric Statistics, Second Edition, PWS-KENT,
Boston.
Hollander &Wolfe, (1973). Nonparametric Statistical Methods, JOHN WILEY &
SONS, New York.
Sidney Siegel, (1956). Nonparametric Statistics: For The Behavioral Sciences,
McGRAW-HILL, New York.
Peter Sprent, (1989). Applied Nonparametric Statistical Methods, CHAPMAN AND
HALL, New York.
Randles &Wolfe, (1979); Introduction to the Theory of Nonparametric Statistics, JOHN
WILEY & SONS, New York.
14