This article was downloaded by: [University of Glasgow]

On: 20 December 2014, At: 03:24

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Journal of Statistical Computation and Simulation

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/gscs20

Note on testing a three state markov chain for

independence
a a
Dong Qian Wang & J.S. Maritz
a
La Trobe University , Bundoora, Victoria, 3083, Australia
Published online: 20 Mar 2007.

To cite this article: Dong Qian Wang & J.S. Maritz (1990) Note on testing a three state markov chain for independence,
Journal of Statistical Computation and Simulation, 37:1-2, 61-68, DOI: 10.1080/00949659008811294

To link to this article: http://dx.doi.org/10.1080/00949659008811294

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of
the Content. Any opinions and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied
upon and should be independently verified with primary sources of information. Taylor and Francis shall not be
liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities
whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of
the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 J . Statist. Comput. Simul.. Vol. 37, pp. 61-68 C 1990 Gordon and Breach Science Publishers S.A.
Reprints available directly from the publtsher
Printed in the United Kingdom
Photocopying permitted by license only

NOTE ON TESTING A THREE STATE MARKOV

CHAIN FOR INDEPENDENCE
DONG QIAN WANG and J. S. MARITZ
La Trobe University, Bundoora, Victoria 3083, Australia
(Received 29 August 1989; in final form 24 May 1990)

A statistic is presented for testing a three state observed Markov chain for independence. The test
procedure is compared with the traditional x2 test. Examples are given in which the proposed test has
better power than the X 2 test.

KEY WORDS: Three state Markov chain, eigenvalues, power, x2 test, statistic.
Downloaded by [University of Glasgow] at 03:24 20 December 2014

1. INTRODUCTION

Many statistical methods in multivariate analysis are concerned with eigenvalue

problems, for example, principal-component analysis and factor analysis, etc. It is
known that several important statistics are functions of eigenvalues in multivariate
analysis. Here we consider a problem of eigenvalues of the observed transition
matrix P of a three state Markov chain. Pate1 and Ahwad (1973) discuss a test of
independence of a Markovian contingency table by the traditional x2 test and
likelihood ratio test. Ahmad (1985) presents a test method of independence for
Markovian contingency tables. Wang and Scott (1988) give a bootstrap method of
testing the hypothesis that a sequence of observations thought to be from a
Markov chain is actually derived from an independent sequence. In this note we
develop a statistic based on sample eigenvalues of the observed transition matrix
to test a three state Markov chain for independence. The test statistic has
asymptotically a standard normal distribution under the independence hypothesis,
and we compare the new test method with the traditional x2 test. It is shown by
simulation that the new test method has better power than the x2-test.
In Section 2 we review some useful results for this note. The asymptotic mean
and variance of a new test statistic for independence are derived in Section 3.
Finally these results are used to test independence of an observed Markov chain.

2. PRELIMINARIES
Consider a Markov chain i r k ,k = 1,2,. . .) with state space S = {1,2,3) and
transition probability matrix
 62 D. Q. WANG AND J. S. MARITZ

where i, j E S, every p i j z O and xjpij=

1. Suppose the Markov chain has been
observed for n + 1 time points and let nij be the number of transitions i to j out of
=xixj
the total n. We write the marginal sum n i = x j n i j , n . j = x i n i j and n = n , ,
Then the joint distribution of observed transition numbers nij, beginning in state r
nij.

and ending in state g, is given (Whittle, 1955) as follows:

where i, j~ S, T,(nij) is the (grth cofactor of the matrix I - p, matrix I =diag

(1, 1, I), matrix P=(fiij) and tij=nij/ni..
Bartlett (1951) shows that the joint distribution of the number of transitions is
asymptotically normal if all eigenvalues of the transition matrix, except the first
simple one, i.e, i.= 1, have modulus less than one. From formula (2.1) the
Downloaded by [University of Glasgow] at 03:24 20 December 2014

asymptotic mean values and second moments of the observed transition prob-
ability matrix can be calculated. If we let

then from the asymptotic joint normality of the nij we deduce the joint asymptotic
normality of ~ e (P).
c
From formula (2.1) the following results are derived.
i) The asymptotic mean value of fiij is

ii) The asymptotic covariance of jijis obtained (Whittle (1955)) as

Hence the covariance matrix C of vec(b) is obtained as follows:

 A THREE STATE MARKOV CHAIN

From the joint asymptotic normality we have

~((dij-~ij)(filrn-~lrn)(fikg-~kg)) =0
Downloaded by [University of Glasgow] at 03:24 20 December 2014

and

3. AN ALTERNATIVE STATISTIC FOR TESTING INDEPENDENCE

In this note we restrict attention to a three state Markov chain. Let us suppose
that the transition probability of a three state Markov chain is

Then for an observed seqence (to,(,, . . ., t,), the transition frequencies nij may be
written as a contingency table. The maximum likelihood estimates of transition
probability pij are fiij=nij/ni.. It is well known that if both the non-unit
eigenvalues of P are zero, then P represents an independent trials process. Hence
for a three state Markov chain, we propose a test of independence based on the
non-unit eigenvalues of the observed transition matrix.
 64 D. Q. WANG AND J. S. MARITZ

In order to determine the test statistic, we need to solve the characteristic

equation.

where
Downloaded by [University of Glasgow] at 03:24 20 December 2014

It is easily seen that the non-unit eigenvalue solutions, 2, and I,, of this equation
are

and

respectively, where

We know if the Markov chain is to be independent we have ILil =0, for i > 1, that
is, the non-unit eigenvalues L2 and 2, are both zero. Hence, according to the
formula (3.1) we can easily obtain condition A + B - 1= 0 if the Markov chain is
independent.
Firstly we find the asymptotic mean and variance of d +B- 1, where 2 and 8
are defined as above. Using the results in Section 2, we obtain after some
calculation the asymptotic results
 A THREE STATE MARKOV CHAIN 65

and

var(A+B-1)

=Var(fill + f i 2 2 + f i 3 3 +fi31fi13 + f i 2 3 f i 3 2 +i)12fi21 - f i l l i ) 2 2 - f i l l f i 3 3 - f i 2 2 f i 3 3 ) *

Under the hypothesis of independence, these results simplify to

d ( A + B - 1)=0

and

where {xi,i ~ S are

j the stationary probabilities which are calculated by
Downloaded by [University of Glasgow] at 03:24 20 December 2014

z l = ( ( l + h l ) + ( I +h.Jhl/hz)-l,

712 =(h3 + h4hl/h2)x1 and n3 = hl/h2xl,

where

b I-f-e
h3 = --- and h,=------.
1- c l -c

Therefore we get the test statistic of the form

whose asymptotic distribution is N(O,1). Under the hypothesis of independence, we

have

J.S.C.S. C
 66 D. Q. WANG AND J. S. MARITZ

Table 1 Frequencies of father-son social class transitions

Son's social class
Upper Middle Lower Sum= ni.
- - - -

Father's social class

Upper. 125 135 19 279
Middle 128 1658 587 2373
Lower 9 425 41 1 845

Hence, in accordance with (3.3), we reject the independence hypothesis if

1W S I>Z,, where Z, is the 100(1- a ) lower percentage point of a standard normal
distribution.

4. A COMPARISON O F TEST METHODS FOR INDEPENDENCE O F A

MARKOV CHAIN
Downloaded by [University of Glasgow] at 03:24 20 December 2014

4.1 Numerical Example

We begin with an illustration, applying the test procedure proposed in Section 3
to a matrix of observed transition proportions. We consider the practical example
discussed by Parzen (1962) and also by Wang and Scott (1988) based on data
presented by Prais (1955). Table 1 gives transition frequencies relating to the social
class of father and son in a random sample of 3497 families. These frequencies are
derived from proportions given by Prais (1955) and the groupings 'upper', 'middle',
'lower' are those used in Parzen (1962). The transition process is a three-state
Markov chain whose transition probabilities are estimated by the proportions
calculated from the frequencies given in Table 1.
The estimated transition probability matrix is

From Table 1 and matrix we derive the following observed values of the
statistics / W S (and x2:

According to both statistics the hypothesis is clearly rejected. This example with its
large number of transitions and rather obvious lack of independence is not
 A THREE STATE MARKOV CHAIN

Table 2 Observed test size being the proportion of rejec-

tions of the hypothesis of independence in 100 trials

Observed test size

n=219 n=401 n=974 n=1897
WS test 0.09 0.09 0.06 0.09
x 2 test 0.06 0.08 0.10 0.10

informative about the relative powers of the two tests. However, we shall use it in
the next section as the basis of a simulation study involving smaller sample sizes.

4.2 A Comparative Simulation Study

The simulation studies presented here have been devised to provide a check on the
accuracy of the asymptotic formula obtained in Section 3, and also to compare the
powers of the IWSI and x2 tests for some finite sample sizes.
The results in Table 2 were generated using the elements of in (4.1) as true
transition probabilities. Because of the process represented by a being so far from
Downloaded by [University of Glasgow] at 03:24 20 December 2014

independent the hypothesis of independence is rejected by both tests in every one

of the simulations.
In order to obtain a more realistic assessment of the relative advantages of the
two test methods one needs to consider a Markov chain in which the degree of
dependence can be varied. We consider a Markov chain which is a function of the
parameter 6. Its transition matrix is P(6) (Wang and Scott, 1988) given by

where 0 1 6 5 1, P,, is the transition matrix of a Markov chain, and matrix PIN,
has for each row the stationary distribution of the chain represented by P,,.
Matrix P(6) represents a Markov chain whose degree of dependence is determined
by the value of 6.
a
For our simulations we have taken P,,= b, where is the matrix in (4.1), and

Table 2 shows some results for the case 6 = 1, i.e. independence. They indicate
that the normal approximation of the distribution of WS, at level c(=0.10, is
accurate enough for practical purposes if the total number of transitions exceeds
200.
To compare the power of the two test methods, we take the parameter 6=0.225,
the new observed Markov chain closes to non-independent, and its transition
matrix is
 D. Q. WANG AND J. S. MARITZ

Table 3 Test power of Markov chain derived from

6 = 0.225
Test power
n=219 n=401 n=974 n=1897
W S test 0.99 0.98 0.93 0.99
l 2 test 0.31 0.43 0.78 0.97

Table 3 shows the results of test power for 6=0.225 under the level rx=O.lO in
100 trials.
These results indicate the new test statistic WS has greater power, and the test
method is better than X 2 test method and bootstrap test.
Downloaded by [University of Glasgow] at 03:24 20 December 2014

References
Ahmad, R. (1985). O n test of independence for the multi-way Markovian contingency tables. Pak. J.
Statist. 1 ( I ) , 39-58.
Bartlett, M . S. (1951), The frequency goodness of fit test for probability chains. Proc. Camb. Phil. Soc.
47, 86-95.
Parzen, E. (1962). Stochastic Processes. Holden-Day, San Francisco, p. 257.
Patel, H . I . and Ahmad, R. (1973). O n test of independence for r x c Markovian contingency tables.
Ann. Inst. Stat. Math. 25, 355-361.
Prais, S. J. (1955). Measuring social mobility. J. Royal Stat. Soc. A , 118, 5 6 6 6 .
Wang, D.Q . and Scott, D. J. (1988). Testing a Markov chain for independence. Communications in
Statistics-Theory and Methods. 18 ( 1 I), 40854103.
Whittle, P. (1955). Some distribution and moment formulae for the Markov chain. J . Royal Stat. Soc.
B, 17, 235-242.