A bit of

history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Lecture 2:
Bayesian Computation
Hedibert Freitas Lopes
The University of Chicago Booth School of Business
5807 South Woodlawn Avenue, Chicago, IL 60637
http://faculty.chicagobooth.edu/hedibert.lopes
hlopes@ChicagoBooth.edu
1 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Outline
1 A bit of history
2 MC integration
3 MC via IS
4 Rejection method
5 SIR method
6 MH algorithms
7 Simulated annealing
8 Gibbs sampler
9 SUR model
10 BVAR
11 References
2 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
MC in the 40s and 50s
Stan Ulam soon realized that computers could be used in this
fashion to answer questions of neutron diusion and
mathematical physics;
He contacted John Von Neumann and they developed many
Monte Carlo algorithms (importance sampling, rejection
sampling, etc);
In the 1940s Nick Metropolis and Klari Von Neumann designed
new controls for the state-of-the-art computer (ENIAC);
Metropolis and Ulam (1949) The Monte Carlo method. Journal of the American Statistical Association.
Metropolis et al. (1953) Equations of state calculations by fast computing machines. Journal of Chemical
Physics.
3 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Monte Carlo methods
We introduce several Monte Carlo (MC) methods for
integrating and/or sampling from nontrivial densities.
MC integration
Simple MC integration
MC integration via importance sampling (IS)
MC sampling
Rejection method
Sampling importance resampling (SIR)
Iterative MC sampling
Metropolis-Hastings algorithms
Simulated annealing
Gibbs sampler
Based on the book by Gamerman and Lopes (1996).
4 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
A few references
MC integration (Geweke, 1989)
Rejection methods (Gilks and Wild, 1992)
SIR (Smith and Gelfand, 1992)
Metropolis-Hastings algorithm (Hastings, 1970)
Simulated annealing (Metropolis et al., 1953)
Gibbs sampler (Gelfand and Smith, 1990)
5 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Two main tasks
1 Compute high dimensional integrals:
E

[h()] =
_
h()()d
2 Obtain
a sample {
1
, . . . ,
n
} from ()
when only
a sample {

1
, . . . ,

m
} from q()
is available.
q() is known as the proposal/auxiliary density.
6 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Bayes via MC
MC methods appear frequently, but not exclusively, in modern
Bayesian statistics.
Posterior and predictive densities are hard to sample from:
Posterior : () =
f (x|)p()
f (x)
Predictive : f (x) =
_
f (x|)p()d
Other important integrals and/or functionals of the posterior
and predictive densities are:
Posterior modes: max

();
Posterior moments: E

[g()];
Density estimation: (g());
Bayes factors: f (x|M
0
)/f (x|M
1
);
Decision: max
d
_
U(d, )()d.
7 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
MC integration
The objective here is to compute moments
E

[h()] =
_
h()()d
If
1
, . . . ,
n
is a random sample from () then

h
mc
=
1
n
n

i =1
h(
i
) E

[h()] as n .
If, additionally, E

[h
2
()] < , then
V

h
mc
] =
1
n
_
{h() E

[h()]}
2
()d
and
v
mc
=
1
n
2
n

i =1
(h(
i
)

h
mc
)
2
V

h
mc
] as n .
8 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example i. MC integration
The objective here is to compute
1
p =
_
1
0
[cos(50) + sin(20)]
2
d
by noticing that the above integral can be rewritten as
E

[h()] =
_
h()()d
where h() = [cos(50) + sin(20)]
2
and () = 1 is the
density of a U(0, 1). Therefore
p =
1
n
n

i =1
h(
i
)
where
1
, . . . ,
n
are i.i.d. from U(0, 1).
1
True value is 0.965.
9 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References

h
((

))
0.0 0.2 0.4 0.6 0.8 1.0
0
1
2
3
4
Sample size (log10)
p
0
.
0
0
.
4
0
.
8
1
.
2
1 2 3 4 5 6
10 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
MC via IS
The objective is still the same, ie to compute
E

[h()] =
_
h()()d
by noticing that
E

[h()] =
_
h()()
q()
q()d
where q() is an importance function.
11 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
If
1
, . . . ,
n
is a random sample from q() then

h
is
=
1
n
n

i =1
h(
i
)(
i
)
q(
i
)
E

[h()]
as n .
Ideally, q() should be
As close as possible to h()(), and
Easy to sample from.
12 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example ii. Cauchy tail
The objective here is to estimate
p = Pr ( > 2) =
_

2
1
(1 +
2
)
d = 0.1475836
where is a standard Cauchy random variable.
A natural MC estimator of p is
p
1
=
1
n
n

i =1
I {
i
(2, )}
where
1
, . . . ,
n
Cauchy(0,1).
13 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
A more elaborated estimator based on a change of variables
from to u = 1/ is
p
2
=
1
n
n

i =1
u
2
i
2[1 + u
2
i
]
where u
1
, . . . , u
n
U(0, 1/2).
14 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
The true value is p = 0.147584.
n p
1
p
2
v
1/2
1
v
1/2
2
100 0.100000 0.1467304 0.030000 0.001004
1000 0.137000 0.1475540 0.010873 0.000305
10000 0.148500 0.1477151 0.003556 0.000098
100000 0.149100 0.1475591 0.001126 0.000031
1000000 0.147711 0.1475870 0.000355 0.000010
With only n = 1000 draws, p
2
has roughly the same precision
that p
1
, which is based on 1000n draws, ie. three orders of
magnitude.
15 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Rejection method
The objective is to draw from a target density
() = c

()
when only draws from an auxiliary density
q() = c
q
q()
is available, for normalizing constants c

and c
q
.
If there exist a constant A < such that
0
()
A q()
1 for all
then q() becomes a blanketing density or an envelope and A
the envelope constant.
16 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Blanket distribution
6 4 2 0 2 4 6
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
0
.
5
pi (target)
q (proposal)
Aq
17 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Bad draw
6 4 2 0 2 4 6
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
0
.
5
pi (target)
q (proposal)
Aq
q
q
pi/Aq=0.41
18 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Good draw
6 4 2 0 2 4 6
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
0
.
5
pi (target)
q (proposal)
Aq
q
q
pi/Aq=0.89
19 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Acceptance probability
6 4 2 0 2 4 6
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8

A
c
c
e
p
t
a
n
c
e

p
r
o
b
a
b
i
l
i
t
y
20 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Algorithm
Drawing from ().
1 Draw

from q();
2 Draw u from U(0, 1);
3 Accept

if u
(

)
A q(

)
;
4 Repeat 1, 2 and 3 until n draws are accepted.
Normalizing constants c

and c
q
are not needed.
The theoretical acceptance rate is
c
q
Ac

.
The smaller the A, the larger the acceptance rate.
21 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example iii. Sampling N(0, 1)
Enveloping the standard normal density
() =
1

2
exp{0.5
2
}
by a Cauchy density q
C
() = 1/((1 +
2
)), or a uniform
density q
U
() = 0.05 for (10, 10).
Bad proposal:The maximum of ()/q
U
() is roughly
A
U
= 7.98 for (10, 10). The theoretical acceptance rate
is 12.53%.
Good proposal: The max of ()/q
C
() is equal to
A
C
=
_
2/e 1.53. The theoretical acceptance rate is
65.35%.
22 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
UNIFORM PROPOSAL
4 2 0 2 4
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
CAUCHY PROPOSAL
4 2 0 2 4
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
Empirical rates: 0.1265 (Uniform) and 0.6483 (Cauchy)
Theoretical rates: 0.1253 (Uniform) and 0.6535 (Cauchy)
23 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
SIR method
No need to rely on the existance of A!
Algorithm
1 Draw

1
, . . . ,

n
from q()
2 Compute (unnormalized) weights

i
= (

i
)/q(

i
) i = 1, . . . , n
3 Sample from {

1
, . . . ,

n
} such that
Pr ( =

i
)
i
i = 1, . . . , n.
4 Repeat m times step 3.
Rule of thumb: n/m = 20.
Ideally,
i
= 1/n and Var () = 0.
24 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example iii. revisited
UNIFORM PROPOSAL
4 2 0 2 4
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
CAUCHY PROPOSAL
4 2 0 2 4
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
Fraction of redraws: 0.391 (Uniform) and 0.1335 (Cauchy)
Variance of weights: 4.675 (Uniform) and 0.332 (Cauchy)
25 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example iv. 3-component mixture
Assume that we are interested in sampling from
() =
1
p
N
(;
1
,
1
) +
2
p
N
(;
2
,
2
) +
3
p
N
(;
3
,
3
)
where p
N
(; , ) is the density of a bivariate normal
distribution with mean vector and covariance matrix . The
mean vectors are

1
= (1, 4)

2
= (4, 2)

3
= (6.5, 2),
the covariance matrices are

1
=
_
1.0 0.9
0.9 1.0
_
and
2
=
3
=
_
1.0 0.5
0.5 1.0
_
,
and weights
1
=
2
=
3
= 1/3.
26 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Target ()

2
2 0 2 4 6 8 10
0
2
4
6
27 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Target ()
th
e
ta
1
5
0
5
10
15
t
h
e
t
a
2
10
5
0
5
10
15
f
(
t
h
e
t
a
)
0.00
0.02
0.04
0.06
0.08
0.10
0.12
28 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Proposal q()
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
q() N(, ) where

2
= (4, 2)

and = 9
_
1.0 0.25
0.25 1.0
_
29 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Rejection method
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
Acceptance rate: 9.91% of n = 10, 000 draws.
30 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
SIR method
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q q
q
q q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
Fraction of redraws: 29.45% of (n = 10, 000, m = 2, 000).
31 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Rejection & SIR
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
2 0 2 4 6 8
0
2
4
6
1

2
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
2 0 2 4 6 8 10
0
2
4
6
1

2
32 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example v. 2-component mixture
Let us now assume that
() =
1
p
N
(;
1
,
1
) +
3
p
N
(;
3
,
3
)
where mean vectors are

1
= (1, 4)

3
= (6.5, 2),
the covariance matrices are

1
=
_
1.0 0.9
0.9 1.0
_
and
3
=
_
1.0 0.5
0.5 1.0
_
,
and weights
1
= 1/3 and
3
= 2/3.
33 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Target ()

2
2 0 2 4 6 8 10
0
2
4
6
34 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Target ()
th
e
ta
1
5
0
5
10
15
t
h
e
t
a
2
10
5
0
5
10
15
f
(
t
h
e
t
a
)
0.00
0.02
0.04
0.06
0.08
0.10
0.12
35 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Proposal q()
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
36 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Rejection method
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
Acceptance rate: 10.1% of n = 10, 000 draws.
37 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
SIR method
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
Fraction of redraws: 37.15% of (n = 10, 000, m = 2, 000).
38 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Rejection & SIR
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
2 0 2 4 6 8 10
0
2
4
6
1

2
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
2 0 2 4 6 8 10
0
2
4
6
1

2
39 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
MCMC history
Dongarra and Sullivan (2000) list the top algorithms with the
greatest inuence on the development and practice of science
and engineering in the 20th century (in chronological order):
Metropolis Algorithm for Monte Carlo
Simplex Method for Linear Programming
Krylov Subspace Iteration Methods
The Decompositional Approach to Matrix Computations
The Fortran Optimizing Compiler
QR Algorithm for Computing Eigenvalues
Quicksort Algorithm for Sorting
Fast Fourier Transform
40 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
70s and 80s
Metropolis-Hastings:
Hastings (1970) and his student Peskun (1973) showed that
Metropolis and the more general Metropolis-Hastings algorithm
are particular instances of a larger family of algorithms.
Gibbs sampler:
Besag (1974) Spatial Interaction and the Statistical Analysis of Lattice Systems.
Geman and Geman (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images.
Pearl (1987) Evidential reasoning using stochastic simulation.
Tanner and Wong (1987). The calculation of posterior distributions by data augmentation.
Gelfand and Smith (1990) Sampling-based approaches to calculating marginal densities.
41 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
MH algorithms
A sequence {
(0)
,
(1)
,
(2)
, . . . } is drawn from a Markov chain
whose limiting equilibrium distribution is the posterior
distribution, ().
Algorithm
1 Initial value:
(0)
2 Proposed move:

q(

|
(i 1)
)
3 Acceptance scheme:

(i )
=
_

com prob.

(i 1)
com prob. 1
where
= min
_
1,
(

)
(
(i 1)
)
q(
(i 1)
|

)
q(

|
(i 1)
)
_
42 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Special cases
1 Symmetric chains: q(|

) = q(

|)
= min
_
1,
(

)
()
_
2 Independence chains: q(|

) = q()
= min
_
1,
(

)
()
_
where (

) = (

)/q(

).
43 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Random walk Metropolis
The most famous symmetric chain is the random walk
Metropolis:
q(|

) = q(|

|)
Hill climbing: when
= min
_
1,
(

)
()
_
a value

with higher density (

) greater than () is
automatically accepted.
44 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example iv. RW Metropolis
0 2 4 6 8
0
2
4
6

2
0 2 4 6 8
0
2
4
6

2
0 2 4 6 8
0
2
4
6

2
0 2 4 6 8
0
2
4
6

2
q(|
i
) N(
i
, 0.25
2
).
45 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example iv. Ind. Metropolis
2 0 2 4 6 8 10
0
2
4
6

2
2 0 2 4 6 8 10
0
2
4
6

2
2 0 2 4 6 8 10
0
2
4
6

2
q
qqq
qqqq
qqq
qqqqqqqq
qqqqqqqqq
q
qqqqqqqqq
qqqqqqq
qq
qq
qqqqqqqqqqqq
qqqqqqqqq
q
q
qqqqq
qqqqqqq
qqqqqqqq
qq
qqq qq
qq
qqq
q
qqqq
q
qqq
qqqqqq
qqqqqqqqqq
qqqqqqq
qqqqqq
qqqqqq
qq
qq
qqq
qqq
q
qqqq
q
qqqqq
q
qqqqq
qqq q
qq
q
qq qqqqqqqqqq
qq
qq
qqq
qqqqqqqq
qqqqqqqqqqq
q
qqqqqqqq
q
q
qqqqq qq
qqqqqqqqqqq
qqqqqqq qqqq
qq qqqqqq
qqqqqqqqq
q
qqqqqq
qqqqqqqqq
qqq
qqqqq
q
q
q
qqqq
qq
qqqq
qqqqqqqqq
qqq
qq
q
qqqqq
q
qqqqqqqq
qqqqqq
qqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqq
qqq
qqq
qqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqq
q
qqq
q
qq
qqq
qqqqq
qqq
qq
q
q
qq
q
qqqqqq
qqq
qqqqqqq
qqq
qq
q
qqq
qqq
qqqq
qqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqq
q
qq
qqqq
q
q
qqqq
qqqqqq
qqqqqqqqq
q
qq
qqqqq
q
qqqqqqqqqqqqqqqq
qqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qqqqqqq
qqqqqqqq
qqqq q
qqqqqqq
qqqq
q
qq
q
q
qqqqqqqqqq
qqqqqqqq
qqqqq
qq
qqqq
qqqq
q
q
q
q
qqqqqqq
q
qq
q
qqqqqqq
qq
qqqqqqq
qq
q
qqq
q
qqqqqqq
qq
q
qqqq
qq
qqqqq
qqqqqq
q
qqqqqq
qqqqqq
qqq
q
qqqqq
qqqqqq
qq
qqqqq
qqqqqqqqqq
qqq
qqqqq
qqq
q
qqqqq
qqqqq
qq
q
q
q
qqqqq
qqqq
qqq
q
qqq
q
qq
qqqqqqqq q
q
qqq
q
qqq
qq
q
qqq
qq
qqqqqqqqqqq qqqqqq
qq
q
qqq
qqq
qq
qqq
q
q
qqq
qqqqqqqqqqq
q
qqqqqqq
qq
qq
qqqqq
q
q q
qq
q
q
q
q
qqqqqqq
qqq
q
qq
q
qqq
qqq
qqqqqqqqq
qqq
qqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqq
qq q
q q
qqqqqqqqqq
q
qqqqqqqqq
qq
q
qqqqq
qqqq
q
qqqqqqqq
qq
q
qqqq
qqq
qqqq
qq
qqqqqqqq
qqq
qqqqqqqqqqqqqqqq
q
qqq
qqq
q
q
qqq
qqq
qq
qqqqqqqqq
qqqqq
q
qqq
q
qq
qqqqq
qq
qqqqqq
q
qqqqqqqqqq
qqqq
q
q
qqqqqqq
qqqqqqqqqqqqq
q
qqq
qqqqqqqqqqq
q
qqqqqqq
qqq q
qqqq
qqqqq
qqq
qqqqq
qq
qq
q
qq
q
qq
qqqqqqqqqqqqq
qqqq
qqqqq
qqqq
q
qqqqqqqq
qqq
qqq
qq
q qq
qqqq
qqqqqq
qqq
qqq
q
q
qqqqqqqqq
qq
qq
qqqqq
q
qqq
qq
q
q
qqq
qq
qq
qqqqqqqqqqqqqqqqqq
qq
qqqqqqq
q qqq
qqqq
qqqqqq
qq
q
qqqq
qq
qqq
q
qq
qq
qqqqqq
qqqqqqqqqqq
q
q
qqq
q
q
qq
qq
q
qq
qqqqqqqqqqqqqq
qqqqqq
qqq
q
q
qq
q
qqqqq
qqqqqqq
q
qqqqqqqqqqq
q
qqq
q
q
qq
q
qqq
q
q
qqq
qqqqq
qqq
qqqqqqqq
qqqq
qqq
q
qqqqqqqq
qqqqqq
q
qqqqq
qqqqqqqqq
qq
qqqqqq
qqqqqqqqqqq
qq
qqqq
q
qqq
qqq
qq
q
q
qqqqqqqqqq
qqqqqq
qq
q
qq
q
qqqq
qqqqqq
q
q
qqq
qqqqq
qqqqq qqqqqqqq
q
q
qqqqqq
qq
q
qqqqqqqq
qqq
q qqqqqqqqqqq
qqq
qqqqqqqqqq
qqqq
q
qqqq
qqq
qq
qqq
qqqqqq
qqq
qqqqq
qqq
qqq
q
qq
qqq
qqq
q
q
q
qqqq
qqq
qq
qqqqqqq
qqqqqq
qqqq
qqqqqqqqqqqqqqq
qq
qq
q
qqqq
qqqqq
qq
qqqq
qq
q
qqqq
qqq
qqqqqqqq
qq
qq
qqq
qq
qqqqq
q
qqqqqqqqqqq
qqqq
qqqqqqqq qqqq
qqq
qqq
qqqq
q
q
qqq
q
qqqqqqq
q
qqq
qqqqqqqqqqqqqqqqqqq
q
q
qqqqqqqqqq
qqqqq
qqqq
qqq
qqq
qq
qqqqq
qqq
q
qq
q
qqq qqqqqqqqqq q
qqq
qqq
2 0 2 4 6 8 10
0
2
4
6

2
q() q
SIR
() N(, ).
46 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example iv. Autocorrelations
Iterations

1
0 500 1000 1500 2000
0
2
4
6
8
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

2
0 500 1000 1500 2000
0
2
4
6
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

1
0 500 1000 1500 2000

2
0
2
4
6
8
1
0
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

2
0 500 1000 1500 2000
0
2
4
6
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
47 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example v. RW Metropolis
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
48 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example v. Ind. Metropolis
q
qqq
qqqqq
qq
qq
qqqqqq
qqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqq
qqqqqqqqqqqqqqqqqqq
q
qq
qqqqqq
qqqq
qqqq
qqqqqq qqqqq
qqq
qqqqqqqqq
qqqqqq
qqqqqq
qq
qqqqq
qqq
q
qqqq qqqqqq
qqqqqqqqq
q
q
q
q
q
q
qqqqqqqqqqqqqq
qqq
qqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqq
qqq
qq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qqqqq
q
qqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqq
qqqqq
qqqqqqqqqqq
qqqqq
q
qqqqqqqq
qqqqqqq
qq
q
qqqqqqqq
qq
qqq
qqqqqqqqqqq
qqq
q
qqqqqq
qqqqqqqqqq
qqq
qqqqqqqq
q
qqqq
qq
q
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqq
q
qq
qqqq
q
q
qq
qq
qqqq
qq
q
qqqqqq
qq
q
qq
qqqqqq
qqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqq
qqqqqqq
qqqqqqqq
qqqqq
qqqqqqqqq
qq
qqq
q
q
qqqqqqq
qqq
qqqqqqq
q
qqqqq
qq
qqqq qqqqq
qq
q
qqqqqqqq
qq
qqqqqqqq
qq
qqqqqqqqq
qqqq
q
qqqqqqq
qq
qqqqq
qqqqqqq
qqqqqqq
qqqqqq
qqqqqq
qqqq
qqqqq
qqqqqq
q
q qqqqq
qqqqqqqqqqqqqqqqqqqqq
q
qqqqq
qqqqqqqqqqqqqqqqqqq
qqq
qqqq
q
qq
qqqqqqqq qq
qqqqqqq
qq
qqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
q
q
q
qqqqqqqqqq
qqqqq
qqqqq
q
q
q
q
qqqqqqq
qqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq q
qqqqq
qq
qq
q
qqqqqqqqqqqqqqq
qqq
qqqq
qq
qqqqqqqq
qqq
qqqqqqqqqqqqqqqq
qqqqqqq
q
qqqq
qqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqq qqqqqqqqqqqqqq
qqq
qqqqqqqqqqq
qqqqqqqq
q
qqq
qqqqqqqqq
qqq
qqqqq
qq
qq
q
qq
q
qqqqqqqqqqqqqqq
qqqq q
qqqq
qqqq
q
qqqqqqqq
qqq
qqq
qq
qqqqqqq
qqqqqqqqqqqq
q
qqqqqqqqqq
qq
qqqqqqqqqqqqq
q qqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
q
qqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
q
qqqqqqqqqqqqqq
qqqqqqqqqqqqq
q
qqqqqqqqqqqqq
qqqqqqqqqqq
q
qqqqq
qqq
qqq
q
q
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqq
qqq
qq
q
qqqqqqqqqqqqqqqqqqq
q
qq
qqqqq
qqqqqqqqqqq
qqqqq
qqqqq qqqqqqqq
q
q
qqq
q
qq
q
q
q
qqqqqqqqqqq
q qqq
qqqqqqqq
qqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqq
qqqqqq
qqq
qqqqqqqq
qqqq
qq
qqq
qqq
q
q
q
qqqqqqqqqqqqqqq
q
qqqqqq
qqqq
qqqq
qqqq
qqqqqqq
qq
qqq
qqqq
qqq
qq qqqqqq qq
qqqqqqqqqqqqqqqqqq
qq
qq
q
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqq
qqqqqqq
q
q
qq
qqqqqqqqqqqqqq
qqqqq
q
q
qqqqqqqqqq
qqqqq
qqqqqqqqqq
qq
qqqqqqqqqqqqqqq
qqqqqqqqqq q
qqq
qqq
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
q
qqq
qqqqq
qq
qq
qqqqqq
qqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqq
qqqqqqqqqqqqqqqqqqq
q
qq
qqqqqq
qqqq
qqqq
qqqqqq qqqqq
qqq
qqqqqqqqq
qqqqqq
qqqqqq
qq
qqqqq
qqq
q
qqqq qqqqqq
qqqqqqqqq
q
q
q
q
q
q
qqqqqqqqqqqqqq
qqq
qqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqq
qqq
qq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qqqqq
q
qqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqq
qqqqq
qqqqqqqqqqq
qqqqq
q
qqqqqqqq
qqqqqqq
qq
q
qqqqqqqq
qq
qqq
qqqqqqqqqqq
qqq
q
qqqqqq
qqqqqqqqqq
qqq
qqqqqqqq
q
qqqq
qq
q
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqq
q
qq
qqqq
q
q
qq
qq
qqqq
qq
q
qqqqqq
qq
q
qq
qqqqqq
qqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqq
qqqqqqq
qqqqqqqq
qqqqq
qqqqqqqqq
qq
qqq
q
q
qqqqqqq
qqq
qqqqqqq
q
qqqqq
qq
qqqq qqqqq
qq
q
qqqqqqqq
qq
qqqqqqqq
qq
qqqqqqqqq
qqqq
q
qqqqqqq
qq
qqqqq
qqqqqqq
qqqqqqq
qqqqqq
qqqqqq
qqqq
qqqqq
qqqqqq
q
q qqqqq
qqqqqqqqqqqqqqqqqqqqq
q
qqqqq
qqqqqqqqqqqqqqqqqqq
qqq
qqqq
q
qq
qqqqqqqq qq
qqqqqqq
qq
qqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
q
q
q
qqqqqqqqqq
qqqqq
qqqqq
q
q
q
q
qqqqqqq
qqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq q
qqqqq
qq
qq
q
qqqqqqqqqqqqqqq
qqq
qqqq
qq
qqqqqqqq
qqq
qqqqqqqqqqqqqqqq
qqqqqqq
q
qqqq
qqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqq qqqqqqqqqqqqqq
qqq
qqqqqqqqqqq
qqqqqqqq
q
qqq
qqqqqqqqq
qqq
qqqqq
qq
qq
q
qq
q
qqqqqqqqqqqqqqq
qqqq q
qqqq
qqqq
q
qqqqqqqq
qqq
qqq
qq
qqqqqqq
qqqqqqqqqqqq
q
qqqqqqqqqq
qq
qqqqqqqqqqqqq
q qqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
q
qqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
q
qqqqqqqqqqqqqq
qqqqqqqqqqqqq
q
qqqqqqqqqqqqq
qqqqqqqqqqq
q
qqqqq
qqq
qqq
q
q
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqq
qqq
qq
q
qqqqqqqqqqqqqqqqqqq
q
qq
qqqqq
qqqqqqqqqqq
qqqqq
qqqqq qqqqqqqq
q
q
qqq
q
qq
q
q
q
qqqqqqqqqqq
q qqq
qqqqqqqq
qqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqq
qqqqqq
qqq
qqqqqqqq
qqqq
qq
qqq
qqq
q
q
q
qqqqqqqqqqqqqqq
q
qqqqqq
qqqq
qqqq
qqqq
qqqqqqq
qq
qqq
qqqq
qqq
qq qqqqqq qq
qqqqqqqqqqqqqqqqqq
qq
qq
q
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqq
qqqqqqq
q
q
qq
qqqqqqqqqqqqqq
qqqqq
q
q
qqqqqqqqqq
qqqqq
qqqqqqqqqq
qq
qqqqqqqqqqqqqqq
qqqqqqqqqq q
qqq
qqq
5 0 5 10 15

1
0

5
0
5
1
0
1
5
1

2
49 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example v. Autocorrelations
Iterations

1
0 500 1000 1500 2000
4
5
6
7
8
9
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

2
0 500 1000 1500 2000
0
2
4
6
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

1
0 500 1000 1500 2000

2
0
2
4
6
8
1
0
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

2
0 500 1000 1500 2000
0
2
4
6
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

1
0 500 1000 1500 2000

5
0
5
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

2
0 500 1000 1500 2000
0
5
1
0
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

1
0 500 1000 1500 2000

5
0
5
1
0
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
Iterations

2
0 500 1000 1500 2000
0
5
1
0
1
5
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
1
.0
Lag
A
C
F
50 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example vi. tuning selection
Tthe target distribution is a two-component mixture of
bivariate normal densities, ie:
() = 0.7f
N
(;
1
,
1
) + 0.3f
N
(;
2
,
2
).
where

1
= (4.0, 5.0)

2
= (0.7, 3.5)

1
=
_
1.0 0.7
0.7 1.0
_

2
=
_
1.0 0.7
0.7 1.0
_
.
51 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Target distribution

2
0.1
0.2
0.3
0.3
0
.4

0.4
0.5
0.6
0.7
0.8
0
.9

2 0 2 4 6
0
2
4
6
8
2
0
2
4
6
0
2
4
6
8
0.0
0.2
0.4
0.6
0.8

1
52 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
RW Metropolis
q(, ) = f
N
(; , I
2
) and =tuning.
q
q q q q q
q
q
qq
q qqq
q q
qq q qqq
q q
qq q q
q
q q q q
q q
q
qq q q q q q q
q
q
qq
q
qq
qqq
q qq
q q qq qqqq qq
q qq
qq
q
q
qq
q
q
q
q
q
q qqq
q
q
qq q
qq
q
q
q
q
q
q q
q
q
qq q
q
qq qq q
q
q
qq
q
q
q
q
q
q
q q
q
q qqq
q
q
q qqq q q q q q
qqq q
q
q q q
q qq qq
q q
q
q qq
q q q
q
q qqqq
q q
qq
qq
q
q qqq
q
q
q
q q q
q q q
q qqq
q q q q qqq q
q
q qq q
q q
qq
q q
q
q
qq
q
qqq
q q qqq
q
q
q q q qqq
q
q
q
q q q qq
qqq
qq
qq q
q
q q
q q
q qq qq q q
qqqqq qq q q
q qq q
q
q
q
q q
q
qqq
q
q q
q qqq
qqq
q q q
qqqq qqq q
q q
q q qq qq qqq
q q
q q
qqq q
q q q
qqq
q
qq
q q
q
q qq
q q
qq
qq
q
q
q
q qqqq
q
qq
q qq q
q
qqq
q qqq q
q
qq
qqq
q q q q q
q
q
q
qq
q
qqq qq qqqq q q
q qq
qq q
q qq q
q
qq
q q
q
q
qq q q
q
q
q q
q q
q
qq
q
q
q q
q qq
qq qq
q qq
qqq
q
q q qq q
q
qqq
q q qq
q q q
q
q q q
q
q q
q q q q
qq q
q q
q q q
q qq
q
qq
q
qq
q qq q q
q q q qq
q q q
qq qqq
q qq q
qq q q qq q
q qqq
q
q qq
q
qq
qq qq qq
q
qq
q q
q q
q
qq q
q
q q
qq q qq q q q qq
q
q
q q
q q q
q
qq q q
q q q q
q
q q q qqq
q
q
q q
qqq q q q q q q
qq
q
q
q q
q qq q
qqq
q q
qq
qqq
q q q
q
q
q
q q
q
q
q q qq
q
q
q q q
q q q
q q q qq qq
q q q q
q
q
qqq q
q q
q q
q
q q
q q
q
q
q
qqq
q
q
q q q
q
q
q
q
q
q
q q q q q q q q
q
q q
q
q q qq
q
qqq q
q q q q qqq qq
q
q
q
q qq q qq
q q q q qq
qq
qq
qq
q qq
qq
qq q q
q
q qqq q
q q
q qq qq
q
q
q
qq q
qq q q qqqq qq
q q
q q q
qqqq q
q q
q
q
q
qqq q q q
q
q
q
q
q q
q qq qq
q
q
q q
qq
q
q
q
q q q q q
q q q qq
qq
q q
qqq q
q
q
q
qq qq
qqq q q qq q q q
q qq
q q
qqq
q q
qq
qq
q qq q q
q qqqq
q
q
q
qq
q q
q
q
q q qq
q
q
q q
q q
q q q q
qq qq
q q q
qq qqq
q
q q
qqqqq q
q
q
q
q
q
q qq qqq q
q
q qq
q
q
q
qq q
qqq q
q qq
qq
q qq q
q qq
q
qq
q
q q q
q
q
q q qq q
qqq q q
qq qq q
q q
qq q q q q q
q
q
q
q
q qqqqqq
q q qqq q qq q q
q q q
qq qq qqq
q
q
q q q
q
qq
qq q
q
qq
q
q
q
q
q
q q
q
q
q q q q q
q
q
qqqqq q qq q
q
q q q q
qq
q
q q q q
q
qq qq q
qq qqqq
q
q
q
q
q
q
q q q
q
q q
q
q
q
q q
qq
q qq
qq qq q q q q
qq
q
q q
qq q q q
q q
q
qq q q
qqqq qq
qq q
q q q qq
qqq
qq q
q
q
qqqq
q
q q q q q q q
q q
q
q
q
q
q q q
qq q q qq
q qq q
qq
q
q
q qq qq
q
q
q
q
q
qq q
q
q
q qq qq q
q
q q qq
q q qq
qq
qq q
q q
q q q
qqq
qq
qq q
q
q q
q q q
qq q
q q
q qq q
q qqqqq q
q
q q qqqq
q
q
q
q q
q qqqq
q qq q
qq
q q q q q
q q
q
q
qq qq
qqq
q
q
q
qq q
qq q
q q q
q q q
qq
q
qq
q
q q qq
qq qqq
q
qqq qq
q
q
q qq qq q
q
qq q
qqq q
q
qq
qq qq qq
q
q
q
qqq q
q
q
qq q
q
q qq
qq q qq q q
qq
qq q
q
q
q
q
q
q
q
q q q q q
qq q q
q
q q
q
q
q qq
qq
q
q q q qqq q
q q q q
q q q qqq
q q
q
qqq qqq q
q
qq
qq q
q q
qq q qqq
qq
q qq q
q
q
q
qq q
q q
q q qq qq q
qqq
q q
q q
q q
q
q
q
q
q qq q
qq qqq
qq
q
qq
qq q qq
q
qq
q
qqq
qq q q q
q
qqq
qq q qq qqq
qq q qq
q
qqqqq
q
q q q qqq q q
qq q q q
q
q
qqqqqq
q
q
q q
qq q qqq qq q qq
q q
q q q q
q q
q
q
q
q
q q
q qq q
q q qqq
q qq q
q q
q
q q q
q
q q
q
q q q q qqq q
q q q q
q
qqq
q q q q q qq q q
q q
q
qq
q q q
q
q q q
q
qq q
q
q
q q qqq
qq q
q
qq
q
q qq
q
q
q
q q qq
qq q
q
q q
qq
q q
q q
qq qqqq
q qq qq
qq
q
qq q
q
q q q
q q
q
qq
q
q q
qq
q qq
q q
q
q
q q
qq
q q
qqqq
q
q
qq q q
q
q
qq
q
q
q q q q q qqqq qq
q
q
q q q
qq
q q
q
q
q
qq
qq qqq q
q
q
q qq q q q
q
q
qqq
q
qq q q qq q q
q
q q qq
q q qq
qq
qqq q
qq q
qq
q
q
q q q
qq
q q
q
qq
q qq
q
q q
qq q q
q q q q qq q
q
q
q q q
q
qq q
q
q q q q q q
q
q q q qqq
qq q
qq q
q
q q
q
qq
q q q q q
q
q q
q
qq qq
q q q
qq qqqqq q q
q qq q q
q
q
qqq
q
qq q
q
qq
qq
q
qqq
q q
q
qqq qq qqqqqq qq
q q
q q q
qq q
q
q q q
q
qq q
q qq q q
qq q
qq
q qqq
q q
q
qq
q q qq
q
q
q q q
q q q
q qq
qq qqq q
q q qq
qqqq q q q
q
q
qq q q
qq
q q
q
qq
q
qq
q q qqqq qq
qq q q
qqq
q q
q
q q
q q qqqq
q q
q
q
qq
q
q
qq
q qqq q
qq
q
qq
q
q
q
q
qqqq qqq q
q
q
q
q
q
q
q q q
q qq
q q
qq qqq q
q
q q
qq
q q
q q
qq
qq q
qq
q q
q q
q q
q
q q
q q qq
qq q qq
qq
q
q q
qqqqqqq
q q qqq
qqq q q
qq
qq q
q
qqq
q
q q qq q qqq q
q
q q q
qqqqq q
q
q
q
q
q q
qq q q
qqq q
qqq
q
q
qq qq q q
q
q q qq
qq
qq q q q
q
q
q
q q qqqq
qq
q q q
q
q q q
q
q q qq q
qq q
q
q q q q qq q qq
qq q
q q q q q
q q
q
q
q
q
q
q q
q
q qq q q q qqq
qq qqq qq
q
q q q
q
q q
qq q
q
q
qq
q
q
q
q qq
q q
q qq
qq
q
q
qq
q
q qqq
q qqqq
qq q qqqq q
q q q
qq qq
q q q q q q
q q
q q q q
q
q
qq q q
q
q
qqqq q
qqq
q qqq qqq q
q q q q qqqqq
q q q
q q
q
q
q q qq q q q q q
qq q
q q
q q
q
q q q qq
qq
qq q
qq qqqq qqq q
q q
q
q
q q
q q
q q q
q q
q
qq qqq qq
qq qq q
q q q qq qq
q
q
q
q q
q
q
qq
q
q
qqq
qq qq q
q q q
q
q q
qq q
q
qqqq q
q qqq qqq q
q q q q
q
q
q qq q
qq
q q q q
q
q
q q q
q
q
q
q q qq
q qq q
q q
q
qq
q qq q q q q
qq q qq q
q
q
q
qq
q
qq
qqqq q
q q
q q qq
q
qq qq q q
q q q q
qqq q
q q q
qqq
qq q
qq qq q q qq
q qq
q
q
q
q
q q q
q
q
q q
q q q q qq q q
qqq q
q
q qqqq q
q q q
q q q q
q q
q
q
q
q q
q q q
q q
q
q q q qq q
q q q
qq
q
q q
q
q
qq
q q q
q
qqq
qq q q
qq q qq qq q q q q q q qq
q
qq q q
q
q q
q q
q qq q
q
q q
q
q q q
q q
q
q q q q q q
qq
q
q q
q q
q q qqq qq q
q
q
qq
q q q
q
q
qq q q q
q
q
q q
q qqqq
qq q q
q
q q q q
q q
q q
q
q
q
q
q q
q q
q q
q
q
q
q
q q qqqq q
q
q
q q
qqq
q q qq
q q q q q
q qq
qq
q q qq
q
q
q q qq qq q q qq
q q
qq
q
qq q
qq
qq
qq q
q
qq
qqqq
q
q
q
qq q
q
q q
q q q q q qq
q
q q qqq
q
qq
qq q q
q q
q
q
q
q qq q qq q q
q qqqq q
q
qqq q
qq q q
q q
q
qqq
qq
q
q q q
q
qqqq qq qq
qq
q
q qqq q
q qq q q
q
q q
q
q q
qq
qq q
q
q q
q q
q
q
q q qq
q q
qqq qq
q q qqqq q
q
q q
q
q
q q
qq qq q
qq q q qq
qq
qq q q qq q
qqqqq
q
q q qq q q q q q
q
qq q
q q
qqq
qq q
q
qqq q q
qq
q q
q qq qq
q qq q q
q
q q
q
q q
q
q
q q
q q
q
q
qq
qq
q q q
q
q
q qq
q
q
q qq
q
q q
q
q
q
q q q
q
q
q
q qq q q qq q qqq
q
q
q q
qq
q
q
q
qqq q
q qq qqq
q
qq q
qqqq
qqqqq
q
q
q
q q q q
q q q q
qq qq
q q q
q qq q q qq q qq
q
q q q q q q
q
q
q q
q qq
qqq q q
q q
q
q q
qqq q
qq q
q q
q
q qq qqq q
q qq
q q
qq
q
q q
qq q qq qq
qq
q qq
q
q
qqq
q
q q
qq
q
qq
q
q q q q q
q
qq q
qq
q q
q q q
q qqqq
q
q
q
q qqq
q q q q
q qq
qq qq qq
q
q
qq
q q
qq q q
q
qq qq
qq
q q
q q qqq q
q
q qq
q
qqq
qq
q
q
q
qq
q
qq q
q
q qq
q q qqqqqq
q
q
q q
q q qq
qqq q
q
q q
qq q q qqqq
qqq
q
q q q q q
qq
qq q
q
q qq
q q
q
q q q qq q
q q
q
q q
qq q q q q q
q
q q q q
q
q
qq q
q q
q
q
q
qqqqqq
q q
q
q q q q q
q q q
q
q
q
q
qq
qq
q
q
q
q
qq qqqq
qqq q
q q q
q
qq q q q
q qqqq
q q
q
qq
q q q
q q
q
qq q
q q
q
q qq
qq qq
q
q
qq
qq
qqqqq qq q q
qqq q
q
qqq
q qq qqqq
q
q q q q q
q qqq q q q
q q
q q
q
q
q
q q q
q q
qq q q qqq
q q
q qqq
q
q
qq q qqq q
q qq
q q
q
q q
q q
q q
qq q q
qq qqqqq
qq
qq q
q qq q
qq
q
qq
q q
qq q q q q qq
qq
q
qq
q
q
q qq q qqq
qq qq
q
q
qq
q q
q
q q q q q q
q
q
q
q q
q q
q q qq q
q q
q
qq
q
q q q qq
qq qq
q qq
q
q
qq
q
qq
qq
qq qq
qq q
q qqq q
q q
q q
q q q q
q
q
q q
q q qq qqq q q
qqq
q q
q
q
qq
q qqqq
q
qq
q
q qq q qq
qqqq q
q
q q q
q q
q q qq q q
q
q
q q q q
q q
q qq q
qq
qq q
q qq q q q
q
q
q q q
q
qq
q
q
q
q
qq
q q q qq
q q q qq
q
q
qq
q q
qqqqqq q
qq
q qqq q q q q q
q
q
q
q q
q q
qqq
q q qq q q
q
q q q q
qq
q q q
qq
q qqq
q qq
q
q q
q
q
q qq
q qq
q q
q
qq q
q
q
q q
q qqqq
qq
qqqq
q q
q q q q q q
q
qq qqqq
q
q
q
qq qq
q
qq q
q
q q qq q qq q qq
q
q q
q
qq
q
q q
qq q qq q qq q
qqqqqq qq
q q
q q
q
qq q
q qqq qq q
q qq
q
q
qq
qqq q
q
q q qq
q
q q q q
q
qqq q q
qq qq q q
qq
qq
q qq q
q
q q
q
q qqq q
q
qq
qq q q
q
qq
q
qq
q
qqq
q
qq q qq
qq
qq
qq
q q
q q
q q
q qq
q q
qq q
q
q q
q
q
qq q
q q q
qq q qqq q q q
q q qq
q
q q q q q
q q
q qq
q
q q q
q qqq q
qq
q
qqqq q
qq q
q
q
q qqq q
q qqq q qqqq
q q
q qq
q
q q
qq q
q
q q q
q q q
q qq
q
qq qqqq
qqq
q
q
qqq
qq q
q
q q
q
q
q q qqq
q
q qq
q
q q q
q qq q
q qq q
q qq
q
q
q
qq qqq
q
qqq
q q
qq
q qq q q
q
q
q q
qqq
q qqq q
q
q
q
q q q q q q
q
q
q qqq
q
qq q q q
qqq q
q
qq
q
q q q q
q
qq
qqqq q
q
q q q
q
qq
q q q q
q q q q
q
qq
q
q qq qq
qq
qq
q
qq
q q
qq
q
q
q q
qq
qq
q q q
q q
q
q
q
q
q qq
q
q
qq q
q q
q q
q
q
q
q
q
q q
q
qq
qq
q q qq qq
q
q q q
q
qq
q q
q
qq q q
q
q
q qq
qq
q
q qq q
q
q
q q
q
q
q q
q q q
q q
qq
q q q
q qq q
qqqq
qq q q q
q
q q
q q
q q q q q
q
q
q qqq q q q
q
q q
q
q
qqq qq
q qq
q
q q q
q q
q
q
q
qqq qqq
q q
qq
q
q
q
q
q q qq q qqqq q
q q
q
qqqq qq
q
qq
q
q
qq q
q
qqq q
q qq q
qq
q q
q
q q
q qqqqq
qq q
q
qqq q
q q q
qq q
q qqq q
qq q
q
q qq
q q q q q q
q q
q q qq
qq qq q
q q
q
qq q
q q q
q
q qqq
q q
qq
q q
qq
q
q q q
qq
qqq q
q
q
q qqq q q qqq
q
qq
q qq q
qq q
qqq
q
q q qqqq qq
q q qqqq q
q
q
q qq
q
q qq qq q
q qq
q q
qq qq
qq
q
qq
q
q qq qqq
qq qq q
q
q
qq
q
qq
q q
q q
q
q
q
q
q
qq q
q
q
q q q
q
q q
q qq q
qqq
qqq
qq
q q qq qq
q qq q
q q
q
q q q
qqq q q q
q
q q q qq
qq
q q qq q q
q
q q
q q
q qq q
q q q q qqq
q
q q q q qq
q q q
qq q
qqq
q q q q
q
qq q qq
q
q q
qq
qq
q qqq
q
q q qq q
qq q q qq q q q q
q qq
qq
q q
qq qq q
q qq
q
q
q q
qq q
q
q
q
q q
q
qqq qq q
q
q qq q
q
q
qq
q
q
q q
q qqq
qq
q
q q
q q
qq q
q qq
q q
qq
qq q q q q qqqq q q
q
q qqqq
qq
q qq
q q
q
q
qq qq
q q q qqqqqq
q
q
q q
q q
qq
q
qq
q q
q q
q
qq
q qq q q qqq
q
q
q
q
q qq q
qq
q
qqqqq
q
2 0 2 4 6
0
2
4
6
8
tuning=0.01
Initial value=(4,5)
Acceptance rate=93.8%
qqqqqqqqqqqq
qq
q
q
q
qq
q q
q q
q
qqq
q
q
q qq
qq
qqqq
q
q
qq
q qqq
q
qq
q
q
qq
qq
qq
q
q
q
qqqq
q
q
q
q q
qqq
qq
qqqq
qq
qq
q q
qqqq
qqq
q
q
q
qq
qq qqq
qq
q
qq
q
q
q
qq
qq
qq
qq
qqq
qqqqq
qq q
qqq
q
qqq
qq
qq
q
qqq qq
qqqq
q
qqqq
q
q
qqqqq
qq qq
qqq
q
q
q
qq
qq
qqqqq
qq
qq q
q
q
qqqq
qqq
q
qqq
q
q
q
q
qqqqq
q
qqq
qqq
q
qq
qq
q
q
q
qq
qq
q
q
qq
qq
q
q
qqq
qqqqqq
qq
q qq
qqqqqq
qqqqqq
qqqqq
qq
q
q
qqqqq
qqqqq
qq
q
q
qq
qq
q qqq
qq
qqq
qqqqq
q
qqq
qqq
qq
q
qq
q
qq
q
qqq q
q
q qqqqq
q
q
q qq
q
q qq
qq
q
qq
qqq
q
q
q
qq
q
q
qq
qq
qqq qqq
qq q
qqqqqqq
qqq
q
q
qq
q
q
qqqqqqqq
q
qqq
q
qqq
qqq
q
q
q
qqqq
q
q
q
qq
qq
qqq
qq
qq
qq
q qq
q
qq
qq
q
qq
qqqq
q
q
qq
q
q
q
qqqqqqq
qqq
qq qqqq
q
qq qqq
q
qqqq
q
q
qq
qqq
q
q
q
q
q
qqqq
qq
q
qqqqqq q
q
qq
q
q
q
q
qq
qqqqqqqqqqqqq
q
q
q
qq
q
qq
qqqq
qqq
q
qqqq
q
q
q
q
qqqqqq
q
qq
qq
q
q
qqqq qq
qqq
q
qqq q
qqq
qqqq
q
qq
qq
q
qqq qqqqqqqq
qq
qqqqq
qqqq
q
qq
qqq
q
q
qqq
qqqqq
qq
qq qqq
qqqqqqqqqqq q
q
q
qq
q
q
q
q
q qq
qq
qq
qq
q
q
q
q
q
qq
qq
qqqq
q
qq
q
q
q
qq
qqqqqqqqqqqq
q
q
qqqqq
qqq
q
qq
qq
q
q
qq
q
q q
qqq
q
qqq
qqq
qqqq
q qq
qq
qqqq
qqqq
q
qq
qqqqq
qqqq
qqqq
qqq
q
q
qq
q
qq
qq
qqqqqq
q
qqqqq
qqqqqq
q
q
qq qqq
qqqq
q
q
qqqqqq
qq
qq
qqq
qqq
qq qq
qqqq
qqqqqqqq
qqq
q qq
qq qqqq
qq
q
qqq
qq
qqqq
qq
qqqqq qqqq
qqqqq
qqq
q
q
qq
q
q
q
q
qq
q
q
qq
qqqq
qq
qqqq
q
qqqqq
q
q
qqq
q
qq
q qq
q
qqq
qq
q
qqqq
q
q
qq
q
q
q
q
qq
qqq
qqq
qq
qq
qqq
q
qq q
qq
q
qqqqqqq
qq
q
q
qq
q
q
qq
q
qq
qqqqq
qqqq
qqqqqqq
qq
qqq
q
qqqqq
q
qqqqqq
qqq
qq
qq
q
qqq q
q
q
q
q
qq
q
q
qq
q
q
qq
q
q
qqqqq
qq
qq
qqq
qqq q
qqqq
q
q
qqq
qqqq
qqqq
q
q
q
q
q
qq
qqq qqq
q
q
qqqqqqq
q
qq
qq
q
qq
qqqq
q
q
q
q
qq
q
qqq
qqq
q
qqqq
q
qqqqq
q
q
qqqqq
qqqq
qq
qqqqqqq
qq
qqqq
qqqqq
q
q
qq
qqq
q
q
qqqqq
q
qq
q
q
q
q
qqq
q
q
q
q
qq
qqqq
q
q
q
q
qqqqqq
q
q
qqqq
qq
qq
qq
qqq
qqqqqq
qqq
qq
q qq
qq
q
qq
qq
qqq
q
qqqqqqq
qqq
q
q
qqq
q
q
q
q
q
q
q q
qqqqq
qq
qq
qq
qq
qqq
q
qq
q
q
qq
q
qqqqqqqq
qqq
qq
qqq
qqqqq
qqq
q
qq
q
qqq
qq
qq
qq
q
q
q
q
qq qqq qqq
qqq
qqq
qqqqqqqqqq q
q
qqqqq
q
q
qqq
q
qq
q
qq
q
q
q q
qq
qqq
qq
q
q
qq
q
q
q
q
qqqq
qq
q
q
qq
q
qq
qq
qq
q
qqq
qqqqqqq
q
qqqqq
q
q
q
q
q
qqq
q
q
q
q
q
qq
q
qq
qqq
q
q
qqq q
q
qqqq q
qq
qq
qqqqq
q
qq
q
q
q
qqq
q
q
q
qqqq
qq q
q
qq
q
q
qq
q
qq
qq
qqqq
qqqqq
qqqq
qqqq
qq
qq
q
q qqq
qqqqqq qqqqq
q
qqqqqq
q
qqq
qq
q
qqqqqqqq
q
qqqq
qq
qq
qq
qqq
q
qqqq
qq
q
qq
q
q q
q
qqqqq q
q
q
qqqqqq
qqq
qqqq
qqq
q
qqq
q
qqqqqqqq
qq
q
qq q
q q
qqq
q qqq
qq
q
qqq
qq
qq q
qqq
qqq
qq q
qq
qqqq q
qqqq
qq qqq
qqqq
q
qqqq
qq
q
qqq
qqqqqqqq
q
qq
q
qqq
q
q
q
qqqq
q
q
q
qq
q
q qqqqqq
qq
qqq
qq
q
q
q
qqqqq
qqqqq
q
qq
q
qq
qq qq
q
qq
qqqqq
qqqqq q
q
q
q
qqqqqq
q
q
q
q
qq
q
qqqq q
qq
q
qq
q
q
qq
q
q
q
q
q
q
q
q
qq
qqq
q
q
qq
q
qq
q
qq
q
q
qq
q
qq
qq
q
q
q
q
qq
q
q
q
q
qqq
q
q
qq
q
qqq
q q
q
qqqq qq
qqqqqq
qqq
qqqq
qq
q
q
qq
qq
qq
qqq
q
q
q
q
qqq
qqqq
q
q
qq
q
qqqq
q
q
q
qq
qqq
q
q q
qq
q
qq q
qqq
q qq
q
q
qq
qqq
q
q
qqqqqq
q
qqqqq
q
q
q
q
q
q
qqqq
q
qqqq
q
q
qq qq
qqqqq
q
qqq
q
qqqqq
qq
qq
qq
qqqqq
q
q
qq
q
qq
q
q
qq qq
q
qqq
q
qqqqq
qqq
qqq
qq
qq
qq
q
q
qqqqqq qq
q
q q
q q
qq
qqq
q
q
qq
qqqq
qqqq
qqq q
qqqqqqq
q
qq
q
q qqq
qqqqq
qqqqqq
q
q
q
q
q
q
q
q
qq
q
qqq
q
q
q
q
qq
qq
q
qq
q
qqqqq
qq
qqqqqqq
qq
q
q
q
q
qqq qqq
q
qqq
qq
q
q
q
qq
qqqqq
qqq
q
qq q
qq
q
q qqq
qq
qq
qqq
qq
q q
qq
qqqqqqq
qq
qq
q
q
q
qq
q
qqq
q
q
q
q
q
qqq
qq
qq q
qq
q
q
qqq
qq
q
qq
qq
qqqq
qqqqq
q
q
qq
qqqqqqq
qq
q
q
q
q
qqqq
q
qqqqqqqq
qq
q
qqqq
q q
qq
q
q
qqqqq
qqqq
q
q
q
q
q
qq
q
q
q
qqqqq
qq
q
q
qqqqqqqq
qq
q
qq qq
qqqqqqq
qqq
qq
q
qqqq
qqqq
q
qq
qqq
qq
q q
q
qq
qq
q
qq
qq
q
qqq
qq q qqqq
qq q
qq
qq q
q
q
qqqqq
q
qqq qqqq
qqq
qq
qqqq
q
q
q
q
q
q
q
qq
q
qqq
qqq
q
q
qq
q
q
qq
qqqqqq
qq
q
q
q
qq
qqqq
qq
q
q
q
qqqqq
q
qqq
qq
qqqq
qq
qq
qq
q
qqq
qqq
q
q
qqqqq
q
q
qqqqqqqqqq
qq
q
qqq
q
qq
qq
q
qqqqqq q
qq
qq
q
q
q
qq
q
q
q
q
q
q
qq
q
q q
q
q
q
qqq
qq
qqq
q
qqq
qqqq
qq
q
q
q
q
q
q
qq
q
q qq
q
q
qqqq
qqq
q qq
q
q
qq
qq
qq
qqqq
q
q
q
qq
qqq
qqqqqqq
q
q
qq
q
q
q
q
q
qqq
qqqqq
q
qq
qqqq
q
q
qq
qqqq
qqqqq
q
q
q
qq q
q
qqqqq
q
q
qqqq
q
q
q
qq
qqqqqq
qqqqqq
qqq
qqqqq
q
qqqq
q
q
qq
qq
q
q
qqq
qq
q
q
qq
q
qqqq
qqq
qqqq
q qq
qq
qqqq
qqq
q
qqq
q
q
qqq
qq
qq
q
qqq
qqq
qqqqqqqq
qq
qq
qq
qq
q
q
q
qq
qq qq
q
q
q
qq
qqqq
q
q
q
qqq
qq
q
qqqq
qq
qq
qq
q
qq
qq
qqq
qq
q
q
q
q
q
q
qqqqqq
qqq
qq
q
qqqq
q
q
qqqqqq
q
q
q
qqqq
q q
qq
q
qqqqqqqq
qqqqqq
q
q
qq
qqqqqqqqq
q
q
q
q
q
q
q
qq q
qq
qqqq
qq q
qq
qq
q
qq
q
qqqqq
q
qq
q
qqqq
q
q
qq
qq
q
q qq
qqqqqq
qqqq
qq
qqq
q
qqq
qq
q
qq
q
qq
q
q
qqqqq
qqqqqqq
q
qq
qqqq
q
qqq
qqqq
q
q
qq
qqq
qqqq
q
q
qq
q
q
qqq
qqqqq
q
qq
qq
qq
q
q
qq
q
q
q qqq
qq
qq qq
qqqqq
qqq
q
qq
q
q
qqq
qq
q
q qq
q
qqq
q
qq
q
q
q
qqqqqq
q q
q q
qqq
q
qqqq
qq
q qq
q
q qq
qq
q
qqqqq
qq
qq
q
q
qq
q
q
q
qq
q
qq
qqq
qqqqqqq
qq
q
q
q
qq q
q
qqqq
qq
qqq
q
qqqq
qqqq
qqq
q
qqq
qqqq
qqqqqqq
qqq
qq
q
q
q
qqq
qq
qqq
q
q
qq qqq
q
qqq
q
qq
qq
qqq
qqqq
q
qq
qq
qq
q
qqqq
q
q q
q
qq
q
q qq
qq
qqqqqqq
qqqq
qq
qq
q
qq
q
q
q q
qqqqqq
q
q
qqq
qq
qqq
qq
qq
q
q
q
qq
qqqqqqqqqqq
q
qqqq
q
qq
q
q
qq
q
qq
q q
q
qqqq
q
q
qq
qqq qq
q
q
q
q
q
q
q
q
qq
q
qqqqq
q
q
qqqqq
qqqq
q
q
qqqq
q
q
q
qqqqqq
q
q
q
qqq
q
q
q
q q
qq
qqqq
q
qqq
q
q
q
q
qqqqqqq
q
q
qq
q
q
qq
q
q
qq
q
q
q
q q
q
q q
q
qq
q
q
qqqqq
q
qq
q
q qq
qqq
q
qqqqqq
q
qq
qq
qq
q
q
qqq q
q
qq q
q
q
qqq
qqqq
q
q
qq
q
q
qqqq
q
q
qqq
qq
q
qq
q
qqq
q
qqq
qqqq
qqq
qq
q
q
q qqq
qq
q
qqq
qq
qqq
q
qqqq
qq
qq
qq
qq
q
qq
q q
qqq q
qq
qq
q
qqq
q
q
qqqqq
qqq
qq
q
q
q
qq
qqqq
qq
q q
q
q
qqq
qq
q
qqqqqq
q q
qqq
q
q
qq
q
q
qqq
q
q
qqq
qq
q
qqq
qqq
q
qq
q
q
qq
q
qqq
qqqq
qq qq
q
q
qqq qq
qqqqq qqq
q
qq
q
q
q
q
q
q
qqq
q
qq
qq
qq
qqqq
q
qq
qqqq
qqqqq
q
qqq
qqq
qqqqq
qqq
q
q
qq
q
q
qq
q qqqqqq
qqq qqqqq
qqqq
qq
q
q
qqqq qqqq
qqqq
qqqq
qq
qqqqqqq
q q
qqq
q
q
qq
q
q
qqq
q
q
q
q
qq
q
q
q
qq
q
q qqq q
q
qq q
qqqqq
qqqq
qq
qqqqqqq
qqqq
qqq
qq q
q q
q
q
q
qqq q
qqq
q
qq
qq
qq
qq
q
q
q
q
qq
q
qq
q
qq
qqq
q qq
qq
q
qq
qq
q
qqqqqqq
qqq
qq
q qqqqqqq
qq
qqq
qqqq
qq
qqq
q
q
q
qqq
qqqqq
q
q
q
qq
qq
qqqq
q
q
qq
qq
q
qq
q
q
qq
qqqq
q
q
q
qqqqq
q
qq
q
q
qqqq
qqq
qq
qq
qq
q
q
q
q
qq qq
qqq
q qq
qq
qqqqqqqqqqqq
qqq
q
qq
qq
qqqq
q
q
q
q
qq
qq
q
q
q
q qq
qq
q
qqqqqqq
q
q
q
qqqqq
q
q
qqqq
qqqqqq
q
q
q
q
q
qqq
q qq
qq
q
qq
q
q
q
q
qqqq qq
q
qq
qqqq qqq
qq
qq
q
qqqqq
q
qq
q
qq
qq
q
qqq
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
qqqq
qqqqqq
q
q
q
q qqqq
qq
q
q
q
q
qq
qqqq
qqqq qqqq
q
q
q
qq
qq
qq
q
qqqq
qqq
q
qq
q
qqqqq
qq
q
qqq
q
qq
q
qqq
q
q
qqqqq
qq
qq
qq
q
q
q
q
qqq
qq
q
q
q
qqq
qq
qq
q
qqq
qqqq
qq
qqqqq
qq
q
q
q
qq
qq
qqq
qqqq
q
qqqq
qq
qq
q
qq
qqq
qq
q
q
qq
q
qqqq
q
q
q
qqq
q
qqqqqq
q
qqqqq
q
qq
qqqq
q
q
qqqqqqq q
q
qqq
q
qq qq
qq
qqq
q
qq
q
qq
q
qqq
qqq
qq
qqqq
qqq
qqq
q
q
q
q qqq
q
q
q
qq
qq
q
qq q
qq
qq
qqqq
q
q
qqqq
q q
q
qqq
q
q
qq
qqqq
qq
q
qqq
q
q
qq qqq
qqq
qqqqqqqqqqq
qqq q qq
qqqqq
qqqq
q
q
q
qq
qqq
q
qq
qqqq
q
qqqqq
qq
qq
q
q
qq
q
q
qq
qqq
qq
q
q
q
qqq
qqqqqqqqqqq
qq
qq
q q
q
qqqqq
q
qq
q
q
q
qqq
qq
q
qqq
qq
qq
q
q
q
q
q
q
qq
qq
qqqqq
qqq
qqqqq
qq
qqqq
qqqq
q
qqqq
q
q
qq
q
q
qq
qq
qqq
q
qq
qqq
q
qqqqqq
q
q
qq
qq
qqq
qqq
qqq
q
qqqqqqq
qq
q
qq
qq
q
qq
qq qqq qq
q
q q
qqqq
qqqq
q
qq
qq
q
qq
qq
q
qq
q
q
qq
q
q
qq
q
q
qqqq
qqqqq
qqqq
qqq q
qqqqq
qqq
q
q
q
qqq
q
qq
qqqqq
q
qqqqq
qqq
qq qq
qq
q
q
qq
q
qqq
q
q
qqqqq
qqq
qqq
q
qq
q
qq
q
qqqqqqqq
qqq
qqqq
q
q
qqqq
qq
q
q
qqqq
q
qq
qq
q
qqq
q
q
qq
qqq
q
qq
q qqq
qqq
qqq
qqqqq
qqq
q
q qq
qqqqqqqqq
qqqq
q
qq
qq
q
qqq
qq qq
qq
q
q
q
q
qq
qq
q
qq
qq
qqq
qqqqq
qqqqqq
qq
q
qq
qqq
q
qqqq
q
qqq
q
q
q
q
q
q
qqqqq q
q q
q q
qq
q q
qqq
qq
qqqqq
q
qqq
q
qqqqqqqqqqqq
qqq
q
qqqq
qqq q
qq
qq
q
qqq
qqq q
qqq
qq
2 0 2 4 6
0
2
4
6
8
tuning=1
Initial value=(4,5)
Acceptance rate=48.5%
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
2 0 2 4 6
0
2
4
6
8
tuning=100
Initial value=(4,5)
Acceptance rate=2.4%
q
q qq qq qqq
q
q
q
q qq
qq q
q q q
qqq
q
q qqqqqq
q q
qq
q qq
q
qq
q qq q qq
q qqq qqq
qq
q
q
q
q
q qq
q
q qq
qqqq q
q qq qq qq
qqq
q
qq q q qqq q q q q q
q
q qq
q
qqqq qqqq q
q
q
q q
q q
q q
q
q
qq q
q q
qq
qqqq
q
qqq
q q
q
qqq q
qq
q q q q q
qq q qq q
q qq
q q qq
q
q qq
qq
qq
q qqqqqq
qq
q
q q
q
q qq
q qq
qq
q
qqq q
qq
qq
q q q qq
q q
q
q q q qq q q
qq
q
q
q qq qq
q q q
q qq
q
q q q
q q
q
q
qqq qq
qq q
qq
qq qqq qq qq
q
q
q
qq q
q
q
q
q
q
q
qqqqq q
q
qqq
q qqq
qq qqq q
q
q qqqqq q
q
q q q
q q q q q
q
q
q
q
qqq q q q
qqq
q
qq
qq
q
q q
qq
q q qq qqq
q
qq q q q q
q q
q q qqq q q
q
q
q qq
qq qq q q q
q q qq q qqqqq q
q
q qqqq qq
q q
qq
q q
q
qq
q
q qq q q q q
q
q
qq
q
q
qq
q q q q
qq
q q
q
q
q
q
qq
q
qq
q
qq
q q
q
qq
q q q
q q qq
q qq
q q q
q qqq q
qqq
q
q q q
q
q q qq
q
q
q qq
qq
q qqq
qq
qq q
qq qq
q q q
q qqqqqq
q
qq q
q
q
q q q
qq q q q qq q
q
qq q
qqq
qq q qq q
qq q q q q q qq qq q q q qqq q q
q q
q
q qqq q
qq q
q
q q qqq
q q
q
q
q
qq
q qqq q qq
q q q q
q
qq q
q qqqq qq
q
q
q q qqq q
qq
q
qq q qqqq qq
q
q qqq q
q q
q q q q
qqq
qq
q
q
qqq qq
q qqq q q q
q
q
q q q
q q q
q
qq q
q q q q
qqqqqq
q q qq
q
qq q
q qq q
q q
q q q
q
qqq q q qq qqq
q
qq
q
q qq
q
q q
qqq qqq
q
q
q
qq
q
qq q q qq
qq q
q q q q q
qq q qq q q
q qq
qqq q q qq
q
qq
q q
q
q
q
q q
q
q qqq
q
q
q q
q qq qq
q q q
q
q qqq q
q q
q q
q q
q
qq q q q
qq q qqq
q
q q
q
q
q qq
q qq qqq q q
qq
q q
q q q qq
q
qq q
qq qqq q
q q q qq q
q q q
q qqq
q q q
q q
q q q qq q q q
q qqqq q
q q
q
q
q q q qq
q
qq
q qqq qq qq qqq
q
qq q
q
q qqq
q
qq
q q q q q
q q q q q q qq
qqq q
q
q q
q
q
q q
q
q
q q
qq q q q
q q
qqq q
q qq
qq q
q q
qq q qqqqqq
qq q
q qq
qq
qq q
q
q q
q
qq q
qq q
q
q
qq q
q q
q qq
q
qq
qq q
q
q
q
q q q q q
qq qq
q
q q
q
q
qq q qq qq
q
qq q
q q
q
qq
q qqq q
q
q qq
q
q
qq q
q q
q q
qqq q qq
qq
q q
q qqqq q q q q q qq q q
qq
q
qq q q
q
qq qq
q
q
q
qq
qq
q q q
qq q
q qq
q
q
q q qqqq
q
q
qq qqq
q
q
qq
qq
q q q q
q
qq
q q
q qq
qq
q
q
q q
qq q qq
q q
q
q
q
q
qq
qqq
q q
q qq
q
q
q qq q q
q
q qq
q
qqqqq qqqq q
q qq
qq
q
q q qq
q q
qq
q
q
q
q
qq q
qqq
q q q
q q q
q
qq q
q qqq q
q q qq
q q
q
q q
q
q
qqqqq
q
q
q q q
qqq qq
q
q
q q
qqq qq q
q
q q
q q qqq
q
q
qq q q
q qq
q
q
q q q
qq q q q
q q
q qqq q
q
qqqqq qq q
q
q q q
q q q
q
q
q q q
q
q
q
q
q
q
q qq q
qq
q
q
q q
qq
q q
q
q qq
qq
qq q
q
qq
qqq
q
q qq qq q
q q
q q q
q
q q
q
q
q
q
q
q
qq qq
q q q q
qq q q q q
qq q q qq
qq qqq
qq
qq q
q q
q
q qq
q qq qq
q
qq q q
qq
q q q qq
q
qqq
qqqq
q q
q
q
q
q
q q qq
q q
q
qq
q
q qq
qq
q
q qqq
q q
q
q q q
q q
q
q q q q q
q
q qq q
q q
q
qqq
q
q q
q q
q q q
q
q q q
qqqq q q q
q q
q q q
q q q
qq
qq
q q
qq
q
q q
q
q q q
q
qq q q q q q
q
qq
q qq
qq qq
q q
q q
qq
q
qq
q
q
q
q
qqqq
q q
q
q qq
q q q
qq
qqqq
q q
qq q
q
q
q
qq
q q
q qqq
q
q
q
q
q
q
q q q q
q
q
qq q
q q qqq
q
q
q
q qqq
qqq q
q q
qq q
q
q q
qq q
q qq
qq
q
q q
q
q
qqq
q
q
qq
qq
qq
qq
q
q
q q
q q
q qqq
q qq q q
q
q
q
qqq
qq q qq
q
q
q
q
q q qq
qq
qqq q q q
q
q q
q qq
q
q qq
q q q
q q q q
q
q
q
q
q q q qq
q q qqq q
qq qq q q q
q
q
q
q q q
q
qq
qq q q
q qq
qq
qq
q
q
q
qq
q q
q
q q
q
q qqq
q
q
q q
q q
q
q
q
q q q q q
q
q q
q
q
qq q q
qq
q
q
qq q
q q
q
q
q q q
q q q
q
q
q qq
q q q q
q
qq
q q q q q
q
q q
q
qq
q q
q
qq q q
qq
q
q
q
q
q q q q q
q
q
q
q q
q qq
q
qq qqqq
q
qq qq
q q q
q
q
q
q
q q
q qqqqq q qq
q q q
q q qq q
q q
q
qq
q
q q q qq
q qq q
qq q q q q
qqq q q
q
q
q qq
q q qq
qq
q
qq qq qq
q q q
q q q
qq qq qqqq
q
qqq q q q
q
qq q q
q qq q
q q
q qq
qq q
qq q
q
qq
q
q q
q qq qqq
q q q
q qqq
q qq
q
q
q
q
q q q q q
qq q
qq
q qq q
q
q q q
q
q
qq
q q
qqqq q
q q q
q q
q
q
q qq
q q
q q q q q
q qq q q qq qq
qqq q q q q qq
q
qq q
qq
qq
q q qq qq
q
q
q
q
qq
q q q
q q
q q
q q qq
q
q
q qq qq
qq
q q
qqq
q q
q q qqq
q qq
q q q
q
q q q
qqqq qq
qqq
q q
qq
q q q
qq q
q
q
qqqqqq
q qq
qqq
q q q
q q qq q q
qq q
q
q
q
q qqq q
q qq q q
q
q
qq q
q
q
q q q
q
q q
q q
q q qq q
q
q q
qqq q q
q
q q qq q q
q
q q q
q q
q
q
q q qq
q q q qq
q
q
q
qqq
qq qqqq
q
q q qqq qq
q
q
q
q q q q
q
q
qq qq
q
q q q q
q q
q q q
q
q q qq
q
q q q
q
qq q q qqq q
q
q q qqq
q q qq
qq
qq q
q q
q
qqq
qq
q q
q q q
qqq
qq
q
q q q
q q qq
q q
q
qqqq
q
q
q q q
q
q q qq q q q
q q q
q
q
qq
q q q
q q
q
qq
qqqq q
q q q
qq q q qq qq q q q
q
qq
qq
q qq q q
qq
q q q
q qqq q q
q q
q
qq q
q q q q
q
qq q
qq qq
q
q q
q
q q q
qqq q q qqqq q
q
qq q q
q
q q
q
q qq q
q qq
qqqq
qqq q
q q
qq
q qqqqq
qq
q q
q
q
q
q q
q q
q
q
q q qq
q
q q q
q
q
q qqq
q
qq
q
qqq q qqq
q q
q q qq
q q
q q
q
q qq q
q qq q
q q q q q
qq
q q
qq
qq
q
qq
qq q
q q q qq qq q q
q
q
q qq
q
q
q
q
q
q q qq
q q
qq
q
q q q
q q q
q q
q
q qq q
qqqq
q
q qq qqqq q q q
q qq
q
q q qq q
q
q
q
q
q
qq
q
qqq q
q
qqq q q q q
qqq
qq q
qq
qqq
q
q
q
q qq
q
q
q
q q
q
q q q
q
q
qq
q
q
q q
qq
qq qq q q q
q q
q q
qq q
q qq q
q
qqqq qq q q
q
q q q
qq
q q qq q
q
q q q q q
qqq
q
qq
q
q q q qq
q
q qq
qq q qq q
q
qqq qq
q
q
qq q
qq
qq qq
q q qq
q q
q q
q qq
q
q
qq
q
q qq qqq q qq
q
qq
q
qq
q q
qqq
q
q qqq qq q q q q
q q
q qq
qq
qq
qqq qqq q q
q q q
q q
q q qq q qq q qqq qqq
q
q q qq q
q
q
q q
qq q q
q q
q
q q q q q qq q qq
q qq q q q q
qqq q
qq qq
q
q q q
q
qqq
qqq qq
q
qq q
qqq q q
q q
q q q
qqqqq
q q
qq q
q
q
q q q q
qq q
q
q qq
q
qq
q
q q q
q q
q q
q
q q qq q qqq
q
qq
qqqq q q
q q
q
q
q q q
q q q q
q
qq
q qq q q
q q
q
q q q
q q
qqq q
q
q
q
q q
qq qq q q qq q qq
q q q q
q qq
q
q
q
q
q qq q qqq qqqq
qq
q
q q q
qq
qq
q q qqq q q q
q
q
q
qq qq q
q qq
qqq
q q
q
q
qq q
q qq
q
qq qq q
q q
q q
q q
q
q
q
q q qqq q q q
q q
qq
q q qq
q q
q
q
q q
q
q
q q qq q
q
qqqq
q
qq q qq
q q qq
qq
qqq q
q qq q q
q
q q
q q
q q q q q qq
q
q
q q
qq
q qq
q
q q
q
q
q q q q q
q
q
q
q
q qq q
qqqq
q q qq q
q
q q qqqq qq
q
q q
q q
q q q qq q q
q
q q qq q
qq q qq
q
qq q q
q q
q
q
qqq q qq q
q
qq q q
q
q
q q q
qq q
qqq qq q q qq q
qq q
qq
q
q q q q
qq
q
q
q
qq q
qq q
qqq
q
q q qq qq
q qq
q qq q
q
q q
q
q q q
q qq
q
qq q
q q q qq
qq qq
q q
q
q
qq
q
q
q q qqqq q q
q
q
q
q
q
q
q
q q q
q
qq
q qqqqqq q
qqq
qqq q
q q
qqq q
q q
q
q
q
q q qqq
q q
q q q
q
q qqqq qq
qq
qqqqq q
q
q qq q
q
q q
qq q
q
qq q
q
q
qq q
qqq
qqqq
q
q
q qq
q q
q q q
q qqq q q q q q
qq
q qq
q
qq
q qqq qq qq
qq
q
qq
q q qq
qq q q
q
q
q
qq
qq q q
q
q
q q
q
q
q
qq q
q q
q q
qq qqq
qq
q
q
qq q
q q
q q
qq
qqq q
q
q
q
q q
q
q q
q
q
q
q
qq q q
q q
q q
q q
q
q q
q
q
q
q q q q
q q
q qq
q
qq q q q q q
q
qq
qqqq q
q
q q
q q q
qq q
q q q q
q
q
q q
q
q q q q q
q
q
qq
q q qq
q
q
qq q q
q
q
q q
q
qqq q
q q
q q qq
q
q
q
q
q
q
qq q
q
qq
q q
q
q
q q
qq
qq q
q q qq
qqq q qq q q q q qq
qqq q q q qq
q
qq q
q
qq
q qqqqq
q q
q
q
q q qqqqqq q
q q q qqq q
q q q q
q q q
q
q
q
q q q
q q
q
q q
qq
q qq
q
q q
qq q
q q
q
q
q
q q
q qq
qq q qq
q q
q qq q q
q
q q qq q q
q q q
q
q
q
q q q qq q q
q q q
q qqq qq
q q
q qqq q q qq q
q
q
q q q q q
qqqq q
qq
q
q
qqq
q
q q qq q
qqq q
qqq q qq
q q q
q
qqqqqq
q
q
qq q q q
qq
q
q
qq
qqq
q
qq q
q qq
q q
qq qq
q qq
qq
q
qq q
qq
q
q q
qq
q
q
q
q
q
q q q
q
qqq
q q
q
q
q
q
q q
qq q q q q q qqq
q
q q
q q
q
qq q q
q q
q q
qq
qqq
q
q q
q q
q
q q
qq
q q q
q
q q q qq
q qq
q qq
q
q
qq
qq
q
q q
qq
q q q q
q
q q
q qq q
qq
q
qq q
q q qq q q q q
q qqqq
qqq q
q q
qqq
q
q
q
q q q
q qqq
q qq
q qqqq
q
q
qq
qq
q qq qq
qqq qq
q
qq q q q qq
qq
q
q
q q q qq
q
q q
q qqq qq
q
qq q
q qq
q
q
q
q q
q
q q
qq q
q q
qq
q q
q q
q
q q
q qqq q q
q
q qq q
q q
q
q q q q q q
q q q q
q
q
q qq q q
qq
qq
qqq
q q
q qq
q q qqq q
q
q
q q
qq q q q
qq
q q q
q
qqq
q
q q qq
q q q q qq q
q
qq qq q q q
qq
q qqq
q
q
q qq
q
q
q
q q qq
qq
q q
q
q
qq
q
q
q q
q q
q
q
q
q
q
q qqqq q qqqqq
qq q qq q
qq q
q q qq
q qq q
qqq q
qqq q
q
q qqq
qq qqqqqq
qq q q
q q
q q
qqq q q
q qq
q
q
q
q
q q
q
q q q q
q
q
q
q q
q qq
qq qqq
qq
q q
q
q qq q
q
q
qq
q q q
q qqq q
qq qq
q
q q q
q
q
q
qq q q
q qqq qqq
q q
q
q
q
q
q qq q q q
qqq
q q
q qqq
q
q qq
q
q q q
q
qq
q qqqq
qq
q q
q q
q q
qq qqq
q qq q
q
q
q
q
qq
qqqq q
q qq
q q q q
q
q
q q
q
qq qq qqq q q q
q qq
q qqq q qq qq
q q
q qq q
q
qq qqq
q q q qq
q
q
q q q
qq
q
q
q
q q qqq q
q
q q
q q q
q
q qq
q q
q
qq qq
qq
q q
qq
q q qq
q q q q q
q
q
q q qq q qq
q qq qq q
qq
q qq qqq q
qq q
qq
q q
q qqqq
qq qq
q q
q
q q
q
q
qqq q q qq
q q
q q
qq
q
q q q q
q q q
q
q
q q q
q
qq
q
qqq q
q
q
q qq
q q q
q
q q
q
q
q
q q q q q
q
q
q qqqq
q
qqq
q
q
q
q
qq
qq q
q q
q q
q
qqq
qq
qq
q qq
q q
q q
q q q qq
q q qq
qq
q q q q
qqq q
qq q
q q
q
qq
q q qq q
q
q qqq q
q
q
q q q
q
qqq q
q
q q q
q
qq
qqq q q q
q
q
q q qqqqqq q
q
q
q q q q
q q qq q q qq
q
qq
qqqq qqq q
q q
q q q
q qq q
q q
q
q
q qqqq q qqq q q
q
q
q
q q
q qq q
q
q q
q q
q q
q
q q
q q q
q q qqq
q
q
q
q
q q
q
q qq q q
qq
q
qqq
qqq
q qq q
qq
q
qq
q qq q
q
q qq q
q
q q
q q qq
qq q qqqqq
qq
q
q
q
q qq
q q
qq qq qq q q
q
q q q q q
qq
qqq
q
q q
qq q q qq
q
q qq
q qqq
qq q
q
q
qq q
q q
q q
qqq
q q qqqq qqq
qq
q
q qq q q q
q
q q qq
qq qq q q
q q
qq
q q
q q q
qq q q
qqq q q
q
q qqq q q
qqq
q
q qq q q q
qq q
q
q qqq q
q
2 0 2 4 6
0
2
4
6
8
tuning=0.01
Initial value=(0,7)
Acceptance rate=93.3%
q
qq qqqqq
q
q
q
q
q
q
qq
qqq
q qqqqqqq
qq
q
q
qq
qqqqqqqqq
q
q
qq
qq
qq
q
q
qqq
qq q
q
q
qq
qq
q
qq q
q
qq
q
q
q
qqqq q
q
qq
qqq
qqqqq
q qq
q
q
q
qq
qqqqq
qqq
q
qq
q
qq
q
q
q
q
q
qq q
qq
qqq
q
q
qq
qqq
qqqq
qq
q
q
qq
q
q
q
qqqq
q
q qqqq
qqqq
q
qqqq q
qqqqqq
qqq
q qqq
qqq
q q
q
qqq
qqqqq
qqqq
q
qqq
q
qqq
qq
q
qqq
q
qqqq
q
qqq
q
q
q
qq
q
qqqq
qqq
q
qqq
q
q
qqqqqqqq
q
q
qqq
qqqqqqqq
q q
q
q
qqqqq
qq
q
qq
qqqq qqq
qqq
q
qq
qq
q
qq
qqq
q
q
q qq
q
q
qqq
qqqq
q
q
qqq
qqqq
qqqqqq
q
qq
qq
q
q
q
q
q
qq
qqqqq
q q
qq q
qqq
q
q
qq
q
q
q
qq
qqqqq
q
qq
qq
qqq
q
q qq
qqq
q
q
qqqq
q
qqq
qqq
qq
q
qq
q
q
qqq
qqq
qqq
qq
q qqqqq
qq
q
qq
q
q
q
qqqqq
q
q
q
qq
qq
q
qq qq
qqq
qq
q
q
q
q
qq
q q
qq
qqq
qqqq
qqqqqqqqqq
qqq
qqqq
qq
qqq
qqq
q
qq
qqqqq
q
qqqqqq
qq
q q
q
q
qqqq
q
qq
qq
qqq
qqqq
qq
q
q
q
q
qqq
qq
qq
q
q
q
q
q
q
qq qq
qqq
qqq
qqq
qq
qqqq
qqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qq
qq
q
q
q qq
qqqqqqqq
q
q
qq
q
qq
q
q
q
q
q
qq
qqqq
qqqqqqqq
q
qqq
q
qq
qqqqq
qqqq
qq
q
q
q
q
q
q
qq
q
q
q q
qqqq
qqq
qqq
q
qqqqq
q
q
qq
q
q
qqq
qq
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qqqqqq
qq
q
q
q
qq
q
q
q
qqqq
q
qq q
q
qq
q
q
qq
q
q
qq
qq
qqqqq
q
qq
qqqq
qqqqqq
qqqqq
qqq
q qq
q
q
q qqqq
q
q
q
qqq q
qqqq q
q
qqq
q
qq
qq
qqqq
qq
qqqq
qqq
q
qq
q
qqq qq
qq
q
q
qqqqqqq
qqqq
q
q
qq q
q
qq
qqqq
q
q
q
qqqq
q
q
q
q qq
qq
qq
q
qq
qqqq
qqq
q
q
q
qqqqqq
q
q
qqq
qq
q
qqqq
qqqqq
qq
qq
q
q
q
qqq
q
qq
q
qqqqqq
q
q
qqqqq
q
q
qqqq
q
q
qqqqq
qqq
q
q
q
qqq
qq
qqqq
qq
qq
qq
q
qq
qqq
q
q
q
qqq
qqqqqq
q
q
q
q
q
qq
qqqqqq
q
qqq
q
q
qqq
qqq
qqqqq
qqqqqqqq q
q q
qqqqq
qqqqqqq
qqqqqq
qqqq
qqqqq
q
q
q
q
qq
qq
qq
q
q
q
qq
qqqq
qqqqqq
qq
qqq q
q
q
q
qqqqq
qq
qq
qq q
q
qqqqq
qqq
qqqq
q
qq
q
qq
qqqq
q
q
qq
qq
qq q
qqqq
q
q qq
q
qqqqq
qq
q
q
qqqq
qqqq
q
qq
q qqqqqqqqq
q
q
q
qq
q
qq
qqqq
q
qqq
qq
qq qqqqqqqq
q
qqq
qqqqq
q
q
qq
q
qq
qq
qqq
qq
qq
q
q
q
q
qq
qq
qq
qqq
qq
q
q q
qq
q
qqqq
qqqq
qqqq
q
q
q
q
qqqq
qqqqqq
q qq
q qq
q q
q
q
q
qqqq
q
q
qqqqq q
q
q
q
qqqq
q
q
q
q
qqq
qqqqqqq
qq
q
q
q q
q
qq
qqq
qq
q
q
q
q
qq
q
qqq
q q
qq
qq
qqq
qq
q
qq qqqqqqqqqq
qq
qq
q
q
q
q
qq
q
qqqq q q
qqqqq
q
qqqq
q
q
qq
qq
qq
qqqq
q
qqqq
q
q
q
qqq
q
qq
qqqq qqqqq
q
qqq
qq
q
qqqq
qq qqq
qq
qqq
q
q
qqqqq
q
qqq qq
qqqq
qqqq
q
q
q
qq
q
qq
q
qq
qqq qq
qq
q
qq
q
qqqqqq
q
qq
q
qqqqq
q qq
qq
qqqqqqqqqq
qq qqqqqq
q
qqq
q
qqqq
q
q
qq
qqq
q
q
qq
q
q
qq
q
q
q
q
qqq
q
q
qqqq
qqq
q
q
q
q
q
qqqq
q
qqq qqqq
qq
q q
qq
q
qq
qqq
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
qq
qqq
q
q
qq
qq q
q
qq qqqqqqqqqqqq
q
q
qqq
qq q
qqqqqqqq q
q
qqqq
q
q
qqqq
qq
q
qqqqq
qq
q
qqqqqqq
qq
q
qqqq
qqq
qqqqq
qqqq
qqq
qqqqqq
qq
qq qqqqqq
q
q
q
q
qqqqqqqq
qqqqq
qq
q
q
q
qqq
qqq
q
q
qqqqq q
q
qq
q
qqq
q
qqq
qq
qqq
qq
qqqqqq
qq q
qq
q
qq
q
qq
q
qqqq
q
q
qq
q
qqq
qqq
qqqqqqq
q
qq
qq
q q qq
q
qqq
q
qqq
q
q
q
q q
qqq
q qqqq
q
qq
q
qq
qq
q qq
qqqqqq
q
qqq
q
qq
q
q
q
qq qq
qq
qq
q
qq
qq
q
qqqq
q
q
qq
q
qqqq qq
qq qqq
q
qqq
q
qq
q
qqqqq
q
qqq q
q
q
qq
qq
q q
qqqqqq
qq
qq
qqqq
qqq qq
q
qqqqq
qqqqqq
q
q
qqqqq
qqqq q
q qqqq
qqq
qq
qq
qqqq
qq
qqqqq
qq
q
qqqqqq
q qqq
q
q
qqq
q
qq
qqqqqq
qqqqq
q
q
qqq
q
q
qq
qq
q
qq
q
qq
q
qq
q
qqqqqq
q
qq
q
qqqqq
q
q
qqqq
qqq
qq qq
q
q
q
q
q q
q
qqq
qqqqq
q
qq
q
q
qqq
q
qq
qq
q
q
q
q
q
qqqqq
q q
qqq
qqq
q
q
q
q
q
qq
q
q
qqqqq
q
qqq
qqqq
qq
qq
q
q
qqqqqqqqq
qq
q
q
qqqqqq
qqqqq
q
q
q
qqq
qqq
qqqqq
qqq
qq
qqqq
qq
q
qqq
qqqq
q
q
qqqqq
q
qq
q
qqqqqqqqq
q
q
q
q
q
q
q q
qq
qq
q
q
q
qqqqqq
q
q q
q
q
qq
q
qq
q
qq
qq
q
q
qqq
qqq
qqq
qq
qq
qq
qq
qqqqqq
q
qqq qq
qqq
q
q
q
q
qqq
qq
qqqq
qq
q
qq
qqqqq
qq
q q
q
q
q
q
qq
qqqq
qq
q
qq
q
q
qq
qqq
q
q
qq
qq
qqq
q
q
qq
qq
q
q
q
qq
q
q
qqq
q
q
q
qqqq
qq
q
q
qq
q
qq
qqq
q
q
q
q q
q
qq q
q
qqq
q
q
qq
q
qqqqqqqqq
qqq
qqqqq
q
qqqqq
qq
q
qq
q
q
qqq
q
q
q
q
qq
q
q
q
q
qq qq
qq
q
qq
q
q
q q
qq
qq
qq
qqq
q
q
qqq
q
qqq
qq
q
q
qq
q qq
qq
q
qqqq
q
q
q
qqq
qq
qqq
q
q
qqq
q
qqqqqqq q q
q
qq
q
q
q
qq
qqqq
q qqqqq
q
qq
q
qq
q
qq
q q q
q
qqq
q
qq q
qqqqq
q
q
q
qq
q
q
q
qq
q
q
qqqq
q
q
qq
q
qq
qqq
q
qqq
qq
q
qq
q
q
qqq
q
qq
qqqqq
q
qqq
qq
q
q
q
qqqq
qq
qqq
q
qq
qq
qq
qq
q
qqqqqqq
q
q
qqq
qqq
qqqq
q
qqq q
q
qq
qqqqq
q
qqqqq
q qq
qqqq
q
qqq
q
qq
q
q
qq
qq
q
qq
q
q
qqq
q qq
qqqq
qq
q
q qq
qq
qq
qqq
q
qq
q
qqqqq
qq
q
qq
qq
qq
qqqq
q
qq
q
qqqqqqqq
qqqqq
qqq
q
qq
qqqq
q
qq
q
qq
q
qq
q
qq
qq
q
qq
q
q
q
qq
qqq
q
q
q q
qqq
q
qqq
qqq
qqq
q
qqq
qq
q
q
q
q
q
qq
q
q
q
qqqqq
qq
qq
qqq
qqqq
q
q
qqq
qqq
qqqqq
qqq
q qqqqqq
qqqq q
qqqqq
qqqqq
q
q
q
q
q
qqq
q
q
qqqq
qqqqqqqqqq
q
q
qq
q
q
qq
q
q
q
q
qqq
qqqq q
q
qqqqq q
q
q
qq
q
qq
qqq
qq
q
qq
qqq
qq
q qqq
qq
q
qqq
q
q
q
q
qqqq
qq
qq
q
qqqq
q
qqqq
q
qq
q
q
qqq
qq
q
qq
q
q
q
q
q
q
qqq
q q
q
qqq
qq
qqq
qqqq
q
qqqqqqq
q
qq
qqq q
q
qqq
q
q
qqq
q
qq
q
q
qq
qqqq
q
q
q
q
qqq
q
q
qq
q
q q
q
qq
qqqq
q
q
q
q
qqq
q
q
qqqq
q
qq
qq
q
qqq
qqqq
qqq qq
qq
q
q
q
q qqqq
q
qq
q
q
q
q
q
q
q
q
q
qqq
q q
q
q
qqqqq
q
q
q
q
q
qq
q
qqqq
qqq
qqqq
qq
q
q
q
qq
q
q
q
q
qqq q
qq
qqqq
qqq
qqqqq
qq qqqq
qq
qq
qqqqq
qqqq
qqqq
qqqq
qqqq
q
q
qqq
q
qq
qq
qq
qqq
q
qq
q
qq
qq
qq
q
qq qqq
qq
qq
q qqqq
q
qq
qqq
q
q
qq
qq qq
qq
qq
q q
q
qqqqq
qq
q
qq
q
qq
q
q
q
qqq
q
q
q
q
q qqqq
qqqq
q
q
q
q
q
qq
qqqq
q
q
q
qqqq
q
q
q
qq
q
qqqqqq
q q
qq qqqq
qq qq
qq
q
q
qqqqqqqq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
qq
qqqq
qq
qq
qq
qqqq
q
qq q
q
qq
q
q qq
qq
q
q q
q
qqqq
q q qqq
q
q
qqqq
q
q
q
qq
q
q
qqqqqq
qq
qqq
qqqq
qqq
q
q
q
q
qqqq
q
q
qq
q
qq
qq
q
q
qqqqq
qqq
q
qqq
qqqqq
qqq
q
qq
qq qq
qqqq
q
q
qqq
q
qqqqq
q
qq
q
qqqqqqqqqq
qq
q
q
qqq
qq
qq
q
qq
q
q
q qqqq
q
q
q
q qq
qq
qq
qqqq q
qq
qqqqq
qqq
q
qqqqqq
qqq
q
q
qqqq
qq
qqqqq
q
q
q
q
qq
qq
qq
qq
q
qqqq
q
qq
qqq
qq
q
qqqqq
qq
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqq
q q
q
qq
qqq
qqq
q
q
q
qqqq
qq q
q
q
qqq
qq
qqqqqq
qq
q
q
q
q
q
qqqq
q
q
qqqqqq
q
qq
qq
q
qq
q
qqqq
qqq
qq q
qq
qq
qq
q q
q
qq
q
qqq
q q q
qqqqqqqq
qqqq
q
qqq
q
qq
q
q
q
qq q
q
q
q
q
qqq
qqqqq
qqqqq
qqq
q
qqq
q
qqq
q
qqqq
q
qqqqqq
qq
qqqq
qqqqqqqqqqq
q
q
q
q
q qq
qqq
q
q
qqqqqqqq
q
q
q
qqq
q
q
q
qqqqq
qq
qq
q
q
q
qqq
q
qqq
qqq
q
q
q
qq
q
qq
q qqqq
q
q
q
q
q
q
q
qqq
qq
qqq
q qq
q qqq
q
qqq
qqqqqqqqqqq
qqqqqq
q
qqq
q
qq
qqq
q
qqq
qq
qqqqqqq
q
q
q
q q
qq
q
q
q qqq
qq
qqqqq qqq
q
q
q
qq
qq
qq
qqq
qq
qq
qq
q
q
q
q
q
q
q
q
q
qqqq
qq
q
qq
qq
qq
qq
qq
q
qqqqqqqq qqqqqq
q
qqq q
qqq
qqq
qqqqq qq
q
qq
qq
qqq
qqq
q
qqq
q
q
qqq
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
qq
q
qq
qqqq
qq
q q
q
qqqqq q
qqqqq
qq
q
qqqqq q
qqqq qq
q
qq
qq
q
q
qq
q
q
qq qqq
qq
qq
q
qqq
q
q
qqqqqqq
q
qq qqqqqq q
qqq
qqqq
q
q
qqq
q
q
q
qq
q
q
qq
q
qqq
qqqqqqqqq
qqq
q
qqqq
q
qqqq
q
q
q
q
q
q
q
q
qq
q
qqq
qq
q
qq
qq
qqqq
qq
qqqq
qqqqqqqq
q
qqq
q
qqqqqq
q
q
q
q
q
q
q
qq
q
q
qq
qq
q
q
q
q
qqq
q
q
qqq
qq
q
qqq
qqq
qqq
q
q
qq
qqqqqq
q
q
qq
qq q
qq qqq
qqq
q
q q
q
q
qq qqqq
qq
qqqqq
qq
qq
qqq
q q
qq
qqqq
qqqq
qqq
qq
qqqq
qq
qqqqqqqqq
q
qqqq
q
q
q
qq q
q
q
q
qqq
q
qq
q
q
qq
q
qq
qqq
qq
q
q
qq
q
q
q
qqqq
qqqqqq
qq
qq
qq
qqqq
qqqq
q
qq
qqq
qqqq
q
q
q q
qqqqqq
q
qq qq
q
q
qqq
qq
qqqq
q
qq
qq
qqq
qqqqq
q
qqqq
qq
q
q
qqqqq
qqqq
qq
qqq
q
q
q
q
q
qqqq
qqqqqq
q
qq
qqq
qqqqqq
q
qqq
qq
q q
q
qq
qqqqqq
qqqqq
qqqqqqq
q
q
q
qq
q qqqq qq
qq
q
q
q
q
q
qq
q
qq
q
q
qq
qqqqq
qq
q
q
qq
qqqq q
q
q
qq
qqq
qq
q
q
qq
q
q
q
qqq
qq
q
qqqqqq
q
qq
q
qqq q qqqq
q
q
qq
qq
q q
q
qq
q
q
q
qqqqqq qqqq
q
qqqqqqqqq
q
q
q
q
q
qq qqq q
q
q
q
q
qqq
qqqq
qq
qq
qq
qq
q
qqqqqq
q
qqqqq
q
q
q
q
qq
q
qq
q
qq
q
qq
q qqq
q
q qqq qqqq
qqq
qqq
qq q q
qqq
qq
q
q
q
q
qq
q
q
q
q
q
qqq
qq
q
q
qq
q
qqqqq
q
q
q
qq
qqqqqqq
qq
q
q
q
q
qqq
q qq
q
qq
qqqqqq
q
q
qqqqqqqqqqqqqqqq
qqqqqq
qq
q
q
qqq
q
qqq q
q
qq
q
qq
q
q
q
qqqq
qqqqqqq
q
q
qqq
qq
q
q
qqq
qq
qq
qq
q
qq q
q q
q
qq
qq qq
q
qqq
qqqq
q
qqq
q
q
qq
qqqq
q
qqqq
q
q
q qq
qqq
qqq
qqq
qq
qq
q
qq
qq
q
2 0 2 4 6
0
2
4
6
8
tuning=1
Initial value=(0,7)
Acceptance rate=49.3%
qqq
q
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqqq
qq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqq
qqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqq
2 0 2 4 6
0
2
4
6
8
tuning=100
Initial value=(0,7)
Acceptance rate=2.4%
53 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Autocorrelations
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=0.01
Initial value=(4,5)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=1
Initial value=(4,5)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=100
Initial value=(4,5)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=0.01
Initial value=(0,7)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=1
Initial value=(0,7)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=100
Initial value=(0,7)
54 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Independent Metropolis
q(, ) = f
N
(;
3
, I
2
) and
3
= (3.01, 4.55)

.
q
qq
q
q
q
q
q
q
qq
q q
q
q
q
q
qqq
q
q
q
qqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
q
q
q
q
q
q
q
qq
q
q
qq
q q
qqq
q
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
q
qq
q
q
q
qqq
q
q
qq qq
q
q
q
qq
q
q
qqqqqqqqqqqqqq
q
q
q q
qqq
qqqq
q
q
qqq
q
qqq
q
qqqqq
qq
qq
qqq
qq
qq
q
q
q
q
q
qqq
q
qq
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
qq
q
q
q
qqqq
qq
qq q
qqq
q
q
q
q
qq
qqqq
q
qqq
qq
q
qq
qq
qqqq
qq
q
q
qqqq
q
q
q q
qqqqqq
q
q
q
q
qq
q
q
qqqqqqqqqqqqqqqqqqqqqqq
qqqqq
q
q
q qqq
q qq
qqqqqqq
q
q
qq
qqqq
qqq
q
qq
qq
qq q
qqqq
q
q
q
q
q
qqqq
q
qq
q
qq
q
q
q
qqqqq
q
qq
qq q
qqq
q
qq
q
q
qq
qq q
q
qq
qqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
q
qqqqq
q
q
qqq
q
qq
q
q
q q
qqq
q
q
q
qqqqqqqqqqqqqqqqqq
qqq
q
qqqqq
q
q
qqq
q
qq
q
qqq
q
qqq q q
q
qq
q
q
qq
q
qq
qqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qq
qq
q
q
q
q
q
qqqqqqq
qqqq
q
qqqqq qqqqqq
qq qq
q
qqqq
q qq
q
qq
q
q
q
q
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
q
q
q
qq
q
q
qqqqq
qqq
q
qqqq
q
qq
q
q
q
q
qq
q
q
qqqqqq
q
q
q
qqq
qqq
q
qq
q
qqq
q
q
q
q
q
q
q
q
qq
q qq
q
qq
q
q
q
q q q qq
q
qqq
q q
q
q
q q
q
q
qqq
q
qq
q
q
q
q
qq
qq
q
qq
qq
qq
qq
qq
q
q
q
qqq
q
qq
q
q
qqq q
q
qq
q
qq
q
q
qq
q
qq
qqqqqqqqqq
q
qqqqqq
q
q
q
q
q
q
q
q
q
qq
q
qq
q
qqqq
q
qq
qq
q
q
qq q
qqq
q
q
q
qqq
qqq
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q
q
qqq
q
qqq
q
q q
q
q
qq
q
qqqqqq
q
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
qqq
q
qq
qq
q
q
q
q
q
q
qqqqqq
qq
qq
q
q
qqq
q
qqqqqq
qqqqqqq
q
q
q
q
q
q
q
q
qqqq
qqqqqq
q
q
qqqqqqqqq
q
q
q
qqq
qqqqqqqqqqqq
qqq
qq
q
qqqqqqqqqq
qq
qqq
q
q
q
q
qqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
q
qqq
q
qq
qq
q
qq
q
qq
qq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
q
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
2 0 2 4 6
0
2
4
6
8
tuning=0.5
Initial value=(4,5)
Acceptance rate=9.9%
Acceptance rate=9.9%
qqqqqqqqqqqqq
q
q
q
qqqqqq
qq
qqqqqq
qq
qq
qqqqq
qqqqqqqq
qqq
q
qqqq
q
qqqq
qqqq
q
qqqqqq
qqq
qqq
qqq
q
qq
q
qqqq
qqqq
qq
qq q
q
qq
q
q
qqqq
qq
q
qq
qqqqq
q
qqq
qqqqq
qqqqq
qq
qqqq
qqqq
q
qq
qqqq
qq
qq
qqqqqq
q
q
qq
qqqqqqqq
q qqq
q
qqqqqq
qq
qq
q
qqqqqqq
qq
qq
qqqqqqqqqqq
qqqqqqq q
qqqq
qqqqq
qq
q
qqqqqqqqqqqq
qqqqqqqqq
q
qqqqq
q
qq
qqqqqqqq
qq
q
q
qqqqqqqqqq
q
qqq
qqqq
q
q
qqqqqq
qqq
qqq
q
qqq
qqqqqqqq
q
qq
q
qqqqqqqqqqqq
q
q
qq
q
qq
q
q
qq
qq
q
qqqq
qqqqqqqqq
qqq
qq
qqqq
qqqqqqq
qqq
q
qq
q
q
q
qqqqqqqq
q
qqqq
qqq
qqqq
qqq
qqqqq
qqq
qqqqqqqqqqq
qqqq
qqqqqqqqq
q
qqqqqqq
qqqqqq
qq
qqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqq
qq
qqqq
qqqqqqqqq
q
qqqqqqqqqq
qqqqqqq
qqqq
qq
qqqqqqq
qq
qqqqq
q
qqqqqq
qq
qqqqqqqq
qqqq
q
qqq
q
qq
q
qq
q
q
qqqqqqqq
qqqqqqq
qqqq
qqqqqqqq
qq
qqqqq
qqqqqqqqqq
qq
qqqqqqqqqqqqqq
q
qqqqqq
qqqqqq
qq
qqqqqq qqqqqqqqqq
q
qq
qqq
qqqqqq
q
q
q qq
qqqqq
qqqqqqqqqq
q
qqqqqqqq
q
q
q
qqq
qq
qq
q
qq
qq
qq
qqqqqq
qqqq
q
qqqqqqqqqqqqq
qq
qqqqq
qqqqqqqq
qqq
q
q
qq
q
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqq
qq
qq
qqqqqqqq
qq
qqqqqqqqqqqq
qqq
q
qq
qqqqqq
qqq
qqqqq
qqqqqqqqqqq
qqqqq
qqqq
qqq
qqqqqq
q
qq
qq
qq
q
qqqqqqqqqq
qq
qqqqq
qqq
q
q
qq
qq
q
qqq
q
q
qqqqqq
q
q
qqqq
qq
qqqq
q
qqq
qqqq
qqqqqqqq
q
qq
q
q
qqq
q
qqq
q
qqq q
qq
qqqqqqqqqqqqqqqqqqq
qqqqq
q
qqqqqqqqqqqqqqq
qqqqq
qqqq
q
qq
qq
qqqqq
q
qqq
qqqqqqq
qqqq
q
qqqq
q
qq
qqq
q
q
qqq
qqq
q
qqqq
q
q
qq
q
q
q
qq
q
qqq
q
q
qqqqqqqqqqq
qqqq
q
q
qqq
qq
qqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqq
qq qqq
qqqqqqq
qqqqqqq
q
q
qqq
q
qqq
q
qqq
qq
qqq
q
qqqq
qqqqqq
qqqqqqq
qqqqqqqq
qq
qq
qqqqqqqqq
q
qq
qqqqqqqq
qq
qq
q
qqq
qqqqqqqqqq
qq
qq
q
q
qq
q
q
qq
q
q
qq
q
qqqqqqqqq
qqqqqq
qq
qq q
qqqq
qqqq
qqqqq
qqq
qq
q
qqqqq
q
qq
qqqqqq
qqqq
q
qqqq
q
qqq
q
qqqqqqqqqqqqqqqqqq
qq
q
qq
q
qq
q
qqq
q
qqqqqqqq
q
q
q
q
qq
q
q
q
q
qqqqqq
qqqq
q
qq
qqq
q
qqqq
qq
q
qq
qqqqq q
qqq
q
qq
qqqqq
q
qqqq
qqqq
qqqq
qqqqqqqq
qq
q
q
qqqqqqqq
q
qqqq
q
q
qqqqqqq qqq
q
qqqqqq
q
qqqq
qqq qq
qqq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqq
q
qqqqq
qqqqq
qqqq
q
qq
qq
qqqqqqqqq
q q
qq
q
qq
q
qqqqqq
qqq
qqqqqq
qqqqq
qqq
q
qq
qqq
qqqqqqq
qqqqqqqqq
q
qq
q
q
q
qqqq
qqq
q
qq
qqqqqqqq
q
qqqqqqq
q
qq
qqqq
qqqq
q
qq
qqq
qqqqqqqqqqq
qqqqqqq
qqq
qq
q
qqqqqqq
qqq
qqqqqq
qq
q
q
qqqqqqqqqqqqqqqq
qqqqq
q
qqqqqqq
qqqqq
qq
qqq
qqq
qqqqqqq
q
qqqqqqqqqq
qqqqqqq
qqqq
qqqqqq
q
qq
qq qqq
q
qqqqqqq
q
q
qqqqqq
qq
qqq
qq
qq
qqq
q
qq
q
qq
q
qq
qqqqqqq
qq
qq
q
q
q
qqqq
qqqqqqqqqq qqqqqqqq
qq
qq
qqqqqq
q
q
qq
qqq
qqqqqqq
q
q
qqqqqqqq
qqqq
q
q
qqq
q
q
q
qqqqqqq
q
q
qqqqq
qqqqqqqqqqqqqqqqqq
qq
qqqq
qqq
qqqq
qq
qqq
qqqqq
qq
qq qqqqqqq
q
qqqq
qqqq
q
q
qqq
qq
qqqq
qq
qq
qq
q
qqqqq
qqq
qqq
q
q
qq
q
qqqqqqqqq
q
q
q
qq
q
q
qqqqq
q
q
q
qqqqqq
qq qq
qqq
q
qqq
qqqq
qq
qqqq
qq
q
q
q
qqqqqqq
q
q
q
q
q
qqq q
q
q
qqq
qq
q
q
qq
q
qqqq
qqqqqqqqqqqqqqq
qq
qq
q
qqqq
qqq
qqq
qqqq
qq
q
qqqqqq
qqqqqq
q
qq
q
q
qq
q
q
q
qq
qqq
qq
q
q
qq
qqqq
qqq qq
qqqqq
qq
q
qq
qq
q
qq
q qqqqqq
qqqqq
q
qqq
qq
q
qqqqqq
qq
q
qqq
qq
q
q
qqqqqqqq
qq
q
q
qqqqq
qqq
qq
qqqqq
qq
qqqqqq
q
qq
qq
qqqqq
qq
qqq
q
q
qqq
qq
qq
qqqqqqqq
q
qq
qqqqqq
qqq
qq
qqqqqqqqqqqq
q
qqqqqqq
qqq
qqqqqq
qqq
q
q
qq
qqqqq
qqq
qqq
qq
qqqqqqq
qq
qqqqqq
q
qqqqq
qqqqqqqqqqqqqqqqqqqq
qqq
qqq
qqqqqqq
q
qqqqqqqqqq
qqqqqq
qqq
qqqq
qqqqqqqqqq
qqq
qqq qqq
qq
q
qq
qqq
qq
q
q
qq
qqqqqqqq
qqqqqqq
qqqq
qqq
qqqqqqqq
qqqqqqq
qqqq
qqq
q
qqqq
q
qq
qqq
qqqq
qqqqqqq
q
q
q
qqqqqqq
q
qq
qq
qqqqqqqqqq q
qqq
qqq qqqqqqq
qqqqqq
qqqqq
q
qqq
qq
q
q
qq
qqqqqq
qq
qq
q
q
qqqq
qqqqqqqq
qqqq
qqqq
qqqq
qqqq
qq
qq
q
qq
q
qqq
qq
q
q
qqqq
q
q qqqqq
qqqq
q
qqqq
q
q
q
qqqq
qqq
qqqq
qq
qq
qqqqq
qq
qq
qqqq
q
qqqq
qqqqqqq
q
qq
qqqqqq
qqq qqqqqqqqq
qq
qqqqqq
qqqqqq
q
qq
qq
qq
q
q
qqqq
qq
q
qqqq
qqq
qqq
qqqqqqq
qqqqqqqqqqqqqqq
q
q
qqqq
qqq
qq
qqqq
qq
qq
qqqqq
q
qqqq
q
qq
q
qqqqqqqqq
q
q
qq
qqqqq
qq q
qqq
qqqqqqqqqqqqqqq
qqqqqqqqqq
q
qqq
q
qqqq
qqqqqqqq
qqqqqqqqq
q
qqqqqqqqqqq
qqqqqqqqqqq
qqq
qqqqqqqq
qqqqq
qqq
qq
q
q
q
qqqqq
qqqqqq
qqqqqqqqq
qqqqqq
qq
qqq
qqq
qq
qqq
q
q
qq
qqqq
q
qq
qq
qqqqq
q
qqq
q
qqq
qqqqqq
qq
qq
q
qq
qqqqq
qqqqqqqqqq
qqq
qq
qq
q
qqqqqqqq
q
q
qqqqqqq
q
qqq
qqq
qqq
qq
qq
q
qq
q
q
qq
qqq
qqqqqq
qqqq
qqqqqq
qqqqq
qqqqqqq
qqqqq
q
q
qqqq
qq
qq
q
q
q
qqqq
qqqq
qqqq
qqq
qq
qq
qqq
qq
qqqqqq
qq
qqq
q
q
qqq
q
qqq
qq
qqqq
qq
q
qqqqqqqqqqqqqqqqq
qqqqqq
q
qqqqq
q
q
q
qq
q
qqq
qqqqqqqqqqqqqqqqqqqq
q
q
q
qqqqqqqqq
qqq qq
qqq
qq
qqq
qqq
q
qq qq
qq
qqqqqqqqqqq
qq
qq
q
q
qqqqqqqqqq
q
q
qqqqq
qq
qq
qq
qq
qq
qq
qq
q
qqq
qq
qqq
qq
qq
qqqqqqqq
qq
q
qqqqq
q q q
qqqq
q
qqq
qqqqqq
qqqqq
qqqqqqqqqqqq
qqqq
qqq
qqqqq
qq
qqq
qqqqq
qqqq qqq
q
qqqq
q
qq
qqqqqqqqqqqq
qq
qqqq
q
qq
qqqq
q
q
qqq
q
q
qqq
q
qqqqqqqqq
qq
qqq
qq
qqq
qqqq
q
qqqqq
q
q
qqq
qqqq
q
qqq
q
qqqqqqqqqq
q
qqq
q
qq qqqqq
q
qqqqqqqq
qq
q
qq
qq
qqq
q
qqqqq
qqq
qq
q
q
qqq
qqq
qq
qqq
q
q
qqqqqqqqqq
qqq qq
qqq
q
qqq
qqqq
qq
qqqqq
q
qq
q
q
q
qqqq
q
qqqq
q
qqqqqqqq
qq
qqqq
q
q
q
qq
q
qq
qq
q
qq
q
q
qq
qqq
qq
q
qqq
qq
qqqq
q
q
qqq
qqq
qqqqq
qqqqqq
qqqqqqqq
qqqq qq
qqqq
qqqqqqqqqq
qqqq
qqq
q
q qq
qq
qqqqqq
qq
qq
qq
qqqqqqq
qqqqqqqq
qqqq
q
q
qq
qqqq qqqqqqq
q
q
q
qq
qq
qq
qq
qq
q
qqqqq
qqqq
qq
qq
qqqqqqqq
qqqq
qqqq
qq
qqqqqqqq
qqqqqqqq
qqq
q
qqq
q
q
qqqq
q
qq
qq
qq
qq
qqqq
qqqqqq
qqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqq
q
q
qqqq
qqqq
qq
qq
qq
qq
qqqq
q
q
q
qqqq
qqq
q
qqq qqq
q
q
q
q
qqqq
qqqqq
qqqqq
qqqqqq
q
q
qqqq
qq
q
q
qqqq
qq
qqqqqq
q
qqqqqq
qqqq
q
qqqq
qqq
qqq
qq qq
qqqqqq
qq
qqq
qqqqqqqq
q
q
qqq
q
q
q
q q
qq
q
qqqqqqqq
q
qq
qqq
q
q
qqqq
qq
q
q
qqqqqqqqq
qq
qqqq
qqqqqqq
q
qqqqq
q
q
q
qqqqq
q
qq
q
q
q
qqqqq
qqq
q
q
q
q
qq
qqqq
q
qqqqqq
q
qq
q
qqq
qq
q
q
q
qq qqqqqqqqqq
qqqqq
q
q
q
qqq
qqqqqqqqq
q
qqqq
qq
qqq
qq
q
q
q
q
qq
q q
qqqq
qq
q
qqqqqqqqqqqqqqq
qq qqq
q
q
qq
qq
qqqqqqqqqq
qqqq
qq
qq
qq
qqqq
qqqqqqqqqqqq
qqq
qqqqqq
qqqq
qq
q
q
qqqqqqqqqqqq
qqqqqqqq
qqqqq
qq
q
qqq
q
qqqqqqqqq
q
qq
q
q
q
qqqqqqqqqq
q
qqqq
qq
qqqqq
q
q
qqqqqq
q
qqq
qqqq
q
qqqq
q
q
q
qq
qqqqqqqq
qq
qq
qq
qq
qqqqqqqqq
q
q
q
q
qqqqq
qq
qqqq
qq
qqqqqqqqqqqqqqqqqqqqq q
qq q
qqqqqqqqqqqqqqqq
qqqqqqq
qq
qqq
qq
qqqqqqq
q
q
qqqq
q
q
qqqqq
q
qq
qqqqqqqqqqqq
q
qqqqqqqq
q
q
q
q
qq qqqqqqqqq q
q
qq
q
qqqqqqq
qq
qqq
qq
qqqqqqqq
qqqq
q
qq
q
qqq
qqqq
q
qq
qq
q
qqqq
qqqq
qqq
qqq
q
q
qqqqqq
qqqq
qqqqqqqqq
q
qqqqqqqqqq
q
q
qq
q
qqqq
q
qq
qq
q
q
qqqqq
qqqqqqq
qqq
qqqqqq
qqqqqq
q
qqq
qqqqqqqq
qq
qqqq
qqq
qqqqqqqq q
q
qq
qqqqq
qqqqqq
q
qqqqqqqq
qq
qq
qq
q
qqq
qqqqqqqq
qqqqqq
q
qqqqqqqqq
qqqqqqqq
q
qqqqqq
qq
qqqq
qqq
qqqqqqqqq
qq
qqqq
q
qq
qqqqqqq
q
q
q
qq
q
q
qq
qqqqq
qq
qq
q
qqqqqqqqq
q
q qqqq
qqq qq
q
qqq
qq
qqq
q qqqqqqqqqqq
qqqqq
qqqqq qq
qqqqqqqqqqqq
qqqqqqqqq
q
q
qqq
q
qqqqq
q
q
qqqqq
qq
qqqq
q
qqq
q
q
qqqqq
qqqqqqqqqq
qq qqq
q
qqqq
qqq
qqq
qqqqqqqqq
q
qq
q
q
q
2 0 2 4 6
0
2
4
6
8
tuning=5
Initial value=(4,5)
Acceptance rate=30.9%
Acceptance rate=30.9%
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqq qqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq qqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqq
qqqqqqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqq
qqqq
qqqqq
qqqqqq
q
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qqqqq
qqq
q
qqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq qqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqq
qq
qqqq
q
qqqqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqq
qq
qqqqqqqqqqq
qqq
qqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqq
qqqq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
q
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqq
q
qqq
qqqqqqqqqqq
qqqqqqqqqq
qqqqq
qqqqqqqqqqqqqq
qqqqq
qqqqqqqqq
qqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
q
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq qqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
q
qqqqqqq
qqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
2 0 2 4 6
0
2
4
6
8
tuning=50
Initial value=(4,5)
Acceptance rate=5%
Acceptance rate=5%
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
q
qq
q
qqqqqq
q
q
q
q
qq
q
qqqqqqqqqq
qq
q
q
q
q
q
qqq
qq
qq
qqq
q
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
q
qqqqqqqq
q q
q
q
q
q
q
q
q
qqqq
q q q
qq
qqq
qqq
qq
q
qqq q
q
qqqqqqqq
q
q
q
qq
q
q q
qqqqqqq
qq
q
q
qqq
q
qq q
q
q
q
q
qqqq
q
q
q
q
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
q
qqq
q
q
qqqq
qqqqq qqq
q
qq
q
q
q q
qq
qqq
q
q
qqqqq
qqqqqqqqqq
q
qq
q
qq
q
q q
q
qq
qq
q
q q
q
q
qqqqqqqqqqqqqqqqqqqqqqqq
q
qq
q
q
qq
q
q
q qq
q
q
qq
qq
qqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqq
qq
q
q
q
qq
qqqq
qqqqq
q
qq
q
q
qq
qqqqqq
q
q
q
qqqqqqqq qq
qqq
qq
qq
q
q
qqqq
q q
q
q
q
qq
q
q
q
q
q q
q
qq
q q
q
q
qqqqqqq
q
qqqqq
qq
q
q qqq
q
q
qqqqqqqq
qqq
q
q
qqqqqqqqq
q
qq
q
q
q qqq
qq
qqq
q
q
q
qqq
qqqqqqq
qq
qq
q
qqqqqqqqqqqqqqqqqqqq
qqq
qqqqqqqqqqqq q
q
q
q
q
qqqqqqqqq
q
q
qqqq
q
q
q qq
q
q
qqq
q
q
qq
qqq
q
q
q
q
q
q
q
qq
q
q
q
q
q qq
qqqq
q
q
qq
qq
qq
q
qq
q
qqq
qq
q
q
q
q
qqq
qqqqqqqqqqqqqq
qqqqqqqq
q
qqq
q
q
q qq
q
qq
q
qq
q
q q
q
q
qqqq
qqqq
qq
q
q
q
qqqq
qq
q
qqq
qqq
q
q
q
qq
qq
q
q
qqqq
q
q
q
qqqqq
q
qq
q
q
q
q
q
qqqq
q
q
q
qq
q
qq
qq
qq
q
q
qqqq
qqqqqqqqqq
qqqqqqqq
qqqqq
qqq
qqqqq
qqq
q
qq qqq
qqq
q
q
q qqq
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
qq
qqqqqqq
q
q
q
q
qqqqqq
qq
q
q
q q
qq
qqqqqqq
qqqq
qqqqqqqqq
q
q
q
qq q
q
qq
q
q
qq
q
q
q
q
q
qq qq
qqq
q
q
q
q
q
q
qqqqqqqqqqqqqq
q
qq
qq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
q
q
q
qq
qq
q
qqq
qq
q
q qqq
qqqq
q
q
q
q q
qqq
qqqqqq
q
qqq
q
qqqqq
qqqqq
q
qqqqqq
q
qq
q
q
q
qqqqqqqqqqqqqqq
q
q
q
qq
qq
q
q
q
qq
q
q
q
qq
qq
q
q
q
q
q
q
q
qq
qqqqqqqqq
qq
q
q
q
qqqqqq
q
qq
qqqqq
q
q
qqq
q
qq q q
qq
qq
q
q
qq
qq
q
q
q
q
q
qq
qqqqqq
q
q
q
q
qq
q
qq
qqqqqqqqqqq
qq
q
q
qq
q
q
qq
q q
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq q
qq
qqq
qqqqqqq
q
q
q
qq
qq
qqqqqqqqqqqqqqqqqqqq
qq
q
qq
q
q
q
q
q
q
qqqqqqqqq
qq
qqqq
q
q
q
q
q
q
qq
q
q
qqqq
q
qqq
qqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
q
qq
qqqq
q
qq
q
qq
q qqq
qqq
qq
q
q
q
qqqqqqq
qqqqqqqqq
q
q
qq
q
q
q qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
q
q
q
qq
q
q
q
q
qq
qq
qqqq
q
q qqqqq
qqq
qq
qq
q
qq qqqqq
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
qqqqqqqqqqq
q
q
q q
q
q
q
qqqq
qq
q
q
qq
q
qq
qqqqqqqq
q
qq
qqq
qqqq
q
q
q
q
q qqqq
q
q
qq
qq
qq
q
q
q
qqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
q
q
q
q
qq
qqqq
qq
q qq
qq
q
q
q
qqqqqqqqqqq
qqq
qq q
q
qqq
q
q
qqq
qq
q
qq
qq
q
q
q
q
qq
qqqqq
qqqqq
q
qqq
qq
qqqqqqq
qqqqqqq
q
q
q
q q
q
qqq q
qqqqqqqqqqqq
q
q
q
q
qqqq
qqq q
q
q
q
qq
q
qq q
q
q
q
qqq
qqqqq
q
qqqq
qqq
q
qqq
q
q
q
q
q
q
qqq q
q
q
q
qqqqq
q
q
qq qqqqqqqqq
qqqqqq
q
q
qq
q q
q
q
qqqqqqq
q
q
q
qq
q
q
qqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqq
qqq
q
q
qq
qq
qqq
q
qqq
qqqq
qqqqqqqqqqq
qqqq
q
q
q
q
q q
q
q
qq
q
q
qqqqqqq
qqqqqqqqqqq
qqq
qq
q q
q
qq
qqq
qq q
qqqq
q
qq
q
q
q
q q
qqq
q
q
q
q
qqq
qqqqq
q
qqq
q
q
q
q
qqqq
q
q
q
qqq
q
q
q
q
q
q
q
q
q
qq
qq
q
qq
q
q
q q
qq q q
q
q
q
qq
q
q
q
qqqq
qqqqqqqqqqqqqqqqqqqqqqqq
q
qq
qqqqqqqq
qqqqqqqq
qq q
qq
qq
q
q
q
qq
q
q
qqqqq
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
q
qq
q
q
qq q
q
q
q
q
q
q
q
qqq
qqqqq
qqqqqqqqqqqqq
q
q
q
q
qqq
q
q
q
q
q
q
q q
qq
qqqqq
qq
qqqq
q
q
qq
qqqqq
q
q qq
q
q
q
q
q
qq
qqq
qq
qq
q
qqqqqq
q
q
qqq
qq
q
qqq
q
qq
qq
qqq
q
qq qqqqqqqqqqqqqqqqqqq
qq
q
qqqqqqqqqqqqqq
q
q
qqqqqqqqqqqq
qq
q
q
q
q
qqqqqqqqqqqqqqqqqqqqqq
q
qqq
qqqqqqqqqqq qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqqq
q
qqqqqq
q
q
qqqqq
q
q
q
q
qqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqq
qqqqq
qqq
qq
q
q
q
q qqq q qq
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
qq
q
qqqqqqqqqqqq
q
q
q
qq
q
q
q
q
q
q
q
q
qqqqqqqqqqqqqq
qq q
qqqqqqqqq
q
q
q
q
qq
qqq
q
q
qqq
qq
q
q
q
qqqq
qq
q
q
q
qq
q
qqq
q
q
q
q
q
qq
q
q
qqq
q
qqqqq
qqqq
q
qqqqqqq
qq
q
q
q
q
qq
q
qqq
q
q q
qqq
qq
q
q
qq
qq
qq
q
q
q
q
qq
q
q
q
qq
q
q q
q
qqqqq
q
q qqqq
qqqqq
qqqqq
q
q
q
q
q
q
q
qqqqqqqqq
qqq
q
q
q
q
q
q
qqqqq
q
qqqq
q
qqqqq
qqqqqq
qq
q q
q q
qq
q
q
q
qqqq q
qq
qq
q
qq
qqq
qq
q
q
qq
q
q
q
q q
qqqqq
qqqqqqqqq
qq
q
qq
q
q
q
qq
q
q
q
qq q
qq
q
qq
qq
q
q q
q
q
qq
q
q
qqq
qqqqqqqqqqqqqqqq
q
qqqqqqqqqq
q
qqq
qqq
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
qq
qqq
q
qqqqq q
qqq
qq
q
q
qq
qq
q
q
q
qq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
q
q
qqqqqqq
qqqqqq
q
q q
q
q q
q
qqq
q
q
qq
q
qqqqqq
q
qq
q
q
qq
q
q
q
qqqq
q
qqq
q
q
q
q
q
q
q
q
q
qqqqqq
qq q q
qqq
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqq
q
q
q
qq
q
q
qq
q
q q
q
qq
q
q q
qqqqqq
q q
qqq
q
q
q
qqqqqqq qqqqq
q
q
q
qq
q q
qq
q
q q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
qqq
qqq
q q
q
q
q
q
q q q
q
q
q
qqq
q
qq
qqq
q
qqqqqq
qqq
q
qq
q
qqq
q
q
q
q
q
qq
qq
qq
qqq
q
qqqq
qq
q
q
q
q
q
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
q
q
qq qq
q
qqqqqqqqqqqq
qqqqqqqqqqqqq
qqq
qqq
qq
q
q
q
qq
qq
qq
q q
q
q
qq
q
qq
q
qqq
q
qqq
q
qq
q
qqqq
q
qq
qqq
q
qq
qq
q
qq
qq
q
q qqqqqqqqqq
qq
q
q
qq
qqq
qq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqq
qqq
q
qq
qqqq
q
q
qqq
qqqqqq
q
q
q
q
qq
q
qqqqqqqqqqqqqqqq qq
q
q
q qq
q
q
q q
q
q
q
qqqqqqqqqqqqqqqq
qq
q
qqq
q
q
q
qqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qq
qqq
q q
q
q
q
q
q
qqqqqq
qq
q
q
qq
q
qq
q
q
q
q
q
q
qqqq
qq
qq
qqq
q q
qqqq
qq
qq
q
qq
q
q
qq
q
q
q
q
qqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
2 0 2 4 6
0
2
4
6
8
tuning=0.5
Initial value=(0,7)
Acceptance rate=29.4%
Acceptance rate=29.4%
q
qq
q
qqqqq
qqqqq
qqqqq
q
qqqqqqqqqq
q
q
qq
q
q
q
q
qq
q
q
q
qqq
q
qq
q
qq
q
qq
qqqqqqqqqq
qq
q
q
qq
q
q
q
qqqq
qqqq
qq
q
qqqqq
q
qq
qqqqq
qqqqqqqqq
qq
q
qqq
qqq
qqq
qqq
qqqqq
qq
qqqqqqqqqqqq
qqqq
qqq
qq
q
qqqqqqqqqq
qqqq
qqqqqq
qqq
q
qqqq
q
qqq
qqqqqqq
qq
qqqqq
qqqq
qqqqqqqqqq qqqq
q
qq
q
q
qqqqq qqqq
qqq
q
qqq
qq
qqqqqqqqqqqqq
q
qqqqqqqq
qqqqqq q
q
qqqqq
qq
qq
q
q
qq
qq
qqq
qqq
q
qqqq
q
qq
q
q
qqq
qqqqqq
q
qqqqq
q
qqqqqqqq
qq
q
qqq
q
qqq
qqq
qqqqq qqqq
qqq
q
qq
qqq
q
q
q
qqqqq
q
qq
qq
qqqq
q
qq
q
qqqqqqqqqqqqqqqqqqqq qq
qqqqqqqqqqqq
q
qq
qqqqqqqq
q
q
qqqqqq
q
qq
qq
qqqq
qq
qqqqq
q
q
q
q
qqqqqqq
qq
qqq
qq
qqqqqqqqq
qqqq
qqqq
qqqqq
qqq
qq
qqqqqq
qq
qqq
qqq
q
qq
qqqq
q
qqqqqqqq
qqqqq
q
q
qqqqqq
qqqq
qqq
qq q
qq
q
qqq
qqqqqqq
q
qqqqqqqqq
qqq
qq
qq
q
qq
qqqqqqqq
qqqq qqqqqq
qqqqqq
qqq
qqqq
q
q
qqq
qqqqq
qqqqqqqq q
qqqqq
qqqq
qqqqq
qq
qqq
q
qqqqqqq
qqq
q q
qq qqqqqqqqqq
qqqq
qqqq
qq
qqqqqqq
q
qq
q
q
qqqqqq
q
q
qqqqq
q
qqqqq
qq
q
q
q
qqqq
qqqq
qq
q
qqq
q
q
qq
qq
q
qq
q
q
q
q
qq
qqqq
qq
qqqq
qqqq
qqqq
q
qq
q
qqqq
qqq
q
q
qqqq
qqqq
qqqqqq
q
q
qq
qqqq
qq
q
qqq
q
qq
qqqq
qqqqq
qq
qqqqqqqqqq
q
qqq
qqq
qqqq
q
qqq
qqqqqqqq
q
q
qqqqqqqqqq
qqqq
qqqq
qqqqqqqq
qq
qq
q
q
qq
qqqq
q
q
qqqqq
qqqq
q
q
q
q
qqqqqq
qqqqqq
qqqqq
qqq
qq
q
qq
qqqqq
qqqqq
q
qqq
q
qqqqq
qq
qq
qqq
qqqqq
qqqqqqqqqq
qq
qqqqqqqqqqqqqqqqq
qqqq
q
qqq
qqqqqqqqqqqqqq
q
q
qqqq
qqqqqqqqqqqqqqqqqq
q
qqqq
qq
q
q
qq
qqqqq
q
qqqqqqqqq
qqqq
qq
qq q
qqqqqqqqqqqqq
qq
qqqqqqq
qqqqqqq
qq
qq
qqq qqqqqqqqq
qqq
qqqqq
q
q
q
qq
q
q
q
q
qqq
q
qq
qq
qqqqq
qqqqqqqqqqq q
q
qq
q
qqqq
q
qqqq
qq
q
qq
q
q
q
q
qq
qqqqq
q
qqq
qqq
qqq
qq
q
q
q
qq
qq
q
qqq
q
q
q
q
qqqqq
qqqqq
qqq
qq
qq
qqqqq
q
qqq
qqqq
q
qqq
q
q
qq
q
qqq q
q
qqqq
qqqqqq
qqqqq
qqq
qqq
qqqqqqqqqq
qqqqqq
q
q
qqqqqq
qqqq
qqq
qqqq
qqqqqq
qqq
qq
q
q
qqqq
q
qq
q
q
q
qqq
qqqqq
qqq
qq
q
qqqqqqq
qq
q
q
qqqq
q
qqqqqq
qq
qq
q
qqqqqqqqqq
qqqqqqq
q
qqqq
q
q
qqqqq
qqq
qq
q
qqq
q
qqqqq
q
qqqqqqqqqqqqqqqqq
q
q
qq qq
qq
qq
q
qqq
qq
q
qq
q
qqqq
qqqq
qqqqqqqqqq
qqqqqqqq
qqqqqq
q
qqqqqqqqqqqqq
qqqqqqqq
qqqq
qqqqqq
q
qq
qqq
qqqq
q
q
qq q
qq
qqqq
qq
q
qqq
qqqq
qqq
qqq
qqq
q
q
qqq
q
qqq
qq
qq
qqqqqqqq
qq
q
q
qqqq
q qqq
qqq
qq
qqq
qq
qqqqqq
q
q
qq
qqqqqqqqqqqqqqqqq
qq
q
qqqqqqqq
qqqqqq
qqqqqq
qq
q
qqqqq
q
q
qqqqqqqqqq
q
qq
qq
qqqqq
q
qqqqqqqqq
qqqqqqqqq
q
qqqq
qq
q
qq
q
qqqqqqqq
qqq
qqqqqq
qqqqqqqq
q
qqqqqqqqqq
qq qqq
qq
q
q q
qq
qq
qq
qqq
qqq
q
q qqqqqqqq
qq
qq
q
q
q
qqq
qqqqq
qqqqq
q
qq
q
q
qqq
q
q
qqqqq
qq
q
qq
qqqqqqq
q
qq
qq
qqqq
q
q
qqq
q
q
qqqqq
q
qq
qqq
qq
qq
qqq
q
q
qq
qq
qqq
qq
q
qqqq
q
q
qqqqqqq
qqqqqqqqqqq
qqqq
qq
q
q
qq
qqq
qq
q
qqqqq
q
qqqqqqq
qqqqqqq
q
qqq
qqq
q
qqqqqqqqq
qqqqq
qq
q
qq
qqqqqqqq
qq
qq
qq
qqqqqqq
qqqq
qqq
qqqqqqq
qq
qqqq
q
q
qq
q
qqqqqqqqqqqqqqqq
q
q
q
q
qqq
qq
qqqq
qq
q
q
qqqq
q
qqqq
qqq
q
qq
qqqqq
qqqqq
qqq
qq
q
qq
q
qqq
q
qqqq
qqq
qqqq
qqqqqq qq
q
qq
qqq
q
q
q
qq
qq
qqq
q
qq
qq
qq
q
qqqqq
qqqqq qqqqqqqq
qqqqqqqqqq
qqq q
qqqqqqq
qqqqqqqqqqq
q
qqqqqqq
qqq
qqqq
q
qqqqqqq
qqq
qq
q
qqqqq
qqq
qqqqqq
qqqq
q
q
qqqqqqqqq
q
q
qq
qqq
q
qqqq
qqqqq
qqq
q
qqqqqqqqqqqq qqq
qq qq
qq
qqqqqqqq
qqqqqq
qqq
q
q
qq
qqq
qq
q
qqq
qq
qqqqqq
qq
qqqq
qqq
q
qqqqqqq
qqqqqq
qqqqq
qq
qqqq
qqqqqqqqqqq
qq
qqqqqqqqqqqqqqq
q
qqq
qqq
qqq
qq
qq
q
q
q
qqq
qqq
qq
qqqqqqqqqqqq
qqq
qq
qqq q
qqq
qqqqqqqqqqqq
qqqqqqq
qqqqqqqqq
qq
qqqqqqq
q
q
qqqqqqqqqq
qq
qq
qqqqqqq
qq
q
qqq
qqq
qqq
q
qq
qqqq
qqqqq
q
qqqqq
q
q
q
qqqqqqqqqqqqq
qqqqqqq
qqq
q
qq
qqqqq
q
qqqq q
qq
q
qqqq qq
q
q
qqqqq
qqq
q
qqqqqq
qqqqqqqqqqqqqqq
qqqq
qqqqqq
q
q
qqq
qq
qqqqqqq
qqq
q
qqqqq
q
q
q
q
q
qqqq
qqqqqqqqqqqqqq
qq
qqqqqq
qq
qq
qq
qq
qqq qq q
qqqqqq
qqqq
q
qqq
q
qq
q
qqqqqq
qqqq
q
qqq
qqq
qqq
q
q
q
q
qq
qqq
q
qqq
q
q
qqqqqqqqqqqqqqqqq qqq
q
qqq
qq
qqqqqqqq
qqq
q
q
q
q
qq
qqqqqqqqqqqq
qqqq
qq
q
q
q
q
qq
qqqqq
qq
q
qqqqqqqqqqqqq
q
qqqqqqqq
q
q
q
qqq
q
q
qqqqq
qqqqqq
qq
q
q
qq
qqq
qqqqqqqqq
qqqqq
qqqqqqq
qqqqqq
qqqqqqq
qq
q
q
q
qqqq qq
qq
qqqq
q
q
qq
q
q
qqq
qqq
qqqq
qq
qq
qqq
qqq
qqqqqqq
qqq
qqqqqqqqqqqqqq
qqq
q
q
qqqqq
q
qq
q
qq
q
q
qqqq
qqqq
q
q
qq
qqq
qqqqq
q
qqqq
q
qqqq
qqq
q q qqqqq
q
qqqqqq
qqq
qqqq
q
q
q
q
qqqqq
q
qqqq
qq
q
qq
qq
q
qqqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqq
qq
qq q
qq
q q
q
q
qqqqqq
qqqqq
qq
qqq
qqq
qqq
qqqqqqqqq
q q
q qqqqqqq
q
qqq
qqqq
q
q
q
q
q
qqq
qqqqqqqqqqqqqqqqqq
qqqqq q
qqq
qq
qqqqqqq
qqqq
qq
q
qq
qqqqq
qq
qqqqqq
qq
q
q
qq
qqq
qq
qqq
qqq
qq
q
qq
qq
qq
qqq
qqq
q
q
qqq
qq
qqq
q
q
q
qq
qq
qqqqq
qqq
q
q
q
qq qqqq
qqqqqqqqqqqqqq
qqqqqq
qq
q
qqqqq qqqq
q
qqqqqqqqq
qqq
q
q
qqqq
qq
qq
qq
q
qq
q
qqqq
q
qq
qqq
q
q
qq
qqqq
q
q
qqq
q
qq
qq
qq
qq
q
qq
qqq
qqq
q
qqq
qqqqqq
qq
qqqqqq
qqqqqq
qqq
qqqq
qqqq q
qqqq
qq
q
qqqq
q
q
qqqqqqq
qqqqqqqqq
q
q
qq
qq
qq
qqqqq
qqqqqqqq
q
q
q
qq
qqq
qqqqqq
q
q
q
q
q
qqqqq
q
qq
qqqqqqqqqq
qq
qqqqqqqqqq
q
qqqq
qq
qqqqqqq
qq
qqqqqqqqq
qq
qqqqq
q
q
qq
qq
qq
q
q
qqqq
q
qq
q
qq
qqqq qqqqq
qqqqqqq
qqq
qq
qqq
qqqqq
q
q
qqq
qqq
qqqqq
q
qq
qqq
qqqq
q
q
q
q
qqqq
q
q
qqqqq
q
q
qqqq
qqq
qq
q
q
qqq
q
q
q
qqqq
qqqq
qqqqq
qqqqqq
qq
q
qq
q
qqqq
qqqqqqqqqq
qq
qqqq
q
q
q
qqqqqq
qqqqqq
qqq
qqq
qqqqq
qqqqq
q
qqqqqqqqqqq
qq
qqqqqq
qqqq
q
qqqqqq qqqqq
q
q
q
qqq
q
qq
q
qq
qqqqq
qqq
qqqqqqq
qq
q
q
qqqqqqq
qqqqq
q
qqqqqqqq
qq
qqq
qqq
q
qq
qqqqqq
qq
q
qqqqq
q
q
qqqqqqqq
qq
qq
qqqq
qqq
qq
qqq q
q
q
qqqqqq
qq
qq
q
qqq
q
qq
q q
qqqqqq
qq
q
qqqq
q
qqqqq
qq
q
qq
qqqqqqq
qqqqqqqqqqqq
q
qq
qq
qq
qq
qq
q
q
qqq
q
q
qqqqqqq
qqqq
q qq
qq
qq
qq
q
qq
q
qq
qq
qqqq
qqq
qqqqqq
qq
qqqqqqq
qqqqqq
qq
q
qqq
qqq
qq
q qqqqq
qqq
qq
qqqqqqqqqqqqqq
q
qq
q
q
qqqq
qq
qqqqqq
qqq
qq
qqqqq
qqqqqqq
q q
qqqqqqqq
q
q
qqqq
q
q
q
qq
qq
q
qq
q
q
q
q
qqq
qqqqq
qqqqqq
qqqqqq
qq
qqqqqqqqqqqqqq
qqqqq
qq
qq
q
qqqqqqq
qqq
qqq
qqqqqqqq
qq
qqqqqqqqqqqqqqqqq
qqq
q
qqq
qqqqq
qqqqq
q
qqqq
qqqqq qqqqq
qq
qqqq q
q
q
q
qq
qqqqq
q
qqq
q
qqqq
q
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqq
qq
qqq q
qqq
qq
qq
q
q
q
qq
qq
qqqqq
q
qqq
q
qq
q
q
qq
q
qq
q
q
qqqq
q
qqq
q
q
qqq
qqqqqqqqq
q
q
qq
qqqq
qq
q
qqqq
q qqq
q
qqq
q
qq
qqqq
qqqq
q
qqqqqqq
qq
qqqqqqqqqq
q
qqq
qq
qqqqq
qqqqqqqqqqq
q
q
qqq
qqqqqqqqq
qqqqqqqqqq
q
q
qq
qq
qqqq
q
qqq
qqq
q
qq
qqqq
qqq
qq
q
qqqq
qqqqqqqqq
qqqqqqqqqq qqqqqqq
q
q
qqq
q
qqqq
q
qqq
qqq
qqq
q
qqq
q
q
q
qqqqqqqqq
qq
q
q
qq
qqqqq
qq
qqq
qq
qqqqqqqqq
qqqqq
qqq
qqqqq
q
q
q
qqqq
q
q
qqqqq
qqq
qq
q
q
qqqqqqqqqqqq
qq
qqqqqq
qqq
qq
qq
qqqqqqqqq
q
qqqqq
qqq
q
qq
qqqqq q
qqqqq
qqqq
q
qqqq
q
qqqq
qq
qqqq
q
q
qqqq
qqqq
qq
q
qqqq
qqqq
qqqqqqqqq
q
q
q
q
qqqqqqqqqqqqqqqqqqq
qqqq
qqqqq
q
q
qqqq
qq
qq
qqq
q
qqq
q
qqq
q
qqqqqqqqqqqqqqqqqqqq
q
q
q
qqqqq
qq
qqqqqq
q
qq
q
qqqqq
qqqqqqqqqq
qqqqqqq
qqqqqqqqqqqqqq
qqq
qqqqqqqqqq
q
qqqqqqqqqqq
q
qqqq
qqqqqqqq
qqqqq
q
q
qqqq
q
qq
qqq
q
qqqq
q
q
qqqqqq
qq
qqq
qqq
q
qqq
qqqqqqqqq
qq
qq
qqqq
qqq
qqqqq
qqqqqq
q
qqqqqqqqqqqq
q
q q
qqqqqq
qq
q qq
qqqqqqqqqq
2 0 2 4 6
0
2
4
6
8
tuning=5
Initial value=(0,7)
Acceptance rate=32.2%
Acceptance rate=32.2%
qqq
qqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqq
qqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qq
qqqqqqqqqqqqqqqqqqqqq
q
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
q
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqq
qqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqq
qqqqqqqqqqqqqq
qqqqqqqqqqqqqqqq
qqqq
qq
qqqqqqqqqqqqqqqqqq
qqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqq
q
qqqqqqqq
qqqqqqqqqq
q
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqqq
qqqqqqqqqq
qqqq
qqqq
qqqqq
qqqqqqqqqqqq
qqqq
qq
q
q
qqqqqqqqqqqqqqqq
qqqq
qqqqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqq qqqqqqqqq
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqq
qq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqqqq
qqqqqqq
q
qqqqqqqqqqqqqqqqqqqqqq
qqqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqq
qqqqqqqqqqq
qqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqq
q
qqqqqqqqq
qqq
qqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqq
q
qqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
q
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqq
qqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqq
q
qqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqqqq
qqqqqqqqqqqq
q
2 0 2 4 6
0
2
4
6
8
tuning=50
Initial value=(0,7)
Acceptance rate=4.3%
Acceptance rate=4.3%
55 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Autocorrelations
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=0.5
Initial value=(4,5)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=5
Initial value=(4,5)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=50
Initial value=(4,5)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=0.5
Initial value=(0,7)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=5
Initial value=(0,7)
0 50 100 150 200
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
lag
Tuning=50
Initial value=(0,7)
56 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Simulated annealing
Simulated annealing
2
is an optimization technique designed to
nd maxima of functions.
It can be seen as a M-H algorithm that tempers with the target
distribution:
q() ()
1/T
where the constant T > 1 receives the physical interpretation
of system temperature, hence the nomenclature used
(Jennison, 1993).
The heated distribution q is attened with respect to and its
density gets closer to the uniform distribution, which is
particularly relevant for the case of a distribution with distant
modes.
By attening the modes, the moves required to cover
adequately the parameter space become more likely.
2
Kirkpatrick, Gelatt and Vecchi (1983)
57 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example vii: Nonlinear surface
Assume that the goal is to nd the mode/maximum of
(
1
,
2
)
4

i =1
e
(
1
+
2
x
i
)y
i
(1 + e

1
+
2
x
i
)
5
,
with x = (0.863, 0.296, 0.053, 0.727) and y = (0, 1, 3, 5).
The simulated annealing algorithm is implemented for four
initial values:
(5, 30) (2, 40) (4, 10) (6, 0)
and two cooling schedules:
T
i
= 1/i and T
i
= 1/[10 log(1 + i )].
The proposal distribution is q(|
(i )
) = f
N
(;
(i )
, 0.05
2
I
2
).
58 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Ti=1/i
1

2
4 2 0 2 4 6

1
0
0
1
0
2
0
3
0
4
0

1
Iterations
0 1000 2000 3000 4000 5000

2
0
2
4
6

2
Iterations
0 1000 2000 3000 4000 5000

1
0
0
1
0
2
0
3
0
4
0
Ti=1/[10log(i+1)]
1

2
4 2 0 2 4 6

1
0
0
1
0
2
0
3
0
4
0

1
Iterations
0 1000 2000 3000 4000 5000

2
0
2
4
6

2
Iterations
0 1000 2000 3000 4000 5000

1
0
0
1
0
2
0
3
0
4
0
Newton-Raphson mode: (0.87, 7.91).
T
i
= 1/i : mode is (0.88, 7.99) when (
(0)
1
,
(0)
2
) = (5, 30).
59 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
3000 3500 4000 4500 5000
0
.
8
2
0
.
8
6
0
.
9
0
0
.
9
4

1
Iterations
3000 3500 4000 4500 5000
7
.
7
7
.
9
8
.
1

2
Iterations
60 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Gibbs sampler
Technically, the Gibbs sampler is an MCMC scheme whose
transition kernel is the product of the full conditional
distributions.
Algorithm
1 Start at
(0)
= (
(0)
1
,
(0)
2
, . . .)
2 Sample the components of
(j )
iteratively:

(j )
1
(
1
|
(j 1)
2
,
(j 1)
3
, . . .)

(j )
2
(
2
|
(j )
1
,
(j 1)
3
, . . .)

(j )
3
(
3
|
(j )
1
,
(j )
2
, . . .)
.
.
.
The Gibbs sampler opened up a new way of approaching
statistical modeling by combining simpler structures (the full
conditional models) to address the more general structure (the
full model).
61 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example viii: Bivariate normal
Assume that the target distribution is the bivariate normal with
mean vector and covariance matrix given by
=
_

1

2
_
and =
_

2
1

12

12

2
2
_
,
respectively.
In this case, the two full conditionals are given by

1
|
2
N
_

1
+

12

2
2
(
2

2
),
2
1

2
12

2
2
_
and

2
|
1
N
_

2
+

12

2
1
(
1

1
),
2
2

2
12

2
1
_
62 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References

1
=
2
= 0

2
1
=
2
2
= 1

12
= 0.95

2
4 2 0 2 4

2
0
2
4
63 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
4 2 0 2 4

2
0
2
4

2
64 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
4 2 0 2 4

2
0
2
4

2
65 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
4 2 0 2 4

2
0
2
4
1

2
4 2 0 2 4

2
0
2
4
1

2
4 2 0 2 4

2
0
2
4
1

2
4 2 0 2 4

2
0
2
4
1

2
0 10 20 30 40
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Lag
A
C
F
0 10 20 30 40
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Lag
A
C
F
0 10 20 30 40
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Lag
A
C
F
0 10 20 30 40
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Lag
A
C
F
0 5 10 15 20 25 30
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Lag
A
C
F
0 5 10 15 20 25 30
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Lag
A
C
F
0 5 10 15 20 25 30
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Lag
A
C
F
0 5 10 15 20 25 30
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Lag
A
C
F
Middle frame: Based on M = 21, 000 consecutive draws.
Bottom frame: Based on M = 1000 draws, after initial
M
0
= 1000 draws and saving every 20th draws.
66 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
4 2 0 2

2
0
2
4
1

1
D
e
n
s
i
t
y
4 2 0 2 4
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4

2
D
e
n
s
i
t
y
4 2 0 2 4
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
67 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example: Seemingly Unrelated
Regressions
Investments
Grunfeld (1958), Boot and White (1960) and Zellner
(1962,1963) study
I
mt
=
m1
+
m2
F
mt
+
m3
C
mt
+
mt
Should the parameters be the same across rms?
Do the
mt
share unobserved common factors?
Staking the observation for rm m:
y
m
= X
m

m
+
m
68 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Capital asset pricing
For a given security, the CAPM species that
r
mt
r
ft
=
m
+
m
(r
mt
r
ft
) +
mt
for return on security m, r
mt
, return on a risk-free security, r
ft
,
and market return, r
mt
.
Are the disturbances correlated across securities?
Are the
m
s and/or
m
s related in any way?
Staking the observation for security m:
y
m
= X
m

m
+
m
69 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Gross State Product
Greene (2008) examines (his examples 9.9 and 9.12) Munnells
(1990) model for output by the 48 continental US states:
log GSP
mt
=
m1
+
m2
log pcap
mt
+
m3
log hwy
mt
+
m4
log water
mt
+
m5
log util
mt
+
m6
log emp
mt
+
m7
unemp
mt
+
mt
Should the coecient vector be the same across states?
Should the disturbances correlated across states?
Should the disturbances correlated across time?
Staking the observation for state m:
y
m
= X
m

m
+
m
70 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
SUR
For m = 1, . . . , M and t = 1, . . . , T
y
mt
= x

mt

m
+
mt
,
with x
mt
a k
m
-dimensional vector of regressors.
Let us stack all equations:
y
t
= (y
1t
, . . . , y
Mt
)

(M 1)

t
= (
1t
, . . . ,
Mt
)

(M 1)
= (

1
, . . . ,

M
)

(k 1)
X
t
= diag(x

1t
, . . . , x

Mt
) (M k)
where k =

M
m=1
k
m
. Therefore,
y
t
= X
t
+
t
71 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
We can now stack all observations t = 1, . . . , T together:
y = (y
1
, . . . , y
T
)

= (
t
, . . . ,
T
)

X = (X

1
, . . . , X

T
)

,
such that
y = X +.
NLRM:
mt
are i.i.d. N(0,
2
) for all m and t.
SUR:
t
are i.i.d. N(0, ) for all t.
This leads to N(0, ), where
= diag(, . . . , ) = I
T

is an MT MT block-diagonal covariance matrix.
72 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Prior distribution
Conditionally conjugate prior for and =
1
:
p(, ) = p()p(),
where
N(
0
, V
0
)
and
Wishart(
0
,
0
).
See Dreze and Richard (1983) and Richard and Steel (1988) for
further discussion regarding alternative prior specications.
73 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Full conditionals
The full conditional distributions are
|y, X, N(
1
, V
1
)
|y, X, Wishart(
1
,
1
)
where
1
=
0
+ T,
V
1
1
= V
1
0
+
T

t=1
X

t
X
t
V
1
1

1
= V
1
0

0
+
n

t=1
X

t
y
t

1
1
=
1
0
+
T

t=1
(y
t
X
t
)(y
t
X
t
)

74 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Grunfelds (1958) data
M=10 U.S. rms over T=20 years, 1935-1954.
3
Variables:
FN = Firm Number; YR = Year;
I = Annual real gross investment;
F = Real value of the rm (shares outstanding); and
C = Real value of the capital stock.
Firms:
General Electric, Westinghouse,
U.S. Steel, Diamond Match,
Atlantic Rening, Union Oil,
Goodyear, General Motors,
Chrysler and IBM
3
Zellner (1971), pages 240-246.
75 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Individual regressions
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
50 100 150 200
5
0
1
0
0
1
5
0
2
0
0
Real gross
F
itte
d
re
a
l g
ro
s
s
General Electric
s2=777.45
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
20 40 60 80
2
0
4
0
6
0
8
0
Real gross
F
itte
d
re
a
l g
ro
s
s
Westinghouse
s2=104.31
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
200 300 400 500 600
2
0
0
3
0
0
4
0
0
5
0
0
6
0
0
Real gross
F
itte
d
re
a
l g
ro
s
s
U.S. Steel
s2=9299.6
q
q
q
qq q
q
q
q
q q q
q
q
q
q
q
q
q
q
1 2 3 4 5 6
1
2
3
4
5
6
Real gross
F
itte
d
re
a
l g
ro
s
s
Diamond Match
s2=1.18
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
40 50 60 70 80 90
4
0
5
0
6
0
7
0
8
0
9
0
Real gross
F
itte
d
re
a
l g
ro
s
s
Atlantic Refining
s2=82.17
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
20 30 40 50 60 70 80 90
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
0
Real gross
F
itte
d
re
a
l g
ro
s
s
Union Oil
s2=88.67
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
20 30 40 50 60
2
0
3
0
4
0
5
0
6
0
Real gross
F
itte
d
re
a
l g
ro
s
s
Goodyear
s2=82.79
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
200 400 600 800 1000 1200 1400
2
0
0
4
0
0
6
0
0
8
0
0
1
0
0
0
1
2
0
0
1
4
0
0
Real gross
F
itte
d
re
a
l g
ro
s
s
General Motors
s2=8423.88
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
50 100 150
5
0
1
0
0
1
5
0
Real gross
F
itte
d
re
a
l g
ro
s
s
Chrysler
s2=176.32
q
q
qq
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
20 40 60 80 100 120 140
2
0
4
0
6
0
8
0
1
0
0
1
2
0
1
4
0
Real gross
F
itte
d
re
a
l g
ro
s
s
IBM
s2=65.33
76 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Regression coecients
General Electric
BETA1
0 200 600 1000

1
0
0

5
0
0
5
0
Westinghouse
BETA1
0 200 600 1000
0
.0
0
0
.0
2
0
.0
4
0
.0
6
U.S. Steel
BETA1
0 200 600 1000

0
.0
5
0
.0
0
0
.0
5
0
.1
0
0
.1
5
0
.2
0
Diamond Match
BETA1
0 200 600 1000

1
0
0
1
0
2
0
3
0
4
0
Atlantic Refining
BETA1
0 200 600 1000
0
.0
2
0
.0
4
0
.0
6
0
.0
8
Union Oil
BETA1
0 200 600 1000

0
.1
5

0
.0
5
0
.0
5
0
.1
0
Goodyear
BETA1
0 200 600 1000

1
0
0
0
1
0
0
2
0
0
3
0
0
General Motors
BETA1
0 200 600 1000
0
.0
5
0
.1
0
0
.1
5
0
.2
0
Chrysler
BETA1
0 200 600 1000

0
.6

0
.4

0
.2
0
.0
0
.2
0
.4
0
.6
IBM
BETA1
0 200 600 1000

5
0
5
1
0
1
5
General Electric
BETA2
0 200 600 1000

0
.2
0

0
.1
0
0
.0
0
0
.0
5
Westinghouse
BETA2
0 200 600 1000
0
.0
0
.2
0
.4
0
.6
U.S. Steel
BETA2
0 200 600 1000
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
Diamond Match
BETA2
0 200 600 1000

0
.1
0
.0
0
.1
0
.2
Atlantic Refining
BETA2
0 200 600 1000
0
.0
0
0
.0
5
0
.1
0
Union Oil
BETA2
0 200 600 1000

4
0

2
0
0
2
0
4
0
Goodyear
BETA2
0 200 600 1000

0
.1
0
.0
0
.1
0
.2
0
.3
General Motors
BETA2
0 200 600 1000
0
.0
5
0
.1
0
0
.1
5
Chrysler
BETA2
0 200 600 1000

4
0

2
0
0
1
0
2
0
3
0
IBM
BETA2
0 200 600 1000
0
.0
0
0
.0
5
0
.1
0
0
.1
5
General Electric
BETA3
0 200 600 1000

0
.0
5
0
.0
0
0
.0
5
0
.1
0
0
.1
5
0
.2
0
Westinghouse
BETA3
0 200 600 1000

3
0
0

2
0
0

1
0
0
0
U.S. Steel
BETA3
0 200 600 1000
0
.0
6
0
.0
8
0
.1
0
0
.1
2
0
.1
4
0
.1
6
Diamond Match
BETA3
0 200 600 1000
0
.2
0
0
.3
0
0
.4
0
0
.5
0
Atlantic Refining
BETA3
0 200 600 1000

2
0
0
2
0
4
0
6
0
Union Oil
BETA3
0 200 600 1000

0
.0
2
0
.0
2
0
.0
6
0
.1
0
Goodyear
BETA3
0 200 600 1000
0
.1
0
0
.2
0
0
.3
0
0
.4
0
General Motors
BETA3
0 200 600 1000

1
0
0
1
0
2
0
Chrysler
BETA3
0 200 600 1000
0
.0
6
0
.0
8
0
.1
0
0
.1
2
0
.1
4
0
.1
6
0
.1
8
IBM
BETA3
0 200 600 1000

0
.2

0
.1
0
.0
0
.1
0
.2
0
.3
77 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Standard deviations
0 200 400 600 800 1000
2
0
3
0
4
0
5
0
General Electric
0 200 400 600 800 1000
1
0
1
5
2
0
2
5
Westinghouse
0 200 400 600 800 1000
6
0
8
0
1
0
0
1
2
0
1
4
0
1
6
0
1
8
0
U.S. Steel
0 200 400 600 800 1000
1
.0
1
.5
2
.0
2
.5
Diamond Match
0 200 400 600 800 1000
1
0
1
5
2
0
Atlantic Refining
0 200 400 600 800 1000
6
8
1
0
1
2
1
4
1
6
Union Oil
0 200 400 600 800 1000
6
8
1
0
1
2
1
4
1
6
Goodyear
0 200 400 600 800 1000
6
0
8
0
1
0
0
1
2
0
1
4
0
General Motors
0 200 400 600 800 1000
1
0
1
5
2
0
2
5
Chrysler
0 200 400 600 800 1000
5
1
0
1
5
2
0
IBM
78 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Correlations
0.2 0.6 1.0
0
2
4
General Electric
Westinghouse
0.2 0.2 0.6
0
.
0
2
.
0
General Electric
U.S. Steel
0.2 0.2 0.6
0
.
0
1
.
5
General Electric
Diamond Match
0.5 0.0 0.5
0
.
0
1
.
0
General Electric
Atlantic Refining
0.5 0.0 0.5
0
.
0
1
.
0
General Electric
Union Oil
0.5 0.0 0.5
0
.
0
1
.
5
General Electric
Goodyear
0.5 0.0 0.5
0
.
0
1
.
5
General Electric
General Motors
0.5 0.0 0.5
0
.
0
1
.
5
General Electric
Chrysler
0.2 0.2 0.6
0
.
0
1
.
5
General Electric
IBM
0.3 0.5 0.7 0.9
0
2
4
Westinghouse
U.S. Steel
0.2 0.2 0.6 1.0
0
.
0
1
.
5
Westinghouse
Diamond Match
0.5 0.0 0.5
0
.
0
1
.
0
Westinghouse
Atlantic Refining
0.5 0.0 0.5
0
.
0
1
.
0
Westinghouse
Union Oil
0.4 0.0 0.4 0.8
0
.
0
1
.
5
Westinghouse
Goodyear
0.6 0.0 0.4 0.8
0
.
0
1
.
5
Westinghouse
General Motors
0.5 0.0 0.5
0
.
0
1
.
0
Westinghouse
Chrysler
0.0 0.4 0.8
0
.
0
1
.
5
Westinghouse
IBM
0.2 0.2 0.6 1.0
0
.
0
2
.
0
U.S. Steel
Diamond Match
0.4 0.0 0.4 0.8
0
.
0
1
.
5
U.S. Steel
Atlantic Refining
0.5 0.0 0.5
0
.
0
1
.
0
U.S. Steel
Union Oil
0.5 0.5 1.0
0
.
0
1
.
0
U.S. Steel
Goodyear
0.5 0.0 0.5
0
.
0
1
.
0
U.S. Steel
General Motors
0.5 0.0 0.5
0
.
0
1
.
5
U.S. Steel
Chrysler
0.2 0.2 0.6 1.0
0
.
0
1
.
5
U.S. Steel
IBM
0.5 0.0 0.5
0
.
0
1
.
5
Diamond Match
Atlantic Refining
0.5 0.0 0.5
0
.
0
1
.
5
Diamond Match
Union Oil
0.5 0.0 0.5
0
.
0
1
.
0
Diamond Match
Goodyear
0.5 0.0 0.5
0
.
0
1
.
0
Diamond Match
General Motors
0.4 0.0 0.4 0.8
0
.
0
1
.
5
Diamond Match
Chrysler
0.2 0.2 0.6 1.0
0
.
0
1
.
5
Diamond Match
IBM
0.5 0.5 1.0
0
.
0
1
.
0
Atlantic Refining
Union Oil
0.5 0.0 0.5
0
.
0
1
.
0
Atlantic Refining
Goodyear
0.8 0.2 0.2
0
.
0
1
.
5
Atlantic Refining
General Motors
0.5 0.0 0.5
0
.
0
1
.
5
Atlantic Refining
Chrysler
0.4 0.0 0.4 0.8
0
.
0
1
.
5
Atlantic Refining
IBM
0.5 0.0 0.5
0
.
0
1
.
0
Union Oil
Goodyear
0.4 0.0 0.4 0.8
0
.
0
1
.
5
Union Oil
General Motors
0.5 0.0 0.5
0
.
0
1
.
0
Union Oil
Chrysler
0.5 0.0 0.5
0
.
0
1
.
0
Union Oil
IBM
0.4 0.0 0.4 0.8
0
.
0
1
.
5
Goodyear
General Motors
0.5 0.0 0.5
0
.
0
1
.
5
Goodyear
Chrysler
0.5 0.0 0.5
0
.
0
1
.
0
Goodyear
IBM
0.8 0.2 0.2
0
.
0
1
.
5
General Motors
Chrysler
0.5 0.0 0.5
0
.
0
1
.
0
General Motors
IBM
0.5 0.5 1.0
0
.
0
1
.
0
Chrysler
IBM
79 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
SUR references
Boot and De Wit (1960) Investment Demand: An Empirical
Contribution to the Aggregation. Problem, International
Economic Review, 1, 3-30.
Zellner (1971) An Introduction to Bayesian Inference and
Econometrics, Wiley.
Koop (2003) Bayesian Econometrics, Wiley.
Kleiber and Zeileis (2010) The Grunfeld Data at 50, German
Economic Review, 11(4), 404-417.
80 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Example: Bivariate BVAR(1)
Let y
t
= (y
t1
, y
t2
)

contain 2 time series observed at time t.

The (basic) VAR(1) can be written as
(y
t
|y
t1
, B, ) N(By
t1
, )
where
B =
_

11

12

21

22
_
and =
_

2
1

12

12

2
2
_
81 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
VAR(1) as a SUR model
The above VAR(1) model can be rewritten as a SUR model as
(y
t
|z
t
, , ) N(z
t
, )
where
z
t
=
_
y
t1,1
y
t1,2
0 0
0 0 y
t1,1
y
t1,2
_
and
=
_
_
_
_

11

12

21

22
_
_
_
_
.
Therefore,
(y|, ) N(Z, )
where y = (y

1
, . . . , y

T
)

and Z = (z

1
, . . . , z

T
)

.
82 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Prior of (, )
We assume that and are independent a priori.
Prior of :
N(b
0
, B
0
).
Prior of :
IW(v
0
, V
0
).
This conditionally conjugate prior DOES NOT lead to closed
form posterior inference, but the implementation of the Gibbs
sampler is straightforward.
83 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Full conditional of
It is easy to see that
p(|, y) exp
_
0.5
_

B
1
0
2

B
1
0

0
_
exp
_
0.5
_

1
Z 2

1
y
_
.
Therefore,
|, , y N(
1
, V
1
)
where

1
= B
1
(B
1
0

0
+ Z

1
y) and B
1
1
= B
1
0
+ Z

1
Z.
84 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Full conditional of
It is easy to see that
p(|, y) ||
q+v
0
+1
2
exp
n
0.5tr(
1
V
0
)
o
||
q+T+1
2
exp
(
0.5
T
X
t=1
(y
t
z
t
)

1
(y
t
z
t
)
)
.
Therefore,
|, y IW(v
1
, V
1
)
where v
1
= v
0
+ T and
S = V
0
+
T

t=1
(y
t
z
t
)(y
t
z
t
)

.
85 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Simulated data
We simulated n = 360 observations (30 years of monthly data)
from the above bivariate VAR(1) with
B =
_
0.85 0.10
0.00 0.95
_
and
=
_
1.0 0.2
0.2 1.0
_
86 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Data
Time Series
Time
0 50 100 150 200 250 300 350

2
0
2
4
6
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
6 4 2 0 2 4 6

2
0
2
4
6
Y1
Y
2
87 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Posterior inference
The prior hyperparameters are
b
0
= 0
4
and B
0
= 1000I
4
and
v
0
= 5 and V
0
= 0.001
We started the Gibbs sampler with B
(0)
= B (true value).
We runt he Gibbs sampler for M = 10, 000 iterations.
88 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
p(B|data)
0 2000 4000 6000 8000 10000
0
.
7
5
0
.
8
0
0
.
8
5
0
.
9
0
0
.
9
5
Iterations
B11
0 2000 4000 6000 8000 10000
0
.
0
0
0
.
0
5
0
.
1
0
0
.
1
5
Iterations
B12
0 2000 4000 6000 8000 10000

0
.
1
0

0
.
0
5
0
.
0
0
0
.
0
5
Iterations
B21
0 2000 4000 6000 8000 10000
0
.
8
5
0
.
9
0
0
.
9
5
1
.
0
0
Iterations
B22
D
e
n
s
it
y
0.75 0.80 0.85 0.90 0.95
0
5
1
0
1
5
Pr(B11>1)=0
D
e
n
s
it
y
0.00 0.05 0.10 0.15
0
5
1
0
1
5
D
e
n
s
it
y
0.10 0.05 0.00 0.05
0
5
1
0
1
5
D
e
n
s
it
y
0.85 0.90 0.95 1.00
0
5
1
0
1
5
Pr(B22>1)=0.0012
89 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
p(|data)
0 2000 4000 6000 8000 10000
0
.
7
0
.
8
0
.
9
1
.
0
1
.
1
1
.
2
Iterations
S11
0 2000 4000 6000 8000 10000
0
.
1
0
.
2
0
.
3
0
.
4
Iterations
S21
0 2000 4000 6000 8000 10000
0
.
7
0
.
8
0
.
9
1
.
0
1
.
1
1
.
2
Iterations
S22
D
e
n
s
it
y
0.7 0.8 0.9 1.0 1.1 1.2
0
1
2
3
4
5
6
D
e
n
s
it
y
0.1 0.2 0.3 0.4
0
2
4
6
8
D
e
n
s
it
y
0.7 0.8 0.9 1.0 1.1 1.2
0
1
2
3
4
5
6
90 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Book references
1 Ripley (1987) Stochastic Simulation. Wiley. (Chapters 5 and 7).
2 Thisted (1988) Elements of Statistical Computing. Chapman & Hall/CRC. (Chapter 5).
3 Gentle (1998) Random Number Generation and Monte Carlo Methods. Springer. (Chapter 5).
4 Liu (2001) Monte Carlo Strategies in Scientic Computing. Springer. (Chapters 2, 5 and 6).
5 Robert and Casella (2004) Monte Carlo Statistical Methods. Springer. (Chapters 3, 4 and 7).
6 Givens and Hoeting (2005) Computational Statistics. Wiley. (Chapters 6, 7 and 8).
7 Gamerman and Lopes (2006) MCMC: Stochastic Simulation for Bayesian Inference. Chapman &
Hall/CRC. (Chapters 3, 5 and 6).
8 Rizzo (2008) Statistical Computing with R. Chapman & Hall/CRC. (Chapters 5 and 9).
9 Rubinstein and Kroese (2008) Simulation and the Monte Carlo Method. Wiley. (Chapters 5 and 6).
10 Robert and Casella (2010) Introducing Monte Carlo Methods with R. Springer. (Chapters 3, 6 and
7).
91 / 92
A bit of
history
MC
integration
MC via IS
Rejection
method
SIR method
MH
algorithms
Simulated
annealing
Gibbs sampler
SUR model
BVAR
References
Seminal papers
1 Metropolis and Ulam (1949) The Monte Carlo method. JASA, 44, 335-341.
2 Metropolis, Rosenbluth, Rosenbluth, Teller and Teller (1953) Equation of state calculations by fast
computing machines. Journal of Chemical Physics, Number 21, 1087-1092.
3 Hastings (1970) Monte Carlo sampling methods using Markov chains and their applications.
Biometrika, 57, 97-109.
4 Peskun (1973) Optimum Monte-Carlo sampling using Markov chains. Biometrika, 60, 607-612.
5 Besag (1974) Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal
Statistical Society, Series B, 36, 192-236.
6 Geman and Geman (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of
images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721-741.
7 Jennison (1993) Discussion of the meeting on Gibbs sampling and other Markov chain Monte Carlo
methods. JRSS-B, 55, 54-6.
8 Kirkpatrick, Gelatt and Vecchi (1983) Optimization by simulated annealing. Science, 220, 671-80.
9 Pearl (1987) Evidential reasoning using stochastic simulation. Articial Intelligence, 32, 245-257.
10 Tanner and Wong (1987) The calculation of posterior distributions by data augmentation. JASA, 82,
528-550.
11 Geweke (1989) Bayesian inference in econometric models using Monte Carlo integration.
Econometrica, 57, 1317-1339.
12 Gelfand and Smith (1990) Sampling-based approaches to calculating marginal densities, JASA, 85,
398-409.
13 Casella and George (1992) Explaining the Gibbs sampler. The American Statistician,46,167-174.
14 Smith and Gelfand (1992) Bayesian statistics without tears: a sampling-resampling perspective.
American Statistician, 46, 84-88.
15 Gilks and Wild (1992) Adaptive rejection sampling for Gibbs sampling. Applied Statistics, 41, 337-48.
16 Chib and Greenberg (1995) Understanding the Metropolis-Hastings algorithm. The American
Statistician, 49, 327-335.
17 Dongarra and Sullivan (2000) Guest editors introduction: The top 10 algorithms. Computing in
Science and Engineering, 2, 22-23.
92 / 92