Bernardo DAuria

Stochastic Processes - 2008/09

Problem Set 1
February 24th , 2009

P1. An urn contains a white and b black balls. After a ball is drawn, it
is returned to the urn if it is white; but if it is black, it is replaced by a white
ball from another urn. Let Mn denote the expected number of white balls in
the urn after the foregoing operation has been repeated n times.
a) Derive the recursive equation

Mn+1 =

1
1
a+b


Mn + 1.

b) Use part (a) to prove that


Mn = a + b b 1

1
a+b

n
.

c) What is the probability that the (n + 1)st ball drawn is white?

P2. Suppose that events occur according to a Poisson process with rate
. Each time an event occurs we must decide whether or not to stop, with our
objective being to stop at the last event to occurr prior to some specified time
T . That is, if an event occurs at time t, 0 t T and we decide to stop, then
we lose if there are any events in the interval (t, T ], and win otherwise. If we
do not stop when an event occurs, and no additional events occur by time T ,
then we also lose. Consider the strategy that stops at the first event that occurs
after some specified time s, 0 s T .
a) If the preceding strategy is employed, what is the probability of winnning?
b) What value of s maximize the probability of winning?
c) Show that the probability of winning under the optimal strategy is 1/e.

P3. Consider an elevator that starts in the basement and travels upward.
Let Ni denote the number of people that get in the elevator at floor i. Assume
the Ni are independent and that Ni is Poisson with mean i . Each person
entering
P at i will, independent of everything else, get off at j with probability
Pij . j>i Pij = 1. Let Oj = {number of people getting off the elevator at floor
j}.
a) Compute E[Oj ].
b) What is the distribution of Oj ?
c) What is the joint distribution of Oj and Ok ?

Universidad Carlos III de Madrid