Assignment #2: Due June 28

1. Suppose is a renewal event. Let {rn } n=0 be the renewal sequence, that is, r0 = 1, rn = P ( occurs on trial n), n 1. Let f0 = 0 and fn = P ( rst occurs on trial n), n 1.
Suppose R (s) and F (s) are the generating functions for {rn } n=0 and {fn }n=0 , respectively.

(a) Show that for n 1,

n n

rn =
k=1

fk rnk =
k=0

fk rnk .

Note that the above formula is not true for n = 0. (Conditional on the rst time we observe the event and use the renewal property.) (b) Based on the above formula, prove the renewal relationship: F (s) = 1 1 . R (s)

2. Suppose a sequence of independent trials X1 , X2 , ... is generated by randomly picking digits (with replacement) from the set {0, 1, 2, ..., 9}. We say the event 235 occurs on trial n if Xn2 = 2, Xn1 = 3, Xn = 5. This is a renewal event (why?). There are a number of ways of showing that 235 is in fact recurrent: (a) Use the renewal sequence. Find rn = P (235 occurs on trial n) and then show that n=1 rn = . (This is the quickest method.) (b) This method requires a little more work but allows us to additionally determine E (T235 ) as well as the probability distribution of T235 . Obtain the generating function R235 (s) of the renewal sequence {rn } n=0 , and use the renewal relation to obtain the probability generating function F235 (s) of the rst waiting time T235 . Evaluate F235 (s) at s = 1 to prove recurrence. (c) Use F235 (s) to obtain E (T235 ). Is 235 positive recurrent? Find the probability f7 that the rst time 235 occurs is on trial 7. 3. In each of the following cases nd the expected waiting time until the rst occurrence of the event. (Hint: use the Renewal Theorem). (a) in a sequence of coin tosses with probability p of H : TTHHTTTH. (b) in a sequence of coin tosses with probability p of H : TTTHHTTT. (c) a sequence of fair die rolls: 262333262. 4. A fair die is rolled repeatedly, producing the sequence X1 , X2 , X3 , . . . where Xn is the outcome on the nth roll. Consider the event = 12121 where we say that occurs on the nth roll if Xn4 = 1, Xn3 = 2, Xn2 = 1, Xn1 = 2 and Xn = 1. This is a delayed renewal event. (Why?)

(a) Find the delayed renewal sequence dn , the associated renewal sequence r n , and use the delayed renewal relation to obtain the probability generating function F (s) of the rst waiting time T . Use this to show that the probability that occurs at least once in the sequence is 1. Use any valid method to determine E (T ). (b) Use the renewal relation on the associated renewal sequence to nd the probability generating function F . Use any valid method to (s) of the between waiting times T determine E (T ). Why is this average dierent from the one obtained in (a)? (c) Use the results of (a) and (b) to prove that 12121 will occur innitely often in the sequence. Find the average number of rolls required to see 5 occurrences of 12121. 5. Suppose we have a random walk with a reecting barrier at 0. That is, the walk takes values only in the nonnegative integers S + = {0, 1, 2, 3, ...} and moves in the following way: if Xn = 0 then at the next step it jumps to state 2. Otherwise it moves like an ordinary random walk, that is, if Xn = j for j = 0 then it jumps to the right with probability p and to the left with probability q = 1 p. (a) For what value of p, 00 =Start at 0, return to 0 is positive recurrent, null recurrent, or transient? (b) Let T00 be the waiting time for the rst 00 . Find E (T00 ). (c) Let V00 be the number of returns to 0. Find E (V00 ). 6. Mary is gambling. She starts out with $1500, and on each game she either wins or loses $100. (Assume that each game is an independent trial.) Identify 1 unit = $100, and let X0 = 15 and let Xn be her total worth (divided by 100) after n games. In this way she goes bankrupt when she hits 0. (Alternatively, you can dene Xn to be her net worth (/100) after n games so that X0 = 0. In this case bankruptcy occurs at -15.) (a) If Marys chance of winning on any game is 0.4 and she continues to play as long as she has money, nd the average number of games required for her to go bankrupt. (b) Find the probability that Marys total worth reaches $3000 at some point. (We assume here that Mary has to quit playing if she goes bankrupt so this is a gamblers ruin problem.) (c) Now suppose Mary has an indulgent rich aunt who constantly resupplies Mary with $1500 whenever she happens to go bankrupt. (Therefore if Marys total worth hits 0, it instantaneously bounces back up to $1500.) Now what is the probability that Marys total worth reaches $3000 at some point? (d) Suppose there is no rich aunt but we enable Mary to play indenitely by convincing the casino to allow her to go arbitrarily deep into debt. (Could this really happen in practice? I think they throw you out on the sidewalk and break your kneecaps once you owe them too much money.) Now what is the probability that Marys total worth ever reaches $3000? Why are your answers in (b), (c), and (d) all dierent?