MA5260: Probability Theory II

Homework 1

Spring 2023

Exercise 1

(i)

Let M1, M2, ..., Mn be a family of monotone classes, and let A, A1, A2,

... be subsets of Ω such that Ai ∈ Mi for all i and either Ai ↑ A or Ai ↓ A.

Then we have:

For any i and j, Ai ∩ Aj ∈ Mi (since Mi is a monotone class),

and therefore, their intersection is in the intersection of all the Mi's:

A1 ∩ A2 ∩ ... ∩ An is a monotone class.

Let B1 ⊂ B2 ⊂ B3 ⊂ ... be a sequence of subsets in A1 ∩ A2 ∩ ... ∩

An that increases to B, then for any i, Bi is in each Mi, since each Mi is a monotone class.

Therefore, their limit B is also in the intersection of all the Mi's, and thus, A1 ∩ A2 ∩ ... ∩ An is
closed under monotone limits.

Now, let C be a class of subsets of Ω, and let M(C) be the smallest monotone class containing C.
We can construct M(C) as the intersection of all monotone classes containing C, which is itself a
monotone class by the above argument.

This is because the family of all monotone classes containing C is non-empty (since the power
set of Ω is a monotone class containing C), and it is closed under the intersection (as shown
above). Therefore, M(C) is well-defined and is the smallest monotone class containing C.

(ii)
To show that any field F which is also a monotone class is a σ-field, we need to show that F is
closed under countable unions and complements.

First, we will show that F is closed under countable unions. Let {A_n} be a sequence of sets in
F.

Define B_1 = A_1 and B_n = A_n \ (A_1 ∪ A_2 ∪ ... ∪ A_{n-1}) for n > 1.

Then, {B_n} is a sequence of disjoint sets in F, and we have A_1 ∪ A_2 ∪ ... = B_1 ∪ B_2 ∪ ....

Moreover, {B_n} is a monotone increasing sequence of sets, since B_n ⊂ B_{n+1} for all n.

Thus, by the monotone class theorem,

we have A_1 ∪ A_2 ∪ ... ∈ F, which shows that F is closed under countable unions.

Next, we will show that F is closed under complements.

Let A be a set in F.

Since F is a field, we have that Ω \ A is also in F.

Now, consider the sequence {A_n} = {A, Ω \ A, A, Ω \ A, ...}.

This sequence is clearly monotone,

since A ⊂ Ω \ A and Ω \ A ⊂ A.

Moreover, the limit of this sequence is Ω, which is in F.

Therefore, by the monotone class theorem, we have A^c = Ω \ A ∈ F, which shows that F is
closed under complements.

Since F is a field and is closed under countable unions and complements, it is a σ-field.

(iii)

We will prove each part separately:

(i) Let F be a field and let A ∈ F. We want to show that M1 := {B ∈ M(F) : A ∪ B ∈ M(F)} is a
monotone class that contains F and hence is equal to M(F).

First, we show that F ⊂ M1. Let B ∈ F, then A ∪ B ∈ F since F is a field, so B ∈ M1. Thus, F ⊂
M1.

Next, we show that M1 is a monotone class.

Let {B_n} be a sequence of sets in M1 and let B = ∪n B_n.

We want to show that B ∈ M1.

Since {B_n} is a monotone increasing sequence of sets,

we have A ∪ B_n ⊂ A ∪ B_{n+1} for all n.

Therefore, A ∪ B = ∪n (A ∪ B_n) is a monotone limit of sets in F.

By the monotone class theorem, we have that B ∈ M1,

so M1 is closed under monotone limits.

(ii) Let F be a field and let B ∈ M(F). We want to show that M2 := {A ∈ M(F) : A ∪ B ∈ M(F)}
is a monotone class that contains F and hence is equal to M(F).

First, we show that F ⊂ M2.

Let A ∈ F, then A ∪ B ∈ F since F is a field,

so A ∈ M2. Thus, F ⊂ M2.

Next, we show that M2 is a monotone class.

Let {A_n} be a sequence of sets in M2 and let A = ∩n A_n.

We want to show that A ∈ M2.

Since {A_n} is a monotone decreasing sequence of sets,

 we have A_n ∪ B ⊃ A_{n+1} ∪ B for all n.

Therefore, A ∪ B = ∩n (A_n ∪ B) is a monotone limit of sets in F.

By the monotone class theorem, we have that A ∈ M2, so M2 is closed under monotone limits.

(iii) Let F be a field. We want to show that M3 := {A ∈ M(F) : A^c ∈ M(F)} is a monotone class
that contains F and hence is equal to M(F).

First, we show that F ⊂ M3.

Let A ∈ F, then A^c ∈ F since F is a field,

so A ∈ M3. Thus, F ⊂ M3.

Next, we show that M3 is a monotone class.

Let {A_n} be a sequence of sets in M3 and let A = ∪n A_n.

We want to show that A ∈ M3.

Since {A_n} is a monotone increasing sequence of sets,

we have A_n^c ⊃ A_{n+1}^c for all n.

Therefore, A^c = (∩n A_n^c)^c is a monotone limit of sets in F.

By the monotone class theorem, we have that A^c ∈ M(F), so A ∈ M3.

Thus, M3 is closed under monotone limits.

Therefore, since F ⊂ M3 and M3 is a monotone class containing F,

we have that M3 = M(F), which shows that for all A, B ∈ M(F), A ∪ B ∈ M(F).

(iv)

From part (ii) we know that any field that is also a monotone class is a σ-field.

We have shown in part (iii) that $M(F)$ is a monotone class.

 Therefore, to show that $M(F)$ is a σ-field,

we only need to show that $M(F)$ is a field.

To show that $M(F)$ is a field,

we need to check that it satisfies the three axioms of a field:

$F \subseteq M(F)$:

This is true by definition.

$A \in M(F) \Rightarrow A^c \in M(F)$:

This is true by definition of $M_3$ in part (iii).

$A, B \in M(F) \Rightarrow A \cup B \in M(F)$:

This is true by definition of $M_1$ and $M_2$ in part (iii).

Therefore, we have shown that $M(F)$ is a field and a monotone class,

and hence by (ii) it is a σ-field.

(v)

Let $M$ be a monotone class that contains a field $F$.

By (i), we know that there is a smallest monotone class containing $F$, denoted by $M(F)$.

We claim that $M(F) \subseteq M$.

First, note that $F \subseteq M(F)$.

To see this, note that $M(F)$ is a monotone class containing $F$,

so by the definition of $M(F)$ as the smallest monotone class containing $F$,

we have $F \subseteq M(F)$.

Next, we show that $M(F)$ is a monotone class.

 To see this, let $(A_n){n=1}^\infty$ be a sequence of sets in $M(F)$

such that $A_n \subseteq A{n+1}$ for all $n$,

and let $A = \bigcup_{n=1}^\infty A_n$.

We want to show that $A \in M(F)$.

Note that $A_n \subseteq A$ for all $n$, so $A_n \in M(F)$ for all $n$.

Since $M(F)$ is a monotone class,

we have that $A \in M(F)$ if $(A_n){n=1}^\infty$ is an increasing sequence,

or $A \in M(F)$ if $(A_n){n=1}^\infty$ is a decreasing sequence.

If $(A_n){n=1}^\infty$ is an increasing sequence,

then $A_n \subseteq A{n+1}$ for all $n$,

so we have $A = \bigcup_{n=1}^\infty A_n \in M(F)$ by the definition of $M(F)$ as the

smallest monotone class containing $F$.

If $(A_n){n=1}^\infty$ is a decreasing sequence,

then $A_n \supseteq A{n+1}$ for all $n$,

so $A = \bigcap_{n=1}^\infty A_n$ and $A^c = \bigcup_{n=1}^\infty (A_n^c)$,

where $(A_n^c)$ is an increasing sequence in $M(F)$.

By the definition of $M(F)$ as the smallest monotone class containing $F$,

we have $A^c \in M(F)$, so $A \in M(F)$.

Therefore, we have shown that $M(F)$ is a monotone class containing $F$, and $M(F) \subseteq
M$.

By the definition of $M(F)$ as the smallest monotone class containing $F$,

we have $M(F) \subseteq M(F')$ for any monotone class $M(F')$ containing $F$.

In particular, we have $M(F) \subseteq \sigma(F)$,

where $\sigma(F)$ is the smallest $\sigma$-field containing $F$.

This completes the proof of the Monotone Class Theorem.

(vi)

Consider the set of all subsets of the interval [0, 1] that are either countable or have a countable
complement.

Let M be the monotone class generated by this set.

M is a monotone class since it is clearly closed under taking countable unions and intersections.

However, M is not a σ-field because it does not contain all subsets of [0, 1].

For example, the set of irrational numbers in [0, 1] is not in M,

since its complement is the countable set of rational numbers, which is in M.

Therefore, M is a monotone class that is not a σ-field.

(vii)

No, not every λ-system is a monotone class.

 A λ-system is closed under set differences,

whereas a monotone class is only required to be closed under monotone limits (either increasing
or decreasing sequences).

consider the set Ω = {1, 2, 3} and

the λ-system L = {{}, {1, 2}, {3}, {1, 2, 3}}.

L is a λ-system since it is closed under complements and set differences,

but it is not a monotone class since it is not closed under increasing limits (for example, the
sequence {1, 2} ⊂ {1, 2, 3} ⊂ {1, 2, 3} ⊂ ...)

or decreasing limits (for example, the sequence {1, 2, 3} ⊃ {1, 2, 3} ⊃ {1, 2} ⊃ ...).

Exercise 2

We will use the Strong Law of Large Numbers (SLLN) to prove that

log ∆r / r → 1 as r → ∞ almost surely.

Let Yr = log(∆r)/r for each r.

Then, we need to show that Yr → 1 almost surely.

First, note that since the Xi are i.i.d. with a continuous distribution,

the probability that Xn = X1 is 0 for all n > 1. Therefore, V1 = 1 almost surely.

Next, we claim that Vr has a geometric distribution with parameter p = 1/r.

To see this, note that for n > Vr-1, we have

P(Vr = n) = P(Vr > n-1) - P(Vr > n) = (1 - P(Xn ≤ Xk for all k ≤ Vr-1)) - (1 - P(Xn ≤ Xk for all k
≤ Vr)).

Now, the events {Xn ≤ Xk for all k ≤ Vr-1} and {Xn ≤ Xk for all k ≤ Vr} are independent,

and each has probability (Vr-1)/n and Vr/n, respectively.

Therefore,

P(Vr = n) = (1 - (Vr-1)/n) - (1 - Vr/n) = Vr/n - (Vr-1)/n = 1/n.

This is precisely the probability mass function of a geometric distribution with parameter p = 1/r.

Using this observation,

∆r = Vr - Vr-1 = Vr - Vr-1 - 1 + 1 = W1 + W2 + ... + Wr,

where Wi = Vi - Vi-1 - 1 is the waiting time for the i-th success in a Bernoulli trial with success
probability p = 1/r.

By the SLLN,

(W1 + W2 + ... + Wr)/r → 1/p = r almost surely.

Therefore,

log ∆r = log(W1 + W2 + ... + Wr) ≤ log(rW1) = log(r(V1 - 1)).

Dividing by r and taking limits, we get

log ∆r / r ≤ log(r(V1 - 1)) / r → log(1) = 0 almost surely.

Therefore,

log ∆r / r → 0 almost surely.

To complete the proof,

log(∆r/r) - log(1+∆r/r) → 0 almost surely.

This follows from the fact that log(1+x) ≤ x for all x > 0.

Therefore,
 0 ≤ log(∆r/r) - log(1+∆r/r) ≤ ∆r/r,

which implies that

log(∆r/r) - log(1+∆r/r) → 0 almost surely.

Putting everything together,

log ∆r / r = (log(∆r/r) - log(1+∆r/r)) + log(1+∆r/r) → 1 almost surely,

which completes the proof.

Exercise 3

(1)

The statement is false. Here is a counterexample:

Let X be a random variable taking values in [0, 1] with the uniform distribution on [0, 1].

Define measures μ and ν on Borel sets of [0, 1] by

μ(A) = ∫_A x dx,

ν(A) = ∫_A x^(−1/2) dx.

It is easy to see that both measures are finite, and ν is absolutely continuous with respect to μ.

Moreover,

dν/dμ(x) = x^(−1/2)

for μ-almost every x.

This function is unbounded near x = 0,

so dν/dμ is not bounded.

Therefore, the statement is false.

(2)

The statement is false. Here is a counterexample:

Let Ω = {0, 1}, and let F be the power set of Ω.

Define probability measures P1 and P2 on (Ω, F) by

P1({0}) = 1, P1({1}) = 0,

P2({0}) = 0, P2({1}) = 1.

It is easy to see that both measures are probability measures.

Now suppose there is a probability measure Q such that P1 ≪ Q and P2 ≪ Q.

Since Q({0, 1}) = 1 and P1({1}) = P2({0}) = 0,

we must have Q({1}) = Q({0}) = 1/2.

But then,

0 = P1({1}) < Q({1}) = 1/2 < P2({0}) = 1,

which is a contradiction.

Therefore, the statement is false.

(3)

The statement is true.

Since μ1 ≪ μ2, there exists a function f such that dμ1 = f dμ2.

Since μ2 ⊥ ν,

∫_A f dμ2 = ∫_A dμ1 = μ1(A) = 0

for all A in F with ν(A) = 0.

Therefore,
 μ1 is absolutely continuous with respect to ν, and we write μ1 ≪ ν.

To show that μ1 is also singular with respect to ν,

we need to show that there exists a set B in F such that B is ν-null and μ1(B^c) = 0.

Since μ1 is absolutely continuous with respect to μ2,

there exists a set A in F such that A is μ2-null and μ1(A^c) = 0.

Since μ2 ⊥ ν,

μ1(A^c) = ∫_A^c f dμ2 = ∫_Ω f 1_A dμ2 = 0,

where 1_A is the indicator function of A.

It follows that μ2(A^c) = 0, and therefore A^c is ν-null.

Since A is μ2-null, we have A ⊆ B, where B is the set of all points in A^c that are ν-null.

Thus, B is ν-null and μ1(B^c) = μ1(A) = 0.

Therefore, μ1 is singular with respect to ν, and we write μ1 ⊥ ν.

Hence, μ1 ⊥ ν and μ1 ≪ μ2 implies μ2 ⊥ ν by the Radon-Nikodym theorem.

Exercise 4

Let us denote the countable collection of measurable sets by ${C_i}_{i\in\mathbb{N}}$.

For every $n\in\mathbb{N}$, let $f_n:\Omega\to\mathbb{R}$ be the characteristic function of

$C_n$, i.e., $f_n(\omega)=1$ if $\omega\in C_n$, and $f_n(\omega)=0$ otherwise. Define $f:\
Omega\to\mathbb{R}$ as follows:

f(ω)=n=1∑∞2n1⋅fn(ω).
 Note that $f$ is well-defined since it converges absolutely and uniformly. Also, $f$ is
measurable since it is a countable sum of measurable functions.

Now, define the function $X:\Omega\to\mathbb{R}$ by

X(ω)=n=1∑∞2n1⋅an,

where $a_n$ is any element of $[n-1,n]$ such that $f_n(\omega)=1$ (such an $a_n$ exists since
$C_n\in F$).

It is easy to see that $X$ is a measurable function. Moreover, we claim that $\sigma(X)=\
sigma(C)$.

First, let us show that $\sigma(C)\subseteq\sigma(X)$. Since $C$ is a collection of measurable

sets, $\sigma(C)\subseteq F$. Thus, it suffices to show that $C_n\in\sigma(X)$ for all $n\in\
mathbb{N}$. Let $E$ be any Borel set in $\mathbb{R}$. Then

\begin{align*}

X^{-1}(E)&={\omega\in\Omega:X(\omega)\in E}\

&=\bigcup_{n=1}^{\infty}\left{\omega\in\Omega:\frac{1}{2^n}\cdot a_n\in E\text{ and } f_n(\

omega)=1\right}\

&=\bigcup_{n=1}^{\infty}\left{\omega\in C_n:a_n\in 2^nE\right},

\end{align*}

which is a countable union of sets in $\sigma(C)$. Thus, $C_n\in\sigma(X)$ for all $n\in\
mathbb{N}$.

Next, we show that $\sigma(X)\subseteq\sigma(C)$. Since ${C_n}{n\in\mathbb{N}}$ is a

countable collection, we can write $\sigma(C)$ as a countable union of $\sigma$-algebras
generated by finite sub collections of $C$, i.e., $\sigma(C)=\bigcup{k=1}^{\infty}\
sigma(C_k^k)$, where $C_k^k={C_{n_1},\ldots,C_{n_k}}$ for some distinct $n_1,\ldots,n_k\
in\mathbb{N}$. It thus suffices to show that $X^{-1} (B)\in\sigma(C_k^k)$ for every Borel set
$B\subseteq\mathbb{R}$ and every $k\in\mathbb{N}$.

Fix a Borel set $B\subseteq\mathbb{R}$ and $k\in\mathbb{N}$.

We claim that $X^ {-1

Exercise 5

Since ν ≪ μ, there exists a measurable function f such that dν/dμ = f, where f is finite a.e.

with respect to μ. Also, since dν/dμ > 0 almost surely with respect to μ, it follows that f > 0
almost surely with respect to μ.

Suppose there exists a set A ∈ F such that μ(A) = 0 and ν(A^c) > 0.

Then, since ν ≪ μ, we have that μ(A^c) = 0.

Thus, we have μ(A) + μ(A^c) = 0, which is a contradiction.

Therefore, we must have μ ≪ ν.

Let g = 1/f. Then, g is finite a.e.

with respect to μ and g > 0 almost surely with respect to ν.

Note that dμ/dν = g, and we want to show that dν/dμ = 1/g almost surely with respect to both μ
and ν.

Let A be a measurable set such that ν(A) = 0.

Then, since ν ≪ μ, we have μ(A) = 0.

Thus,

∫_A (1/g) dμ = 0
 since g > 0 almost surely with respect to μ.

This implies that dν/dμ = 1/g almost surely with respect to μ.

Now, let B be a measurable set such that μ(B) = 0.

Then, since μ ≪ ν, we have ν(B) = 0. Thus, we have:

∫_B g dν = 0

since g > 0 almost surely with respect to ν.

This implies that dν/dμ = 1/g almost surely with respect to ν.

Therefore, we have shown that if dν/dμ > 0

almost surely with respect to μ, then μ ≪ ν and dν/dμ = (dν/dμ)^(-1)

almost surely with respect to both μ and ν.

Exercise 6

Let $X_n$ and $Y_n$ be two independent fair coin tosses,

i.e., taking values $+1$ and $-1$ each with probability $\frac{1}{2}$.

Let $F_0$ be the trivial $\sigma$-algebra ${\emptyset,

\Omega}$ and $F_1$ be the $\sigma$-algebra generated by the first coin toss,

i.e., $F_1={\emptyset, {-1}, {+1}, \Omega}$.

Define $X_0 = Y_0 = 0$, $X_1 = X_0+X_1$ and $Y_1 = Y_0+Y_1$.

Let $Z_n = X_n + Y_n$, and $G_0$ be the trivial $\sigma$-algebra ${\emptyset,

\Omega}$ and $G_1$ be the $\sigma$-algebra generated by both the coin tosses,

i.e., $G_1={\emptyset, {-1,-1}, {-1,+1}, {+1,-1}, {+1,+1}, \Omega}$.

Note that $X_n$ and $Y_n$ are martingales with respect to the filtration $(F_n)$,
 but $Z_n$ is not even a sub or super martingale with respect to any filtration.

This can be seen by observing that

$P(Z_2=2) = P(X_2=Y_2=1) = \frac{1}{4} $,

$P(Z_2=0) = P(X_2=1, Y_2=-1)+P(X_2=-1, Y_2=1) = \frac{1}{2}$, and

$P(Z_2=-2) = P(X_2=Y_2=-1) = \frac{1}{4}$.

Hence, $E[Z_2|G_1] = 0$ and $E[Z_2|G_1] \neq Z_1$,

which implies that $Z_n$ is not even a sub or super martingale with respect to any filtration.

Exercise 7

(i)

To show that $Y_n$ is a martingale, we need to verify that $\mathbb{E}[Y_n | \mathcal{F}{n-

1}] = Y{n-1}$ for all $n \geq 2$, where $\mathcal{F}{n-1} = \sigma(X_1,\ldots,X{n-1})$ is the
natural filtration.

We have:

\begin{align*}

\mathbb{E}[Y_n | \mathcal{F}{n-1}] &= \mathbb{E}\left[ \frac{X_n}{Y{n-1}} \cdot Y_{n-

1} \bigg| \mathcal{F}{n-1} \right] \

&= X_n \cdot \mathbb{E}\left[ \frac{1}{Y{n-1}} \bigg| \mathcal{F}{n-1} \right] \

&= X_n \cdot \frac{1}{Y{n-1}} \cdot \mathbb{E}[Y_{n-1} | \mathcal{F}{n-1}] \

&= X_n \cdot \frac{1}{Y{n-1}} \cdot Y_{n-1} \

&= X_n
 \end{align*}

where the second last equality follows from the martingale property of $Y_{n-1}$.

Therefore, $Y_n$ is a martingale.

(ii)

If $\sum_{n=1}^{\infty} a_n < \infty$ and $b_n \geq 0$ for all $n$,

then $\lim_{n\to \infty} b_n = 0$ almost surely, where $b_n = \prod_{m=1}^n (1 + a_m)$.

For each $n \geq 1$, define $a_n = \ln(X_n) - E(\ln(X_1))$.

Then $\sum_{n=1}^{\infty} a_n < \infty$

since $E(\ln(X_1)) > \ln(P(X_1 = 1)) > -\infty$ and $X_n$ is non-negative.

Let $b_n = Y_n$ for $n \geq 1$.

Then $b_n \geq 0$ for all $n$, and we can write

\begin{align*}

b_n &= \prod_{m=1}^n X_m \

&= \exp\left(\sum_{m=1}^n \ln(X_m)\right) \

&= \exp\left(\sum_{m=1}^n a_m + nE(\ln(X_1))\right) \

&= \exp\left(\sum_{m=1}^n a_m\right) \exp(nE(\ln(X_1)))

\end{align*}

Since $\sum_{n=1}^{\infty} a_n < \infty$,

it follows that $\lim_{n\to \infty} \sum_{m=1}^n a_m = 0$ almost surely.

Therefore, $\lim_{n\to \infty} b_n = \lim_{n\to \infty} \exp(nE(\ln(X_1))) = 0$ almost surely, as

$E(\ln(X_1)) < 0$.
Thus, $Y_n \to 0$ almost surely.