Probability and measures

Key Objectives:

ˆ Probability and measures

Notation:

ˆ Ω : sample space

ˆ A: sigma algebra, where A ⊂ P (Ω)

ˆ P: probability measure, where P : A −→ R

ˆ (Ω, A, P): Probability Space

ˆ B(R): Borel sigma algebra of the set of real numbers

Note. In a probability space (Ω, A, P)

1. P(Ω) = 1

2. Elements A ∈ A are called events

3. P(A) ∈ [0, 1] for all A ∈ A

!
∞
S ∞
P
4. P Aj = Aj for pairwise disjoint sets Ai , Aj where Ai ∩ Aj = ∅
j=1 j=1

1
Proposition 1. Let (Ω, A, P) be a probability space and B ∈ A where P(B) ̸= 0.
P(A ∩ B)
Let PB : A −→ R be the real-valued function defined as PB (A) := .
P(B)
Then (Ω, A, PB ) is a probability space.
Definition 1. Given a probability space (Ω, A, P), B ∈ A where P(B) ̸= 0.
The conditional probability of A given B is defined to be
P(A ∩ B)
P(A | B) :=
P(B)
P( ·|B) : A −→ [0, 1] is called the conditional probability measure given B. P(B|B) = 1 by definition.
Remark. For P(B) = 0, should we set P(A | B) = 0 ?
Definition 2. Let (Ω, A) and (Ω̃, Ã) be measurable spaces.
A map X : Ω −→ Ω̃ is called a random variable if
X −1 (Ã) ∈ A for all à ∈ Ã
.
Notation: Let (Ω, A) and (Ω̃, Ã) be measurable spaces, P : A −→ [0, 1] probability measure and
X : Ω −→ Ω̃ a random variable.
Define P(X ∈ Ã) := P(X −1 (Ã)) = P({ω ∈ Ω : X(ω) ∈ Ã}.
Definition 3. Let (Ω, A, P) be a probability space, (R, B(R)) be a measurable space and X : Ω −→ R
be a random variable.
Then PX : B(R) −→ [0, 1] defined by
PX (B) := P(X −1 (B)) = P(X ∈ B)
is called a probability distribution of X.
Proposition 2. PX is a probability measure of (R, B(R)).
Notation: Write X ∼ P̃ if P̃ is a probability measure and PX = P̃.
Example 1. Consider a Bernoulli process which is a sequence of independent identically distributed
Bernoulli trials.
n tosses of the same coin, probability of head is p
Consider (Ω, A, P), where Ω = {0, 1}n , A = P(Ω) and P is a Bernoulli measure. If k:= num of 1’s, then
P({ω}) = pnum of 1’s · (1 − p)num of 0’s = pk · (1 − p)n−k
Let X : Ω −→ R be a random variable, X(ω) := num. of 1’s in ω.
Then X ∼ Bin(n, p)
Compute PX :
PX ({k}) = P(X −1 ({k}))
= P({ω ∈ Ω : X(ω) = k})
 
n
= · pk · (1 − p)n−k
k

n


So P(X = k) = P(X ∈ {k}) = PX ({k}) = k
pk (1−p)n−k , which is the same as a binomial distribution.
Hence, X ∼ Bin(n, p).

2
Remark. PX : B(R) −→ [0, 1]
n
n

k
p (1 − p)n−k δk , where
P
PX = k
k=0

1 if k = j
δk ({j}) =
0 else
For all B ∈ B(R), 
1 if k ∈ B
δk (B) =
0 else
Therefore,
n  
X n
PX (B) = pk (1 − p)n−k δk (B)
k=0
k
 
X n k
= p (1 − p)n−k
k
k∈B∩{0,··· ,n}

Also,
n  
X n
PX (R) = pk (1 − p)n−k δk (R)
k=0
k
n  
X n k
= p (1 − p)n−k
k=0
k
= (p + (1 − p))n
=1
Definition 4. Let (Ω, A, P) be a probability space. Consider (R, B(R), PX ), and let X : Ω −→ R be a
random variable.
Define FX : R −→ [0, 1] to be:
FX (x) := PX ((−∞, x]) = P(X ≤ x)
FX is called the cumulative distribution function of X.

Properties:
ˆ lim FX (x) = 0 and lim FX (x) = 1
x→−∞ x→∞

ˆ FX is monotonically increasing.

ˆ FX is right continuous, lim+ FX (x) = FX (x0 )

x→x0

Definition 5. Consider (Ω, A, P) , (R, B(R), PX ), and X : Ω −→ R random variable.

Define the expectation of X: Z
E(X) := X dP
Ω

3
Notation: Denote fX to be the probability density function of PX . If such a function exists, then
Z
PX (A) = fX (x) dx
A

Example 2. X ∼ Exp(λ) exponential distribution

 −λx
λe x>0
fX (x) =
0 x≤0
Z Z Z ∞
1
E(X) := X dP = xfX (x) dx = xλe−λx dx =
Ω R 0 λ
Definition 6. Consider (Ω, A, P) and X : Ω −→ R random variable.
Z
E(X) := X dP
Ω
Change of variables:
X
A⊂Ω R
g
g ◦ X=g(X)
R

Then g(X) is a new random variables.

Z Z
g(X) dP = g(X(ω)) dP(ω)
A ZA

= g(x) d(P ◦ X −1 )(x) (x := X(ω), X −1 (x) = ω)

X(A)
Z
= g(x) dPX (x) (PX = P ◦ X −1 )
X(A)
 R
 X(A) g(x)fX (x) dx continuous case
= P
g(x) · px discrete case

x∈X(A)

where fX is pdf of PX , px is pmf of PX and px = PX ({x})

 R
Z 
 xfX (x) dx continuous case
X(Ω)
E(X) := X dP = P
Ω 
 x · px discrete case
x∈X(Ω)

Properties: Consider (Ω, A, P) and X, Y : Ω −→ R random variables, where E(X) and E(Y ) exist.
(a) E(aX + bY ) = aE(X) + bE(Y )
(b) If X, Y are independent, then E(XY ) = E(X)E(Y )
(c) If PX = PY , then E(X) = E(Y )
(d) If X ≤ Y almost surely, that is P({ω ∈ Ω : X(ω) ≤ Y (ω)}) = 1, then E(X) ≤ E(Y )

4
Bibliography

[1] Rick Durrett, Probability: theory and examples. Cambridge University Press, Fourth edition, 2010.

[2] Kai Lai Chung, A course in probability theory. Third edition, 1968

[3] David Williams, Probability with Martingales. Cambridge University Press, 1991.

[4] Patrick Billingsley, Probability and measure. Second edition. John Wiley & Sons, Inc., New York,
1986.