Professional Documents
Culture Documents
Week01 Workshop Soln
Week01 Workshop Soln
Week01 Workshop Soln
Workshop aims
• Probability revision
1 Random variables
1.1 Let X : Ω → R and Y : Ω → R be two random variables with probability density functions fX : R → R and
fY : R → R respectively, and with joint probability density function fX,Y : R2 → R. Show that, for any α, β ∈ R,
we have
E (αX + βY ) = αE (X) + βE (Y ) .
Now we use1
Z ∞
fX,Y (x, y) dy = fX (x) ,
Z −∞
∞
fX,Y (x, y) dx = fY (y) ,
−∞
leading to
Z ∞ Z ∞
E (αX + βY ) = α xfX (x) dx + β yfY (y) dy
−∞ −∞
= αE (X) + βE (Y ) .
V (X + Y ) = V (X) + V (Y ) .
1
To see these identities, consider e.g. any expectation E (g (X)), taken over both X and Y . This allows us to identify the PDF for
X, as used here.
1
Solution We have
h i
V (X + Y ) = E (X + Y − E (X + Y ))2
h i
= E (X + Y − E (X) − E (Y ))2
h i
= E (X − E (X))2 + (Y − E (Y ))2 + 2 (X − E (X)) (Y − E (Y ))
h i h i
= E (X − E (X))2 + E (Y − E (Y ))2 + 2E [XY − XE (Y ) − Y E (X) + E (X) E (Y )]
= V (X) + V (Y ) + 2E (XY ) − 2E (X) E (Y )
= V (X) + V (Y ) + 2E (X) E (Y ) − 2E (X) E (Y )
= V (X) + V (Y ) .
Here E (XY ) = E (X) E (Y ) has been used, which follows as X and Y are independent.
2 Gaussian distribution
2.1 Consider a Gaussian random variable X : Ω → R with X ∼ N µ, σ 2 , where σ > 0. Show, using the
Solution First
Z ∞
E (X − µ) = (x − µ) fX (x) dx
−∞
Z ∞
1 (x−µ)2
=√ (x − µ) e− 2σ2 dx.
2πσ 2 −∞
With the substitution
x−µ
y=
,
σ
we obtain Z ∞
σ 1 2
E (X − µ) = √ ye− 2 y dy.
2π −∞
The integral is zero as it is the integral of the product of odd and even functions, specifically
Z ∞ Z 0 Z ∞
− 12 y 2 − 12 y 2 1 2
ye dy = ye dy + ye− 2 y dy
−∞ −∞ 0
Z 0 1 2
Z 0 1 2
= ye− 2 y dy − (−y) e− 2 (−y) d (−y)
−∞ −∞
= 0.
Hence
E (X − µ) = 0,
2
leading to
E (X) = µ.
Next
h i
V (X) = E (X − µ)2
Z ∞
1 (x−µ)2
=√ (x − µ)2 e− 2σ2 dx.
2πσ 2 −∞
With the substitution
x−µ
y= ,
σ
we obtain
Z ∞
σ2 1 2
V (X) = √ y 2 e− 2 y dy
2π −∞
Z ∞
σ2 1 2
= −√ y −ye− 2 y dy
2π −∞
Z ∞
σ 2 h − 1 y2 i∞ σ2 1 2
= −√ ye 2 +√ e− 2 y dy,
2π −∞ 2π −∞
where the final equality follows by integrating by parts. Using the given standard integral leads to
V (X) = σ 2 .
which displays
E_X = mu
V_X = sigma **2
Solution
Z ∞
1 (x−µ)2
E eaX = √ eax e− 2σ2 dx
2πσ 2 −∞
Z ∞
1 −2axσ 2 +x2 −2µx+µ2
=√ e− 2σ 2 dx
2πσ 2 −∞
2
1
Z ∞
−
(x−µ−aσ2 ) −2aµσ2 −a2 σ4
=√ e 2σ 2 dx
2πσ 2 −∞
2
1
Z ∞ (x−µ−aσ2 )
aµ+ 21 a2 σ 2 −
=e √ e 2σ 2 dx
2πσ 2 −∞
1 2 2
= eaµ+ 2 a σ
.
3
2.3 Let g : R → R be a smooth bounded function. Show that
Solution
Z ∞ (x−µ)2
1
E (Xg (X)) = √ xg (x) e− 2σ2 dx
2πσ 2 −∞
Z ∞ 2
Z ∞
(x−µ)2
1 x − µ − (x−µ) 1
= −σ 2
√ g (x) − 2 e 2σ 2 dx + µ √ g (x) e− 2σ2 dx
2πσ 2 −∞ σ 2πσ 2 −∞
2
∞ Z ∞ Z ∞
(x−µ)2 (x−µ)2
1 −
(x−µ) 1 − 1
2
= −σ √ g (x) e 2σ 2 +σ √2 0
g (x) e 2σ dx + µ √
2 g (x) e− 2σ2 dx
2πσ 2 −∞ 2πσ 2 −∞ 2πσ 2 −∞
2
0
= σ E g (X) + µE (g (X)) .
We have
E [(X − E (X)) (Y − E (Y ))] = Sxy .
For completeness this is derived from the probability density function. First2
Z ∞Z ∞
1 1 T −1
E [(X − E (X)) (Y − E (Y ))] = q (x − mx ) (y − my ) e− 2 (q−m) S (q−m) dxdy.
(2π)2 det S −∞ −∞
Define
1 Sxx √ 0
L= √ ,
Sxx Sxy det S
noting that Sxx 6= 0 follows from S being symmetric positive definite. After some algebra we have
LLT = S,
2
E (X) = µx and E (Y ) = µy can also be derived from the PDF – e.g. via an approach similar to that used in question 2.1.
4
√
from which it follows that det L = det S.3 With the substitution4
x̃ −1 x
=L −m ,
ỹ y
1 1 T −1
fQ (q) = q e− 2 (q−m) S (q−m)
(2π)2 det S
2 2
1 (x−mx ) (y−my )
− − 2S
=q e 2Sxx yy
2
(2π) Sxx Syy
!
(x−mx )2 (y−my )2
1 1 −
= √ e− 2Sxx p e 2Syy
2πSxx 2πSyy
= fX (x) fY (y) ,
where fX and fY are the probability density functions for X and Y respectively.5 Hence X and Y are independent.
3
Technical note: L is the Cholesky factor of the covariance matrix S.
4
This is a higher dimensional generalization of the substitution used in the solution to question 2.1.
5
To see that these are the PDFs for X and Y , consider the expectations of functions depending only on X or Y , E (g (X)) and
E (g (Y )).