Chapter 3: Expectations

3.1 Introduction
Traditionally -> get involved in betting and
other games of chance with monetary reward
Thus, mathematical expectation is AVERAGE
AMOUNT you can get when participating in such
games
Example: Roll a die and receive the money
according to the number that shows up
1 => $1; 2 => $2; 6 => $6
What is the average amount one can get if this game is
repeated in a long run?
1 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Example: Roll a die and receive the money
according to the number(continued)
Suppose the die is fair, then the law of large number
suggests that the six sides will turn up about equally
often.
Thus, mathematical expectation is just:
If you play this game long enough, you expect to get
$3.50. Why this information is important in this game of
chance?
2 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
50 . 3 $
6
1
6 $
6
1
5 $
6
1
4 $
6
1
3 $
6
1
2 $
6
1
1 $ =
|
.
|

\
|
+
|
.
|

\
|
+
|
.
|

\
|
+
|
.
|

\
|
+
|
.
|

\
|
+
|
.
|

\
|
Chapter 3: Expectations
3.2 Expected values
Definition: Let X => a discrete r.v. defined on ,
and its possible values are defined x
1
, x
2
, with
the corresponding probabilities f(x
1
), f(x
2
),
Thus, the expected value for X is:
Other notations used:

X
EX
3 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Probability of
each x
i
Value of each
x
i
Chapter 3: Expectations
Suppose that X is a discrete r.v. and we have a
function, Y = g(X)
Because Y is a function of X => Y is also a r.v.
Whenever we have a r.v., we can find its expectation
The formula is:
4 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Similar concept to what weve discussed
Value of function g(x
i
)
Probability
for each x
i
Chapter 3: Expectations
Example: An experiment consists of tossing 3
coins. A random variable X represents the
number of heads. Find what is the expected
value of X.
5 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Example: Suppose a random variable X can take
values {1, 2, 3} and its corresponding
probabilities are given as 0.1, 0.5 and 0.4. If a
function Y is defined as Y = X
2
+ 2X, find the
mean value of Y.
Solution:
6 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Note that probabilities are already given, so
we are left to find the values of the function
Chapter 3: Expectations
If X and Y are discrete r.v., then:
7 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Example: Suppose X denotes the number of
heads appearing in 3 coin tosses and E(X) is
given as 3/2. Suppose a function Y is defined as
Y = 2X + 3. Find the expected value of Y.
Solution:
8 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Example: Suppose X and Y are two discrete
random variables and its joint p.f. is illustrated
in the following table. Find the expected value
of X and Y; that is E(X + Y).
9 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
1 2 3
1 1/12 1/6 1/12
2 1/6 0 1/6
3 0 1/3 0
Y
X
Chapter 3: Expectations
Definition: If X is a continuous r.v. with p.d.f.
f(x), the mean of X or the expected value of X:
If the interest is on a function Y = g(X), then the
mean value or the expected value of Y is:
10 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Example: Find the mean value of X when X has
the following p.d.f. f(x) = 6x (1-x) for 0<x<1.
Solution:
11 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Example: Suppose X is uniform on interval (-1,
1) and define Y = X
2
. Find the mean value of Y.
Solution:
If X is uniform, then p.d.f. of X is just 1/2. How do you get
this result?
Next, find E(Y)
12 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
3.3 Functions of random variables
Suppose we have a random vector (X, Y) and its
p.d.f. f(x, y). Then the expected value of a
function g(X, Y) is just:
If X and Y have expected values, then for
constants a, b and c:
13 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Example: Suppose the joint p.d.f. for random
vector (X, Y) is f(x,y) = 24xy for x>0, y>0, x+y<1.
Find the expected value of X and the expected
value of XY?
Solution:
14 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Example: Suppose we have the following
random variables, X
1
, X
2
,, X
n
and the expected
values for each X
i
is . Suppose function Y is
defined as the sum of X
1
, X
2
,X
n
. Find the
expected value of Y?
Solution:
15 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Two random variables can be independent of
each other.
Suppose X and Y are independent, then
Example: If E(X) is given as 9 and E(Y) is 3/2 and
events X and Y are independent, find the mean
value of E(XY).
16 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
3.4 Moments
Concept of E(X) => describes the middle or
center of the distribution
Concept can be extended to the followings:
Expected value of integer powers of X
Powers of deviations about a particular value
17 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Moments
( )
'
k
k
X E =
kth moment
( ) ( )
k
k
X E =
kth central
moment
Chapter 3: Expectations
What is the first moment of X?
What is the second moment of X?
What is the first central moment of X?
What is the second central moment of X?
18 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
3.4 Variance
Definition: Variance is the second central
moment:
If X is a discrete r.v., then var X can be obtained
by implementing the expectation concept
Recall:
19 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
E[ (X - )
2
]
Chapter 3: Expectations
If X is continuous r.v. then:
It is always easy, however to calculate variance
X using this formula:
20 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
( ) ( ) ( )
( ) ( )
( )
2 2
2 2
2 2
2

2
2

=
+ =
+ =
X E
X E X E
X X E X E
How to get this???
Chapter 3: Expectations
Prove that for any constant a,
Solution:
21 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
( ) | | ( ) | |
2 2
a X E a X E
X X
+ =
( ) | | ( ) ( ) ( ) | |
2 2
X X
a X E a X E =
( ) | | ( ) ( )( ) ( ) | |
2 2 2
2
X X X X
a a X X E a X E + =
( ) | | ( ) | | ( ) ( ) | | ( )
2 2 2
2
X X X X
a X E a X E a X E + =
Chapter 3: Expectations
Example: Suppose that the p.f. is given as f(x) =
1/6 for x = 1, 2,6 and the mean of X is = 7/2.
Find the standard deviation of r.v. X
Solution:
22 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Example: Suppose that X is uniform on the
interval (0, b). What is the p.d.f. of X? Find the
variance of X.
Solution:
23 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
3.5 Chebyshev inequality
For any constant c > 0, and if X has mean and
standard deviation , then
The equivalence form is
24 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
Interested in: finding the probability that X is
greater than k standard deviations from the
mean:
What is the probability X is greater than 2 standard
deviations from the mean?
What is the probability X is greater than 3 standard
deviations from the mean?
25 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
( )
2 2
2
o
o
o
k
k X P s >
Chapter 3: Expectations
Example: Consider the discrete r.v. X defined by
P(X=0) = 3/4; P(X=a) = P(X=-a) = 1/8. Find the
expectation of X and the variance of X. Next,
find what is the probability X is greater than 2
standard deviations from the mean.
Solution:
26 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
3.6 Covariance
The concept of expectation can be extended to
find the covariance.
Recall that variance X is
Now, covariance is a term used to describe the
association of 2 variables, say X and Y.
27 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Positive: X is large when Y is large
OR X is small when Y is small
Negative: X is large when Y is
small OR vice versa
Chapter 3: Expectations
Some other useful results:
cov (X, X) is just variance of X => how to prove this?
Given some constants a and b, then:
If X and Y are independent, then
28 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Chapter 3: Expectations
3.7 Correlation
Covariance suffers from being dependent on
the units of measurement
Example: Suppose X denotes the weight of a person
and Y denotes the height of a person, so the unit
measurement in covariance involves say kg and cm.
Eliminate the problem => correlation coefficient
Also measures the association between 2 variables
But unit-less
29 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
Can you see why
correlation is unit-less?
Chapter 3: Expectations
3.7 Correlation
Correlation is 0 when covariance is 0. Can easily
see this from the formula.
Value of ranges from -1 to 1.
Highly correlated when the values are close to
either -1 or 1
Positive correlation and negative correlation
Follows the same definition of +ve and ve covariance
30 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
The variables are uncorrelated
X and Y independent
NOT TRUE
Chapter 3: Expectations
Example: Suppose a random vector X and Y are
described by the joint p.f. given as follows. Find
the covariance of (X, Y). Next find the
correlation of (X, Y).
31 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12
1 2 3 4
1 0 1/12 1/12 1/12
2 1 2 3 4
3 1/12 1/12 0 1/12
4 1/12 1/12 1/12 0
X
Y
Chapter 3: Expectations
Example: Suppose X, Y and Z are i.i.d
(independent and identically distributed)
random variables with mean 0 and st. deviation
1. Find var (2X + 3Y Z). Next, find cov (X 2Y,
3X + Y + 2Z).
32 NAA, Universiti Pendidikan Sultan Idris, Sem1, 2011/12