Professional Documents
Culture Documents
Probability and Random Variables (Sheffield)
Probability and Random Variables (Sheffield)
Probability and Random Variables (Sheffield)
Counting tricks
Binomial coefficients
Problems
18.440: Lecture 1
Permutations and combinations, Pascals
triangle, learning to count
Scott Sheffield
MIT
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
Outline
Permutations
Counting tricks
Binomial coefficients
Problems
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
Outline
Permutations
Counting tricks
Binomial coefficients
Problems
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
Permutations
I
Answer: 52 51 50 . . . 1 = 52! =
80658175170943878571660636856403766975289505440883277824
1012
Answer: n!
n (n
1) (n
2) . . . (n
k + 1) = n!/(n
k)!
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
Permutation notation
I
I
I
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
Cycle decomposition
I
For example, taking n = 7, we write (2, 3, 5), (1, 7), (4, 6) for
the permutation such that (2) = 3, (3) = 5, (5) = 2 and
(1) = 7, (7) = 1, and (4) = 6, (6) = 4.
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
Outline
Permutations
Counting tricks
Binomial coefficients
Problems
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
Outline
Permutations
Counting tricks
Binomial coefficients
Problems
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
n
k
notation
I
Answer: nk
Answer: n!/(n
Answer:
Answer:
18.440 Lecture 1
n
k
n
k
:=
k)!
n!
k!(n k)!
Permutations
Counting tricks
Binomial coefficients
Problems
Pascals triangle
Arnold principle.
A simple recursion:
Answer:
1 + n1 x 1 + n2 x 2 + . . . +
P
Question: what is nk =0 kn ?
I
I
(x
+ 1)n
n
k
n
k
n 1
k 1
n 1
k
k
x in
.
the expansion of
.
n
0
Answer: (1 + 1)n = 2n .
18.440 Lecture 1
n
n 1
xn
n
n
x n.
Permutations
Counting tricks
Binomial coefficients
Problems
Outline
Permutations
Counting tricks
Binomial coefficients
Problems
18.440 Lecture 1
Permutations
Counting tricks
Binomial coefficients
Problems
More problems
13
13
4
3
4
2
18.440 Lecture 1
12
4
2
12
4
2
11
4
1
/2
Permutations
Counting tricks
Binomial coefficients
Problems
More problems
I
I
I
I
I
I
I
I
How many hands that have four cards of the same suit, one
card of another suit?
13
4 13
4 3 1
How many 10 digit numbers with no consecutive digits that
agree?
If initial digit can be zero, have 10 99 ten-digit sequences. If
initial digit required to be non-zero, have 910 .
How many 10 digit numbers (allowing initial digit to be zero)
in which only 5 of the 10 possible digits are represented?
This is one is tricky, can be solved with inclusion-exclusion (to
come later in the course).
How many ways to assign a birthday to each of 23 distinct
people? What if no birthday can be repeated?
36623 if repeats allowed. 366!/343! if repeats not allowed.
18.440 Lecture 1
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Multinomial coefficients
Integer partitions
More problems
18.440: Lecture 2
Multinomial coefficients and more counting
problems
Scott Sheffield
MIT
18.440 Lecture 2
Multinomial coefficients
Integer partitions
More problems
Outline
Multinomial coefficients
Integer partitions
More problems
18.440 Lecture 2
Multinomial coefficients
Integer partitions
More problems
Outline
Multinomial coefficients
Integer partitions
More problems
18.440 Lecture 2
Multinomial coefficients
Integer partitions
More problems
Partition problems
I
Answer: 8!/(3!2!3!)
Answer
18.440 Lecture 2
n
n1 ,n2 ,...,nr
:=
n!
n1 !n2 !...nr ! .
Multinomial coefficients
Integer partitions
More problems
Multinomial coefficients
Answer: yes.
I
n
n
x n1 x n2 . . . xrnr
n1 , . . . , nr 1 2
18.440 Lecture 2
Multinomial coefficients
Integer partitions
More problems
By the way...
I
I
I
I
I
I
I
I
18.440 Lecture 2
Multinomial coefficients
Integer partitions
More problems
Outline
Multinomial coefficients
Integer partitions
More problems
18.440 Lecture 2
Multinomial coefficients
Integer partitions
More problems
Integer partitions
18.440 Lecture 2
1 bars and n
Multinomial coefficients
Integer partitions
More problems
Outline
Multinomial coefficients
Integer partitions
More problems
18.440 Lecture 2
Multinomial coefficients
Integer partitions
More problems
27!
(3!)9 9!
I
I
18.440 Lecture 2
Multinomial coefficients
Integer partitions
More problems
I
I
I
I
13
3
13
5
13
1
26
13
18.440 Lecture 2
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
18.440: Lecture 3
Sample spaces, events, probability
Scott Sheffield
MIT
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
Outline
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
Outline
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
Outline
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
Similarly, B
of A).
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
I
I
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
Venn diagrams
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
Venn diagrams
A\B
A\B
18.440 Lecture 3
Ac \ B
Ac \ B c
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
Outline
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
DeMorgans laws
It will not snow or rain means It will not snow and it will
not rain.
I
I
I
I
It will not both snow and rain means Either it will not
snow or it will not rain.
(S \ R)c = S c [ R c
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
Outline
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
Axioms of probability
I
I
I
I
18.440 Lecture 3
Formalizing probability
Sample space
DeMorgans laws
Axioms of probability
18.440 Lecture 3
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Axioms of probability
Consequences of axioms
Inclusion exclusion
18.440: Lecture 4
Axioms of probability and
inclusion-exclusion
Scott Sheffield
MIT
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
Outline
Axioms of probability
Consequences of axioms
Inclusion exclusion
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
Outline
Axioms of probability
Consequences of axioms
Inclusion exclusion
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
Axioms of probability
I
I
I
I
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
Outline
Axioms of probability
Consequences of axioms
Inclusion exclusion
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
Intersection notation
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
Consequences of axioms
I
I
I
I
P(A)?
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
Outline
Axioms of probability
Consequences of axioms
Inclusion exclusion
18.440 Lecture 4
Axioms of probability
Consequences of axioms
Inclusion exclusion
Inclusion-exclusion identity
I
I
I
n
X
P(Ei )
i=1
+ ( 1)(r +1)
P(FG ) + P(EFG )?
P(Ei1 Ei2 ) + . . .
i1 <i2
i1 <i2 <...<ir
n+1
= + . . . + ( 1)
I
P(E1 E2 . . . En ).
18.440 Lecture 4
n
r
Axioms of probability
Consequences of axioms
Inclusion exclusion
m
It is counted m1
2 +
inclusion exclusion sum.
18.440 Lecture 4
m
3
+ ...
m
m
1 times in the
1)m .)
Axioms of probability
Consequences of axioms
Inclusion exclusion
n people toss hats into a bin, randomly shue, return one hat
to each person. Find probability nobody gets own hat.
Answer:
I
I
I
(n r )!
n! .
There are nr terms like that in the inclusion exclusion sum.
What is nr (n n!r )! ?
Answer: r1! .
1
1
1
1
P([ni=1 Ei ) = 1 2!
+ 3!
4! + . . . n!
1
1
1
1 P([ni=1 Ei ) = 1 1 + 2!
. . . n1! 1/e .36788
3! + 4!
18.440 Lecture 4
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Equal likelihood
A few problems
Hat problem
A few more problems
18.440: Lecture 5
Problems with all outcomes equally like,
including a famous hat problem
Scott Sheffield
MIT
18.440 Lecture 5
Equal likelihood
A few problems
Hat problem
A few more problems
Outline
Equal likelihood
A few problems
Hat problem
A few more problems
18.440 Lecture 5
Equal likelihood
A few problems
Hat problem
A few more problems
Outline
Equal likelihood
A few problems
Hat problem
A few more problems
18.440 Lecture 5
Equal likelihood
A few problems
Hat problem
A few more problems
Equal likelihood
18.440 Lecture 5
Equal likelihood
A few problems
Hat problem
A few more problems
Outline
Equal likelihood
A few problems
Hat problem
A few more problems
18.440 Lecture 5
Equal likelihood
A few problems
Hat problem
A few more problems
Problems
Roll two dice. What is the probability that their sum is three?
18.440 Lecture 5
Equal likelihood
A few problems
Hat problem
A few more problems
Outline
Equal likelihood
A few problems
Hat problem
A few more problems
18.440 Lecture 5
Equal likelihood
A few problems
Hat problem
A few more problems
P([ni =1 Ei )
n
X
P(Ei )
i=1
+ ( 1)(r +1)
P(Ei1 Ei2 ) + . . .
i1 <i2
i1 <i2 <...<ir
n+1
= + . . . + ( 1)
I
P(E1 E2 . . . En ).
P
The notation i1 <i2 <ir means a sum over all of the
subsets of size r of the set {1, 2, . . . , n}.
18.440 Lecture 5
n
r
Equal likelihood
A few problems
Hat problem
A few more problems
n people toss hats into a bin, randomly shue, return one hat
to each person. Find probability nobody gets own hat.
Answer:
I
I
I
(n r )!
n! .
There are nr terms like that in the inclusion exclusion sum.
What is nr (n n!r )! ?
Answer: r1! .
1
1
1
1
P([ni=1 Ei ) = 1 2!
+ 3!
4! + . . . n!
1
1
1
1 P([ni=1 Ei ) = 1 1 + 2!
. . . n1! 1/e .36788
3! + 4!
18.440 Lecture 5
Equal likelihood
A few problems
Hat problem
A few more problems
Outline
Equal likelihood
A few problems
Hat problem
A few more problems
18.440 Lecture 5
Equal likelihood
A few problems
Hat problem
A few more problems
Problems
18.440 Lecture 5
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 6
Conditional probability
Scott Sheffield
MIT
18.440 Lecture 6
Outline
Examples
Multiplication rule
18.440 Lecture 6
Outline
Examples
Multiplication rule
18.440 Lecture 6
Conditional probability
I
18.440 Lecture 6
Outline
Examples
Multiplication rule
18.440 Lecture 6
More examples
I
.9p
.9p+.1(1 p) .
P(disease|positive) =
18.440 Lecture 6
There are only two taxi companies in town, Blue Cabs and
Green Cabs. On the night in question, 85 percent of all taxis
on the road were green and 15 percent were blue.
The witness has undergone an extensive vision test under
conditions similar to those on the night in question, and has
demonstrated that he can successfully distinguish a blue taxi
from a green taxi 80 percent of the time.
18.440 Lecture 6
Outline
Examples
Multiplication rule
18.440 Lecture 6
Multiplication rule
I
I
P(E1 E2 E3 . . . En ) =
P(E1 )P(E2 |E1 )P(E3 |E1 E2 ) . . . P(En |E1 . . . En
1)
For example, let Ei be event ith person gets own hat in the
n-hat shue problem.
Another example: roll die and let Ei be event that the roll
does not lie in {1, 2, . . . , i}. Then P(Ei ) = (6 i)/6 for
i 2 {1, 2, . . . , 6}.
18.440 Lecture 6
If prize behind that door, host tosses a coin, opens one of the
other two doors.
You then get to open a door and claim whats behind it.
Should you stick with door you pointed to or choose other
door?
18.440 Lecture 6
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Bayes formula
Independence
18.440: Lecture 7
Bayes formula and independence
Scott Sheffield
MIT
18.440 Lecture 7
Bayes formula
Independence
Outline
Bayes formula
Independence
18.440 Lecture 7
Bayes formula
Independence
Outline
Bayes formula
Independence
18.440 Lecture 7
Bayes formula
Independence
I
I
I
18.440 Lecture 7
Bayes formula
Independence
What is P(D|T )?
18.440 Lecture 7
Bayes formula
Independence
Bayes theorem
I
So P(A|B) is
Ratio
18.440 Lecture 7
P (B |A)
P (B )
P(B|A)
P(B)
times P(A).
Bayes formula
Independence
Bayes theorem
P(B|A)P(A)
P(B)
18.440 Lecture 7
Bayes formula
Independence
Updated odds
I
P(A|B)/P(Ac |B)
P(A)/P(Ac )
18.440 Lecture 7
=?
P(A|B)/P(Ac |B)
P(A)/P(Ac )
= P(B|A)/P(B|Ac )
Bayes formula
Independence
I
I
18.440 Lecture 7
Bayes formula
Independence
Outline
Bayes formula
Independence
18.440 Lecture 7
Bayes formula
Independence
Independence
I
I
I
I
I
I
I
I
18.440 Lecture 7
Bayes formula
Independence
No. Consider these three events: first coin heads, second coin
heads, odd number heads. Pairwise independent, not
independent.
18.440 Lecture 7
Bayes formula
Independence
18.440 Lecture 7
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 8
Discrete random variables
Scott Sheffield
MIT
18.440 Lecture 8
Outline
Recursions
18.440 Lecture 8
Outline
Recursions
18.440 Lecture 8
Random variables
Example: toss n coins (so state space consists of the set of all
2n possible coin sequences) and let X be number of heads.
Answer:
18.440 Lecture 8
n
k
No. Consider these three events: first coin heads, second coin
heads, odd number heads. Pairwise independent, not
independent.
18.440 Lecture 8
Examples
I
I
I
Now say we roll three dice and let Y be sum of the values on
the dice. What is P{Y = 5}?
18.440 Lecture 8
Indicators
I
I
I
18.440 Lecture 8
Outline
Recursions
18.440 Lecture 8
18.440 Lecture 8
I
I
I
I
xa p(x).
18.440 Lecture 8
Another example
I
I
I
I
I
18.440 Lecture 8
Outline
Recursions
18.440 Lecture 8
I
I
I
I
I
18.440 Lecture 8
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Defining expectation
Functions of random variables
Motivation
18.440: Lecture 9
Expectations of discrete random variables
Scott Sheffield
MIT
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
Outline
Defining expectation
Motivation
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
Outline
Defining expectation
Motivation
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
Simple examples
I
What is E [X ]?
Answer: p.
Answer:
18.440 Lecture 9
1
61
+ 16 2 + 16 3 + 16 4 + 16 5 + 16 6 =
21
6
p. Then what
= 3.5.
Defining expectation
Functions of random variables
Motivation
I
I
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
A technical point
If the state
P space S is countable, is it possible that the sum
E [X ] = s2S P({s})X (s) somehow depends on the order in
which s 2 S are enumerated?
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
Outline
Defining expectation
Motivation
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
Answer:
E [g (X )] =
g (x)p(x).
x:p(x)>0
I
What is E [X + b]?
Generally, E [aX + b] = aE [X ] + b = a + b.
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
More examples
I
I
I
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
Additivity of expectation
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
More examples
Answer: 1/n.
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
Outline
Defining expectation
Motivation
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
18.440 Lecture 9
Defining expectation
Functions of random variables
Motivation
Contract one: Ill toss 10 coins, and if they all come up heads
(probability about one in a thousand), Ill give you 20 billion
dollars.
Can you find a function u(x) such that given two random
wealth variables W1 and W2 , you prefer W1 whenever
E [u(W1 )] < E [u(W2 )]?
18.440 Lecture 9
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Defining variance
Examples
Properties
Decomposition trick
18.440: Lecture 10
Variance
Scott Sheffield
MIT
18.440 Lecture 10
Defining variance
Examples
Properties
Decomposition trick
Outline
Defining variance
Examples
Properties
Decomposition trick
18.440 Lecture 10
Defining variance
Examples
Properties
Decomposition trick
Outline
Defining variance
Examples
Properties
Decomposition trick
18.440 Lecture 10
Defining variance
Examples
Properties
Decomposition trick
Also,
E [g (X )] =
x:p(x)>0
18.440 Lecture 10
g (x)p(x).
Defining variance
Examples
Properties
Decomposition trick
Defining variance
I
(x
)2 p(x).
x:p(x)>0
I
18.440 Lecture 10
Defining variance
Examples
Properties
Decomposition trick
18.440 Lecture 10
)2 ].
2X + 2 ].
Defining variance
Examples
Properties
Decomposition trick
Outline
Defining variance
Examples
Properties
Decomposition trick
18.440 Lecture 10
Defining variance
Examples
Properties
Decomposition trick
Variance examples
Var[X ] = E [X 2 ] E [X ]2 =
1 2
1 2
1 2
1 2
1 2
1 2
61 + 62 + 63 + 64 + 65 + 66
(7/2)2 =
91
6
49
4
Then Var[Y ] = E [Y 2 ]
18.440 Lecture 10
E [Y ]2 = 14 02 + 21 12 + 41 22
35
12 .
12 = 12 .
Defining variance
Examples
Properties
Decomposition trick
Variance?
18.440 Lecture 10
Defining variance
Examples
Properties
Decomposition trick
Outline
Defining variance
Examples
Properties
Decomposition trick
18.440 Lecture 10
Defining variance
Examples
Properties
Decomposition trick
Identity
Yes.
18.440 Lecture 10
E [aX ]2 = a2 E [X 2 ]
a2 E [X ]2 =
Defining variance
Examples
Properties
Decomposition trick
Standard deviation
p
Var[X ].
Write SD[X ] =
18.440 Lecture 10
Defining variance
Examples
Properties
Decomposition trick
Outline
Defining variance
Examples
Properties
Decomposition trick
18.440 Lecture 10
Defining variance
Examples
Properties
Decomposition trick
Number of aces
I
Answer:
Answer:
I
I
I
52
5
4
k
.
48
5 k
(4)( 48 )
So P{A = k} = k 525 k .
(5)
P4
So E [A] = k =0 kP{A = k},
P
and Var[A] = 4k =0 k 2 P{A = k}
18.440 Lecture 10
E [A]2 .
Defining variance
Examples
Properties
Decomposition trick
18.440 Lecture 10
E [A]2 =
105
1317
25
1313 .
Defining variance
Examples
Properties
Decomposition trick
n
X
i=1
18.440 Lecture 10
Xi
n
X
j=1
Xj ] =
n X
n
X
i=1 j=1
E [Xi Xj ].
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 11
Binomial random variables and repeated
trials
Scott Sheffield
MIT
18.440 Lecture 11
Outline
More problems
18.440 Lecture 11
Outline
More problems
18.440 Lecture 11
Answer:
Answer:
Writing q = 1
I
I
n
k
n
k
/2n .
p k (1
p)n
k.
n
k
pk qn
18.440 Lecture 11
Examples
18.440 Lecture 11
Other examples
18.440 Lecture 11
n
i
pi qn
Outline
More problems
18.440 Lecture 11
Expectation
I
What is E [X ]?
Direct approach:
by definition of expectation,
P
E [X ] = ni=0 P{X = i}i.
18.440 Lecture 11
Recall that
n
i
but important
I
i=0
Rewrite this as E [X ] = np
Substitute j = i
E [X ] = np
n 1
X
n
j=0
18.440 Lecture 11
n
X
n
=
n
i
i=1
Pn
i=1
1 to get
implies a simple
p j q (n
n 1
i 1
1) j
p (i
1 i n i
pq .
1
1) q (n 1) (i 1) .
= np(p + q)n
= np.
n
X
j=1
18.440 Lecture 11
E [Xj ] =
n
X
j=1
p = np.
Identity gives
E [X ] = np
n
X
n
i=1
np
n 1
X
j=0
E [X k ]
1
j
p j (1
Thus
= E [(Y + 1)k
parameters (n 1, p).
18.440 Lecture 11
1 i
p
1
1]
p)n
(1
1 j
p)n i i k
(j + 1)k
We know E [X ] = np.
1]
1)p + 1].
E [X 2 ]
E [X ]2
where Y is
(np)2 =
So Var[X ] =
= np(n
np(1 p) = npq, where q = 1 p.
18.440 Lecture 11
1)p + np
I
I
I
I
P
X = nj=1 Xj , so
P
P
P P
E [X 2 ] = E [ ni =1 Xi nj =1 Xj ] = ni =1 jn=1 E [Xi Xj ]
E [Xi Xj ] is p if i = j, p 2 otherwise.
Pn Pn
i=1
j=1 E [Xi Xj ] has n terms equal to p and (n
terms equal to p 2 .
So E [X 2 ] = np + (n
Thus
Var[X ] = E [X 2 ]
18.440 Lecture 11
1)np 2 = np + (np)2
E [X ]2 = np
np 2 = np(1
1)n
np 2 .
p) = npq.
Outline
More problems
18.440 Lecture 11
More examples
I
An airplane seats 200, but the airline has sold 205 tickets.
Each person, independently, has a .05 chance of not showing
up for the flight. What is the probability that more than 200
people will show up for the flight?
18.440 Lecture 11
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 12
Poisson random variables
Scott Sheffield
MIT
18.440 Lecture 12
Outline
18.440 Lecture 12
Outline
18.440 Lecture 12
18.440 Lecture 12
Its the amount of money that one dollar grows to over a year
when you have an interest rate of 100 percent, continuously
compounded.
Its the amount of money that one dollar grows to over a year
when you have an interest rate of 100 percent, continuously
compounded.
Its also the amount of money that one dollar grows to over
years when you have an interest rate of 100 percent,
continuously compounded.
18.440 Lecture 12
= limn!1 (1
/n)n .
I
I
18.440 Lecture 12
satisfies
Outline
18.440 Lecture 12
I
I
I
18.440 Lecture 12
satisfies
Expectation
I
I
I
I
I
1
X
P{X = k}k =
k=0
Setting j = k
18.440 Lecture 12
1
X
k=0
1, this is
P1
k!
j
j=0 j! e
1
X
k=1
= .
(k
1)!
Variance
I
I
I
I
Given P{X = k} = k! e
for integer k 0, what is Var[X ]?
Think of X as (roughly) a Bernoulli (n, p) random variable
with n very large and p = /n.
This suggests Var[X ] npq (since np and
q = 1 p 1). Can we show directly that Var[X ] = ?
Compute
E [X 2 ] =
1
X
P{X = k}k 2 =
k=0
Setting j = k 1, this is
0
1
j
X
@ (j + 1) e
j!
j=0
Then Var[X ] = E [X 2 ]
18.440 Lecture 12
1
X
k=0
k2
k!
1
X
k=1
k 1
(k
A = E [X + 1] = ( + 1).
E [X ]2 = ( + 1)
= .
1)!
Outline
18.440 Lecture 12
I
I
18.440 Lecture 12
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 13
Poisson processes
Scott Sheffield
MIT
18.440 Lecture 13
Outline
18.440 Lecture 13
Outline
18.440 Lecture 13
(1
p)n
e .
k!
k!
General idea: if you have a large number of unlikely events
that are (mostly) independent of each other, and the expected
number that occur is , then the total number that occur
should be (approximately) a Poisson random variable with
parameter .
18.440 Lecture 13
I
I
I
I
18.440 Lecture 13
A cautionary tail
I
Example: Joe works for a bank and notices that his town sees
an average of one mortgage foreclosure per month.
18.440 Lecture 13
Outline
18.440 Lecture 13
18.440 Lecture 13
Outline
18.440 Lecture 13
18.440 Lecture 13
Outline
18.440 Lecture 13
I
I
I
I
I
18.440 Lecture 13
18.440 Lecture 13
t)k /k!.
Binomial formula:
n k
p)n k =
k p (1
p)n
k.
t.
This is
18.440 Lecture 13
Summary
I
Let Tk be time elapsed, since the previous event, until the kth
event occurs. Then the Tk are independent random variables,
each of which is exponential with parameter .
18.440 Lecture 13
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 14
More discrete random variables
Scott Sheffield
MIT
18.440 Lecture 14
Outline
Problems
18.440 Lecture 14
Outline
Problems
18.440 Lecture 14
Answer: P{X = k} = (1
is tails probability.
18.440 Lecture 14
p)k
1p
= qk
1 p,
where q = 1
18.440 Lecture 14
What is E [X 2 ]?
By definition E [X 2 ] =
P1
k =1 q
k 1 pk 2 .
1,
18.440 Lecture 14
2E [X ] + 1 =
q)E [X 2 ] = pE [X 2 ] = 2/p
1, so
Example
Answer: (5/6)k
What is E [X ]?
Answer: 6.
What is Var[X ]?
Answer: 1/p 2
18.440 Lecture 14
1 (1/6).
1/p = 36
6 = 30.
Outline
Problems
18.440 Lecture 14
So P{X = k} =
sum to 1?
18.440 Lecture 14
k 1
r 1
pr
1 (1
p)k
r p.
1 tosses
I
I
I
18.440 Lecture 14
Outline
Problems
18.440 Lecture 14
Problems
I
I
I
I
I
I
I
I
Nate and Natasha have beautiful new baby. Each minute with
.01 probability (independent of all else) baby cries.
Additivity of expectation: How many times do they expect
the baby to cry between 9 p.m. and 6 a.m.?
Geometric random variables: Whats the probability baby is
quiet from midnight to three, then cries at exactly three?
Geometric random variables: Whats the probability baby is
quiet from midnight to three?
Negative binomial: Probability fifth cry is at midnight?
Negative binomial expectation: How many minutes do I
expect to wait until the fifth cry?
Poisson approximation: Approximate the probability there
are exactly five cries during the night.
Exponential random variable approximation: Approximate
probability baby quiet all night.
18.440 Lecture 14
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 15
Continuous random variables
Scott Sheffield
MIT
18.440 Lecture 15
Outline
18.440 Lecture 15
Outline
18.440 Lecture 15
I
I
I
I
18.440 Lecture 15
Simple example
(
1/2
0
x 2 [0, 2]
x 62 [0, 2].
Suppose f (x) =
I
I
What is F ?
18.440 Lecture 15
Another example
x/2 x 2 [0, 2]
0
02
6 [0, 2].
Suppose f (x) =
What is F ?
18.440 Lecture 15
Outline
18.440 Lecture 15
I
I
I
I
I
18.440 Lecture 15
Next, if g =R g1 + g2 then R
E
R [g (X )] = g1 (x)f (x)dx + g2 (x)f (x)dx =
g1 (x) + g2 (x) f (x)dx = E [g1 (X )] + E [g2 (X )].
18.440 Lecture 15
Examples
1/2
0
What is Var[X ]?
What is Var[X ]?
18.440 Lecture 15
x 2 [0, 2]
.
x 62 [0, 2].
(
x/2 x 2 [0, 2]
0
02
6 [0, 2].
Outline
18.440 Lecture 15
I
I
I
I
One of the
( very simplest probability density functions is
1 x 2 [0, 1]
f (x) =
.
0 0 62 [0, 1].
18.440 Lecture 15
I
I
I
I
I
I
I
I
18.440 Lecture 15
I
I
I
I
18.440 Lecture 15
Perspective
I
18.440 Lecture 15
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 16
Lectures 1-15 Review
Scott Sheffield
MIT
18.440 Lecture 16
Outline
18.440 Lecture 16
Outline
18.440 Lecture 16
Answer
18.440 Lecture 16
n
n1 ,n2 ,...,nr
:=
n!
n1 !n2 !...nr ! .
1 bars and n
Axioms of probability
I
I
I
P(S) = 1.
18.440 Lecture 16
Consequences of axioms
P(Ac ) = 1
I
I
P(A)
18.440 Lecture 16
P(AB)
Inclusion-exclusion identity
I
I
I
Also, P(E [ F [ G ) =
P(E ) + P(F ) + P(G ) P(EF )
P(AB).
P(EG )
P(FG ) + P(EFG ).
More generally,
P([ni=1 Ei ) =
n
X
P(Ei )
i=1
+ ( 1)(r +1)
P(Ei1 Ei2 ) + . . .
i1 <i2
i1 <i2 <...<ir
n+1
= + . . . + ( 1)
I
P(E1 E2 . . . En ).
P
The notation i1 <i2 <...<ir means a sum over all of the
subsets of size r of the set {1, 2, . . . , n}.
18.440 Lecture 16
n
r
n people toss hats into a bin, randomly shue, return one hat
to each person. Find probability nobody gets own hat.
Answer:
I
I
I
(n r )!
n! .
There are nr terms like that in the inclusion exclusion sum.
What is nr (n n!r )! ?
Answer: r1! .
1
1
1
1
P([ni=1 Ei ) = 1 2!
+ 3!
4! + . . . n!
1
1
1
1 P([ni=1 Ei ) = 1 1 + 2!
. . . n1! 1/e .36788
3! + 4!
18.440 Lecture 16
Conditional probability
I
I
1)
For example, let Ei be event ith person gets own hat in the
n-hat shue problem.
18.440 Lecture 16
18.440 Lecture 16
Bayes theorem
So P(A|B) is
Ratio
18.440 Lecture 16
P (B |A)
P (B )
P(B|A)
P(B)
times P(A).
I
I
18.440 Lecture 16
Independence
18.440 Lecture 16
No. Consider these three events: first coin heads, second coin
heads, odd number heads. Pairwise independent, not
independent.
18.440 Lecture 16
Outline
18.440 Lecture 16
Random variables
18.440 Lecture 16
Indicators
I
I
I
18.440 Lecture 16
18.440 Lecture 16
18.440 Lecture 16
Answer:
E [g (X )] =
x:p(x)>0
18.440 Lecture 16
g (x)p(x).
Additivity of expectation
18.440 Lecture 16
(x
)2 p(x).
x:p(x)>0
I
18.440 Lecture 16
(E [X ])2 .
Identity
18.440 Lecture 16
E [aX ]2 = a2 E [X 2 ]
a2 E [X ]2 =
Standard deviation
p
Var[X ].
Write SD[X ] =
18.440 Lecture 16
Answer:
Answer:
Writing q = 1
I
I
n
k
n
k
/2n .
p k (1
p)n
k.
n
k
pk qn
18.440 Lecture 16
n
X
j=1
18.440 Lecture 16
E [Xj ] =
n
X
j=1
p = np.
I
I
I
I
I
P
X = nj =1 Xj , so
P
P
P P
E [X 2 ] = E [ ni=1 Xi nj=1 Xj ] = ni=1 nj=1 E [Xi Xj ]
E [Xi Xj ] is p if i = j, p 2 otherwise.
Pn Pn
i=1
j=1 E [Xi Xj ] has n terms equal to p and (n
terms equal to p 2 .
So E [X 2 ] = np + (n
Thus
Var[X ] = E [X 2 ]
1)np 2 = np + (np)2
E [X ]2 = np
np 2 = np(1
1)n
np 2 .
p) = npq.
Can P
show generally
P that if X1 , . . . , Xn independent then
Var[ nj =1 Xj ] = nj =1 Var[Xj ]
18.440 Lecture 16
I
I
18.440 Lecture 16
satisfies
18.440 Lecture 16
satisfies
and Var[X ] = .
Let Tk be time elapsed, since the previous event, until the kth
event occurs. Then the Tk are independent random variables,
each of which is exponential with parameter .
18.440 Lecture 16
t.
Answer: P{X = k} = (1
is tails probability.
18.440 Lecture 16
p)k
1p
= qk
1 p,
where q = 1
Then P{X = k} =
So E [X ] = r /p.
18.440 Lecture 16
k 1
r 1
pr
1 (1
p)k
r p.
Outline
18.440 Lecture 16
I
I
I
I
18.440 Lecture 16
I
I
I
I
I
18.440 Lecture 16
Next, if g =R g1 + g2 then R
E
R [g (X )] = g1 (x)f (x)dx + g2 (x)f (x)dx =
g1 (x) + g2 (x) f (x)dx = E [g1 (X )] + E [g2 (X )].
18.440 Lecture 16
I
I
I
I
I
I
I
I
18.440 Lecture 16
I
I
I
I
18.440 Lecture 16
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 18
Uniform random variables
Scott Sheffield
MIT
18.440 Lecture 18
Outline
18.440 Lecture 18
Outline
18.440 Lecture 18
I
I
I
I
18.440 Lecture 18
I
I
I
Suppose X is a random
variable with probability density
(
1 x 2 [0, 1]
function f (x) =
0 x 62 [0, 1].
a.
18.440 Lecture 18
I
I
I
I
I
I
I
Suppose X is a random
variable with probability density
(
1 x 2 [0, 1]
function f (x) =
0 x 62 [0, 1].
What is E [X ]?
So Var[X ] =
18.440 Lecture 18
E [X 2 ]
(E [X ])2
= 1/3
1/4 = 1/12.
Suppose X is a random
variable with probability density
(
1 x 2 [0, 1]
function f (x) =
0 x 62 [0, 1].
What is the cumulative distribution function
FX (a) = P{X < a}?
8
>
<0 a < 0
FX (a) = a a 2 [0, 1] .
>
:
1 a>1
What is the general moment E [X k ] for k
18.440 Lecture 18
0?
Outline
18.440 Lecture 18
Fix <
I
I
b a
.
18.440 Lecture 18
I
I
I
I
I
I
I
Suppose X is a random
variable with probability density
(
1
x 2 [, ]
function f (x) =
0
x 62 [, ].
What is E [X ]?
Intuitively, wed guess the midpoint +
2 .
Whats the cleanest way to prove this?
One approach: let Y be uniform on [0, 1] and try to show that
)Y + is uniform on [, ].
X =(
Then linearity of
)E [Y ] + = (1/2)(
) + = +
E [X ] = (
2 .
Using similar logic, what is the variance Var[X ]?
Answer: Var[X ] = Var[(
)Y + ] = Var[(
)Y ] =
2
2
(
) Var[Y ] = (
) /12.
18.440 Lecture 18
Outline
18.440 Lecture 18
Toss n = 300 million Americans into a hat and pull one out
uniformly at random.
Is the height of the person you choose a uniform random
variable?
Maybe in an approximate sense?
No.
Is the percentile of the person I choose uniformly random? In
other words, let p be the fraction of people left in the hat
whose heights are less than that of the person I choose. Is p,
in some approximate sense, a uniform random variable on
[0, 1]?
The way I defined it, p is uniform from the set
{0, 1/(n 1), 2/(n 1), . . . , (n 2)/(n 1), 1}. When n is
large, this is kind of like a uniform random variable on [0, 1].
18.440 Lecture 18
3. The amount of time you have to wait until the next subway
train come (assuming trains come promptly every six minutes
and you show up at kind of a random time).
4. The amount of time you have to wait until the next subway
train (without the parenthetical assumption above).
18.440 Lecture 18
18.440 Lecture 18
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Tossing coins
Normal random variables
Special case of central limit theorem
18.440: Lecture 19
Normal random variables
Scott Sheffield
MIT
18.440 Lecture 19
Tossing coins
Normal random variables
Special case of central limit theorem
Outline
Tossing coins
18.440 Lecture 19
Tossing coins
Normal random variables
Special case of central limit theorem
Outline
Tossing coins
18.440 Lecture 19
Tossing coins
Normal random variables
Special case of central limit theorem
Tossing coins
18.440 Lecture 19
Tossing coins
Normal random variables
Special case of central limit theorem
Tossing coins
Answer: 2
Answer: p k (1
18.440 Lecture 19
k n
k
p)n
k n
k
Tossing coins
Normal random variables
Special case of central limit theorem
Outline
Tossing coins
18.440 Lecture 19
Tossing coins
Normal random variables
Special case of central limit theorem
so I =
18.440 Lecture 19
p
2.
1
0
re
r 2 /2
dr =
2e
r 2 /2
1
0
Tossing coins
Normal random variables
Special case of central limit theorem
I
I
x 2 /2
1
xdx = p e
2
x 2 /2
= 0.
2
p1 e x /2 x 2 dx?
2
18.440 Lecture 19
x 2 /2 dx.
x 2 /2 dx)
= 1.
Tossing coins
Normal random variables
Special case of central limit theorem
E [Y ] = E [X ] + = and Var[Y ] =
18.440 Lecture 19
if
2 Var[X ]
2.
Tossing coins
Normal random variables
Special case of central limit theorem
I
I
I
I
I
x 2 /2 dx.
( 3) .0013,
( 2) .023 and
( 1) .159.
18.440 Lecture 19
Tossing coins
Normal random variables
Special case of central limit theorem
Outline
Tossing coins
18.440 Lecture 19
Tossing coins
Normal random variables
Special case of central limit theorem
Sn np
lim P{a p
b} ! (b)
n!1
npq
I
(a).
This is (b)
(a) = P{a X b} when X is a standard
normal random variable.
18.440 Lecture 19
Tossing coins
Normal random variables
Special case of central limit theorem
Problems
I
I
I
I
18.440 Lecture 19
( 2.19).
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 20
Exponential random variables
Scott Sheffield
MIT
18.440 Lecture 20
Outline
18.440 Lecture 20
Outline
18.440 Lecture 20
x a
0
=1
a.
18.440 Lecture 20
e
a
dx =
Moment formula
I
I
I
I
1
.
0
1 ].
E [X 0 ] = E [1] = 1, E [X ] = 1/ , E [X 2 ] = 2/
E [X n ] = n!/ n .
Variance: Var[X ] = E [X 2 ]
18.440 Lecture 20
(E [X ])2 = 1/
2.
2,
Outline
18.440 Lecture 20
I
I
I
I
I
I
18.440 Lecture 20
Outline
18.440 Lecture 20
Memoryless property
I
I
I
I
I
I
I
18.440 Lecture 20
If we plan to toss a coin until the first heads comes up, then
we have a .5 chance to get a heads in one step, a .25 chance
in two steps, etc.
Given that the first 5 tosses are all tails, there is conditionally
a .5 chance we get our first heads on the 6th toss, a .25
chance on the 7th toss, etc.
18.440 Lecture 20
I
I
I
I
I
I
18.440 Lecture 20
Bob: No, I dont think you get it yet. Its a subtle point in
statistics. Its very important.
18.440 Lecture 20
I
I
18.440 Lecture 20
I
I
I
I
I
18.440 Lecture 20
Outline
18.440 Lecture 20
18.440 Lecture 20
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Gamma distribution
Cauchy distribution
Beta distribution
18.440: Lecture 21
More continuous random variables
Scott Sheffield
MIT
18.440 Lecture 21
Gamma distribution
Cauchy distribution
Beta distribution
Outline
Gamma distribution
Cauchy distribution
Beta distribution
18.440 Lecture 21
Gamma distribution
Cauchy distribution
Beta distribution
Outline
Gamma distribution
Cauchy distribution
Beta distribution
18.440 Lecture 21
Gamma distribution
Cauchy distribution
Beta distribution
so that () = (
is defined on (0, 1)
18.440 Lecture 21
1)!
Gamma distribution
Cauchy distribution
Beta distribution
Answer:
18.440 Lecture 21
k 1
n 1
pn
1 (1
p)k
n p.
Gamma distribution
Cauchy distribution
Beta distribution
For large N,
(k
k 1
n 1
1)(k
pn
p)k
2) . . . (k
(n 1)!
kn 1 n
p
(n 1)!
18.440 Lecture 21
1 (1
np
is
n + 1)
pn
p)k
(1
1 ( x)(n
p=
N
(n
1) e
1)!
by
Gamma distribution
Cauchy distribution
Beta distribution
Defining
distribution
(n
The probability from previous side, N1 ( x)(n
the form for a continuum random variable.
1) e
1)!
suggests
18.440 Lecture 21
Gamma distribution
Cauchy distribution
Beta distribution
Outline
Gamma distribution
Cauchy distribution
Beta distribution
18.440 Lecture 21
Gamma distribution
Cauchy distribution
Beta distribution
Cauchy distribution
FX (x ) = P {X x } = P {tan x } = P { tan
1
1
1 x.
2 + tan
Find fX (x) =
18.440 Lecture 21
d
dx F (x )
1 1
1+x 2 .
1x }
Gamma distribution
Cauchy distribution
Beta distribution
I
I
I
18.440 Lecture 21
Gamma distribution
Cauchy distribution
Beta distribution
18.440 Lecture 21
Gamma distribution
Cauchy distribution
Beta distribution
Outline
Gamma distribution
Cauchy distribution
Beta distribution
18.440 Lecture 21
Gamma distribution
Cauchy distribution
Beta distribution
I
I
1
( n )x a 1 (1 x)b 1
P p = x|h = (a 1) = 11 a P1{h=(a 1)}
which is
xa
1 (1
18.440 Lecture 21
x)b
Gamma distribution
Cauchy distribution
Beta distribution
Beta distribution
I
I
I
I
I
I
I
18.440 Lecture 21
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these mateials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 22
Joint distributions functions
Scott Sheffield
MIT
18.440 Lecture 22
Outline
18.440 Lecture 22
Outline
18.440 Lecture 22
18.440 Lecture 22
Outline
18.440 Lecture 22
18.440 Lecture 22
18.440 Lecture 22
I
I
@ @
@x @ y F (x, y ).
18.440 Lecture 22
Outline
18.440 Lecture 22
I
I
I
I
18.440 Lecture 22
f (x, y ) = fX (x)fY (y ) =
Using
polar coordinates, we want
R1
1
r 2 /2 dr = e r 2 /2 1 = 1
0 (2r ) 2 e
0
18.440 Lecture 22
2
2
p1 e x /2 p1 e y /2
2
2
1/2
1
r 2 /2
2 e
.39.
Outline
18.440 Lecture 22
Roll a die repeatedly and let X be such that the first even
number (the first 2, 4, or 6) appears on the X th roll.
If j
1, then
P{X = j, Y = 2} = P{X = j, Y = 4}
= P{X = j, Y = 6} = (1/2)j
18.440 Lecture 22
I
I
I
18.440 Lecture 22
Exponential
E [Ttiger ] = 1/
hours squared.
tiger
= .2/hour. So P{Ttiger a} = e
tiger
= 5 hours, Var[Ttiger ] = 1/
2
tiger
.2a .
= 25
= .6.
18.440 Lecture 22
18.440 Lecture 22
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these mateials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 23
Sums of independent random variables
Scott Sheffield
MIT
18.440 Lecture 23
P{X + Y a} =
=
I
I
1
1
1
1
FX (a
a y
1
fX (x)fY (y )dxdy
y )fY (y )dy .
18.440 Lecture 23
Worth remembering.
18.440 Lecture 23
I
I
18.440 Lecture 23
18.440 Lecture 23
I
I
I
y(
y )n
(n)
So fZ (y ) =
18.440 Lecture 23
x(
x)s 1
and fY (y ) = e
(s)
R1
y )fY (y )dy .
1 fX (a
y(
y )t
(t)
So fX (x) =
Now fX +Y (a) =
Up to an a-independent multiplicative constant, this is
Z a
Z a
(a y )
s 1
y t 1
a
e
(a y ) e
y dy = e
(a y )s 1 y t
I
I
dy .
18.440 Lecture 23
2
1,
x2
2 2
1
p 1
2
Y is normal with
y2
2 2
2
fX (x) =
and fY (y ) =
I
I
e
R2 1
a2
2+ 2)
1
2
1 fX (a
y )fY (y )dy .
18.440 Lecture 23
Other sums
Yes, Poisson 1 +
interpretation.
18.440 Lecture 23
2.
and Poisson
2?
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these mateials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 24
Conditional probability, order statistics,
expectations of sums
Scott Sheffield
MIT
18.440 Lecture 24
Outline
Order statistics
Expectations of sums
18.440 Lecture 24
Outline
Order statistics
Expectations of sums
18.440 Lecture 24
Conditional distributions
I
I
18.440 Lecture 24
I
I
18.440 Lecture 24
A word of caution
I
I
I
I
I
18.440 Lecture 24
Outline
Order statistics
Expectations of sums
18.440 Lecture 24
18.440 Lecture 24
I
I
I
Are
Yes.
18.440 Lecture 24
Example
I
Up to a constant, f (x) = x 7 (1
18.440 Lecture 24
x)2 .
Outline
Order statistics
Expectations of sums
18.440 Lecture 24
Properties of expectation
I
I
I
I
I
I
I
18.440 Lecture 24
Properties of expectation
I
I
I
I
I
I
I
18.440 Lecture 24
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these mateials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 25
Covariance and some conditional
expectation exercises
Scott Sheffield
MIT
18.440 Lecture 25
Outline
18.440 Lecture 25
Outline
18.440 Lecture 25
A property of independence
I
I
I
18.440 Lecture 25
Then
X E [Y ]
Y E [X ] + X Y = E [XY ]
E [X ]E [Y ].
18.440 Lecture 25
m
X
i=1
ai Xi ,
n
X
bj Y j ) =
j=1
m X
n
X
ai bj Cov(Xi , Yj ).
i=1 j=1
Special case:
Var(
n
X
i=1
18.440 Lecture 25
Xi ) =
n
X
i=1
Var(Xi ) + 2
(i,j):i<j
Cov(Xi , Xj ).
Defining correlation
I
I
I
I
I
I
I
E [X ]E [Y ].
Cov(X , Y )
Var(X )Var(Y )
18.440 Lecture 25
Important point
I
18.440 Lecture 25
Examples
I
I
I
I
I
I
I
I
18.440 Lecture 25
Outline
18.440 Lecture 25
Famous paradox
I
18.440 Lecture 25
I
I
I
I
18.440 Lecture 25
Precise details not important, but lets say you have 1/4 in
the 0th pile and 38 5j in the jth pile for each j > 0. Important
thing is that pile size is increasing exponentially in j.
Every pile is bigger after transfer (and this can be true even if
banker takes a portion of each pile as a fee).
Banker seemed to make you richer (every pile got bigger) but
really just reshued your infinite wealth.
18.440 Lecture 25
18.440 Lecture 25
Moral
18.440 Lecture 25
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 26
Conditional expectation
Scott Sheffield
MIT
18.440 Lecture 26
Outline
Conditional expectation
18.440 Lecture 26
Outline
Conditional expectation
18.440 Lecture 26
I
I
18.440 Lecture 26
Example
I
I
I
I
18.440 Lecture 26
Outline
Conditional expectation
18.440 Lecture 26
Conditional expectation
I
I
18.440 Lecture 26
f (x,y )
fY (y ) .
So
Example
What is E [X |Y = 5]?
I
I
What is E [Z |Y = 5]?
What is E [Y |Z = 5]?
18.440 Lecture 26
18.440 Lecture 26
Conditional variance
I
I
I
I
I
I
Definition:
Var(X |Y ) = E (X E [X |Y ])2 |Y = E X 2 E [X |Y ]2 |Y .
Var(X |Y ) is a random variable that depends on Y . It is the
variance of X in the conditional distribution for X given Y .
Note E [Var(X |Y )] = E [E [X 2 |Y ]] E [E [X |Y ]2 |Y ] =
E [X 2 ] E [E [X |Y ]2 ].
If we subtract E [X ]2 from first term and add equivalent value
E [E [X |Y ]]2 to the second, RHS becomes
Var[X ] Var[E [X |Y ]], which implies following:
Useful fact: Var(X ) = Var(E [X |Y ]) + E [Var(X |Y )].
One can discover X in two stages: first sample Y from
marginal and compute E [X |Y ], then sample X from
distribution given Y value.
Above fact breaks variance into two parts, corresponding to
these two stages.
18.440 Lecture 26
Example
2
X and Y an
2
Y and write
18.440 Lecture 26
Outline
Conditional expectation
18.440 Lecture 26
Interpretation
18.440 Lecture 26
g (X ))2 ] is
Examples
18.440 Lecture 26
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these mateials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 27
Moment generating functions and
characteristic functions
Scott Sheffield
MIT
18.440 Lecture 27
Outline
Characteristic functions
18.440 Lecture 27
Outline
Characteristic functions
18.440 Lecture 27
I
I
I
18.440 Lecture 27
d
d
0
tX
dt M (t) = dt E [Xe ]
E [X 2 ]. Same argument
Also M 00 (t) =
= E [X 2 e tX ].
So M 00 (0) =
of M at zero is E [X n ].
18.440 Lecture 27
18.440 Lecture 27
18.440 Lecture 27
Other observations
18.440 Lecture 27
Examples
I
p)e 0 . In
18.440 Lecture 27
18.440 Lecture 27
2 /2
.
2?
I
I
I
t.
> 0 and
18.440 Lecture 27
1
.
(1+x 2 )
18.440 Lecture 27
Outline
Characteristic functions
18.440 Lecture 27
Characteristic functions
I
For example,
And
18.440 Lecture 27
aX (t)
X +Y
X (at)
Y,
just as MX +Y = MX MY .
Outline
Characteristic functions
18.440 Lecture 27
Perspective
I
18.440 Lecture 27
Continuity theorems
I
L
evys continuity theorem (see Wikipedia): if
limn!1 Xn (t) = X (t) for all t, then Xn converge in law to
X.
18.440 Lecture 27
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 28
Lectures 18-27 Review
Scott Sheffield
MIT
18.440 Lecture 28
Outline
18.440 Lecture 28
Outline
18.440 Lecture 28
I
I
I
I
18.440 Lecture 28
18.440 Lecture 28
I
I
I
18.440 Lecture 28
(a).
This is (b)
(a) = P{a X b} when X is a standard
normal random variable.
18.440 Lecture 28
Problems
I
I
I
I
18.440 Lecture 28
( 2.19).
I
I
I
Values:
( 3) .0013,
( 2) .023 and
and
( 1) .159.
18.440 Lecture 28
x a
0
=1
a.
e
0
a
dx =
18.440 Lecture 28
e
a
defined for
Defining
distribution
18.440 Lecture 28
Outline
18.440 Lecture 28
I
I
Suppose X is a random
variable with probability density
(
1
x 2 [, ]
function f (x) =
0
x 62 [, ].
Then E [X ] =
+
2
18.440 Lecture 28
)Y + ] = Var[(
)2 /12.
)Y ] =
I
I
I
I
18.440 Lecture 28
18.440 Lecture 28
Density: f (x, y ) =
18.440 Lecture 28
@ @
@x @ y F (x, y ).
18.440 Lecture 28
P{X + Y a} =
=
I
I
1
1
1
1
FX (a
a y
1
fX (x)fY (y )dxdy
y )fY (y )dy .
18.440 Lecture 28
Conditional distributions
18.440 Lecture 28
ANSWER: an .
18.440 Lecture 28
I
I
I
Are
Yes.
18.440 Lecture 28
Properties of expectation
I
I
I
I
I
I
I
18.440 Lecture 28
Properties of expectation
I
I
I
I
I
I
I
18.440 Lecture 28
A property of independence
I
I
I
18.440 Lecture 28
18.440 Lecture 28
Cov(X , Y ) = Cov(Y , X )
Cov(X , X ) = Var(X )
Cov(aX , Y ) = aCov(X , Y ).
Cov(X1 + X2 , Y ) = Cov(X1 , Y ) + Cov(X2 , Y ).
General statement of bilinearity of covariance:
Cov(
m
X
i=1
ai Xi ,
n
X
bj Y j ) =
j=1
m X
n
X
ai bj Cov(Xi , Yj ).
i=1 j=1
Special case:
Var(
n
X
i=1
18.440 Lecture 28
Xi ) =
n
X
i=1
Var(Xi ) + 2
(i,j):i<j
Cov(Xi , Xj ).
Defining correlation
I
E [X ]E [Y ].
Cov(X , Y )
Var(X )Var(Y )
Satisfies
18.440 Lecture 28
1 (X , Y ) 1.
I
I
18.440 Lecture 28
Conditional expectation
I
I
18.440 Lecture 28
f (x,y )
fY (y ) .
So
18.440 Lecture 28
Conditional variance
I
I
I
I
I
I
Definition:
Var(X |Y ) = E (X E [X |Y ])2 |Y = E X 2 E [X |Y ]2 |Y .
Var(X |Y ) is a random variable that depends on Y . It is the
variance of X in the conditional distribution for X given Y .
Note E [Var(X |Y )] = E [E [X 2 |Y ]] E [E [X |Y ]2 |Y ] =
E [X 2 ] E [E [X |Y ]2 ].
If we subtract E [X ]2 from first term and add equivalent value
E [E [X |Y ]]2 to the second, RHS becomes
Var[X ] Var[E [X |Y ]], which implies following:
Useful fact: Var(X ) = Var(E [X |Y ]) + E [Var(X |Y )].
One can discover X in two stages: first sample Y from
marginal and compute E [X |Y ], then sample X from
distribution given Y value.
Above fact breaks variance into two parts, corresponding to
these two stages.
18.440 Lecture 28
Example
2
X and Y an
2
Y and write
18.440 Lecture 28
18.440 Lecture 28
18.440 Lecture 28
Examples
> 0 then
18.440 Lecture 28
2,
2 /2
then
t.
Cauchy distribution
FX (x ) = P {X x } = P {tan x } = P { tan
1
1
1 x.
2 + tan
Find fX (x) =
18.440 Lecture 28
d
dx F (x )
1 1
1+x 2 .
1x }
Beta distribution
I
18.440 Lecture 28
a
a+b
(a 1)
(a 1)+(b 1) .
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these mateials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 30
Weak law of large numbers
Scott Sheffield
MIT
18.440 Lecture 30
Outline
18.440 Lecture 30
Outline
18.440 Lecture 30
P{|X
k}
.
k2
Proof: Note that (X )2 is a non-negative random variable
and P{|X | k} = P{(X )2 k 2 }. Now apply
Markovs inequality with a = k 2 .
18.440 Lecture 30
P{|X
I
I
I
k}
k2
18.440 Lecture 30
n
Then the value An := X1 +X2 +...+X
is called the empirical
n
average of the first n trials.
Indeed, weak law of large numbers states that for all > 0
we have limn!1 P{|An | > } = 0.
18.440 Lecture 30
Similarly, Var[An ] =
By Chebyshev P |An
n 2
n2
=
|
2 /n.
Var[An ]
2
n 2
18.440 Lecture 30
Outline
18.440 Lecture 30
n
Is it always the case that if we define An := X1 +X2 +...+X
then
n
An is typically close to some fixed value when n is large?
What if X is Cauchy?
But in this case E [|X |] was infinite. Does the weak law hold
as long as E [|X |] is finite, so that is well defined?
18.440 Lecture 30
Characteristic functions
I
For example, X +Y = X
and Y are independent.
And
18.440 Lecture 30
aX (t)
X (at)
Y,
just as MX +Y = MX MY , if X
Continuity theorems
I
I
n!1
I
Xn (t)
X (t)
18.440 Lecture 30
I
I
I
I
g ( nt )
t
n
= 0 if t is fixed.
18.440 Lecture 30
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these mateials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 31
Central limit theorem
Scott Sheffield
MIT
18.440 Lecture 31
Outline
18.440 Lecture 31
Outline
18.440 Lecture 31
n!1
I
I
I
I
(a).
Here (b)
(a) = P{a Z b} when Z is a standard
normal random variable.
Sp
n np
npq describes number of standard deviations that Sn is
above or below its mean.
Question: Does a similar statement hold if the Xi are i.i.d. but
have some other probability distribution?
Central limit theorem: Yes, if they have finite variance.
18.440 Lecture 31
Example
I
I
I
What is Var[X ]?
18.440 Lecture 31
Example
I
What is E [X ]?
What is Var[X ]?
18.440 Lecture 31
General statement
I
I
I
n!1
18.440 Lecture 31
(a).
Outline
18.440 Lecture 31
For example, X +Y = X
and Y are independent.
And
18.440 Lecture 31
aX (t)
X (at)
Y,
just as MX +Y = MX MY , if X
18.440 Lecture 31
Continuity theorems
I
L
evys continuity theorem (see Wikipedia): if
lim
n!1
Xn (t)
X (t)
18.440 Lecture 31
n( pt )2 /2
n e
n
But e
= et
2 /2
t
e
as n tends to infinity.
18.440 Lecture 31
2 /2
18.440 Lecture 31
Y (t)
Now Bn is
So
Bn (t)
ng ( pt
p1
n
=
)
So
n e
n
But e
=e t
2
to e t /2 as n tends to infinity.
18.440 Lecture 31
Y (t).
2 /2
Perspective
I
I
I
18.440 Lecture 31
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 32
Strong law of large numbers and Jensens
inequality
Scott Sheffield
MIT
18.440 Lecture 32
Outline
Jensens inequality
18.440 Lecture 32
Outline
Jensens inequality
18.440 Lecture 32
18.440 Lecture 32
I
I
18.440 Lecture 32
How would you advise Pedro to invest over the next 10 years
if Pedro wants to be completely sure that he doesnt lose
money?
18.440 Lecture 32
We wrote Tn = R1 . . . Rn . P
Taking logs, we can write
Xi = log Ri and Sn = log Tn = ni=1 Xi .
Now Sn is a sum of i.i.d. random variables.
E [X1 ] = E [log R1 ] = .53(log 1.15) = .47(log .85) .0023.
By the law of large numbers, if we take n extremely large,
then Sn /n .0023 with high probability.
This means that, when n is large, Sn is usually a very negative
value, which means Tn is usually very close to zero (even
though its expectation is very large).
Bad news for Pedros grandchildren. After 100 years, the
portfolio is probably in bad shape. But what if Pedro takes an
even longer view? Will Tn converge to zero with probability
one as n gets large? Or will Tn perhaps always eventually
rebound?
18.440 Lecture 32
Outline
Jensens inequality
18.440 Lecture 32
n
Then the value An := X1 +X2 +...+X
is called the empirical
n
average of the first n trials.
Recall: weak law of large numbers states that for all > 0
we have limn!1 P{|An | > } = 0.
18.440 Lecture 32
| > } P{Y
n.
n}.
So limn!1 P{|An
If the right limit is zero for each (strong law) then the left
limit is zero for each (weak law).
18.440 Lecture 32
n} = 0.
18.440 Lecture 32
Outline
Jensens inequality
18.440 Lecture 32
Jensens inequality: E [g (X )]
g (E [X ]).
g is convex), then
18.440 Lecture 32
I
I
I
I
C , 0}.
18.440 Lecture 32
Perspective
I
18.440 Lecture 32
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Markov chains
Examples
Ergodicity and stationarity
18.440: Lecture 33
Markov Chains
Scott Sheffield
MIT
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
Outline
Markov chains
Examples
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
Outline
Markov chains
Examples
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
Markov chains
I
Precisely,
P{Xn+1 = j|Xn = i, Xn
= in
1 , . . . , X1
= i1 , X0 = i0 } = Pij .
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
Simple example
I
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
Matrix representation
I
I
B
B
B
A=B
B
B
@
1
C
C
C
C
C
C
A
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
I
I
An
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
@
A
(n)
(n)
(n)
P
PM1 . . . PMM
M0
P
P
... P
M0
M1
MM
18.440 Lecture 33
1n
C
C
C
C
C
C
A
Markov chains
Examples
Ergodicity and stationarity
Questions
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
Outline
Markov chains
Examples
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
Simple example
I
.5 .5
A=
.2 .8
Note that
2
A =
I
Can compute
18.440 Lecture 33
A10
.64 .35
.26 .74
.285719 .714281
.285713 .714287
Markov chains
Examples
Ergodicity and stationarity
Its complicated
Single
Married
I
I
I
Engaged
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
Outline
Markov chains
Examples
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
I
I
0 1 . . . M
18.440 Lecture 33
Markov chains
Examples
Ergodicity and stationarity
Simple example
I
If A =
.5 .5
.2 .8
A =
I
, then we know
0 1
.5 .5
.2 .8
0 1
= .
.5 .5
A = 2/7 5/7
= 2/7 5/7 = .
.2 .8
Recall
that
.285719 .714281
2/7 5/7
10
A =
=
.285713 .714287
2/7 5/7
18.440 Lecture 33
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Entropy
Noiseless coding theory
Conditional entropy
18.440: Lecture 34
Entropy
Scott Sheffield
MIT
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
Outline
Entropy
Conditional entropy
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
Outline
Entropy
Conditional entropy
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
What is entropy?
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
Information
I
I
I
I
I
I
for each x 2 S.
Since there are 2k values in S, it takes k bits to describe an
element x 2 S.
Intuitively, could say that when we learn that X = x, we have
learned k = log P{X = x} bits of information.
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
Shannon entropy
I
n
X
i=1
pi ( log pi ) =
n
X
pi log pi .
i=1
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
Other examples
I
n
X
i=1
pi ( log pi ) =
n
X
pi log pi .
i=1
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
Outline
Entropy
Conditional entropy
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
B $ 01
C $ 10
Or by
D $ 11
A$0
B $ 10
C $ 110
I
I
I
D $ 111
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
ni p(xi )
H(X ) =
N
X
i=1
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
Outline
Entropy
Conditional entropy
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
H(X , Y ) =
XX
i
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
Conditional entropy
I
I
I
I
I
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
P
Definitions:PHY =yj (X ) =
i p(xi |yj ) log p(xi |yj ) and
HY (X ) = j HY =yj (X )pY (yj ).
18.440 Lecture 34
Entropy
Noiseless coding theory
Conditional entropy
I
I
P
Definitions:PHY =yj (X ) =
i p(xi |yj ) log p(xi |yj ) and
HY (X ) = j HY =yj (X )pY (yj ).
Important property two: HY (X ) H(X ) with equality if
and only if X and Y are independent.
In words, the expected amount of information we learn when
discovering X after having discovered Y cant be more than
the expected amount of information we would learn when
discovering X before knowing anything about Y .
P
Proof: note that E(p1 , p2 , . . . , pn ) :=
pi log pi is concave.
The vector v = {pX (x1 ), pX (x2 ), . . . , pX (xn )} is a weighted
average of vectors vj := {pX (x1 |yj ), pX (x2 |yj ), . . . , pX (xn |yj )}
as j ranges over possible values. By (vector version of)
Jensens inequality, P
P
H(X ) = E(v ) = E( pY (yj )vj )
pY (yj )E(vj ) = HY (X ).
18.440 Lecture 34
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 35
Martingales and the optional stopping
theorem
Scott Sheffield
MIT
18.440 Lecture 35
Outline
18.440 Lecture 35
Outline
18.440 Lecture 35
Definitions
I
18.440 Lecture 35
Martingale examples
I
I
I
I
18.440 Lecture 35
18.440 Lecture 35
Outline
18.440 Lecture 35
Precisely, if you buy the asset at some time and adopt any
strategy at all for deciding when to sell it, then the expected
price at the time you sell is the price you originally paid.
18.440 Lecture 35
18.440 Lecture 35
18.440 Lecture 35
I
I
18.440 Lecture 35
Martingale examples
18.440 Lecture 35
Examples
18.440 Lecture 35
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 36
Risk Neutral Probability and Black-Scholes
Scott Sheffield
MIT
18.440 Lecture 36
Outline
Black-Scholes
18.440 Lecture 36
Outline
Black-Scholes
18.440 Lecture 36
Overview
I
18.440 Lecture 36
I
I
rT .
18.440 Lecture 36
At first sight, one might think that PRN (A) describes the
markets best guess at the probability that A will occur.
Answer: PRN (A) < .5. People are risk averse. In second
scenario they need the money more.
18.440 Lecture 36
Non-systemic event
Suppose that A is the event that the Boston Red Sox win the
World Series. Would we expect PRN (A) to represent (a
market assessment of) true probability in that case?
18.440 Lecture 36
18.440 Lecture 36
Listener: Then why not make some bets and get rich? If your
estimates are so much better, law of large numbers says youll
surely come out way ahead eventually.
18.440 Lecture 36
Prices as expectations
I
I
I
I
I
I
18.440 Lecture 36
Outline
Black-Scholes
18.440 Lecture 36
I
I
I
I
rT
= ERN [g (e N )]e
rT
2.
I
I
rT
18.440 Lecture 36
log K ]
X
ln( K0 )+(r +
p
18.440 Lecture 36
2
)(T )
2
(d1 )X0
and d2 =
(d2 )Ke
X
ln( K0 )+(r
p
rT
2
)(T )
2
where
18.440 Lecture 36
18.440 Lecture 36
I
I
18.440 Lecture 36
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 37
Review: practice problems
Scott Sheffield
MIT
18.440 Lecture 37
18.440 Lecture 37
1
8
+ 56
1
56
= 2.
18.440 Lecture 37
Conditional distributions
Roll ten dice. Find the conditional probability that there are
exactly 4 ones, given that there are exactly 4 sixes.
18.440 Lecture 37
I
I
Ratio is
18.440 Lecture 37
6
4
42 /56 =
6
4
( 15 )4 ( 45 )2 .
18.440 Lecture 37
E [E 2 ] = 2 and Cov[E , V ] = 0.
18.440 Lecture 37
(2+1)x
for
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 38
Review: practice problems
Scott Sheffield
MIT
18.440 Lecture 38
Order statistics
18.440 Lecture 38
Var[X 2 ] = E [X 4 ] (E [X 2 ])2
Z 1
Z 1
1 4
1 2
1 1
4
=
x dx (
x dx)2 =
= .
2
2
5
9
45
1
1
Note that for x 2 [ 1, 1] we have
Z 1
1
1 x
P{X > x} =
dx =
.
2
x 2
If x 2 [ 1, 1], then
P{min{X1 , . . . , Xn } > x}
x
2
)n .
18.440 Lecture 38
Write Y =
Pn
i =1 Xi /n.
Then
MY (t) = E [e tY ] = E [e t
18.440 Lecture 38
Pn
i=1
Xi /n
] = (MX (t/n))n .
Entropy
18.440 Lecture 38
Entropy answers
18.440 Lecture 38
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
18.440: Lecture 39
Review: practice problems
Scott Sheffield
MIT
18.440 Lecture 39
Markov chains
I
I
I
18.440 Lecture 39
I
I
18.440 Lecture 39
1
9
18.440 Lecture 39
25 before the
I
I
p
p
25 25
18.440 Lecture 39
Martingales
Yn
Yn
Yn
Yn
18.440 Lecture 39
Pn
= Pi=1 iXi
n
= Q i =1 Xi2 n
n
= Qi=1 (1 + Xi )
n
= i=1 (Xi 1)
Martingales
18.440 Lecture 39
E [e 3X 3 ].
E [e X 1X 2(a,b) ] for fixed constants a < b.
18.440 Lecture 39
E [e
3X 3
]=
Z
Z
1
1
1
e 3x
1
p e
2
x 2 /2
dx
x 2 6x+6
1
2
p e
dx
2
1
Z 1
x 2 6x+9
1
2
p e
=
e 3/2 dx
2
1
Z 1
(x 3)2
1
2
p e
= e 3/2
dx
2
1
= e 3/2
18.440 Lecture 39
1
2
e x p e x /2 dx
2
a
Z b
x2
1
=
e x p e 2 dx
2
a
Z b
x 2 2x +1 1
1
2
p e
=
dx
2
a
Z b
(x 1)2
1
2
p e
= e 1/2
dx
2
a
Z b 1
x2
1
p e 2 dx
= e 1/2
2
a 1
E [e 1X 2(a,b) ] =
= e 1/2 ( (b
18.440 Lecture 39
1)
(a
1))
Farewell
18.440 Lecture 39
MIT OpenCourseWare
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.