3E3 (Probability and Statistics):

Quiz
Anthony Quinn

Thursday 7th November 2019 (14:00–14:50) Goldsmith Hall

Instructions:

• The duration of the quiz is FIFTY MINUTES.

• Answer ALL QUESTIONS. They carry equal marks.

• Enter your answers directly onto this test paper.

• Enter your student number below. Enter your name, student number and signature
at the bottom of the last page, and fold over.

Student Number:

1
Q1: Consider three propositions, S1 , S2 and S3 , such that S1 is a necessary condition
for S2 . Also, S1 is a necessary condition for S3 . There are no other logical constraints.
Furthermore:

Pr[S1 ] = 0.7,

Pr[(S2 ∧ S3 ) | S1 ] = 0.4.

Complete your probability model in any consistent way, and hence evaluate

Pr[S1 | S2 ∨ S3 ].

2
Q2: Consider three Bernoulli nodes (i.e. random variables), C, E1 and E2 , such that E1
and E2 are conditionally independent and identically distributed given C. Furthermore:
1
Pr[C = 1] = ,
4
1
Pr[E1 = 1 | C = 1] = Pr[E1 = 0 | C = 0] = .
3
Evaluate Pr[E2 = 0 | E1 = 0].

3
Q3: The probability parameters of a 4-state homogeneous Markov chain are:
 2
0 61 1
  1 
3 6 2
   
 1 2

1 
 1 

 6 3
0 6 
 
 4 
T=  1
 , p1 =   ,
  1 
1 2

 6 6 3
0 

 
 8 
   
1 1 2 1
0 6 6 3 8

where, as usual, T is the transition probability matrix and p1 is the initial probability
vector.
Evaluate the probability that the first and third symbols have the same state.

4
Q4: Each component mass-produced by a factory is faulty with probability 0.1. In a
quality control exercise, a sample of 99 of these components is examined.

(i) Most probably, how many components will be faulty in this sample, and what is
this probability?

(ii) What is the least probable number of faulty components in this sample, and what
is this probability?

5
Q5: Images in a database are each classified into one of four classes, Cj , j = 1, 2, 3, 4,
where Pr[C1 ] = 0.5, Pr[C2 ] = 0.3 and Pr[C3 ] = 0.15. Consider the next four classified im-
ages. Write down an expression—which you do not need to evaluate—for the probabilty
that fewer of these four images will be classified as C1 than C2 .

6
Q6: Cars pass a particular point on a two-way, east-west road in both directions, with
the average rate being 5 cars per minute from east to west, and 3 cars per minute from
west to east. Given that altogether 7 cars passed this point in the last minute, what is
the probability that the majority of these pased from east to west?

7
Q7: A mobile phone user checks her phone on average 1.2 times per minute during a
particular period of the day. Consider the following three propositions relating to her
usage at this time of the day:
S1 : she checks her phone once in the next minute;
S2 : she checks her phone once in the second-next minute;
S3 : she checks her phone once in the interval of time between 90 seconds and 150 seconds
from now.
Demonstrate that these three propositions are not independent.

Name:

Student Number:

Signature:

8
 313 Annual Quiz 2019 Solutions 9 1119
Q1

3
u 22

Prfs Szasz a
Bfsi 53 5 Prk 0.4 0.7 s 0.28

partition 5 4 prob parameters orb 2 specified two

more requiredG.s.in red

15 n
sing 0.22

serfs Isis FIT 42

Oc e
1
L N

I 0

I To 73
E
11
s
2
3 43

BE Ben I Be
Pfc II D BE j s's
Isis
Pfc.nl ojE7zyj ys

prff.io E oJETPPrfEz o c iJPrfcafE.so tPrfEz o c o Pr eo E o

2g t Ix
element

77 144

Pr proposition It If 3

4
N Binomial n 99 o
p
D
Dp to modes are at N 9 and N 10

RIN 9 Vio
In p 2x Prfw b n p

2 193 ix 989

Binomial monotonically decreasing away from models

 min probability ether at N D or N 99 obviously the
latter

QIN 9 n e 3 P I 199

Green Ps fn In po 95 ix 999 I

IN k A Nih NN5hshnyshytn 4 es 3gs

Multinomial n e
Pr Proposition
RIN 0 A Nil PIN AN 2 t ftp.oANE3
tPrfNioANz 4JtPrfNilANz 2 tPrfN lNNz 3
4 o 3 I
Ix 3 z4 5 37 It
x as x 3x 2 t
o to 0 3 1

to f Ex 34 x I t is x 35.21
to
54.33 x L
QI Jew 5 min
y 5 3 8 min
use 3 min

Prfi ezwft4ANwzeft37V n.V NewDANMorrell Nf 71

flows w at wze are independent

e III x e E e te e tessie
8 1 18 137
e
7
 Tutorial 5 i
Solutions 22 10 19
22
M she misses a lecture
Prfm 21.520 V L2 on 521

Lecture
p p
Student
is in chooses
M27 M 20

f x L t f x s s

Lex Mi ist 5 denote the sequential proposition that she

misses the Ah next lecture These are iid BTs with ps's
s

RIN 31ns p d
P n
the no of
missedlectures
among the next 5
attempts
23
s A 133,43 15,6
U
12,3435 15,6
U
12,3436 15,6
U
countably infinite

344
Prfs
26
Denote N k the no of error free days is 3h
4 0,1 2

0.95k
Prf N k m

Morning shift worker

afternoon
7
0.9k
BIN 3h A

PrIN h N a 85
night 9
Totp
0ghx Fo 0.8k x
Prf N 3h 0.95 s x t
 27
Ri E
f 2 3 4 4 Candidates i l 2 12 42voters

Assume voters act independently of each other nos of votes for

each candidate N Nr Nz Ny can be modelled as multinomial
with
a 12

B
a

3 313 3

Prf RIN Nz Ns Nys 3 E

O Ol

b
Prf P
In 7 7 E 3 binomial amputation

I interpreting the
am
proposition asmeaning that
I receives the majority of 12votes
rather than assuming it means she receives
more than each of the others in which case we must
measure evaluate the probability of ma More elementary
propositions

Pr exactly 1 of the candidates receives 0 Votes

The difficulty we face in computing this is that there are combinatorically

many partitions of 12 into four numbers satisfying the stated
constraints
0228 07 I 4 I 290

An easiertask is to attempt to find a fairly tight upper bound for

the required probability via the union bound lemma i
 Prf I Pr a
soft PRICEY Prf so Pr for D
EET
receives 0 votes

four NOT mutually exclusive propositions

e.g o o o n satisfies

Prfc so
to 13 binomial the first 3 propositions

Prfcaso 5 E Pfosso Rfa so

Pr n
s 3 t 3 E 0.21
28 i

ri G C A T with the marginal stationary probability

vector
being Bs
J
Clearly there will be dependence between proximate bases that's
the nature of DNA code will assume for great
so we

simplification that bases are probed at well separated points along

the strand the realizedbases at each probing point are
independent with identical multinomial probabilities Bs above

we can ahopt the multinomial probability law to measure

compute probabilities of count prepositions

i 44 least probable PrlTtt TJ f

most probable
Prf6666 4g

Pr n
Prfan permutation of thebases

4 x
F x F x
3

f is Prf F geometric prob law

Pr
iv I PYNE
syn 3
n
6 Pc
s
i
 It would be best to assume this means atleast 2 C's
at atleast 2 A's might have been d d eh less ambiguously

2 2 6
Prf Na
n n
c

Prf i 2 Novo 2
I
a
nose

Pr 3 2,1 t
Prf2,3 I t
Prf4 2,0 t Prf2,4 J t Prf3,3OJ

is
ta5f5f5i
aI tI i t a

µ Interpreting consistently

Aldo l n
Nca Naa

Prf1,1 I 2 t Prf2,1 I l t 8 1,2 I l t

Alli2,1
Prf3,1 I o t 8 1,3 I s t 041,1 3,0

Pr 2,2 l o t 8 2,12,0J t Prfl 2,2 J

n ia 13513 t t t

Pr n
Prf6666 v Cccc v
AAAA v TT TT

HI't t
Lviii Pr n
I Ps any way of partitioning 6 bases in 4
numbers all different

The only way to partition 6 in this way is into

6 0 t l t Z t 3

Pr of bases permuting 0,1 2,3

n
I Prf any numbers

There are 4 24 distinct

24 permutations mutually exclusive

propositions contradicting the proposition of interest

Prfnj l fPrf o oJnfncsDnfNn.s 2 n n 3

Prf 1,0 2,3 t.fr 0 2

1,3 t t
Pf3,21,0
i
f pop a o 3
 3E3 Tutorial 5 (Sequential Experiments with
Independence)
Anthony Quinn
11th October 2019

Q1: Consider a digital source generating length-5 quaternary (i.e. four-state) words.
How many distinct such words are there, and under what assumptions is it valid
to assign equal probability to each?

Q2: Three of the weekly lectures in a course take place in M20, the 4th in M21. A
particular student does not read the timetable, and so randomly chooses one of
these venues for each lecture. What is the probability that she misses at least 3 of
her next 5 lectures?

Q3: A gambler repeatedly throws a die until he observes a 5 or a 6. Specify the set of
events associated with the proposition, S =‘he throws the die more than 4 times’,
and evaluate Pr[S].

Q4: A mechanical system comprises 5 components, each of which is operating with

probability 0.8. What is the probability that (i) none, (ii) all, and (iii) the majority
of the switches are not functioning?

Q5: The sequential output of a binary source is observed until a 1 is released. Consider
propositions “N = k” meaning “the position of the first 1 is k”, so that k =
1, 2, 3, ....

(a) Find Pr[N > k], stating all assumptions.

(b) Find an expression for Pr[N > k|N > k0 ] as a function of k and k0 .
(c) Formulate a proposition that is independent of the proposition, “N > k0 ”.

Q6: A particular factory has 10 employees, 5 being morning-shift employees, 3 afternoon-

shift employees, and 2 night-shift employees. The probability of an employee mak-
ing an error during their shift is, respectively, 0.05, 0.1 and 0.2, for the three shifts
respectively. Stating your assumptions, evaluate and sketch the probability that the
number of consecutive shifts for which an employee of this factory works error-free
is k ≥ 0, 1, 2, ....

Q7: 4 candidates stand for election before an electorate of 12 voters. Historically, the first
candidate has been twice as popular as any of the others. Stating any assumptions,
and justifying any probability model you use, compute

(i) the probability that all the candidates receive the same number of votes;

1
 (ii) the probability that the first candidate receives the majority of the votes;
(iii) the probability that only 3 of the 4 candidates receive votes.

Q8: The CRISPR gene editing technology probes a DNA sequence at various points,
recording the base (G, C, A or T) at each point. For this sequence, it is known
that the frequencies of occurrence of the bases are in a ratio of 4:2:2:1, respectively.
Imposing and stating any reasonable assumptions, address each of the following:

(i) How many distinct recordings of length 4 are there? Which is the most prob-
able, and which the least? Evaluate the probability of each.
(ii) What is the probability that each base appears in a length-4 recording?
(iii) What is the probability that the first T appears somewhere between the 5th
and the 10th points?
(iv) What is the probability that at least 2 C’s appear in a length-6 recording?
(v) What is the probability that 2 C’s and 2 A’s appear in a length-6 recording?
(vi) What is the probability that a G, a C and an A appear in a length-5 recording?
(vii) What is the probability that a length-4 recording contains only one base?
(viii) What is the probability that a length-6 recording contains the same number
of certain bases?
 3E3 Tutorial 4 Solutions 3 12 19

3 D element of T
a
PrfSz B
Is R

L's t
toy
b There are 3 6 elementary propositions making this true

being all the permutations of R G and B Gives the probability

parameters the States are indistinguishable

Pfm 6 x Prfan elon proposition

6 x Pr Rko x
Hal B x A B

G x f x t x

Psfs Is prfsit.lsi.pt PrfFftPrfSi

i Si NFJ
hyJPr
gives he gives he hoes
forgets not forget

Fj Ip x t TNF I
p o
n
his new I above
dynamics must be
Amc etciteyspecifeh
Specify T s

I I column sums si
Then revert to a ash b proceeding as before In b the
States remain indistinguishable

IE E E a Ex's Exit
45
144

b 6 x x zx f
 Qb
10100101010001 for example

sl
I Rfs sit 2

elicitingIspecifying
ftp si Pr si oJ

Llb7Prfm
l Prf3bitsequaY

l 2xfx'gxtg I

a as above a

8 10 bit word PRES Is.is xPr Si s si a x xPrsi siJPr si

p
9 transitions note as in

The maximal value is F7th occurring when there are 9

toggles 0101010.101 or lo 10

value is 9
The minimal 5 2 occurring when there are no toggles
0000000000 or Il 11
 3E3 Tutorial 4 (Sequential Experiments with
Dependence, and Markov Chains)
Anthony Quinn
8th October 2019

Q1: Students can take as many as three exams in a particular subject, in an attempt
to gain credits for that subject. In the summer exam, the pass rate is 70%, it’s
60% in the autumn, and 30% in the ‘specials’. Stating any assumptions, answer
the following:

(a) What is the probability that a student gains credits in this subject?
(b) In a class of 100 students, what is the probability that exactly 15 gain credits
in the autumn?

Q2: A certain football team has a 50:50 chance of making it through the first round of
a league, and a 60% chance of making it through the next round to the final. If it
gets to the final, it has a 70% chance of winning.

(a) Set up a probability space for this context, specifying the three elements of the
probability triple.
(b) In a sense, this is a sequential experiment. Define the sample space for each
of the sub-experiments.
(c) Find the probability that the team will reach the final.

Q3: Consider a sequential experiment with sample space, Ω. The sample space for
the ith sub-experiment is Ωi , i = 1, . . . , n. Bearing in mind your experience in
the previous questions, comment on the general relationship between ]Ω and ]Ωi ,
i = 1, . . . , n.

Q4: Rafael returns the tennis ball five times more often than not. In this respect, he is
50% better than Marin. Propose a simple two-state dynamic for the ball, specifying
all parameters.

(a) If Rafael currently has the ball, what’s the probability that he also has it after
two strokes?
(b) What is the probability that Marin has the ball after 10 strokes?

Q5: A student owns 3 pairs of socks: red, green and blue ones. He tries not to wear the
same pair two days in a row.

(a) If he wears the red pair on the first day, what is the probability he wears the
blue pair on the third?

1
 (b) What is the probability that he wears all three pairs on three consecutive days?
(c) Re-evaluate your answers to (a) and (b) if he forgets his intention half the time.

Q6: A ternary homogeneous Markov chain, with states denoted by Ωi = {ω1 , ω2 , ω3 },

respectively, i = 1, 2, 3, ..., has transition matrix:
 
.8 x .1
T =  .1 z y  .
.1 .3 .6

(a) Choose any appropriate values for x, y and z.

(b) Specify exactly the operations needed (say in Matlab) to evaluate p6 if (i)
p1 = [1/3 1/3 1/3]0 , and (ii) p1 = [1 0 0]0 , respectively.
(c) Evaluate Pr[S6 = ω3 |S1 = ω1 ], where Si denotes the state of the process at
sample i.

Q7: A homogeneous Markov chain has transition matrix:

0 12 0 1
 
 0 0 1 0 
T= 2
 1 1 0 0 .

2
0 0 12 0

(a) Draw the state transition diagram.

(b) If the Markov chain is in the 4th state at sample (i.e. sub-experiment) i = 5,
draw the trellis diagram for the next 4 steps, and hence compute the condi-
tional probability of each of the 4 states at i = 9.
(c) Verify your results in (b) directly via T (use Matlab).
(d) If the Markov chain is in state 3 at i = 3, then compute the probability that
it was in state 2, two steps earlier. State any assumptions that you make.
(e) Can you guess what the behaviour of the Markov chain is in the long-run?
Validate your ideas using Matlab.

Q8: An object moves between regions A, B and C, being reflected always from A to B,
and absorbed by C. Complete your Markov chain model of these dynamics.

(a) Intuitively, what is the long-run behaviour of this system? Can you verify this?
(b) If the object does not start in C, what is the probability it is still not there
after 4 steps?
(c) It is noted that the object is in region B after 2 steps. Re-evaluate your last
answer.

Q9: In Q7, let there now be four regions, A, B, C and D, such that A is reflective (into
B), and D is also reflective (into C). You identify a drift in the dynamics, with the
object advancing (in the direction of higher-lettered regions) 10% more often than
retreating. Repeat Q7 in this case.
Q10: A binary source in telecommunications (i.e. a sequential generator of 1s and 0s)
toggles twice as often as not.

(a) Suggest typical outputs from this source.

(b) What is the probability that 2 bits, if separated by 3 bits, are equal?
(c) What is the probability that the bits in a 3-bit word are not all equal?
(d) What is/are the most probable (least probable) 10-bit word(s)?

Q11: In a computer game, the square screen of side l is divided into a number of regions:
(i) the inside and outside of a centred circle, C, of radius 4l ; (ii) the inside and outside
of a centred circle, c, of radius 10l ; and (iii) the upper-half, H, and lower-half screens.
The dynamics of the ball during one-second intervals are studied and the following
are noted:

1. If inside c at the start of the interval, the ball always appears outside c but
inside C at the end of that interval; if inside C initially, it appears inside c a
second later, in 10% of cases;
2. If inside C initially, it is still there a second later, in 30% of cases; if outside
C, it stays there in 80% of cases;
3. If inside H initially, it is still there a second later, in 70% of cases; if outside
H, it stays there in 40% of cases;

Stating any assumptions that you make, specify an appropriate Markov chain model
for the dynamics of the ball. Hence, specify the computations needed—in Matlab
for instance—to evaluate each of the following probabilities:

(a) if the ball starts inside c, the probability that it appears inside C after 5
seconds;
(b) if the ball starts inside c, the probability that it appears outside H after 5
seconds;
(c) the probability that the ball appears inside H but outside C after 5 seconds;
after 10 seconds.

Q12: [HARD] A carpark with 5 places has a barrier control system which ensures that
a maximum of one car can enter in any one-hour period. There is no restriction
on the number of cars that can leave. There is no overnight parking allowed, and
the operating hours of the carpark are from 08:00 to midnight. In any hour, the
probability of there being a car seeking entry is 0.1 and, if admitted, it will not
leave within the hour. The probability that any previously parked car leaves in the
next hour is also 0.1. Specify a model for the number of cars in the carpark at any
hour of the day, stating your assumptions. Using Matlab judiciously, address each
of the following:

(a) What is the probability that there are no cars in the carpark at noon?
(b) If there are 3 cars at 14:00, what is the probability that the carpark will be
full at 17:00?
 (c) If the carpark is full at 17:00, what is the probability that there were at least
3 cars at 14:00?
(d) What is the probability that the carpark is not full at midnight?

Q13: [EASIER] Address Q12 again, where the only change in wording is ”The proba-
bility that a car leaves in the next hour is also 0.1”. Criticize this model.
 353 Tutorial 3 7 11 19

23
a
Prfl I I V O o o

Prf 1104114041 t 0401904401819

2
3 43 xp t 43 43 4 P
41g
b Prf I 9g

PrifB 1 Bz o BI RIB 1 a Bz o

PrlBz o

V
Numerator Prf I I O I o o

sprfol.IR iljerlytPrplojPrplDPrlD
P 3
Y3x43xptYzxY3xp
Denominator PslI I o v l o o v Ost o v Osoof
73 73 xp t 43 5 xp t 13 43 fp t 4343 4 p

Ya s e

Answer so we need pep i to specify the

answer
QI
he fE m t

Re
fi 2,34
iii

ER Isr X B RI is the most probable

b
PfH1 Rl GIRI117 Prl't 05 x I
0.02
72g
Prf
 210 L o s N o 3 I as

Os TE 5

5 S TE lo
AT

t is

2 0.2
12 partition sets at most 11 probability parameters required
Those parameters 6 specified in the question are indicated
At least the following 2 parameters must be elicited specified
o 06 o 02

Pyo te 51 I Pr I 0.1 0.2 0.02

t.io
must be E K
eg O l readyspecified
0.2

TES N PRINT 0.2 0.3 0 06

Must be E I t
e g O2 A seat
specified
0.3

Under these assignments Psfo Te 5 1 0.5 t 0.06 t 0.02 0.33

This inference requires at least specification of the probability split

across the two propositions whose event set is

prfoaieis.INPrlN
must be E Ya f Tt2f
o 03
e g 0.4 Specified 0.3
0.4 0.3 0.12
Similarly we at theleast of it must split
the probability specified here as shown o z
1
0.05
x 0.5
eg Oy x 0.5 I 0.05

With this minimal parameterization our inverse inference is

ss
814tsisf o.rs oz z o

Note Later in the course when we have encountered the exponential

probability model for timing data we may choose to model the

times as conditionally exponential as follows s
s l l 1
TIL n
Expp d s.tn BIT 51L e
p 2
I
Jio 2h2

TN
1 IN Exp in s.tn
BIT lol N e If In oi 2

1 II n
Expt s.tn Prftsis I e In 2
GIVEN
BI PIT 15
4 Prf
Then PrfLIT is

Prft is

5
I 0.5

e Io s t e
n
Io.rs t e 0.2
3
2 0.5
0 19
3 5
2 0.5 t 2 0.3 t 2 xD 3
212

2 I 15 Max noof probability parameters if no

stochastic find logical
A sufficient way to impose these constraints
stochastic constraints is via the
model

13 pps

8
S H 53 4 52

531st
I

ss n San Sznsy

Psfs nsyszns.dk sznSD

w
Il Prayer
8 9 nsa 8 5453NSD
64
11

Hsfsa x Prky
 3E3 Tutorial 3 (Conditional Modelling and
Independence)
Anthony Quinn
3rd October 2019

Q1: There are two coins, one unbiased, one showing heads twice as often as tails. A coin
is chosen and tossed.

(a) What is the probability of heads?

(b) If the result of the exercise is heads, what is the probability that it was the
unbiased coin that was tossed.

Q2: In a gambling game of two steps, a player tosses a coin. If heads, she then rolls
a die. If tails, she then tosses the coin again. Stating your assumptions, evaluate
each of the following:

(a) the probability that she tosses heads;

(b) the probability that she rolls a six;
(c) if the game changes, and she tosses the coin until she gets to roll the die, what
is the probability that she rolls a six as her eleventh action?

Q3: A digital source releases 3-bit binary words. In any word, each bit is twice as
probable to have the same value as the preceding one than otherwise. Stating any
other assumptions, find

(a) the probability that all three bits in a word are equal.
(b) the probability that not all three bits are equal.
(c) the probability that the first bit is 1 if the last bit is 0.

Q4: A computer game can be played a one of three levels: easy, medium and hard. 30%
of players play at the easy level and 60% at the medium level. All players have a
game-assigned skill level, ranging from 1 (super) to 4 (awful). 50% of easy-level
players have rank 1, 20% of medium-level players have this rank, and 5% of hard-
level players have this rank. At each level, the numbers achieving the other ranks
are about equal.

(a) What is the most probable rank for a player of this game?
(b) What is the probability that a rank-1 player is playing at the hard level?

Q5: Prove that two mutually exclusive propositions cannot be independent unless at
least one of them has probability zero.

1
Q6: A dart is thrown at a dartboard of radius one unit. Consider the two propositions,
S1 ≡ “the dart lands at most a half a unit from the centre/bullseye”; and Sx ≡
“the dart lands between 41 and x units from the centre/bullseye”. Stating your
assumptions, deduce a value for x such that these propositions are independent,
and interpret x geometrically.

Q7: Two real numbers are selected at random in the interval [0, 1]. Consider the propo-
sitions:

• S1 ≡ “first number is > 12 ”

• S2 ≡ “second number is > 12 ”
• S1 ≡ “first number is > second number”

(a) Construct propositions that, together, induce a partition of Ω. These proposi-

tions should involve (i) only S1 , (ii) S2 and S3 , and (iii) S1 , S2 and S3 .
(b) Prove that S1 is not independent of S3 , but that S1 is independent of S2 .

Q8: Consider three propositions, S1 , S2 and S3 , with probabilities 0.7, 0.6 and 0.6 re-
spectively. Given S3 , the first two propositions have probabilities 0.9 and 0.8 re-
spectively. S1 and S2 are jointly true with probability 0.72 if S3 is true, and they
are jointly true with probability 0.2 if S3 is false. Verify that S1 and S2 are condi-
tionally independent given S3 , but they are not (marginally) independent. Deduce
the probability of all the propositions inducing the partition of the sample space.

Q9: 100 patients participate in an experimental drug trial. 60 are women. After 6
months, the drug is found to be effective in 50% of the women, and 60% of the
men. At this time, all patients for whom the drug was ineffective are changed to a
new drug, while everyone else is kept on the original drug. After 6 more months, it
is found that the new drug is effective in 60% of the women, and 80% of the men.
Find the probability that

(a) a woman is put on both drugs and fails;

(b) a patient in the study is put on the new drug;
(c) a patient has an ineffective response after 6 months, but effective after 12
months;
(d) a patient who has had an effective response in the study was (i) a woman, (ii)
had been put on only one drug, and (iii) had been put on both drugs.

Q10: 50% of calls handled by a telephone exchange are local, 30% are national, and
the remainder are international. It is noted that half of the international calls last
more than 15 minutes, half of the national calls last more than 10 minutes, and
half of local calls last more than 5 minutes. Establish a complete probability model
(e.g. specify the probability triple, (Ω, A, Pr[·])) for this problem, and hence find a
bound on the probability that

(a) a call lasts more than 5 minutes;

(b) a call was local if it lasted more than 15 minutes.
 Fig. Q11

Q11: In Fig. Q11, A and B are locations of two base stations and 1, 2 and 3 are the pos-
sible locations of a mobile phone. x is a measure of distance. A phone is connected
to one, and only one, base station with probability proportional to the inverse of
the phone’s distance from the base station. Stating any other assumptions, find

(a) the probability that one phone is connected to A and the other to B, given
that there are 2 phones;
(b) the probability that all phones are at location 1, given that there are 6 phones
in all, with 3 of them connected to base station A, and three to B.

Q12: Consider 4 propositions, S1 , . . . , S4 associated with an uncertain experiment.

(a) In general, how many probability parameters need to be specified?

(b) In my probability model, S1 is conditionally independent of S3 and S4 , given
S2 . Furthermore, S3 and S4 are independent. Answer (a) again, exploiting
this structure. Write down the implied chain rule factorization.
(c) Choose probability parameters in (b), and illustrate your choice via (i) a graph-
ical model, and (ii) a Venn diagram.
(d) For your choice in (c), find Pr[S1 ∧ S2 ].
(e) Choose any other chain rule factorization. How many probability parameters
are now required? Comment.
 3E3 Tutorial 2 (Discrete Probability Modelling)
Anthony Quinn
23rd September 2019

Q1: Consider a proposition, S, and conditional propositions, SN , SS and SN S , using

the notation from the notes. Arrange these 4 propositions in order of increasing
probability.

Q2: Two cards are drawn from a standard pack without jokers. Propose a (strictly)
necessary condition, and also a (strictly) sufficient one for the proposition that the
sum of the two cards is 17. Here, the number value of a picture card is 0. Do your
answers depend on whether there is replacement of the first card? Adopting any
consistent probability model, evaluate the probability of the proposition, both with
and without replacement.

Q3: The time, TA (days), to failure of mechanical device, A, is measured. Prior experi-
ence leads to the assignment of the probabilities, 0.34 and 0.54, to the propositions,
TA > 4.3 and TA > 2.5, respectively. Are these assignments consistent? Deduce
the probability that TA ≤ 2.5 given that TA ≤ 4.3.

Q4: The uncertain experiment described in Q3 is augmented also to test the proposition,
TB ≤ 1.0, in respect of another mechanical device, B, with which A interacts.
Complete your probability model for this UE in any consistent way you wish,
making sure to state your assumptions, and hence evaluate the probability that
TB > 1 given that 2.5 < TA ≤ 4.3.

Q5: Consider three propositions, S1 , S2 and S3 . The latter two are mutually exclusive.
Write down an expression for the probability of Pr[S1 ∨ S2 ∨ S3 ], stating the number
of degrees of freedom in your probability model.

Q6: Prove the identity, Pr[S1 ∨ S2 ] = Pr[S1 ] + Pr[S2 ] − Pr[S1 ∧ S2 ]. Can you extend your
answer to N ≥ 2 propositions? Can you prove the union bound in this way?

Q7: Consider three propositions, S1 , S2 and S3 , with probabilities 0.2, 0.1 and 0.6 re-
spectively. Write down an inequality for Pr[S1 ∨ S2 ∨ S3 ].

Q8: In Q7, S1 and S2 are mutually exclusive. The probability of each is 0.1 if S3 is true.
(i) Deduce Pr[S3 |S 2 ]; and (ii) verify your answer to Q7.

Q9: ECG electrodes are used to measure the cardiac potential difference between various
attachment points on a patient’s chest. The following is a set of n = 14 such pairs
of measurements, (v1,i , v2,i ), i = 1, . . . , n (in mV, rounded to one decimal place):

1
 v1,i -2.0 -2.8 -4.9 1.3 -0.1 2.5 -1.8 -0.6 -0.7 -2.7 4.1 -3.1 -0.4 -3.3
v2,i -1.4 -0.8 1.4 -0.0 -1.1 -0.2 1.9 -0.8 -1.2 3.6 0.8 -1.2 1.1 -3.3

We wish to use these data to predict a future measurement pair, (V1 , V2 ). In

particular, we are interested in the following three propositions:

(i) S1 : |V1 | > |V2 | > 0.7;

(ii) S2 : −0.6 < V1 < 0.0;
(iii) S3 : V2 > 0.

Stating any assumptions that you make, use the data above to establish a complete
probability model for this uncertain experiment, specifying the probability triple.
Hence, evaluate S2 ∨ S3

Q10: Compute the probability of winning the Irish national lottery next Saturday night,
stating your assumptions.

Q11: An urn contains three red balls and one green ball. Three balls are selected at
random, without replacement. What is the probability that the selection includes
the green ball. Compare your probability model with the one you used in Q10 and
comment.

Q12: Repeat Q11 if you replace each ball after drawing it.

Q13: An urn contains 2 red and 3 green balls. Three balls are drawn at random without
replacement. What is the probability of (i) 2 red balls in the draw, and (ii) the
sequence RRG. Repeat your work if each ball is replaced after drawing.

Q14: Using the axioms of probability, state and prove the identity relating Pr[S] to
Pr[S].

Q15: Deduce the chain rule of probability for N > 1 probabilities. Return to Q8 and
calculate Pr[S1 ∧ S2 ∧ S3 ] using this rule, and also directly via the Venn diagram.

Q16: The DNA is composed of sequences of 4 bases, G, C, A and T. (i) What is the
probability of a run of 3 bases, each with the same state? (ii) What is the probability
of a run of n bases, each with the same state, expressing your answer as a function
of n? State your assumptions.

Q17: I receive 100 emails in a 5-day period and note that 35 of them are spam. What
is the probability that the next 20 emails I receive include no more than 5 spam
emails? Explain your assumptions and reasoning fully.
 Tutorial 2 Solutions 22 10 19

23

I3

Prftps2si Tpsy.zJ PrfTaE2 8ATaE4.3

PrfT nE4.3
46
O 70
I 734
serfs us us E
mins ERM I

by the union bound theorem

min 0.2 0.1 0.6 I

min o 9 I

0.9
 There are 3,4 4 distinguishable selections only one
of which satisfies the proposition Adopting Bayes postulate
uniform probability modelling Prf I s
f
Qb
For maximal simplicity but not accuracy AMC model tater
will capture dependency assume He bases are sequentially
independent with
Po Prfs G O SP L I
R s Pr Isis c EB L
Pp Jefee 3
P
3
Prf run of 3 equal bases s
Ps t P Pf t P
is Prfan of n 1 equal bases Pan t Pan t Pan t
En
 217
The relative frequency of spam emails in the historic random
sample was 34 so estimate Prfanemail will be spam in future p 35
0.35k x o 6520
PIN 55 20 0.351 7 x

adopting the binomial probability law under the assumptionthat

the email classifications are ish BTS

ABQ 241419
 3E3 Tutorial 2 (Discrete Probability Modelling)
Anthony Quinn
23rd September 2019

Q1: Consider a proposition, S, and conditional propositions, SN , SS and SN S , using

the notation from the notes. Arrange these 4 propositions in order of increasing
probability.

Q2: Two cards are drawn from a standard pack without jokers. Propose a (strictly)
necessary condition, and also a (strictly) sufficient one for the proposition that the
sum of the two cards is 17. Here, the number value of a picture card is 0. Do your
answers depend on whether there is replacement of the first card? Adopting any
consistent probability model, evaluate the probability of the proposition, both with
and without replacement.

Q3: The time, TA (days), to failure of mechanical device, A, is measured. Prior experi-
ence leads to the assignment of the probabilities, 0.34 and 0.54, to the propositions,
TA > 4.3 and TA > 2.5, respectively. Are these assignments consistent? Deduce
the probability that TA ≤ 2.5 given that TA ≤ 4.3.

Q4: The uncertain experiment described in Q3 is augmented also to test the proposition,
TB ≤ 1.0, in respect of another mechanical device, B, with which A interacts.
Complete your probability model for this UE in any consistent way you wish,
making sure to state your assumptions, and hence evaluate the probability that
TB > 1 given that 2.5 < TA ≤ 4.3.

Q5: Consider three propositions, S1 , S2 and S3 . The latter two are mutually exclusive.
Write down an expression for the probability of Pr[S1 ∨ S2 ∨ S3 ], stating the number
of degrees of freedom in your probability model.

Q6: Prove the identity, Pr[S1 ∨ S2 ] = Pr[S1 ] + Pr[S2 ] − Pr[S1 ∧ S2 ]. Can you extend your
answer to N ≥ 2 propositions? Can you prove the union bound in this way?

Q7: Consider three propositions, S1 , S2 and S3 , with probabilities 0.2, 0.1 and 0.6 re-
spectively. Write down an inequality for Pr[S1 ∨ S2 ∨ S3 ].

Q8: In Q7, S1 and S2 are mutually exclusive. The probability of each is 0.1 if S3 is true.
(i) Deduce Pr[S3 |S 2 ]; and (ii) verify your answer to Q7.

Q9: ECG electrodes are used to measure the cardiac potential difference between various
attachment points on a patient’s chest. The following is a set of n = 14 such pairs
of measurements, (v1,i , v2,i ), i = 1, . . . , n (in mV, rounded to one decimal place):

1
 v1,i -2.0 -2.8 -4.9 1.3 -0.1 2.5 -1.8 -0.6 -0.7 -2.7 4.1 -3.1 -0.4 -3.3
v2,i -1.4 -0.8 1.4 -0.0 -1.1 -0.2 1.9 -0.8 -1.2 3.6 0.8 -1.2 1.1 -3.3

We wish to use these data to predict a future measurement pair, (V1 , V2 ). In

particular, we are interested in the following three propositions:

(i) S1 : |V1 | > |V2 | > 0.7;

(ii) S2 : −0.6 < V1 < 0.0;
(iii) S3 : V2 > 0.

Stating any assumptions that you make, use the data above to establish a complete
probability model for this uncertain experiment, specifying the probability triple.
Hence, evaluate S2 ∨ S3

Q10: Compute the probability of winning the Irish national lottery next Saturday night,
stating your assumptions.

Q11: An urn contains three red balls and one green ball. Three balls are selected at
random, without replacement. What is the probability that the selection includes
the green ball. Compare your probability model with the one you used in Q10 and
comment.

Q12: Repeat Q11 if you replace each ball after drawing it.

Q13: An urn contains 2 red and 3 green balls. Three balls are drawn at random without
replacement. What is the probability of (i) 2 red balls in the draw, and (ii) the
sequence RRG. Repeat your work if each ball is replaced after drawing.

Q14: Using the axioms of probability, state and prove the identity relating Pr[S] to
Pr[S].

Q15: Deduce the chain rule of probability for N > 1 probabilities. Return to Q8 and
calculate Pr[S1 ∧ S2 ∧ S3 ] using this rule, and also directly via the Venn diagram.

Q16: The DNA is composed of sequences of 4 bases, G, C, A and T. (i) What is the
probability of a run of 3 bases, each with the same state? (ii) What is the probability
of a run of n bases, each with the same state, expressing your answer as a function
of n? State your assumptions.

Q17: I receive 100 emails in a 5-day period and note that 35 of them are spam. What
is the probability that the next 20 emails I receive include no more than 5 spam
emails? Explain your assumptions and reasoning fully.
 3153 Tutorial 1 22 10 19

22
N fs E 2 s
Sn Tz 25 S
b N 3s is 2 Ss
NG S e 2 I
5ms

A Acb Ac

o.cm

i Es
in

772 4 5 inlauceh sets from

unsurtably infinite
cases Continuous
 S h
Sz A N Az its event set as illustrated

S as S
ie 9h52 is SUFFICIENT for S and alsofor 52
S as s

Proceeding similarly
r

s V Sz D U Az
s

I fi e.s.us
Ii
r
F
a
5 5 a nai
S a S sufficient for 9
S a
SI and Sa are mutually exclusive i e

r
Sasi A Sz Sp
F
Isr Asi Sol mutual exclusive
SIG Nsa sp l n n
24

X a.m
exclusive
Ss and Ingo are mutually
exclusive
125 r
z

Ay

partition to

Jasa v sans K 10
S S Sz Sy US partitionsubsets
O n O
0 h l n

u
O n I n
O n
l

U v
O n O n
I n
I I n
Sr n
Sz SI
V
E n
Sz n
Sz n Su
V
B n Iz n
Sz a
Sy
ne logical constraint S a
52 Sz

partition 7

Az
 3E3 Tutorial 1 (Propositional Logic and Partitions)
Anthony Quinn
18th September 2019

Q1: I have a coin and a die, and I play a game of chance. I toss the coin until I see
heads (H), and then I roll the die. If, however, I have not tossed H after 5 tosses,
I advance to the die anyway.
(i) What type of experiment is this? Specify the sample space, Ω.
(ii) Propose any pair of propositions that are logically related. State that rela-
tionship, and specify the induced partition of Ω.
(iii) If the game is changed to remove the stopping condition, attempt (i) and (ii)
again.
Q2: β-particles (i.e. electrons) impinge on a detector, and a measuring device records
the absolute time of each such event. Stating your assumptions, address each of
the following:
(i) Consider (indexed) propositions, N (t) ≤ k (meaning “the number of particles
in the next t seconds (s) is no greater than k”), and T3 > t3 (meaning “the
time to the third-next particle is greater than t3 (s)”). Provide any suitable
values of k, t and t3 that induce (a) necessity, (b) sufficiency and (c) equiva-
lence, respectively, between the two propositions. In each of these three cases,
specify the induced partition of Ω and provide one elementary outcome in each
partition subset.
(ii) Consider all k-indexed propositions above. Specify the induced partition of
Ω. Is it discrete or continuous?
(iii) Consider all t3 -indexed propositions above. Specify the induced partition of
Ω. Is it discrete or continuous?
Q3: Consider two propositions, S1 and S2 , defined in the the context of an uncertain
experiment. Also consider (compound) propositions, S1 ∧ S2 , S1 ∨ S2 , S1 ∧ S2 and
S1 ∨ S2 . For each of these four propositions, state the logical relationship (if any)
between it and each of S1 and S2 , respectively.
Q4: Consider proposition, S, and propositions, SN and SS , which are, respectively,
necessary and sufficient conditions for S. In respect of each of the following pairs
of propositions, what, if any, logical constraint exist between them: (i) SN and SS ;
(ii) S and SS ; (iii) S ∧ SN and SS ?
Q5: Consider four propositions, S1 , . . . , S4 , defined in the the context of an uncertain
experiment. S1 ⇒ S2 and S1 ∧ S3 ≡ S∅ . No other logical constraints are to be
assumed.

1
 (i) Illustrate the induced partition of Ω (via a Venn diagram), and specify its size.


(ii) Consider the proposition, S1 ∧ S2 ∨ (S3 ∧ S4 ). Illustrate its event set on
your diagram in (i). Hence, re-express this proposition (only) via propositions
whose events are partition subsets.

Q6: We monitor two moving components in a mechanical assembly, and are interested
in when each will fail. In particular, we are concerned with each of the following
propositions:

• S1 ≡ “it will take more than a year for the first component to fail”;
• S2 ≡ “it will take more than 6 months for the second component to fail”;
• S3 ≡ “the second component will fail before the first one does”.

(i) Illustrate the induced partition of Ω (via a Venn diagram), and specify its size.
(ii) Specify one elementary outcome in each partition subset.

Q7: I roll three dice, one red, one green and one blue. The respective outcomes (number
on the face) are R, G and B, respectively. Consider the propositions:

• S1 ≡ “R ≥ G”;
• S2 ≡ “G ≥ B”;
• S3 ≡ “R, G and B are all odd”;
• S4 ≡ “R + G + B > 10”.

(i) Repeat (i) and (ii) in Q5.

(ii) Recall the adapted version of this question in today’s lecture (lecture 6). Cor-
rect an error in the discussion of that case.
 3E3 Probability Modelling Equations
TCD Engineering
April 27, 2015

1 Three Types of Probabilistic Experiments

1. The PE has a finite number of outcomes

Ω = {ω1 , . . . , ωn }, #Ω = N (cardinality) (1)

2. The PE has a countably infinite number of outcomes

Ω = {ω1 , ω2 , ω3 , . . . } (2)

3. The PE has an uncountably infinite number of outcomes

Ω = R (3)

2 Stochastic Independence
S1 ⊥
⊥ S2 ⇒ P [S1 ∩ S2 ] = P [S1 ]P [S2 ] (4)

3 OR Union Lemmas
P [S1 ∪ S2 ] = P [S1 ] + P [S2 ] − P [S1 ∩ S2 ] (5)
P [S1 ∪S2 ∪S3 ] = P [S1 ]+P [S2 ]+P [S3 ]−P [S1 ∩S2 ]−P [S1 ∩S3 ]−P [S2 ∩S3 ]+P [S1 ∩S2 ∩S3 ] (6)

3.1 Boole’s Inequality

Prove: n n
[ X
P[ Si ] ≤ {1, P [Si ]} (7)
i=1 i=1
Proof:
n
[ n−1
[ n−1
[
P[ Si ] = P [ Si ] + P [Sn ] − P [ Si ∩ Sn ]
i=1 i=1 i=1

1
 n−1
[
P[ Si ∩ Sn ] ≥ 0
i=1
n
[ n−1
[
P[ Si ] ≤ P [ Si ] + P [Sn ]
i=1 i=1
n
[ n−1
X Xn
∴ P[ Si ] ≤ P [ Si ] + P [Sn ] = P [ Si ]
i=1 i=1 i=1

4 Conditional Probability (Chain Rule)

P [S1 ∩ S2 ]
P [S1 |S2 ] = (8)
P [S2 ]

5 Conditional Independence
S1 ⊥
⊥ S2 ⇒
P [S1 ∩ S2 ∩ S3 ] = P [S1 |S2 ∩ S3 ]P [S2 |S3 ]P [S3 ] (9)
= P [S1 |S3 ]P [S2 |S3 ]P [S3 ]

6 Theorem of Inverse Probability (Bayes’ Rule)

P [S1 |S2 ]P [S2 ]
P [S2 |S1 ] = (10)
P [S1 ]

7 Theorem of Total Probability

Partition of Ω into n partitions

P [SΩ ] = P [SΩ1 ] + P [SΩ2 ] + · · · + P [SΩn ] = 1

n
X n
X
P [S] = P [S ∩ SΩi ] = P [S|SΩi ]P [SΩi ] (11)
i=1 i=1

8 Binomial Probability Law

 
n n!
=
k (n − k)!k!
  (12)
n k n−k
P [Sk ] = p q
k

2
9 Multinomial Probability Law
m
X
ni = n
i=1
m
X
pi = 1
i=1

Number of elements in each particular set:

n!
Qm
i=1 ni !
m
" #" m
#
\ n! Y
P[ (Ni = ni )] = Qm pni i (13)
i=1 i=1 ni ! i=1

If m = 2, this becomes the Binomial Probability Law

10 Geometric Probability Law

P [X = k] = (1 − p)k−1 p = q k−1 p (14)

11 Transition Probability
The M xN transition matrix is a stochastic matrix because each column sums to 1
t11 . . . t1N
T̄ = . . . tij ... (15)
tM 1 . . . tM N

11.1 Degrees of Freedom

There are (M − 1)N probability parameters (degrees of freedom) needed to specify T̄

For square T̄ , (N 2 − 1) probability parameters are needed

12 Homogenous Markov Chains

n
\
P[ Si ] = P [Sn |Sn−1 ]P [Sn−1 |Sn−2 ] . . . P [S2 |S1 ]P [S1 ] (16)
i=1

3
12.1 Time Invariance
Simplest relaxation of independence is for each outcome to depend only on the previous
outcome
P [Sn = j] = P [(Sn = j) ∩ SΩ ]
⇒ P [Sn = j] = P [(Sn = j) ∩ ((Sn−1 = 1) ∪ (Sn−1 = 2) ∪ · · · ∪ (Sn−1 = M ))]
Using axiom 3 and the Chain Rule:

P [Sn = j] = P [(Sn = j)|(Sn−1 = 1)]P [Sn−1 = 1]

+P [(Sn = j)|(Sn−1 = 2)]P [Sn−1 = 2]

+...
+P [(Sn = j)|(Sn−1 = M )]P [Sn−1 = M ]
M
X M
X
⇒ P [Sn = j] = P [(Sn = j)|(Sn−1 = i)]P [Sn−1 = i] = P [Ai ]P [Bi ]
i=1 i=1

P [Ai ]: These are the M probabilities along the j th row of T̄

P [Bi ]: These are the M probabilities in P n−1 , where P n−1 is the marginal (total) proba-
bility vector at step n − 1

The ith element of P n is obtained by multiplying the j th row of T̄ by P n−1

P n = T̄ P n−1 n = 1, 2, 3, . . .

P n = T̄ (T̄ P n−2 ) = T̄ 2 (T̄ P n−3 )

∴ P n = T̄ n−1 P 1 (17)

12.2 Long-Run Behaviour

T̄ k = V̄ Λ̄k V̄ −1 , k≥1 (18)

¯ =0
det(T̄ − λI) (19)

λk1 0
Λ̄k = (20)
0 λk2

¯ n=0
(T̄ − λn I)V (21)

V11 V21
V̄ = (22)
V12 V22

4
 1 V22 −V21
V̄ −1 = (23)
det(V̄ ) −V12 V11

a b e f (ae + bg) (af + bh)

= (24)
c d g h (ce + dg) (cf + dh)

12.3 Colour-Blind Homogenous Markov Chains

Depends on the ordering of states, but not on which state the chain is currently in. T̄
is doubly stochastic, the rows and columns sum to 1. All states are equiprobable in the
long-run.

13 Bernoulli Random Variable

Shaping parameter P [1] = p ⇒ q = (1 − p)

X ∼Ber(p) (25)

Summarise the Bernoulli Model using the binomial PMF, n ≡ # iid Bernoulli Trials

N ∼Bi(n, p) (26)

Maximiser: the most probable outcome

k̂  {0, 1, . . . , n}

k̂ = np − 1 (if np integer) = bnpc (if np non-integer) (27)

14 Geometric Random Variable

M (ω) = L  {1, 2, 3, . . . }
(28)
P [M = L] = q L−1 p

14.1 Memoryless Property of Geometric Random Variable

Prove:
P [(M > (L + L0 )) | (M > L0 )] = P [M > L], L, L0 ≥ 0 (29)
Proof:

Definition of Conditional Probability

P [(M > (L + L0 )) ∩ (M > L0 )]
P [(M > (L + L0 )) | (M > L0 )] =
P [M > L0 ]

5
(M > (L + L0 )) is sufficient for (M > L0 )

P [(M > (L + L0 ))]

⇒ P [(M > (L + L0 )) | (M > L0 )] =
P [M > L0 ]

q L+L0
⇒ = qL
q L0
∴ P [(M > (L + L0 )) | (M > L0 )] = P [M > L], L, L0 ≥ 0

15 Probability Mass Function

Discrete random variable k → P [X = k] = Pk , ∀k  Z (30)

16 Properties of Cumulative Distribution Function

1. Function of all x  R

2. FX (x) ≥ 0, by Axiom 1

3. FX (x) ≤ 1, by Axiom 2

4. limx→+∞ FX (x) = 1

5. limx→−∞ FX (x) = 0

6. FX (x2 ) ≥ FX (x1 ), for any x2 > x1 , i.e. non-decreasing function

7. P [x1 < x ≤ x2 ] = P [x ≤ x2 ] − P [x < x1 ] = FX (x2 ) − FX (x1 )

8. fX (x), the probability density function, is always differentiable from the right

d
fX (x) = (FX (x)) (31)
dx

17 Properties of Probability Density Function

1. limx→−∞ fX (x) = limx→+∞ fX (x) = 0, FX (x) constant at limits

2. fX (x) ≥ 0, FX (x) non-decreasing

3. Inverse of the differentiation property

Z x
FX (x) = fX (x)dx (32)
−∞

6
 R∞
4. fX (x)dx = 1, the normalisation property of the PDF
−∞
Rx
5. P [x1 < x ≤ x2 ] = x12 fX (x)dx = FX (x2 ) − FX (x1 ) = P [x ≤ x2 ] − P [x < x1 ]

6. x1 = x − dx, x2 = x ⇒ P [x − dx < X ≤ x] ≈ fX (x)dx

(a) fX (x) is a graph of any belief that the unknown measurement, X = x, is “around”
value x
(b) Belief is proportional to the PDF, dx is the constant of proportionality

18 Exponential Random Variable

T (seconds) ∼ Exp(λ), λ > 0 (per second) (33)
The normalising constant is determined by noting that
Z ∞  ∞
−λt −1 −λt 1
e dt = e =
0 λ 0 λ

Need to multiply fT (t) by λ to achieve an area of 1 under the exponential PDF

 −λt
λe , t≥0
fT (t) = (34)
0 , t<0

P [T > t] = 1 − FT (t) (the CDF), t > 0, λ > 0

Z ∞
(35)
=λ e−λt dt = e−λt
t

Mean, m
1
m= (36)
λ

Variance, VAR
1
VAR = (37)
λ2

18.1 Memoryless Property

t, τ > 0
Prove:
P [T > (t + τ )|T > τ ] = P [T > t] (38)

7
Proof:
P [T > (t + τ ) ∩ T > τ ]
P [T > (t + τ )|T > τ ] =
P [T > τ ]
P [T > (t + τ )] e−λ(t+τ )
⇒ = = e−λt = P [T > t]
P [T > τ ] e−λτ

19 Poisson’s Distribution
λk e−λ
f (k : λ) = P [X = k] = (39)
k!

20 Gaussian (Normal) Distribution

Two shaping parameters, m = mean and ζ = standard deviation; r = ζ 2 is the variance

X(ω) = x ∼ Nx (m, r) (40)

 
1 −1 2
NX (m, r) = fX (x) = √ exp (x − m) (41)
2πr 2r
  
1 x−m
FX (x) = 1 + erf √ (42)
2 ζ 2

21 Bivariate Probability Distribution Function

Z ∞ Z ∞
fXY (x, y)dxdy = 1 (43)
y=−∞ x=−∞

Proposition of interest IXY  R2

ZZ
P [(X, Y )  IXY ] = fXY (x, y)dxdy (44)
IXY

21.1 Marginals
Projections onto each axis
Z ∞
fX (x) = fXY (x, y)dy (45)
y=−∞
Z ∞
fY (y) = fXY (x, y)dx (46)
x=−∞

8
21.2 Conditionals
fXY (x, y0 )
fX (x|y0 ) = , fY (y0 ) > 0 (47)
fY (y0 )
fXY (y, x0 )
fY (y|x0 ) = , fX (x0 ) > 0 (48)
fX (x0 )

21.3 Bayes’ Rule for Probability Distribution Functions

fXY (x, y) = fX (x|Y = y)fY (y) = fY (y|X = x)fX (x) (49)

fY (y|X = x)fX (x)

fX (x|Y = y) = (50)
fY (y)
fX (x|Y = y)fY (y)
fY (y|X = x) = (51)
fX (x)

For X ⊥
⊥ Y , Conditionals = Marginals

fXY (x, y) = fX (x)fY (y), X⊥

⊥Y (52)

fX (x|Y = y) = fX (x) (53)

fY (y|X = x) = fY (y) (54)

22 Bivariate Gaussian (Normal) Probability Distribu-

tion Function
Bivariate distribution to express believe in two random variables X and Y , where neither
has any constraint on its possible values. m = marginal mean, r = marginal variance.

x ∼ fX (x) = NX (mx , rx ) (55)

y ∼ fY (y) = NY (my , ry ) (56)

Correlation coefficient ρxy controls the dependence between x and y.

ρxy  [−1, +1] (57)

ρxy = 0 ⇔ X ⊥
⊥Y (58)

9
 ρxy = −1 → (negatively correlated) (59)

ρxy = +1 → (positively correlated) (60)

22.1 Mean and Covariance Vectors

mx
m= , m  R2 (61)
my

rx ρxy δx δy
Σ̄ = , rx = δx2 , ry = δy2 (62)
ρxy δx δy ry
2
δx|y = δx2 (1 − ρ2xy ) (63)
2
If δxy = 0, if you know one, you know the other, no variance for this condition

23 Expectation
Discrete random variable X
X
E[X] = mx = kP [X = k] (64)
∀k

Continuous random variable Y

Z ∞
E[Y ] = my = yfY (y)dy (65)
−∞

Law of large numbers

n
1X
lim xi = mx (66)
n→∞ n
i=1

Expectation rules

E[X + Y ] = E[X] + E[Y ] (67)

E[X + c] = E[X] + c (68)

E[aX] = aE[X] (69)

10
24 Variance
VAR[X] = E[X 2 ] − m2x = rx = δx2 (70)

X⊥
⊥Y ⇒ VAR[X + Y ] = VAR[X] + VAR[Y ] (71)

VAR[αX] = α2 VAR[X] (72)

Continuous
Z ∞
VAR[X] = (x − mx )2 fX (x)dx (73)
−∞

Discrete
X
VAR[X] = (k − mx )2 P [X = k] (74)
∀k

Variance rules

VAR[αX] = α2 V AR[X] (75)

VAR[X] = E[(X − µ)2 ] = E[X 2 ] − E[X]2 (76)

25 Signal to Noise Ratio

!
V2
SNR(dB) = 10log10 (77)
δ2

26 Conditional Mean and Variance

Continuous
Z ∞
mx|y = xfX (x|y)dx (78)
−∞
Z ∞
my|x = yfY (y|x)dy (79)
−∞
Z ∞
2
δx|y = (x − mx|y )2 fX (x|y)dx (80)
−∞

11
 Z ∞
2
δy|x = (y − my|x )2 fY (y|x)dy (81)
−∞

Discrete
X
mx|y = kP [(X = k)|Y ] (82)
∀k
X
my|x = lP [(Y = l)|X] (83)
∀l
X
2
δx|y = (k − mx|y )2 P [(X = k)|Y ] (84)
∀k
X
2
δy|x = (l − my|x )2 P [(Y = l)|X] (85)
∀l

27 Covariance
δxy = E[(X − mx )(Y − my )] = E[XY ] − E[X]E[Y ] (86)

δxy = ρxy δx δy (87)

δxy
−1 ≤ ρxy = ≤1 (88)
δx δy
Continuous Z ∞ Z ∞
δxy = (x − mx )(y − my )fXY (x, y)dxdy (89)
y=−∞ x=−∞

Discrete XX
δxy = (k − mx )(l − my )P [X = k, Y = l] (90)
∀l ∀k

28 Independence Between Random Variables X and Y

Continuous
X⊥
⊥ Y ⇔ fXY (x, y) = fX (x)fY (y)
Discrete
X⊥
⊥ Y ⇔ P [X = k, Y = l] = P [X = k]P [Y = l]
⊥
⊥ is sufficient for ρxy = 0
ρXY = 0 is necessary for ⊥
⊥
⊥
⊥ ⇒ ρxy = 0 (91)

12
One exception where ⊥
⊥ and ρxy = 0 are equivalent is the Bivariate Gaussian Distribution

⊥
⊥ ⇔ ρxy = 0, NXY (m, Σ̄) (92)

Proof

δx2 δxy
Σ̄ =
δxy δy2

δx2 0
If (ρxy = 0) ⇒ (δxy = 0) ⇒ Σ̄ =
0 δy2

⇒ NXY (m, Σ̄) = NX (mx , δx2 )NY (my , δy2 ) = fX (x)fY (y) or P [X = k]P [Y = l] ⇔ X ⊥
⊥Y

29 Sifting Property of Dirac-δ Function

Z ∞
g(x)δ(x − x0 )dx = g(x0 ) (93)
−∞

30 Sampling
X i = xi ∼ fX (x)

n
1X
fe(x) = δ(x − xi ) (94)
n i=1
n
1X
m
ex = xi (95)
n i=1
n
1X
VAR[X]
] = rex = e x )2
(xi − m (96)
n i=1

30.1 Non-parametric and Parametric Approximation of Bivariate

PDF’s/PMF’s
Given random sample:

(x, y)n = {(x1 , y1 ), (x2 , y2 ), . . . , (xn , yn )}

Consider each of the two marginals separately and use moment-matching to capture corre-
lation

13
 n
1X
E[XY
e ]= xi y i (97)
n i=1

cov[XY
f ] = E[XY
e ]−m
e xm
ey (98)

cov[XY
f ] cov[XY
f ]
ρexy = p p = in the Gaussian case (99)
2
δx δy2 δx δy

31 Error Function
Z x
2 2
erf(x) = √ e−t dt (100)
π 0

14
Figure 1: Error Function Table

15
 EE3E3-1

FACULTY OF ENGINEERING, MATHEMATICS & SCIENCE

SCHOOL OF ENGINEERING

Electronic & Electrical Engineering

Engineering Trinity Term, 2018
Junior Sophister
Annual Examinations

Probability and Statistics (3E3) [SAMPLE]

Date: 2018 Venue: — Time: —

Anthony Quinn

Instructions to Candidates:
ANSWER QUESTION 1, and any THREE of the remaining five questions.

Question 1 is worth 30 marks in toto.

All remaining questions are worth equal marks (i.e. 70/3 marks each). The percentage divi-
sion of marks within each question is indicated on the paper.

Materials Permitted for this Examination:

Non-programmable calculators are permitted for this examination. Please indicate the make
and model of your calculator on each answer book used.

Page 1 of 6
c Trinity College Dublin, The University of Dublin 2018
 EE3E3-1

Q.1 [Compulsory]

Answer ALL the following questions.

(a) Consider three propositions, S1 , S2 and S3 , each with probability 0.4. S1 and S2 are
independent, as are S1 and S3 , while S2 and S3 are mutually exclusive. Evaluate
Pr[S1 ∨ S2 |S3 ].

[20 %]

(b) In tests on batches of 100 lightbulbs produced by a manufacturer, on average 6 fail to

last longer than a year. What is the most probable number of such faulty lightbulbs in a
future batch of 49 lightbulbs, and what is this probability?

[20 %]

(c) Cars pass a particular point on an uncongested road at an average rate of 3 per minute.
Evaluate the probability that the second-next car will pass at a time more than 40 sec-
onds from now.

[20 %]

(d) Particles impinge uniformly on a triangular detector with vertices at Cartesian coordinates,
(2, 0), (−2, 0) and (0, 2) (cm), respectively. Draw a calibrated plot of the cumulative
distribution function of Y , being the y-coordinate of the next particle.

[20 %]

(e) The numbers of phone calls arriving at two related call centres in a fixed interval of time
are recorded on five occasions, as follows:

N1 7 5 12 9 4
N2 5 4 2 2 8

Use these data to estimate the correlation coefficient between N1 and N2 , and interpret
your answer.

[20 %]

Page 2 of 6
c Trinity College Dublin, The University of Dublin 2018
 EE3E3-1

Q.2

(a) Define conditional probability. Hence, state and prove Bayes’ rule. [45 %]

(b) In a device manufacturing plant, 70% of the devices are produced by day-shift workers,
and 30% by night-shift workers. The manufactured device includes a key component
which is sourced in equal numbers from either factory A or factory B. In a quality control
study, it is found that 90% of the devices produced by the day-shift workers meet the
required standard, regardless of the origin of the key component. In the case of the
night-shift workers, 85% of their devices are satisfactory if the key component is from
factory A, and this drops to 80% when the key component is from factory B.
Using these statistics, and stating any assumptions that you make, complete your prob-
ability model and evaluate each of the following:

(i) the probability that a device is defective if the key component is from factory A;
[15 %]
(ii) the probability that a device produced by this plant is defective;
[20 %]
(iii) the probability that the key component in a defective device is from factory B.
[20 %]

Q.3

(a) State and prove the multinomial probability law.

[50 %]

(b) The average lifetime of a particular implanted medical device is twelve years. Seven
such devices are monitored after implantation. Stating your assumptions, evaluate the
probability that the majority of these are still functional after ten years and that none of
those which fail will do so in less than eight years.

[50 %]

Page 3 of 6
c Trinity College Dublin, The University of Dublin 2018
 EE3E3-1

Q.4

(a) The probability model of a continuous random variable (rv), X, is completely specified by
its cumulative distribution function (cdf), FX (x). Define the two phrases in italics. Also,
state—and briefly justify—the key properties of any cdf.

[50 %]

(b) Evaluate the probability that a normally distributed measurement is more than two stan-
dard deviations from the mean.

[25 %]

(c) An uncertain voltage level in an electronic circuit is uniformly distributed in the interval
(−v, +v] (Volts), for some fixed v > 0. Derive an expression for the variance of this
voltage.

[25 %]

Q.5

(a) Define random Poisson points, and hence prove that the (time) interval between two
consecutive points is exponentially distributed.

[45 %]

(b) Trains arrive at a station at an average rate of one every five minutes. Stating any as-
sumptions that you make, address each of the following:

(i) evaluate the probability that the time to the second-next train is less than eight
minutes?
[15 %]
(ii) if no train arrives in the next three minutes, evaluate the probability that more than
one will arrive in the following three minutes?
[15 %]
(iii) evaluate the probability that five trains will arrive in the next twenty minutes, with
the majority of these arriving in the next fifteen minutes.
[25 %]

Page 4 of 6
c Trinity College Dublin, The University of Dublin 2018
 EE3E3-1

Q.6
(a) The probability density function of a random variable, X, is either f1 (x) or f2 (x). A set of
n ≥ 1 realizations of X is observed. Stating your assumptions, propose a statistical test
for choosing between the two possible probability models for X.

[30 %]

(b) For a particular patient, a pair of biophysical measurements, (X, Y ), is recorded on each
of 7 consecutive days, as follows:

X 0.9 -1.5 -1.0 2.5 1.3 -0.2 0.7

Y 1.1 -1.6 0.8 1.2 0.3 0.6 1.3

Using these data, and stating any assumptions that you make, address each of the
following:

(i) adopting a nonparametric model, estimate Pr[X ≥ 0.7 | Y > 0] for a future mea-
surement pair, (X, Y );
[20 %]
(ii) adopting a bivariate normal model, estimate and plot a line regressing Y onto X.
Evaluate Pr[Y > 0 | X = 0].
[50 %]

Note on the bivariate Gaussian (normal) distribution:

If
σX2
 
" # " # σXY
X mX
∼ N (m, Σ), where m = , Σ= ,

Y mY

σXY σY2
with
σXY = ρXY σX σY ,
then
2
f (x|Y = y) = N (mX|y , σX|y ),
where
σX
mX|y = mX + ρXY (y − mY ),
σY
 
2
σX|y = σX2 1 − ρ2XY .

Page 5 of 6
c Trinity College Dublin, The University of Dublin 2018
 EE3E3-1

Page 6 of 6
c Trinity College Dublin, The University of Dublin 2018
EE3E03 2017 Solutions
Exam Paper: https://www.tcd.ie/academicregistry/exams/assets/local/past-papers2017/EE/EE3E3-1.PDF
Copyright © 2018, ExamHack
 3E3: Probability and Statistics
Mid-Term Test [SAMPLE]
Anthony Quinn

Thursday 24th March 2022, 13:00–13:40

Goldsmith Hall
Instructions:
• The duration of this test is FORTY MINUTES.

• Answer all THREE questions, which carry equal marks (6 marks each). The divi-
sion of marks is indicated within each question.

• This is an open-book test, and Matlab may be used. The test is strictly subject to
the PROCEDURE which has been distributed to candidates in advance. Scripts
must be imaged and uploaded to Blackboard by 13:55, accompanied by the signed
declaration quoted in the Procedure.

Q.1: S1 and S2 are Boolean propositions with probabilities 0.4 and 0.5, respectively,
while S3 = i ∈ {1, 2, 0} is a q = 3-state multinomial, with prior probability vector,
 
0.4
p3 =  0.5  .
0.1

The following constraints apply:

• S1 ⊥
⊥ S2 ,

• (S1 ∧ S2 ) ⇒ (S3 = 2).

(a) Sketch the induced partition of Ω, and specify the number of probability parameters.

[3 marks]

(b) Complete your probability model in any consistent manner, and hence evaluate

Pr[S1 | S2 ∨ (S3 = 1)].

[3 marks]

[OVER]

1
Q.2: Consider four propositions (Boolean), S1 , . . . , S4 . The following stochastic structure
applies:

• the (marginal) probabilities of S1 and S2 are each 21 ;

• if either (or both) proposition(s), S1 and S2 , is (are) true, then S3 is true. Otherwise,
S3 is true with probability 32 ;

• if either (or both) proposition(s), S1 and S2 , is (are) true, then S4 is true. Otherwise:
1
if S3 is true, then S4 is true with probability 2
;
1
if S3 is false, then S4 is true with probability 4
.

(a) Draw an implied—fully parameterized—probability graph. [3 marks]

(b) Evaluate Pr[S3 | S4 ]. [3 marks]

Q.3: The cooccurrence matrix of a q = 4-state sequential experiment, Si ∈ {1, 2, 3, 0},

i ∈ Z, is as follows:  
2 18 8 9
 22 16 25 40 
 
C=  .
 7 51 0 15 


6 18 40 2

(a) Adopting a homogeneous Markov chain model, and given that S1 = 1, evaluate the
probability that S2 and S4 are equal;

[3 marks]

(b) If, instead, an iid model is adopted, then evaluate the probability that—in a sample
of n = 6 sub-experiments—the first two states are both 0, and the next four states
comprise equal numbers of 1’s and 2’s.

[3 marks]
 i

2.1

31 032 30

This partition captures the sole logical constraint that

S n
52 53 2
With 10 partition propositions sets induced the max number

I Ez gg 1I ingPrls.nsy
of is 9 However o 4xo.s o.z
b pips
Much freedom beyond

03
or

718
Az
25

31 032 30
 03 02
P 151s is gfY

I t 05 t 03 02

p
0.25
2.2

O O
e e
I
Is L
1

Pfs 154 8154

Pr
53
fuggygion
Pry s I er sins Ting sj y
Pr
sylszisznsix.fi xmxn

I Ex x2 I

Pry l Il It's Is

Prust
2.3

fo

3 o

1 I 1 s
73 8
2
37 3 73 86

ie counts are

11
the marginal In i Column sums

Is s
Es s

D
n where n n s sum of cosec aunts
I
279

3
1a
Prf Pr Isps si x Pr sis s n

rigged
transition probs
Taft

4 diagonal elements of Fj x
É x
b Pr equal nos of I's 2s in stages 3 6 of exp

Pr n O
A 7 0 A Boso 4 n 4 Rs

Pr N Rst N 2 4 Bs
n I
B o
7

Pr ni A 02 A 0 n't Is
2 2
3,0
with Is as estimated above

71 411 4 5
II y f x
t
t
o fix fix P
T
where 0 I 1

Call this probability P

Finally Pr 1st 2 States both 20s A statement above

Ea E xp

732 22 3 22