PROBABLITY AND STATISTICS LECTURE NOTE 2021

CHAPTER- FIVE Definition 2: Let A and B be sets

INTRODUCTION TO 1) The union of sets A and B is A ∪ B =

{x: x ∈ A 𝐨𝐫 x ∈ B}.
PROBABILITY
2) The intersection of sets A and B is A ∩ B =
If an experiment is conducted under {x: x ∈ A 𝐚𝐧𝐝 x ∈ B}.
homogeneous and similar conditions, we
Definition 3: a set A is a subset of set B if
come across two types of situations or
each object in set A is also included in set B
phenomena.
or if every element of set A is an element of
The results or the outcome is unique or set B and written asA ⊆ B.
certain which are called as deterministic
phenomena by which the result can be Definition 4: Two sets, A and B, are equal,
predicted certainly from an experiment denoted A=B, if and only if all element in A
performed. For instance, in case of ohm’s law belongs to the set B and every element in B
voltage = I*R. On the other hand the results belongs to set A, i.e., A ⊆ B and A ⊇ B.
or the outcomes of an experiment are not
Definition 5: Set B is a proper subset of set
unique, but can be one of the several possible
A if B is a subset of A, but B does not equal
outcomes; this type of model is called
A. i.e. B⊏A
probabilistic or non-deterministic
phenomena. Example: In random tossing of
Definition 6: Suppose that A ⊂ S. The
a coin, we are not sure of getting head or tail.
complement of set A, denoted as A̅ or c, is
Probability: can be defined as a measure of the set containing all elements in S that are
the likelihood that a particular event will not in A. i.e. {x: x ∈ S and x ∉ A}.
occur or it is a science of decision making
Definition 7: Two sets, A and B, are called
with calculated risk in face of uncertainty.
disjoint of mutually exclusive if they contain
Set theory: Before we talk about probability, no common element. i.e., ifA ∩ B = ∅.
it is useful to review some basic definitions
De Morgan's Laws:
and theorem from set theory.
a) (A ∪ B)c = Ac ∩ Bc
Definition1: Set is a collection of all possible
objects. Members of a set are called elements.
b) (A ∩ B)c = Ac ∪ Bc
The set is represented by capital letters like
A, B, C... etc. and the members represented
by small letters like a, b, c... etc.

BY: ABDULMENAN M.(Msc)

1
PROBABLITY AND STATISTICS LECTURE NOTE
Definitions of some probability terms
1. Experiment: An experiment is any
activity that generates outcomes. For
example, tossing of a fair coin is
considered as a statistical experiment.
For example, tossing of a fair coin,
rolling a die etc.
2. Random experiment: - is a process of
measurement or observation which is
repeated at any time and who's out
come can't be predicted with certainty.
E.g. tossing a coin
3. Outcome: a particular result of an
experiment (result of single trial of an
experiment)
4. Sample space: - is the set of all
possible outcomes of a random
experiment. Each possible outcome is
called sample point.
5. Event: - is a subset of a sample space
(one or more outcomes of an
experiment).
Example1: if we toss a coin the sample
space (S) of this experiment S = (head; tail)
where head and tail are two faces of a coin.
If we are interested in the outcome head then
the event E= head
There are two types of events. These are:
 Elementary event /simple event/
sample point is a single possible
outcome of an experiment.
 Composite event (compound event) is
an event having two or more
elementary events in it.
For example, rolling a die (6 sided object)
with a sample space, S ={1,2,3,4,5,6) an
2
2021
event having {3} is simple event where as
having even number {2,4,6} is compound
(composite)event.
6. Equally likely outcomes: In a certain
experiment, if each outcome in the sample
space has equal chance to occur, then we say
that the outcomes are equally likely outcomes
7. Mutually exclusive events: two events are
said to be mutually exclusive, if both events
cannot occur at the same time (i.e., they have
no occurrences in common). For example, If
we are interested in the outcome of event A
getting even numbers and B odd numbers
through rolling a fair die , then A= {2, 4, 6}
and B= {1, 3, 5}.Thus, A and B are mutually
exclusive events.
Example: In tossing a coin twice. Write the
event consisting of
a) At least one head b) Exactly two
heads c) At most two tails d)
Identify for a, b and c as elementary
or compound events, and which are
mutually exclusive events.
Solution: S = {HH, HT, TH, TT}.
a) Let A= an event consisting of at least
one head, hence the possible
outcomes of event A are {HH, HT,
TH} which is compound event.
8. Independent Events: Two events A and
B are said to be independent events if the
occurrence of event A has no influence
(bearing) on the occurrence of event B. For
example, if two fair coins are tossed, then
the result of one toss is totally independent
of the result of the other toss.
BY: ABDULMENAN M.(Msc)
PROBABLITY AND STATISTICS LECTURE NOTE
The probability that a head will be the
outcome of any one toss will always be ½,
irrespective of whatever the outcome is of
the other toss. Hence, these two events are
independent.
Counting rules
If the number of possible outcomes in the
experiment are small, it is relatively easy to
list and count all possible outcomes where as
if it is very large number of possible
outcomes occur an enumeration of cases is
often difficult or tedious. Therefore, to
overcome this problem we have four
different counting techniques.
I. Addition principle: Suppose that a
procedure designated by 1, can be performed
in n1 ways. Assume the second procedure
designated by 2 can be performed in n2 ways.
Furthermore suppose that both procedures
are not possible to perform together. The
possible number of ways which can perform
1 or 2 procedure is in n1 + n2 different ways.
th
Generally, if there are k procedures and i
procedure may be performed in ni ways,
where i=1, 2… k then the number of ways in
which we perform procedure 1 or 2 or … or
k is given by 𝑛1 + 𝑛2 + ⋯ + 𝑛𝑘 = ∑𝑘𝑖=1 𝑛𝑖 ,
assuming that no two procedures performed
together.
Example: Suppose that we are planning a trip
and are deciding between Minibus and Bus
transportation. If there are 5 Minibus routes
and 3 Bus routes to go from Aksum city to
Mekelle city, find the available routes for the
trip?
Solution: we have two opportunities or
procedures either using Minibus or Bus,
3
2021
hence k= 2. The first procedure has 5routes
(ways), 𝑛1 = 5 and the second procedure
has 3 routes (ways) 𝑛2 = 3. Therefore, we
have n1+n2 = 5+3=8 possible routes for
someone to go from Aksum to Mekelle city.
II. Multiplication principle: In a sequence
of n events in which the first event has k1
possibilities… the nth event has kn
possibilities, then the total possibilities of the
sequence will be k1*k2*….*kn.
Example: An Instructor gives 6 question of
multiple choose examination. There are 4
possible response to each question. How
many answer keys can be made?
Solution: k1 =k2 = k3 =k4=k5=k6=4 4.4.4.4.4.4
= 4096 d/t answer key can be made.
Exercise 1: If a snack bar offers 4 kind of
sandwich which it offers with macchiato,
cappuccino, tea or milk. How many different
ways are there in which one can order
sandwich and a drink for breakfast?
Exercise 2. The number in the Ethiopian
telephone number is 10 digit sequence, but
the 1st and the 2nd digit number take only 0
and 9 respectively. How many distinct
telephone numbers are there in Ethiopia?
Exercise 3: suppose that in a medical study
patients are classified according to their
blood type as A, B , AB, and O; according
to their RH factors as + or - and according
to their blood pressure as high, normal or low
,then in how many different ways can a
patient be classified ?
III. Permutation Rule: An arrangement of n
objects in a specific order. We have different
rules in permutation.
Rule 1: The no. of permutation of n objects
taken all together is n!
BY: ABDULMENAN M.(Msc)
PROBABLITY AND STATISTICS LECTURE NOTE 2021

Note: In definition 0! = 1! = 1. Note: When the method of selection or

Example 1: A photographer wants to arrange arrangement of r objects from n-objects is
three persons in a row for photograph. In how with repetition the possible numbers of
many ways can the arrangement be done? arrangements are nr.
Solution: n=3, n! =3! =3x2x1=6 d/t ways.
Exercise 1: - Aksum University Registrar
Rule-2: A permutation of n different objects Office wants to give identity number for
taken r at a time is an arrangement of r out of students by using 4 digits. The number
the n objects, with attention given to the order should be considered by the following
of arrangement. The number of permutations numbers only: {0, 1, 2, 3, 4, 5, and 6}. Hence,
of n objects taken r at a time is denoted by nPr, how many different ID Numbers could be
and is given by: given by the Registrar?
n!
nPr = a. Without repeating the number
n  r !
b. With repetition of numbers
Example 1: out of 6 colors a flag of 3 color
is to be formed. In how many ways can the Exercise 2: In a board of directors composed
flag be formed? of 6 people, how many ways can a chief
Solution: here order is important. n=6, r=3. executive officer, a director and a secretary
n! 6!
be selected?
Therefore nPr = n! = (6−3)! = 6x5x4
n  r !
IV. Combination Rule: counting technique
= 120 ways
in which the order of the objects is
immaterial. The number of combinations of n
Rule-3: The number of permutation of n
objects taken r at a time is denoted by the
objects taken all at a time, when n1 objects are 𝑛!
alike of one kind, n2 objects are alike of symbol (nr) or nCr is given by: nCr = (n−r)!r!
second kind, …, nk objects are alike of kth
kind is given by: Example 1:- A printed circuit board has eight
different locations in which a component can
 k  be placed. If five identical components are to
  ni !
be placed on the board, how many different
  ki 1 
n!
designs are possible?
 ni !
n1! n2 ! n3!...nk !
i 1
Solution: Each design is a subset of the eight
Example 1: The total number of arrangement locations that are to contain the components.
of the letters of the word STATISTICS taken The number of possible designs is, therefore,
8! 8∗7∗6∗5! 8∗7∗6∗5!
10! (85) = (8−5)!5! = = 3∗2∗1∗5! = 8 ∗
all at a time is given by = 50,400 3!5!
3!3!1!2!1! 7 = 56 different designs are possible.
since there are 3s’s, 3t’s, 1a, 2i’s and 1c.

BY: ABDULMENAN M.(Msc)

4
PROBABLITY AND STATISTICS LECTURE NOTE
Exercise: A committee of 5 people must be
selected from 5 men and 8 women. In how
many ways can the selection be done if there
are at least 3 women on the committee?
Axiomatic Approach
Given a sample space of a random
experiment, the probability of the occurrence
of any event A is defined as a set function P
(A) satisfying the following axioms:
1) P(S)=1
2) Let A be an event and A' be the
compliment of A with respect to a
given sample space of an experiment,
then p(A' )=1-P(A)
3) For any event A: 0 < P(A) < 1
4) If A and B are mutually exclusive
events,
P (A or B) = P (AUB) = P (A) + P(B)
5) Suppose A and B are two any events
of a sample space, then
P (AUB) = P (A) + P (B) – P (A∩ B)
6. If A and B are two independent events
then the chance of occurrence of both
two events is
P (A  B) = P (A) P (B)
Example 1: A day of the week is selected at
random. Find the probability that it is a
weekend day.
Solution:
P (Saturday or Sunday) =
P (Saturday) + P(Sunday) = 1/7+1/7=2/7
5
2021
Conditional probability
Let there be two events A and B. Then the
probability of event A given that the outcome
of event B is given by:
P [ AnB ]
P [A/B] =
P[ B ]
Where P [A/B] is interpreted as the
probability of event A on the condition that
event B has occurred. In this case P [A n B]
is the joint probability of event A and B, and
P [B] is not equal to zero.
Independence: The two events A and B
are said to independent if 𝑷(𝐴 ∩ 𝐵) =
𝑷(𝐴)𝑷(𝐵). If in addition,𝑷(𝐵) > 0,
independence is equivalent to the condition
𝑷(𝐴|𝐵) = 𝑷(𝐴).
Example 1: Suppose that the construction
company has 100 mixer machines. Some of
them use electric power (E) while others are
manual (M) and some machines are new (N)
while others are used (U). If 𝐸 ∩ 𝑁 = 40, the
total number of new machines ( 𝑁) is 60
and 𝑀 ∩ 𝑈 = 10. Find the probability of
getting calculating new machine (N) given
that the selected machine is manual (M).
P(N∩M) 20⁄ 2
Solᵰ: P (N│M) = = 30⁄100 = 3 = 0.67
P (M) 100
Partition, Total probability and
Baye’s Theorems
Definition: A partition of a sample space is
simply the decomposition of the sample
space into a collection of mutually exclusive
events with positive probability. That is,
{B1, B2,…,Bk} form a partition of S if
i) S = B1∪B2 ∪…∪Bk
BY: ABDULMENAN M.(Msc)
PROBABLITY AND STATISTICS LECTURE NOTE
ii) Bi ∩ Bj = ø for all i ≠j
iii) P (Bi) > 0 for all i.
Theorem : Let B1, B2,…,Bk form partition
of the sample space S with P(Bi) ≠0 for
all i and let A be an arbitrary event. Then:
 The total proaility of an event can be
constructe as: P (A) = P (A|B1) P(B1) +
P(A|B2) P(B2) +..........+ P(A|Bk) P(BK)
 Bayes Theorem: recalling the total
probability and multiplication rules, we
have:
P (Bi ∩A)
P (Bi |A) = P(A)
P (A|Bi) P(Bi)
= (HTH) =X (THH) = 2, X (HTT) =X (THT)
P(A)
Example: In a blocket factory, there are three
machines. Machine 1 and Machine 3
manufacture 25% and 40 % of the total
output, respectively. Of their outputs, 5%,
4%, and 2%, respectively, are defective
blokets. A blocket is chosen at random and
found to be defective. What is the probability
that the blocket came from machine 1?
P (D|M1) P(M1)
Solution: P (𝑀1 |D) P(D)
0.0125 0.0125
= 0.0125+0.014+0.008 = 0.0345 = 0.36 (36%)
Exercise: The chance that a doctor A will
diagnose a disease X correctly is 60%. The
chance that a patient will die by his treatment
after correct diagnosis is 40% and the chance
of death by wrong diagnosis is 70%. A
patient of doctor A who had disease X, died.
What is the chance that his disease was
diagnosed correctly?
6
2021
CHAPTER- SIX
ONE DIMENSION RANDOM
VARIABLE
Random Variable (RV): is a variable whose
values are determined by chance. Let X is a
function that assigns a real number X (s) to
every element s ∈ S, and then X is called
Random Variable.
Example 1: Flip a coin three times, find the
number of heads in three tosses. Let X be the
number of heads: X (HHH) = 3, X (HHT) =X
=X (TTH) = 1 and X (TTT) = 0 Hence,
RX= {0, 1, 2, 3} are possible values of
random variable X.
We have two types of random variables:
1. Discrete RV: is a function X(s) from a
finite or countable infinite sample space S
to the real numbers R.
Example: The number of bugs in a
computer program.
2. Continuous RV: is a function X(s) from
an uncountable infinite sample space S to the
real numbers R.
Example: The time to assemble a product
(e.g. a Computer).
Probability distribution (PD): is the
combination of the distinct values 𝑥𝑖 of a
random variable 𝑋 with their probabilities
𝑃(𝑥) or𝑓(𝑥).
Discrete PD (probability mass function):
is a distribution whose random variable is
discrete. Let X be a discrete random variable
whose possible values are X1, X2 …., Xn
with the probabilities P1, P2,..Pn respectively,
BY: ABDULMENAN M.(Msc)
PROBABLITY AND STATISTICS LECTURE NOTE 2021

Then the following two conditions must Table 1: Expectation and Variance of a PD
n
pi ≥ 0 and  P( xi )  1 .
Types of Expected value Variance
satisfy:
i 1
PD (mean)
Discrete E(X) = Var (X) = E(X-  ) 2
Continuous PD (probability density (PMF) n

function): is a distribution whose random  X P X 

i 1
i i
= E (X2) - [( E ( X )]2 =
n

 ( xi ) P  x  - E(X) 2
2
variable is continuous. i
i 1
Note: Probability of a single value is zero
and probability of an interval is the area Continuous E(X) is equal Var (X) = E(X2) -[E(X)]2
bounded by curve of probability density (PDF) b Where:
function and interval on x-axis. Let 𝑎 and 𝑏
to x i f ( xi )dx b

x
2
a E(X2) = i f ( xi )dx
be any two values; then P (a  x  b) area for i= 1, 2,...,n a

under curve b/n 𝑎 and𝑏. Since X must assume

some value, it follows that the total area and a  x  b
under the density curve must equal 1. Example2: Considering example1 above,
≤
construct the discrete PD.
Note: Let X is a continuous random variable
and there exists a function f called X=x 0 1 2 3 Total
probability density function (Pdf) of X if
P(X=x) 1/8 3/8 3/8 1/8 1
the following two conditions satisfy.
(a) Is p(x) legitimate probability function?
I. f (x) ≥ 0 for all x
∞ Why?
II. ∫−∞ f (x)dx = 1
(b) Compute p(x is odd)?
As third important property for any a, b 3 1 1
with −∞< a < b < +∞ we have p(x=1) + p(x=3)= 8 + 8 =2

b (c) Find the probability of getting at most one

P (a ≤ X ≤ b) = ∫a f (x)dx. 1 3 1
head? p(x=0) + p(x=1)= 8 + 8 =2
General properties of expected value and
variance (d) Find the probability of getting at least one
head? P (x=1) + p(x=2) + p(x=3)
If C is a constant then:
1 3 3 7
= + 8 + 8 =8
 E(C) = C and Var (C) = 0 8

 E (CX) = CE(X) and (e) Find the probability of getting exactly

Var (CX) = C2 Var (X) 1
three heads? P (x=3) = 8
 E (X+C) =E(X) +C and Var (X +C)
= Var (X) +0 = Var (X) (f) What is the expected value and variance?
 E(X + Y) = E(X) +E(Y) and 4

Var (X ± Y) = Var (X) + Var (Y), If

E (X) =  X P X  x  = 0(1/8)
i 1
i i +1(3/8) +
X and Y are independent. 2(3/8) +3(1/8) = 1.5

BY: ABDULMENAN M.(Msc)

7
PROBABLITY AND STATISTICS LECTURE NOTE 2021
4
Cumulative Distribution of
Var (X) =  X P X  x   [ E ( X )]
i 1
i
2
i
2

Random Variables
We have been dealing with discrete and
4

 X P X  x 
i 1
i
2
i = 2
0 (1/8) 2
+1 (3/8) + continuous random variables in a somewhat
different manner, using PMFs and PDFs,
22(3/8) +32(1/8) = 3 respectively. It would be desirable to describe
Hence, Var (X) = 3-(1.5)2 = 0.75 all kinds of random variables with a single
mathematical concept. This is accomplished
Example 3: Let X be a continuous r.v and its by the cumulative distribution function or
pdf be: CDF for short. The CDF of a random variable
2𝑥 0≤𝑥≤1 X is denoted by 𝐹𝑋 and provides the
f(x) = { probability P(X ≤ x). In particular, for every
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
x we have
(a) Verify that f is a pdf
1 3 𝐹𝑋 (𝑥) = 𝑃(𝑋 ≤ 𝑥)
(b) Find p (2 ≤ 𝑥 ≤ 4),
∑ 𝑝𝑋 (𝑥) , 𝑖𝑓 𝑋 𝑖𝑠 𝑑𝑖𝑠𝑐𝑟𝑒𝑡𝑒 𝑅. 𝑉
Solution: 𝑘≤𝑥
= 𝑥
a) To be a pdf ∫ 𝑓𝑋 (𝑥)𝑑𝑥, 𝑖𝑓 𝑋 𝑖𝑠 𝑐𝑜𝑛𝑡𝑖𝑛𝑢𝑜𝑢𝑠 𝑅. 𝑉
{ −∞
I. f(x) ≥ 0 for 0 < 𝑥 < 1
∞ 0 Loosely speaking, the CDF 𝐹𝑋 (𝑥)
II. ∫∞ f(x)dx = ∫−∞ f(x)dx +
1 ∞ “accumulates” probability “up to” the value
∫0 f(x)dx + ∫1 f(x)dx = 1 x.
1
= 0 + ∫0 2xdx + 0 = x 2 |10 = 1There for it
is a pdf.
3 3
1 3
b) P (2 X < 4) = ∫14 f(x)dx =∫14 2xdx
2 2
3
𝟑 𝟏 𝟓
== x 2 | = (𝟒)𝟐 − (𝟐)𝟐 = 𝟏𝟔.
4
1
2

Exercise 1: Suppose X is a continuous

random variable with pdf
1+x −1≤x≤0
f(x) = {
1−x 0≤x≤1
Find the expected value and variance of X.

BY: ABDULMENAN M.(Msc)

8
PROBABLITY AND STATISTICS LECTURE NOTE 2021

TWO DIMENSION RV
(JOINT DISTRIBUTIONS)
Probabilistic models often involve several
random variables of interest. For example, in
a medical diagnosis context, the results of
several tests may be significant, or in a
networking context, the workloads of several
gateways may be of interest. All of these
random variables are associated with the
same experiment, sample space, and
probability law, and their values may relate
in interesting ways. This motivates us to
consider probabilities involving
simultaneously the numerical values of
several random variables and to investigate This page intentionally left
blank
their mutual couplings. In this section, we
will extend the concepts of 𝑝𝑚𝑓 or 𝑝𝑑𝑓 and
expectations developed so far to multiple
random variables.

i) Discrete RV Conditions
1. 𝑃(𝑋 = 𝑥, 𝑌 = 𝑦) ≥ 0

2. ∑𝑛𝑖 ∑𝑚
𝑗 𝑃(𝑋 = 𝑥, 𝑌 = 𝑦) =1

ii) Continuos RV Conditions

1. 𝑓(𝑋 = 𝑥, 𝑌 = 𝑦) ≥ 0
∞ ∞
2. ∫−∞ ∫−∞ 𝑓𝑋,𝑌 (𝑥, 𝑦)dydx =1

Definition: If X and Y are any two random

variables, the conditional expectation of X
given that Y = y is defined to be
 E(X |Y=y) = ∑𝑛𝑖 𝑥𝑖 𝑃(X |Y = y)
if X and Y are jointly discrete
∞
 E(X |Y=y) = ∫−∞ 𝑥 𝑓𝑋 (X |Y = y)dx
if X and Y are jointly continuous.

BY: ABDULMENAN M.(Msc)

9
 PROBABLITY AND STATISTICS LECTURE NOTE 2021

Properties Discrete Distribution Function Continuous Distribution Function

Definitions It is called probability mass It is also known as probability
function (pmf) density function (pdf)
A joint pmf of two random A joint pdf of two random
variables 𝑋 and 𝑌 can be variables 𝑋 and 𝑌 can be
construct as: construct as:
𝑝𝑋,𝑌 (𝑥, 𝑦) = 𝑃(𝑋 = 𝑥, 𝑌 = 𝑦) 𝑓𝑋,𝑌 (𝑥, 𝑦) = 𝑓(𝑋 = 𝑥, 𝑌 = 𝑦)
∞
Marginal distribution 𝑝𝑋 (𝑥) = 𝑃(𝑋 = 𝑥) , for 𝑎𝑙𝑙 𝑥 𝑓𝑋 (𝑥) = ∫−∞ 𝑓𝑋,𝑌 (𝑥, 𝑦)dy,
functions 𝑝𝑌 (𝑦) = 𝑃(𝑌 = 𝑦) , for 𝑎𝑙𝑙 𝑦 for 𝑎𝑙𝑙 𝑥
∞
𝑓𝑌 (𝑦) = ∫−∞ 𝑓𝑋,𝑌 (𝑥, 𝑦)dx ,
for 𝑎𝑙𝑙 𝑦

Independence random Two discrete random variables 𝑋 and Two continuous random variables 𝑋
variables via marginal 𝑌 are independent if and 𝑌 are independent if
distribution functions 𝑝𝑋,𝑌 (𝑥, 𝑦) = 𝑝𝑋 (𝑥) 𝑝 𝑌 (𝑦) , 𝑓𝑋,𝑌 (𝑥, 𝑦) = 𝑓𝑋 (𝑥) 𝑓𝑌 (𝑦)
for all x and y for all x and y
Conditional distributions 𝑃(𝑋=𝑥,𝑌=𝑦) 𝑓(𝑋=𝑥,𝑌=𝑦)
𝑃(𝑋 = 𝑥|𝑌 = 𝑦) = , 𝑓(𝑋 = 𝑥|𝑌 = 𝑦) = ,
𝑃(𝑌=𝑦) 𝑓𝑌 (𝑦)
𝑃(𝑌 = 𝑦) > 0 𝑓𝑌 (𝑦) ≠ 0
𝑃(𝑋=𝑥,𝑌=𝑦) 𝑓(𝑋=𝑥,𝑌=𝑦)
𝑃(𝑌 = 𝑦|𝑋 = 𝑥) = , 𝑓(𝑌 = 𝑦|𝑋 = 𝑥) = ,
𝑃(𝑋=𝑥) 𝑓𝑋 (𝑥)
𝑃(𝑋 = 𝑥) > 0 𝑓𝑋 (𝑥) ≠ 0

∞
Expectation 𝐸[𝑋] = ∑𝑛𝑖 𝑥𝑖 𝑃(𝑋 = 𝑥𝑖 ) 𝐸[𝑋] = ∫−∞ 𝑥 𝑓𝑋 (𝑥)dx
𝐸[𝑌] = ∑𝑚 𝑗 𝑦𝑗 𝑃(𝑌 = 𝑦𝑗 )
∞
𝐸[𝑌] = ∫−∞ 𝑦 𝑓𝑌 (𝑦)dy
𝐸[𝑋𝑌] has the form of 𝐸[𝑋𝑌] has the form of
∑𝑛𝑖 ∑𝑚
𝑗 (𝑥𝑖 , 𝑦𝑗 ) 𝑃(𝑋 = 𝑥𝑖 , 𝑌 = ∞ ∞
∫−∞ ∫−∞ 𝑥𝑦 𝑓𝑋,𝑌 (𝑥, 𝑦)dydx
𝑦𝑗 )
Variance 𝑉𝑎𝑟(𝑋) = 𝐸[𝑋 2 ] - (𝐸[𝑋])2 𝑉𝑎𝑟(𝑋) = 𝐸[𝑋 2 ] - (𝐸[𝑋])2
𝑉𝑎𝑟(𝑌) = 𝐸[𝑌 2 ] - (𝐸[𝑌])2 𝑉𝑎𝑟(𝑌) = 𝐸[𝑌 2 ] - (𝐸[𝑌])2
Standard deviation 𝛿(𝑋) = √𝑉𝑎𝑟(𝑋) 𝛿(𝑋) = √𝑉𝑎𝑟(𝑋)
𝛿(𝑌) = √𝑉𝑎𝑟(𝑌) 𝛿(𝑌) = √𝑉𝑎𝑟(𝑌)
Covariance The Covariance of two discrete The Covariance of two continuous
random variables 𝑋 and 𝑌 are given random variables 𝑋 and 𝑌 are given
as: as: 𝐶𝑜𝑣(𝑋, 𝑌) = 𝐸[𝑋𝑌] - 𝐸[𝑋] 𝐸[𝑌]
𝐶𝑜𝑣(𝑋, 𝑌) = 𝐸[𝑋𝑌] - 𝐸[𝑋] 𝐸[𝑌]
Independence random If the two discrete random variables 𝑋 If the two continuous random variables
variables via Expectation and and 𝑌 are independent, then 𝑋 and 𝑌 are independent, then
Covariance 𝐸[𝑋𝑌] = 𝐸[𝑋] 𝐸[𝑌] 𝐸[𝑋𝑌] = 𝐸[𝑋] 𝐸[𝑌]
i.e, 𝐶𝑜𝑣(𝑋𝑌) = 0 i.e, 𝐶𝑜𝑣(𝑋𝑌) = 0
Table2: General concepts of joint probability distribution for discrete and continuous cases.

BY: ABDULMENAN M.(Msc)

10
PROBABLITY AND STATISTICS LECTURE NOTE 2021

Exercise 1: Throw a fair die. Suppose: CHAPTER-SEVEN

X equals 1 if we get a number greater than or
equal to 4 and X equals 0 otherwise.
Y equals 1 if we get an even number Y equals
0 otherwise.Then,
a) Find the marginal distribution
functions.
b) Find the conditional distributions.
c) Show their independency through
marginal distribution function case.
d) Find the expectation, variance,
standard deviation and covariance.
e) Show their independency through
expectation and covariance.
Exercise 2: The fraction X of male runners
and the fraction Y of female runners who
compete in marathon races is described by
the joint density function
8xy, 0  x  1,0  y  1.
𝑓(𝑥, 𝑦) =
0, elsewhere
Then,
a. Find the marginal distribution
functions.
b. Find the conditional distributions.
c. Show their independency through
marginal distribution function case.
d. Find the expectation, variance,
standard deviation and covariance.
e. Show their independency through
expectation and covariance.
COMMON PROBABILITY
DISTRIBUTIONS
7.1 Common Discrete Probability
Distributions
Even though there are several Common
discrete Probability distributions, but in this
section we will discuss briefly only about the
two most important binomial and Poisson
distributions.
Binomial Distribution
Definition : A discrete random variable X
is said to follow a Binomial distribution with
parameters n and p, written as X ~ Bi(n , p)
or X ~ B(n , p), if its distribution is given by
P(X = x) =
(n )px (1 − p)n−x for x = 0,1,2, … , n
{ x .
0 elsewhere
Where, x = number of successes assumes
values.
n = number of trails.
p = probability of success; 0 ≤ p ≤ 1.
The trials must meet the following
requirements to say the random
variable follows binomial distribution
I. The total number of trials is fixed in
advance.
II. There are just two outcomes of each
trial success and failure
III. the outcomes of all the trials are
statistically independent
IV. All the trials have the same
probability of success.

BY: ABDULMENAN M.(Msc)

11
PROBABLITY AND STATISTICS LECTURE NOTE 2021

Let X be a binomially distributed 10

𝑃(𝑋 = 0) = ( ) (0.25)0 (0.75)10
random variable with parameters p, 0
based on n repetitions of an = 0.0563
experiment. Then, E(X) = np
and Var(X) = npq. 10
𝑃(𝑋 = 1) = ( ) (0.25)1 (0.75)9
1
Example 1:- A given mid-exam contains 10 = 0.1877
multiple choice questions, and each question
𝑃(𝑋 = 2)
has four alternatives with one exact answer.
10
Find the probability that the student exactly = ( ) (0.25)2 (0.75)8
2
answered:
= 0.2816
i. 3 questions
. ’. 𝑃(𝑋 ≥ 3)
ii. At least 3 questions
= 1 – (0.0563 + 0.1877
iii. Find the mean and variance of RV
+ 0.2816) = 0.4744
X.
iii. The mean = 𝑛𝑝 = 2.5
Solution: Using binomial distribution we
The variance = 𝑛𝑝𝑞 = 1.875
can get the probability value easily. That
is n =10, p = ¼ (the chance of getting
answer from 4 alternatives)
Poisson distribution
Definition: A random variable X is said to
q = 1- p = 1- ¼ = ¾ have a Poisson distribution with parameter
λ>0 if its distribution is given by,
The possible marks for a student from 10
questions are X = 0, 1, 2, 3. . . 10. 𝐞−𝛌 𝛌𝐱
𝐏(𝐗 = 𝐱) = { 𝐟𝐨𝐫 𝐱 = 𝟎, 𝟏, 𝟐, …. .
𝐱!
𝑛 𝟎𝐞𝐥𝐬𝐞𝐰𝐞𝐫𝐞
𝑃(𝑋 = 𝑥) = ( ) 𝑝 𝑥 𝑞 𝑛−𝑥
𝑥 Where x=count of the number of events that
occur in a certain time interval or spatial area.
i. 𝑃(𝑋 = 3) =
(10)(0.25)3 (0.75)7 = 0.250 We write as 𝐗~𝐏𝐢𝐬(𝛌).
3
ii. 𝑃(𝑋 ≥ 𝑥) = 1 − 𝑃(𝑋 <
 = np = mean number of times an event
𝑥).
occurs.
Hence 𝑃(𝑋 ≥ 3) = 1 – 𝑃(𝑋 < 3)
When an event occurs rarely, the
number of occurrences of such an
= 1 – {𝑃(𝑋 = 0) + 𝑃(𝑋 = 1) +
event may be assumed to follow a
𝑃(𝑋 = 2)}
Poisson distribution .The following
are some of the examples, which can

BY: ABDULMENAN M.(Msc)

12
PROBABLITY AND STATISTICS LECTURE NOTE 2021

be analyzed using Poisson 7.2 Common Continuous

distribution: Probability Distribution
 The number of Bacteria in a given
Normal Distribution
volume of water.
 The number of defective articles in a The most important and widely used
packet of 200. probability distribution is normal
 The number of road accidents distribution. It is also known as Gaussian
reported in a city per day. distribution. Most of the distributions
occurring in practice, for instance, binomial,
Note: The special property of Poisson Poisson, etc., can be approximated by normal
distribution is that its mean and variance distribution.
are same to  . I.e. mean = variance =  . Further, many of the sampling distributions
like Student’s t, F, & χ2 distributions tend to
Example 1:- Connections arrive at a switch
normality for large samples. Therefore, the
at a rate of 11 per minutes. The arrival
normal distribution finds an important place
distribution is Poisson. Then what is the
in statistical inference. Its probability density
probability that
function is expressed by the relation,
(a) Exactly 11 calls arrive in one minutes?
1  X   
2
1
f(x) = e 2   
(b) Exactly 100 calls arrive in 10 minutes?  2  
(c) The number of calls arriving in 2 minutes
is greater than 7 and less than or equal to 10? In the above formula,  = population mean.

Solutions: Let Xi be the random variable  =population standard deviation.

giving the number of arrivals during t-
minutes. We know that Xi has a Poisson x= a given value of the rv in the
distribution, which implies thatP(X = x) = range -   x  .
𝐞−𝛌 𝛌𝐱
,
𝐱! For a normal distribution the
a) Probability of exactly 11 arrivals in one
frequency curve will be symmetrical
(11)11 ∗e−11
minute is P(X1 = 11) = = 0.119 or bell shaped. However, not all
11!
b) Probability of 100 calls in 10 minute is symmetrical curves are normal.
(11∗10)100 ∗e−10∗11 The shape of the normal curve is
P(X10 = k) = = 0.025
100! completely determined by two
(11∗2)k ∗e−11∗2
c) P(7 < X2 ≤ 10) = ∑10
k=8 = parameters  &  . For any given  ,
k!
228 1 22 222 228 794 there can be a number of normal
(8! + + )= ( 10! ) = 0.003
e22 9! 10! e22 curves each with a different  .
Likewise, for any given  , there can
be a number of normal curves each
with a different  . In order to make

BY: ABDULMENAN M.(Msc)

13
PROBABLITY AND STATISTICS LECTURE NOTE 2021

such all distributions readily

comparable with each other, their
individuality as expressed by their
mean and standard deviation has to be
suppressed. This is done by
transforming the normal variable into
standard normal variable.
The standard normal variable is
denoted by Z and is given by Z =
X 
. The distribution of the
 ii) P(X > 0) = P(Z > −1) =
standard normal variable is known as 1 − P(Z < 1)= 0 .8413.
standard normal distribution. It is
1 1 2
given by f(x) = e z in the
2 2
range -   z   . For standard
normal distribution,
  0 and   1. Tables are
readily available for different values
of Z. Because of the symmetrical
nature of the normal distribution the iii) P(X > 9) = 𝑃(𝑍 > 2.0) =
tables are presented only for the 0.5 − 0.4772 = 0 .0228
positive values of Z.

Note: Area under curve is equal to one. But

area above or below z = 0 is 0.5.

Example: If X is a normal random variable

with parameters μ = 3 and σ2 = 9, find

i) P (2 < X <5) Exercise: The length of life of a certain type

ii) P(X >0) of automatic washer is approximately
iii) P(X >9). normally distributed, with a mean of 3.1
Solution: First we have to standardize the years and standard deviation of 1.2 years. If
random variance X. this type of washer is guaranteed for 1 year,
what fraction of original sales will require
2−3
i) P(2 < X <5) = P ( <𝑍< replacement?
3
5−3
) =P(−0.33 < Z <0.67)
3
=0.3779 (from table)

BY: ABDULMENAN M.(Msc)

14
PROBABLITY AND STATISTICS LECTURE NOTE 2021

Standard normal table(Z-table)

Area between 0 and z

0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

0.0 0.0000 0.0040 0.0080 0.0120 0.0160 0.0199 0.0239 0.0279 0.0319 0.0359

0.1 0.0398 0.0438 0.0478 0.0517 0.0557 0.0596 0.0636 0.0675 0.0714 0.0753

0.2 0.0793 0.0832 0.0871 0.0910 0.0948 0.0987 0.1026 0.1064 0.1103 0.1141

0.3 0.1179 0.1217 0.1255 0.1293 0.1331 0.1368 0.1406 0.1443 0.1480 0.1517

0.4 0.1554 0.1591 0.1628 0.1664 0.1700 0.1736 0.1772 0.1808 0.1844 0.1879

0.5 0.1915 0.1950 0.1985 0.2019 0.2054 0.2088 0.2123 0.2157 0.2190 0.2224
0.6 0.2257 0.2291 0.2324 0.2357 0.2389 0.2422 0.2454 0.2486 0.2517 0.2549

0.7 0.2580 0.2611 0.2642 0.2673 0.2704 0.2734 0.2764 0.2794 0.2823 0.2852

0.8 0.2881 0.2910 0.2939 0.2967 0.2995 0.3023 0.3051 0.3078 0.3106 0.3133

0.9 0.3159 0.3186 0.3212 0.3238 0.3264 0.3289 0.3315 0.3340 0.3365 0.3389

1.0 0.3413 0.3438 0.3461 0.3485 0.3508 0.3531 0.3554 0.3577 0.3599 0.3621
1.1 0.3643 0.3665 0.3686 0.3708 0.3729 0.3749 0.3770 0.3790 0.3810 0.3830

1.2 0.3849 0.3869 0.3888 0.3907 0.3925 0.3944 0.3962 0.3980 0.3997 0.4015

1.3 0.4032 0.4049 0.4066 0.4082 0.4099 0.4115 0.4131 0.4147 0.4162 0.4177

1.4 0.4192 0.4207 0.4222 0.4236 0.4251 0.4265 0.4279 0.4292 0.4306 0.4319

1.5 0.4332 0.4345 0.4357 0.4370 0.4382 0.4394 0.4406 0.4418 0.4429 0.4441

1.6 0.4452 0.4463 0.4474 0.4484 0.4495 0.4505 0.4515 0.4525 0.4535 0.4545

1.7 0.4554 0.4564 0.4573 0.4582 0.4591 0.4599 0.4608 0.4616 0.4625 0.4633

1.8 0.4641 0.4649 0.4656 0.4664 0.4671 0.4678 0.4686 0.4693 0.4699 0.4706

1.9 0.4713 0.4719 0.4726 0.4732 0.4738 0.4744 0.4750 0.4756 0.4761 0.4767
2.0 0.4772 0.4778 0.4783 0.4788 0.4793 0.4798 0.4803 0.4808 0.4812 0.4817
2.1 0.4821 0.4826 0.4830 0.4834 0.4838 0.4842 0.4846 0.4850 0.4854 0.4857

2.2 0.4861 0.4864 0.4868 0.4871 0.4875 0.4878 0.4881 0.4884 0.4887 0.4890

2.3 0.4893 0.4896 0.4898 0.4901 0.4904 0.4906 0.4909 0.4911 0.4913 0.4916

2.4 0.4918 0.4920 0.4922 0.4925 0.4927 0.4929 0.4931 0.4932 0.4934 0.4936
2.5 0.4938 0.4940 0.4941 0.4943 0.4945 0.4946 0.4948 0.4949 0.4951 0.4952

2.6 0.4953 0.4955 0.4956 0.4957 0.4959 0.4960 0.4961 0.4962 0.4963 0.4964
2.7 0.4965 0.4966 0.4967 0.4968 0.4969 0.4970 0.4971 0.4972 0.4973 0.4974
2.8 0.4974 0.4975 0.4976 0.4977 0.4977 0.4978 0.4979 0.4979 0.4980 0.4981
2.9 0.4981 0.4982 0.4982 0.4983 0.4984 0.4984 0.4985 0.4985 0.4986 0.4986
3.0 0.4987 0.4987 0.4987 0.4988 0.4988 0.4989 0.4989 0.4989 0.4990 0.4990

BY: ABDULMENAN M.(Msc)

15