General Definitions probability and

Concepts
Roba HS (MPH, Assistant Professor)
School of Public Health|CHMS|HU
January 2021

1/4/2021 1
 Basic Probability Concepts
• Probability is the foundation for making inference about the
population based on the sample as representative part of the
population.
• Frequently encountered in everyday communication
• E.g. we may hear a say that a patient has a 50–50 chance of
surviving a certain operation.
• Another physician may say that she is 95 percent certain that a
patient has a particular disease

1/4/2021 2
 Probability

• A measure used to measure the chance of the occurrence of

some event.

• The more likely the event, the closer the number is to one

• The more unlikely the event, the closer the number is to zero.

• An event that cannot occur has a probability of zero, and an

event that is certain to occur has a probability of one.

1/4/2021 3
• Experiment
– a procedure or process that we perform whose outcomes
are not predictable in advance
– Example:
• Experiment with coin tossing results in two outcomes,
head or tail.
• the two outcomes, head and tail, are exhaustive, because
one of these two outcomes must occur in each
experiment and there are no other possible outcomes in
this experiment

1/4/2021 4
• Sample Space
– The sample space of an experiment is the set of all possible
outcomes of an experiment.

– it is the universal set and is denoted by Ω.

– Example

• In the coin tossing experiment with a single coin, the

possible outcomes are head (H) or tail (T).

• Hence, the sample space is Ω= (H, T)

1/4/2021 5
• Event
– Any subset of the sample space X is called an event.

– E.g. in the coin tossing experiment, an event called success

may occur if the outcome is a head (H).

– If a tail (T) appears, then it may be called failure

1/4/2021 6
• Let us consider selecting a patient from a hospital room with
six beds numbered from 1 to 6 and observing the patient of the
selected bed.

• Here, the patients are identified by their respective bed

numbers.

• This experiment has six possible outcomes or elements.

1/4/2021 7
• The sample space is Ω={1, 2, 3,4,5, 6}.
• Consider the following events and the elements corresponding
to the events:

Notation n(Ω) = number of outcomes (elements) in Ω and

n(E)=number of outcomes (elements) in the event E.

1/4/2021 8
• Equally Likely Outcomes
– The outcomes of an experiment are equally likely if the
outcomes have the same chance of occurrence

– an experiment resulting in equally likely outcomes is

equally probable

– It implies that if the sample space contains n equally likely

outcomes, then it is likely that probability of each outcome
is 1/n.

1/4/2021 9
• Mutually Exclusive Outcomes
– In an experiment, if only one outcome is observed at a time
excluding the occurrence of any other outcome, then it is
called mutually exclusive.

– Example:

• In the coin tossing experiment, if the outcome is

observed to be head, the occurrence of tail is not
possible at the same time and vice versa

1/4/2021 10
 Probability of an Event
• If the experiment has n(Ω) equally likely outcomes, then the
probability of the event E is denoted by P(E) and is defined
by

• This is the classical definition of probability and the

experiment should satisfy the condition of equally likely as an
essential precondition which may not be true in many practical
situations.

1/4/2021 11
• If the experiment doesn’t equally likely outcomes, alternative
definition known as the relative frequency or empirical
definition to measure the probability

• Let an event E occurs n(E) times in a series of n trials where n

is the total number of trials or sample size, the trials are
conducted under the same conditions in the experiment

1/4/2021 12
• Here, the ratio n(E)
n
is the relative frequency of the event E in n
trials.
• If n tends to infinity, then we can define the probability of E as
follows:
𝑛(𝐸)
P(E)= lim
𝑛→∞ 𝑛
• suppose the bed number is selected at random for six patients
identified
• Determine the probabilities of the following events:

1/4/2021 13
1/4/2021 14
 Some Operations on Events
• Let A and B be two events defined on the sample space Ω.
1. Union of Two events: (AUB)
– The event AUB consists of all outcomes in A or in B or in
both A and B.
– The event AUB occurs if A occurs, or B occurs, or both A
and B occur.

1/4/2021 15
2. Intersection of Two Events: (AՈB)

– The event AՈB consists of all outcomes in both A and B.

– The event AՈB occurs if both A and B occur.

1/4/2021 16
3. Complement of an Event: 𝑨 or (AC) or (A′)
– The complement of the event A is denoted by 𝐴.

– The event A consists of all outcomes of Ω that are not in A.

– The event 𝐴 occurs if A does not.

1/4/2021 17
 Example: (Classical Probability)
• Experiment: Selecting a patient randomly from a hospital
room having six beds numbered 1, 2, 3, 4, 5, and 6.

• Define the following events:

(1) E1UE2= {1; 2, 3, 4, 6} selecting an even number or a number

less than 4:

1/4/2021 18
2) E1UE4= {1, 2, 3, 4, 5, 6}= Ω =selecting an even number or an
odd number.

• It can be shown that E1UE4 = Ω where E1 and E4 are called

exhaustive events.

• The union of these events gives the whole sample space.

1/4/2021 19
3) E1ՈE2 ={2} = selecting an even number and a number less
than 4.

4) E1ՈE4 =∅= selecting an even number and an odd number.

1/4/2021 20
• E1ՈE2 =∅= In this case, E1 and E4 are called disjoint (or
mutually exclusive) events.

• These kinds of events cannot occur simultaneously (together at

the same time).

1/4/2021 21
5) The complement of E1

1/4/2021 22
• Mutually Exclusive (Disjoint) Events
– The events A and B are disjoint (or mutually exclusive) if
E1ՈE2 =∅
I. P(AՈB)=0
II. P(AUB)=P(A) + P(B)

1/4/2021 23
• Exhaustive Events

1/4/2021 24
 General Probability Rules

1/4/2021 25
 The Addition Rule

1/4/2021 26
1/4/2021 27
 Marginal Probability
• Given some variable that can be broken down into (m) categories
designated by A1, A2, . . ., Am and another jointly occurring
variable that is broken down into (s) categories designated by B1,
B2, . . . , Bs
• The marginal probability of Ai, P(Ai), is equal to the sum of the
joint probabilities of Ai with all categories of B.
• That is

1/4/2021 28
 Example: Relative Frequency or Empirical
• Let us consider a bivariate table for variables A and B.
• There are three categories for both the variables, A1, A2,and A3
for A and B1, B2, and B3 for B
Joint frequency distribution for m categories of A and s categories of B

1/4/2021 29
• Joint probability distribution for m categories of A and s
categories of B

1/4/2021 30
• Number of elements in each cell

Probabilities of events

1/4/2021 31
 Applications of Relative Frequency (Empirical Probability)

• Let us consider a hypothetical data on four types of diseases of

200 patients from a hospital as shown below:
Disease type A B C D Total
Number of patients 90 80 20 10 200

• Experiment: Selecting a patient at random and observe his/her

disease type. Total number of trials, sample size, in this case, is
n =200

1/4/2021 32
1/4/2021 33
 Conditional Probability
• The conditional probability of the event A when we know that
the event B has already occurred is defined by

1/4/2021 34
1/4/2021 35
 Multiplication Rules of Probability

Let us consider a hypothetical set of data on 600 adult males

classified by their ages and smoking habits as summarized
Consider the following event:
(B1|A2) = smokes daily given that age is between 30 and 39

1/4/2021 36
• Two-way table displaying number of respondents by age and
smoking habit of respondents smoking habit

1/4/2021 37
1/4/2021 38
 Probability distribution

• The relationship between the values of a random variable and

the probabilities of their occurrence

• Expressed in the form of a table, graph, or formula

• Provides the clinician and researcher with a powerful tool for

summarizing and describing a set of data and for reaching
conclusions about a population of data on the basis of a sample
of data drawn from the population.

1/4/2021 39
 Probability Distributions for Discrete Random
Variables
• A table, graph, formula, or other device used to specify all
possible values of a discrete random variable along with their
respective probabilities
• shows the probability associated with each value of the
discrete random variable.

• The relative frequency based on the sample data approximates

the probability distribution for a large sample size.

1/4/2021 40
 Probability Distributions,…
• The main properties of a discrete random variable are:
– the probability of every value of a variable lies between 0
and 1
– the sum of all probabilities of the variable is equal to 1
– the probabilities of a discrete random variable are additive
• If we let the discrete probability distribution be represented by
P(x), then P(x)= P(X=x) is the probability of the discrete
random variable X to assume a value x.

1/4/2021 41
 Probability Distributions,…
• The probability distribution of a discrete variable, X, can be
shown for different number of outcome values of the variable.

• Some examples are shown below:

• P(X =x); x =0, 1.
• P(X=x); x = 0, 1, . . . K.
• P(X = x); x =0, 1, . . .∞.

1/4/2021 42
 Probability Distributions,…
Frequency distribution of number of injury accidents in a day

Possible values of the random variable X are X = 0, 1, 2, 3, 4, 5.

Experiment: Selecting a day at random

1/4/2021 43
 Probability Distributions,…
• (X = 0) = The event that on the selected day there was no accident,

• (X = 1) = The event that on the selected day there was one accident,

• (X = 2) = The event that on the selected day there were 2 accidents,

• (X = 3) = The event that on the selected day there were 3 accidents,

• (X = 4) = The event that on the selected day there were 4 accidents,

• (X = 5) = The event that on the selected day there were 5 accidents.

1/4/2021 44
 Probability Distributions,…

• In general
– (X = x) = the event that on the selected day there were x
accidents.
– Using the relative frequency definition, we know that n =
639.
– The number of elements of the event (X = x) is n(X = x) =
frequency of x.
𝑛 𝑋=𝑥 𝐹𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦
– P(X =x)= =relative frequency =
639 639

1/4/2021 45
 Probability Distributions,…
Probability distribution of number of accidents in a day

1) What the probability that one accident would occur?

Solution: P(X=1)=0.3380

1/4/2021 46
 Probability Distributions,…
2) What is the probability that two or more accidents?

Solution:

P(X≥2)=P(X=2) + P(X=3) + P(X=4) + P(X=5)

= 0.1140 + 0.1469 + 0.0219 + 0.0016

= 0.2144

1/4/2021 47
 Probability Distributions,…
0.45
0.4
0.35
0.3
P(X=x)

0.25
0.2
0.15
0.1
0.05
0
0 1 2 3 4 5
X=x
The graphical presentation of this probability distribution

1/4/2021 48
 Expected Value and Variance of a Discrete Random
Variable
• The population mean and variance are defined using
corresponding probabilities of a variable

• The population mean is defined as the expected value of a

variable and it can be expressed as follows:

– The population mean or expected value of a discrete

random variable X is denoted by µ or µx

E(X)=μ= 𝐱 𝐱(𝐏(𝐗 = 𝐱))

1/4/2021 49
 Similarly, the population variance of a discrete random variable
X is denoted by 𝜎2 or 𝜎x2 and it is defined by
Var(X)=E(X-μ)2 =σ2 = 𝒙 (X−μ)2P(X=x)
• Example: calculate the μ and the 𝜎2 of the discrete random
variable X whose probability distribution is given by the
following table

1/4/2021 50
• Solution
X P(X=x) xP(=x) (x-μ) (x-μ)2 (x-μ)2 P(X=x)
0 0.05 0 -1.9 3.61 0.1805
1 0.25 0.25 -0.9 0.81 0.2025
2 0.45 0.90 0.1 0.01 0.0045
3 0.25 0.75 1.1 1.21 0.3025
μ= 𝒙 𝒙𝑷(𝑿 = 𝒙) σ2 = 2P(X=x)=0.69
Total 𝒙 (X−μ)
=1.9

1/4/2021 51
 Cumulative Distributions

• The cumulative probability distribution for the discrete

variable whose probability distribution

• Obtained by successively adding the probabilities, P (X =xi)

• The cumulative probability for xi is written as F(xi)=P(X≤ xi)

• It gives the probability that X is less than or equal to a

specified value, xi

1/4/2021 52
 Cumulative Distributions,…

Cumulative probability distribution

1/4/2021 53
 The binomial distribution

• Widely encountered probability distributions in applied

statistics

• Derived from a process known as a Bernoulli trial

• Arises from trial that result in only one of two mutually

exclusive outcomes, such as dead or alive, sick or well, full-
term or premature
• Outcomes which are termed traditionally as success or failure.

1/4/2021 54
 The bernoulli process
1. Each trial results in one of two possible, mutually exclusive,
outcomes denoted (arbitrarily) as a success, and the other is
denoted a failure.
2. The probability of a success, denoted by p, remains constant
from trial to trial. The probability of a failure, 1- p, is denoted
by q.
3. The trials are independent; that is, the outcome of any
particular trial is not affected by the outcome of any other
trial.

1/4/2021 55
 The Bernoulli Process,…

4. The probabilities of success or failure are complementary to

each other so that if the probability of success is denoted by p
then the probability of failure is 1-P, where p satisfies 0≤p ≤
1.

1/4/2021 56
 Binomial Distribution,…
• example;

– if all birth records for a calendar year shows that 85.8% of

the pregnancies had delivery in week 37 or later.

– The 85.8% interpreted as the probability of a recorded birth

in week 37 or later

– If we randomly select five birth records from this

population, what is the probability that exactly three of the
records will be for full-term births?

1/4/2021 57
 Binomial Distribution,…

• Let us designate the occurrence of a record for a full-term birth

(F) as a “success” and hasten to add that a premature birth (P)
is not a failure

• It will also be convenient to assign the number 1 to a success

and the number 0 to a failure (record of a premature birth).

1/4/2021 58
 Binomial Distribution,…

• Suppose the five birth records selected resulted in this

sequence of full term births

PFPPF
• In coded form we would write this as
10110
P(1, 0, 1, 1, 0,) =pqppq=q2p3

1/4/2021 59
 Binomial Distribution,…
• Three successes and two failures could occur in any one of the
following additional sequences as well:

From the addition rule we know that this

probability is equal to the sum of the
individual probabilities. In the present
example we need to sum the 10q2p3’s or,
equivalently, multiply q2p3 by 10.

1/4/2021 60
 Binomial Distribution,…

• Answer for original question is

• Since in the population, p=0.858;

q=(1-p)=(1- 0.858)=0.142

10q2p3 =10(0.142)2(0.858)3
=10 (0.0202)(0.6316)
= 0.1276

1/4/2021 61
 Combinations
• A combination of n objects taken x at a time is an unordered
subset of x of the n objects
• Combination is used in large sample procedures
• The number of combinations of n objects that can be formed
by taking x of them at a time is given by
𝑛!
n𝐶 x=𝑥! 𝑛−𝑥 !

• where x!, read x factorial, is the product of all the whole

numbers from x down to 1. That is,

x! =x(x-1)(x-2)…(1). We note that, by definition, 0!=1.

1/4/2021 62
 Binomial Distribution,…
• Let us return to our example in which we have a sample of
n=5 birth records and we are interested in finding the
probability that three of them will be for full-term births

5! 5𝑥4𝑥3𝑥2𝑥1 120
5𝐶 3= 3! 5−3 ! = (3𝑥2𝑥1) 2𝑥1
= 12
= 10

1/4/2021 63
 Binomial Distribution,…
• In our example we let x =3, the number of successes, so that n-
x = 2, the number of failures. We then may write the
probability of obtaining exactly x successes in n trials as

f(x)= n𝑪x 𝒒𝒏−𝒙 𝒑𝒙 =n𝑪x 𝒑𝒙 𝒒𝒏−𝒙 , for x=0, 1, 2, …,n

• The Binomial Parameters
– binomial distribution has two parameters, n and p
– μ=np,
– σ2=np(1-p) = npq

1/4/2021 64
 Poisson Distribution
• Used to model a discrete random variable representing the
number of occurrences or counts of some random events in an
interval of time or space (or some volume of matter)

• The possible values of X = x are x = 0, 1, 2, 3,…

• The discrete random variable, X, is said to have a Poisson

distribution with parameter (mean) λ if the probability

𝑒 −λ λ𝑥
distribution of X is given by f(x) = 𝑥!

1/4/2021 65
 Poisson Distribution,…
• where e = 2.71828 (the natural number).

• λ (lambda) is the parameter of the distribution and is the

average number of occurrences of the random event in the
interval

• The Poisson Process

– The occurrences of the events are independent

– The probability of the single occurrence of the event in a
given interval is proportional to the length of the interval

1/4/2021 66
 Poisson Distribution,…
– In any infinitesimally small portion of the interval, the
probability of more than one occurrence of the event is
negligible
Example

– In a study of drug-induced anaphylaxis among patients

taking rocuronium bromide as part of their anesthesia the
occurrence of anaphylaxis followed a Poisson model with
λ=12 incidents per year in Norway

1/4/2021 67
 Poisson Distribution,…
– Find the probability that in the next year, among patients
receiving rocuronium, exactly three will experience
anaphylaxis

𝑒 −12 123
f(x=3) = = 0.00177
3!

• What is the probability that at least three patients in the next

year will experience anaphylaxis if rocuronium is administered
with anesthesia?

1/4/2021 68
 Poisson Distribution
• Example: Suppose that the number of accidents per day in a city has
a Poisson distribution with average 2 accidents.

1. What is the probability that in a day

I. the number of accidents will be 5,

II. the number of accidents will be less than 2.

2. What is the probability that there will be six accidents in 2

days?

3. What is the probability that there will be no accidents in an

hour?

1/4/2021 69
 Poisson Distribution,…

𝑒 −2 25
1. P (X =5) = = 0.036089
5!

2. P(X<2)=P(X=0) + P(X=1) =

𝑒 −2 20 𝑒 −2 21
= + =0.135335 + 0. 270670= 0.406005.
0! 1!

1/4/2021 70
 Probability Distributions of Continuous data
• A non-negative function f(x) of the continuous random
variable X if the total area bounded by its curve and the x -axis
is equal to 1 and if the subarea under the curve bounded by the
curve, the x -axis, and perpendiculars erected at any two points
a and b give the probability that X is between the points a and
b.

• Also known as probability density function

1/4/2021 71
 Probability Distributions of Continuous data…

Graph of a continuous distribution showing area between a

and b.

1/4/2021 72
 Normal distribution
• Known as the Gaussian distribution

• The normal density is given by

1 − 𝑥−𝜇 2/2𝜎2
f(x)= 𝑒 , −∞ < 𝑥 < ∞;
2𝜋𝜎

where (e = 2.71828) and (π = 3.14159).

• The parameters of the distribution are the µ and the σ2

X ~N (μ,σ2).

1/4/2021 73
 Normal Distribution,…

• The density function of X, f(x), is a bell-shaped curve

– The highest point of the curve of f(x) is at the mean μ.

Hence, the mode = mean = μ.

– The curve of f(x) is symmetric about the mean μ.

– In other words, mean = mode = median

– The area under the curve is 1

1/4/2021 74
 Standard Normal Distribution
• the standard normal distribution with mean µ = 0 and variance
σ2 =1

• Denoted by Normal (0,1) or N(0,1).

• The standard normal random variable is denoted by Z, and we

write Z~N(0,1)

𝑥−𝜇
Z= 𝜎

• The equation for the standard normal distribution is written

1 −𝑧2/2
Z= 2𝜋
𝑒 , −∞ < 𝑧 < ∞
1/4/2021 75
 Standard Normal Distribution,…
The standard normal distribution

The z-transformation is useful in application of normal distribution

1/4/2021 76
 Standard Normal Distribution,…

• Z-transformation that yields a value of Z, Z=1 indicates that

the value of x used in the transformation is 1 standard
deviation above 0.

• A value of Z = -1 indicates that the value of x used in the

transformation is 1 standard deviation below 0.

1/4/2021 77
 Standard Normal Distribution,…
• Example;
– What is the probability that a z picked at random from the
population of z’s will have a value between -2.55 and 2.55?
answer:
P(-2.55<z<2.55)=0.9946-
0.0054

=0.9892

1/4/2021 78
 Standard Normal Distribution,…

1/4/2021 79
 Application of normal distribution
• Normal distribution is not a law that is adhered to by all
measurable characteristics occurring in nature
• However, many of these characteristics are approximately
normally distributed
• Used to model the distribution of many variables that are of
interest
• Allows us to make useful probability statements about some
variables conveniently than would be the case if some more
complicated model had to be used

1/4/2021 80
 Application,…
• Example:
– Let us consider weight of women in reproductive age
follows a normal distribution with mean 49 kg and variance
25 kg2
a. Find the probability that a randomly chosen woman in
her reproductive age has weight less than 45 kg.
b. What is the percentage of women having weight less
than 45 kg?
c. In a population of 20,000 women of reproductive age,
how many would you expect to have weight less than
45 kg?

1/4/2021 81
• Solution
– Here the random variable, X = weight of women in
reproductive age, population mean = 49 kg, population
variance= σ2 = 25 kg2, population standard deviation = σ =
5 kg. Hence, X~Normal (49,25).

a. The probability that a randomly chosen woman in

reproductive age has weight less than 45 kg is P(X<45)

1/4/2021 82
 Application,…

– The percentage of women of reproductive age who have

weight less than 45 kg is P(x<45) x100% = 0.2119 x100%
= 21.19%

• In a population of 20,000 women of reproductive age, we

would expect that the number of women with weight less than
45 kg is P(X <45)x 20,000 =0.2119 x20,000 = 4238.

1/4/2021 83
1/4/2021 84