Introduction to Basic Probability

Prof(Dr.) Harsh Vardhan

3-4-10/08/2023
Man can not discover new oceans unless he has
courage to lose the sight of the shore.

BS/IIFT/Harsh/4 2
 Session Objective

 To provide conceptual information regarding Probability and its

application in Managerial Decision Making
 To understand Conditional Probability
 To use Bayes’ theorem

BS/IIFT/Harsh/4 3
Content

 Fundamental Concept of Probability

 Sample Spaces and Events, Different types of Events,
Simple Probability, and Joint Probability
 Events-Mutually Exclusive ,Exhaustive, Equally Likely & Independent
 Classical definition of Probability, limitations of the classical definition
 Rules of Probability
 Conditional Probability, Marginal Probability and Total Probability
 Bayes’ Theorem

BS/IIFT/Harsh/4 4
 Why Learn Probability?
 Nothing in life is certain. In everything we do, we gauge the chances
of successful outcomes, from business to medicine to the weather
 Probability provides a quantitative description of the chances or
likelihoods associated with various outcomes
 It provides a bridge between Descriptive and Inferential Statistics

Probability
Population Sample
Statistics
BS/IIFT/Harsh/4 5
 Probabilistic vs. Statistical Reasoning

 Suppose you know exactly the proportions of car makes in

Gurugram. Then you can find the probability that the first
car you see in the street is a Maruti Brezza. This is
Probabilistic reasoning as you know the population and
predict the sample.
 Now suppose that you do not know the proportions of car
makes in Gurugram, but would like to estimate them. You
observe a random sample of cars in the street and then
you have an estimate of the proportions of the
population. This is Statistical reasoning.

BS/IIFT/Harsh/4 6
 What is Probability?
Probability is chance of something to happen.

 We used graphs and numerical measures to

describe data sets which were usually samples.
 We measured “how often” using
Relative frequency = f/n
• As n gets larger,

Sample Population
And “How often”
= Relative frequency Probability

BS/IIFT/Harsh/4 7
Basic Concepts

 An Experiment is the process by which a well defined

observation (or measurement) is obtained.
 An Event is an outcome of an experiment, usually
denoted by a capital letter.
 The basic element to which probability is applied
 When an experiment is performed, a particular event
either happens, or it doesn’t!

BS/IIFT/Harsh/4 8
Experiments & Events

 Experiment: Record an age

 A: person is 20 years old
 B: person is older than 45
 Experiment: Toss a dice
 A: observe an odd number
 B: observe a number greater than 2

BS/IIFT/Harsh/4 9
 Sample Spaces

Collection of all Possible Outcomes (events)

e.g. All 6 faces of a die:

e.g. All 52 cards of a bridge deck:

A sample point is an element of the sample space,

any one particular experimental outcome.
BS/IIFT/Harsh/4 10
 Events

 Simple Event: Outcome from a Sample

Space with one Characteristic
e.g. A Red Card from a deck of cards.
 Joint or Compound Event: Involves two
outcomes simultaneously
e.g. An Ace which is also a Red Card from a
deck of cards.
An Ace given that it is a Red Card.

BS/IIFT/Harsh/4 11
 Basic Concepts

 An event that cannot be decomposed is called a Simple event.

 Denoted by E with a subscript.
 Each simple event will be assigned a probability, measuring “how
often” it occurs.
 The set of all simple events of an experiment is called the Sample
Space, S.

BS/IIFT/Harsh/4 12
 Example
The dice toss:
 Simple events: Sample space:

1 E1
2
S ={E1, E2, E3, E4, E5, E6}
E2
S
3 E3 •E1 •E3
4 E4 •E5
5
E5 •E2 •E4 •E6
6 E6
BS/IIFT/Harsh/4 13
 Basic Concepts

 An event is a collection of one or more simple events.

S
•E1 •E3
The die toss:
A •E5
–A: an odd number B
–B: a number > 2 •E2 •E6
•E4
A ={E1, E3, E5}
B ={E3, E4, E5, E6}
BS/IIFT/Harsh/4 14
 Simple Events

The Event of a Happy Face

There are 5 happy faces in this collection of 18 objects

BS/IIFT/Harsh/4 15
 Joint Events

The Event of a Happy Face AND Light Colored

3 Happy Faces which are light in color

BS/IIFT/Harsh/4 16
 Compound Events

The Event of Happy Face OR Light Colored

12 Items, 5 happy faces and 7 other light objects

BS/IIFT/Harsh/4 17
 Special Events

 Null Event Null Event

 Club & Diamond on
1 Card Draw 
 Complement of Event
 For Event A,
 All Events Not In A:A’

BS/IIFT/Harsh/4 18
 Dependent or Independent Events
The Event of a Happy Face GIVEN it is Light Colored

E = Happy Face  Light Color

3 Items, 3 Happy Faces Given they are Light Colored

BS/IIFT/Harsh/4 19
 Visualizing Events

•Contingency Tables Ace Not Ace Total

Black 2 24 26
Red 2 24 26
Total 4 48 52

•Tree Diagrams

BS/IIFT/Harsh/4 20
 Contingency Table 3/08/23

A Deck of 52 Cards
Red Ace
Not an Total
Ace
Ace
Red 2 24 26
Black 2 24 26
Total 4 48 52
Sample Space
BS/IIFT/Harsh/4 21
 Tree Diagram

Event Possibilities
Ace
Red
Cards Not an Ace
Full
Deck
of Cards Ace
Black
Cards
Not an Ace
BS/IIFT/Harsh/4 22
 Basic Concepts
Mutually Exclusive Events

 Two events are mutually exclusive if, when one

event occurs, the other cannot, and vice versa.

•Experiment: Toss a die Not Mutually

–A: observe an odd number Exclusive

–B: observe a number greater than 2

–C: observe a 6 B and C?
Mutually
–D: observe a 3 Exclusive B and D?

BS/IIFT/Harsh/4 23
 Exhaustive Events

 The total number of possible outcomes in

any experiment are known as exhaustive
events or exhaustive cases.
 Examples:
 Toss a coin there are two possibilities - Head or Tail,
these two are exhaustive cases.
 Throw a dice there are 6 exhaustive cases having
faces 1,2,3,4,5, and 6.
 Throw two dice there will be 62 =36 exhaustive
number of cases.

BS/IIFT/Harsh/4 24
 Favorable Events
 The number of cases favorable to an event in a trial is
the number of outcomes which entail the happening of
the event.
Example
 In drawing a card from a pack of cards the number of
cases favorable to drawing of an ace is 4, for drawing a
spade is 13 and for drawing a red card is 26.
 In throwing of two dice ,the number of favorable cases to
get a sum of 5 is :(1,4),(4,1), (2,3), (3,2) i.e 4.

BS/IIFT/Harsh/4 25
 Equally Likely Events

 Outcomes of a trial are said to be equally likely if

there is no reason to expect one in preference to
other.
Examples
 In tossing an unbiased coin the event of getting
head or tail are equally likely to occur.
 In throwing an unbiased dice ,all the six faces are
equally likely occur.

BS/IIFT/Harsh/4 26
 Independent Events
 Events are said to be independent if happening or non
happening of an event is not affected by the
supplementary knowledge concerning the occurrence of
any number of the remaining events.
Examples
 In tossing an unbiased coin, the event of getting a head
in the first toss is independent of getting a head in the
second ,third and subsequent throws.
 Drawing a card from a pack of a well-shuffled cards and
replacing it before drawing other ,the second draw is
independent of the first. But if the card drawn in the first
draw is not replaced then second draw is dependent on
the first draw.
BS/IIFT/Harsh/4 27
 The Probability of an Event

 The Probability of an event A measures “how often” A

will occur. We write P(A).
 Suppose that an experiment is performed n times.
The relative frequency for an event A is

Number of times A occurs f


n n
• If we let n get infinitely large,
f
P( A)  lim
n n
BS/IIFT/Harsh/4 28
 The Probability of an Event

 P(A) must be between 0 and 1.

 If event A can never occur, P(A) = 0. If event A
always occurs when the experiment is performed,
P(A) =1.
 The sum of the probabilities for all simple events in S
equals 1.
• The probability of an event A is found
by adding the probabilities of all the
simple events contained in A.
BS/IIFT/Harsh/4 29
 Finding Probabilities

 Probabilities can be found using

 Estimates from empirical studies
 Common sense estimates based on equally likely
events.

• Examples:
–Toss a fair coin. P(Head) = 1/2
– Suppose that 10% of the Indian population has
brown hair. Then for a person selected at random,
P(Brown hair) =0.10
BS/IIFT/Harsh/4 30
 Example 2

A bowl contains three marbles, one red, one blue and

one green. A child selects two marbles at random.
What is the probability that at least one is red?

1st 2nd Ei P(Ei)

m RB 1/6
m
m RG P(at least 1 red)
1/6
m = P(RB) + P(RG)+ P(BR)
BR 1/6
m +P(GR)
m
BG 1/6 = 4/6 = 2/3
m
m GB
1/6
m GR
1/6
BS/IIFT/Harsh/4 31
 Using Simple Events

 The Probability of an event A is equal to the sum of

the probabilities of the simple events contained in A
 If the simple events in an experiment are equally
likely, you can calculate

n A number of simple events in A

P( A)  
N total number of simple events

BS/IIFT/Harsh/4 32
 Example 1

Toss a fair coin twice. What is the probability of

observing at least one head?

1st Coin 2nd Coin Ei P(Ei)

H HH
1/4 P(at least 1 head)
H
T HT = P(E1) + P(E2) + P(E3)
1/4
= 1/4 + 1/4 + 1/4 = 3/4
H TH 1/4
T
T TT 1/4

BS/IIFT/Harsh/4 33
 Example 2
The sample space of throwing a pair of dice is

BS/IIFT/Harsh/4 34
 Example 3

Event Simple events Probability

Dice add to 3 (1,2),(2,1) 2/36
Dice add to 6 (1,5),(2,4),(3,3), 5/36
(4,2),(5,1)
Red dice show 1 (1,1),(1,2),(1,3), 6/36
(1,4),(1,5),(1,6)
Green dice show 1 (1,1),(2,1),(3,1), 6/36
(4,1),(5,1),(6,1)

BS/IIFT/Harsh/4 35
 Counting Rules

 Sample space of throwing 3 dice has 216 entries, sample

space of throwing 4 dice has 1296 entries, …
 At some point, we have to stop listing and start thinking …
 We need some counting rules

BS/IIFT/Harsh/4 36
 The mn Rule
 If an experiment is performed in two stages, with m
ways to accomplish the first stage and n ways to
accomplish the second stage, then there are mn ways
to accomplish the experiment.
 This rule is easily extended to k stages, with the
number of ways equal to
n1 n2 n3 … nk

Example: Toss two coins. The total number of

simple events is:
22=4
BS/IIFT/Harsh/4 37
 Examples m

m
Example: Toss three coins. The total number of
simple events is: 222=8
Example: Toss two dice.
The total number of 6  6 = 36
simple events is:
Example: Toss three dice. The total number of
simple events is: 6  6  6 = 216

Example: Two marbles are drawn from a dish

containing two red and two blue. The total number
of simple events is ( without replacement):
BS/IIFT/Harsh/4 4  3 = 12 38
 Three Approaches of Probability
Classical approach
 Relative frequency approach
 Subjective approach

Classical Probability
Assigning probabilities based on the assumption of
equally likely outcomes.
Probability for an event to occur
= Number of outcomes where the event occurs
Total number of possible out comes.
P(H)=1/2, probability of getting a head.
P(3)=1/6, probability of getting 3 on rolling a dice.
Sample Space: S = {1, 2, 3, 4, 5, 6}
If an experiment has n possible outcomes, this method
would assign a probability of 1/n to each outcome.
BS/IIFT/Harsh/4 39
 Contd.-Shortcoming of Classical approach
 A priori probability, it is another name of classical
probability as we can answer about out come in
advance without tossing a fair coin, rolling an
unbiased dice or drawing a card from standard
deck.
 For a prior probability ,probability estimates are
made prior to receiving new information.
Do you encounter such situations in real
management world? …No!
Classical approach of probability assumes a world
that does not exist!
BS/IIFT/Harsh/4 40
 Relative frequency of occurrence
 Relative Frequency of occurrence is proportion of times
that an event occurs in long run when the conditions are
stable or it is the observed relative frequency of an event in
a very large number of trials.
It is assigning probabilities based on experimentation or
historical data.
 By increasing number of observations we increase
accuracy.

Number of times A occurs f

P(A)= 
n n
f
P( A)  lim
n n
BS/IIFT/Harsh/4 41
 Example: Lucas Tool Rental
 Relative Frequency Method
Lucas would like to assign probabilities to the
number of floor polishers it rents per day. Office
records show the following frequencies of daily rentals
for the last 40 days.
Number of Number
Polishers Rented of Days
0 4
1 6
2 18
3 10
4 2

BS/IIFT/Harsh/4 42
 Example: Lucas Tool Rental
 Relative Frequency Method
The probability assignments are given by dividing
the number-of-days frequencies by the total frequency.
Number of Number
Polishers Rented of Days Probability
0 4 4/40
1 6 6/40
2 18 18/40
3 10 10/40 .25
4 2 2/40 .05
total 40 1.00

BS/IIFT/Harsh/4 43
 Subjective Probability
Subjective Method
 Assigning probabilities based on the assigner’s judgment.
 Probabilities are based solely on historical data.
 We can use any data available as well as our experience and
intuition, but ultimately a probability value should express our
degree of belief that the experimental outcome will occur.
 The best probability estimates are often obtained by combining
the estimates from the classical or relative frequency approach
with the subjective estimates.

BS/IIFT/Harsh/4 44
 Example
 Subjective probability plays role when event occurs once or at
most few times.
 Suppose there are three candidates for social service worker and
each three have attractive appearance, high level of energy and
self confidence, a record of past accomplishments and attitude to
face challenges. What are the chances each will relate to client
successfully? Choosing among three will require to assign a
subjective probability to each persons’ potential.
 Choosing a site for construction of nuclear power plant. You don't
have any past record for geological fault at the site. What is
probability of major nuclear disaster. It becomes subjective
probability issue.
BS/IIFT/Harsh/4 45
 Permutations

 The number of ways you can arrange

n distinct objects, taking them r at a time is

n!
P n

(n  r )!
r

where n! n(n  1)(n  2)...(2)(1) and 0! 1.

Example: How many 3-digit lock combinations
can we make from the numbers 1, 2, 3, and 4?
4!
The order of the choice is
important!
P   4(3)(2)  24
3
4

1!
BS/IIFT/Harsh/4 46
 Ans 12,
Example: How many 2-
(1,2),(1,3),(1,4)
digit lock combinations
can we make from the (2,1),(2,3),(2,4)
numbers 1, 2, 3, and 4? (3,1),(3,2),(3,4)
(4,1),(4,2),(4,3)

Example: How many Ans. 24

3-digit lock (1,2,3),(1,2,4),(1,3,2),(1,3,4),(1,4,2),(1,4,3)

combinations can we (2,1,3),(2,1,4),(2,3,1),(2,3,4),(2,4,1),(2,4,3),
make from the numbers (3,1,2),(3,1,4),(3,2,1),(3,2,4),(3,4,1),(3,4,2),
1, 2, 3, and 4? (4,1,2),(4,1,3),(4,2,2),(4,2,3),(4,3,1),(4,3,2).

BS/IIFT/Harsh/4 47
 Examples

Example: A lock consists of five parts and

can be assembled in any order. A quality
control engineer wants to test each order for
efficiency of assembly. How many orders are
there?
The order of the choice is
important!
5!
P   5(4)(3)(2)(1)  120
5
5

0!
BS/IIFT/Harsh/4 48
 Combinations
 The number of distinct combinations of n distinct
objects that can be formed, taking them r at a time is

n!
C 
n

r!(n  r )!
r

Example: Three members of a 5-person committee must

be chosen to form a subcommittee. How many different
subcommittees could be formed?
5! 5(4)(3)(2)1 5(4)
The order of C 
5
   10
3!(5  3)! 3(2)(1)(2)1 (2)1
3
the choice is
not important!
BS/IIFT/Harsh/4 49
 Event Relations
The beauty of using events, rather than simple
events, is that we can combine events to make other
events using logical operations: and, or and not.
The union of two events, A and B, is the event
that either A or B or both occur when the experiment
is performed. We write
AB

A B A B

BS/IIFT/Harsh/4 50
 Event Relations
The intersection of two events, A and B, is the
event that both A and B occur when the
experiment is performed. We write A  B.

A B A B

• If two events A and B are mutually

exclusive, then P(A  B) = 0.
BS/IIFT/Harsh/4 51
 Event Relations

The complement of an event A consists of all

outcomes of the experiment that do not result in
event A. We write AC.
S
AC

BS/IIFT/Harsh/4 52
 Example

Select a student from the classroom and

record his/her hair color and gender.
 A: student has brown hair
 B: student is female
 C: student is male
Mutually exclusive; B = CC
What is the relationship between events B and C?
•AC: Student does not have brown hair
•BC: Student is both male and female = 
•BC: Student is either male and female = all students = S
BS/IIFT/Harsh/4 53
Calculating Probabilities for Unions and Complements

 There are special rules that will allow you to calculate

probabilities for composite events.
 The Additive Rule for Unions:
 For any two events, A and B, the probability of their
union, P(A  B), is

P( A  B)  P( A)  P( B)  P( A  B)
A B

BS/IIFT/Harsh/4 54
 Example: Additive Rule

Example: Suppose that there were 120 students

in the classroom, and that they could be classified
as follows:
Brown Not Brown
A: brown hair Male 20 40
P(A) = 50/120
B: female Female 30 30
P(B) = 60/120,Find P(AUB)

P(AB) = P(A) + P(B) – P(AB)

= 50/120 + 60/120 - 30/120
= 80/120 = 2/3 Check: P(AB)
BS/IIFT/Harsh/4
= (20 + 30 + 30)/120 55
 Example: Two Dice

A: red dice show 1

B: green dice show 1
Find P(AUB).

P(AB) = P(A) + P(B) – P(AB)

= 6/36 + 6/36 – 1/36
= 11/36
BS/IIFT/Harsh/4 56
 A Special Case
When two events A and B are mutually
exclusive, P(AB) = 0
and P(AB) = P(A) + P(B).

A: male with brown hair Brown Not Brown

P(A) = 20/120 Male 20 40
B: female with brown hair Female 30 30
P(B) = 30/120
Find P(AUB)
A and B are mutually P(AB) = P(A) + P(B)
= 20/120 + 30/120
exclusive, so that
= 50/120
BS/IIFT/Harsh/4 57
 Example: Two Dice

A: dice add to 3
(1,2 2,1)=2
B: dice add to 6
(1,5 5,1 2,4 4,2 3,3) =5
Find P(AUB).

A and B are mutually P(AB) = P(A) + P(B)

= 2/36 + 5/36
exclusive, so that
= 7/36
BS/IIFT/Harsh/4 58
 Calculating Probabilities AC
for Complements A

 We know that for any event A:

 P(A  AC) = 0
 Since either A or AC must occur,
P(A  AC) =1
 so that P(A  AC) = P(A)+ PC) (A = 1

P(AC) = 1 – P(A)

BS/IIFT/Harsh/4 59
 Example

Select a student at random from the

classroom. Define:
A: male Brown Not Brown
Male 20 40
P(A) = 60/120
B: female Female 30 30
P(B) = ?

A and B are P(B) = 1- P(A)

complementary, so that = 1- 60/120 = 60/120

BS/IIFT/Harsh/4 60
 Calculating Probabilities for Intersections

. The rule for calculating P(A  B) depends on the idea of

independent and dependent events.

Two events, A and B, are said to be

independent if the occurrence or
nonoccurrence of one of the events does
not change the probability of the
occurrence of the other event.
BS/IIFT/Harsh/4 61
Joint and Marginal Probability

 Joint Probability
The probability of two events occurring together or in
succession.
 Marginal Probability
The unconditional probability of one event occurring or the
probability of a single event.

BS/IIFT/Harsh/4 62
 Joint Probability Using Contingency Table

Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)

A2 P(A2 and B1) P(A2 and B2) P(A2)

Total P(B1) P(B2) 1

Joint Probability Marginal (Simple) Probability

BS/IIFT/Harsh/4 63
 Compound Probability :Addition Rule
P(A1 or B1 ) = P(A1) +P(B1) - P(A1 and B1)
Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)

A2 P(A2 and B1)) P(A2 and B2) P(A2)

Total P(B11) P(B2) 1

For Mutually Exclusive Events: P(A or B) = P(A) + P(B)

BS/IIFT/Harsh/4 64
 Conditional Probabilities
The probability that A occurs, given that event
B has occurred is called the conditional
probability of A given B and is defined as

P( A  B)
P( A | B)  if P( B)  0
P( B)

“given”

BS/IIFT/Harsh/4 65
Conditional Probability Using Contingency Table

Conditional Event: Draw 1 Card. Note Kind & Color

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
P(Ace and Black) 2 / 52 2
P(Ace | Black) =  
P(Black) 26 / 52 26
BS/IIFT/Harsh/4 66
 Example 1
Toss a fair coin twice. Define
 A: head on second toss
 B: head on first toss

P(A|B) = ½
HH
1/4 P(A|not B) = ½
HT 1/4
P(A) does not A and B are
TH 1/4
change, whether independent!
TT 1/4 B happens or
not…
BS/IIFT/Harsh/4 67
 Example 2

A bowl contains five marbles, two red and three blue.

Randomly select two one by one , and define
 A: second marble is red.
 B: first marble is blue.

m P(A|B) =P(2nd red|1st blue)= 2/4 = 1/2

m m
P(A|not B) = P(2nd red|1st red) = 1/4
m m

P(A) does change,

depending on A and B are
whether B happens dependent!
or not… BS/IIFT/Harsh/4 68
 Example 3: Two Dice 4/08
Toss a pair of fair dice.
Define
 A: red die show 1
 B: green die show 1

P(A|B) = P(A and B)/P(B)

=1/36/1/6=1/6=P(A)

P(A) does not

change, whether A and B are
B happens or independent!
not…
BS/IIFT/Harsh/4 69
 Example 4: Two Dice
Toss a pair of fair dice.
Define
 A: add to 3
 B: add to 6
B: {(1,5),(2,4),(3,3),(4,2),5,1)}

P(A|B) = P(A and B)/P(B)

=0/36/5/36=0

P(A) does change A and B are dependent!

when B happens In fact, when B happens,
A can’t
BS/IIFT/Harsh/4 70
 Defining Independence
 We can redefine independence in terms of
conditional probabilities:
Two events A and B are independent if and
only if
P(A|B) = P(A) or P(B|A) = P(B)
Otherwise, they are dependent.
• Once you’ve decided whether or not two events
are independent, you can use the following rule
to calculate their intersection.
BS/IIFT/Harsh/4 71
 The Multiplicative Rule for Intersections

 For any two events, A and B, the probability that both A

and B occur is

P(A  B) = P(A) P(B given that A occurred)

= P(A)P(B|A)
 If the events A and B are independent, then the
probability that both A and B occur is

P(A  B) = P(A) P(B)

BS/IIFT/Harsh/4 72
 Example 1
In a certain population, 10% of the people can be
classified as being high risk for a heart attack. Three
people are randomly selected from this population.
What is the probability that exactly one of the three
are high risk?
Define H: high risk N: not high risk
P(N)=1 - P(H)=1-0.1=0.9

P(exactly one high risk) = P(HNN) + P(NHN) + P(NNH)

= P(H)P(N)P(N) + P(N)P(H)P(N) + P(N)P(N)P(H)
= (.1)(.9)(.9) + (.9)(.1)(.9) + (.9)(.9)(.1)= 3(.1)(.9)2 = .243
BS/IIFT/Harsh/4 73
 Example 2
Suppose we have additional information in the
previous example. We know that only 49% of the
population are female. Also, of the female patients, 8%
are high risk. A single person is selected at random. What
is the probability that it is a high risk female?
Define H: high risk F: female
From the example, P(F) = .49 and P(H|F) = .08.
Use the Multiplicative Rule:
P(high risk female) = P(HF)
= P(F)P(H|F) =.49(.08) = .0392
BS/IIFT/Harsh/4 74
 The Law of Total Probability

Let S1 , S2 , S3 ,..., Sk be mutually exclusive and exhaustive

events (that is, one and only one must happen). Then
the probability of any event A can be written as

P(A) = P(A  S1) + P(A  S2) + … + P(A  Sk)

= P(S1)P(A|S1) + P(S2)P(A|S2) + … + P(Sk)P(A|Sk)

BS/IIFT/Harsh/4 75
 The Law of Total Probability

S1

A Sk
A

A  S1 Sk
S2….

P(A) = P(A  S1) + P(A  S2) + … + P(A  Sk)

= P(S1)P(A|S1) + P(S2)P(A|S2) + … + P(Sk)P(A|Sk)

BS/IIFT/Harsh/4 76
 Bayes’ Rule
Let S1 , S2 , S3 ,..., Sk be mutually exclusive and exhaustive
events with prior probabilities P(S1), P(S2),…,P(Sk). If an event
A occurs, the posterior probability of Si, given that A occurred
is

P( Si ) P( A | Si )
P( Si | A)  for i  1, 2,...k
 P( Si ) P( A | Si )
Proof
P( AS i )
P( A | Si )  
 P( AS i )  P( Si ) P( A | Si )
P( Si )
P( AS i ) P( Si ) P( A | Si )
P( Si | A)  
P( A)  P( Si ) P( A | Si )
BS/IIFT/Harsh/4 77
 Bayes’ Theorem
 Often we begin probability analysis with initial or prior
probabilities.
 Then, from a sample, special report, or a product test we
obtain some additional information.
 Given this information, we calculate revised or posterior
probabilities.
 Bayes’ theorem provides the means for revising the prior
probabilities.
Application
Prior New Posterior
of Bayes’
Probabilities Information Probabilities
Theorem

BS/IIFT/Harsh/4 78
 Bayes’ theorem is applicable when the events for which
we want to compute posterior probabilities are mutually
exclusive and their union is the entire sample space.
 A Priori Probability- Probability estimate made prior to
receiving new information. [P(Si) i=1,2,….n]
 Posterior Probability- A probability that has been revised
after information was obtained. [P(Si/A) i=1,2,….n]
 Likelihoods- Probabilities [P(A/Si) i=1,2,….n] are called
Likelihoods as they indicate how likely the event A under
consideration is to occur, given each and every a priori
probability.

BS/IIFT/Harsh/4 79
 Bayes’ Theorem

P( A Bi )  P( Bi )
P(Bi A) =
P( A B1 )  P( B1 )      P( A Bk )  P( Bk )

P( Bi and A)

P( A) Adding up
the parts
of A in all
Same
the B’s
Event

BS/IIFT/Harsh/4 80
 Problem
 The contents of three urns are as follows:
 Urn 1 1W,2B,3 R
 Urn 2 2W,1B, 1R
 Urn 3 4W,5B, 3R
 One urn is chosen at random and two balls are drawn.
They happen to be white and red. What is the
probability that they came from urn I,II or III?

BS/IIFT/Harsh/4 81
 Solution-Assignment
Urn 1 1W,2B,3 R
Urn 2 2W,1B, 1R
Urn 3 4W,5B, 3R
Let E1 E2 E3 are the events that Urn 1,Urn 2,Urn 3 is chosen respectively.
Let A be the event that the two balls taken from the selected urn are
White and Red.
P(E1)= P(E2)= P(E3)=1/3
P(A/ E1) =1*3/6C2 =1/5
P(A/ E2) =2*1/4C2 =1/3
P(A/ E3) =4*3/12C2 =2/11
P(E2/A)= P(E2)* P(A/ E2)
P(E1)* P(A/ E1)+ P(E2)* P(A/ E2)+ P(E3)* P(A/ E3)
=55/118=0.466 BS/IIFT/Harsh/4 82
Definitions and Learning
 The probability of any outcome of a random phenomenon is the
proportion of times the outcome would occur in a very long series of
repetitions.
 In classical approach, probability is known as long run relative
frequency. That is if we repeat the experiment long number of time,
or in other way, suppose we have a large sample size, then proportion
of times the event A occurs (that is the relative frequency) will be
very close to the true value of the probability of that event.

BS/IIFT/Harsh/4 83
Probability Rules:
 Rule 1:
 Let N(A) = Number of events favorable to event A.
 N= Total number of events in the Sample Space S.
𝑁(𝐴)
P(A)=
𝑁

BS/IIFT/Harsh/4 84
Probability Rules:

 Rule 2: The probability P(A) of any event A satisfies 0 <P(A) < 1

 Rule 3: If S is the sample space in a probability model, the P(S) = 1.
 Rule 4: For any event A,
P(A does not occur) = 1 – P(A)

BS/IIFT/Harsh/4 85
Probability Rules
 Rules 5: Addition Rule:
Two events A and B are disjoint if they have no outcomes in
common and so can never occur simultaneously. If A and B are
disjoint,
P(A or B) = P(A) + P(B)

BS/IIFT/Harsh/4 86
Probability Rules:
 Rule 6: General Addition Rule for Any Two Events.

For any two events A and B,

P(A or B) = P(A) + P(B) – P(A and B)

BS/IIFT/Harsh/4 87
Probability Rules:
 Rule 7: Conditional Probability
When P(A) > 0, the conditional probability of B given A is,

P(B|A) = P( A and B)
P(A)

BS/IIFT/Harsh/4 88
Probability Rules:
 Rule 8: Multiplication Rule for Independent Events
Two events A and B are independent if knowing that one occur
does not change the probability that the other occurs. If A and B are
independent,
P( A and B) = P(A)P(B)

BS/IIFT/Harsh/4 89
Probability Rules:

 Rule 9: Independent Events

Two events A and B that both have positive
probability are independent if
P(B|A) = P(B)

BS/IIFT/Harsh/4 90
Probability Rules:
 Rule 10: General Multiplication Rule for Any Two Events
The probability that both of two events A and B happen together can
be found by
P( A and B) = P(A) P( B|A)
Here P (B|A) is the conditional probability that B occurs given the
information that A occurs.

BS/IIFT/Harsh/4 91
 Bayes’ Theorem
 To find the posterior probability that event Bi
will occur given that event A has occurred we
apply Bayes’ theorem.

P( A Bi )  P( Bi )
P(Bi A) = P( A B1 )  P( B1 )      P( A Bk )  P( Bk )

P( A / B1) * P( B1)
P( B1 / A) 
P( A / B1) * P( B1)  P( A / B 2) * P( B 2)
BS/IIFT/Harsh/4 92
Thanks

BS/IIFT/Harsh/4 93