EE313 Probability for EE

Conditional Probability
Murat Saraçlar
Prior Probability and Conditional Probability
• 𝑃[𝐴] reflects our knowledge of the occurrence of 𝐴 prior to
performing an experiment. We refer to 𝑃[𝐴] as the a priori
probability, or the prior probability, of A.
• Conditional probability 𝑃[𝐴|𝐵] (the probability of 𝐴 given 𝐵)
describes our knowledge of 𝐴 when we know that 𝐵 has occurred.
• Recall Example 1.14 and compare 𝑃 Long and 𝑃 Long|Fax
Voice Data Fax
Long 0.30 0.12 0.15
Brief 0.20 0.08 0.15

EE313 Probability for EE -- Murat Saraçlar 2

Definition 1.6
• The conditional probability of the event 𝐴 given the occurrence of the
event 𝐵 is
![#$]
𝑃[𝐴|𝐵] = ![$]
.

• Conditional probability is defined only when 𝑃 𝐵 > 0.

• 𝑃[𝐵] = 0 means that it is certain that 𝐵 never occurs.

EE313 Probability for EE -- Murat Saraçlar 3

Theorem 1.9
A conditional probability measure 𝑃[𝐴|𝐵] has the following properties
that correspond to the axioms of probability.

Property 1 𝑃[𝐴|𝐵] ≥ 0.

Property 2 𝑃[𝐵|𝐵] = 1.

Property 3 If A = 𝐴& ∪ 𝐴' ∪ ⋯ with 𝐴( ∩ 𝐴) = ∅ for 𝑖 ≠ 𝑗, then

𝑃 𝐴|𝐵 = 𝑃 𝐴&|𝐵 + 𝑃 𝐴'|𝐵 + ⋯.

EE313 Probability for EE -- Murat Saraçlar 4

Example 1.15
• Consider an experiment that consists of testing two integrated circuits that
come from the same silicon wafer, and observing in each case whether a
circuit is accepted (𝑎) or rejected (𝑟).
• The sample space of the experiment is 𝑆 = {𝑟𝑟, 𝑟𝑎, 𝑎𝑟, 𝑎𝑎}.
• Let 𝐵 = {𝑟𝑟, 𝑟𝑎} denote the event that the first chip tested is rejected.
• Similarly, let 𝐴 = {𝑟𝑟, 𝑎𝑟} denote the event that the second circuit is a failure.
• The circuits come from a high-quality production line. Therefore the prior
probabilities 𝑃[𝐴] and 𝑃[𝐵] are very low.
• However, some wafers become contaminated by dust, and these wafers
have a high proportion of defective chips.
• With the event B that the first chip is a reject, the probability 𝑃[𝐴|𝐵] that
the second chip will also be rejected is higher than the a priori probability
𝑃[𝐴] because of the likelihood that dust contaminated the entire wafer.

EE313 Probability for EE -- Murat Saraçlar 5

Example 1.16
With respect to Example 1.15, consider the a priori probability model
𝑃[𝑟𝑟] = 0.01, 𝑃[𝑟𝑎] = 0.01, 𝑃[𝑎𝑟] = 0.01, 𝑃[𝑎𝑎] = 0.97.
• Find the probability of
• A = “second chip rejected”
• B = “first chip rejected.”
• Also find the conditional probability that the second chip is a reject
given that the first chip is a reject.

EE313 Probability for EE -- Murat Saraçlar 6

Example 1.16
With respect to Example 1.15, consider the a priori probability model
𝑃[𝑟𝑟] = 0.01, 𝑃[𝑟𝑎] = 0.01, 𝑃[𝑎𝑟] = 0.01, 𝑃[𝑎𝑎] = 0.97.
• Find the probability of
• A = “second chip rejected” 𝑃 𝐴 = 𝑃 𝑟𝑟 + 𝑃 𝑎𝑟 = 0.02
• B = “first chip rejected.” 𝑃 𝐵 = 𝑃 𝑟𝑟 + 𝑃 𝑟𝑎 = 0.02
• Also find the conditional probability that the second chip is a reject
given that the first chip is a reject.
![#$] *.*&
𝑃 𝐴𝐵 = 𝑃 𝑟𝑟 = 0.01 and thus 𝑃 𝐴 𝐵 = = = 0.5
![$] *.*'

EE313 Probability for EE -- Murat Saraçlar 7

Example 1.18
Roll two fair four-sided dice. Let 𝑋& and 𝑋' denote the number of dots
that appear on die 1 and die 2, respectively.
• Let 𝐴 be the event 𝑋& ≥ 2. What is 𝑃[𝐴]?

• Let B denote the event 𝑋' > 𝑋&. What is 𝑃[𝐵]?

• What is 𝑃[𝐴|𝐵]?

EE313 Probability for EE -- Murat Saraçlar 8

Example 1.18
Roll two fair four-sided dice. Let 𝑋& and 𝑋' denote the number of dots
that appear on die 1 and die 2, respectively.
X2
• Let 𝐴 be the event 𝑋& ≥ 2. What is 𝑃[𝐴]? We b
" A elem
4 B • • • #• X 1 an
• Let B denote the event 𝑋' > 𝑋&. What is 𝑃[𝐵]? 3
# fair, t
• • #• •
# bility
2 • • • •
• What is 𝑃[𝐴|𝐵]? # circle
1 #• • • • repre
! X 1 pair o
1 2 3 4 It con
To find P[A], we add up the proba
EE313 Probability for EE -- Murat Saraçlar 9
The triangle represents B. It conta
The event A B has three outcomes
definition of conditional probability,
Example 1.18
Roll two fair four-sided dice. Let 𝑋& and 𝑋' denote the number of dots
that appear on die 1 and die 2, respectively.
X2
• Let 𝐴 be the event 𝑋& ≥ 2. What is 𝑃[𝐴]? We b
𝑃 𝐴 = 12⁄16 = 0.75 " A elem
4 B • • • #• X 1 an
• Let B denote the event 𝑋' > 𝑋&. What is 𝑃[𝐵]? 3 # fair, t
• • #• •
bility
2 • #
• • •
• What is 𝑃[𝐴|𝐵]? # circle
1 #• • • • repre
! X 1 pair o
1 2 3 4 It con
To find P[A], we add up the proba
EE313 Probability for EE -- Murat Saraçlar 10
The triangle represents B. It conta
The event A B has three outcomes
definition of conditional probability,
Example 1.18
Roll two fair four-sided dice. Let 𝑋& and 𝑋' denote the number of dots
that appear on die 1 and die 2, respectively.
X2
• Let 𝐴 be the event 𝑋& ≥ 2. What is 𝑃[𝐴]? We b
𝑃 𝐴 = 12⁄16 = 0.75 " A elem
4 B • • • #• X 1 an
• Let B denote the event 𝑋' > 𝑋&. What is 𝑃[𝐵]? 3 # fair, t
𝑃 𝐵 = 6⁄16 = 0.375 • • #• •
bility
2 • #• • •
• What is 𝑃[𝐴|𝐵]? # circle
1 #• • • • repre
! X 1 pair o
1 2 3 4 It con
To find P[A], we add up the proba
EE313 Probability for EE -- Murat Saraçlar 11
The triangle represents B. It conta
The event A B has three outcomes
definition of conditional probability,
Example 1.18
Roll two fair four-sided dice. Let 𝑋& and 𝑋' denote the number of dots
that appear on die 1 and die 2, respectively.
X2
• Let 𝐴 be the event 𝑋& ≥ 2. What is 𝑃[𝐴]? We b
𝑃 𝐴 = 12⁄16 = 0.75 " A elem
4 B • • • #• X 1 an
• Let B denote the event 𝑋' > 𝑋&. What is 𝑃[𝐵]? 3 # fair, t
𝑃 𝐵 = 6⁄16 = 0.375 • • #• •
bility
![#$] ,/&. 2 • #
• • • circle
• What is 𝑃[𝐴|𝐵]? 𝑃 𝐴 𝐵 = = = 0.5 #
![$] ./&. 1 #• • • • repre
or simply focusing on the conditioning event 𝐵,
! X 1 pair o
𝑃 𝐴 𝐵 = 3⁄6 = 0.5 1 2 3 4 It con
To find P[A], we add up the proba
EE313 Probability for EE -- Murat Saraçlar 12
The triangle represents B. It conta
The event A B has three outcomes
definition of conditional probability,
Law of Total Probability
Theorem 1.10: For an event space {𝐵&, 𝐵', … , 𝐵/ } with 𝑃 𝐵( > 0 for
all 𝑖,
/
𝑃 𝐴 = K 𝑃 𝐴 𝐵( 𝑃[𝐵( ]
(0&

EE313 Probability for EE -- Murat Saraçlar 13

Law of Total Probability
Theorem 1.10: For an event space {𝐵&, 𝐵', … , 𝐵/ } with 𝑃 𝐵( > 0 for
all 𝑖,
/
𝑃 𝐴 = K 𝑃 𝐴 𝐵( 𝑃[𝐵( ]
(0&
Proof: This follows from Theorem 1.8 (𝑃 𝐴 = ∑/ (0& 𝑃[𝐴 ∩ 𝐵( ])
and the identity 𝑃[𝐴 ∩ 𝐵( ] = 𝑃[𝐴|𝐵( ]𝑃[𝐵( ],
which is a direct consequence of the definition of conditional
probability.

EE313 Probability for EE -- Murat Saraçlar 14

Bayes’ Theorem
• In many situations, we have advance information about 𝑃[𝐴|𝐵] and
need to calculate 𝑃[𝐵|𝐴].
• Theorem 1.11:
𝑃 𝐴 𝐵 𝑃[𝐵]
𝑃 𝐵|𝐴 =
𝑃[𝐴]

EE313 Probability for EE -- Murat Saraçlar 15

Bayes’ Theorem
• In many situations, we have advance information about 𝑃[𝐴|𝐵] and
need to calculate 𝑃[𝐵|𝐴].
• Theorem 1.11:
𝑃 𝐴 𝐵 𝑃[𝐵]
𝑃 𝐵|𝐴 =
𝑃[𝐴]
• Proof:
𝑃[𝐴𝐵] 𝑃 𝐴 𝐵 𝑃[𝐵]
𝑃 𝐵|𝐴 = =
𝑃[𝐴] 𝑃[𝐴]

EE313 Probability for EE -- Murat Saraçlar 16

Bayes’ Theorem for Classification
What is the probability of class 𝐶( , 𝑖 = 1, … , 𝑚
given the observation 𝑋?
The a posteriori probability or posterior probability
𝑃 𝑋 𝐶( 𝑃[𝐶( ] 𝑃 𝑋 𝐶( 𝑃[𝐶( ]
𝑃 𝐶( |𝑋 = = /
𝑃[𝑋] ∑(0& 𝑃 𝑋 𝐶( 𝑃[𝐶( ]

EE313 Probability for EE -- Murat Saraçlar 17

Bayes’ Theorem for Classification
What is the most probable class 𝐶( , 𝑖 = 1, … , 𝑚
given the observation 𝑋?
The a posteriori probability or posterior probability

𝑃 𝑋 𝐶( 𝑃[𝐶( ] 𝑃 𝑋 𝐶( 𝑃[𝐶( ]
𝑃 𝐶( |𝑋 = = /
𝑃[𝑋] ∑(0& 𝑃 𝑋 𝐶( 𝑃[𝐶( ]
Noting that 𝑃 𝑋 is a constant (does not depend on 𝑖),
the most likely class can be found by simply using

arg max 𝑃 𝐶( |𝑋 = arg max 𝑃 𝑋|𝐶( 𝑃 𝐶( .

( (
EE313 Probability for EE -- Murat Saraçlar 18
Bayes’ Theorem for Classification
When the classes have equal prior probabilities,
what is the most probable class 𝐶( , 𝑖 = 1, … , 𝑚
given the observation 𝑋?
The most likely class can be found by simply using
arg max 𝑃 𝐶( |𝑋 = arg max 𝑃 𝑋|𝐶( 𝑃 𝐶( .
( (
If the classes 𝐶( are equally likely,
(that is, the a priori class probabilities 𝑃 𝐶( are equal)
arg max 𝑃 𝐶( |𝑋 = arg max 𝑃 𝑋|𝐶( .
( (

EE313 Probability for EE -- Murat Saraçlar 19