Topic 4: Uncertainty

Management in Rule
Based (Bayes Reasoning)

ITS662
BASIC PROBABILITY THEORY

The basic concept of probability plays a significant role in our everyday

life.

Human try to determine the probability of rain and the prospects of

our promotion, the odds that the favourite football team will win the
next test match, and the likelihood of passing the test at college.

Probability can be expressed mathematically as a numerical index

with a range between zero (an absolute impossibility) to unity (an
absolute certainty).
BASIC PROBABILITY THEORY

• Most events have a probability index strictly between 0 and 1, which means that each event has at
least two possible outcomes:
• favourable outcome or success, and
• unfavourable outcome or failure.

P(success ) =
the number of successes
the number of possible outcomes

P( failure ) =
the number of failures
the number of possible outcomes
• A probability of zero means that an event is impossible.
BASIC PROBABILITY THEORY

• If s is the number of times success can occur, and f is the number of times failure can occur, then
P(success ) = p =
s
s+ f
p+q=1
P( failure ) = q =
f
s+ f

• Example:
• If we throw a coin, the probability of getting a head will be equal to the probability of getting a
tail.
• In a single throw, s = f = 1, and therefore the probability of getting a head (or a tail) is 0.5.

If you toss a coin, you cannot get both a head and a tail at
the same time, so this has zero probability.
CONDITIONAL PROBABILITY

• In probability theory, Conditional Probability is the probability of one event

occurring with some relationship to one or more other events. For example:
• Event A is that it is raining outside, and it has a 0.3 (30%) chance of raining
today.
• Event B is that you will need to go outside, and that has a probability of 0.5
(50%).
• A conditional probability would look at these two events in relationship with
one another, such as the probability that it is both raining and you will need to
go outside.
 CONDITIONAL PROBABILITY
The conditional probability of event A occurring given that event B has occurred equals:

p( A  B )
p(A B ) =
the number of times A and B can occur p(A B ) =
the number of times B can occur p (B )

The conditional probability of event B occurring given that event A has occurred equals:

p (B  A)
p (B A) = p (B A) =
the number of times B and A can occur
the number of times A can occur p ( A)
BAYES’S THEOREM

• Bayes' theorem is also known as Bayes' rule, Bayes' law, or Bayesian reasoning, which determines the
probability of an event with uncertain knowledge. In probability theory, it relates the conditional
probability and marginal probabilities of two random events.
• It is a way to calculate the value of P(B|A) with the knowledge of P(A|B).
• Bayes' theorem allows updating the probability prediction of an event by observing new information
of the real world.
BAYES’S THEOREM

p (B A) p ( A)
p(A B ) =
p (B )

where:
p(A|B) is the conditional probability that event A occurs given that event B has occurred.
p(B|A) is the conditional probability of event B occurring given that event A has occurred.
p(A) is the probability of event A occurring.
p(B) is the probability of event B occurring.
EXAMPLE 1

In Exton School, 60% of the boys play football and 36% of the boys play ice hockey.
Given that 40% of those that play football also play ice hockey, what percent of those that play ice
hockey also play football?
Solution :
Let’s A = Play football
B = Play ice hockey ( )
p AB =
( )
p B A  p ( A)
p (B )
P(A) = 60% = 0.6
P(B) = 36% = 0.36
P(B|A) = 40% = 0.4

Therefore p(A|B) = (0.4 * 0.6) / 0.36

= 0.66
Conclusion: 66% of boys who play ice hockey is also play football.
EXAMPLE 2

75% of the children in Exton school have a dog, and 30% have a cat.
Given that 60% of those that have cat also have a dog, what percent of those that have a dog also have
a cat?
Solution :

p (B A) p ( A)
Let’s A = Have a cat

p(A B ) =
B = Have a dog
p (B )
P(A) = 30% = 0.3
P(B) = 75% = 0.75
P(B|A) = 60% = 0.6

Therefore p(A|B) = (0.6 * 0.3) / 0.75

= 0.24

Conclusion: 24% of children who have a dog also have a cat.

BAYESIAN REASONING

• The Bayesian logic states that the probability of the occurrence of an event can be found if the value
of another event is known, provided that they are dependent on each other. The dependency
establishes a mathematical relation between both the events, thereby making it possible for the
technicians and other scientists to predict the knowledge which they like to have.
• Suppose all rules in the knowledge base are represented in the following form:
• IF H is true
• THEN E is true {with probability p}
• This rule implies that if event E occurs, then the probability that event H will occur is p.

p (E H ) p (H )
p (H E ) =
p (E H ) p (H ) + p (E H ) p (H )
BAYESIAN REASONING
p (E H ) p (H )
p (H E ) =
p (E H ) p (H ) + p (E H ) p (H )
where:
p(H) is the prior probability of hypothesis H being true.
p(E|H) is the probability that hypothesis H being true will result in evidence E.
p(¬H) is the prior probability of hypothesis H being false.
p(E|¬H) is the probability of finding evidence E even when hypothesis H is false.

P(E|H) * P(H) is on both the top and the bottom of the equation. It represents “expected number of times
it’s true” in the generic formula above.
P(E|¬H) * P(¬H) represents "expected number of times it's false".
EXAMPLE
• Patients with chess pains are often given an electrocardiogram (ECG) test.
• Test results are classified as either positive (+ECG) suggesting heart disease (+HD) or
negative (-ECG) suggesting no heart disease (-HD).
• Assume now that the patient has produced a +ECG and we want to know how
probable it is that he has heart disease, that is : P(+HD|+ECG)
• The following information apply in this case :
• 10 people out of hundred have heart disease
• 90 people out of 100 who have HD will produce +ECG
• 95 people out of 100 who have –HD will produce –ECG
• Solution
• First, obtain the prob value :
• P(+HD) = 10/100 = 0.1
• P(-HD) = 1 – P(+HD) = 1 – 0.1 = 0.9
• P(+ECG|+HD) = 90/100 = 0.9
• P(-ECG|-HD) = 95/100 = 0.95
• P(+ECG|-HD) = 1 – P(-ECG|-HD) = 1 – 0.95 = 0.05
EXAMPLE
• Assume now that the patient has produced a +ECG and we want to know how probable it is
that he has heart disease, that is : P(+HD|+ECG)
• Solution
• First, obtain the prob value : p (E H ) p (H )
p (H E ) =
p (E H ) p (H ) + p (E H ) p (H )
• P(+HD) = 10/100 = 0.1
• P(-HD) = 1 – P(+HD) = 1 – 0.1 = 0.9
• P(+ECG|+HD) = 90/100 = 0.9
• P(-ECG|-HD) = 95/100 = 0.95
• P(+ECG|-HD) = 1 – P(-ECG|-HD) = 1 – 0.95 = 0.05

Therefore :
P(+HD|+ECG) = P(+ECG|+HD) * P(+HD)
P(+ECG|+HD) + P(+ECG|-HD) * P(-HD)
= 0.1 * 0.9
0.9*0.1 + 0.05*0.9
= 0.67