Professional Documents
Culture Documents
BN Lecture
BN Lecture
BN Lecture
3
Expert Systems
1 Rule Based Systems
4
2 Uncertainty
5
3 Normative Expert Systems
6
Causality - 1 Icy Roads
Icy Roads
7
- 2 Wet Grass
Rain Sprinkler
Watson’s Holmes’
Grass Wet Grass Wet
8
- 3 Earthquake or Burglar
Burglary Earthquake
Alarm
10
Conjunctions and Disjunctions
Venn Diagrams
• P(A v B)
=P(A)+P(B) - P(A & B)
(not mutually exclusive) A B
11
Conditional probability & independence
• Independence:
P(B|A)=P(B)
E.g. P(Heads|Even) = P(Heads)
12
Probability Distributions
• Probability Distribution:
0.5
– p(Weather=Sunny) = 0.5
– p(Weather=Rain)= 0.2
– p(Weather=Cloud)= 0.2
– p(Weather=Snow)= 0.1 0.2
0.1
• NB Distribution sums to 1.
S R C S
13
Joint Probability
14
Joint Distribution Example
• Domain with 2
variables each of
P(Toothache, Cavity)
which can take on 2
states.
Toothache ¬Toothache
15
Bayes’ Theorem
Simple:
P(Y|X) = P(X|Y)P(Y)
P(X)
General:
P(Y|X,E) = P(X|Y,E)P(Y|E)
P(X|E)
16
Bayesian Probability
• No need for
repeated Trials
• Appear to follow
rules of Classical
Probability
• How well do we The Probability Wheel:
A Tool for Assessing Probabilities
assign probabilities?
17
Bayesian Network - Definition
• Causal Structure
• Interconnected Nodes
• Directed Acyclic Links
• Joint Distribution formed
from conditional
distributions at each node.
18
Earthquake or Burglar
Burglary Earthquake
Alarm
B E P(A)
T T .95
T F .94
F T .29 Alarm
F F .001
A P(M) A P(J)
T .70 T .90
F .01 F .05
Mary Calls John Calls
20
Retrieving Probabilities from the
Conditional Distributions
• P(x1,…xn) = n P(xi|Parents(xi))
• E.g. i=1
21
Constructing A Network
- Node Ordering and Compactness
• Mary Calls Mary Calls John Calls
• John Calls
• Alarm
• Burglary Alarm
• Earthquake
Burglary
Earthquake
22
Node Ordering and Compactness contd.
• Mary Calls
• Johns Calls
• Earthquake
• Burglary
• Alarm
23
Node Ordering and Compactness contd.
Alarm
24
Conditional Independence
revisited - D-Separation
25
D-Separation contd.
26
D-Separation contd.
• A set of nodes, E, d-separates two sets of nodes, X and Y, if every undirected path from a
node in X to a node in Y is Blocked given E.
27
Blocking
X Y
E
Z
28
D-Separation - Example
Battery
• Moves and Battery are
independent given it is
known about Ignition Radio Ignition
• Moves and Radio are
independent if it is known
that Battery works Petrol
• Petrol and Radio are
independent given no Starts
evidence. But are
dependent given evidence
of Starts Moves
29
Inference
• Diagnostic Inferences (effects to causes)
• Causal Inferences (causes to effects)
• Intercausal Inferences - or ‘Explaining
Away’ (between causes of common effect)
• Mixed Inferences (combination of two or
more of the above)
30
Inference contd.
Q E Q E E
E Q E
31
Inference contd.
Burglary Earthquake
Alarm
• E.g. P(X|E):
Involves computing two values:
– Causal Support (evidence variables above X
connected through it’s parents)
– Evidential Support (evidence variables below X
connected through it’s children
35
Clustering
Cloudy Cloudy
36
Conditioning
- -
Cloudy+ Cloudy+ Cloudy Cloudy
37
Stochastic Simulation - Example
P(A=1) = 0.2
A
A P(B=1) A p(C=1)
0 0.2 B C 0 0.05
1 0.8 1 0.2
D E
38
Stochastic Simulation
Run repeated simulations to estimate the probability
distribution
• Let Wx = the states of all other variables except x.
• Let the Markov Blanket of a node be all of its parents,
children and parents of children.
• Distribution of each node, x, conditioned upon Wx can be
computed locally from their own probability with their
children’s :
• P(a|Wa) = . P(a) . P(b|a) . P(c|a)
• P(b|Wb) = . P(b|a) . P(d|b,c)
• P(c|Wc) = . P(c|a) . P(d|b,c) . P(e|c)
• Therefore, only the Markov blanket of a node is required to
compute the distribution
39
The Algorithm
• Set all observed nodes to their values
• Set all other nodes to random values
• STEP 1
• Select a node randomly from the network
• According to the states of the node’s markov blanket,
compute P(x=state, Wx) for all states
• STEP 2
• Use a random number generator that is biased according to
the distribution computed in step 1 to select the next value
of the node
• Repeat
40
Algorithm contd.
41
Case Study - Pathfinder
• Diagnostic Expert System for Lymph-Node Diseases
4 Versions of Pathfinder :
1) Rule Based
2) Experimented with Certainty Factors/Dempster-Shafer
theory/Bayesian Models
3) Refined Probabilities
4) Refined dependencies
42
Section 2 - Research Issues in
Uncertainty
1 Learning Belief Networks from Data
• Assume no
Case Fraud Gas Jewellery Age Sex
Knowledge of 1 No No No 30-50 F
Probabilities 2 No No No 30-50 M
3 Yes Yes Yes >50 M
Distributions or 4 No No No 30-50 M
Causal Structure. 5 No Yes No <30 F
6 No No No <30 F
• Is it possible to 7 No No No >50 M
8 No No Yes 30-50 F
infer both of these 9 No Yes No <30 M
from data? 10 No No No <30 F
43
Some Methods
Gas Jewelry
44
2 Dynamics - Markov Models
State Transition Model
Sensor Model
45
Updating over time
State t-1 State t
State t
Percept t
Supply Supply
47
3 Other approaches to modeling
Uncertainty
?
• Default Reasoning
• Dempster - Shafer
Theory
• Fuzzy Logic
48