Scribd Logo
Upload

Welcome to Scribd!

  • Bayesian Net B HBN

    Uploaded by

    G J
    7 views31 pages
    1. The document provides an introduction to graphical models and Bayesian networks. It discusses directed and undirected probabilistic graphical models, including Bayesian networks, hidden Markov models, and Markov networks. 2. The document uses a dataset with three random variables, A, B, and C, to demonstrate calculating probabilities from the joint distribution and building Bayesian networks representing the conditional dependencies between the variables. 3. Larger Bayesian networks with more than three variables are constructed from the dataset using different structure learning algorithms, and queries are performed on the learned networks.

    Original Description:

    Original Title

    Bayesian Net b Hbn

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    1. The document provides an introduction to graphical models and Bayesian networks. It discusses directed and undirected probabilistic graphical models, including Bayesian networks, hidden Markov models, and Markov networks. 2. The document uses a dataset with three random variables, A, B, and C, to demonstrate calculating probabilities from the joint distribution and building Bayesian networks representing the conditional dependencies between the variables. 3. Larger Bayesian networks with more than three variables are constructed from the dataset using different structure learning algorithms, and queries are performed on the learned networks.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    7 views31 pages

    Bayesian Net B HBN

    Uploaded by

    G J
    1. The document provides an introduction to graphical models and Bayesian networks. It discusses directed and undirected probabilistic graphical models, including Bayesian networks, hidden Markov models, and Markov networks. 2. The document uses a dataset with three random variables, A, B, and C, to demonstrate calculating probabilities from the joint distribution and building Bayesian networks representing the conditional dependencies between the variables. 3. Larger Bayesian networks with more than three variables are constructed from the dataset using different structure learning algorithms, and queries are performed on the learned networks.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 31

    lOMoARcPSD|37278598

    Bayesian-net-b - hbn

    Artificial intelligence (Indian Institute of Information Technology Vadodara)

    Scan to open on Studocu

    Studocu is not sponsored or endorsed by any college or university


    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    An Elementary Introduction to Graphical Models

    Pratik Shah

    February 2021

    Probabilistic Graphical Models


    1. Directed
    a. Bayesian Network (Belief Network)
    b. Hidden Markov Model
    c. Kalman Filter
    d. etc. . .
    2. Undirected
    a. Markov Network (Markov Random Field)
    b. Markov Logic Network

    Essence of GM
    1. Capture the variable dependency
    2. Reduce parameters
    3. Compact representation
    4. Factorization of Joint Probability

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    DATA: Three Random Variables


    df<-read.table("abcSampleTable.txt",head=TRUE)
    N<-nrow(df)
    df

    ## A B C
    ## 1 yes no yes
    ## 2 no no yes
    ## 3 yes yes no
    ## 4 yes yes no
    ## 5 yes no no
    ## 6 no no no
    ## 7 yes yes no
    ## 8 yes yes no
    ## 9 no no yes
    ## 10 yes no no
    ## 11 no no yes
    ## 12 yes no no
    ## 13 no no yes
    ## 14 yes yes no
    ## 15 yes yes yes
    ## 16 yes yes no
    ## 17 no no no
    ## 18 no no yes
    ## 19 yes yes no
    ## 20 no no no

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    library(dplyr)

    ##
    ## Attaching package: 'dplyr'
    ## The following objects are masked from 'package:stats':
    ##
    ## filter, lag
    ## The following objects are masked from 'package:base':
    ##
    ## intersect, setdiff, setequal, union
    library(bnlearn)

    Random Variables and Probability Table


    Answer the following probability queries:
    1. P (A = yes, B = yes):
    df.cond1<-df %>%
    filter(A=="yes",B=="yes")
    n1<-nrow(df.cond1)
    PAB<-n1/N
    PAB

    ## [1] 0.4
    2. P (A = yes|B = yes)
    df.cond2<-df %>%
    filter(B=="yes")
    n2<-nrow(df.cond2)
    PAgB<-n1/n2
    PAgB

    ## [1] 1
    3. P (A = yes)
    n3<-nrow(filter(df,df$A=="yes"))
    PA<-n3/N
    PA

    ## [1] 0.6
    4. P (B = yes)

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    n4<-nrow(filter(df,df$B=="yes"))
    PB<-n4/N
    PB

    ## [1] 0.4
    5. P (B = yes|A = yes)
    PBgA<-n1/n3
    PBgA

    ## [1] 0.6666667

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Recollect Bayes’ Rule


    P (B|A)P (A)
    P (A|B) =
    P (B)
    where,
    P (A|B) is posterior, P (B|A) is likelihood, P (A) is prior and, P (B) is evidence

    Joint Probability
    P (AB) = P (B|A)P (A)
    P (AB) = P (A|B)P (B)

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Interpretation-1
    DF <- df
    df <- lapply(DF, as.factor)
    df <- data.frame(df)
    bayes_bn1<-model2network('[A][B|A]')
    plot(bayes_bn1)

    bayes_bn1.fit<-bn.fit(bayes_bn1,df[,-3])
    bayes_bn1.fit

    ##
    ## Bayesian network parameters
    ##
    ## Parameters of node A (multinomial distribution)
    ##
    ## Conditional probability table:
    ## no yes
    ## 0.4 0.6
    ##
    ## Parameters of node B (multinomial distribution)
    ##

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    ## Conditional probability table:


    ##
    ## A
    ## B no yes
    ## no 1.0000000 0.3333333
    ## yes 0.0000000 0.6666667

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Interpretation-2
    bayes_bn2<-model2network('[B][A|B]')
    plot(bayes_bn2)

    bayes_bn2.fit<-bn.fit(bayes_bn2,df[,-3])
    bayes_bn2.fit

    ##
    ## Bayesian network parameters
    ##
    ## Parameters of node A (multinomial distribution)
    ##
    ## Conditional probability table:
    ##
    ## B
    ## A no yes
    ## no 0.6666667 0.0000000
    ## yes 0.3333333 1.0000000
    ##
    ## Parameters of node B (multinomial distribution)
    ##

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    ## Conditional probability table:


    ## no yes
    ## 0.6 0.4

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    What about a larger set of variables?


    bayes_bn31<-hc(df,score="k2")
    bayes_bn31.net<-bn.fit(bayes_bn31,df)
    plot(bayes_bn31)

    A B

    bayes_bn31.net

    ##
    ## Bayesian network parameters
    ##
    ## Parameters of node A (multinomial distribution)
    ##
    ## Conditional probability table:
    ##
    ## C
    ## A no yes
    ## no 0.2307692 0.7142857
    ## yes 0.7692308 0.2857143
    ##
    ## Parameters of node B (multinomial distribution)
    ##

    10

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    ## Conditional probability table:


    ##
    ## A
    ## B no yes
    ## no 1.0000000 0.3333333
    ## yes 0.0000000 0.6666667
    ##
    ## Parameters of node C (multinomial distribution)
    ##
    ## Conditional probability table:
    ## no yes
    ## 0.65 0.35
    bayes_bn32<-hc(df,score="bic")
    bayes_bn32.net<-bn.fit(bayes_bn32,df)
    plot(bayes_bn32)

    A B

    bayes_bn32.net

    ##
    ## Bayesian network parameters
    ##
    ## Parameters of node A (multinomial distribution)

    11

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    ##
    ## Conditional probability table:
    ## no yes
    ## 0.4 0.6
    ##
    ## Parameters of node B (multinomial distribution)
    ##
    ## Conditional probability table:
    ##
    ## A
    ## B no yes
    ## no 1.0000000 0.3333333
    ## yes 0.0000000 0.6666667
    ##
    ## Parameters of node C (multinomial distribution)
    ##
    ## Conditional probability table:
    ##
    ## A
    ## C no yes
    ## no 0.3750000 0.8333333
    ## yes 0.6250000 0.1666667

    12

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Let’s query these networks


    set.seed(0)
    ep1 <- cpquery(bayes_bn31.net, event = (A == "yes" & B == "yes"),
    evidence = TRUE, n = 1000)
    ep2 <- cpquery(bayes_bn32.net, event = (B == "yes" ),
    evidence = (A == "yes"), n = 1000)
    ep1

    ## [1] 0.394
    ep2

    ## [1] 0.6484245

    13

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Joint Probability
    bayes_bn31

    ##
    ## Bayesian network learned via Score-based methods
    ##
    ## model:
    ## [C][A|C][B|A]
    ## nodes: 3
    ## arcs: 2
    ## undirected arcs: 0
    ## directed arcs: 2
    ## average markov blanket size: 1.33
    ## average neighbourhood size: 1.33
    ## average branching factor: 0.67
    ##
    ## learning algorithm: Hill-Climbing
    ## score: Cooper & Herskovits' K2
    ## tests used in the learning procedure: 10
    ## optimized: TRUE
    bayes_bn32

    ##
    ## Bayesian network learned via Score-based methods
    ##
    ## model:
    ## [A][B|A][C|A]
    ## nodes: 3
    ## arcs: 2
    ## undirected arcs: 0
    ## directed arcs: 2
    ## average markov blanket size: 1.33
    ## average neighbourhood size: 1.33
    ## average branching factor: 0.67
    ##
    ## learning algorithm: Hill-Climbing
    ## score: BIC (disc.)
    ## penalization coefficient: 1.497866
    ## tests used in the learning procedure: 7
    ## optimized: TRUE

    P (ABC) = P (C)P (A|C)P (B|A)


    P (ABC) = P (A)P (B|A)P (C|A)

    14

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Aha. . . you have seen a Graphical Model!


    nbr(bayes_bn32,node="A")

    ## [1] "B" "C"


    parents(bayes_bn32,node="A")

    ## character(0)
    children(bayes_bn32,node="A")

    ## [1] "B" "C"


    mb(bayes_bn32,node="A")

    ## [1] "B" "C"


    plot(bayes_bn32,highlight=list(nodes="A"))

    A B

    arcs(bayes_bn32)

    ## from to
    ## [1,] "A" "B"
    ## [2,] "A" "C"

    15

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    path(bayes_bn32, from="B", to="C")

    ## [1] FALSE
    dsep(bayes_bn32, "B","C")

    ## [1] FALSE
    dsep(bayes_bn32, "B","C","A")

    ## [1] TRUE

    16

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Can we go back to Sample!


    set.seed(0)
    samples.bn32<-cpdist(bayes_bn32.net, nodes=nodes(bayes_bn32.net), evidence=TRUE, n=20)
    samples.bn32

    ## A B C
    ## 1 no no no
    ## 2 yes no yes
    ## 3 yes yes no
    ## 4 yes no no
    ## 5 no no yes
    ## 6 yes yes no
    ## 7 no no yes
    ## 8 no no yes
    ## 9 no no yes
    ## 10 no no no
    ## 11 yes no no
    ## 12 yes yes no
    ## 13 yes no no
    ## 14 no no yes
    ## 15 yes yes no
    ## 16 no no no
    ## 17 yes yes no
    ## 18 no no no
    ## 19 no no yes
    ## 20 yes yes no

    17

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    How many three node Directed Acyclic Graphs?

    Figure 1: Guesses?

    18

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Need a Larger One?


    course.grades<-read.table("2020_bn_nb_data.txt",head=TRUE)
    head(course.grades)

    ## EC100 EC160 IT101 IT161 MA101 PH100 PH160 HS101 QP


    ## 1 BC CC BB BC CC BC AA BB y
    ## 2 CC BC BB BB CC BC AB BB y
    ## 3 AB BB AB AB BB CC BC AB y
    ## 4 BC CC BB BB BB BB BC BB y
    ## 5 BC AB CD BC BC BC BC CD y
    ## 6 DD CC DD CD CD CC BC BC n
    course.grades<-lapply(course.grades,as.factor)
    course.grades<-data.frame(course.grades)
    course.grades.net<-hc(course.grades[,-9],score="k2")
    plot(course.grades.net)

    HS101
    EC100 PH160

    EC160 PH100

    IT101 MA101
    IT161

    course.grades.net.fit<-bn.fit(course.grades.net, course.grades[,-9])
    bn.fit.barchart(course.grades.net.fit$EC100)

    ## Loading required namespace: lattice

    19

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Conditional Probabilities for Node EC100


    0.0 0.2 0.4 0.6 0.8 1.0

    AA AB BB
    F
    DD
    CD
    CC
    BC
    BB
    AB
    AA
    BC CC CD
    F
    DD
    Levels

    CD
    CC
    BC
    BB
    AB
    AA
    DD F
    F
    DD
    CD
    CC
    BC
    BB
    AB
    AA
    0.0 0.2 0.4 0.6 0.8 1.0

    Probabilities

    20

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Naive Bayes Classifier


    library(bnclassify)

    ##
    ## Attaching package: 'bnclassify'
    ## The following objects are masked from 'package:bnlearn':
    ##
    ## modelstring, narcs, nparams
    ## The following object is masked from 'package:dplyr':
    ##
    ## vars
    course.grades<-read.table("2020_bn_nb_data.txt",head=TRUE)
    course.grades<-lapply(course.grades,as.factor)
    course.grades<-data.frame(course.grades)
    nb.grades<-nb(class="QP",dataset=course.grades)
    plot(nb.grades)

    QP

    EC100 EC160 IT101 IT161 MA101 PH100 PH160 HS101

    nb.grades<-lp(nb.grades, course.grades, smooth=0)


    p<-predict(nb.grades, course.grades)

    21

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    cm<-table(predicted=p, true=course.grades$QP)
    cm

    ## true
    ## predicted n y
    ## n 70 1
    ## y 2 159

    Remove MA101
    nb.grades<-nb(class="QP",dataset=course.grades[,-5])
    plot(nb.grades)

    QP

    EC100 EC160 IT101 IT161 PH100 PH160 HS101

    nb.grades<-lp(nb.grades, course.grades[,-5], smooth=1)


    p<-predict(nb.grades, course.grades[,-5])
    cm<-table(predicted=p, true=course.grades$QP)
    cm

    ## true
    ## predicted n y
    ## n 69 0
    ## y 3 160

    22

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Conditional Independence Check


    ci.test("MA101","IT101","QP",course.grades)

    ##
    ## Mutual Information (disc.)
    ##
    ## data: MA101 ~ IT101 | QP
    ## mi = 105.55, df = 98, p-value = 0.2832
    ## alternative hypothesis: true value is greater than 0

    23

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    Something More Interesting


    tn <- tan_cl("QP", course.grades)
    tn <- lp(tn, course.grades, smooth = 1)
    plot(tn)

    QP

    EC100

    EC160 PH100 MA101

    IT101

    IT161 HS101

    PH160

    tn <- lp(tn, course.grades, smooth = 1)


    p <- predict(tn, course.grades)
    cm1<-table(predicted=p, true=course.grades$QP)
    cm1

    ## true
    ## predicted n y
    ## n 71 0
    ## y 1 160

    24

    Downloaded by G J (rmpatel2197@gmail.com)
    lOMoARcPSD|37278598

    References
    1. Bayesian Network without Tears by Eugene Charniak
    2. Bayesian Networks with R by Bojan Mihaljevic
    3. bnstruct: an R package for Bayesian Network Structure Learning with missing data by
    Francesco Sambo and Alberto Franzin
    4. Introduction to Artificial Intelligence by Stuart Russell and Peter Norvig

    25

    Downloaded by G J (rmpatel2197@gmail.com)

    You might also like