Write some theory of search algorithms and discuss random search with example .

Based on the search problems we can classify the search algorithms into

uninformed (Blind search) search and informed search (Heuristic search) algorithms.
UNIFORMED/BLIND SEARCH:
The uninformed search does not contain any domain knowledge such as closeness, the location
of the goal. It operates in a brute-force way as it only includes information about how to traverse
the tree and how to identify leaf and goal nodes. Uninformed search applies a way in which
search tree is searched without any information about the search space like initial state operators
and test for the goal, so it is also called blind search.It examines each node of the tree until it
achieves the goal node.
It can be divided into five main types:
o Breadth-first search
o Uniform cost search
o Depth-first search
o Iterative deepening depth-first search
o Bidirectional Search
Breadth-first Search:
o Breadth-first search is the most common search strategy for traversing a tree or graph.
This algorithm searches breadthwise in a tree or graph, so it is called breadth-first search.
o BFS algorithm starts searching from the root node of the tree and expands all successor
node at the current level before moving to nodes of next level.
o The breadth-first search algorithm is an example of a general-graph search algorithm.
o Breadth-first search implemented using FIFO queue data structure.
Advantages:
 o BFS will provide a solution if any solution exists.
o If there are more than one solutions for a given problem, then BFS will provide the
minimal solution which requires the least number of steps.
Disadvantages:
o It requires lots of memory since each level of the tree must be saved into memory to
expand the next level.
o BFS needs lots of time if the solution is far away from the root node.
Depth-first Search
o Depth-first search is a recursive algorithm for traversing a tree or graph data structure.
o It is called the depth-first search because it starts from the root node and follows each
path to its greatest depth node before moving to the next path.
o DFS uses a stack data structure for its implementation.
o The process of the DFS algorithm is similar to the BFS algorithm.
Note: Backtracking is an algorithm technique for finding all possible solutions using
recursion.
Advantage:
o DFS requires very less memory as it only needs to store a stack of the nodes on the path
from root node to the current node.
o It takes less time to reach to the goal node than BFS algorithm (if it traverses in the right
path).
Disadvantage:
o There is the possibility that many states keep re-occurring, and there is no guarantee of
finding the solution.
o DFS algorithm goes for deep down searching and sometime it may go to the infinite loop.

Informed Search
Informed search algorithms use domain knowledge. In an informed search, problem information
is available which can guide the search. Informed search strategies can find a solution more
efficiently than an uninformed search strategy. Informed search is also called a Heuristic search.
A heuristic is a way which might not always be guaranteed for best solutions but guaranteed to
find a good solution in reasonable time.
Informed search can solve much complex problem which could not be solved in another way.
An example of informed search algorithms is a traveling salesman problem.
1. Greedy Search
 2. A* Search
1.Best-first Search Algorithm (Greedy Search):
Greedy best-first search algorithm always selects the path which appears best at that moment. It
is the combination of depth-first search and breadth-first search algorithms. It uses the heuristic
function and search. Best-first search allows us to take the advantages of both algorithms. With
the help of best-first search, at each step, we can choose the most promising node. In the best
first search algorithm, we expand the node which is closest to the goal node and the closest cost
is estimated by heuristic function, i.e.
1. f(n)= g(n).
Were, h(n)= estimated cost from node n to the goal.
The greedy best first algorithm is implemented by the priority queue.
Best first search algorithm:
o Step 1: Place the starting node into the OPEN list.
o Step 2: If the OPEN list is empty, Stop and return failure.
o Step 3: Remove the node n, from the OPEN list which has the lowest value of h(n), and
places it in the CLOSED list.
o Step 4: Expand the node n, and generate the successors of node n.
o Step 5: Check each successor of node n, and find whether any node is a goal node or not.
If any successor node is goal node, then return success and terminate the search, else
proceed to Step 6.
o Step 6: For each successor node, algorithm checks for evaluation function f(n), and then
check if the node has been in either OPEN or CLOSED list. If the node has not been in
both list, then add it to the OPEN list.
o Step 7: Return to Step 2.
A* Search Algorithm:
A* search is the most commonly known form of best-first search. It uses heuristic function h(n),
and cost to reach the node n from the start state g(n). It has combined features of UCS and
greedy best-first search, by which it solve the problem efficiently. A* search algorithm finds the
shortest path through the search space using the heuristic function. This search algorithm
expands less search tree and provides optimal result faster. A* algorithm is similar to UCS
except that it uses g(n)+h(n) instead of g(n).
In A* search algorithm, we use search heuristic as well as the cost to reach the node. Hence we
can combine both costs as following, and this sum is called as a fitness number.

At each point in the search space, only those node is expanded which have the lowest value of
f(n), and the algorithm terminates when the goal node is found.
Algorithm of A* search:
Step1: Place the starting node in the OPEN list.
Step 2: Check if the OPEN list is empty or not, if the list is empty then return failure and stops.
Step 3: Select the node from the OPEN list which has the smallest value of evaluation function
(g+h), if node n is goal node then return success and stop, otherwise
Step 4: Expand node n and generate all of its successors, and put n into the closed list. For each
successor n', check whether n' is already in the OPEN or CLOSED list, if not then compute
evaluation function for n' and place into Open list.
Step 5: Else if node n' is already in OPEN and CLOSED, then it should be attached to the back
pointer which reflects the lowest g(n') value.
Step 6: Return to Step 2.

Random search:
Random search is a group of numerical optimization methods that do not require the gradient of
the problem to be optimized, and Random search can be applied to functions that are not
continuous or differentiable.
Random search is a technique where random combinations of the hyperparameters are used to
find the best solution for the built model. It is similar to grid search, and yet it has proven to yield
better results comparatively. The drawback of random search is that it yields high variance
during computing. Since the selection of parameters is completely random; and since no
intelligence is used to sample these combinations, luck plays its part.
Below is a visual description of search pattern of the random search:
As random values are selected at each instance, it is highly likely that the whole of action space
has been reached because of the randomness, which takes a huge amount of time to cover every
aspect of the combination during grid search. This works best under the assumption that not all
hyperparameters are equally important. In this search pattern, random combinations of
parameters are considered in every iteration. The chances of finding the optimal parameter are
comparatively higher in random search because of the random search pattern where the model
might end up being trained on the optimised parameters without any aliasing.
Algorithm : Random search
Step 1: n= initial node
Step 2: finish If n is goal
Step 3: expand n to get a set S of child nodes
Step 4:select a node n' from S at random
Step 5 : n= n' return to 2

● This method selects randomly a new state from the current one. If the goal state is reached, the
search terminates .Otherwisethe method randomly selectan other operator to move to the next
State .Goal achievement is not insured in finite time .
● Failure to detect the repeated states can turn a linear problem into an exponential problem

Avoiding repeated states:

● Do not regenerate the state you just came from .
● Do not create path with cycles.
Advantages:
● The random search algorithms are very easy to implement; the objective function does not
have to be smooth; it is enough if the function can be evaluated (even by simulations)
Disadvantages:
● The random search algorithms are not necessarily convergent in the usual sense ; in most cases
they are only convergent in a probabilistic sense.
Example:
● The simplest idea is to replace the search direction corresponding to the gradient with a
random direction .

Write in detail about Search with closed and open list with example
Heuristics function: Heuristic is a function which is used in Informed Search, and it finds the
most promising path. It takes the current state of the agent as its input and produces the
estimation of how close agent is from the goal. The heuristic method, however, might not always
give the best solution, but it guaranteed to find a good solution in reasonable time. Heuristic
function estimates how close a state is to the goal. It is represented by h(n), and it calculates the
cost of an optimal path between the pair of states. The value of the heuristic function is always
positive.
Pure Heuristic Search: Pure heuristic search is the simplest form of heuristic search algorithms.
It expands nodes based on their heuristic value h(n). It maintains two lists, OPEN and CLOSED
list. In the CLOSED list, it places those nodes which have already expanded and in the OPEN
list, it places nodes which have yet not been expanded.
On each iteration, each node n with the lowest heuristic value is expanded and generates all its
successors and n is placed to the closed list. The algorithm continues unit a goal state is found. In
the informed search we will discuss two main algorithms which are given below:
o Best First Search Algorithm(Greedy search)
o A* Search Algorithm
Best-first Search Algorithm (Greedy Search): Greedy best-first search algorithm always
selects the path which appears best at that moment. It is the combination of depth-first search and
breadth-first search algorithms. It uses the heuristic function and search. Best-first search allows
us to take the advantages of both algorithms. With the help of best-first search, at each step, we
can choose the most promising node. In the best first search algorithm, we expand the node
which is closest to the goal node and the closest cost is estimated by heuristic function, i.e. f(n)=
g(n).
Were, h(n)= estimated cost from node n to the goal.
The greedy best first algorithm is implemented by the priority queue
Example: Consider the below search problem, and we will traverse it using greedy best-first
search. At each iteration, each node is expanded using evaluation function f(n)=h(n) , which is
given in the below table.

In this search example, we are using two lists which are OPEN and CLOSED Lists. Following
are the iteration for traversing the above example.
Expand the nodes of S and put in the CLOSED list
Initialization: Open [A, B], Closed [S]
Iteration 1: Open [A], Closed [S, B]
Iteration 2: Open [E, F, A], Closed [S, B]
: Open [E, A], Closed [S, B, F]
Iteration 3: Open [I, G, E, A], Closed [S, B, F]
: Open [I, E, A], Closed [S, B, F, G]
Hence the final solution path will be: S----> B----->F----> G

A* Search Algorithm: A* search is the most commonly known form of best-first search. It uses
heuristic function h(n), and cost to reach the node n from the start state g(n). It has combined
features of UCS and greedy best-first search, by which it solve the problem efficiently. A* search
algorithm finds the shortest path through the search space using the heuristic function. This
search algorithm expands less search tree and provides optimal result faster. A* algorithm is
similar to UCS except that it uses g(n)+h(n) instead of g(n). In A* search algorithm, we use
search heuristic as well as the cost to reach the node. Hence we can combine both costs as
following, and this sum is called as a fitness number.
Example:
In this example, we will traverse the given graph using the A* algorithm. The heuristic value of
all states is given in the below table so we will calculate the f(n) of each state using the formula
f(n)= g(n) + h(n), where g(n) is the cost to reach any node from start state.
Here we will use OPEN and CLOSED list.
Solution:

Initialization: {(S, 5)}

Iteration1: {(S--> A, 4), (S-->G, 10)}
Iteration2: {(S--> A-->C, 4), (S--> A-->B, 7), (S-->G, 10)}
Iteration3: {(S--> A-->C--->G, 6), (S--> A-->C--->D, 11), (S--> A-->B, 7), (S-->G, 10)}
Iteration 4 will give the final result, as S--->A--->C--->G it provides the optimal path with cost
6.
Write in detail Depth first and Breadth first search with example.

Graph Traversal Algorithm: In this part of the tutorial we will discuss the techniques by using
which, we can traverse all the vertices of the graph.
Traversing the graph means examining all the nodes and vertices of the graph. There are two
standard methods by using which, we can traverse the graphs. Lets discuss each one of them in
detail.

Breadth First Search

Depth First Search

-Breadth First Search (BFS) Algorithm Breadth first search is a graph traversal algorithm that
starts traversing the graph from root node and explores all the neighbouring nodes. Then, it
selects the nearest node and explore all the unexplored nodes. The algorithm follows the same
process for each of the nearest node until it finds the goal.

The algorithm of breadth first search is given below. The algorithm starts with examining the
node A and all of its neighbours. In the next step, the neighbours of the nearest node of A are
explored and process continues in the further steps. The algorithm explores all neighbours of all
the nodes and ensures that each node is visited exactly once and no node is visited twice.

-Algorithm:

Step 1: SET STATUS = 1 (ready state) for each node in G

Step 2: Enqueue the starting node A

and set its STATUS = 2

(waiting state)

Step 3: Repeat Steps 4 and 5 until

QUEUE is empty

Step 4: Dequeue a node N. Process it

and set its STATUS = 3

(processed state).

Step 5: Enqueue all the neighbours of N that are in the ready state

(whose STATUS = 1) and set

their STATUS = 2

(waiting state)

[END OF LOOP]
Step 6: EXIT

-Example:

Consider the graph G shown in the following image, calculate the minimum path p from node A
to node E. Given that each edge has a length of 1.

Solution:

Minimum Path P can be found by applying breadth first search algorithm that will begin at node
A and will end at E. the algorithm uses two queues, namely QUEUE1 and QUEUE2. QUEUE1
holds all the nodes that are to be processed while QUEUE2 holds all the nodes that are processed
and deleted from QUEUE1.

Lets start examining the graph from NodeA.

1. Add A to QUEUE1 and NULL to QUEUE2.

QUEUE1 = {A}

QUEUE2 = {NULL}

2. Delete the Node A from QUEUE1 and insert all its neighbours. Insert Node A into QUEUE2

QUEUE1 = {B, D}

QUEUE2 = {A}

3. Delete the node B from QUEUE1 and insert all its neighbours. Insert node B into QUEUE2.

QUEUE1 = {D, C, F}

QUEUE2 = {A, B}

4. Delete the node D from QUEUE1 and insert all its neighbours. Since F is the only neighbour
of it which has been inserted, we will not insert it again. Insert node D into QUEUE2.
QUEUE1 = {C, F}

QUEUE2 = { A, B, D}

5. Delete the node C from QUEUE1 and insert all its neighbours. Add node C to QUEUE2.

QUEUE1 = {F, E, G}

QUEUE2 = {A, B, D, C}

6. Remove F from QUEUE1 and add all its neighbours. Since all of its neighbours has already
been added, we will not add them again. Add node F to QUEUE2.

QUEUE1 = {E, G}

QUEUE2 = {A, B, D, C, F}

7. Remove E from QUEUE1, all of E's neighbours has already been added to QUEUE1 therefore
we will not add them again. All the nodes are visited and the target node i.e. E is encountered
into QUEUE2.

QUEUE1 = {G}

QUEUE2 = {A, B, D, C, F, E}

Now, backtrack from E to A, using the nodes available in QUEUE2.

The minimum path will be A → B → C → E.

-Depth First Search (DFS) Algorithm

Depth first search (DFS) algorithm starts with the initial node of the graph G, and then goes to
deeper and deeper until we find the goal node or the node which has no children. The algorithm,
then backtracks from the dead end towards the most recent node that is yet to be completely
unexplored.

The data structure which is being used in DFS is stack. The process is similar to BFS algorithm.
In DFS, the edges that leads to an unvisited node are called discovery edges while the edges that
leads to an already visited node are called block edges.

-Algorithm:

Step 1: SET STATUS = 1 (ready state) for each node in G

Step 2: Push the starting node A on the stack and set its STATUS = 2 (waiting state)
Step 3: Repeat Steps 4 and 5 until STACK is empty

Step 4: Pop the top node N. Process it and set its STATUS = 3 (processed state)

Step 5: Push on the stack all the neighbours of N that are in the ready state (whose STATUS = 1)
and set their

STATUS = 2 (waiting state)

[END OF LOOP]

Step 6: EXIT

-Example :

Consider the graph G along with its adjacency list, given in the figure below. Calculate the order
to print all the nodes of the graph starting from node H, by using depth first search (DFS)
algorithm.

-Solution :

Push H onto the stack

STACK : H

DEFINE HEURISTIC SEARCH WITH EXAMPLE

It is named so because there is some extra information about the states. This extra information is
useful to compute the preference among the child nodes to explore and expand.There would be a
heuristic function associated with each node. Best First Search (BFS), A*, Mean and Analysis
are the examples of informed search.

Heuristic search algorithm contains an array of knowledge such as how far we are from the
goal, path cost, how to reach to goal node, etc. This knowledge help agents to explore less to the
search space and find more efficiently the goal node.
The informed search algorithm is more useful for large search space. Informed search algorithm
uses the idea of heuristic, so it is also called Heuristic search.

Heuristics function: Heuristic is a function which is used in Informed Search, and it finds the
most promising path. It takes the current state of the agent as its input and produces the
estimation of how close agent is from the goal. The heuristic method, however, might not always
give the best solution, but it guaranteed to find a good solution in reasonable time. Heuristic
function estimates how close a state is to the goal. It is represented by h(n), and it calculates the
cost of an optimal path between the pair of states. The value of the heuristic function is always
positive.

Admissibility of the heuristic function is given as:

h(n) <= h*(n)

Here h(n) is heuristic cost, and h*(n) is the estimated cost. Hence heuristic cost should be less
than or equal to the estimated cost.

BEST-FIRST SEARCH ALGORITHM (greedy search)

The Best first search uses the concept of a Priority queue and heuristic search. To search the
graph space, the BFS method uses two lists for tracking the traversal. An ‘Open’ list which keeps
track of the current ‘immediate’ nodes available for traversal and ‘CLOSED’ list that keeps track
of the nodes already traversed

Best First Search Algorithm

1. Create 2 empty lists: OPEN and CLOSED

2. Start from the initial node (say N) and put it in the ‘ordered’ OPEN list

3. Repeat the next steps until GOAL node is reached

1. If OPEN list is empty, then EXIT the loop returning ‘False’ 2. Select the first/top node (say N)
in the OPEN list and move it to the CLOSED list. Also capture the information of the parent
node

3. If N is a GOAL node, then move the node to the Closed list and exit the

loop returning ‘True’. The solution can be found by backtracking the path

4. If N is not the GOAL node, expand node N to generate the ‘immediate’ next nodes linked to
node N and add all those to the OPEN list
5. Reorder the nodes in the OPEN list in ascending order according to an

evaluation function f(n)

This algorithm will traverse the shortest path first in the queue. The time complexity of the
algorithm is given by O(n*logn) .

Best First Search Example

Let’s have a look at the graph below and try to implement both Greedy BFS and A* algorithms

step by step using the two list, OPEN and CLOSED

g(n) = Path Distance

h(n) = Estimate to Goal
f(n) = Combined Hueristicsi.e. g(n) + h(n)
Even though you would find that both Greedy BFS and A* algorithms find the path equally
efficiently, number of steps, you may notice that the A* algorithm is able to come up with is a
more optimal path than Greedy BFS. So in summary, both Greedy BFS and A* are Best first
searches but Greedy BFS is neither complete, nor optimal whereas A* is both complete and
optimal. However, A* uses more memory than Greedy BFS, but it guarantees that the path found
is optimal.
Write in detail about A* algorithm with 2 examples.
A* ALGORITHM:
A * algorithm is a searching algorithm that searches for the shortest path between the initial and
the final state. It is used in various applications, such as maps.
In maps the A* algorithm is used to calculate the shortest distance between the source (initial
state) and the destination (final state).
HOW IT WORKS:
Imagine a square grid which possesses many obstacles, scattered randomly. The initial and the
final cell is provided. The aim is to reach the final cell in the shortest amount of time.
The diagram below shows how the initial cell reaches the final cell in the shortest time by using
A* algorithm:

A* algorithm has 3 parameters:

● g: the cost of moving from the initial cell to the current cell. Basically, it is the sum of all
the cells that have been visited since leaving the first cell.
● h: also known as the heuristic value, it is the estimated cost of moving from the current
cell to the final cell. The actual cost cannot be calculated until the final cell is reached.
Hence, h is the estimated cost. We must make sure that there is never an over estimation
of the cost.
● f: it is the sum of g and h. So, f = g + h

The way that the algorithm makes its decisions is by taking the f-value into account. The
algorithm selects the smallest f-valued cell and moves to that cell. This process continues until
the algorithm reaches its goal cell.
Example 1:
A* algorithm is very useful in graph traversals as well. Suppose there is the following graph,
apply A* algorithm on it. The initial node is A and the goal node is E.

The traversal from node A to E is explained below –

At every step, the f-value is being re-calculated by adding together the g and h values. The
minimum f-value node is selected to reach the goal state. Notice how node B is never visited.
Probabilistic reasoning:
Probabilistic reasoning is a way of knowledge representation where we apply the concept of
probability to indicate the uncertainty in knowledge. In probabilistic reasoning, we combine
probability theory with logic to handle the uncertainty.
We use probability in probabilistic reasoning because it provides a way to handle the uncertainty
that is the result of someone's laziness and ignorance.
In the real world, there are lots of scenarios, where the certainty of something is not confirmed,
such as "It will rain today," "behaviour of someone for some situations," "A match between two
teams or two players." These are probable sentences for which we can assume that it will happen
but not sure about it, so here we use probabilistic reasoning.

Need of probabilistic reasoning in AI:

● When there are unpredictable outcomes.
● When specifications or possibilities of predicates becomes too large to handle.
● When an unknown error occurs during an experiment.
Probability
Probability can be defined as a chance that an uncertain event will occur. It is the numerical
measure of the likelihood that an event will occur. The value of probability always remains
between 0 and 1 that represent ideal uncertainties.
● 0 ≤ P(A) ≤ 1, where P(A) is the probability of an event A.
● P(A) = 0, indicates total uncertainty in an event A.
● P(A) =1, indicates total certainty in an event A.
We can find the probability of an uncertain event by using the below formula.

● P(¬A) = probability of a not happening event.

● P(¬A) + P(A) = 1.
Event: Each possible outcome of a variable is called an event.
Sample space: The collection of all possible events is called sample space.
Random variables: Random variables are used to represent the events and objects in the real
world.
Prior probability: The prior probability of an event is probability computed before observing
new information.
Posterior Probability: The probability that is calculated after all evidence or information has
taken into account. It is a combination of prior probability and new information.
Conditional probability:
Conditional probability is a probability of occurring an event when another event has already
happened.
 Let's suppose, we want to calculate the event A when event B has already occurred, "the
probability of A under the conditions of B", it can be written as:

Where P(A⋀B)= Joint probability of a and B

P(B)= Marginal probability of B.
If the probability of A is given and we need to find the probability of B, then it will be given as:
It can be explained by using the below Venn diagram, where B is occurred event, so sample
space will be reduced to set B, and now we can only calculate event A when event B is already
occurred by dividing the probability of P(A⋀B) by P( B ).

Bayes' theorem:
Bayes' theorem is also known as Bayes' rule, Bayes' law, or Bayesian reasoning, which
determines the probability of an event with uncertain knowledge.
In probability theory, it relates the conditional probability and marginal probabilities of two
random events.
Bayes' theorem was named after the British mathematician Thomas Bayes. The Bayesian
inference is an application of Bayes' theorem, which is fundamental to Bayesian statistics.
It is a way to calculate the value of P(B|A) with the knowledge of P(A|B).
Bayes' theorem allows updating the probability prediction of an event by observing new
information of the real world.
Example: If cancer corresponds to one's age then by using Bayes' theorem, we can determine the
probability of cancer more accurately with the help of age.
Bayes' theorem can be derived using product rule and conditional probability of event A with
known event B:
As from product rule we can write:
1. P(A ⋀ B)= P(A|B) P(B) or
Similarly, the probability of event B with known event A:
1. P(A ⋀ B)= P(B|A) P(A)
P(A|B) = P(B|A) P(A) /P(B)
Equating right hand side of both the equations, we will get:
P(A|B) = P(B|A)P(A)/ P(B)
The above equation (a) is called as Bayes' rule or Bayes' theorem. This equation is basic of
most modern AI systems for probabilistic inference.
 It shows the simple relationship between joint and conditional probabilities. Here,
P(A|B) is known as posterior, which we need to calculate, and it will be read as Probability of
hypothesis A when we have occurred an evidence B.
P(B|A) is called the likelihood, in which we consider that hypothesis is true, then we calculate
the probability of evidence.
P(A) is called the prior probability, probability of hypothesis before considering the evidence
P(B) is called marginal probability, pure probability of an evidence.
In the equation (a), in general, we can write P (B) = P(A)*P(B|Ai), hence the Bayes' rule can be
written as:
P(Ai|B) = P(Ai)*P(B|Ai) / ∑ P(Ai) * P(B|Ai)
Where A 1, A 2, A 3,........, A n is a set of mutually exclusive and exhaustive events.
Applying Bayes' rule:
Example-1:
Question: what is the probability that a patient has diseases meningitis with a stiff neck?
Given Data:
A doctor is aware that disease meningitis causes a patient to have a stiff neck, and it occurs 80%
of the time. He is also aware of some more facts, which are given as follows:
o The Known probability that a patient has meningitis disease is 1/30,000.
o The Known probability that a patient has a stiff neck is 2%.
Let a be the proposition that patient has stiff neck and b be the proposition that patient has
meningitis. , so we can calculate the following as:
P(a|b) = 0.8
P(b) = 1/30000
P(a)= .02
Hence, we can assume that 1 patient out of 750 patients has meningitis disease with a stiff neck.

Q. Discuss Bayesian Networks- representation with example.

Bayesian Networks are a type of Probabilistic Graphical Model that uses the Bayesian
inferences for probability computations. It represents a set of variables and its conditional
probabilities with a Directed Acyclic Graph (DAG). They are primarily suited for considering an
event that has occurred and predicting the likelihood that any one of the several possible known
causes is the contributing factor.
 As mentioned above, by making use of the relationships which are specified by the Bayesian
Network, we can obtain the Joint
Probability Distribution (JPF) with the conditional probabilities. Each node in the graph
represents a random variable and the arc (or directed arrow) represents the relationship between
the nodes. They can be either continuous or discrete in nature.
In the above diagram A, B, C and D are 4 random variables represented by nodes given in the
network of the graph. To node B, A is its parent node and C is its child node. Node C is
independent of Node A.
Before we get into the implementation of a Bayesian Network, there are a few probability basics
that have to be understood.
Example of Bayesian Networks
Let us now understand the mechanism of Bayesian Networks and their advantages with the help
of a simple example. In this example, let us imagine that we are given the task of modeling a
student’s marks (m) for an exam he has just given. From the given Bayesian Network Graph
below, we see that the marks depend upon two other variables. They are,
• Exam Level (e)– This discrete variable denotes the difficulty of the exam and has two values
(0 for easy and 1 for difficult)
• IQ Level (i) – This represents the Intelligence Quotient level of the student and is also discrete
in nature having two values (0 for low and 1 for high)
Additionally, the IQ level of the student also leads us to another
variable, which is the Aptitude Score of the student (s). Now, with marks the student has scored,
he can secure admission to a particular university. The probability distribution for getting
admitted (a) to a university is also given below.
In the above graph, we see several tables representing the probability distribution values of the
given 5 variables. These tables are called the Conditional Probabilities Table or CPT. There are a
few properties of the CPT given below –
• The sum of the CPT values in each row must be equal to 1 because all the possible cases for a
particular variable are
exhaustive (representing all possibilities).
• If a variable that is Boolean in nature has k Boolean parents,
then in the CPT it has 2K probability values.
Conclusion
There are innumerable applications to Bayesian Networks in Spam Filtering, Semantic Search,
Information Retrieval, and many more. For
example, with a given symptom we can predict the probability of a disease occurring with
several other factors contributing to the disease. Thus, the concept of the Bayesian Network is
introduced in
this article along with its implementation with a real-life example.

Q. Write about construction and inference of Bayesian Network

Bayesian Networks Definition: A Bayesian network consists of the following:
• A set of variables and a set of directed edges between variables. • Each variable has a finite set
of mutually exclusive states.
• The variables together with the directed edges form an acyclic directed graph (DAG).
• To each variable A with parents B1, . . ., Bn, a conditional probability table P(A|B1, . . . , Bn) is
attached
Constructing a Bayesian network
• Step 1: Identify the random variables
• Step 2: Determine the conditional dependencies
• Step 3: Populate the CPTs Can be learned from observation data!

Factoring joint distributions

• Using the chain rule we can always factor a joint distribution as follows:
P(A,B,E,J,M) =
P(A | B,E,J,M) P(B,E,J,M) =
P(A | B,E,J,M) P(B | E,J,M) P(E,J,M) =
P(A | B,E,J,M) P(B | E, J,M) P(E | J,M) P(J,M)
P(A | B,E,J,M) P(B | E, J,M) P(E | J,M)P(J | M)P(M)
• This type of conditional dependencies can also be represented graphically.
Constructing a Bayesian network:
• Step 1: Identify the random variables
• Step 2: Determine the conditional dependencies - Select on ordering of the variables - Add
them one at a time - For each new variable X added select the minimal subset of nodes as parents
such that X is independent from all other nodes in the current network given its parents.
• Step 3: Populate the CPTs - We will discuss this when we talk about density estimations
Bayesian network: Inference
• Once the network is constructed, we can use algorithm for inferring the values of unobserved
variables.
• For example, in our previous network the only observed variables are the phone call and the
radio announcement. However, what we are really interested in is whether there was a burglary
or not.
 Inference
• Lets start with a simpler question
- How can we compute a joint distribution from the network? - For example, P(B,¬E,A,J, ¬M)?
• Answer: - That’s easy, lets use the network
Other inference methods
• Convert network to a polytree
- In a polytree no two nodes have more than one path between them
- We can convert arbitrary networks to a polytree by clustering (grouping) nodes. For such a
graph there is a algorithm which is linear in the number of nodes
- However, converting into a polytree can result in an exponential increase in the size of the
CPTs
Q) Write about temporal model with example
• Agents in uncertain environments must be able to keep track of the current state of the
environment, just as logical agents must.
• This is difficult by partial and noisy data, because the environment is uncertain over time.
• At best, the agent will be able to obtain only a probabilistic assessment of the current situation.
• Two section in Temporal Model.
1)Time and Uncertainty
• States and observations
• Stationary processes and the Markov assumption
2)Inference in Temporal Model
Time and Uncertainty:
• A changing world is modeled using a random variable for each aspect of the environment state, at
each point in time.
• The relations among these variables describe how the state evolves.
Example: Treating a Diabetic Patient.
• We have evidence, such as, recent insulin doses, food intake, blood sugar measurements, and other
physical signs.
• The task is to assess the current state of the patient, including the actual blood sugar level and
insulin level.
• Give this information, the doctor (or patient) makes a decision about the patient’s food intake and
insulin dose.
• The dynamic aspects of the problem are essential.
• Blood sugar levels and measurements thereof can change rapidly over time, depending on one’s
recent food intake and insulin doses, one’s metabolic activity, the time of day, and so on.
• To assess the current state from the history of evidence and to predict the outcomes of treatment
actions, we must model these changes.
Inference tasks in Temporal Models:
• Basic inference tasks
▫ Filtering
▫ Prediction
▫ Smoothing
▫ Most likely explanation
Filtering: -
1) maintain a current state estimate and update it.
2)no need to go back over the entire history of percepts for each update.
Example:
In the umbrella example, it might mean computing the probability that it rained last Wednesday,
given all the observations of the umbrella carrier made up to today. Hindsight provides a better
estimate of the state than was available at the time, because it incorporates more evidence
Prediction: -
1) To compute a future belief state, given current evidence (it’s like filtering without all
evidence).
Example:
In the umbrella example, it might mean computing the probability that it rained last Wednesday,
given all the observations of the umbrella carrier made up to today. Hindsight provides a better
estimate of the state than was available at the time, because it incorporates more evidence
Smoothing: -
1) Smoothing is the process of computing the distribution over past states given evidence up to
the present that is
Example: In the umbrella example, it might mean computing the probability that it rained last
Wednesday, given all the observations of the umbrella carrier made up to today. Hindsight
provides a better estimate of the state than was available at the time, because it incorporates more
evidence
Most likely explanation: -
Given a sequence of observations, we might wish to find the sequence of states that is most
likely to have generated those observations. That is, we wish to compute argmax..., P(X1:t |e1:t)
For example, if the umbrella appears on each of the first three days and is absent on the fourth,
then the most likely explanation is that it rained on the first three days and did not rain on the
fourth. Algorithms for this task are useful in many applications, including speech
recognition-where the aim is to find the most likely sequence of words, given a series of
sounds--and the reconstruction of bit strings transmitted over a noisy channel.
Markov Decision Process
Reinforcement Learning is a type of Machine Learning. It allows machines and software agents
to automatically determine the ideal behavior within a specific context, in order to maximize its
performance. Simple reward feedback is required for the agent to learn its behavior; this is
known as the reinforcement signal.
There are many different algorithms that tackle this issue. As a matter of fact, Reinforcement
Learning is defined by a specific type of problem, and all its solutions are classed as
Reinforcement Learning algorithms. In the problem, an agent is supposed to decide the best
action to select based on his current state. When this step is repeated, the problem is known as
a Markov Decision Process.
A Markov Decision Process (MDP) model contains:
● A set of possible world states S.
● A set of Models.
● A set of possible actions A.
● A real valued reward function R(s,a).
● A policy the solution of Markov Decision Process.

What is a State?
A State is a set of tokens that represent every state that the agent can be in

What is a Model?
A Model (sometimes called Transition Model) gives an action’s effect in a state. In particular,
T(S, a, S’) defines a transition T where being in state S and taking an action ‘a’ takes us to state
S’ (S and S’ may be same). For stochastic actions (noisy, non-deterministic) we also define a
probability P(S’|S,a) which represents the probability of reaching a state S’ if action ‘a’ is taken
in state S. Note Markov property states that the effects of an action taken in a state depend only
on that state and not on the prior history.

What is Actions?

An Action A is set of all possible actions. A(s) defines the set of actions that can be taken being
in state S.

What is a Reward?

A Reward is a real-valued reward function. R(s) indicates the reward for simply being in the
state S. R(S,a) indicates the reward for being in a state S and taking an action ‘a’. R(S,a,S’)
indicates the reward for being in a state S, taking an action ‘a’ and ending up in a state S’.

What is a Policy?

A Policy is a solution to the Markov Decision Process. A policy is a mapping from S to a. It

indicates the action ‘a’ to be taken while in state S.

MDP formulation
Multi-period decision problem is formulated as a Markov decision process (MDP) with supply
and demand uncertainty. Decision policies are obtained from solving the MDP problem through
exact value iteration, as well as approximate approaches intended to overcome the 'curses of
dimensionality.' We compare the results from applying them with those from the popular (s, S)
policy fo
Markov Decision Process:
A dynamical system specified in terms of a transition model (ie., a model which gives, for each
state s and action a, the distribution of states that will result if a was executed in s), and a reward
function is called a Markov Decision Process.

The objective in MDP is to find the optimal policy for the dynamicalsystems.

An MDP requires, as input:

1) a set of states,
2) a set of actions, and
3) a reward function.
The reward function tells us what reward to expect when in a given state.

The actions are represented in terms of transition probabilities from one state to another state.
For example, when in state s1, action a1 might have probability 0.9 of taking you to state s2.

Example:

how to solve a Markov decision problem through a series of examples, all using the same
environment. Introducing grid world:

+-------+-------+-------+
| | | |
2| | | +1 |
| | | |
+-------+-------+-------+
| | | |
1| | | -1 |
| | | |
+-------+-------+-------+
| | | |
0| | | |
| | | |
+-------+-------+-------+
0 1 2

Each square is a state. The squares with +1 and -1 are termination states. You want to reach the
+1 state and stay away from the -1 state. So, somehow, you need to know what to do in each of
the others states.

In this environment, you have actions: up, down, left, right, where each action moves the agent
one position in the specified direction. However, we want to learn about handling uncertainty,
so let's make the outcomes non-deterministic. Lets say that there is an 80% chance that the
operator will move the agent in the specified direction, and a 20% chance that the operator will
move the agent at a right angle to the specified direction. For instance, if the agent executes the
action up, then the agent has an 80% chance of moving up, a 10% change of moving left, and a
10% chance of moving right. These probabilities describe a
transition model.

In a problem like this, the utility of a state really depends on where you can get from that state.
Therefore, the utility of the state depends on the utilities of a sequence of states (an environment
history).
A policy is a complete mapping from states to actions. You can calculate a policy from a
transition model (the probabilities) and a utility function. Once you have a policy, then the
agent simply does what the policy says for a given state.

In an accessible environment, the agent's observations at each step will identify the current state.

In an inaccessible environment, the agent's observations do not provide enough information to

identify the current state or the associated transition probabilities.

As stated above, a Markov decision problem (MDP) is the problem of calculating an optimal
policy in an accessible, stochastic environment with a known transition model.

The Markov property is said to hold if the transition probabilities from any given state depend
only on the state and not on previous history.

A partially observable Markov decision problem (POMDP) is the problem of calculating an

optimal policy in an inaccessible environment. These problems generally don't use the same
kinds of solutions as the MDPs, and this paper will not cover them.

We are interested in MDPs, so the environment is accessible and the transition model is known.

Given an MDP and a reward function, we can calculate an optimal policy. Specificaly, an
optimal policy directs the agent towards the next possible state that has the highest promise i.e.,
will lead to state sequnces whose expected value will be the highest. The promise
of a state is called the utility of the state. To understand the utility of a state, and how it fits in to
an MDP, we need a little more background.