Unit 2

Search Techniques
 AI Techniques
1. Search Methods
• It is a way of solving the problem by selecting a best solution by employing a
direct or indirect approach.
• Two ways namely Brute Force Search and Heuristics Search techniques are
used to solve problems.
2. Knowledge Representation
• In order to solve problems in AI knowledge plays a very important role.
• Solving AI problems requires a lot of knowledge of context, language,
visuals, particular domain or knowledge of how to reason.
• Hence, it is required that knowledge must be represented efficiently and
meaningfully.
3. Abstraction
• It is a way of separating important features and variations from unimportant
ones.
2
 Contd…
• Be it ‘Deep Blue’ defeating the legendary Gary Kasparov in Chess in 1997
or ‘Alpha Go’ defeating Lee Sudol in 2016, the potential of AI to mimic and
surpass human mental capabilities has exponentially increased over time.
• https://www.professional-ai.com/deep-blue-algorithm.html
• https://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov
• Search algorithms form the core of such Artificial Intelligence
programs. And while we may be inclined to think that this has limited
applicability only in areas of gaming and puzzle-solving, such algorithms are
used in many more AI areas like route and cost optimizations, action
planning, knowledge mining, robotics, autonomous driving, computational
biology, software and hardware verification, theorem proving etc.

3
 Search Techniques

4
 Breath First Search (BFS)
• Uses QUEUE as data structure
• Breadth-First Search algorithm is a graph traversing technique, where you
select a random initial node (source or root node) and start traversing the
graph layer wise in such a way that all the nodes and their respective
children nodes are visited and explored.
• Crawlers in search engine, GPS Navigation, Broadcasting in networking are
applications of BFS
• Time and Space Complexity = b^d ; b= branching factor and d = depth
Algorithm
• Step 1: Take an Empty Queue.
• Step 2: Select a starting node (visiting a node) and insert it into the Queue.
Step 3: Provided that the Queue is not empty, extract the node from the
Queue and insert its child nodes (exploring a node) into the Queue.
• Step 4: Print the extracted node.
5
 Example

6
 Example

Output : S ABCDGHEFIK
7
 Depth First Search(DFS)
• Depth-First Search or DFS algorithm is a recursive algorithm that uses
the backtracking principle.
• It entails conducting exhaustive searches of all nodes by moving forward if
possible and backtracking, if necessary.
• To visit the next node, pop the top node from the stack and push all of
its nearby nodes into a stack.
• Topological sorting, scheduling problems, graph cycle detection, and solving
puzzles with just one solution, such as a maze or a Sudoku puzzle, all
employ depth-first search algorithms.
• Time Complexity: O(b^d)
• Space Complexity: O(bd)

8
 Depth First Search (DFS)
• Step 1: Create a stack with the total number of vertices in the graph as the
size.
• Step 2: Choose any vertex as the traversal's beginning point. Push a visit to
that vertex and add it to the stack.
• Step 3 - Push any non-visited adjacent vertices of a vertex at the top of the
stack to the top of the stack.
• Step 4 - Repeat steps 3 and 4 until there are no more vertices to visit from
the vertex at the top of the stack.
• Step 5 - If there are no new vertices to visit, go back and pop one from the
stack using backtracking.
• Step 6 - Continue using steps 3, 4, and 5 until the stack is empty.
• Step 7 - When the stack is entirely unoccupied, create the final spanning
tree by deleting the graph's unused edges.

9
 Example

10
 Example

11
 Iterative Deepening Search(IDS)

• The iterative deepening algorithm is a combination of DFS and BFS

algorithms.
• This search algorithm finds out the best depth limit and does it by
gradually increasing the limit until a goal is found.
• This algorithm performs depth-first search up to a certain "depth limit", and it
keeps increasing the depth limit after each iteration until the goal node is
found.
• The iterative search algorithm is useful uninformed search when search
space is large, and depth of goal node is unknown.
• Iterative Deepening = BFS (fast search) +DFS (memory-efficiency)
• The main drawback of IDDFS is that it repeats all the work of the previous
phase.

12
 Algorithm
bool IDDFS(src, target, max_depth)
for limit from 0 to max_depth
if DLS(src, target, limit) == true
return true;
return false;
bool DLS(src, target, limit)
if (src == target)
return true;
if (limit <= 0)
return false;
for each adjacent i of src
if DLS(i, target, limit?1)
return true ;
return false;
13
 Example
• There can be two cases-
a) When the graph has no cycle
• This case is simple. We can DFS multiple times with different height limits.
b) When the graph has cycles
• This is interesting as there is no visited flag in IDDFS.

14
 Contd…
• Following tree structure is showing the iterative deepening depth-first
search.
• IDS algorithm performs various iterations until it does not find the goal node
• 1'st Iteration A
2'nd Iteration A, B, C
3'rd Iteration A, B, D, E, C, F, G
4'th Iteration A,B, D, H, I, E, C, F, K,G
• In the third iteration,
the algorithm will find the goal node.

15
 Contd…
Completeness:
• This algorithm is complete if the branching factor is finite.
Time Complexity:
• Let's suppose b is the branching factor and depth is d then the worst-case
time complexity is O(bd).
Space Complexity:
• The space complexity of IDDFS will be O(bd).
Optimal:
• IDS algorithm is optimal if path cost is a non- decreasing function of the
depth of the node.

16
 Heuristic Function
• Heuristic is a function which is used in Informed Search, and it finds the
most promising path.
• It takes the current state of the agent as its input and produces the
estimation of how close agent is from the goal.
• The heuristic method, however, might not always give the best
solution, but it guaranteed to find a good solution in reasonable time.
• Heuristic function estimates how close a state is to the goal.
• It is represented by h(n), and it calculates the cost of an optimal path
between the pair of states.
• The value of the heuristic function is always positive.
• Euclidean Distance

• Manhattan Distance = #of tiles on right position in 8 puzzle

17
 Contd…

18
Contd..

19
 Best First Search
• An informed search, like BFS would use an evaluation function to decide
which among the various available nodes is the most promising (or ‘BEST’)
before traversing to that node.
• BFS uses the concept of a Priority queue and heuristic search.
• To search the graph space, the BFS method uses two lists for tracking the
traversal.
• An ‘Open’ list that keeps track of the current ‘immediate’ nodes available for
traversal and a ‘CLOSED’ list that keeps track of the nodes already
traversed.
• BFS is called Greedy Search Algorithm too.

20
 Contd…
• Start with OPEN containing just initial node
• Until a goal is found or there are no nodes left on OPEN do
– Pick the best node on OPEN
– Generate its successors
– For each successor do:
• If it has not been generated before, evaluate it, add it to OEPN, record its parents
• If it has been already generated before, change the parent if this new path is better
than previous one. Update the cost of getting to this node

– Space and Time complexity: O(b^d)

21
 Contd…
• The two variants of BFS are Greedy Best First Search and A* Best First Search
• For Greedy BFS the evaluation function is f(n) = h(n)

• For A* the evaluation function is f(n) = g(n) + h(n)

• Distance is an approximation of how close we are to the goal from a given node and
is denoted by the heuristic function h(n)
• The sum of the distance from the start node to each of these immediate next nodes
is denoted by the function g(n).
• At any point, the decision on which node to go to next is governed by our evaluation
function f(n)

22
 Example 1

OPEN CLOSE

C NULL
TOEBP C
OEBP CT
IEBPN CTO
ZEBPN CTOI
EBPN CTOIZ

Final Path
C – O – I -- Z

23
 Example 2

OPEN CLOSE

S NULL
B A S
F EA SB
GIEA SBF
I EA SBFG

24
 Example 3

25
 8 Puzzle Problem

26
 A* Algorithm
• It combines the strength of both a greedy search and a uniform cost search.
• The heuristic of A* search is calculated as the summation of heuristic cost in
greedy search and uniform cost search.
f(x) = h(x) + g(x)
• h(x) is the forward cost of a node from current state to goal state.
• g(x) is the backward cost of a node from the initial state.
• f(x) is fitness number
• The node with smallest f(x) is selected.
• The time and space complexity of A* search algorithm is O(b^d)

27
 Example 1
Initial Node A to Goal Node J

28
 Contd…

29
 Contd…

30
 Example 2 – Cost Effective

Initial node A to Goal node G

31
 Contd….

32
 Example 3

Initial State S to Goal G

33
 AO* Algorithm
• AO* algorithm is a best first search algorithm.
• AO* algorithm uses the concept of AND-OR graphs to decompose any
complex problem given into smaller set of problems which are further solved.
• AND-OR graphs are specialized graphs that are used in problems that can
be broken down into sub problems where AND side of the graph represent a
set of task that need to be done to achieve the main goal , whereas the OR
side of the graph represent the different ways of performing task to achieve
the same main goal.
f(n) = g(n) + h(n)
where,
• g(n): The actual cost of traversal from initial state to the current state.
• h(n): The estimated cost of traversal from the current state to the goal state.
• f(n): The actual cost of traversal from the initial state to the goal state.

34
 Example

f(A-B) = g(B) + h(B) = 1+4= 5

f(A-C-D) = g(C) + h(C) + g(D) + h(D) = 1+2+1+3 = 7

The minimum cost path is chosen i.e A-B.

f(B-E) = 1 + 6 = 7

f(B-F) = 1 + 8 = 9

f(A-B) = g(B) + updated((h(B)) = 1+7=8

35
 Contd
f(C-G) = 1+2 = 3

f(C-H-I) = 1+0+1+0 = 2

f(D-J) = 1+0 = 1

f(A-C-D) = g(C) + h(C) + g(D) +

updated((h(D)) = 1+2+1+1 =5

36
 Example 2
F(A-D)= 1+10 = 11
F(A-BC) = 1 + 1 + 6 +12 = 20

F(A-D-FE) = 1+1+ 4 +4 =10

F(B-G)= 5+1 =6
F(B-H)= 7 + 1 = 8

F(G-I) = 1 +1 = 2
new cost of F(B-G)= 1+2 =3

F(C-J) = 1+1= 2

New Cost F(A-BC) = 1+1+2+3 = 7

New Cost F(A-D)=11

37
 Hill Climbing
• local search algorithm which continuously moves in the direction of
increasing elevation/value to find the peak of the mountain or best solution to
the problem.
• It terminates when it reaches a peak value where no neighbor has a higher
value.
• It is a technique which is used for optimizing the mathematical problems
• A node of hill climbing algorithm has two components which are state and
value.
• Hill Climbing is mostly used when a good heuristic is available.
• Local search, Greedy approach and No backtracking are features of it

38
 Contd..

• Graph between various states of

algorithm and Objective function/Cost.
• On Y-axis we have taken the function
which can be an objective function or
cost function, and state-space on the
x-axis.
If the function on Y-axis is cost then,
the goal of search is to find the global
minimum and local minimum.
If the function of Y-axis is Objective
function, then the goal of the search
is to find the global maximum and
local maximum.

39
 Contd…
• Local maximum is a state which is better than its neighbor states, but there
is also another state which is higher than it.
• Global maximum is the best possible state of state space landscape. It has
the highest value of objective function
• Current state is a state in a landscape diagram where an agent is currently
present.
• Flat local maximum is a flat space in the landscape where all the neighbor
states of current states have the same value.
• Shoulder is a plateau region which has an uphill edge.

40
 Problems in Hill Climbing
• A local maximum is a peak state in the landscape which is better than each
of its neighboring states, but there is another state also present which is
higher than the local maximum.

• Backtracking technique can be a solution of the local maximum

41
 Contd….
• A plateau is the flat area of the search space in which all the neighbor
states of the current state contains the same value, because of this
algorithm does not find any best direction to move.
• A hill-climbing search might be lost in the plateau area.
• The solution for the plateau is to take big steps or very little steps while
searching, to solve the problem

42
 Contd…
• A ridge is a special form of the local maximum.
• It has an area which is higher than its surrounding areas, but itself has a
slope, and cannot be reached in a single move.
• Solution: With the use of bidirectional search, or by moving in different
directions, we can improve this problem.

43
 Example
h(x) = +1 for all the blocks in the support structure if the block is correctly
positioned otherwise -1 for all the blocks in the support structure.

44
 Means End Analysis (MEA)
• It starts with a focus on your goal, then helps you see the steps needed to
get there.
• Means end analysis problem solving is equally useful for project tracking
and management.
• This algorithm deals with an initial state and end state and the action plan
and movement in forward and backward directions.
• This recursive algorithm optimizes the search operation and achieves the
end goal with minimum effort..
• Goals are split into sub goals, and each sub goal is further split into action
steps.
• Breaking up goals into actionable tasks helps management to reach the end
state successfully.

45
 How it works?

Figure: Working of MEA

46
Eg. Want to make career as Web Developer
 Algorithm
• Step 1: Compare CURRENT and GOAL, and if there are no discrepancies,
return Success and Exit.
• Step 2: If not, choose the most significant difference and follow the steps
below until success or failure is achieved.
• Choose a new operator O that applies to the current difference, and if no
such operator exists, signal failure.
• Try to apply operator O to CURRENT. Make description of two states.
– O-Start, a state in which O’s preconditions are satisfied.
– O-Result, the state that would result if O were applied In O-start.
• If
(First-Part <------ MEA (CURRENT, O-START) and
(LAST-Part <----- MEA (O-Result, GOAL), are successful,
• then signal Success and return the result of combination of FIRST-
PART, O, and LAST-PART.
47
 Example 1

Figure : Current State

Figure: Solution using MEA

48
 Constraint Satisfaction Problem(CSP)

• Constraint satisfaction is a technique where a problem is solved when its

values satisfy certain constraints or rules of the problem.
• Constraint satisfaction depends on three components, namely:
• A finite set of variables which stores the solution (V = {V1, V2, V3,....., Vn})
• A set of discrete values known as domain from which the solution is picked
(D = {D1, D2, D3,.....,Dn})
• A finite set of constraints (C = {C1, C2, C3,......, Cn}).
• The constraint value consists of a pair of {scope, rel}.
• The scope is a set of variables which participate in the constraint and rel is
a relation which includes a list of values which the variables can take to
satisfy the constraints of the problem.
• Consider the Sudoku problem, Suppose that a row, column and block
already have 3, 5 and 7 filled in. Then the domain for all the variables in that
row, column and block will be {1, 2, 4, 6, 8, 9}.
49
 Contd…
• Crypt Arithmetic (Coding alphabets to numbers.)
• N-Queen (In an n-queen problem, n queens should be placed in an nXn
matrix such that no queen shares the same row, column or diagonal.)
• Map Coloring (coloring different regions of map, ensuring no adjacent
regions have the same color)
• Crossword (everyday puzzles appearing in newspapers)
• Sudoku (a number grid)
• Latin Square Problem
• Zebra Puzzle

50
 Example – Crypt Arithmetic
• The rules or constraints on a crypt arithmetic problem are as follows:

1. There should be a unique digit to be replaced with a unique alphabet.

2. The result should satisfy the predefined arithmetic rules, i.e., 2+2 =4, nothing
else.
3. Digits should be from 0-9 only.
4. There should be only one carry forward, while performing the addition
operation on a problem.
5. The problem can be solved from both sides, i.e., left hand side (L.H.S), or
right hand side (R.H.S)

51
 Contd…
• Given a cryptarithmetic problem, i.e., S E N D + M O R E = M O N E Y

52
 Example 2
• Given a cryptarithmetic problem, i.e., B A S E + B A L L = G A M E S

53
 Example 3
• Given a cryptarithmetic problem, i.e., S O M E + T I M E = S P E N T

54
 Example 4
• Given a cryptarithmetic problem, i.e.,
1. CROSS+ROADS=DANGER
2. EAT+THAT=APPLE
3. BLACK+GREEN=ORANGE
4. CRASH+HACKER=REBOOT

55
 Mini-Max Algorithm

• Mini-max algorithm is a recursive or backtracking algorithm which is

used in decision-making and game theory.
• It provides an optimal move for the player assuming that opponent is also
playing optimally.
• In this algorithm two players play the game, one is called MAX and other is
called MIN
• Both Players of the game are opponent of each other, where MAX will select
the maximized value and MIN will select the minimized value.
• It performs a depth-first search algorithm for the exploration of the
complete game tree.
• It proceeds all the way down to the terminal node of the tree, then backtrack
the tree as the recursion.

56
 Pseudo-code
function minimax(node, depth, maximizingPlayer)
if depth ==0 or node is a terminal node then
return static evaluation of node

if MaximizingPlayer then // for Maximizer Player

maxEva= -infinity
for each child of node do
eva= minimax(child, depth-1, false)
maxEva= max(maxEva,eva) //gives Maximum of the values
return maxEva

else // for Minimizer player

minEva= +infinity
for each child of node do
eva= minimax(child, depth-1, true)
minEva= min(minEva, eva) //gives minimum of the values
return minEva

57
 Working
Step 1:
• The algorithm generates the entire game-tree and apply the utility function to
get the utility values for the terminal states. Suppose maximizer takes first
turn which has worst-case initial value =- infinity, and minimizer will take next
turn which has worst-case initial value = +infinity.

58
 Contd…..
Step 2:
• first we find the utilities value for the Maximizer, its initial value is -∞, so we
will compare each value in terminal state with initial value of Maximizer and
determines the higher nodes values. It will find the maximum among the all.

• For node D max(-1, -∞) => max(-1,4)= 4

• For Node E max(2, -∞) => max(2, 6)= 6
• For Node F max(-3, -∞) => max(-3,-5) = -3
• For node G max(0, -∞) = max(0, 7) = 7

59
 Contd….
Step 3:
• it's a turn for minimizer, so it will compare all nodes value with +∞, and will
find the 3rd layer node values.
• For node B= min(4,6) = 4
• For node C= min (-3, 7) = -3

60
 Contd…
Step 4:
• it's a turn for Maximizer, and it will again choose the maximum of all nodes
value and find the maximum value for the root node.
• For node A max(4, -3)= 4

61
 Properties
• Complete- Min-Max algorithm is Complete. It will definitely find a solution (if
exist), in the finite search tree.
• Optimal- Min-Max algorithm is optimal if both opponents are playing
optimally.
• Time complexity- As it performs DFS for the game-tree, so the time
complexity of Min-Max algorithm is O(bm), where b is branching factor of the
game-tree, and m is the maximum depth of the tree.
• Space Complexity- Space complexity of Mini-max algorithm is also similar
to DFS which is O(bm).
Limitation
• The main drawback of the minimax algorithm is that it gets really slow for
complex games such as Chess, go, etc.
• This type of games has a huge branching factor, and the player has lots of
choices to decide.

62
 Alpha Beta Pruning
• Alpha-beta pruning is a modified version of the minimax algorithm.
• It is an optimization technique for the minimax algorithm.
• As we have seen in the minimax search algorithm that the number of game
states it has to examine are exponential in depth of the tree.
• Since we cannot eliminate the exponent, but we can cut it to half.
• Hence there is a technique by which without checking each node of the
game tree we can compute the correct minimax decision, and this technique
is called pruning.
• This involves two threshold parameter Alpha and Beta for future expansion,
so it is called alpha-beta pruning or as Alpha-Beta Algorithm.
• Alpha-beta pruning can be applied at any depth of a tree, and sometimes it
not only prune the tree leaves but also entire sub-tree.

63
 Contd….
• The two-parameter can be defined as:
– Alpha: The best (highest-value) choice we have found so far at any point
along the path of Maximizer. The initial value of alpha is -∞.
– Beta: The best (lowest-value) choice we have found so far at any point
along the path of Minimizer. The initial value of beta is +∞.
• The Alpha-beta pruning returns the same move as the standard algorithm
does, but it removes all the nodes which are not really affecting the final
decision but making algorithm slow.
• Hence by pruning these nodes, it makes the algorithm fast.

64
 Contd….
• Condition for Alpha-beta pruning: α>=β
• Key points about alpha-beta pruning:
• The Max player will only update the value of alpha.
• The Min player will only update the value of beta.
• While backtracking the tree, the node values will be passed to upper nodes
instead of values of alpha and beta.
• We will only pass the alpha, beta values to the child nodes.

65
 Pseudo-Code
function minimax(node, depth, alpha, beta, maximizingPlayer) is
if depth ==0 or node is a terminal node then
return static evaluation of node
if MaximizingPlayer then // for Maximizer Player
maxEva= -infinity
for each child of node do
eva= minimax(child, depth-1, alpha, beta, False)
maxEva= max(maxEva, eva)
alpha= max(alpha, maxEva)
if beta<=alpha
break
return maxEva
else // for Minimizer player
minEva= +infinity
for each child of node do
eva= minimax(child, depth-1, alpha, beta, true)
minEva= min(minEva, eva)
beta= min(beta, eva)
if beta<=alpha break ; return minEva

66
 Working
Step 1:
• Max player will start first move from node A where α= -∞ and β= +∞, these
value of alpha and beta passed down to node B where again α= -∞ and β=
+∞, and Node B passes the same value to its child D

67
 Contd….
Step 2:
• At Node D, the value of α will be
calculated as its turn for Max.
• The value of α is compared with
firstly 2 and then 3, and the max (2,
3) = 3 will be the value of α at node
D and node value will also 3.
Step 3:
• Now algorithm backtrack to node B,
where the value of β will change as
this is a turn of Min, Now β= +∞, will
compare with the available
subsequent nodes value, i.e. min (∞,
3) = 3,
• hence at node B now α= -∞, and β=
3.

68
 Contd….
Step 4:
• At node E, Max will take its turn, and the value of alpha will change. The
current value of alpha will be compared with 5, so max (-∞, 5) = 5, hence at
node E α= 5 and β= 3, where α>=β, so the right successor of E will be
pruned, and algorithm will not traverse it, and the value at node E will be 5

69
 Contd….
Step 5:
• Algorithm again backtrack the tree, from
node B to node A.
• At node A, the value of alpha will be
changed the maximum available value is
3 as max (-∞, 3)= 3, and β= +∞, these
two values now passes to right
successor of A which is Node C.
• At node C, α=3 and β= +∞, and the same
values will be passed on to node F.
Step 6:
• At node F, again the value of α will be
compared with left child which is 0, and
max(3,0)= 3, and then compared with
right child which is 1, and max(3,1)= 3
still α remains 3, but the node value of F
will become 1.

70
 Contd….
Step 7:
• Node F returns the node value 1
to node C, at C α= 3 and β= +∞,
here the value of β will be
changed, it will compare with 1.
min (∞, 1) = 1.
• Now at C, α=3 and β= 1, and
again it satisfies the condition
α>=β, so the next child of C
which is G will be pruned, and
the algorithm will not compute
the entire sub-tree G.

71
 Contd….
Step 8:
• C now returns the value of 1 to A.
• Here the best value for A is max (3, 1) = 3. Hence the optimal value for the
maximizer is 3 for this example.

72
 Move Ordering in Alpha-Beta Pruning

• The effectiveness of alpha-beta pruning is highly dependent on the order in

which each node is examined.
• Move order is an important aspect of alpha-beta pruning.
1. Worst ordering
• In some cases, alpha-beta pruning algorithm does not prune any of the
leaves of the tree, and works exactly as minimax algorithm.
• It also consumes more time because of alpha-beta factors, such a move of
pruning is called worst ordering.
• The best move occurs on the right side of the tree. The time complexity for
such an order is O(bm).

73
 Contd…
2. Ideal ordering
• The ideal ordering for alpha-beta pruning occurs when lots of pruning
happens in the tree, and best moves occur at the left side of the tree.
• We apply DFS hence it first search left of the tree and go deep twice as
minimax algorithm in the same amount of time.
• Complexity in ideal ordering is O(bm/2).

74
75
76
77