Artificial Intelligence

1
 OBJECTIVES
• To have a basic proficiency in a traditional AI language including
ability to write simple to moderate programs and to understand
code written in that language
• To improve analytical and problem solving skills based on the
characteristics of the problem using various heuristic search
techniques and to improve designing and playing a game
• To have knowledge on propositional calculus, proportional and
predicate logic to understand few systems such as natural
deduction, axiomatic system, etc.

2
Contd..
• To have an understanding of the basic issues of knowledge
representation and blind and heuristic search, as well as an
understanding of other topics such as minimax, resolution, etc. that
play an important role in AI programs.
• To have a basic understanding of some of the more advanced topics
of AI such as learning, natural language processing, agents and
robotics, expert systems, and planning
• To have basic knowledge on probabilistic analysis and networks as
well as fuzzy systems and fuzzy logics.

3
 OUTCOMES:
• Identify problems that are amenable to solution by AI methods

• And which AI methods may be suited to solve a given problem

• Formalize a given problem in the language/ framework of different

AI methods
• Implement basic AI algorithms

• Design and carry out an empirical evaluation of different algorithms

on problem formalization, and state the conclusions that the
evaluation supports

4
 TEXT BOOKS:
• Artificial Intelligence- Saroj Kaushik, CENGAGE Learning
• Artificial intelligence, A modern Approach , 2nded,
Stuart Russel, Peter Norvig, PEA
• Artificial Intelligence- Rich, Kevin Knight, Shiv Shankar B
Nair, 3rded, TMH
• Introduction to Artificial Intelligence, Patterson, PHI

5
 REFERNCE BOOKS
• Artificial Intelligence, structures and Strategies for
Complex problem solving, - George F Lugar, 5th
edition, PEA
• Introduction to Artificial Intelligence, Ertel, Wolf
Gang, Springer
• Artificial Intelligence, A new Synthesis, Nils J
Nilsson, Elsevier
6
 Why Artificial Intelligence?
With the help of AI, you can create such software or devices which can solve
real-world problems very easily and with accuracy such as health issues,
marketing, traffic issues, etc.

With the help of AI, you can create your personal virtual Assistant, such as
Cortana, Google Assistant, Siri, etc.

With the help of AI, you can build such Robots which can work in an
environment where survival of humans can be at risk.

AI opens a path for other new technologies, new devices, and new
Opportunities.

7
 UNIT - I
• Introduction to artificial intelligence:
– Introduction - 1
– History - 2
– Intelligent systems - 3
– Foundations of AI - 7
– Applications - 9
– Tic-tac-toe game playing - 9
– Development of AI languages - 16
– Current trends in AI - 16
8
 Introduction
• Foundation of AI was laid with Boolean theory by a
mathematician, Boole & other researchers
• Since the invention of computer in 1943, AI has been of
interest to the researchers
• They always aimed to make machines more intelligent than
humans
• AI has grown substantially for last six decades starting from
simple to intelligent programs

9
• AI comprises of a numerous subfields ranging from general purpose
areas to specific tasks
• General purpose areas: perception, logical reasoning, et.
• Specific tasks: game playing, theorem proving, diagnosing diseases,
etc.
• Scientists of other fields use AI to systematize and automate the
intellectual tasks
• AI is engaged in two different significant fields:
– Science of human intelligence
– Engineering discipline

10
11
12
13
14
15
16
17
18
19
 Goals of Artificial Intelligence
• Replicate human intelligence
• Solve Knowledge-intensive tasks
• An intelligent connection of perception and action
• Building a machine which can perform tasks that requires
human intelligence such as:
• Proving a theorem
• Playing chess
• Plan some surgical operation
• Driving a car in traffic
• Creating some system which can exhibit intelligent behaviour,
learn new things by itself, demonstrate, explain, and can
advise to its user.

20
 Advantages
High Accuracy with less errors
High-Speed
Useful for risky areas
Digital Assistant
Useful as a public utility.

21
 Disadvantages

High Cost
Can't think out of the box
No feelings and emotions
Increase dependency on machines
No Original Creativity

22
 History
 In 400 B.C, philosophers said that, mind operates on knowledge
encoded in some internal language

 Psychologists strengthened the idea that, living creatures are

information processing machines

 Mathematicians provided tools to manipulate certain or uncertain

logical statements of certainty as well as probabilistic statements

23
• John McCarthy organized a conference on machine intelligence in
1956, since then the field is known as AI
• In1957, a program named GPS (General Problem Solver)
was developed and tested on problems requiring common sense
• John McCarthy announced his new development i.e., LISP
(LISt Processing language) in 1958
• MarvinMinsky of MIT demonstrated that computer programs could
solve spatial and logic problems
• 1960, another program named as STUDENT was developed to solve
algebraic problems

24
• L.Zadeh has developed Fuzzy set and logic to make decisions
under uncertain conditions
• Terry Winograd at MIT, developed SHRDLU, a program which
carries a simple dialogue with a user in English. This is written
in MACLISP
• Minsky also developed Frame Theory around 1970 used for
storing structured programs to be used by AI programs
• R.Kowalski developed PROLOG language around 1970. (Expert
Systems are also developed in the same year)
25
26
 Intelligent systems
• AI is combination of Computer Science, Physiology &
Philosophy
• AI is a broad area from Machine Vision to Expert
Systems
• John Mc Carthy defines intelligence as the computational
part of the ability to achieve goals in the world
• Different people think of AI differently & there is no unique
definition
• AI is the study of making machines which do things
intelligently 27
• Hence, AI programs must have capability and
characteristics of intelligence such as
– Learning – ex. Trail and method,
– Reasoning - a way to infer facts from existing data
– Problem solving -  finding winning moves in board
games; identifying people from their photographs
– Inferencing - the process of using a trained neural
network model to make a prediction.
– perceiving - the process of interpreting vision, sounds,
smell, and touch
– comprehending information – reading passage and
getting answers

• AI requires an understanding of related terms such as

intelligence, knowledge, reasoning, cognition, learning, and 28
• AI programs:
– are not perfect and even make mistakes like humans

– also can be treated as non-numeric (common sense)

ways of solving problems
– need not be perfect but gives reasonably good
solutions

• There are two views of AI goals

– Duplicating what brain does (cognitive science)

– Duplicating what brain should do (logical thinking)

29
Eliza
• The very first intelligent system by Joseph Weizenbaum
developed in 1964 - 1966
• A program that conversed with user in English
• Able to converse about any subject, since it stored the
subjects information in data banks
• It was able to pick up speech patterns from user’s
questions and provide responses
• The main characteristics of Eliza are briefly mentioned
as:
30
31
32
• Simulation of intelligence
– Eliza programs are not intelligent since they
cannot understand the meaning
– They understand the keywords and phrases

• Quality of response
– It is limited by the sophistication of the ways in
which they can process the input
– Ex. The no.of templates available is a limitation
33
• Coherence
– Earlier versions had no structure on the conversation
– Every statement was based on entirely on the current
input and no context info was maintained
– More complex versions can do better
– Any intelligence strongly depends on coherence

• Semantics
– Have no semantic representation of content
– It does not have intelligence of understanding
– It imitates the human conversation style
34
 Categorization of Intelligent Systems

• In order to design Intelligent Systems, they are

categorized into four
– Systems that think like humans

– Systems that act like humans

– Systems that think rationally

– Systems that act rationally

35
36
37
• Systems that think like humans
– Requires cognitive modelling approaches
– Should know the functioning of brain and its
mechanism for processing information
– It is an area of cognitive science
– The stimuli are converted into mental
representation
– Cognitive processes manipulate it to build new
representation to generate actions
– Neural Network is a computing model for processing
information similar to brain
38
• Systems that act like humans
– Requires the overall behaviour of the system should be human like

• Systems that think rationally

– Relies on logic to measure correctness

– Logical formulae and theories are used for synthesizing outcomes

– E.g. John is a human, and all humans are mortal, hence the conclusion is John
is mortal
– not that all intelligent behaviours are mediated by logical deliberation

• Systems that act rationally

– By rational behaviour we mean doing the right thing

– Though the method is illogical, the observed behaviour must be rational

39
• To summarize, it can be defined that intelligence is the property of
mind which encompasses capabilities like:
– Reason and draw meaningful conclusions

– Plan sequences of actions to complete a goal

– Solve problems

– Think abstractly

– Comprehend ideas and help computers to communicate in NL

– Store knowledge provided before or during interrogation

– Learn new ideas from environment and new circumstances

– Offer advice based on rules and situations

– Learn new concepts and tasks that require high levels of intelligence

40
 Components of AI Program
• AI Program should have:
– Knowledge base and
– navigational capacity which contains control strategy and
Inference mechanism
• Knowledge base:
– AI programs should be learning in nature and update its
knowledge accordingly
– it consists of facts and rules and characteristics like
voluminous, incomplete, imprecise, dynamic and keep
changing
41
• Control strategy:
– It determines which rule to be applied

– For this heuristics, or related thumb rules are used

• Inference mechanism:
– It requires search through knowledge base

– Derives new knowledge using existing knowledge

with the help of inference rules

42
 Foundations of AI
• Commonly used AI techniques are rule-based, fuzzy logic,
neural networks, decision theory, statistics, probability
theory, genetic algorithms, etc.
• Since AI is interdisciplinary in general, foundations of AI
are
– Mathematics
– Neuroscience
– Control theory
– Linguistics
43
 Applications
• Business: financial strategies, advising
• Engineering: check design, offer suggestions to create new products,
expert systems
• Manufacturing: assembly, inspection, maintenance
• Medicine: monitoring, diagnosing, prescribing
• Education: teaching
• Fraud detection
• Object identification
• Space shuttle scheduling
• Information retrieval
44
Tic-tac-toe game playing
• The objective is to write a program which never
loses
• Three approaches are presented to play the game
which increases
– Complexity
– Use of generalization
– Clarity of their knowledge
– Extensibility of their approach
45
Approach 1
• 3x3 board is represented as nine element vector
• Each element in a vector can contain any of the
following three digits
– 0- represents blank position
– 1 – indicates X player move
– 2 – indicates O player move

• The program makes use of a move table that

consist of vector or 39 (19683) elements
46
47
• Initially the board is empty and represented by nine
zeros (000 000 000)
• The best new board position is 000 010 000
• All possible board positions are stored in ‘Current
Board Position’ column with its corresponding next
best possible board position in ‘New Board
Position’ column
• Once the table is designed, the program has to simply
do the table look up
• The algorithm is as follows

48
1. View the vector as a ternary number

2. Get an index by converting this vector to its

corresponding decimal number

3. Get the vector from new board position stored at

the index. The vector thus selected represents the
way the board will look after the move that should
be made

4. So set board position equal to that vector

49
42
Disadvantages
• Requires lot of space to store move table
• Lot of work is required to create move table
• Move table creation is highly error prone
• Cannot be extended to 3D since 327 board
positions are required
• This program is not intelligent at all since it does
not meet any of AI requirements
50
Approach 2
• The board B[1..9] is represented by a nine element vector
• Here
– 2 – represents blank position
– 3 – indicates X player move
– 5 – indicates O player move

• There are totally 9 moves in game

• They are represented by an integer 1 for first move and 9
for last move
• We use the following 3 sub procedures
51
Human strategy
• If human is winning in the next move, then he
plays in the desired square
• If he is not winning in next move, then he
checks if the opponent is winning
• If so, then he blocks that square

52
• Go(n) – use this function computer can make a move in
square n
• Make_2 – this function helps the computer to make valid
2 moves
• PossWin(p) – if player p can win in the next move, then it
returns the index (from 1 to 9) of the square that constitutes a
winning move
• Otherwise it returns a ‘0’
• The function PossWin operates by checking, one at a time,
for each of rows/ columns/ diagonals

53
• If PossWin(p) = 0, then p cannot win
• Find whether opponent can win
• If so, then block it, this can be achieved as
– If (3*3*2 = 18), then X player can win as there is one blank
square
– If (5*5*2 = 50), then O player wins

• Let us represent computer as C & human as H

• First player is X and second player is O
• The following table consists of rules to be applied by
computer for all nine moves
54
55
Disadvantages

• Though it can be treated as AI approach since

it applies heuristics, it has following
disadvantages:
– Not as efficient as first one w.r.t time

– Several conditions are checked before each move

– Still cannot generalize to 3D

56
Approach 3
• The board is chosen as a magic square of
order 3
• The magic square of order n consists of n2
distinct numbers from 1 to n2
• The numbers of columns, rows and diagonals
sum to the same constant

57
How to create magic square 3x3

• Begin by placing 1 in the middle of first row

• Subsequently place next numbers diagonally up
and to right
• Go down if the work is over
• The only thing to be remembered is - the
matrix is wrapped-around

58
59
• In this approach, a list of blocks played by each player
is maintained
• Each pair of blocks a player owns is considered
• Difference D between 15 and the sum of the blocks is
calculated
• If D<0 or D>9, then those two blocks are not collinear
and so can be ignored
• Else if the block representing difference in blank, i.e.
not in the list, then player can move in that block

60
61
1. Suppose H plays in block 8
2. C plays in block 5
3. H plays in block 1
4. C checks if H can win or not
– Compute sum of blocks played by H
• S=8+1=9
• Compute D=15-9=6
– The sixth block is a winning block for H and not there on either list. So, C blocks 6
5. H plays in block 4
6. C checks if C can win
– Compute sum of blocks played by C
• S=5+6=11
• Compute D=15-11=4; discard this block as it already is in list
– Now C checks whether H can win
• Compute sum of pair of square from list of H which have not used earlier
– S=8+4=12
– Compute D=15-12=3
• Block 3 is free, so C plays in block 3.
7. If H plays in block 2 or 9, then computer wins. Lets assume h plays 2
8. C checks if it can win
– Compute sum of blocks played by C which has not used earlier
• S=5+3=8
• Compute D=15-8=7
– Block 7 is free, so C plays in block 7 and wins the game
9. If H plays in block 7 then there is a draw
62
 Advantages & Disadvantages

• Program requires much time than other two

approaches as it must search all possible moves
• But the advantage is that the approach could be
extended to handle 3D tic-tac-toe
• It could also be extended to handle games more
complicated than tic-tac-toe

63
 3 Dimensional Tic-Tac-Toe

• This is similar to the traditional Tic-Tac-Toe

• But played on a cube of order 3
• Cube is created by stacking 3 grids
• Now the 3 in a row can be on any of the three levels, or between
levels
• To win, a player must place three of his symbols on three squares
that line up vertical, horizontal or diagonal on a single grid or over
all other grids

64
65
• In this game, one may use the magic cube as
shown in fig.
• Numbers from 1 to 27 are arranged in a 3x3x3
pattern
• Sum of the numbers on each row, column,
diagonal is 42, called magic constant of cube
• The magic cube of order n has a magic constant
equal to n[(n3+1)/2]

66
67
 Development of AI languages
• AI languages stress on knowledge representation schemes, pattern
matching, flexible search, and programs on data
• E.g. LISP, Pop-2, ML, Prolog, etc.
• LISP is a functional language based on calculus
• Prolog is a logic lang. based on 1st order predicate logic
• Pop-2 is a stack based language providing greater flexibility and has
similarity with LISP
• Pop 11 is embedded in an AI programming environments which focuses on
domain level
• AI programming can exploit any language from BASIC through C to
Smalltalk
68
 Current trends in AI

• Current trends in AI are basically towards the

development of technologies which have origin with
biological or behavioral phenomena related to human
or animals such as evolutionary computation
• Evolutionary computation uses iterative progress, such
as growth or development
• It is inspired by biological mechanism or evolution

69
• Evolutionary paths of AI and simulation started to converge
in cognitive psychology
• Simulation has been developed to study and understand
complex time varying behaviours exhibited by real physical
systems
• ANN has been developed based on functioning of human
brain to predict features based on previous details
• Evolutionary techniques mostly involve meta-heuristic
optimization algorithms such as evolutionary algorithms and
swarm intelligence

70
• Genetic algorithms based on Darwin theory of evolution
were developed mainly by emulating the nature and
behaviour of biological chromosome
• Swarm intelligence is based on the collective behaviour
of decentralized, self-organized systems
• Ants, bees and termites etc. solve complex problems by
mutual cooperation
• This emergent behaviour of self-organization by a group
of social insects is known as swarm intelligence

71
• In computational sense, the set of mobile agents
which are liable to communicate directly or
indirectly with each other and which collectively
carry out a distributed problem solving is known as
swarm intelligence
• Expert systems continues to remain an attractive
field for its practical utility in all walks of real life
• Emergence of agent technology as a subfield of AI, is
a significant paradigm shift for s/w development

72
Contents
• Problem solving: state-space search and control
strategies:
– Introduction 23
– General problem solving 24
– Characteristics of problem 32
– Exhaustive searches 34
– Heuristic search techniques 44
– Iterative deepening A* 57
– Constraint satisfaction 57

73
 Problem solving: state-space search and
control strategies:

Introduction
• Problem solving is a method of deriving solution steps beginning from initial
description of the problem to the desired solution
• In AI, the problems are frequently modelled as a state space problem where
the state space is a set of all possible states from start to goal states
• A set of states form a graph in which two states are linked if there is

an operation which can be executed to transform one state to other

74
General problem solving
The following sections describe facilitating the modelling
of problems and search processes
• Production system
– Water jug problem
– Missionaries and
cannibals problem

• State – Space search

– Eight puzzle problem

• Control strategies
75
 Production System (PS)

• PS helps AI programs to do search process more conveniently in

state-space problems
• PS consists of start & goal states & DBs that consists related
info.
• PS consists of no.of production rules in which each production rule
has left and a right side
• Left side determines the applicability of rule
• Right side describes the action to be performed if the rule is
applied
• Rule’s left side is current state & right side describes the new state
76
that is obtained by applying the rule
• In addition to usefulness of PS in describing search,
following are other advantages:
– A good way to model the strong state-driven nature of
intelligent action
– New rules can be easily added to a/c for new situations
without disturbing rest of system
– It is important in RT applications where new i/p to the DB
changes the behaviour of the system

77
 Water jug problem
Problem statement:
• We have two jugs, a 5 Lt. and a 3 Lt. without measuring
marker on them.
• There is endless water supply through tap
• Our task is to get 4 Lt. of water in the 5 Lt. jug
Solution
• State space for this problem can be described as the set
of ordered pairs of integers (X,Y)
– X – no.of Lt. of water in 5 Lt. jug
– Y – no.of Lt. of water in 3 Lt. jug
• Operations are defined as production rules as in Table
78
 Production rules for water jug problem

Rule Left of rule Right of rule Description

No.
1 (X, Y | X < 5) (5, Y) Fill 5 Lt. jug
2 (X, Y | X >0) (0, Y) Empty 5 Lt. jug
3 (X, Y | Y < 3) (X, 3) Fill 3 Lt. jut
4 (X, Y | Y > 0) (X, 0) Empty 3 Lt. jug
5 (X, Y | X +Y <= 5 & Y > 0) (X+Y, 0) Empty 3 Lt. into 5 Lt. jug
6 (X, Y | X + Y <= 3 & X > 0) (0, X+Y) Empty 5 Lt. into 3Lt. jug
7 (X, Y | X + Y >= 5 & Y > 0) (5, Y – (5-X)) Until 5 Lt. jug is full Pour water from 3 Lt.
jug into 5 Lt. jug
8 (X, Y | X + Y >= 3 & X > 0) (X – (3 - Y), 3) Pour water from 5 Lt.
jug into 3 Lt. jug is full

79
 Solution path 1
Rule applied 5 Lt. jug 3 Lt. jug Step no.
Start state 0 0
1 5 0 1
8 2 3 2
4 2 0 3
6 0 2 4
1 5 2 5
8 4 3 6
Goal state 4 -

80
 Solution path 2
Rule applied 5 Lt. jug 3 Lt. jug Step no.
Start state 0 0
3 0 3 1
5 3 0 2
3 3 3 3
7 5 1 4
2 0 1 5
5 1 0 6
3 1 3 7
5 4 0 8
Goal state 4 -

81
Missionaries and cannibals problem
Problem statement
• Three missionaries and three cannibals want to cross a
river.
• There is a boat on their side of the river that can be used by
either one or two persons
• If the cannibals outnumber the missionaries, they will be
killed by cannibals
• How can they all cross over safely

82
Solution
• State space of the problem can be described as the set of
ordered pairs of left and right banks of the river as (L, R)
• Each bank is represented as a list [nM, mC, B]
– n- no.of missionaries M
– m- no.of cannibals C
– B- boat

• The table consists of production rules based on the chosen

representation
• One of the possible solution path trace is given in table

83
1. Start state: ([3M, 3C, 1B], [0M, 0C, 0B])
– 1B means the boat is present & 0B means absent

2. Any state: ([n1M, m1C, _ ], [n2M, m2C, _]), with

constraints/ conditions at any state as n1(!
=0)>=m2; n2(!=0)>=m2; n1+n2=3,
m1+m2=3; boat can be either side

3. Goal state: ([0M, 0C, 0B], [3M, 3C, 1B])

84
 Production rules for missionaries and cannibals problem

RN Left side of rule -> Right side of rule

Rules for boat going from left bank to right bank of the river
L1 ([n1M, m1C, 1B], [n2M, m2C, 0B]) -> ([(n1-2)M, m1C, 0B], [(n2+2)M, m2C, 1B])
L2 ([n1M, m1C, 1B], [n2M, m2C, 0B]) -> ([(n1-1)M, (m1-1)C, 0B], [(n2+1)M,(m2+1)C, 1B])
L3 ([n1M, m1C, 1B], [n2M, m2C, 0B]) -> ([n1M, (m1-2)C, 0B], [n2M,(m2+2)C, 1B])
L4 ([n1M, m1C, 1B], [n2M, m2C, 0B]) -> ([(n1-1)M, m1C, 0B], [(n2+1)M, m2C, 1B])
L5 ([n1M, m1C, 1B], [n2M, m2C, 0B]) -> ([n1M, (m1-1)C, 0B], [n2M,(m2+1)C, 1B])

85
 Solution path

Rule number ([3M, 3C, 1B], [0M, 0C, 0B]) <- START STATE
L2 ([2M, 2C, 0B], [1M, 1C, 1B])
R4 ([3M, 2C, 1B], [0M, 1C, 0B])
L3 ([3M, 0C, 0B], [0M, 3C, 1B])
R5 ([3M, 1C, 1B], [0M, 2C, 0B])
L1 ([1M, 1C, 0B], [2M, 2C, 1B])
R2 ([2M, 2C, 1B], [1M, 1C, 0B])
L1 ([0M, 2C, 0B], [3M, 1C, 1B])
R5 ([0M, 3C, 1B], [3M, 0C, 0B])
L3 ([0M, 1C, 0B], [3M, 2C, 1B])
R5 ([0M, 2C, 1B], [3M, 1C, 0B])
L3 ([0M, 0C, 0B], [3M, 3C, 1B]) -> GOAL STATE

87
88
 State – Space search

• Similar to Production System

• This facilitates easy search
• This consists of four parts basically:
1. Set ‘S’ containing start state
2. Set ‘G’ containing goal state
3. Set of nodes in the graph/tree/path, each node represents the state in
problem-solving process
4. Set of arcs connecting nodes, each arc corresponds to operator that is a
step in problem solving process

90
• Solution path is a path through the graph from a node in S to
a node in G
• Objective of search algorithm is to find a solution path in the
graph
• There may be more than one solution path

• Exercise a choice between various solution paths based on:

– Some criteria of goodness or

– Some heuristic function

• Lets take missionaries & cannibals problem

91
• Possible operators (applied in this problem) are {2M0C,
1M1C, 0M2C, 1M0C, 0M1C}
– 2M - means two missionaries
– 1C – means one cannibal, etc.

• These operators can be used for both the sides of the

river
• If boat is on left side, use “->”
• If boat is on right side, use “<-”
• For the sake of simplicity, lets use (L:R), here L=n1Mm1C1B,
R=n2Mm2C0B
• Here B means boat, 1 is present & 0 is absent 92
• Start state: (3M3C1B : 0M0C0B) or (331:000)

• Goal state: (0M0C0B : 3M3C1B) or (000:331)

• Following are filtered out:

– Invalid states

– Illegal operators not applicable to some states

– Some states that are not at all required

• (1M2C1B : 2M1C0B) is an invalid state, which leads to one M & two Cs on

the left side

• In case of (2M2C1B : 1M1C0B), operator 0M1C or 0M2C would be illegal

• Applying same operator both the ways is waste since it leads to previous
state, which is said to be looping situation

93
• To illustrate the progress, develop a tree of nodes, each
node represents a state
• Root node represents the start state
• Arcs represents, application of one of the operators
• Nodes for which no operator is applied, are leaf nodes
• Search space generated using valid operators are
shown in fig
• The sequence of operators applied to solve the
problem is given in table

94
95
96
 Eight puzzle problem
Problem statement:
• A 3x3 grid with 8 randomly numbered (1-8) tiles
arranged on it with one empty cell
• At any point, the adjacent tile can move to the empty
cell, creating a new empty cell
• Solving this problem involves arranging tiles such that
we get the goal state

97
98
• A state of the problem should keep track of the position of
all tiles on the game board
• 0 represents blank position
• The states are
– Start state: [[3,7,6], [5,1,2], [4,0,8]]
– Goal state: [[5,3,6], [7,0,2], [4,1,8]]
– Operations can be {up, down, left, right}

• To simplify, a search tree up to level 2 is shown in the fig. to

illustrate the use of operators to generate next state
• Continue search like this to reach goal state

99
10
0
 Control strategies
• One of the most important components of problem solving
• It describes the order of application of the rules to the
current state
• It should move towards solution by exploring the solution
space in a systematic way
• DFS & BFS are systematic approaches
• Both are blind searches

10
1
• In DFS, single branch of the tree is followed until it
yields a solution or some pre-specified depth has
reached & then go back to immediate previous
node & explore other branches
• In BFS, a search space tree is generated level wise until
a solution is found or some specified depth is reached
• Both are exhaustive, uninformed and blind
• To solve real-world problems, effective control strategy
must be used

10
2
• To find the correct strategy for a given
problem, there are two directions
– Data-driven search, called forward chaining, from
start state
– Goal-driven search, called backward chaining,
from the goal state

10
3
 Forward chaining
• Begins with known facts and works towards a conclusion
• Ex. In eight puzzle problem, we start from start state and
work forward to the goal state
• In this a tree is started building with move sequences with
the start state as root of tree
• The states of next level are generated by all rules
• This process is continued until a configuration that matches
the goal state
10
4
 Backward chaining
• It is a goal-directed strategy
• Begins with goal state and continued working backward
• In many cases this is useful strategy
• The eight puzzle problem have single start state and single
goals state
• It makes no difference when the problem is solved using
either chaining strategy since computational effort in both
the strategies is same
10
5
 Characteristics of problem
• Analysing the problem along several key characteristics
is a must before solving it
• Some of the types are:
– Type of problem
– Decomposability of problem
– Role of knowledge
– Consistency of knowledge base used
– Requirement of solution

10
6
 Type of problem
• There are 3 types of problems in real life
– Ignorable
• Solution steps can be ignored for these problems
• E.g. in proving a theorem, a lemma can be ignored
– Recoverable
• Solution steps can be undone for these problems
• E.g. water jug prob, if a jug is filled, it can be emptied
• Used in single player puzzles, solved by back tracking, using
push-down stack
– Irrecoverable
• Solution steps cannot be undone for these problems
• E.g. any two player games, like chess, snakes and ladders, etc.

35
 Decomposability of problem

• Divide problem into a set of independent smaller

sub-problems

• Solve them & combine solutions for final answer

• Each sub-problem is solved by a different

processor in parallel processing environment

• Divide & conquer is a commonly used method

10
8
 Role of knowledge

• Knowledge plays an important role in solving

any problem
• Knowledge could be in the form of rules and
facts
• They help in generating search space for
finding the solution

10
9
 Consistency of knowledge base used

• The knowledge base used to solve the problem should

be consistent
• Inconsistent knowledge base will lead to wrong
solutions
• E.g. if the knowledge is as rules and facts as follows:
– It is humid, it will rain,
– If it is sunny then it is daytime
– It is sunny day It is night-time

• This is not consistent as there is a contradiction

 Requirement of solution
• Problem should be analysed whether the solution is absolute or relative
• Exact solution is absolute solution, reasonably good or approximate
solution is relative solution
• E.g. 1 - for water jug problem, one solution is considered and no need to
find better solution
• The solution is absolute, it is any path solution
• E.g. 2 - for travelling salesman problem, unless all routes are known, it is
difficult to find the shortest path
• The solution is relative, it is best path solution
• Best-path problems are computationally harder than any-path problems

11
1
 Exhaustive searches

• Following are the few uninformed exhaustive

searches:
– Breadth-First-Search (BFS)

– Depth-First-Search (DFS)

– Depth-First-Iterative Deepening

– Bidirectional Search

11
2
 Breadth-First-Search (BFS)
• BFS expands all the states one step away from start state, and then
expands two steps away, and then three steps away, and so on until a goal
state is reached
• BFS always give an optimal solution

• BFS uses two lists, viz. OPEN & CLOSED

• OPEN is a Queue, that contains the states that are to be expanded

• CLOSED is a Stack, that contains the states that are already expanded

• E.g. a BFS is implemented to check whether a goal node exists or not for
water jug problem

11
3
11
4
11
5
• At each state, applicable rule is applied first and

if it generates one of previous states, cross it

• If new state is generated, expand it in BFS style

• A path is visible from start to goal states by tracing

the tree in reverse through parent link

• Path is optimal & more shorter path is not there

• Solution path:
(0,0) -> (5,0) -> (2,3) -> (2,0) -> (0,2) -> (5,2) -> (4,3) 11
6
Depth-First-Search (DFS)
• DFS is implemented using two stacks OPEN &
CLOSED
• OPEN list contains states that are to be
expanded
• CLOSED list keeps track of states already
expanded
• If we discover the first element of the OPEN as
goal state, get track of the path as we traversed 11
7
11
8
11
9
S.NO BFS DFS
1. BFS stands for Breadth First Search. DFS stands for Depth First Search.
BFS(Breadth First Search) uses Queue data DFS(Depth First Search) uses Stack data
2. structure for finding the shortest path. structure.

BFS can be used to find single source

shortest path in an unweighted graph, In DFS, we might traverse through more
3. because in BFS, we reach a vertex with edges to reach a destination vertex from a
minimum number of edges from a source source.
vertex.

4. BFS is more suitable for searching vertices DFS is more suitable when there are
which are closer to the given source. solutions away from source.
DFS is more suitable for game or puzzle
BFS considers all neighbours first and problems. We make a decision, then
5. therefore not suitable for decision making explore all paths through this decision. And
trees used in games or puzzles. if this decision leads to win situation, we
stop.

The Time complexity of BFS is O(V + E)
when Adjacency List is used and O(V^2)
6.
when Adjacency Matrix is used, where V
stands for vertices and E stands for edges.
The Time complexity of DFS is also O(V + E)
when Adjacency List is used and O(V^2)
when Adjacency Matrix is used, where V
stands for vertices and E stands for edges.

7. Here, siblings are visited before the Here, children are visited before the
children siblings
     

12
 Depth-First-Iterative Deepening
• DFID takes the advantages of BFS & DFS
• DFID expands all nodes at a given depth before expanding any
nodes at greater depth
• DFID gives optimal solution of shortest path from start state to
goal state
• At any time it is performing a DFS and never searches deeper than
depth ‘d’
• Thus it uses the space O(d)
• Disadvantage, is it performs wasted computation
• Working of DFID algorithm is as follows
 Bidirectional Search
• It runs two simultaneous searches
• One search moves forward from start to goal and another moves back
from goal to start
• Searching is stopped when both meet in middle
• Useful if there are only one start and goal states
• If match is found, path can be traced from start to match state and
match to goal state
• Each node has link to its successors and parents
• Each of two searches has time complexity O(bd/2) and O(bd/2+bd/2) is
much less running time
 Analysis of search methods

• Effectiveness of any search strategy in problem solving is

measured in terms of:
– Completeness: Algorithm guarantees a solution if it exists
– Time complexity: Time required to find a solution
– Space complexity: Space required to find a solution
– Optimality: The algorithm is optimal if it finds the highest
quality solution when there are several different solutions
for the problem
 Travelling salesman problem

• There are n cities & distance between each pair of the cities is given

• Find shortest route of visiting all cities once & return to starting point

• This require (n-1)! Paths to be examined

• This phenomenon is called ‘combinatorial explosion’

• Generate complete paths, keeping track of shortest path found so far

• Stop exploring any path as soon as its partial length becomes greater
than the shortest path length found so far
 Heuristic search techniques

• This is a criterion to find out most effective way to

achieve the goal among all choices
• It no longer guarantees to find the best solution but
finds a very good solution (by using good
heuristics)
• There are two types of heuristics
– General purposes heuristics – useful in various problem
domains
– Special purpose heuristics – domain specific
 General purposes heuristics

• A General purpose heuristics for combinatorial problem is Nearest

Neighbour Algorithms that work by selecting the locally superior
alternative
• In many AI problems it is often difficult to measure precisely the
goodness of a solution
Branch and bound search
d
 Hill climbing
• It is a heuristic search used for mathematical optimisation problems of AI
• Given a large set of inputs and a good heuristic function, the algorithm
tries to find the best possible solution to the problem in the most
reasonable time period
• This solution may not be the absolute best (global optimal maximum) but
it is sufficiently good considering the time allotted
• Hill-climbing solves the problems where we need to maximise or
minimise a given real function by selecting values from the given inputs
• E.g. Travelling Salesman Problem where we need to minimise the
distance travelled by the salesman
Features of Hill Climbing

• It carries out a Heuristic search.

• Heuristic function ranks all potential alternatives in a search

algorithm based on the information available.

• It helps the algorithm to select best route to its solution.

• This basically means that this search algorithm may not find
the optimal solution to the problem

• But it will give possible solution in a reasonable time.

• It is a variant of the generate-and-test algorithm.

The algorithm is as follows :
• Step1: Generate possible solutions.
• Step2: Evaluate to see if this is the expected solution.
• Step3: If the solution has been found quit else go back to
step 1.
• Hill climbing takes the feedback from the test
procedure and the generator uses it in deciding the
next move in the search space
• Hence, we call it as a variant of the generate-and-
test algorithm
• It uses the Greedy approach
• At any point in state space, the search moves in that
direction only which optimises the cost of function
with the hope of finding the most optimum solution at
the end
State Space diagram for Hill Climbing
• State-space diagram is a graphical representation of the set of states
(input) our search algorithm can reach vs the value of our objective
function (function we intend to maximise / minimise).
• Here:
1. The X-axis denotes the state space i.e. states or configuration our
algorithm may reach.
2. The Y-axis denotes the values of objective function
corresponding to a particular state.
• Best solution will be that state space where objective function has
maximum value (global maxima)
State space
Following are the different regions in the Diagram;
• Local maxima: It is a state which is better than its neighbouring state however
there exists a state which is better than it (global maximum)
• This is better because the value of objective function is higher than its
neighbours.
• Global maxima: It is the best possible state in the state space diagram
• This because at this state, objective function has the highest value.
• Plateau/flat local maxima: It is a flat region of state space where neighbouring
states have the same value.
• Ridge: It is a region which is higher than its neighbour’s but itself has a slope
• It is a special kind of local maximum.
• Current state: The region of state space diagram where we are currently present
during the search. (Denoted by the highlighted circle in the given image.)okk5K%
%
%555ktkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
Types of Hill Climbing
• 1. Simple Hill Climbing
– It only evaluates the neighbour node state at a time and selects
the first one which optimizes current cost & set it as current
state
– It only checks it’s one successor state, and if it finds better than
the current state, then move else be in the same state
– This algorithm has the following features:
• Less time consuming
• Less optimal solution
• The solution is not guaranteed
Algorithm for Simple Hill Climbing

• Step 1: Evaluate initial state, if it is goal state then return success and Stop.

• Step 2: Loop Until a solution is found or no new operator left to apply.

• Step 3: Select and apply an operator to the current state.

• Step 4: Check new state:

– If it is goal state, then return success and quit.

– else if it is better than current state then assign new state as a current state.

– else if not better than the current state, then return to step 2.

• Step 5: Exit.
2. Steepest-Ascent hill climbing
• The steepest-Ascent algorithm is a variation of
the simple hill-climbing algorithm
• This algorithm examines all the neighbouring
nodes of the current state and selects one
neighbour node which is closest to the goal state
• This algorithm consumes more time as it searches
for multiple neighbours.
Algorithm for Steepest-Ascent hill climbing

• Step 1: Evaluate the initial state, if it is goal state then return success and stop, else
make the current state as your initial state.

• Step 2: Loop until a solution is found or the current state does not change.

– Let S be a state such that any successor of current state will be better than it.

– For each operator that applies to the current state;

• Apply the new operator and generate a new state.

• Evaluate the new state.

• If it is goal state, then return it and quit, else compare it to the S.

• If it is better than S, then set new state as S.

• If the S is better than the current state, then set the current state to S.

• Step 5: Exit.
3. Stochastic hill climbing
• It does not examine all the neighbouring nodes
before deciding which node to select
• It just selects a neighbouring node at random and
decides (based on the amount of improvement
in that neighbor) whether to move to that
neighbour or to examine another
Algorithm for Stochastic Hill climbing
• Step 1: Evaluate the initial state. If it is a goal state then stop and return success.
Otherwise, make initial state as current state.
• Step 2: Repeat these steps until a solution is found or current state does not
change.
• a) Select a state that has not been yet applied to the current state.

• b) Apply successor function to current state & generate all neighbour states.

• c) Among the generated neighbour states which are better than current state
choose a state randomly (or based on some probability function)
• d) If the chosen state is goal state, then return success, else make it
current state and repeat step 2: b) part.
• Step 3: Exit.
Problems in different regions in Hill climbing

Hill climbing cannot reach the best possible state if it enters any of the following
regions :
• 1. Local maximum: At a local maximum all neighbouring states have values which
are worse than the current state
• Since hill-climbing uses a greedy approach, it will not move to the worse state and
terminate itself
• The process will end even though a better solution may exist.
• To overcome the local maximum problem: Utilise the backtracking technique
• Maintain a list of visited states.
• If the search reaches an undesirable state, it can backtrack to the previous
configuration and explore a new path.
• 2. Plateau: On the plateau, all neighbours have the same value.
Hence, it is not possible to select the best direction.

• To overcome plateaus: Make a big jump.

• Randomly select a state far away from the current state.

• Chances are that we will land at a non-plateau region

• 3. Ridge: Any point on a ridge can look like a peak because the
movement in all possible directions is downward.

• Hence, the algorithm stops when it reaches such a state.

To overcome Ridge: You could use two or more rules before testing.

• It implies moving in several directions at once

Simulated Annealing
• A hill-climbing algorithm which never makes a move
towards a lower value guaranteed to be incomplete
because it can get stuck on a local maximum
• And if algorithm applies a random walk, by moving a
successor, then it may complete but not efficient.
• Simulated Annealing is an algorithm which yields
both efficiency and completeness.
• Mechanically, Annealing is a process of hardening a metal or
glass to a high temperature then cooling gradually
• This allows metal to reach a low-energy crystalline state.
• Same process is used in simulated annealing in which the
algorithm picks a random move, instead of picking best move
• If random move improves the state, then it follows the same
path
• Otherwise, the algorithm follows the path which has a
probability of less than 1 or it moves downhill and chooses
another path.
Applications of Hill Climbing Technique
• It can be used to solve many problems, where current
state allows for an accurate evaluation function, e.g.
Network-Flow, Travelling Salesman problem, 8-Queens
problem, Integrated Circuit design, etc.
• This technique is also used in robotics for coordinating
multiple robots in a team
 Best First Search
• Combines the benefits of both depth first and breadth first
search by moving along a single path at a time
• But change paths whenever some other path looks more
promising than the current path
• In Best First Search although one path might be selected at
a time but others are not thrown
• They can be revisited in future if the selected path becomes
less promising.
• It is sometimes important to search a graph instead of
a tree so we have to take care that the duplicate
paths are not pursued
• To perform this job, an algorithm will work by
searching a directed graph in which a node represents
a point in the problem space
• Each node, in addition to describing the problem space
and the heuristic value associated with it, will also
contain a link or pointer to its best parent and points to
its successor node
• Once the goal node is found, the parent link will allow
us to trace the path from source node to the goal node
• OPEN list (queue): is the list of nodes which have been
found but yet not expanded
• CLOSED list: contains expanded/visited nodes i.e., the
anodes whose successors are also generated.
Algorithm for Best First Search

1. Place the start node on the OPEN list.

2. Create a list called CLOSED i.e., initially empty.

3. If the OPEN list is empty search ends unsuccessfully. Else

4. Remove the first node on OPEN list and put this node on CLOSED list.

5. If this is a goal node, search ends successfully. else

6. Generate successors of this node:

For each successor :

(a) If it has not been discovered before i.e., it is not on OPEN, evaluate this node by applying the
heuristic function, add it to the OPEN and record its parent.

(b) If it has been discovered before, change the parent if the new path is better than the previous
one. Update cost of getting to this node & to any successors that this node may already have.

7. Reorder the list OPEN, according to the heuristic merit.

8. Go to step 3.
• Best first searches will always find good paths to a goal node if there is
any

• But it requires a good heuristic function for better estimation of

the distance to a goal node

16
7
 Beam Search
A heuristic search algorithm that examines a graph by extending
the most promising node in a limited set is known as beam
search.

 it always expands the W number of the best nodes at each

level.

Beam Search constructs its search tree using breadth-first

search. It generates all the successors of the current level’s state
at each level of the tree. However, at each level, it only
evaluates a W number of states. Other nodes are not taken into
account.

W = 1, the search becomes a hill-climbing search in which the

best node is always chosen from the successor nodes 16
8
16
9
Beam search
 A* algorithm
• It is an advanced BFS algorithm that searches for shorter paths first rather
than the longer paths
• A* is optimal as well as a complete algorithm.
• Optimal meaning that A* is sure to find the least cost from the source to
the destination
• It is going to find all the paths that are available to us from the source to
the destination.
• But A* is slow and also the space it requires is a lot as it saves all the
possible paths that are available
So why choose A* over other faster algorithms?
• Let the graphs below answer that for you. I have taken the
Dijkstra’s algorithm and A* Algorithm for comparison.
• We know that the Dijkstra’s Algorithm finds all the paths
that can be taken without finding or knowing which is the
most optimal one for the problem that we are facing.
• This leads to the unoptimized working of the algorithm and
unnecessary computations
• A* algorithm, on the other hand, finds the
most optimal path that it can take from the
source in reaching the destination.
• It knows which is the best path that can be
taken from its current state and how it needs
to reach its destination.
• It uses a heuristic or evaluation function f(X)

• F(n) = g(n) + h(n)

• Function g is the cost of starting node to node n

• Function h is the cost of node n to goal node

• A* algorithm incrementally searches all the

routes starting from starting node until it finds
the shortest path to goal node
17
6
• Starting with a given node, the algorithm expands the node
with the lowest f(X) value
• It maintains a set of partial solutions
• Unexpanded leaf nodes of expanded leaf nodes are stored
in a queue with f values
• E.g. solve eight puzzle problem using A* algo.
• F(X) = g(X)+h(X)
• h(X) -> no.of tiles, not in their goal position in given state X
• g(X) -> depth of node X in the search tree
 Iterative deepening a*
• IDA* is the combination of the DFID & A* algo
• Here, successive iterations are corresponding to increasing values of the
total cost of a path rather than increasing depth of the search
• For each iteration, perform a DFS pruning off a branch when its total cost
(g+h) exceeds a given threshold
• Initial threshold starts at the estimate cost of the start state and increase
for each iteration of the algorithm
• Threshold used for the next iteration is the minimum cost of all values
exceeded the current threshold
• Repeat until find a goal
• The IDA* finds a least cost or optimal solution
• Uses far less space than A*
• It expands approximately the same number of
nodes as the A* in a tree search
• Simpler to implement as there are no OPEN and
CLOSED lists to be maintained
• A simple recursion performs DFS inside an outer
loop to handle iterations
Informed Search Uninformed Search

It uses knowledge for the searching It doesn’t use knowledge for

process. searching process.

It finds solution more quickly. It finds solution slow as compared

to informed search.

It may or may not be complete. It is always complete.

Cost is low. Cost is high.

It consumes less time. It consumes moderate time.

It provides the direction regarding No suggestion is given regarding

the solution. the solution in it.

It is less lengthy while It is more lengthy while

implementation. implementation.

Greedy Search, A* Search, Graph Depth First Search, Breadth First

Search Search

18
5
 Constraint Satisfaction
• Many AI problems can be viewed as problems of constraint
satisfaction
• Objective is to solve problem state instead of optimal path
• These are Constraint Satisfaction (CS) Problems
• Search can be made easier in those cases in which the solution
is required to satisfy local consistency conditions
• E.g. Cryptography, n-Queen Problem, map colouring,
crossword puzzle, etc.
 Crypt-Arithmetic puzzle
• Problem statement: solve the puzzle by assigning 0-9 in
such a way that each letter is assigned unique digit
which satisfy the following addition:
B A SE

+B A LL

GAMES

• Constrai
nts: no
two
 C 4C 3C 2C 1
B A S E
+B A L L
GAME S
• Constraints equations are:
• E+L = S
• S+L+C1=E
• 2A+C2=M
• 2B+C3=A
• G=C4
19
0