FOUNDAMENTALS

IN AI and ML
CSA2001

J. Jayanthi
UNIT -2
contents

What are
State space Tree Uninformed Informed
search
search search search
problems?
What are search
problems?
• We will consider the problem of designing goal-based agents
in
known, fully observable, and deterministic environments.
• Example environment:

Start

Exit
Remember: Goal-based
Agent
• The agent has the task to reach a defined goal state.
• The agent needs to move towards the goal. It can use search algorithms
to plan actions that lead to the goal.
• The performance measure is typically the cost to reach the goal.
Agent’s
Maze
location

Result of
moving

Exit
location

𝑎𝑠 = argmina∈A[𝑐𝑜𝑠𝑡 𝑠, 𝑠1, 𝑠2, … , 𝑠𝑛 𝑠𝑛 ∈ 𝑆𝑔𝑔𝑜𝑎𝑙 ]

What are search
problems?
• We will consider the problem of designing Initial state
goal-based agents in, known, fully
observable, deterministic environments. 1

• For now, we consider only a discrete

environment using an atomic state
representation (states are just labeled 1, 2,
3,
…).
• The state space is the set of all possible
z Goal
states of the environment and some states stat
are marked as goal states. e
1)Phases:
Search/Planning: the process of looking for the sequence of actions that
reaches a goal state. Requires that the agent knows what happens when it
moves!
2) Execution: Once the agent begins executing the search solution in a
deterministic, known environment, it can ignore its percepts (open-loop
Search problem
components
•Initial state: state description Initial state Actions: {N, E, S, W}
Transitions
•Actions: set of possible actions 𝐴 1
g i
•Transition model: a function 4
that defines the new state resulting
from performing an action in the
current state
a
•Goal state: state description
•Path cost: the sum of step costs
z Goal
stat
Notes grid
Discretization
e
: • The state space is typically too large to be enumerated or it is continuous.
Therefore, the problem is defined by initial state, actions and the
transition model and not the set of all possible states.
• The optimal solution is the sequence of actions (or equivalently a sequence
of states) that gives the lowest path cost for reaching the goal.
S- set of all states State represent --- all nodes
S0-> initital state Opeartor – edges b/w the nodes
S0 belongs S
A: s->s set of operators
G: G belongs S final state
 Original Description

Example: Romania
Vacation
• On vacation in Romania; currently in
Arad
• Flight leaves tomorrow from Bucharest
• Initial state: Arad State Space/Transition
model
• Actions: Drive from
Defined as a graph
one city to
another.
• Transition model
and states: If you
go from city A to
city B, you end up
in city B.
• Goal state:
Bucharest
• Path cost: Sum
Distance in miles
Example: Vacuum
world
State Space

Goal states

•Initial State: Defined by agent location and dirt

location. Left, right, suck
•Actions:
There are 8 possible
•Transition model: Clean a location or atomic states of the
move. system.
Why is the number of
•Goal state: All locations are clean. states for n possible
•Path cost: E.g., number if actions locations 𝑛 2𝑛 ?
Example: Sliding-tile
puzzle
• Initial State: A given configuration.
• Actions: Move blank left, right, up, down
• Transition model: Move a tile
• Goal state: Tiles are arranged empty and 1-8 in order
• Path cost: 1 per tile move.

State space size

Each state describes the location of each tile (including the
empty one). ½ of the permutations are unreachable.
• 8-puzzle: 9!/2 = 181,440 states
• 15-puzzle: 16!/2 ≈ 1013 states
• 24-puzzle: 25!/2 ≈ 1025 states
 Example: Robot motion
planning

• Initial State: Current arm position.

• States: Real-valued coordinates of robot joint angles.
• Actions: Continuous motions of robot joints.
• Goal state: Desired final configuration (e.g., object is
grasped).
• Path cost: Time to execute, smoothness of path, etc.
Example Problems – Towers of
Hanoi States: combinations of poles and disks

Operators: move disk x from pole y to pole z

subject to constraints
• cannot move disk on top of smaller disk
• cannot move disk if other disks on top

Goal test: disks from largest (at bottom) to

smallest on goal pole

Path cost: 1 per move

Towers of Hanoi applet

Example Problems
– Rubik’s Cube
States: list of colors for each cell on each face

Initial state: one specific cube configuration

Operators: rotate row x or column y on face

z direction a

Goal: configuration has only one color on

each face

Path cost: 1 per move

Rubik’s cube applet

Example Problems
– Eight Queens
States: locations of 8 queens on chess board

Initial state: one specific queens

configuration

Operators: move queen x to row y and

column z

Goal: no queen can attack another (cannot

be in same row, column, or diagonal)

Path cost: 0 per move

Eight queens applet

Example Problems
–Water Jug
States: Contents of 4-gallon jug and 3-gallon
jug

Initial state: (0,0)

Operators:
• fill jug x from faucet
• pour contents of jug x in jug y until y full
• dump contents of jug x down drain

Goal: (2,n)

Path cost: 1 per fill

Saving the world, Part I

Saving the world, Part II

Solving search
problems
Given a search
problem definition How do we find the Construct a
optimal solution search tree
• Initial state for the state
• Actions (sequence of
actions/states)? space graph!
• Transition model
• Goal state
• Path cost
Initial state
State space

Goal states
 Issue: Transition model is not a
tree! It has cycles vs. redundant
paths Cycles
Return to the same state. The search tree will create a new node!
Initial state

Redundant paths
Multiple paths to get to the same state
Goal states
Initial state

Path Path
1 2

Goal states
 Search
•tree
Superimpose a “what if” tree of possible
actions and outcomes (states) on the Root node =
state space graph. Initial state
• The Root node represents the initial stare. a
• An action child node is reached by an edge Edge = Action
representing an action. The corresponding
state is defined by the transition model. Child node
• Trees have no cycles or redundant paths. b c
Cycles in the search space must be broken. Redundant
To prevent infinite loops. Removing path
redundant paths improves
search efficiency. d e e
• A path through the tree corresponds to a Cycle
sequence of actions (states).
• A solution is a path ending in a node
representing a goal state.
b … … …
• Nodes vs. states: Each tree node Solution path
represents a state of the system. If Node representing
redundant path cannot be prevented then f
a Goal state
state can be represented by
multiple nodes.
Differences between typical Tree search
and AI search
Typical tree search AI tree/graph search
• Assumes a given tree that fits
in memory.
• Trees have by construction
no cycles or redundant
paths.
• The search tree is too large to fit
into
a. Builds parts of the tree from the
memory.
initial state using the transition
function representing the graph.
b. Memory management is very
important.
• The search space is typically a very
large and complicated graph.
Memory-efficient cycle checking is
very important to avoid infinite
loops or minimize searching parts of
the search space multiple times.
• Checking redundant paths often
requires too much memory and we
accept searching the same part
multiple times.
Tree Search Algorithm
Outline
1. Initialize the frontier (set of unexplored know
nodes) using the starting state/root node.
2. While the frontier is not empty:
a) Choose next frontier node to expand according to
search strategy.
b) If the node represents a goal state, return it as
the solution.
c) Else expand the node (i.e., apply all possible actions
to the transition model) and add its children nodes
representing the newly reached states to the frontier.
Tree search
example Frontier

Transition model
Tree search
example 1. Expand Arad

Frontier

Transition model
 Tree search
example
Frontier
2. Expand Sibiu

Transition model
Example of
a cycle

We could
have also
expanded
Timisoara or
Zerind!
Search
strategies
• A search strategy is defined by picking the order of node expansion.

• Strategies are evaluated along the following dimensions:

• Completeness: does it always find a solution if one exists?
• Optimality: does it always find a least-cost solution?
• Time complexity: how long does it take?
• Space complexity: how much memory does it need?

• Worst case time and space complexity are measured in terms of the
size of the state space n (= number of nodes in the search tree).
Metrics used if the state space is only implicitly defined by initial
state, actions and a transition function are:
• d: depth of the optimal solution (= number of actions needed)
• m: the number of actions in any path (may be infinite with loops)
• b: maximum branching factor of the search tree (number of successor nodes for
a parent)
State Space for
Search
State
Space
• Number of different states the agent and
environment can be in.
State
representation
• Reachable states are defined by the initial state and
the 𝑥1
𝑥2
transition model. Not all states may be reachable …
from the initial state.
• Search tree spans the state space. Note that a single
state can be represented by several search tree nodes
if we have redundant paths.
• State space size is an indication of problem size.
State Space Size Estimation

• Even if the used algorithm represents the state space

using atomic states, we may know that internally
they have a factored representation that can be
used to estimate the problem size.
• The basic rule to calculate (estimate) the state space
size for factored state representation with 𝑛 variables
is:
𝑥1 × 𝑥2 × ⋯ × 𝑥𝑛
where ⋅ is the number of possible values.
 In how many ways can
we order/arrange n
objects?

2×2=4
3×2×1=6

Factorial: 𝑛! = 𝑛 × 𝑛 − 1 × ⋯ × 2 × 1

#Python
import math

print (math.factorial(23))

Source: Permutations/Combinations Cheat Sheets by Oleksii Trekhleb

https://itnext.io/permutations-combinations-algorithms-cheat-sheet-68c14879aba5
 𝑛
Binomial Coefficient: = 𝐶 𝑛, 𝑟 = 𝐶 𝑛 𝑟
3 𝑟
=3 Read as “n choose r” because it is the number
2 of ways can we choose 𝑟 out of 𝑛 objects?
𝑛
Special case for 𝑟 = 2: 𝑛
= 2
2
(𝑛−1)

#Python
import scipy.special

# the two give the same

results
scipy.special.binom(10, 5)
scipy.special.comb(10, 5)
Source: Permutations/Combinations Cheat Sheets by Oleksii Trekhleb
https://itnext.io/permutations-combinations-algorithms-cheat-sheet-68c14879aba5
Examples: What is the state space
size?

Dirt
• Order of A and B does not matter!
• Repetition: Dirt can be in both
places
• There are 2 options (clean/dirty)

→ 22

Robot
• Can be in 1 out of 2 squares.
→2

Total: 2 × 22= 23
Examples: What is the state space
size?
Often a rough upper limit is sufficient to determine how hard the search problem is.

Maze 8-queens problem 8-puzzle problem Tic-tac-toe

Examples: What is the state space
size?
Often a rough upper limit is sufficient to determine how hard the search problem is.

Maze 8-queens problem 8-puzzle problem Tic-tac-toe

Positions the All arrangements All All possible

agent can be in. with 8 queens on the arrangements boards.
board. of 9 elements. 𝑛 < 39 = 19,683
n = Number 𝑛 ≤ 9!
of white 𝑛 < 264 ≈ 1.8 × 1019 Many boards are
squares. Half is not legal (e.g., all
We can only have unreachable x’s)
8 queens: : 9!
𝑛 = 64 9 𝑛 = 2= 181,440
8 ≈ 4.4 × 10
Search Function –
Uninformed Searches
Open = initial state // open list is all generated states
// that have not been “expanded”
While open not empty // one iteration of search algorithm
state = First(open) // current state is first state in open
Pop(open) // remove new current state from open
if Goal(state) // test current state for goal condition
return “succeed” // search is complete
// else expand the current state by
// generating children and
// reorder open list per search strategy
else open = QueueFunction(open, Expand(state))
Return “fail”
Uninformed
Search
Uninformed search
strategies
The search algorithm/agent is not provided information about
how close a state is to the goal state.

It blindly searches following a simple strategy until it finds the

goal state by chance.

Search strategies we will discuss:

Breadth-first search
Uniform-cost search
Depth-first search
Iterative deepening
search
 Breadth-first search
(BFS)
Expansion rule: Expand shallowest unexpanded node in the
frontier (=FIFO).

Data Structures
• Frontier data structure: holds references to the green nodes (green) and
is implemented as a FIFO queue.
• Reached data structure: holds references to all visited nodes (gray and
green) and is used to prevent visiting nodes more than once (cycle
checking).
• Builds a tree with links from parent to child.
Implementation:
BFS

Expand adds the next level

below node to the search tree.

reached makes sure we do

not visit nodes twice (e.g., in
a cycle or a redundant path).
Fast lookup is important.

31
Implementation: Expanding the search
tree
• AI tree search creates the search tree while searching.
• The EXPAND function uses the current search tree node (i.e.,
current state) and the problem description to create new nodes for
all reachable states.
• It tries all actions in the current state by checking the
transition function (RESULTS) and then returns a list of new
nodes for the frontier.
Node structure for
the search tree.
Yield can also be
Transitio implemented by
n returning a list of
function Nodes.
Properties of breadth-first
search d: depth of the optimal solution
• Complete? m: max. depth of tree
Yes b: maximum branching factor

• Optimal?
Yes – if cost is the same per step (action). Otherwise: Use uniform-cost search.

• Time?
Sum of the number of nodes created in at each level in a b-ary tree of depth d:
1 + 𝑏 + 𝑏2 + ⋯ + 𝑏𝑑 = 𝑂(𝑏𝑑)
• Space?
Stored nodes: 𝑂(𝑏𝑑)

Note:
Disadv:The large space complexity is usually a bigger problem than time!
Adv: minimal path to goal
 d: depth of the optimal solution

Breadth-first m: max. depth of tree

b: maximum branching factor

search
• Time and Space: 𝑂 𝑏𝑑 - all paths to the depth of the goal are expanded

expanded
d=2
b=2
m=4
B C Goal

D E F G

C Goal
 Analysis
See what happens with b=10
expand 10,000 nodes/second
1,000 bytes/node

Depth Nodes Time Memory

.11
2 1110 1 megabyte
seconds
106
4 111,100 11 seconds
megabytes
6 107 19 minutes 10 gigabytes
8 109 31 hours 1 terabyte
10 1011 129 days 101 terabytes
12 1013 35 years 10 petabytes
15 1015 3,523 years 1 exabyte
Depth-first
search
(DFS)
• Expansion rule:
Expand deepest
unexpanded
node in the
frontier
(last
added).
• Frontier: stack
(LIFO)
• No reached data
structure!
Cycle checking
checks only the
current path.
Redundant paths
can not be
identified and
lead to
replicated work.
Implementation:
DFS
• DFS could be implemented like BFS/Best-first search and just taking the
last element from the frontier (LIFO).
• However, to reduce the space complexity to 𝑂(𝑏𝑚), the reached data structure
needs to be removed! Options:
• Recursive implementation: cycle checking is a problem leading to infinite loops.
• Iterative implementation: Build tree and abandoned branches are removed from
memory. Cycle checking is only done against the current path. This is similar to
Backtracking search.
DFS uses 𝑃 = ∞

If we only keep the current

path in memory, then we
can only check against the
path from the root to the
current node and the
frontier to prevent cycles.

See BFS for function

Properties of depth-first
search
• Complete?
• Only in finite search spaces. Some cycles can be avoided by checking for
repeated states along the path.
• Incomplete in infinite search spaces (e.g., with cycles).

• Optimal? d: depth of the optimal solution

m: max. depth of tree
No – returns the first solution it finds.
b: maximum branching factor

• Time?
Could be the time to reach a solution at maximum depth m in the last path: 𝑂 𝑏𝑚
Terrible if 𝑚 ≫ 𝑑, but if there are many shallow solutions, it can be much faster than
BFS.

• Space?
𝑂 𝑏𝑚 ⬄ linear in max. tree depth (only if no reached data structure is used!)

Note:
Disadv: not complete/ infinite (depth not limit )
 d: depth of the optimal solution
m: max. depth of tree
Depth-first b: maximum branching factor

search
•Time: 𝑂(𝑏𝑚) – worst case is expanding all paths.
• Space: 𝑂(𝑏𝑚) - if it only stores the frontier nodes and the current path.

A
b=2 d=2

B C Goal
m=4

D E

H Goal DFS finds this goal first ⬄ Not optimal!

Note: The order in which we add new nodes to the frontier can change what goal we
find!
 Comparison of
Search Techniques
DF BF
S S
Complet N Y
e
Optimal N N
Heuristic N N
Time bm bd+1
Space bm bd+1
Iterative deepening search
(IDS)
Can we
a) get DFS’s good memory footprint,
b) avoid infinite cycles, and
c) preserve BFS’s optimality guaranty?

Use depth-restricted DFS and gradually increase the

depth.

1. Check if the root node is the goal.

2. Do a DFS searching for a path of length 1
3. If goal not found, do a DFS searching for a path of length 2
4.If goal not found, do a DFS searching for a path of length 3 5.
…
Iterative
deepenin
g search
(IDS)
Implementation:
IDS

See BFS for function

EXPAND.
Properties of iterative deepening
search
• Complete? d: depth of the optimal solution
Yes m: max. depth of tree
b: maximum branching factor
• Optimal?
Yes, if step cost =
1
• Time?
Consists of rebuilding trees up to d times
⬄ Slower than BFS, but the same
𝑑 𝑏1 + (𝑑 − 1)𝑏2 + … + 1𝑏𝑑 = 𝑂(𝑏𝑑)
complexity!

• Space?
O(bd) ⬄ linear space. Even less than DFS since m<=d. Cycles need to be handled by
the depth-limited DFS implementation.

Note: IDS produces the same result as BFS but trades better space complexity
for worse run time.
This makes IDS/DFS into
the workhorse of AI.
 Bidirectional Search
Search forward from initial state to
goal AND backward from goal state to
initial state
Can prune many options
Considerations
Which goal state(s) to use
How determine when searches
overlap
Which search to use for each
direction
Here, two BFS searches
Time and space is O(bd/2)
 Uniform Cost Search
(Branch&Bound)
priority queue
QueueingFn is SortByCostSoFar
Cost from root to current node n is g(n)
Add operator costs along path
First goal found is least-cost solution
Space & time can be exponential because large subtrees with inexpensive steps
may be explored before useful paths with costly steps
If costs are equal, time and space are O(bd)
Otherwise, complexity related to cost of optimal solution
Uniform-cost search
(= Dijkstra’s shortest path
• Expansion rule: Expand node in the frontier with the least path cost from the initial state.
algorithm)
• Implementation: best-first search where the frontier is a priority queue ordered by lower 𝑓𝑓(𝑛) =
path cost (cost of all actions starting from the initial state).
• Breadth-first search is a special case when all step costs being equal, i.e., each action costs
the same!

• Complete? d: depth of the optimal solution

Yes, if all step cost is greater than some small positive constantε > 0 m: max. depth of tree
b: maximum branching factor
• Optimal?
Yes – nodes expanded in increasing order of path cost

• Time?
( is O(b1+C*/ ε).
Number of nodes with path cost ≤ cost of optimal solution C*)
This can be greater than O(bd): the search can explore long paths consisting of small steps before exploring
shorter paths consisting of larger steps

• Space?
1+C*/ ε
O(b )

See Dijkstra's algorithm on Wikipedia

Implementation: Best-First Search
Strategy

The order for expanding

the frontier is determined
by f(n) = path cost to node
n.

This check is the difference

to BFS! It visits a node again
See BFS for function if it can be reached by a
EXPAND. better (cheaper) path.
36
UCS Example
Graph-search

Note:
1) Uses “explored” set to avoid visiting already
explored states.
2) Uses “frontier” set to store states that remain to be
explored and expanded.
3) However, with eg uniform cost search, we need to
make a special check when node (i.e. state) is on
frontier. Details later.
Tree-search algorithms
Basic idea:
○ simulated exploration of state space by generating successors of
already-explored states (a.k.a. ~ expanding states)

Note: 1) Here we only check a node for possibly being a goal state,
after we select the node for expansion.
2) A “node” is a data structure containing state + additional info
(parent
node, etc.
Informed
Informed
search
• AI search problems are typically very large. We would like to
improve efficiency by expanding as few nodes as possible.

• The agent can use additional information in the form of “hints” about
how promising different states/nodes are to lead to the goal. These hints
are derived from
• information the agent has (e.g., a map) or
• percepts coming from a sensor.
• Heuristic –” Rule of thumb”
• The agent uses a heuristic function ℎ(𝑛) to rank nodes in the frontier and
select the most promising state in the frontier for expansion using a best-
first search strategy.

• Algorithms:
• Greedy best-first search
• A* search
 Informed Searches
Best-first search, Hill climbing, Beam search, A*, IDA*, RBFS, SMA*
New terms
Heuristics
Optimal solution
Informedness
Hill climbing problems
Admissibility
New parameters
g(n) = estimated cost from initial state to state n
h(n) = estimated cost (distance) from state n to closest goal
h(n) is our heuristic
Robot path planning, h(n) could be Euclidean distance
8 puzzle, h(n) could be #tiles out of place
Search algorithms which use h(n) to guide search are heuristic search algorithms
Heuristic
function
• Heuristic function ℎ(𝑛) estimates the cost of reaching a node representing
the goal state from the current node 𝑛.
• Examples:
Euclidean distance Manhattan distance
Start state Start state

Goal state Goal state

Travelling Salesman Problem
Tower of Hanoi Problem
Water-Jug Problem
N-Queen Problem
Chess
Sudoku
Crypt-arithmetic Problems
Magic Squares
Logical Puzzles and so on.
 BBe
BFS-Best First Search
 A*

Here, from A to B, the sum of cost and

heuristics is 1 + 3 = 4.
From A to C, it is 2 + 4 = 6.
From A to H, it is 7 + 0 = 7.

From A to B to D, the cost is 1 + 4 + 2 = 7.

From A to B to E, it is 1 + 6 + 6 = 13.

From A to C to F, the cost is 2 + 3 + 3 = 8.

From A to C to G, the cost is 2 + 2 + 1 = 5.
G, it can go to H whose cost is 2 + 2 + 2 + 0 = 6.
Heuristic for the Romania
problem
Drive from Arad to Bucharest using a table with straight-line distances.

h(n)
Greedy best-first search
example
Expansion rule: Expand the
node that has the lowest
of the heuristic function h(n)
value h(n)=
Greedy best-first search
example
Greedy best-first search
example
Greedy best-first search
example

Total:
140 + 99 + 211 = 450 miles
 Properties of greedy best-first
search
• Complete?
Yes – Best-first search if complete in finite
spaces.

• Optimal?
No

Total:
140 + 99 + 211 = 450 miles

Alternative through Rimnicu Vilcea:

140 + 80 + 97 + 101 = 418 miles
Implementation of greedy best-first
search

Best-Firs Expand the

frontier using
t 𝑓𝑓 𝑛 = ℎ(𝑛)

Search
Implementation of greedy best-first
search
Heuristic ℎ(𝑛) so we expand the node with the lowest estimated cost

The order for expanding

the frontier is determined
by f(n)

This check is the different to

BFS! It visits a node again if
See BFS for function itcan be reached by a
EXPAND. better (cheaper)
path.
55
Properties of greedy best-first
search
• Complete?
Yes – Best-first search if complete in finite
spaces.
• Optimal?
No d: depth of the optimal solution
m: max. depth of tree
b: maximum branching factor
• Time?
Worst case: O(bm) ⬄ like DFS
Best case: O(bm) – If ℎ(𝑛) is 100% accurate

• Space?
Same as time complexity.
How can we fix the optimality problem
with greedy best-first search?

ℎ = 1 is always better than ℎ = 2.

Greedy best-first will go this way
and never reconsider!
A* search example
Expansion rule: 𝑓𝑓 𝑛 = 𝑔(𝑛) + ℎ(𝑛) =
Expand the node with
the smallest f(n)

ℎ(𝑛)
A* search example
𝑓𝑓 𝑛 = 𝑔 𝑛 + ℎ(𝑛)

ℎ(𝑛)
A* search example
𝑓𝑓 𝑛 = 𝑔 𝑛 + ℎ(𝑛)

ℎ(𝑛)
A* search example
𝑓𝑓 𝑛 = 𝑔 𝑛 + ℎ(𝑛)

ℎ(𝑛)
A* search example
𝑓𝑓 𝑛 = 𝑔 𝑛 + ℎ(𝑛)

ℎ(𝑛)
A* search example
𝑓𝑓 𝑛 = 𝑔 𝑛 + ℎ(𝑛)

ℎ(𝑛)
BFS vs. A*
search
BFS

A*

Source: Wikipedia
Implementation of A*
Search

Best-Firs Expand the frontier

using
t 𝑓𝑓 𝑛 = 𝑔 𝑛 + ℎ(𝑛)

Search
Implementation of A*
Search
Path cost to 𝑛 + heuristic from 𝑛 to goal = estimate of the total cost
𝑔 𝑛 + ℎ(𝑛)

The order for expanding

the frontier is determined
by
𝑓𝑓(𝑛)

This check is different to

BFS! It visits a node again if
See BFS for function itcan be reached by a
EXPAND. better (cheaper)
redundant path.
67
Optimality: Admissible
heuristics
Definition: A heuristic ℎ is admissible if for every node 𝑛,
ℎ 𝑛 ≤ ℎ∗(𝑛), where ℎ∗(𝑛) is the true cost to reach the
goal state from 𝑛.
I.e., an admissible heuristic is a lower bound and
never overestimates the true cost to reach the goal.

Example: straight line distance never overestimates the

actual road distance.

Theorem: If ℎ is admissible, A* is optimal.

Proof of Optimality of 𝑓𝑓(𝑛) = 𝑔(𝑛) + ℎ(𝑛)
For goal states:
𝑓𝑓(𝑛∗) = 𝑔(𝑛) + 0
A*

n* (goal)
Any unexplored node 𝑛 has:
𝐶∗ = 𝑓𝑓(𝑛∗) = 𝑔(𝑛∗) + 0 n
𝑓𝑓 𝑛 ≥ 𝑓𝑓(𝑛∗)

n’ (other goal)
𝑔 𝑛′ ≥ 𝑓𝑓 𝑛 ⟺ 𝑔 𝑛′ ≥
𝐶∗

• Suppose A* terminates its search at goal 𝑛∗ at a cost of 𝐶∗ = 𝑓𝑓 𝑛∗ .

• All unexplored nodes 𝑛 have 𝑓𝑓 𝑛 ≥ 𝑓𝑓(𝑛∗) or they would have been explored before 𝑛∗.
• Since 𝑓𝑓(𝑛) is an optimistic estimate, it is impossible for 𝑛 to have a successor goal state 𝑛𝑛 with
𝐶𝑠 < 𝐶∗.
• This proofs that 𝑛∗ must be an optimal solution.
Guarantees of
A*
A* is optimally efficient

a. No other tree-based search algorithm that uses the same heuristic can
expand fewer nodes and still be guaranteed to find the optimal
solution.

b. Any algorithm that does not expand all nodes with 𝑓𝑓(𝑛) < 𝐶∗ (the lowest
cost of going to a goal node) cannot be optimal. It risks missing the optimal
solution.
Properties of
A*
•Complete?
Yes

•Optimal?
Yes

•Time?
Number of nodes for which 𝑓𝑓(𝑛) ≤ 𝐶∗ (exponential)

•Space?
Same as time complexity.
Designing heuristic
functions
Heuristics for the 8-puzzle
ℎ1(𝑛) = number of misplaced tiles
ℎ2(𝑛) = total Manhattan distance (number of squares from desired
location of each tile)

ℎ1(𝑠𝑡𝑎𝑟𝑡) = 8
ℎ2(𝑠𝑡𝑎𝑟𝑡) = 3 + 1 + 2 + 2 + 2 + 3 + 3 + 2 = 18

1 needs to move 3
Are ℎ1 and ℎ2 admissible? positions
Heuristics from relaxed
problems
• A problem with fewer restrictions on the actions is called a
relaxed problem.
• The cost of an optimal solution to a relaxed problem is an
admissible heuristic for the original problem. I.e., the true cost is
never smaller.
• ℎ1: If the rules of the 8-puzzle are relaxed so that a tile can move
anywhere, then ℎ1(𝑛) gives the shortest solution.
• ℎ2: If the rules are relaxed so that a tile can move to any adjacent square,
then ℎ2(𝑛) gives the shortest solution.

ℎ1(𝑠𝑡𝑎𝑟𝑡) = 8

ℎ2(𝑠𝑡𝑎𝑟𝑡)
= 3+1+2+2+2+3+3+2
Heuristics from relaxed
problems
What relaxations are used in these two
cases?
Euclidean distance Manhattan distance
Start state Start state

Goal state Goal state

Heuristics from
subproblems
•Let ℎ3(𝑛) be the cost of getting a subset of tiles
(say, 1,2,3,4) into their correct positions. The final order
of the * tiles does not matter.
•Small subproblems are often easy to solve.
•Can precompute and save the exact solution cost for every
or many possible subproblem instances – pattern
database.

** * *
* * * *
Dominance: What heuristic is
better?
Definition: If ℎ1 and ℎ2 are both admissible heuristics
and ℎ2(𝑛) ≥ ℎ1(𝑛) for all 𝑛, then
ℎ2 dominates ℎ1

Is ℎ1 or ℎ2 better for A* search?

•A* search expands every node with
𝑓𝑓(𝑛) < 𝐶∗ ⬄ ℎ(𝑛) < 𝐶∗ – 𝑔(𝑛)
•ℎ2 is never smaller than ℎ1. A* search with ℎ2 will expand
less nodes and is therefore better.
Dominanc
e
•Typical search costs for the 8-puzzle (average number
of nodes expanded for different solution depths 𝑑):

•𝑑 = 12 IDS = 3,644,035 nodes

A*(ℎ1) = 227 nodes
A*(ℎ2) = 73 nodes

•𝑑 = 24 IDS ≈ 54,000,000,000 nodes

A*(ℎ1) = 39,135 nodes
A*(ℎ2) = 1,641 nodes
Combining
heuristics
•Suppose we have a collection of
admissible heuristics ℎ1, ℎ2, … , ℎ𝑚, but
none of them dominates the others.
•Combining them is easy:
ℎ(𝑛) = max{ℎ1(𝑛), ℎ2(𝑛), … , ℎ𝑚(𝑛)}

•That is, always pick for each node the heuristic

that is closest to the real cost to the goal ℎ∗(𝑛).
Satisficing Search: Weighted A*
search
• Often it is sufficient to find a “good enough” solution if it can be found
very quickly or with way less computational resources. I.e., expanding
fewer nodes.

• We could use inadmissible heuristics in A* search (e.g., by multiplying ℎ(𝑛)

with a factor W) that sometimes overestimate the optimal cost to the goal
slightly.
1. It potentially reduces the number of expanded nodes significantly.
2. This will break the algorithm’s optimality guaranty!

f𝑛 =𝑔𝑛 + W × ℎ(𝑛)
Weighted A* search: 𝑔𝑛 +W×ℎ𝑛 (𝟏𝟏 < W < ∞)

The presented algorithms are special cases:

A* search: 𝑔 𝑛 +ℎ 𝑛 (W = 1)
Uniform cost search: 𝑔𝑛 (W = 0)
Greedy best-first search: ℎ𝑛 W=∞
 Example of weighted A*
search

BFS Exact A* Search Weighted A* Search

𝑓𝑓 𝑛 = # actions to reach n 𝑓𝑓 𝑛 = 𝑔(𝑛) + ℎ𝐸𝑢𝑐𝑙(𝑛) 𝑓𝑓(𝑛) = 𝑔(𝑛) + 5 ℎ𝐸𝑢𝑐𝑙(𝑛)

Source and Animation: Wikipedia

Memory-bounded
search
• The memory usage of A* (search tree and frontier) can still
be exorbitant.
• How can we make A* more memory-efficient while
maintaining completeness and optimality?
• Iterative deepening A* search.
• Recursive best-first search, SMA*: Forget some subtrees but remember
the best f-value in these subtrees and regenerate them later if
necessary.

• Problems: memory-bounded strategies can be complicated to

implement and suffer from “memory thrashing”
(regenerating forgotten nodes like IDS).
Implementation as Best-first
Search
• All discussed search strategies can be implemented using Best-first
search.
• Best-first search expands always the node with the minimum value of
an evaluation function 𝑓(𝑛).
Search Strategy Evaluation function 𝑓(𝑛)
BFS 𝑔(𝑛) (=uniform path cost)
Uniform-cost Search 𝑔(𝑛) (=path cost)
DFS/IDS (see note below) −𝑔(𝑛)
Greedy Best-first Search ℎ(𝑛)
(weighted) A* Search 𝑔 𝑛 + W × ℎ(𝑛)

• Note: DFS/IDS is typically implemented differently to achieve the

lower space complexity.
 Summary: Uninformed search
strategies
Time Space
Algorithm Complete? Optimal?
complexity complexity
Yes If all step 𝑂(𝑏𝑑) 𝑂(𝑏𝑑)
BFS
costs are equal

Uniform-cos Yes Yes Number of node s with 𝑔 𝑛 ≤ 𝐶∗

t Search
In finite spaces
DFS (cycle checking) No 𝑂(𝑏𝑚) 𝑂(𝑏𝑚)

If all step
IDS Yes 𝑂(𝑏𝑑) 𝑂(𝑏𝑑)
costs are equal

b: maximum branching factor of the search tree

d: depth of the optimal solution
m: maximum length of any path in the state space
C*: cost of optimal solution
 Summary: All search
strategies
Time Space
Algorithm Complete? Optimal?
complexity complexity
Yes If all step 𝑂(𝑏𝑑) 𝑂(𝑏𝑑)
BFS
costs are equal

Uniform-cos
Yes Yes Number of nodes with 𝑔(𝑛) ≤ 𝐶∗
t Search

DFS In finite spaces No 𝑂(𝑏𝑚) 𝑂(𝑏𝑚)

(cycles checking)

If all step
IDS Yes 𝑂(𝑏𝑑) 𝑂(𝑏𝑑)
costs are equal

Greedy best- In finite spaces Depends on Wor st case: 𝑂(𝑏𝑚)

No heuristic
first Search (cycles checking) t case: 𝑂(𝑏𝑑)

Bes
Number of nodes with
A* Search Yes Yes 𝑔(𝑛) + ℎ (𝑛) ≤ 𝐶∗
Conclusio
n
• Tree search can be used for planning actions for goal-based agents
in known, fully observable and deterministic environments.

• Issues are:
• The large search space typically does not fit into memory. We use
a description using a compact transition model.
• The search tree is built on the fly, and we have to deal with cycles
and redundant paths.

• IDS is a memory efficient methods used in AI often for

uninformed search.
• Informed search uses heuristics based on knowledge or percepts
to improve search (i.e., A* to expand fewer nodes).
CS 5/7320 Artificial
Intelligence

Local Local
Search Chapters 4.1 &
4.2
 Recap: Uninformed and Informed
Search

Tries to plan the Initial state

best path
from a
given initial state
to a
given goal state.
Typically searches a large
portion of the search space Goal
(needs time and memory). state
Often comes with optimality
guarantees.
 Local Search Algorithms
Local move from
state to state

x
x
Utility
U

x
y

Idea:
Start with a current solution (a state) and improve the solution by moving from the
current state to a “neighboring” better state (a.k.a. performing a series of local moves).
Local Search Algorithms
states Example:
n-Queens
Goal: Put n queens on an n × n board with no two
Problem
queens on the same row, column, or diagonal.

Defining the search problem:

State space: All possible n-queen configurations.
How many are there?
State representation: How do we define a
structured representation?
Objective function: What is a possible utility
function given the state representation?
Local neighborhood: What states are close to each
other?
4
2 conflicts = utility of -2
Example:
3 n-Queens
2

1
Problem
A B C D

0 conflicts = utility of 0
Example: Traveling salesman problem
Goal: Find the shortest tour connecting a given set of cities
State space: all possible tours (states are not individual cities!)
State representation: Order of cities in the tour.
Objective function: minimize the length of the tour
Local neighborhood: Change the order of visiting a few cities.
Note: We have solved a different problem with uninformed/informed search! Each
city was defined as a state and the path was the solution.
Hill-Climbing Search
aka Greedy Local Search
Idea: keep a single “current” state and try to find better
neighboring states.

MotorCycleUSA.com
 Example:
Goal: Put n queens on an n × n board with no two queens on the same row, column,

n-Queens Problem
or diagonal.
State space: all possible n-queen configurations. We can restrict the state space:
Only one queen per column.
State representation: row position of each queen in its column (e.g., 2, 3, 2, 3)
Objective function: minimize the number of pairwise conflicts.
Local neighborhood: Move one queen anywhere in its column.

Improvement strategy
Find a local neighboring state (move one queen within its column) to reduce
conflicts

1
 Example:
n-Queens Problem
To find the best local move, we must evaluate all local neighbors (moving a queen in
its column) and calculate the objective function.
Objective value after moving
the queen to this square
Example:
n-Queens Problem

Remember: This makes

the problem a lot easier.
 Example: Traveling Salesman
Problem
Goal: Find the shortest tour connecting n cities
State space: all possible tours
State representation: tour (order in which to visit the cities) = a
permutation
Objective function: length of tour
Local neighborhood: reverse the order of visiting a few cities

Local move to reverse the order of cities C, E and D:

A B A B

C C

D E D E

State representation: ABDEC ABCED

 Example: Traveling Salesman
Problem

Local move to reverse the order of cities C, E and D:

A B A B

C C

D E D E

State representation: ABDEC ABCED

 Hill-Climbing Search (= Greedy Local
Search)

Typically, we start with a random state

Variants:
Steepest-ascend hill climbing
Check all possible successors and choose the highest-valued successors.

Stochastic hill climbing

choose randomly among all uphill moves, or
generate randomly one new successor at a time until a better one is found =
first-choice hill climbing – the most popular variant, this is what people often
mean when they say “stochastic hill climbing”
 Local Optima
Hill-climbing search is like greedy best-first search with the objective
function as a (maybe not admissible) heuristic and no frontier (just stops in
a dead end).
Is it complete/optimal?
No – can get stuck in local optima

Example: local optimum for the

8-queens problem. No single
queen can be moved within its
column to improve the objective
function.

Simple approach that can help with local optima:

Random-restart hill climbing
• Restart hill-climbing many times with random initial states and return the best
solution.
The State Space
“Landscape”
max.

Neighbors placed
next to each other
How to escape local maxima?
Random restart hill-climbing can help.
What about “shoulders” (called “ridges” in higher dimensional space)?
Hill-climbing that allows sideways moves and uses momentum.
Minimization vs.
Maximization
Convex vs. Non-Convex
Optimization Problems
Minimization problems
Convex Problem Non-convex Problem

One global optimum +

Many local optima hard
smooth function calculus
makes it easy
Many discrete optimization
problems are like this.
Simulated Annealing
Use heat to escape local optima…
Simulated Annealing
Idea: First-choice stochastic hill climbing + escape local minima by
allowing some “bad” moves but gradually decrease their frequency.
Inspired by the process of tempering or hardening metals by decreasing
the temperature (chance of accepting bad moves) gradually.
“bad” local move
Objective function (minimize)

State space
Simulated Annealing

Typically, we start with a random state

VALUE(next) – Value(current) Always do good moves

Uses the Metropolis

acceptance criterion to
accept “bad” moves
Note: Use VALUE(current) – VALUE(next) for minimization
 The Effect of
Temperature
Prob. of accepting a worse state exp(ΔE/T)

ΔE (neg. means the solution gets worse)

The lower the temperature, the less likely the algorithm will accept a worse state.
Cooling Schedule
 Simulated
Annealing Search
Guarantee: If temperature decreases slowly
enough, then simulated annealing search will
find a global optimum with probability
approaching one.
However:
This usually takes impractically long.
Evolutionary Algorithms
A Population-based Metaheuristics
 Evolutionary Algorithms / Genetic
Algorithms

A metaheuristic for population-based optimization.

Uses mechanisms inspired by biological evolution
(genetics):
Reproduction: Random selection with probability based on a fitness function.
Random recombination (crossover)
8
Random mutation 7
6
Repeated for many generations 5
4
3
2
Example: 8-queens problem 1

Individual = state
representation as
a chromosome:
row of the queen
in each column
next
generation
Search in Continuous Spaces
Discretization of Continuous Space

x
 Discretization of Continuous Space

How did we discretize this space?

Initial state

Goal
state
Search in Continuous Spaces:
Gradient Descent
 Search in Continuous Spaces:
Empirical Gradient Methods

What if the mathematical formulation of the objective function is

not known?
We may have objective values at fixed points, called the training
data.
In this case, we can estimate the gradient using the data and use
empirical gradient search.
We will talk more about search in continuous spaces with loss
functions using gradient descend when we talk about parameter
learning for machine learning.