INT6070

Advanced Programming for

Artificial Intelligence

Searching Algorithms

For some of the lectures in this course, we will use “Word” style rather than “PowerPoint” style notes to facilitate explanation of ideas and review.
Searching 搜索是一种系统地寻找或构造问题的解决方案的技术。

⚫ Searching are the techniques for systematically finding or

constructing solutions to problems.
⚫ Basic technique: generate-and-test. 基本技术：生成和测试。
1. Generate (or find) a possible candidate solution.
2. Test the candidate solution.
2.1 IF solution is found, announce the result
2.2 ELSE go to step 1.
1.生成（或查找）可能的候选解决方案。
2.测试候选方案。
2.1 IF找到解决方案，宣布结果
2.2 ELSE转至步骤1。
Searching in AI
⚫ Searching techniques are used in many AI problems
⚫ Examples
◼ Path planning

◼ Two players Games

◼ Natural Language Processing

人工智能搜索的基础：树和图形

Basic of Searching in AI: Trees and Graphs

人工智能中的许多搜索算法都是基于树和图的
⚫ Many searching algorithms in AI are based on trees and graphs

Tree Graph
Tree 树有许多节点（一个根节点、叶节点和其他节点）
⚫ A tree has a number of nodes (A root node, leaf nodes, and other

nodes)
每个节点都可能有通向其他节点的链接。
⚫ Each node may have links leading to other nodes.
每个链路将一个父节点与一个子节点连接起来
⚫ Each link connect a parent node with a child node
从根节点到某个节点的路由称为其路径
⚫ The route from the root node to a certain node is called its path
树不能包含循环（到任何节点的根路径）
⚫ A tree cannot contain cycles (only one path form root to any node)
Graph
一个图也由节点和链接组成
然而，在图中允许循环（循环）（这对于基
⚫ A graph also consist of nodes and links 于树的搜索算法来说可能是个问题）

⚫ However, cycles (loops) are allowed in graphs (which can be

problematic for tree-based searching algorithms)

⚫ Examples

- Road networks in a country

Undirected graph
- Computer networks

- Social media friends connections

Directed graph
Graphs and Trees
许多搜索算法都是基于树形的（不允许进行循环）。
⚫ Many searching algorithms are tree-based (no loops allowed).
然而，许多问题实际上都是图。
⚫ However, many problems are actually graphs.
解决方案：使用树表示一个图，在必要时重复一些节点
⚫ Solution: represent a graph using a tree, repeating some nodes if

necessary

- The tree on the right is a tree-representation of the graph on

the left (note that some nodes are repeated).

右边的树是左边图的树表示（注意，一些节点是重复的）。
Representing Problems using Trees (or Graphs)

Example: 8-puzzle problem

We want to find a way to transform an initial state (e.g., the example on

the left) to a goal state (the example on the right) by applying a series of

operators (e.g., moving an adjacent number into the blank square)

Initial State Goal State

Try it out:

https://murhafsousli.github.io/8puzzle/#/
我们希望找到一种方法，通过应用一系列操作符（例如，将相邻的数字移动到空
白方块中），将初始状态（例如，左边的例子）转换到目标状态（右边的例子）
在这种情况下，我们感兴趣的是从根状态到目标状态（隐藏在树的某个地方）的路径
⚫ In this case, we are interested in the path that leads from the root to

a goal state (hidden somewhere down the tree below)

节点：可能的状态。 链接：操作（即，移动一个数字）。
⚫ Node: a possible state. Link: operation (i.e., moving a number).

Tree:

- Note: We can only examine one node (state) at a time!

注意：我们一次只能检查一个节点（状态）！
8-puzzle problem

Input:

• What does each node represent?

• A state of the puzzle (the position of each

piece (0 – 8, blank))

• Start state (The root node)

• A given starting position (varies case by

case) 给定的起始位置（因情况而异）

• Operators

• Swapping the blank square with a neighbor

square (UP/DOWN/LEFT/RIGHT)

• Cost per move: uniform cost (1 per move)

• Test for goal state

• Are the pieces in pre-defined goal positions?

Output:

• A path from start state to goal state

[Some problem may require the shortest path]

We will implement a simple AI to solve this problem.

Example: Route planning problem

E.g., find the shortest path from Chengdu to Harbin

Graph:
Route planning problem

Input:

• What does each node represent?

• A state of the puzzle (the location of the

traveler)

• Start state (The root node)

• A given city (e.g., Chengdu)

• Operators

• Travel to a connected city

• Cost per move: the distance travelled

• Test for goal state

• The current state equals some given goal

state (e.g., Harbin)

Output:

• Path: a path from start-state (e.g.,

Chengdu) to goal-state (e.g., Harbin)

• Shorter paths are more preferable

Common Searching Algorithms
不知情的搜索（盲搜索）
⚫ Uninformed search (blind search)
◆ Given no extra knowledge on how to find the solution.
◆ Examples 因为没有关于如何找到解决方案的额外知识。
⚫ Depth first search
⚫ Breadth first search
⚫ Informed search (heuristic) 知情搜索（启发式）
◆ Utilize problem-specific knowledge to speed up the search
◆ Example: Hill climbing 利用特定于问题的知识来加快搜索速度
Depth First Search (DFS)
在尝试下一个分支之前，尽量跟随树的某个分支
Try to follow a certain branch of the tree as deep as possible, before
trying the next branch

本示例中的DFS将按顺序检查节点
For example: DFS in this example would inspect the nodes in the order
[A, B, D, E, C, F, G]

Or, alternatively, from right to left as

[A, C, G, F, B, E, D]
Breadth First Search (BFS)
节点被逐级访问（从上到下，从左到右或从左到右）
Nodes are visited level by level (from top to bottom, from left to right
OR from left to right)

BFS would inspect the nodes in the order

A, B, C, D, E, F, G

or, Alternatively, from right to left as

A, C, B, G, F, E, D
 Note: 我们一次只能检查一个节点。
⚫ We can only inspect one node at a time.
⚫ We can only check whether a node is the goal state when we get
there. 当我们到达节点时，我们只能检查节点是否是目标状态。
⚫ We often do not know what further options are available (i.e.,
whether a node has children) until we arrive at the node.
在我们到达该节点之前，我们通常不知道还有哪些其他可用的选项（例如，一个节点
是否有子节点）。
Exercise 1
Show the order of visiting each node if the

tree is searched using i) Depth First Search, ii)

Breadth First Search

DFS:
[A, B, E, I, J, C, F, G, K, L, H, D]
A B E I J C F G K L H D
or, from right to left:
A D C H G L K F B E J I
[A, D, C, H, G, L, K, F, B, E, J, I]

BFS
[A, B, C, D, E, F, G, H, I, J, K, L]

or from right to left:

A B C D E F G H I J K L
[A, D, C, B, H, G, F, E, L, K, J, I]
A D C B H G F E L K J I
Tree Searching Algorithm
(Basic version)
1. Start at the root node (called the next node).
设置已访问节点的列表（最初为空）
2. Setup a list of visited nodes (initially empty)
设置to_visis节点列表（最初为空）
3. Setup a list of to_visit nodes (initially empty)
重复以下操作，直到成功或失败：
4. Repeat until SUCCESS or FAILURE:
如果以前没有访问过下一个节点：
i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the

end of the to_visit list

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove one of the nodes from the list

and select it as the next node

B. Else if no more nodes in the list

Report FAILURE
Tree Searching Algorithm (Basic version)

We will now illustrate the idea using this tree

Start

Goal
Tree Searching algorithm: Basic version

1. Start at the root node (called the next node).

2. Setup a list of visited nodes (initially empty)

3. Setup a list of to_visit nodes (initially empty)

⚫ next_node = A Tree:

⚫ visited = [ ]
⚫ to_visit = [ ]
Tree Searching algorithm: Basic version
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

[Not shown here]

⚫ next_node = A Tree:

⚫ visited = [A]
⚫ to_visit = [ ]
Tree Searching Algorithm: Basic Idea
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

iii. Look at the to_visit list

[Not shown here]

Tree:
⚫ next_node = A
⚫ visited = [A]
⚫ to_visit = [B,C]
Tree Searching Algorithm: Basic Idea
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

A. If there are more nodes in the list

从列表中删除其中一个节点，并选择它作为下一个节点
Remove one of the nodes from the list and select it as the next node

B. Else If no more nodes in the list

Tree:
Report FAILURE

⚫ next_node = ?
⚫ visited = [A]
⚫ to_visit = [ ? ] #(was: [B, C])
 A. If there are more nodes in the list

Remove one of the nodes from the list and select it as the next node

从to_visisList中删除哪个节点？
Question: which node to remove from the to_visit List?
The first one? The last one? Random?

Answer: It depends:

For DFS: remove the last one (i.e., from the END of the LIST)
For BFS: remove the first one (i.e., from the FRONT of the LIST)
Depth First Search Algorithm
1. Start at the root node (called the next node).

2. Setup a list of visited nodes (initially empty)

3. Setup a list of to_visit nodes (initially empty)

4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the

end of the to_visit list

iii. Look at the to_visit list

A. If there are more nodes in the list

Remove one node from the END of the

list and select it as the next node

B. Else If no more nodes in the list

Report FAILURE
Breadth First Search Algorithm
1. Start at the root node (called the next node).

2. Setup a list of visited nodes (initially empty)

3. Setup a list of to_visit nodes (initially empty)

4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the

end of the to_visit list

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove one node from the FRONT of the

list and select it as the next node

B. Else If no more nodes in the list

Report FAILURE
我们将继续使用深度优先搜索继续前面的示例
We will now continue the previous example using Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove the LAST nodes from the list and select it as the next node

B. If no more nodes in the list

Tree:
Report FAILURE

⚫ next_node = C
⚫ visited = [A]
⚫ to_visit = [B C] #(was: [B, C])
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

[Not shown here]

Tree:
⚫ next_node = C
⚫ visited = [A, C]
⚫ to_visit = [B C]
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

⚫ next_node = C
⚫ visited = [A, C]
⚫ to_visit = [B, F, G]
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove the LAST nodes from the list and select it as the next node

B. Else If no more nodes in the list

Report FAILURE Tree:

⚫ next_node = G
⚫ visited = [A, C]
⚫ to_visit = [B,F ] #(was: [B, F, G])
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

ii. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

iii. Look at the to_visit list

Tree:
[Not shown here]

⚫ next_node = G
⚫ visited = [A, C, G]
⚫ to_visit = [B, F]
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

⚫ next_node = G
⚫ visited = [A, C, G]
⚫ to_visit = [B, F] (No child to add)
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove the LAST nodes from the list and select it as the next node

B. Else if no more nodes in the list

Report FAILURE Tree:

⚫ next_node = F
⚫ visited = [A, C, G]
⚫ to_visit = [B] (was: B, F)
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

⚫ next_node = F
⚫ visited = [A, C, G, F]
⚫ to_visit = [B] (F has no child to add)
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

C. If there are more nodes in the list

Remove the LAST nodes from the list and select it as the next node

D. Else if no more nodes in the list

Report FAILURE Tree:

⚫ next_node = B
⚫ visited = [A, C, G, F]
⚫ to_visit = [ ] (was: B)
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]
⚫ next_node = B
⚫ visited = [A, C, G, F, B]
⚫ to_visit = [ ]
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

⚫ next_node = B
⚫ visited = [A, C, G, F, B]
⚫ to_visit = [D, E]
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove the LAST nodes from the list and select it as the next node

B. Else if no more nodes in the list

Report FAILURE Tree:

next_node = E
visited = [A, C, G, F, B]
to_visit = [D ]
Tree Searching Algorithm: Depth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

next_node = E (Goal!)
visited = [A, C, G, F, B, E]
to_visit = [D ]
Note: The path from root to goal is [A, B, E] Goal
Next, we repeat the search using Breadth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove the FIRST nodes from the list and select it as the next node

B. If no more nodes in the list

Tree:
Report FAILURE

⚫ next_node = ?
⚫ visited = [A]
⚫ to_visit = [ ? ] #(was: [B, C])
Tree Searching Algorithm: Breadth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

next_node = B
visited = [A, B]
to_visit = [ C ]
Tree Searching Algorithm: Breadth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

next_node = B
visited = [A, B]
to_visit = [ C, D, E ]
Tree Searching Algorithm: Breadth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove the FIRST nodes from the list and select it as the next node

B. If no more nodes in the list

Tree:
Report FAILURE

⚫ next_node = C
⚫ visited = [A, B ]
⚫ to_visit = [ D, E] (was: C, D, E)
Tree Searching Algorithm: Breadth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

next_node = C
visited = [A, B, C]
to_visit = [ D, E] (was: C, D, E)
Tree Searching Algorithm: Breadth First Search
5. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

iii. Look at the to_visit list

Tree:
[Not shown here]

next_node = C
visited = [A,B,C]
to_visit = [D,E,F,G]
Tree Searching Algorithm: Breadth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove the FIRST nodes from the list and select it as the next node

B. If no more nodes in the list

Tree:
Report FAILURE

⚫ next_node = D
⚫ visited = [A, B, C]
⚫ to_visit = [E,F,G] (was:[D,E,F,G])
Tree Searching Algorithm: Breadth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

next_node = D
visited = [A, B, C, D]
to_visit = [E, F, G]
Tree Searching Algorithm: Breadth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

[Not shown here]

ii. Look at the to_visit list

A. If there are more nodes in the list

Remove the FIRST nodes from the list and select it as the next node

B. If no more nodes in the list

Tree:
Report FAILURE

⚫ next_node = E
⚫ visited = [A, B, C, D]
⚫ to_visit = [F,G] (was: E, F, G)
Tree Searching Algorithm: Breadth First Search
4. Repeat until SUCCESS or FAILURE:

i. If the next node has NOT been visited before:

A. Add the next node to visited nodes list

B. If it is the goal, report SUCCESS, and stop.

C. If it is not the goal, add its children to the end of the to_visit list

ii. Look at the to_visit list

Tree:
[Not shown here]

next_node = E GOAL REACHED

visited = [A, B, C, D, E]
to_visit = [F, G]

Goal
Depth First Search
Implementation (Version 1)
In this example, we represent a tree in a Python

program using a dictionary as follow:

graph = {1: [2 , 3],

2: [4, 5],

3: [6, 7]}

goal = 5

Goal

And a simple function for checking whether a

given node (or a state) is the goal:

def is_goal(x):

return x==goal
Depth First Search
Implementation (Version 1)
And the search function closely follows the

algorithm described above.

 ⚫ Note that, in DFS version 1, we are directly

referring to the pre-defined dictionary for the

graph data:

然而，在一些问题中，预先预先定义整个graph是不现实的。
⚫ However, in some problems, it is not practical

to pre-define the whole graph in advance.

⚫ To take care of this possibility, we can replace

the line marked in red by a more generic

find_children( )function for generating the

children, which can be changed for different

problems, as follows:
为了解决这种可能性，我们可以用更通用的find_chiilds（）函数替换红
色标记的行，以生成子代，可以针对不同的问题进行更改，如下所示：
Depth First Search
Implementation (Version 2)
上述版本在查找目标状态时工作得很好。但是，它不记得从根目录到
目标的路径。
Depth First Search Implementation
(Version 3): Remember the path
⚫ The above versions work fine in finding a goal
state. However, it does not remember the path

from the root to the goal.

The path is
[1, 2, 5]

Goal

为此，我们可以记录从根到每个节点的路径，通过将其存储在to_
visit List中的每个状态旁边。
⚫ To do so, we can record the path from the root
to each node, by storing it next to each state
in the to_visit List.
因此，to_visit列表中的每个元素现在是一个两部分的元组（state,
path），
⚫ Thus, each element of the to_visit list is
其中
now a 2-part tuple (state, path), where
state是问题中的一种状态（与以前相同）
◼ state is a state in the problem (same as

before)
path是从根节点到当前节点的状态列表
◼ path is a list of the states from the root to

the current node

Exercise 2: Breadth First Search
Implementation
通过修改DFS版本2的程序来实现BFS。
⚫ Implementation BFS by modifying the program

of DFS version 2.

⚫ Hint: Which line(s) (see below) should we

change?

⚫ Test your program using the same graph as

before. Show the order in which the nodes are

visited.

⚫
Exercise 3: Maze
Write a program in Python to find a path through

the following maze using DFS!

Hints:

⚫ DFS version 3 could be useful.

⚫ What are the states?

⚫ What does the tree look like?

⚫ How can we represent the tree represented in

Python?

For your information, the solution of the maze is

[A, B, D, I, K, M, N]
Comparison of DFS and BFS
Depth First Search Breadth First Search
Problems Solutions occur deep in the tree Solutions occur at a reasonable level

suitable (not too deep)

Time Potentially large (exponential) Potentially large (exponential)

requirement O(bm) O(bm)

Memory Modest (linear). O(b* m) Potentially large (exponential) O(bm)

requirement Need to store to_visit nodes Need to store all nodes of the current

mainly in a single branch only at level in the to_visit.

any time

Complete? Cannot find solution if a branch Can always find solution (if it exists)

can be infinitely deep

Note. b is the branch factor and m is the deepest level. O(n) means it is roughly proportional to n