Professional Documents
Culture Documents
Module 1
Module 1
Module 1
Module 1
Definition
It is the study of how to make computers do things which at the moment people
do better. It fails to include some areas of potentially large impact namely problems that
cannot now be solved well by either computers or people.
Intelligence +System AI
Applications of AI
1. Game playing
You can buy machines that can play master level chess for a few hundred dollars.
There is some AI in them, but they play well against people mainly through brute force
computation--looking at hundreds of thousands of positions. To beat a world champion
by brute force and known reliable heuristics requires being able to look at 200 million
positions per second.
2. Speech recognition
In the 1990s, computer speech recognition reached a practical level for limited
purposes. Thus United Airlines has replaced its keyboard tree for flight information by a
system using speech recognition of flight numbers and city names. It is quite convenient.
On the other hand, while it is possible to instruct some computers using speech, most
users have gone back to the keyboard and the mouse as still more convenient.
3. Understanding natural language
Just getting a sequence of words into a computer is not enough. Parsing sentences
is not enough either. The computer has to be provided with an understanding of the
domain the text is about, and this is presently possible only for very limited domains.
4. Computer vision
The world is composed of three-dimensional objects, but the inputs to the human
eye and computers' TV cameras are two dimensional. Some useful programs can work
solely in two dimensions, but full computer vision requires partial three-dimensional
information that is not just a set of two-dimensional views. At present there are only
limited ways of representing three-dimensional information directly, and they are not as
good as what humans evidently use.
5. Expert systems
6. Heuristic classification
One of the most feasible kinds of expert system given the present knowledge of
AI is to put some information in one of a fixed set of categories using several sources of
information. An example is advising whether to accept a proposed credit card purchase.
Information is available about the owner of the credit card, his record of payment and
also about the item he is buying and about the establishment from which he is buying it
(e.g., about whether there have been previous credit card frauds at this establishment).
7. Commonsense reasoning
State space search is a powerful technique for solving problems. However, in order to
apply this technique we need to formulate the problem. A problem can be defined
formally by 4 components:
Initial state :- Specify one or more states within that space that describe possible
situations from which the problem solving process may start. These states are
called the initial sates.
Successor function S(x) :- Description of all possible actions available. Given a
particular state x, S(x) returns a set of < action, successor> ordered pairs in which
each action is one of the legal actions in state x and each successor is a state that
can be reached from x by applying the action.
Goal Test :– Goal states are one or more states that would be acceptable as
solutions to the problem. Goal test determines whether a given state is a goal
state. They may or may not be explicitly specified.
Path cost :- Function that assigns a numeric cost to a path. It is usually the sum
of the costs of individual actions along the path. Step cost of taking action ‘a’ to
go from state ‘x’ to ‘y’ is denoted by c(x,a,y).
Problem Space is the environment in which search takes place. Problem Instance is
the problem space combined with initial and goal state.
State space is the set of all states reachable from the initial state.(initial state +
successor function).State space forms a graph in which nodes are states and the arcs
between nodes are actions. A path in state space is a sequence of states connected by a
sequence of actions.
A solution is a sequence of actions (path) leading from the initial state to a goal state.
Solution quality is measured by the path cost function. An Optimal solution has the
lowest path cost among all solutions.
Example Problems
1. 8 Puzzle
The eight puzzle consists of a three by three board with eight numbered tiles and a
blank space. A tile adjacent to the blank space can slide into the space. A game consists
of a starting position and a specified goal position. The goal is to transform the starting
position into the goal position by sliding the tiles around. The object is to figure out the
steps needed to get from one configuration of the tiles to another.
Arrange the tiles so that all the tiles are in the correct positions. You do this by
moving tiles. You can move a tile up, down, left, or right, so long as the following
conditions are met:
A) there's no other tile blocking you in the direction of the movement; and
B) you're not trying to move outside of the boundaries/edges.
For example, suppose we wanted to get from the initial state to the goal state given
below.
and the is a blank tile you can slide around. This 8 puzzle instance can be solved in 5
steps.
1. States – A state description specifies the location of each of the eight tiles and the
blank in one of the nine squares.
2. Initial state – Any state can be designated as the initial state. Any given goal can be
reached from exactly half of the possible initial states.
3. Successor function. – generates the legal states that result from trying the four
actions(blank moves left, right, up, down).
4. Goal test – this checks whether the state matches the goal configuration.
5. Path cost - Number of actions to reach goal.
2. 8-Queens Problem
7 X
6 X
5 X
4 X
3 X
2 X
1 X
0 X
0 1 2 3 4 5 6 7
Although efficient special purpose algorithms exist for this problem and the
whole ‘n’ queens family, it remains an interesting test problem for search algorithms.
There are 2 main kinds of formulation.
1.Incremental formulation – involves operators that augment the state description
starting with an empty state , ie each action adds a queen to the board.
2.Complete-state formulation – starts with all 8 queens on the board and moves them
abroad.
In either case the path cost is of no interest because only the final state counts.
For eg: Incremental formulation
1. States - Any arrangement of 0 to 8 queens on the board
2. Initial state - No queens on the board
3. Successor function - Add a queen to any empty square
4. Goal test - 8 queens on board and none attacked
3. Water-Jug Problem
You are given two jugs a 4-gallon one and other 3-gallon one. Neither has any
measuring marker on it. There is a pump that can be used to fill the jugs with water. The
Problem : How can you get exactly 2 gallons of water into the 4-gallon jug?
The State Space for this problem can be described as a set of Ordered pairs of
integers (x, y) such that x = 0, 1, 2, 3, or 4 and y = 0, 1, 2 , or 3; x represents the number
of gallons of water in the 4-gallon jug and y represents the number of gallons of water in
the 3-gallon jug.
The Start State is (0, 0).The Goal States are (2, 0), (2, 1), (2, 2), (2, 3).The actions
include :
4. Towers of Hanoi
There are three posts 1, 2, and 3. One of which, say post 1, has ‘n’ disks on it in
ascending size. The Problem is: How to transfer all disks from the post 1 to post 2,
keeping the order of ascending size. Each time only one disk can be moved. Post 2 is
used for the intermediate storage
The goal of this puzzle is start with the rings arranged as shown above, and to move them
all to this configuration:
Only the top ring on a peg can be moved, and it may only be placed on a smaller ring, or
on an empty peg. For the Tower of Hanoi, the problem space is the set of all possible
configurations of rings on the pegs. Some of the configurations or states in the space are
given special interpretations:
Initial States
States where a given episode of problem solving starts. In the Tower of Hanoi
puzzle, there is one initial state, as shown above. In other domains there may be a number
of possible initial states.
This is also a Route finding problem. A salesperson has to visit a number of cities.
(S)He can start at any city and must finish at that same city. The salesperson must visit
each city only once. A simple control structure could in principle solve the problem. It
would simply explore all possible paths in the tree and return the one with the shortest
length. This approach will work for very short list of cities. But it breaks down quickly as
the no of cities grows.
If there are n cities then the number of possible paths among them is
1*2*3*4………*(n-1) or (n-1)! .The time required to examine a single path is
proportional to n!. Assume there are 35 cities to visit. To solve this problem he would
take more time than he would be willing to send. This phenomenon is called
combinatorial explosion.
We can beat this using a simple strategy called Branch and bound. Begin
generating complete paths keeping track of the shortest path found so far. Give up
exploring any path as soon as its partial length becomes greater than the shortest path
found so far. Using this technique we are still guaranteed to find the shortest path.
Although this algorithm is efficient than the first one it still requires exponential time.
3. VLSI layout
4. Robot Navigation
Problem Characteristics
We can solve this problem by breaking down it into 3 smaller problems each of
which we can be solved independently. Decomposable problem can be solved easily.
This can be solved by a simple integration program that works as follows. At each
step it checks to see whether the problem it is working on is immediately solvable. If
so then the answer is returned directly. If the problem is not easily solvable, the
integrator checks to see whether it can decompose the problem into smaller problems.
If it can it creates those problems and calls itself recursively on them. The problem
tree is shown below:
29
In attempting to solve the 8-Puzzle we might make a stupid move. For eg in the
game shown above we might start by sliding tile 5 into the empty space. Having done
that we cannot change our mind and immediately slide tile 6 into the empty space since
the empty space will essentially have moved. But we can backtrack and undo the first
move sliding tile 5 back to where it was. Then we can move tile 6.
Problem Classification
1) Ignorable problems - in which solution steps can be ignored. For eg in the case
of Theorem Proving a lemma that has been proved can be ignored for next steps.
Ignorable problems can be solved using a simple control structure that never
backtracks. Such a control structure is easy to implement.
In Bridge we cannot know exactly where all the cards are or what the other
players will do on their turns. What we would like to do is to plan the entire hand before
making the first play. These problems are called uncertain outcome problems. For
uncertain-outcome problems, a sequence of generated operators can only have a good
probability of leading to a solution.
Plan revision is made as the plan is carried out and the necessary feedback is
provided. One of the hardest types of problems to solve is the irrecoverable uncertain
outcome. Eg : controlling a robot arm.
Different reasoning paths lead to the answer. It does not matter which path we
follow. For eg in the case of travelling Salesman Problem we have to try all paths to find
the shortest one. Any-path problems can be solved in a reasonable amount of time using
heuristics that suggest good paths to explore. If the heuristics are not perfect the search
for the solution may not be as directly as possible. For best-path problems, much more
exhaustive search will be performed. Best –path problems are computationally harder
than any – path problems.
Consider the problem of playing chess. Suppose you had unlimited computing
power available. How much knowledge would be required by a perfect program? It
requires just the rules for determining legal moves and some simple control mechanism
that implements an appropriate search procedure. In this case knowledge is important
only to constrain the search for a solution.
But now consider the problem of scanning daily newspaper to decide who will
win in the upcoming elections. Suppose you had unlimited computing power available
.But in this case lot of knowledge is required even to be able to recognize a solution.
For eg while proving mathematical theorem it should be made sure that the proof
exists. If there is that much difficult it may not know where to start.
Problem solving agents decide what to do by finding sequences of actions that lead to
desirable states. Search algorithms classified as
1. Uninformed Search/Blind Search - in the sense that they are given no information
about the problem other than its problem definition.
2. Informed Search/Heuristic Search – ones that have some idea of where to look for
solutions.
Goals help organize behavior by limiting the objectives that the agent is trying to
achieve. Goal formulation based on the current situation and the agent’s performance
measure is the first step in problem solving. The agent’s task is to find out which
sequence of actions will get it to a goal state. Problem formulation is the process of
deciding what actions and states to consider given a goal.
An agent first formulates a goal and a problem, searches for a sequence of actions
that would solve the problem and then executes the actions one at a time. The process of
looking for such a sequence is called search. A search algorithm takes a problem as input
and returns a solution in the form of an action sequence. Once a solution is found the
actions it recommends can be carried out. This is called execution phase. Thus we have a
simple formulate, search, execute design for an agent.
Usually the environment is assumed to be static because formulating and solving
the problem is done without paying attention to any changes that might be occurring in
the environment.
Solution can be found out by searching in state space. We can generate a search
tree that is generated by the initial state and the successor function that together define
the state space. The root of search tree is a search node corresponding to the initial state.
The first step is to find whether it is a goal state. If it is not we need to consider other
states. This is done by expanding the current state ie applying the successor function to
the current state thereby generating a new set of states.
1. State – the state in the state space to which the node corresponds
2. Parent node – node in the search tree that generated this node
3. Action – action that was applied to the parent to generate the node.
4. Path cost – cost of the path from the initial state to the node as indicated by parent
pointers
5. Depth – no of steps along the path from the initial state.
Collection of nodes that have been generated but not yet expanded is called
fringe. Each element of the fringe is a leaf node i.e. a node with no successors in the tree.
Time and space complexity are measured in terms of problem difficulty defined by:
b - Maximum branching factor of the search tree
d - Depth of the least-cost solution
m - Maximum depth of the state space (may be infinity)
Time is often measured in terms of no of nodes generated during search and space
in terms of max no of nodes stored in memory.
Production Systems
Productions consist of two parts: a sensory precondition (or "IF" statement) and
an action (or "THEN"). If a production's precondition matches the current state of the
world, then the production is said to be triggered. If a production's action is executed, it is
said to have fired. A production system also contains a database, sometimes called
working memory, which maintains data about current state or knowledge, and a rule
interpreter. The rule interpreter must provide a mechanism for prioritizing productions
when more than one is triggered.
Matching
The rules of a production consist of a condition and action in the form: (if x then
y). The left-hand-side conditions (x and y may be arbitrarily complex conjunctions of
expressions) are compared against the elements of working memory to determine if the
conditions are satisfied.
Conflict Resolution
At any point in processing, several productions may match to the elements of
working memory simultaneously. Since production systems are normally implemented on
serial computers, this result in a conflict: there is a non-unique choice about which action
to take next. Most conflict resolution schemes are very simple, dependent on the number
of conditions in the production, the time stamps (ages) of the elements to which the
conditions matched, or completely random. One of the advantages of production systems
is that the computational complexity of the matcher, while large, is deterministically
finite.
Actions
Search Strategies
1. Exhaustive (uninformed) search
Breath-first search
Uniform-cost search
Depth-first search
Depth-limited search
Iterative deepening search
Bidirectional search
Uninformed/Blind Search
1. Breadth First Search
It is a simple strategy in which root node is expanded first then all successors of
root node are expanded next and then their successors and so on. Consider again the
search space shown below:
Let us suppose that the state labeled G is a goal state in this space, and that, as shown, no
operators apply to the states I, J, E, F and H.
A program will start with the initial state A, and try to find the goal by applying
operators to that state, and then to the states B and/or C that result, and so forth. The idea
is to find a goal state, and often one is also interested in finding the goal state as fast as
possible, or in finding a solution that requires the minimum number of steps, or satisfies
some other requirements.
One approach to deciding which states that operators will be applied to is called
"Breadth-First Search". In a breadth-first search, all of the states at one level of the tree
are considered before any of the states at the next lower level. So for the simple search
state pictured above, the states would be applied in the order as indicated below:
Algorithm:
Usually BFS is implemented using data structure called queue. You add the newly
formed paths to the back of the list. When you remove them to expand them, you remove
them from the front.
Completeness: Does it always find a solution if one exists? YES, if shallowest goal node
is at some finite depth d
Time complexity: Assume a state space where every state has b successors. Root node
has b successors, each node at the next level has again b successors (total b2), and so on.
Assume solution is at depth d. In Worst case we would expand all but the last node at
depth d since the goal itself is not expanded generating bd+1 –b nodes at level d+1.Then
total number of nodes generated is
Advantages:
1. BFS will not get trapped anywhere .This contrasts with DFS which may follow a
single unfruitful path for a very long time before the path actually terminates in a
state that has no successors.
2. If there is solution then BFS is guaranteed to find it. Furthermore if there are
multiple solutions then a minimal solution will be found. This is guaranteed by
the fact that longer paths are never explored until all shorter ones have already
been examined. This contrasts with DFS which may find a long path to a solution
in one part of the tree when a shorter path exists in some other unexplored part of
the tree.
Disadvantages:
Uniform-cost search is similar to breadth-first search. BFS is optimal when all the
step costs are equal, because it always expands the shallowest unexpanded node. By a
simple extension we can find an algorithm that is optimal with any step cost function.
Instead of expanding the shallowest node UCS expands the node with lowest path cost.
UCS is the same as BFS when all step-costs are equal. This is implemented by means of a
queue ordered by path cost.
We associate with each node n a cost g(n) that measures the cost of getting to
node n from the start node. g(start) = 0. If ni is a successor of n, then g(ni) = g(n) +
c(n,ni) , where c(n,ni) is the cost of going from node n to node ni .
Time complexity: Assume C* the cost of the optimal solution. Assume that every action
costs at least ε. In the Worst-case time complexity will be O (b C*/ ε)
Optimality: It will be optimal since nodes expanded in order of increasing path cost.
(Also if complete)
Advantage:
1. Guaranteed to find the least-cost solution.
Disadvantages:
DFS is another control strategy which always expands the deepest node. We
could pursue a single branch of the tree until it yields a solution or until a decision to
terminate the path is made. In depth-first search, the search is conducted from the start
state as far as it can be carried until either a dead-end or the goal is reached. If a dead-end
is reached, backtracking occurs.
The most recently created state from which alternative moves are available will
be revisited and a new state is created. This form of backtracking is called chronological
backtracking because the order in which steps are undone depends only on the temporal
sequence in which steps where originally made. Specifically the most recent step is
always the first to be undone. This form of backtracking is what is usually meant by the
simple backtracking. An example is shown below:
In a depth-first search, after operators are applied to a state, one of the resultant
states is considered next. So the order in which the states of the simple example space
will be considered is:
Algorithm:
1. If the initial state is a goal state, terminate with success and return that state.
a) Generate a successor E of the initial state. If there are no more successors signal
failure.
b) Call DFS with E as initial state.
c) If success is returned signal success otherwise continue in this loop
Completeness: Does it always find a solution if one exists? NO, unless search space is
finite and no loops are possible.
Time complexity: O(bm ) .Terrible if m is much larger than d (depth of optimal solution).
But if many solutions are present, then faster than BF-search
Space complexity: O(bm) .Backtracking search uses even less memory. In that it
generates only one successor instead of all
Optimality: No, unless search space is finite and no loops are possible.
Advantages:
1. DFS requires less memory since only the nodes on the current path are stored.
This contrasts with BFS where all of the tree that has so far been generated must
be stored. Once a node has been expanded it can be removed from memory as
soon as all its descendants have been fully explored.
2. Easily programmed: function call stack does most of the work of maintaining
state of the search.
3. DFS may find a solution without examining much of the search space at all. This
contrasts with BFS in which all parts of the tree must be examined to level ‘n’
before any nodes at level ‘n+1’ can be examined. This is particularly significant if
many acceptable solutions exist. DFS can stop when one of them is found.
Disadvantages:
1. May find a sub-optimal solution (one that is deeper or more costly than the best
solution).
2. Incomplete: without a depth bound, may not find a solution even if one exists.
Depth-first search can spend much time (perhaps infinite time) exploring a very
deep path that does not contain a solution, when a shallow solution exists. An easy way to
solve this problem is to put a maximum depth limit ‘l’ on the search ie nodes at depth
limit are treated as if they have no successors. Beyond the depth limit, a failure is
generated automatically without exploring any deeper. This approach is called depth
limited search. The depth limit solves the infinite path problem.
If a solution exists at depth d or less, then d -limited DFS will find the solution,
and the algorithm is very efficient in space. However, depth-limited DFS is not
complete: If a solution exists but only at depth greater than d, then depth-limited
DFS will not find the solution.
For eg : in the case of a TSP consider a map with 20 cities. Therefore we know
that if there is a solution it must be of length 19 at the longest so l =19 is a possible
choice. But if we study the map carefully we can see that any city can be reached from
any other city in at most 9 steps .This is known as diameter of state space gives us a
better depth limit which leads to a more efficient depth limited search.
Algorithm :
1. Determine the vertex where the search should start and assign the maximum
search depth
2. Check if the current vertex is within the maximum search depth
o If not: Do nothing
o If yes:
1. Expand the vertex and save all of its successors in a stack
2. Call DLS recursively for all vertices of the stack and go back to
Step 2
Completeness : In general the algorithm is not complete since it does not find any
solution that lies beyond the given search depth. But if you choose the maximum search
depth to be greater than the depth of a solution the algorithm becomes complete.
Optimality: Depth-limited search is not optimal. It still has the problem of depth-first
search that it first explores one path to its end, thereby possibly finding a solution that is
more expensive than some solution in another path.
Problems:
The main advantage of this algorithm in game tree searching is that the earlier
searches tend to improve the commonly used heuristics, such as the killer heuristic and
alpha-beta pruning, so that a more accurate estimate of the score of various nodes at the
final depth search can occur, and the search completes more quickly since it is done in a
better order (it can be shown that alpha-beta pruning works much better if the moves are
generated with the moves likely to be the best moves evaluated first). An eg : is shown
below:
In an iterative deepening search, the nodes on the bottom level are expanded once,
those on the next to bottom level are expanded twice, and so on, up to the root of the
search tree, which is expanded d + 1 times. So the total number of expansions in an
iterative deepening search is
level d: once
level d-1: 2
level d-2: 3
………….
level 2: d-1
level 1: d
Algorithm :
1. Set DEPTH-LIMIT = 0
2. Conduct depth-first search to a depth of DEPTH-LIMIT If a solution path is
found, then return it.
3. Increment DEPTH-LIMIT by 1 and go to step 2.
Advantages:
Disadvantage:
1. Some computer time is wasted re-exploring the higher parts of the search tree.
However, this actually is not a very high cost.
6. Bi-Directional Search
This does not come without a price: Aside from the complexity of searching two
times in parallel, we have to decide which search tree to extend at each step; we have to
be able to travel backwards from goal to initial state - which may not be possible without
extra work; and we need an efficient way to find the intersection of the two search trees.
This additional complexity means that the A* search algorithm is often a better choice if
we have a reasonable heuristic.
It is implemented by having one or both of the searches check each node before it
is expanded to see if it is in the fringe of the other search tree. At least one of the search
tree needs to be kept in memory so that membership check can be done hence the space
complexity is also O(bd / 2).This space requirement is the most significant weakness of
bidirectional search. The algorithm is complete and optimal if both searches are BF.
Completeness: Yes
Space Complexity: O(bd/2) need to retain at least that many nodes to guarantee
completeness
b = branching factor
d = depth of the shallowest solution
m = maximum depth of the search tree
l = depth limit
ε = step-cost
C* = cost of the optimal solution
Constraint Satisfaction
Firstly, the representation as a CSP is often much closer to the original problem:
the variables of the CSP directly correspond to problem entities, and the
constraints can be expressed without having to be translated into linear
inequalities. This makes the formulation simpler, the solution easier to
understand, and the choice of good heuristics to guide the solution strategy more
straightforward.
Secondly, although CSP algorithms are essentially very simple, they can
sometimes find solution more quickly than if integer programming methods are
used.
The first step propagation arises from the fact that there are usually dependencies
among the constraints. These dependencies occur because many constraints involve more
than one object and many objects participate in more than one constraint. For e.g.:
assume we start with one constraint N=E+1. Then if we added the constraint N=3 we
could propagate that to get a stronger constraint on E namely E=2.
Initial state: the empty assignment in which all variables are unassigned.
Successor function: a value can be assigned to any unassigned variable, provided that it
does not conflict with previously assigned variables.
Goal test: the current assignment is complete.
Path cost: a constant cost (e.g., 1) for every step.
At this point the second step begins. Some hypothesis about a way to strengthen
the constraints must be made. In this case of crypt arithmetic problem this usually means
guessing a particular value for some letter. Once this has been done constraint
propagation can begin again from this new state. If a solution is found it can be reported.
If a contradiction is detected then backtracking can be used to try a different guess and
proceed with it.
Algorithm:
1. Propagate available constraints. To do this first set OPEN to the set of all objects
that must have values assigned to them in a complete solution. Then do until an
inconsistency is detected or until OPEN is empty.
a) Select an object OB from OPEN. Strengthen as much as possible the set of
constraints that apply to OB
b) If this set is different from the set that was assigned the last time OB was
examined or if then first time OB has been examined then add to OPEN all
objects that share any constraint with OB.
c) Remove OB from OPEN.
2. If the union of the constraints discovered above defines a solution the quit and
report the solution.
3. If the union of the constraints discovered above defines a contradiction the quit
then return failure.
4. If neither of the above occurs then it is necessary to make a guess at something in
order to proceed. To do this loop until a solution is found or all possible solutions
have been eliminated.
a) Select an object whose value is not yet determined and select a way of
strengthening the constraints on that object.
b) Recursively invoke constraint satisfaction with the current set of
constraints augmented by the strengthening constraint just selected.
S E N D
M O R E
=========
M O N E Y
We have to replace each letter by a distinct digit so that the resulting sum is
correct. Furthermore, numbers must not begin with a zero.
Initial state: No 2 letters have the same value.
The sum of the digits must be as shown in the problem.
The goal state is a problem state in which all letters have been assigned a digit in
such a way that all the initial constraints are satisfied. The solution proceeds in cycles. At
each cycle 2 significant things are done.
A few useful heuristics can help the best guess to try first. For e.g. if there is a
letter that has only 2 possible values and another with 6 possible values, there is a better
chance of guessing right on the first than on the second. Another useful heuristic is that if
there is a letter that participates in many constraints then it is a good idea to prefer it to a
letter that participates in a few. A guess on such a highly constrained letter will usually
lead quickly either to a contradiction (if it is wrong) or to the generation of many
additional constraints (if it is right).A guess on a less constrained letter on the other hand
provides less information.