Artificial Intelligence

Contents
Unit 1:
1.1 Introduction to AI
1.2 AI problems
1.3 AI techniques
1.4 Level of model
1.5 Criteria for success
1.6 Problem, problem spaces and searches
1.7 Production system characteristics
1.8 Issues in design of search programs

3
5
7
13
15
18
30
33

Unit 2 :
2.1 Game AI
2.2 Types of AI
2.3 Chasing and evading
2.4 Backtracking
2.5 Strategical AI
2.6 How to create strategical AI in games

36
37
40
44
47
49

Unit 3:
3.1 Game AI
3.2 Importance of good game AI
3.3 Difference between Game AI and Academic AI
3.4 Deterministic & Non-deterministic

55
55
61
62

Unit 4:
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8

Pathfinding, A* & its derivatives

Flocking and Stearing
Rule based system
Finite state machine
Patterning & Waypoints
Fuzzy logic and fuzzy state machine
Genetic algorithm
Artificial Neural Network

Unit 5:
5.1 Future for AI in games

64
78
85
90
102
106
107
110

114

ARTIFICIAL INTELLIGENCE
UNIT-I
1.1 Introduction to AI
The field of artificial intelligence, or AI, attempts to understand intelligent
entities. Thus, one reason to study it is to learn more about ourselves. But
unlike philosophy and psychology, which are also concerned with intelligence,
AI strives to build intelligent entities as well as understand them. Another
reason to study AI is that these constructed intelligent entities are interesting
and useful in their own right. AI has produced many significant and impressive
products even at this early stage in its development. Although no one can
predict the future in detail, it is clear that
computers with human-level intelligence (or better) would have a huge impact
on our everyday lives and on the future course of civilization.
AI addresses one of the ultimate puzzles. How is it possible for a slow, tiny
brain, whether biological or electronic, to perceive, understand, predict, and
manipulate a world far larger and more complicated than itself? How do we go
about making something with those properties? These are hard questions, but
unlike the search for faster-than-light travel or an antigravity device,the
researcher in AI has solid evidence that the quest is possible. All the
researcher has to do is look in the mirror to see an example of an intelligent
system.
AI is one of the newest disciplines. It was formally initiated in 1956, when
the name was coined, although at that point work had been under way for
about five years. Along with modern genetics, it is regularly cited as the "field
I would most like to be in" by scientists in other disciplines. A student in
physics might reasonably feel that all the good ideas have already been taken
by Galileo, Newton, Einstein, and the rest, and that it takes many years of
study before one can contribute new ideas. AI, on the other hand, still has
openings for a full-time Einstein.
For over 2000 years, philosophers have tried to understand how seeing,
learning, remembering, and reasoning could, or should,be done.' The advent
of usable computers in the early 1950s turned the learned but armchair
speculation concerning these mental faculties into a real experimental and
theoretical discipline.Many felt that the new "Electronic Super-Brains" had
unlimited potential for intelligence. "Faster Than Einstein" was a typical
headline. But as well as providing a vehicle for creating artificially intelligent
entities, the computer provides a tool for testing theories of intelligence, and
many theories failed to withstand the testa case of "out of the armchair,
into the fire." AI has turned out to be more difficult than many at first
imagined, and modem ideas are much richer, more subtle, and more
interesting as a result.
3

AI currently encompasses a huge variety of subfields, from generalpurpose areas such as perception and logical reasoning, to specific tasks such
as playing chess, proving mathematical theorems, writing poetry, and
diagnosing diseases. Often, scientists in other fields move gradually into
artificial intelligence, where they find the tools and vocabulary to systematize
and automate the intellectual tasks on which they have been working all their
lives. Similarly, workers in AI can choose to apply their methods to any area of
human intellectual endeavor. In this sense, it is truly a universal field.
1.1.1 Definitions of Artificial Intelligence
The definitions vary along two main dimensions. The ones on top are
concerned
with thought processes and reasoning, whereas the ones on the bottom
address behavior. Also,the definitions on the left measure success in terms of
human performance, whereas the ones on the right measure against an ideal
concept of intelligence, which we will call rationality.
"The exciting new effort to make computers think . . . machines with minds, in
the full and literal sense" (Haugeland, 1985)
"[The automation of] activities that we associate with human thinking,
activities such as decision-making, problem solving, learning..."(Bellman,
1978)
"The art of creating machines that perform functions that require intelligence
when performed by people" (Kurzweil, 1990)
"The study of how to make computers do things at which, at the moment,
people are better" (Rich and Knight, 1991 )
"The study of mental faculties through the use of computational models"
(Charniak and McDermott, 1985)
"The study of the computations that make it possible to perceive, reason, and
act" (Winston, 1992)
"A field of study that seeks to explain and emulate intelligent behavior in
terms of computational processes" (Schalkoff, 1990)
"The branch of computer science that is concerned with the automation of
intelligent behavior" (Luger and Stubblefield, 1993)

1.2 AI Problems
There are four essentially different types of problems
Singlestate problems,
Multiple-state problems,
Contingency problems, and
Exploration problems
Let the world contain just two locations. Each location may or may not
contain dirt, and the agent may be in one location or the other. There are 8
possible world states, as shown in figure 1.1. The agent has three possible
actions in this version of the vacuum world: Left, Right, and Suck.Assume, for
the moment, that sucking is 100% effective. The goal is to clean up all the
dirt. That is, the goal is equivalent to the state set {7,8}.

Fig 1.1 The eight possible states of the simplified vacuum World
First, suppose that the agent's sensors give it enough information to tell
exactly which state it is in (i.e., the world is accessible); and suppose that it
knows exactly what each of its actions does. Then it can calculate exactly
which state it will be in after any sequence of actions. For example, if its initial
state is 5, then it can calculate that the action sequence [Right,Suck] will get
to a goal state. This is the simplest case, which we call a single-state
problem.

Second, suppose that the agent knows all the effects of its actions, but
has limited access to the world state. For example, in the extreme case, it
may have no sensors at all. In that case, it knows only that its initial state is
one of the set {1,2,3,4,5,6,7,8}. One might suppose that the agent's
predicament is hopeless, but in fact it can do quite well. Because it knows
what its actions do, it can, for example, calculate that the action Right will
cause it to be in one of the states {2,4,6,8}. In fact, the agent can discover
that the action sequence [Right,Suck,Left,Suck] is guaranteed to reach a goal
state no matter what the start state. To summarize: when the world is not fully
accessible, the agent must reason about sets of states that it might get to,
rather than single states. We call this a multiple-state problem.
Although it might seem different, the case of ignorance about the effects
of actions can be treated similarly. Suppose, for example, that the
environment appears to be nondeterministic in that it obeys Murphy's Law:
the so-called Suck action sometimes deposits dirt on the carpet but only if
there is no dirt there already. For example, if the agent knows it is in state 4,
then it knows that if it sucks, it will reach me of the states {2,4}. For any
known initial state, however, there is an action sequence that is guaranteed to
reach a goal state.
Sometimes ignorance prevents the agent from finding a guaranteed
solution sequence. Suppose, for example, that the agent is in the Murphy's
Law world, and that it has a position sensor and a local dirt sensor, but no
sensor capable of detecting dirt in other squares. Suppose further that the
sensors tell it that it is in one of the states {1,3}. The agent might formulate
the action sequence [Suck,Right,Suck]. Sucking would change the state to one
of {5,7}, and moving right would then change the state to one of {6,8}. If it is
in fact state 6, then the action sequence will succeed, but if it is state 8, the
plan will fail. If the agent had chosen the simpler action sequence [Suck], it
would also succeed some of the time, but not always. It turns out there is no
fixed action sequence that guarantees a solution to this problem.
Obviously, the agent does have a way to solve the problem starting from
one of {1,3}: first suck, then move right, then suck only if there is dirt there.
Thus, solving this problem requires sensing during the execution phase.
Notice that the agent must now calculate a whole tree of actions, rather than
a single action sequence. In general, each branch of the tree deals with a
possible contingency that might arise. For this reason, we call this a
contingency problem.
Finally, consider the plight of an agent that has no information about the
effects of its actions. This is somewhat equivalent to being lost in a strange
country with no map at all, and is the hardest task faced by an intelligent
6

agent.The agent must experiment, gradually discovering what its actions do

and what sorts of states exist. This is a kind of search, but a search in the real
world rather than in a model thereof. Taking a step in the real world, rather
than in a model, may involve significant danger for an ignorant agent. If it
survives, the agent learns a "map" of the environment, which it can then use
to solve subsequent problems. We discuss this kind of exploration problem.

1.3 AI Techniques
According to the area of research, different techniques are applied:
Finite State Morphology
Parsing and Generation Techniques
Techniques for Speech Synthesis
Hidden Markov models
Kernel methods
Bayesian networks etc.
Let us discuss about two techniques briefly in this context.
1.3.1 Hidden Markov models
Some words have very simple pronunciation models.For example : Take the
word "tomato," you say [tow mey tow] and I say [tow maa tow]. The alternative
pronunciations are specified as a Markov model. In general, a Markov model is
a way of describing a process that goes through a series of states. The model
describes all the possible paths through the state space and assigns a
probability to each one. The probability of transitioning from the current state
to another one depends only on the current state, not on any prior part of the
path.

Fig 1.2 Two pronunciation models of the word "tomato." The top one accounts
for dialect differences. The bottom one does that and also accounts for a
coarticulation effect.
The top of Figure 1.2 is a Markov model with seven states (circles), each
corresponding to the production of a phone. The arrows denote allowable
transitions between states, and each transition has a probability associated
with it. There are only two possible paths through the model, one
corresponding to the phone sequence [t ow m ey t ow] and the other to [t ow
m aa t ow]. The probability of a path is the product of the probabilities on the
arcs that make up the path. In this case, most of the arc probabilities are 1
and we have
P([towmeytow] ("tomato") = P([towmaatow] ("tomato") = 0.5

The second source of phonetic variation is coarticulation. The bottom half

of Figure 1.2 gives a more complicated pronunciation model for "tomato" that
takes this coarticulation effect into account. In this model there are four
distinct paths and we have
P([towmeytow]\"tomato") = P([towmaatow] ("tomato") = 0.1
P([tahmeytow]\"tomato") = P([tahmaatow]\ "tomato") = 0.4
Similar models would be constructed for every word we want to be able to
recognize.
8

A hidden Markov model is just like a regular Markov model in that it

describes a process that goes through a sequence of states. The difference is
that in a regular Markov model, the output is a sequence of state names, and
because each state has a unique name, the output uniquely determines the
path through the model. In a hidden Markov model, each state has a
probability distribution of possible outputs, and the same output can appear in
more than one state. HMMs are called hidden models because the true state
of the model is hidden from the observer. In general, when you see that an
HMM outputs some symbol, you can't be sure what state the symbol came
from.
Suppose our speech signal is processed to yield the sequence of vector
quantization values [C1 ,C4,C6]. From the HMM in Figure 24.36, we can
compute the probability that this sequence was generated by the phone [m]
as follows. First, we note that there is only one path through the model that
could possibly generate this sequence: the path from Onset to Mid to End,
where the output labels from the three states are C1, C4, and C6, respectively.
By looking at the probabilities on the transition arcs, we see that the
probability of this path is 0.7 x 0.1 x 0.6 (these are the values on the three
horizontal arrows in the middle of the Figure 24.36). Next, we look at the
output probabilities for these states to see that the probability of [C 1 ,C4,C6]
given this path is 0.5 x 0.7 x 0.5 (these are the values for P(Cl\Onset),
P(C4\Mid) and P(C6\End), respectively). So the probability of [Cl ,C4,C6] given
the [m] model is
P([Cl,C4,C6]|[m]) = (0.7 x 0.1 x 0.6) x (0.5 x 0.7 x 0.5) = 0.00735
We could repeat the calculation for all the other phone models to see
which one is the most probable source of the speech signal.
Actually, most phones have a duration of 50-100 milliseconds, or 5-10
frames at 10 msec/frame. So the [C1,C4,C6J sequence is unusually quick.
Suppose we have a more typical speaker who generates the sequence
[C1,C1,C4,C4,C6,C6J while producing the phone. It turns out there are two
paths through the model that generate this sequence. In one of them both
C4s come from the Mid state (note the arcs that loop back), and in the other
the second C4 comes from the End state. We calculate the probability that
this sequence came from the [m] model in the same way: take the sum over
all possible paths of the probability of the path times the probability that the
path generates the sequence.

Fig., 1.3 An HMM for the phone [m]. Each state has several possible
outputs, each with its own probability.
P([Cl,C\,C4, C4, C6, C6] | [m]) =
(0.3 x 0.7 x 0.9 x 0.1 x 0.4 x 0.6) x (0.5 x 0.5 x 0.7 x 0.7 x 0.5 x 0.5) +
(0.3 x 0.7 x 0.1 x 0.4 x 0.4 x 0.6) x (0.5 x 0.5 x 0.7 x 0.1 x 0.5 x 0.5)
= 0.0001477
We see that the loops in the phone model allow the model to represent both
fast and slow speech, a very important source of variation. The multiple
vector quantization values on each state represent other sources of variation.
Altogether, this makes for a fairly powerful model. The hard part is getting
good probability values for all the parameters.

1.3.2 Bayesian Networks

Bayesian networks provide a means of expressing joint probability
distribution over many interrelated hypothesis. A Bayesian network consists of
Directed Acyclic Graph(DAG) and a set of local distributions. Each node in the
graph represents a random variable. A random variable denotes an attribute,
feature or hypothesis about which we may be uncertain.
Bayesian Networks: A review
A Bayesian network (BN) is a probabilistic graph model. It can be defined
as a pair (G, p), where G = (V, E) is a directed acyclic graph (DAG) (Jensen,
1996; Heckerman, 1998). Here, V is the vertex set which represents variables
in the problem and E is the edge set which denotes probabilistic relationships
among the variables. For a variable X V, a parent of X is a node from which
10

there is a directed link to X. Let pa(X) denote the set of parents of X, then the
conditional independence property can be represented as follows:

This property can simplify the computations in a Bayesian network model.

For example, the joint distribution of the set of all variables in V can be written
as a product of conditional probabilities as follows:

Fig 1.4 ASIA model

The conditional independence between variables is either obtained
from a priori expert knowledge or discerned from data, or a combination
of both (Jensen, 1996). The set of conditional distributions {P (X | pa(X)), X
V} are called the parameters of a Bayesian network. Note that if
variable X has no parents, then P (X | pa(X)) = P (X) is the marginal
distribution of X.
Figure 1 is a Bayesian network called the ASIA model (adapted from
Lauritzen and Spiegelhalter (1988)). The variables are Dyspnoea,
Tuberculosis, Lung cancer, Bronchitis, Asia, X-ray, Either, and Smoking.
They are all binary variables. The joint distribution of all variables is

The ordering of variables constitutes a constraint on the structure of a

Bayesian network. If variable X appears before variable Y , then Y can not
be a parent of X. We use the ordering (A, S, T, L, B, E, X, D) as prior
knowledge in our example.
11

Two important issues in using a Bayesian network are : (a) learning a

Bayesian network and (b) probabilistic inferencing. Learning a BN involves
learning the structure of the network (the directed graph), and obtaining
the conditional probabilities (parameters) associated with the network.
Once a Bayesian network is constructed, we usually need to determine
various probabilities of interest from the model. This process is referred to
as probabilistic inference. For example, in the ASIA model, a diagnosis
application would require finding the probability P (B | D) of Bronchitis,
given the (observed) symptom Dyspnoea. This probability (usually called
posterior probability) can be computed using the Bayes rule.
Collective Bayesian Network Learning Strategy
We now present a collective strategy to learn a Bayesian network (both
structure
and parameters) when data is distributed among different sites. The
centralized solution to this problem is to download all datasets from
distributed sites to a central site. In many applications, this would not be
feasible because of the size of the data, available communication bandwidth,
or due to security considerations. Learning a BN for the homogeneous case
was studied by Kenji (1997). In this case, every distributed site has the same
set of features but has different set of observations. We address here the
heterogenous case, where each distributed site has all the observations for
only a subset of the features.
The primary steps in our approach are:
Compute local BNs (local model) involving the variables observed at
each site (local variables) based on local data.
At each site, based on the local BN, identify the observations that
are most likely to be evidence of coupling between local and non-local
variables. Transmit a subset of these observations to a central site.
At the central site, a limited number of observations of all the
variables are now available. Using this, compute a non-local BN consisting of
links between
variables across two or more sites.
Combine the local models with the links discovered at the central
site to obtain a collective BN.
The non-local BN thus constructed would be effective in identifying
associations between variables across sites, whereas the local BNs would
detect associations among local variables at each site. The conditional
probabilities can also be estimated in a similar manner. Those probabilities
that involve only variables from a single site can be estimated locally,
whereas the ones that involve variables from different sites can be estimated
12

at the central site. The same methodology could be used to update the
network based on new data. First, the new data is tested for how well it fits
with the local model. If there is an acceptable statistical fit, the observation is
used to update the local conditional probability estimates. Otherwise, it is also
transmitted to the central site to update the appropriate conditional
probabilities (of cross terms). Finally, a collective BN can be obtained by
taking the union of nodes and edges of the local BNs and the nonlocal BN,
along with the conditional probabilities from the appropriate BNs. Probabilistic
inference can now be performed based on this collective BN. Note that
transmitting the local BNs to the central site would involve a significantly
lower communication as compared to transmitting the local data.
It is quite evident that learning probabilistic relationships between
variables that belong to a single local site is straightforward and does not
pose any additional difficulty as compared to a centralized approach.2 The
important objective is to correctly identify the coupling between variables that
belong to two (or more) sites. These correspond to the edges in the graph that
connect variables between two sites and the conditional probability(ies) at the
associated node(s). In the following, we describe our approach to selecting
observations at the local sites that are most likely to be evidence of strong
coupling between variables at two different sites.
Selection of samples for transmission to global site
For simplicity, we will assume that the data is distributed between two
sites and will illustrate the approach using the BN in Figure 1. The extension of
this approach to more than two sites is straightforward. Let us denote by A
and B the variables in the left and right groups, respectively, in Figure 1. We
assume that the observations for A are available at site A, whereas the
observations for B are available at a different site B. Furthermore, we assume
that there is a common feature (key or index) that can be used to associate
a given observation in site A to a corresponding observation in site B.
Naturally, V = A B.
At each local site, a local Bayesian network can be learned using only
samples
in this site. This would give a BN structure involving only the local variables at
each site and the associated conditional probabilities. Let p A(.) and pB(.)
denote the estimated probability function involving the local variables. This is
the product of the conditional probabilities as indicated by (2). Since p A(x),
pB(x) denote the probability or likelihood of obtaining observation x at sites A
and B, we would call these probability functions the likelihood functions l A(.)
and lB(.), for the local model obtained at sites A and B, respectively. The
observations at each site are ranked based on how well it fits the local model,
using the local likelihood functions. The observations at site A with large
13

likelihood under lA(.) are evidence of local relationships between site A

variables, whereas those with low likelihoods under lA(.) are possible evidence
of cross relationships between variables across sites. Let S(A) denote the set
of keys associated with the latter observations (those with low likelihood
under lA(.)). In practice, this step can be implemented in different ways. For
example, we can set a threshold A and if lA(x) A, then x SA. The sites A
and B transmit the set of keys S A, SB, respectively, to a central site, where the
intersection S = SA SB is computed.
The observations corresponding to the set of keys in S are then obtained
from each of the local sites by the central site. The following argument
justifies our selection strategy. Using the rules of probability, and the assumed
conditional independence in the BN of Figure 1, it is easy to show that:

where nb(A) = {B, L} is the set of variables in B, which have a link

connecting it to a variable in A. In particular,

where nb(B) = {E, D} and

Therefore, an observation {A = a, T = t, E = e, X = x, D = d, S = s, L = l,
B = b} with low likelihood at both sites A and B; i.e. for which both P(A) and
P(B) are small, is an indication that both P(A | nb(A)) and P(B | nb(B)) are large
for that observation (since observations with small P(V) are less likely to
occur). Notice from (5) and (7) that the terms common to both P(A | nb(A))
and P(B | nb(B)) are precisely the conditional probabilities that involve
variables from both sites A and B. In other words, this is an observation that
indicates a coupling of variables between sites A and B and should hence be
transmitted to a central site to identify the specific coupling links and the
associated conditional probabilities.
In a sense, our approach to learning the cross terms in the BN involves a
selective sampling of the given dataset that is most relevant to the
identification of coupling between the sites. This is a type of importance
sampling, where we select the observations that have high conditional
probabilities corresponding to the terms involving variables from both sites.
Naturally, when the values of the different variables (features) from the
different sites, corresponding to these selected observations are pooled
together at the central site, we can learn the coupling links as well as
14

estimate the associated conditional distributions. These selected observations

will, by design, not be useful to identify the links in the BN that are local to the
individual sites.
1.4 Level of model
Before we set out to do something it is good to decide exactly what we
are trying to do. So we must ask ourselves, what is our goal in trying to
produce programs that do the intelligent things that people do? Are we trying
to produce programs that simply do the task the same way people do? Or, are
we attempting to produce programs that simply do the task in whatever way
appears easiest? There have been AI projects motivated by each of these
goals.
Efforts to build programs that performs tasks the way people do can be
divided into two classes. Programs in the first class attempts to solve
problems that do not really fit our definition of an AI task. They are problems
that a computer can easily solve, although that easy solution would exploit
mechanisms that do not seem to be available to the people. A classical
example of this class of program is Elementary Perceiver and Memoraizer
(EPAM) [Feigenbaum, 1963], which memoraized associated pairs of nonsense
syllables is easy for computer. Simply input them. To retrieve a responsible
syllable given its associated stimulus one, the computer just scans for
stimulus syllable and responds with the one stored next to it. But this task is
hard for people. EPAM simulated one way people might perform the task. It
built a discrimination net through which it could find images of syllables it had
seen. It also stored with each stimulus image, a cue that it could later pass
through the discrimination net to try to find correct response image. But it
stored as a cue only as much information about response syllable as was
necessary to built ambiguity at the time the association was stored. This
might be just the first letter, for example: But, ofcourse as the discrimination
net grew and more syllable were added, an old cue might no longer be
sufficient to identify response syllable uniquely. Thus EPAM, like people
sometimes forget previously learned response. Many people regard program
in the first class to be uninteresting, and to some extent they are probably
right. These programs can, however, be useful tools for psychologists who
want to test theories of human performance.
The second class of program that attempt to model human performance
are those that do things that fall more clearly about our definition of AI tasks;
they do things that are not trivial for the computers. There are several reasons
one might want to model human performance at these sorts of tasks:
1.
To test psychological theories of human performance. One
example of a program that was written for this reason is PARRY [Colby, 1975],
which exploited a model of human paranoid behavior to simulate the
conversational behavior of a paranoid person. The model was good enough
15

that when several psychologists were given the opportunity to converse the
program via a terminal; they diagnosed its behavior as paranoid.
2.
To enable computers to understand human reasoning. For
example, for a computer to be able to read a newspaper story and then
answer a question, such as why did the terrorists kill the hostages? its
program must be able to simulate the reasoning processes of people.
3.
To enable people to understand computer reasoning. In many
circumstances, people are reluctant to rely on computer unless they can
understand how the machine arrived at its result. If the computers reasoning
process is similar to that of people, then producing an acceptable explanation
is much easier.
4.
To exploit what knowledge we can glean from people. Since
people are the best-known performers of most of tasks with which we are
dealing, it makes a lot of sense to look to them for clues as to how to proceed.
This last motivation is probably the most pervasive of the four. It
motivated
Several early systems that attempted to produce intelligent behavior by
imitating people at the level of individual neurons. For example of this, see
the early theoratical work of McCulloch and Pits[1943], the work on
perceptrons originally developed by Frank Rosenblatt but best described in
Perceptron [minsky and Papert, 1969] and Design for a Brain[Ashby, 1952]. It
proves impossible, however, to produce even minimally intelligent behavior
with such simple devices. One reason was that there were severe theoretical
limitations, to the particular neural, net architecture that was being used.
More recently, several new neural net architectures have been proposed.
These structures are not subject to same theoretical limitations as were
perceptrons. These new architectures are loosely called connectionist, and
they have been used as a basis for several learning and problem-solving
programs. Also, we consider that while human brain are highly parallel
devices, most current computing devices are essentially serial engines. A
highly successful parallel technique may be computationally interactable on a
serial computer. But recently, partly because of the existence of the new
family of parallel cognitive model, as well as because of general promise of
parallel computing, there is low substantial design of massively parallel
machines to support AI programs.
Human cognitive theory have also influenced AI to look for higher-levels
(i.e, far above the neural level), theories that do not require massive
parallelism for their implementation. An early example of this approach can
be seen in GPS. This same approach can also be seen in much current work in
natural language understanding. The failure of straightforward syntactic
parsing mechanisms to make much of a dent in the problem of interpreting
English sentences has led many people who are interested in natural
16

language understanding by machine to look seriously for inspiration at what

little we know about how people interpret language. And when people who are
trying to build programs to analyze pictures discover that a filter function they
have developed is very similar to what we think people use, they take hart
that perhaps they are on the right track.
As you can see, this last motivation pervades a great many areas of AI
research. Infact, it, in conjunction, with the other motivations we mentioned,
tends to make the distinction between the goal of simulating human
performance and the goal of building an intelligent agent any way we can
seem much less different than they at first appeared. In either case, what we
really need is a good model of processes involved in intelligent reasoning. The
goal of cognitive science, in which psychologist, linguists, and computer
scientists all work together, has as its goal the discovery of such a model. For
a good survey of variety of approaches contained within the field, see
Norman[1981], Anderson[1985], and Gardner[1985].
1.5 Criteria for success
One of the most basic philosophical questions for AI is "Can machines
think?" We will not attempt to answer this question directly, because it is not
clearly defined.
To see why, consider the following questions:
Can machines fly?
Can machines swim?
Most people would agree that the answer to the first question is yes,
airplanes can fly, but the answer to the second is no; boats and submarines
do move through the water, but we do not normally call that swimming.
However, neither the questions nor the answers have any impact at all on the
working lives of aeronautic and naval engineers. The answers have very little
to do with the design or capabilities of airplanes and submarines, and much
more to do with the way we have chosen to use words. The word "swim" has
come to mean "to move along in the water by movements of the limbs or
other body parts," whereas the word "fly" has no such limitation on the means
of locomotion.
To complicate matters, words can be used metaphorically, so when we say
a computer (or an engine, or the economy) is running well, we mean it is
operating smoothly, not that it is propelling itself with its legs in an admirable
fashion. Similarly, a person who says, "My modem doesn't work because the
computer thinks it is a 2400-baud line" is probably using "thinks"
metaphorically, and may still maintain that computers do not literally think.
The practical possibility of "thinking machines" has been with us only for
about 40 years, not long enough for speakers of English to settle on an agreed
17

meaning for the word "think." In the early days of the debate, some
philosophers thought that the question of thinking machines could be settled
by means of linguistic analysis of the kind hinted at earlier. If we define "think"
to mean something like "make decisions or deliberations by means of an
organic, natural brain," then we must conclude that computers cannot think.
Ultimately, the linguistic community will come to a decision that suits its need
to communicate clearly, but the decision will not tell us much about the
capabilities of machines.
Alan Turing, in his famous paper "Computing Machinery and Intelligence"
(Turing, 1950), suggested that instead of asking "Can machines think?" we
should instead ask if they can pass a behavioral test (which has come to be
called the Turing Test) for intelligence. He conjectured that by the year 2000, a
computer with a storage of 10 9 units could be programmed well enough to
have a conversation with an interrogator for 5 minutes and have a 30%
chance of fooling the interrogator into thinking it was human. Although we
would certainly not claim that anything like general, human-level intelligence
will be achieved by that time, his conjecture may not be that far off the truth.
Turing also examined a wide variety of possible objections to the possibility of
intelligent machines, including virtually all of those that have been raised in
the 44 years since his paper appeared.
Some of the objections can be overcome quite easily. For example, Lady
Ada Lovelace, commenting on Babbage's Analytical Engine, says, "It has no
pretensions to originate anything. It can do whatever we know how to order it
to perform." This objection, that computers can only do what they are told to
do and are therefore not capable of creativity, is commonly encountered even
today. It is refuted simply by noting that one of the things we can tell them to
do is to learn from their experience. For example, Samuel's checker-playing
program performed very poorly with its original programming. However, it
was able to learn, over the course of a few days of self-play, to play checkers
far better than Samuel himself. One can try to preserve Lady Lovelace's
objection by maintaining that the program's ability to learn originated
in Samuel, and so too did its checker-playing ability. But then one would also
be led to say that Samuel's creativity originated in his parents, and theirs
originated in their parents, and so on.
The "argument from disability" takes the form of a claim, usually
unsupported, to the effect that "a machine can never do X." As examples of X,
Turing lists the following:
Be kind, resourceful, beautiful, friendly, have initiative, have a sense of
humor, tell right from wrong, make mistakes, fall in love, enjoy strawberries
and cream, make someone fall in love with it, learn from experience, use
18

words properly, be the subject of its own thought, have as much diversity of
behavior as man, do something really new.
Turing suggests that scepticism of this nature arises from experience of
machines as devices for carrying out repetitive tasks requiring little sensory
and no reasoning ability. He points to the fact that in the late 1940s, the
general population found it difficult to believe that machines could find
numerical solutions of equations or predict ballistic trajectories. Even today,
however, many technically literate people do not believe that machines can
learn.
The supposed inability to make mistakes presents an interesting problem
when considering the Turing Test. Certainly, instantaneous and correct
answers to long division problems would be a giveaway, and some attempt to
simulate human fallibility would be required. But this is not a mistake in the
normal sense, because the program is doing exactly what its designer
intended. Something more akin to human mistakes will arise when intractable
problems are involved. For example, given only a small amount of time to find
a chess move, the computer must essentially guess that its move is correct.
Similarly, a program that is trying to induce hypotheses from a small amount
of data is bound to make mistakes when using such hypotheses for prediction.
When unavoidably irrational behavior on the part of the computer matches
corresponding failings of humans, this provides evidence that similar
mechanisms are in operation. Rational behavior, on the other hand, provides
much weaker constraints on mechanisms.
What Turing calls the mathematical objection concerns the proven
inability of computers to answer certain questions. In-practice objections
center on the so-called "argument from informality," which claims that
intelligent behavior cannot be captured by formal rules.The final, and most
interesting, objection claims that even if computers behave as intelligently as
humans, they still will not be intelligent.
1.6 Problem spaces and searches
A problem consists of four parts:
the initial state,
a set of operators,
a goal test function,
and a path cost function.
The environment of the problem is represented by a state space.
A path through the state space from the initial state to a goal state is a
solution.
We can distinguish problems so-called, toy problems, which are intended
19

to illustrate or exercise various problem-solving methods, and so-called realworld problems, which tend to be more difficult and whose solutions people
actually care about.In this section, we will give examples of both. By nature,
toy problems can be given a concise, exact description. This means that they
can be easily used by different researchers to compare the performance of
algorithms. Real-world problems, on the other hand, tend not to have a single
agreed-upon description, but we will attempt to give the general flavor of
their formulations.
1.6.1 Toy problems
i) The 8-puzzIe
The 8-puzzle, an instance of which is shown in Figure 3.4, consists of a
3x3 board with eight numbered tiles and a blank space. A tile adjacent to the
blank space can slide into the space.The object is to reach the configuration
shown on the right of the figure. One important trick is to notice that rather
than use operators such as "move the 3 tile into the blank space," it is more
sensible to have operators such as "the blank space changes places with the
tile to its left." This is because there are fewer of the latter kind of operator.
This leads us to the following formulation:
States: a state description specifies the location of each of the eight tiles in
one of the nine squares. For efficiency, it is useful to include the location of
the blank.
Operators: blank moves left, right, up, or down.
Goal test: state matches the goal configuration as shown in figure.
Path cost: each step costs 1, so the path cost is just the length of the path.

Fig 1.5 A typical instance of the 8-puzzle.

The 8-puzzle belongs to the family of sliding-block puzzles. This general
class is known to be NP-complete, so one does not expect to find methods
significantly better than the search algorithms described in this chapter and
20

the next. The 8-puzzle and its larger cousin, the 15-puzzle, are the standard
test problems for new search algorithms in Al.
ii) The 8-queens problem
The goal of the 8-queens problem is to place eight queens on a
chessboard such that no queen attacks any other. (A queen attacks any piece
in the same row, column or diagonal.) Figure shows an attempted solution
that fails: the queen in the rightmost column is attacked by the queen at top
left.

Fig 1.6 Almost a solution to the 8-queens problem

Although efficient special-purpose algorithms exist for this problem and
the whole n-queens family, it remains an interesting test problem for search
algorithms. There are two main kinds of formulation. The incremental
formulation involves placing queens one by one, whereas the complete-state
formulation starts with all 8 queens on the board and moves them around. In
either case, the path cost is of no interest because only the final state counts;
algorithms are thus compared only on search cost. Thus, we have the
following goal test and path cost:
Goal test: 8 queens on board, none attacked.
Path cost: zero.
There are also different possible states and operators. Consider the following
simple-minded formulation:
States: any arrangement of 0 to 8 queens on board.
21

Operators: add a queen to any square.

In this formulation, we have 648 possible sequences to investigate. A more

sensible choice would use the fact that placing a queen where it is already
attacked cannot work, because subsequent placings of other queens will not
undo the attack. So we might try the following:
States: arrangements of 0 to 8 queens with none attacked.
Operators: place a queen in the left-most empty column such
that it is not attacked by any other queen.
It is easy to see that the actions given can generate only states with no
attacks; but sometimes no actions will be possible. For example, after making
the first seven choices (left-to-right) in Figure, there is no action available in
this formulation. The search process must try another choice. A quick
calculation shows that there are only 2057 possible sequences to investigate.
The right formulation makes a big difference to the size of the search space.
Similar considerations apply for a complete-state formulation. For example,
we could set the problem up as follows:
States: arrangements of 8 queens, one in each column.
Operators: move any attacked queen to another square in the
same column.
This formulation would allow the algorithm to find a solution eventually,
but it would be better to move to an unattacked square if possible.
1.6.2 Real-world problem
i) Touring and travelling salesperson problems
Consider the problem, "Visit every city in Figure below at least once,
starting and ending in Bucharest." This seems very similar to route finding,
because the operators still correspond to trips between adjacent cities. But for
this problem, the state space must record more information. In addition to the
agent's location, each state must keep track of the set of cities the agent has
visited. So the initial state would be "In Bucharest; visited {Bucharest}," a
typical
intermediate
state
would
be
"In
Vaslui;
visited
{Bucharest,Urziceni,Vaslui}," and the goal test would check if the agent is in
Bucharest and that all 20 cities have been visited.

22

Fig 1.7 Route-finding map

The travelling salesperson problem (TSP) is a famous touring problem in

which each city must be visited exactly once. The aim is to find the shortest
tour.6 The problem is NP-hard (Karp, 1972), but an enormous amount of effort
has been expended to improve the capabilities of TSP algorithms. In addition
to planning trips for travelling salespersons, these algorithms have been used
for tasks such as planning movements of automatic circuit board drills.
ii) Robot navigation
Robot navigation is a generalization of the route-finding problem
described earlier. Rather than a discrete set of routes, a robot can move in a
continuous space with (in principle) an infinite set of possible actions and
states. For a simple, circular robot moving on a flat surface, the space is
essentially two-dimensional. When the robot has arms and legs that must also
be controlled, the search space becomes many-dimensional. Advanced
techniques are required just to make the search space finite. In addition to the
complexity of the problem, real robots must also deal with errors in their
sensor readings and motor controls.
1.6.3 Search strategies
23

The majority of work in the area of search has gone into finding the right
search strategy for a problem. In our study of the field we will evaluate
strategies in terms of four criteria:

Completeness: is the strategy guaranteed to find a solution when there is

one?
Time complexity: how long does it take to find a solution?
Space complexity: how much memory does it need to perform the search?
Optimality: does the strategy find the highest-quality solution when there
are several different solutions?

There are six search strategies that come under the heading of
uninformed search. The term means that they have no information about the
number of steps or the path cost from the current state to the goalall they
can do is distinguish a goal state from a nongoal state. Uninformed search is
also sometimes called blind search.

Consider again the route-finding problem. From the initial state in Arad,
there are three actions leading to three new states: Sibiu, Timisoara, and
Zerind. An uninformed search has no preference among these, but a more
clever agent might notice that the goal, Bucharest, is southeast of Arad, and
that only Sibiu is in that direction, so it is likely to be the best choice.
Strategies that use such considerations are called informed search strategies
or heuristic search strategies. Not surprisingly, uninformed search is less
effective than informed search. Uninformed search is still important, however,
because there are many problems for which there is no additional information
to consider.
The six uninformed search strategies are distinguished by the order in
which nodes are expanded. It turns out that this difference can matter a great
deal, as we shall shortly see.
Breadth-first search
One simple search strategy is a breadth-first search. In this strategy,
the root node is expanded first, then all the nodes generated by the root node
are expanded next, and then their successors, and so on. In general, all the
nodes at depth d in the search tree are expanded before the nodes at depth d
+ 1. Breadth-first search can be implemented by calling the GENERAL-SEARCH
algorithm with a queuing function that puts the newly generated states at the
end of the queue, after all the previously generated states:
function BREADTH-FlRST-SEARCH(problem) returns a solution or failure
return GENERAL-SEARCH( problem,ENQUEUE-AT-END)
24

Breadth-first search is a very systematic strategy because it considers all

the paths of length 1 first, then all those of length 2, and so on. Figure 1.8
shows the progress of the search on a simple binary tree. If there is a solution,
breadth-first search is guaranteed to find it, and if there are several solutions,
breadth-first search will always find the shallowest goal state first. In terms of
the four criteria, breadth-first search is complete, and it is optimal provided
the path cost is a nondecreasing function of the depth of the node.
So far, the news about breadth-first search has been good. To see why it is
not always the strategy of choice, we have to consider the amount of time
and memory it takes to complete a search. To do this, we consider a
hypothetical state space where every state can be expanded to yield b new
states. We say that the branching factor of these states (and of the search
tree) is b. The root of the search tree generates b nodes at the first level, each
of which generates b more nodes, for a total of b2 at the second level. Each of
these generates b more nodes, yielding b3 nodes at the third level, and so on.
Now suppose that the solution for this problem has a path length of d. Then
the maximum number of nodes expanded before finding a solution is
1 + b + b2 + b3 + + bd
This is the maximum number, but the solution could be found at any point
on the dth level. In the best case, therefore, the number would be smaller.
Those who do complexity analysis get nervous (or excited, if they are the sort
of people who like a challenge) whenever they see an exponential complexity
bound like O(bd). Figure 3.12 shows why. It shows the time and memory
required for a breadth-first search with branching factor b = 10 and for
various values of the solution depth d. The space complexity is the same as
the time complexity, because all the leaf nodes of the tree must be
maintained in memory at the same time.Figure 1.9 assumes that 1000 nodes

Fig 1.8 Breadth-first search trees after 0, 1, 2, and 3 node expansions

25

can be goal-checked and expanded per second, and that a node requires 100
bytes of storage. Many puzzle-like problems fit roughly within these
assumptions (give or take a factor of 100) when run on a modern personal
computer or workstation.

Fig 1.9 Time and memory requirements for breadth-first search. The figure
shown assumes the branching factor b=10; 1000 nodes/ second; 100 bytes/
node

There are two lessons to be learned from Figure 1.9. First, the memory
requirements are a bigger problem for breadth-first search than the execution
time. Most people have the patience to wait 18 minutes for a depth 6 search
to complete, assuming they care about the answer, but not so many have the
111 megabytes of memory that are required. And although 31 hours would
not be too long to wait for the solution to an important problem of depth 8,
very few people indeed have access to the 11 gigabytes of memory it would
take. Fortunately, there are other search strategies that require less memory.
The second lesson is that the time requirements are still a major factor. If your
problem has a solution at depth 12, then (given our assumptions) it will take
35 years for an uninformed search to find it. Of course, if trends continue then
in 10 years, you will be able to buy a computer that is 100 times faster for the
same price as your current one. Even with that computer, however, it will still
take 128 days to find a solution at depth 12and 35 years for a solution at
depth 14. Moreover, there are no other uninformed search strategies that fare
any better. In general, exponential complexity search problems cannot be
solved for any but the smallest instances.
Uniform cost search
Breadth-first search finds the shallowest goal state, but this may not always
be the least-cost solution for a general path cost function. Uniform cost
search modifies the breadth-first strategy by always expanding the lowestcost node on the fringe (as measured by the path cost g(n)), rather than the
26

lowest-depth node. It is easy to see that breadth-first search is just uniform

cost search with g(n) = DEPTH(n).
When certain conditions are met, the first solution that is found is
guaranteed to be the cheapest solution, because if there were a cheaper path
that was a solution, it would have been expanded earlier, and thus would
have been found first. A look at the strategy in action will help explain.
Consider the route-finding problem in Figure 1.10. The problem is to get from
S to G,and the cost of each operator is marked. The strategy first expands the
initial state, yielding paths to A, B, and C. Because the path to A is cheapest, it
is expanded next, generating the path SAG, which is in fact a solution, though
not the optimal one. However, the algorithm does not yet recognize this as a
solution, because it has cost 11, and thus is buried in the queue below the
path SB, which has cost 5. It seems a shame to generate a solution just to
bury it deep in the queue, but it is necessary if we want to find the optimal
solution rather than just any solution. The next step is to expand SB,
generating SBG, which is now the cheapest path remaining in the queue, so it
is goal-checked and returned as the solution.
Uniform cost search finds the cheapest solution provided a simple
requirement is met: the cost of a path must never decrease as we go along
the path. In other words, we insist that
g(SUCCESSOR(n)) > g(n)
for every node n.
The restriction to nondecreasing path cost makes sense if the path cost of
a node is taken to be the sum of the costs of the operators that make up the
path. If every operator has a nonnegative cost, then the cost of a path can
never decrease as we go along the path, and uniform-cost search can find the
cheapest path without exploring the whole search tree. But if some operator
had a negative cost, then nothing but an exhaustive search of all nodes would
find the optimal solution, because we would never know when a path, no
matter how long and expensive, is about to run into a step with high negative
cost and thus become the best path overall.

27

Fig 1.10 A route-finding problem, (a) The state space, showing the cost for
each operator, (b) Progression of the search. Each node is labelled with g(n).
At the next step, the goal node with g = 10 will be selected.
Depth-first search
Depth-first search always expands one of the nodes at the deepest level
of the tree. Only when the search hits a dead end (a nongoal node with no
expansion) does the search go back and expand nodes at shallower levels.
This strategy can be implemented by GENERAL-SEARCH with a queuing
function that always puts the newly generated states at the front of the
queue. Because the expanded node was the deepest, its successors will be
even deeper and are therefore now the deepest. The progress of the search is
illustrated in Figure 1.11.
Depth-first search has very modest memory requirements. As the figure
shows, it needs to store only a single path from the root to a leaf node, along
with the remaining unexpanded sibling nodes for each node on the path. For a
state space with branching factor b and maximum depth m, depth-first search
requires storage of only bm nodes, in contrast to the b d that would be required
by breadth-first search in the case where the shallowest goal is at depth d.
Using the same assumptions as Figure 1.9, depth-first search would require 12
28

kilobytes instead of 111 terabytes at depth d=12,a factor of 10 billion times

less space.
The time complexity for depth-first search is O(b m). For problems that
have very many solutions, depth-first may actually be faster than breadthfirst, because it has a good chance of finding a solution after exploring only a
small portion of the whole

Figure 1.11 Depth-first search trees for a binary search tree. Nodes at depth 3
are assumed to have no successors.
space. Breadth-first search would still have to look at all the paths of length d
- 1 before considering any of length d. Depth-first search is still O(b m) in the
worst case. The drawback of depth-first search is that it can get stuck going
down the wrong path. Many problems have very deep or even infinite search
trees, so depth-first search will never be able to recover from an unlucky
choice at one of the nodes near the top of the tree. The search will always
continue downward without backing up, even when a shallow solution exists.
Thus, on these problems depth-first search will either get stuck in an infinite
29

loop and never return a solution, or it may eventually find a solution path that
is longer than the optimal solution. That means depth-first search is neither
complete nor optimal. Because of this, depth-first search should be avoided
for search trees with large or infinite maximum depths.
It is trivial to implement depth-first search with GENERAL-SEARCH:
function DEPTH-FlRST-SEARCH(problem) returns a solution, or failure
GENERAL-SEARCH(problem,ENQUEUE-AT-FRONT)

It is also common to implement depth-first search with a recursive

function that calls itself on each of its children in turn. In this case, the queue
is stored implicitly in the local state of each invocation on the calling stack.
Depth-limited search
Depth-limited search avoids the pitfalls of depth-first search by
imposing a cutoff on the maximum depth of a path. This cutoff can be
implemented with a special depth-limited search algorithm, or by using the
general search algorithm with operators that keep track of the depth. For
example, on the map of Romania, there are 20 cities, so we know that if there
is a solution, then it must be of length 19 at the longest. We can implement
the depth cutoff using operators of the form "If you are in city A and have
travelled a path of less than 19 steps, then generate a new state in city B with
a path length that is one greater." With this new operator set, we are
guaranteed to find the solution if it exists, but we are still not guaranteed to
find the shortest solution first: depth-limited search is complete but not
optimal. If we choose a depth limit that is too small, then depth-limited search
is not even complete. The time and space complexity of depth-limited search
is similar to depth-first search. It takes O(b') time and O(bl) space, where / is
the depth limit.
Iterative deepening search
The hard part about depth-limited search is picking a good limit. We picked 19
as an "obvious" depth limit for the Romania problem, but in fact if we studied
the map carefully, we would discover that any city can be reached from any
other city in at most 9 steps. This number, known as the diameter of the
state space, gives us a better depth limit, which leads to a more efficient
depth-limited search. However, for most problems, we will not know a good
depth limit until we have solved the problem.
Iterative deepening search is a strategy that sidesteps the issue of
choosing the best depth limit by trying all possible depth limits: first depth 0,
30

then depth 1, then depth 2, and so on. The algorithm is shown in Figure 1.12.
In effect, iterative deepening combines the benefits of depth-first and
breadth-first search. It is optimal and complete, like breadth-first search, but
has only the modest memory requirements of depth-first search. The order of
expansion of states is similar to breadth-first, except that some states are
expanded multiple times. Figure 1.13 shows the first four iterations of
ITERATIVE-DEEPENING-SEARCH on a binary search tree. Iterative deepening
search may seem wasteful, because so many states are expanded multiple
times. For most problems, however, the overhead of this multiple expansion is
actually rather small.
function iTERATiVE-DEEPENiNG-SEARCH(praWem) returns a solution sequence
inputs: problem, a problem
for depth < 0 to do
if DEPTH-LlMITED-SEARCH(proWem, depth) succeeds then return its result
end
return failure
Figure 1.12 The iterative deepening search algorithm

Figure 1.13 Four iterations of iterative deepening search on a binary tree.

31

Intuitively, the reason is that in an exponential search tree, almost all of the
nodes are in the bottom level, so it does not matter much that the upper
levels are expanded multiple times. Recall that the number of expansions in a
depth-limited search to depth d with branching factor b is
1+b+b2+....+bd-2+bd-1+bd
To make this concrete, for b = 10 and d = 5. the number is
1 + 10+100+1,000+10,000+ 100,000= 111,111
In an iterative deepening search, the nodes on the bottom level are
expanded once, those on the next to bottom level are expanded twice, and so
on, up to the root of the search tree, which is expanded d + 1 times. So the
total number of expansions in an iterative deepening search is
(d + 1)1 + (d)b + (d- \)b2 + + 3bd-2 + 2bd-1 + bd
Again, for b = 10 and d = 5 the number is
6 + 50 + 400 + 3,000 + 20,000+100,000= 123,456
All together, an iterative deepening search from depth 1 all the way down
to depth d expands only about 11 % more nodes than a single breadth-first or
depth-limited search to depth d, when b = 10. The higher the branching
factor, the lower the overhead of repeatedly expanded states, but even when
the branching factor is 2, iterative deepening search only takes about twice as
long as a complete breadth-first search. This means that the time complexity
of iterative deepening is still O(bd), and the space complexity is O(bd). In
general, iterative deepening is the preferred search method when there is a
large search space and the depth of the solution is not known.
Bidirectional search
The idea behind bidirectional search is to simultaneously search both forward
from the initial state and backward from the goal, and stop when the two
searches meet in the middle (Figure 1.14). For problems where the branching
factor is b in both directions, bidirectional search can make a big difference. If
we assume as usual that there is a solution of depth d, then the solution will
be found in O(2bd/2) = O(bd/2) steps, because the forward and backward
searches each have to go only half way. To make this concrete: for b = 10 and
d = 6, breadth-first search generates 1,111,111 nodes, whereas bidirectional
search succeeds when each direction is at depth 3, at which point 2,222
nodes have been generated. This sounds great in theory. Several issues need
to be addressed before the algorithm can be implemented.

The main question is, what does it mean to search backwards from the
32

goal? We define the predecessors of a node n to be all those nodes that

have n as a successor. Searching backwards means generating
predecessors successively starting from the goal node.
When all operators are reversible, the predecessor and successor sets are
identical; for some problems, however, calculating predecessors can be
very difficult.
What can be done if there are many possible goal states? If there is an
explicit list of goal states, such as the two goal states in Figure 1.1, then
we can apply a predecessor function to the state set just as we apply the
successor function in multiple-state search. If we only have a description
of the set, it may be possible to figure out the possible descriptions of
"sets of states that would generate the goal set," but this is a very tricky
thing to do. For example, what are the states that are the predecessors of
the checkmate goal in chess?
There must be an efficient way to check each new node to see if it already
appears in the search tree of the other half of the search.
We need to decide what kind of search is going to take place in each half.
For example, Figure 1.14 shows two breadth-first searches. Is this the best
choice?

Fig 1.14 A schematic view of a bidirectional breadth-first search that is about

to succeed, when a branch from the start node meets a branch from the goal
node.
The O(bd/2) complexity figure assumes that the process of testing for
intersection of the two frontiers can be done in constant time (that is, is
independent of the number of states). This often can be achieved with a hash
table. In order for the two searches to meet at all, the nodes of at least one of
them must all be retained in memory (as with breadth-first search). This
means that the space complexity of uninformed bidirectional search is O(bd/2).

33

Comparing search strategies

Figure 1.15 Evaluation of search strategies, b is the branching factor; d is the

depth of solution; m is the maximum depth of the search tree; / is the depth
limit.
1.7 Production system characteristics
The production system are a good way to describe the operations that can
be performed in a search for a solution to a problem. Two questions we might
reasonably ask at this point are:
1.

2.

Can production system, like problems be described by a set of

characteristics that shed some light on how they can easily be
implemented?
If so what relationships are there between types and types of production
systems best suited to solving the problems?

The answer to the first question is yes. Consider the following definitions
of classes of production system. A monotonic production system is a
production system is one in which this is not true. A partially commutative
production system is a production system with the property that if the
application of a particular sequence of rules transform state x into state y
then any permutation of those rules that is allowable also transforms state x
into state y. A commutative production system is a production system that is
both monotonic and partially commutative.
The significance of these categories of production system lies in the
relationship between categories and appropriate implementation strategies.
But before discussing relationship, it might be helpful to make the meanings
of the definitions cleverer by showing how they relate to specific problems.
Thus we arrive at the second question above, which asked whether there
is interesting relationship between classes of production systems and classes
of problems. For any solvable problem, there exist an infinite number of
production systems that describe way to find solutions. Some will be more
34

natural or efficient than others. Any problem that can be solved by any
production system can be solved by a commutative one, but the commutative
one may be so unwieldy as to be practically useless. It may use individual
states to represent entire sequence of applications of rules of a simpler, non
commutative system. So in a formal sense there is no relationship between
kinds of problem and kinds of production systems since all problems can be
solved by all kind of production systems. But in a practical sense, there
definitely is such a relationship between kinds of problem and kinds of
systems that lend themselves naturally to describing those problems. To see
this let us look at a few examples. The table below shows the four categories
of production systems produced by dichotomies, monotonic versus nonmonotonic and partially commutative versus nonpartially commutative, along
with some problems that can naturally be solved by each type of system.

Partially commutative
Not
partially
commutative

Monotonic
Theorem proving
Chemical synthesis

Non-monotonic
Robot navigation
Bridge

Four categories of production system

The upper left corner represents commutative systems.
Partially commutative, monotonic production systems are useful for
solving ignorable problems. This is not surprising since the definitions of the
two are essentially the same. But recall that ignorable problems are those for
which a natural formulations lead to solution steps that can be ignored. Such
a natural formulation will then be a partially commutative, monotonic system.
Problems that involve creating new things rather than changing old ones are
generally ignorable. Theorem proving as we described is one example of such
a creative process. Making deductions from some known facts is a similar
creative process. Both of those processes can easily be implemented with a
partially commutative, monotonic system.
Partially commutative, monotonic production system are important from
an implementation standpoint because they can be implemented without the
ability to backtrack to previous states when it is discovered that an incorrect
path has been followed. Although it is often useful to implement such systems
with backtracking in order to guarantee a systematic search, the actual
database representing the problem state need not be restored. This often
results in a considerable increase in efficiency, particularly because, since the
database will never have to be restored, it is not necessary to keep track of
where in the search process every change was made.

35

We have now discussed partially commutative production system that are

also monotonic. They are good for problems where things do not change; new
things get created. Nonmonotonic, partially commutative systems, on the
other hand, are useful for problems in which changes occur but can be
reversed and in which order of operations is not critical. This is usually the
case in physical manipulation problems, such as robot navigation on a flat
plane. Suppose that a robot has the following operation: go north(N), go
east(E), go south(S), go west(W). To reach its goal, it does not matter whether
the robot executes N-N-E or N-E-N. Depending on how the operators are
chosen, the 8-puzzle and the blocks world problem can also be considered
partially commutative.
Both types of partially commutative production systems are significant
from an implementation point of view because they tend to lead to many
duplications of individual states during the search process.
Production systems are not partially commutative are useful for many
problems in which irreversible changes occur. For example, consider the
problem of determining a process to produce a desired chemical compound.
The operators available include such things as Add chemical x to the pot or
Change the temperature to t degrees. These operators may cause
irreversible changes to the portion being brewed. The order in which they are
performed can be very important in determining the final output. It is possible
if x is added to y, a stable compound will be formed, so later addition of z will
have no effect; if z is added to y, however, a different stable compound may
be formed, so later addition of x will have no effect. Nonpartially commutative
production systems are less likely to produce the same node many times in
the search process. When dealing with one that describes irreversible
processes, it is particularly important to make correct decision the first time,
although if the universe is predictable, planning can be used to make less
important.

1.8 Issues in design of search programs

Every search process can be viewed as a traversal of tree structure in
which each node represents a problem state and each arc represents
relationship between the states represented by the nodes it connects. For
example, the tree structure given below shows part of a search tree for a
water jug problem. The arcs have not been labeled in the fig., but they
correspond to a particular water pouring operations. The search process must
find a path or paths in principle, be constructed in its entirety from the rules
that define allowable moves in the problem space. But, in practice, most of it
never is. It is too large and most of it need never be explored. Instead of first
36

building the tree explicitly and then searching it, most search programs
represent the tree implicitly in the rules and generate explicitly only those
parts that they decide to explore. Throughout our discussion of search
methods, it is important to keep in mind this distinction between implicit
search trees and explicit partial trees that are actually constructed by the
search program.

A search tree for water jug problem

We need to discuss about some important issues that arise in general
search problem techniques.
The direction in which to conduct the search(forward versus backward
reasoning). We can search forward through the state space from the start
state to goal state, or we can search backward from the goal.
How to select applicable rules(matching). Production systems partially
spend most of their time looking for rules to apply, so it is critical to have
efficient procedures for matching rules against states.
How to represent each node of the search process(the knowledge
representation process and the frame process). For problems like chess, a
node can be fully represented by a single array. In more complex problem
solving, however, it is inefficient and/or impossible to represent all of the
facts in the world and to determine all of the side effects an action may
have.
One other issue we should consider at this point is that of search trees
versus search graphs. As mentioned above we can think of production rules of
generating nodes in a search tree. Each node can be expanded in turn,
generating a set of successors. This process continues until a node
representing solution is found. Implementing such a procedure requires little
bookkeeping. However, this process often results in the search node being
generated as part of several paths so being processed more than once. This
happens because the search space may really be an arbitrary directed graph
rather than a tree.
For example, in the tree shown above, the node (4,3), representing 4gallons of water in one jug and 3-gallons of in other, can be generated either
37

by first filling 4-gallon jug and then the 3-gallon one or by filling them in
opposite order. Since the order does not matter, continuing to process both
these nodes would be redundant. This example also illustrates another
problem that often arises when the search process operate as a tree walk. On
the third level, the node (0,0) appears.(In fact it appears twice). But this is the
same as the top node of the tree, which has already been expanded. Those
two path have not gotten Us anywhere. So we would like to eliminate them
and continue only along the other branches.
The waste of effort that arises when the same node is generated more
than once can be avoided at the price of additional bookkeeping. Instead of
traversing a search tree, we traverse a directed graph. This graph differs from
a tree in that several paths may come together at a node. The graph
corresponding to the above tree is:

A search graph for the water jug problem

Any tree search procedure that keeps track of all the nodes that have
been generated so far can be converted to a graph search procedure by
action performed each time a node is generated. Notice that of the two
systematic search procedures we have discussed so far, this requirement that
nodes be kept track of it met by breadth-first search but not depth-first
search. But, of course, depth-first search could be modified, at the expense of
additional storage, to retain in memory nodes that have been expanded and
then backed-up over. Since all nodes are saved in the search graph, we must
use the following algorithm instead of simply adding a new node to the graph.

Algorithm: Check duplicate nodes

1. Examine the set of nodes that have been created so far to see if the new
node already exists.
2. If it does not-simply add it to the graph just as for a tree.
3. If it does not already exist, then do the following:
(a) Set the node that is being expanded to point to the already existing
node corresponding to its successor rather than to the new one. The new one
38

can simply been thrown away.

(b) If you are keeping track of the best(shortest or otherwise least cost)
path to each node, then click to see if the new path is better or worse than
the old one. If worse, do nothing. If better, record the new path as the correct
path to use to get to the node and propagate the corresponding change in
cost down through successor nodes as necessary.
One problem that may arise here is that cycles may be introduced into
search graph. A cycle is a path in which a given node appears more than
once. For example, the graph above contains two cycles of length two. One
includes the node (0,0)and (4,0); the other includes the nodes (0,0) and (0,3).
Whenever there is a cycle, there can be paths of arbitrary length. Thus it may
become more difficult to show that a graph traversing algorithm is guaranteed
to terminate.
Treating the search process as a graph search rather than as a tree search
reduces the amount of effort that is spent exploring essentially the same path
several times. But it requires additional efforts each time a node is generated
to see if it has been generated before. Whether this effort is justified depends
on the particular problem. If it is very likely that the same node will be
generated in several different ways, then it is more worthwhile to use a graph
procedure than if such duplication will happen only rarely.
Graph search procedures are especially useful for dealing with partially
commutative problem systems in which a given set of operations will produce
the same results regardless of the order in which the operations are applied. A
systemic search procedure will try many of the permutations of these
operators and so will generate the same node many times. This is exactly
what happened in water jug example shown above.

39

UNIT II

2.1 Game Artificial Intelligence

AI is catagorised into two types namely

Weak AI non-sentient computer intelligence, typically focused on a

narrow task. The intelligence of weak AI is limited.

Artificial general intelligence (strong AI) hypothetical artificial intelligence

at least as smart as a human. Such an AI would be recursive, in that it
could improve itself. In successive intervals of increased intelligence, such
an entity could theoretically achieve superintelligence in a relatively short
period of time. One or more superintelligences could potentially change
the world so profoundly and at such a high rate, that it may result in
a technological singularity. Strong AI does not yet exist. The prospect of its
creation inspires expections of both promise and peril, and has become the
subject of an intense ongoing ethical debate.
Game AI : a weak AI which gives non-player characters hum an-level
intellect, appearance of having different personalities, portraying emotions
or various dispositions, etc., making game more immersive, challenging
and fun.

2.2 Types of game AI

The many applications areas of Artificial Intelligence (AI) include neural
networks, fuzzy logic, genetic algorithms, virtual reality, and intelligent
agents. AI in gaming is an attempt to build intelligent virtual worlds and
artificial characters, or to create environments that are similar to the human
decision making in certain aspect.
There are three fundamental types of AI in games:

1- Roaming AI
2- Behavioral AI
3- Strategic AI
1- Roaming AI: It models the movement of game objects. i.e. the decisions
made by game objects that determine how they roam around in a virtual
game world.

Basically, roaming AI is used whenever a computer controlled object must make a

decision to alter its current path, either to achieve the desired result in the game or
simply to conform to a particular movement pattern.

40

There are three different types of Roaming AI namely:

Chasing
Evading
Patterned roaming
Chasing: It is a type of roaming AI in which a game object tracks and goes after
another game object or objects. This approach is used in many shoot-em-up games
where an alien or enemy chases after players ship or car or bike or the player himself.
It is implemented by altering the aliens velocity or position based on the current
position of the players ship.
Evading: It is the logical counterpart to chasing. it is another type of roaming
AI in which a game object tries to get away from another object or objects.
Patterned roaming: It is a type of roaming AI that uses a predefined set of
movements for a game object. Patterns can include circles,figure eights,
zigzags, or even more complex movements.

2- Behavioral AI: This type of gaming AI uses a mixture of all three roaming AI
algorithms to give game objects specific behaviors. eg. if you want the alien or enemy
to chase sometimes, evade other times, follow a pattern some other times, and may be
even act randomly every once in a while. behavioral AI can also be used to alter the
difficulty of the game. eg. You would like to favor a chasing algorithm more than
random or patterned movement to make the aliens or enemies more aggressive in the
higher levels of a game.
3- Strategic AI: Any AI designed to play a game with a fixed set of well defined rules
is known as strategic AI. eg. a computer controlled chess player would use strategic
AI to determine each move based on trying to improve the chances of winning the
game.
Strategic AI tend to vary more based on the nature of the game because it is so tightly
linked to the rules of the game.
Example:
The prototypes of two games have been created integrating roaming,
behavioral and strategic AI: Family Guy Strategy game based on themes from
the television cartoon Family Guy. This game is trying to provide another
way for Family Guy fans to pass the time until a new episode or another movie
is created. This game is a combination of few smaller games merged as
levels. These individual levels depict particular Family Guy episodes or the
movie and elaborated to create a way for a Family Guy fan to take part in a
particular event. The objective of the game is to navigate the family
(FatherPeter, Mother-Lois, Daughter-Meg, Son-Chris, Baby-Stewie, Dog-Brian)

41

through their hometown of Quahog to reach the Quahog Airport for a family
vacation to Milwaukee. Each different level displays different characters and
different types of gameplay. Level 1 (Lois Hunter) This level is based off of the
game called Deer Hunter. he only character in this level is Lois. When the
level is started Lois automatically appears on the screen and starts moving in
a random direction. The goal of the level is to click on Lois as many times as
possible to get a high score. Each time the player clicks on Lois the score
increases by ten points. Also, each time the player clicks on her she jumps to
a random position on the screen and starts moving in a random direction. Her
speed increases slightly each time you click on her as well which increases
the difficulty of playing. After about 1.5 seconds Lois direction changes
automatically to prevent predictability. The level ends after a counter runs out
of time and if the players score is high enough then the player goes to the
next level. If not, then this level restarts and the player tries again. Level 2
(Peter vs. The Giant Chicken) Peter vs. The Giant Chicken In two episodes
Peter battles The Giant Chicken. The theme of this level is a spoof off of
Mortal Kombat, an old school fighting game which involves handto-hand
combat. At the beginning of the level Peter and The Giant Chicken start off on
different sides of the screen. The player controls Peter using the arrow keys.
The Giant Chicken is set to randomly move about the level and to react in an
attack or block when the player (Peter) is too close. Other features in this level
include a health counter, life counter, and a timer. If either Peter or The Giant
Chicken has 0 health then that character will lose a life and then the two
players will restart at their original positions, thus starting a new round. There
is a maximum of three rounds and 30 seconds allowed for each. To pass this
level, the player must beat The Giant Chicken two times or else if The Giant
Chicken wins twice then the level restarts all of the counters and the player
must go through it all again. Ong Bak: Quest of the Thai Warrior Action game
based on the movie Ong Bak. Sacred incarnation of Buddha, you must
retrieve this statue head in order to regain the protection for your village, time
is running out. While searching the world for the sacred head of Ong Bak you
will travel to many exciting and different places. Ting from Pradu is an
incredibly gifted fighter who has honed his skills and abilities over a lifetime.
He has been named a protector of the village and has sworn to retrieve the
sacred head of Ong Bak. Although he upholds his teaching to fight only when
absolutely necessary, he will need all his skill in order to save the village from
disaster. It was decided that the game should be implemented as a sidescrolling fighting game with AI components. The tracking system for all
characters of the game is based off chasing and evading tactics. Several
markers were set in place to log each movement and corresponding (x,y)
coordinates. These markers were set on the main character and enemies. By
using a chance variable with the probability of 75%, the enemy moves
towards the player character analyzing the distance and (x,y) coordinates of
the characters. These coordinates are taken from the collision boxes created
42

for each sprite. With the probability of 25%, the enemy moves away from the
player in a random direction to simulate the evasion.
Features
The following components have been integrated into the engine of both
games:
completed storylines;
developed levels, character sprites, worlds, and AI;

implemented scripts to manage the following: character controls, AI

movements, miscellaneous world elements.
Other AI relevant features include:

multiplayer environment;
ability to chose a path in the game; ability to change level difficulty;
penalty for cheating, etc. Our games run on the Windows platform and
require DirectX 8 or higher compatible graphics card with at least 16MB of
video memory (preferably 32 MB or more). These games also require a
monitor that is capable of 1024x768 viewing resolution. The characters
are rendered in a 2D format. Figures 2.1 and 2.2 give some screenshots of
these games.

43

Figure 2.1: Game screenshots from Ong Bak: single enemy, and group attack

44

Figure 2.2: Game screenshots from Family Guy: Lois Hunter, and Peter vs. The
Giant Chicken
2.3 Chasing and evading
The chasing and evading problem (evading is virtually the opposite of
chasing) consists of 3 parts: the decision to initiate a chase or evade, the
effecting the chase or evade, and obstacle avoidance.
The environment can be either tiled or continuous
tile-based: the game domain is divided into discrete tiles, and the
player's position is represented by integers, the position is fixed to a discrete
tile, movement goes tile by tile, the number of directions in which the player
can make is limited, and characters appear to move faster when moving along
a diagonal path

45

continuous environment: position is represented by floating-point

coordinates, which can represent any location in the game domain, and the
player also is free to head in any direction.

Basic chasing: correcting the predator's coordinates based on the prey's

coordinates so as to reduce the distance between their positions, the
chasing or evading seems almost too mechanical.
// basic chasing
if ( predatorX > preyX )
{
predatorX--;
}
else if ( predatorX < preyX )
{
predatorX++;
}
if ( predatorY > preyY )
{
predatorY--;
}
else if ( predatorY < preyY )
{
predatorY++;
}

Line-of-sight chasing appears m ore natural, direct, and intelligent than

basic chasing in a tiled-base game, there are only 8 possible direction of
movements, to minimize the jaggy and sometimes jumpy appearance, it
is important to move only adjacent tiles when changing positions.
The Bresenham 's algorithm will never draw 2 adjacent pixels along a
line's shortest axis, and it will walk along the shortest possible path
between the starting and ending points.
Once the target moves, the pre-calculated path becomes obsolete, and
need to be re-calculated.
The algorithm will determine which axis is the longest, then traverse the
long axis, calculating each point of the line along the way.

// path direction calculation

deltaRow = endRow - currRow; deltaCol = endCol - currCol;
stepRow = ( deltaRow < 0 ) ? -1 : 1;
stepCol = ( deltaCol < 0 ) ? -1 : 1;
deltaRow = abs(deltaRow * 2);
deltaCol = abs(deltaCol * 2);

46

// path initialization
for ( currStep = 0; currStep < kMaxPathLength; currStep++ )
{
pathRows[currStep] = pathCols[currStep] = -1;
}
currStep = 0;
pathRows[currStep] = currRow;
pathCols[currStep] = currCol;
currStep++;
// bresenham algorithm
if ( daltaCol > deltaRow )
{
fraction = deltaRow * 2 - daltaCol;
while ( currCol != endCol )
{
if ( fraction >= 0 )
{
currRow += stepRow; fraction -= deltaCol;
}
currCol += stepCol;
fraction += deltaRow;
pathRow[currentStep] = currRow; pathCol[currentStep++] = currCol;
}
}
else
{
fraction = deltaCol * 2 - daltaRow; while ( currRow != endRow )
{
if ( fraction >= 0 )
{
currCol += + stepCol;
fraction -= deltaRow;
}
currRow += stepRow;
fraction += deltaCol;
pathRow[currentStep] = currRow;
pathCol[currentStep++] = currCol;
}
}

In a continuous environment, use physics engine, where the game entities

47

are driven by applied forces and torques.

A simple 2-dimensional rigid-body physics engine, the line-of-sight
algorithm controls the predator by applying thrust for forward motion and
activate steering forces for turning so as to keep its heading toward the
prey at all times.
The predator can't always turn on a dim e, but turning radius is a function
of their linear speed.
Global and local coordinate systems
A global (earth-fixed) coordinate system is fixed and doesn't move,
whereas a local (body-fixed) coordinate system rotates with the object
to which it is attached and is locked onto objects that move around
within the global coordinate system
x = X cosQ + Y sinQ; y = -X sinQ + Y cosQ;
where (x, y) are the local coordinates of the global point (X, Y)

// line-of-sight chasing
left = right = false;
v = vRotate2D( -predator.fOrientation, prey.vPosition - predator.vPosition );
v.normalize( );
if ( v.x < -TOL )
{
left = true;
}
else if ( v.x > TOL )
{
right = true;
}
predator.setThrusters(left, right);

The unit vector v points from the predator directly toward the prey.
If the x-position of the prey, in term s of the predator's local coordinate
system , is negative, the prey is somewhat to the starboard side of the
predator (turn right); if the prey's x-coordinate is positive, it is somewhat
on the port side of the predator (turn left).
The predator always head directly toward the prey and most likely end up
right behind it, unless it is moving so fast that it overshoots the prey, in
which case it will loop around and head toward the prey again.
To prevent overshooting, use speed control logic to allow the predator to
slow down as it gets closer to the prey.
Heading directly toward the prey is not always the shortest path in terms
of range to target, or, perhaps, time to target.

48

2.4 Backtracking
Backtracking is a general algorithm for finding all (or some) solutions to
some computational problems, notably constraint satisfaction problems, that
incrementally builds candidates to the solutions, and abandons each partial
candidate c ("backtracks") as soon as it determines that c cannot possibly be
completed to a valid solution.
The classic textbook example of the use of backtracking is the eight
queens puzzle, that asks for all arrangements of eight chess queens on a
standard chessboard so that no queen attacks any other. In the common
backtracking approach, the partial candidates are arrangements of k queens
in the first k rows of the board, all in different rows and columns. Any partial
solution that contains two mutually attacking queens can be abandoned.
Backtracking can be applied only for problems which admit the concept of
a "partial candidate solution" and a relatively quick test of whether it can
possibly be completed to a valid solution. It is useless, for example, for
locating a given value in an unordered table. When it is applicable, however,
backtracking is often much faster than brute force enumeration of all
complete candidates, since it can eliminate a large number of candidates with
a single test.
Backtracking is an important tool for solving constraint satisfaction
problems, such as crosswords, verbal arithmetic,Sudoku, and many other
puzzles. It is often the most convenient technique for parsing, for
the knapsack problem and other combinatorial optimization problems. It is
also the basis of the so-called logic programming languages such
as Icon, Planner and Prolog.
Backtracking depends on user-given "black box procedures" that define
the problem to be solved, the nature of the partial candidates, and how they
are extended into complete candidates. It is therefore a metaheuristic rather
than a specific algorithm although, unlike many other meta-heuristics, it is
guaranteed to find all solutions to a finite problem in a bounded amount of
time.
2.4.1 Constraint Satisfaction problem
A constraint satisfaction problem (or CSP) is a special kind of problem
that satisfies some additional structural properties beyond the basic
requirements for problems in general. In a CSP, the states are defined by the
values of a set of variables and the goal test specifies a set of constraints
that the values must obey. For example, the 8-queens problem can be viewed
49

as a CSP in which the variables are the locations of each of the eight queens;
the possible values are squares on the board; and the constraints state that
no two queens can be in the same row, column or diagonal. A solution to a
CSP specifies values for all the variables such that the constraints are
satisfied.Many kinds of design and scheduling problems can be expressed as
CSPs, so they form a very important subclass. CSPs can be solved by generalpurpose search algorithms, but because of their special structure, algorithms
designed specifically for CSPs generally perform much better.
Constraints come in several varieties. Unary constraints concern the value
of a single variable. For example, the variables corresponding to the leftmost
digit on any row of acryptarithmetic puzzle are constrained not to have the
value 0. Binary constraints relate pairs of variables. The constraints in the 8queens problem are all binary constraints. Higher-order constraints involve
three or more variablesfor example, the columns in the cryptarithmetic
problem must obey an addition constraint and can involve several variables.
Finally, constraints can be absolute constraints, violation of which rules out a
potential solution, or preference constraints that say
which solutions are preferred.
Each variable Vi in a CSP has a domain Di, which is the set of possible
values that the variable can take on. The domain can be discrete or
continuous. In designing a car, for instance, the variables might include
component weights (continuous) and component manufacturers (discrete). A
unary constraint specifies the allowable subset of the domain, and a binary
constraint between two variables specifies the allowable subset of the crossproduct of the two domains. In discrete CSPs where the domains are finite,
constraints can be represented simply by enumerating the allowable
combinations of values. For example, in the 8-queens problem, let V1 be the
row that the first queen occupies in the first column, and let V2 be the row
occupied by the second queen in the second column. The domains of V1 and
V2 are {1,2,3,4,5,6,7,8}. The no-attack constraint linking V1 and V2 can be
represented by a set of pairs of allowable values for V1 and V2: {{1,3}, (1,4),
{1,5},... ,(2,4), (2,5),...} and so on. Altogether, the no-attack constraint
between V\ and Vi rules out 22 of the 64 possible combinations. Using this
idea of enumeration, any discrete CSP can be reduced to a binary CSP.
Constraints involving continuous variables cannot be enumerated in this
way, and solving continuous CSPs involves sophisticated algebra. In this
chapter, we will handle only discrete, absolute, binary (or unary) constraints.
Such constraints are still sufficiently expressive to handle a wide variety of
problems and to introduce most of the interesting solution methods.
Let us first consider how we might apply a general-purpose search
50

algorithm to a CSP. The initial state will be the state in which all the variables
are unassigned. Operators will assign a value to a variable from the set of
possible values. The goal test will check if all variables are assigned and all
constraints satisfied. Notice that the maximum depth of the search tree is
fixed at n, the number of variables, and that all solutions are at depth n. We
are therefore safe in using depth-first search, as there is no danger of going
too deep and no arbitrary depth limit is needed.

In
the
most
naive
implementation, any unassigned
i
i
variable in a given state can be
assigned a value by an operator, in
which case the branching factor would be as high as |Di | or 64 in the 8-queens
problem. A better approach is to take advantage of the fact that the order of
variable assignments makes no difference to the final solution. Almost all CSP
algorithms therefore generate successors by choosing values for only a single
variable at each node. For example, in the 8-queens problem, one can assign
a square for the first queen at level 0, for the second queen at level 1, and so
on. This results in a search space of size |Di|, or 88 in the 8-queens problem. A
straight forward depth-first search will examine all of these possibilities.
Because CSPs include as special cases some well-known NP-complete
problems such as 3SAT,
we cannot expect to do better than exponential complexity in the worst case.
In most real problems, however, we can take advantage of the problem
structure to eliminate a large fraction of the search space. The principal
source of structure in the problem space is that, in CSPs, the goal test is
decomposed into a set of constraints on variables rather than being a "black
box.
Depth-first search on a CSP wastes time searching when constraints have
already been violated. Because of the way that the operators have been
defined, an operator can never redeem a constraint that has already been
violated. For example, suppose that we put the first two queens in the top
row. Depth-first search will examine all 8 6 possible positions for the remaining
six queens before discovering that no solution exists in that subtree. Our first
improvement is therefore to insert a test before the successor generation step
to check whether any constraint has been violated by the variable
assignments made up to this point. The resulting algorithm, called
backtracking search, then backtracks to try something else.
Backtracking also has some obvious failings. Suppose that the squares
chosen for the first six queens make it impossible to place the eighth queen,
because they attack all eight squares in the last column. Backtracking will try
all possible placings for the seventh queen, even though the problem is
already rendered unsolvable, given the first six choices. Forward checking
avoids this problem by looking ahead to detect unsolvability. Each time a
51

variable is instantiated, forward checking deletes from the domains of the asyet-uninstantiated variables all of those values that conflict with the variables
assigned so far. If any of the domains becomes empty, then the search
backtracks immediately. Forward checking often runs far faster than
backtracking and
is very simple to implement.
Forward checking is a special case of arc consistency checking. A state
is arc-consistent if every variable has a value in its domain that is consistent
with each of the constraints on that variable. Arc consistency can be achieved
by successive deletion of values that are inconsistent with some constraint. As
values are deleted, other values may become inconsistent because they relied
on the deleted values. Arc consistency therefore exhibits a form of
constraint propagation, as choices are gradually narrowed down. In some
cases, achieving arc consistency is enough to solve the problem completely
because the domains of all variables are reduced to singletons. Arc
consistency is often used as a preprocessing step, but can also be used during
the search.
2.5 Strategy AI

The strategic AI itself is most common in real-time strategy (RTS) games but has
been making its way more and more into tactical FPS games. The player-controlled
commander can be its own system or set up as an empty entity-one that does not have
a place or graphic in the world but is updated and thinks.
The commanders are going to be guided by hierarchical rules systems and FSMs,
which govern elements such as resource gathering, researching up the tech tree,
building the army, and so on. For the most part, this basic upkeep of the game
elements does not require much actual thought. What does require intelligence is the
interactions with the other players.
Governing these interactions (or warfare) is where the real meat and potatoes
lie with strategic AI. The commander needs to explore the game map to find the
player, identify key points of interest such as choke points, build defenses, and
analyze other player's defenses. How one would do this is not obvious but can be
made easier with decision maps.
Decision Maps
Decision maps are two-dimensional arrays approximating the game map. Each cell in
the array corresponds to a region in the game and is filled with important information
about the region. These maps are used to help your strategic AI make meaningful
decisions about the game as a whole.

52

Resource Maps
Resource maps contain information about where the resources are laid out in a
strategy game. Knowing where on the maps resources are concentrated can influence
many decisions that a commander will make. Where to deploy expansions or satellite
bases (resources near the commanders base), where the enemy is likely to deploy its
expansions (resources near their base) and places likely to be contested (resources in
the middle).
Getting a count of potential available resources also influences which units are
supported and how to deploy the army. If resources are scarce, each unit must be used
more carefully, because replacements are less likely. If resources are plentiful, it opens
up the strategies of massing cheap units or building powerful units.
Objective Maps
These maps are filled with information about the goals of the commander-for
example, locations of enemy bases, the positions of map objectives (blow something
up, protect something, hack something, etc.), and key units of the commanders army
(main base, hero units, etc). Tracking this information guides the use of the
commander's army. Points that need to be protected should be surrounded with
defensive structures, and a contingent of units should always be near these points.
Attack objectives will be sought out and defenses tested. Analysis of defenses around
targets must be undertaken to figure out the optimal way to overcome them. This
leads to the backbone of military games-conflict maps.
Conflict Maps
Conflict maps (see Figure 2.3) are used and updated far more often than the previous
maps. Conflict maps keep track of where battle occurs throughout the level.
Whenever one of the commanders units engages in a fight with an enemy, the unit
will update the conflict map with key information:

Type of conflict. Units, buildings, or both

Capabilities of units. Hits ground or air or both

Numbers. How many where encountered
Strength. How much damage potential is in this area

This information can be analyzed to determine the following:

When the enemy attacks, the AI will be able to determine whether the
defenses deployed are effective, whether the defenses are engaged or

53

ignored, and whether conflict arises close to defensive objectives. This can
lead to the AI changing the layout and composition of its defenses.

When attacking the enemy (preferably with scouts), weaknesses in their

defense can quickly be identified. If paths exist to the goal that can avoid
conflict or keep it reduced to a minimum, you have an optimal strategy to
deploy the troops.
Composition of the enemys military capabilities dictates the proper
countermeasures. Because most strategy games attempt to keep some level
of balance among the units, the computer will be able to determine the
number and type of units needed to break through defenses at different
points.

2.3 An example of how a conflict map might look laid over a view of the
terrain. The darker the red, the more conflicts encountered.

Building and Applying Maps

As mentioned earlier that these maps are built by the units of the commanders army.
It will be part of the rules that govern the AI to send scouts out to explore as soon as
possible in order to start building the maps. Good AI should periodically make sure
that these maps stay up to date. Early in the game, when only a few units are
maintaining these maps, updating should be no problem to the game engine. Later in
the game, with massive numbers of units all providing information at the same time, it
has the potential to affect performance.

54

Ensuring that the decision maps are maintained quickly is not too hard. You can
do so by putting the decision map system into its own thread. Indeed, each AI
controlled player should have its own thread to handle its own set of decisions maps.
The real performance gain happens if all of the entities are already broken up into
multiple threads. The decision map threads only need to handle the requests from the
parallelized entity update messages.
2.6 How to Create Strategically AI in Games
The biggest challenge in computer strategy games is the creation of a fun
computer opponent. The hardest element of "fun" is "good". For the Artificial
Intelligence to be good, it must do much of what a human player would do: situational
& map analysis, resource allocation, and strategy execution. Yhe framework and
some of the strategies which are developed at Activision in the development of games
such as Dark Reign: The Future Of War, Battlezone, Civilization: Call To Power, and
others are given here.
A relatively large number of high profile military strategy games was developed
by Ian Lane Davis, which includes: Dark Reign includes The Future Of War,
Battlezone, and Civilization: Call To Power.Perhaps the hardest part of developing
strategy games is the design and implementation of the Artificial Intelligence. This
model first breaks down the AI tasks into Tactical AI and Strategic AI, and then
further breaks down the Strategic AI into three parts: Analysis, Resource Allocation,
and High Level AI (personality).
Problem Definition
Before describing the details of the Dark Reign Model, let us define the
context of the problem and solutions. To generate a successful computer
opponent in a strategy game (or any computer game, for that matter), it
important to focus on what the developers true goal is:
Davis First Law of Computer Game AI: The goal of any AI is to lose the game.
This may seem counter-intuitive! Most programmers and researchers want to
develop algorithms and approaches that are optimal and perfect. There are
two problems, however. The first problem is that optimal is always hard and
often impossible. In classic games, such as chess and checkers, all players
can see the entire board at once, whereas in most modern computer strategy
games all players (including the computer) have incomplete knowledge of the
location and composition of the opponents pieces. And while its true that the
AI players can cheat knowledge, its very hard to hide that cheating from
players, and if the cheating is obvious, the game is not fun. Incomplete
knowledge precludes optimal decision-making. Furthermore, in comparison to
55

tradition games, the number of pieces, the number of types of pieces, and the
number of actions available to each piece are colossal. Optimal solutions such
as game trees [Tanimoto87] that try to predict several moves ahead (useful
for tic-tac-toe, and some other small games) are inappropriate and infeasible
due to the relatively enormous branching factor.
The second problem with trying to make an optimal and perfect AI player
is that, even if it were possible, it would be undesirable. Although most
players want a challenging AI opponent, nobody likes to lose all the time. The
goal is to make the player feel threatened, but then to make the player feel as
if she has heroically fought off a superior enemy [Millar96]. Frequently this
involves pushing the player to the edge and then intentionally backing off.
Such a tactic makes the player feel like they just barely held off the enemy
horde, but sent them fleeing with a to-the-lastman defense.
Fortunately or not, its easy enough to create an AI player that will lose a
game. The trick is to make an opponent that can create an exciting ebb and
flow of power. Towards this end, the computer player needs to be able to play
a devastatingly good game, and be able to back off when needed. The AI
must dynamically evaluate its situation, formulate a plan, and execute it. As a
final wrinkle, the AI system must be straightforward to use, as the games
level designers are often non-technical.

2.4 Strategy AI diagram

Turn-Based versus RTS
The approach is called as Dark Reign Model because it was first used on
Dark Reign: the Future of War. In the realm of computer games, Dark Reign is
known as an RTS, or Real-Time Strategy game. In an RTS, you can instruct a
unit, or a number of units, to move across the board, and as they move you
have time to move other pieces, and the opponent may also move his/her/its
pieces at the same time. In contrast, Civilization: Call To Power is a Turn-Based
game, which means that only one player can move units at a time, and within
56

a team, only one unit moves at once. The fundamental difference between an
RTS game and a Turn-Based game for an AI developer is that in a TurnBased
game the feedback for any decision is immediate: if you decide to move a
unit, that unit can be moved before anything else happens. However, in an
RTS, you can (and often have to) separate strategic troop and resource
commitment decisions from the tactical/low-level actions of units, whereas in
a Turn-Based game there is no such distinction.
Strategic AI
The goal of the Dark Reign Model for Strategic AI is to allow the creation of
computer opponents that respond to changes in fortune and circumstance. If
a computer opponent can retreat and regroup when things look bleak or
recognize a weakness in the enemys position and attack it,then the gameplay
can be vastly improved. To create this dynamic gameplay, our strategic AI
consists of three modules, an analysis module, a resource allocation module,
and a high level AI module. The first two modules performance is defined by
a set of parameters called an AI Personality (AIP, pronounced "ape"), and the
third module is a logical system which is responsible for changing AIPs when
new situations arise, and for triggering special actions based on certain
events occurring.Each module requires a different kind of AI techniques, and
each is described below.
Analysis Module
The ultimate goal of the analysis module is to define the current strategic
goals and rate them by priority. The strategic goals can include exploration,
reconnaissance, base construction, defensive goals, and offensive goals. In
Dark Reign, the goals were geographically defined: the map was divided into a
number of regions and each region has some offensive, defensive,
information, and/or resource values. These values are combined to make
some overall priority for each region. The analysis system would simply tell
the resource allocation system which areas troops should be sent to (and the
tactical AI handles attacks). Civilization: Call To Power, the goals are targetoriented, which means that each enemy city or unit can become its own goal.
Each goal is still rated by several values. The contribution of each value to the
importance of a goal is set by the level designer through an AIP configuration
file.
The question is "How do we arrive at these values, and what techniques
are useful?" We need to know things such as how far each spot on the map is
from our empire, how far each spot is from the enemy bases, which areas of
the map are dangerous, and which areas can we reach from which other
areas. These are map analysis problems, which are quite similar to computer
vision problems, and our solutions come from that realm (see [Davis96] for
applications of the following techniques in computer vision).
57

For example, if we want to concentrate our troops near areas we control,

we need to know how far any potential target is from those areas. The naive
approach is to look at a target and search for the nearest piece of our
territory. This is a major computational burden, so we can use a technique
known to the computer vision community as the grassfire algorithm [Duda73]
to compute the distance to our territory from every other spot on the map in a
simple two-pass raster scan algorithm.
Next, if we want to know how dangerous a given section of map is, we
start by finding the locations of all known enemy troops. We can divide the
map into regions and sum up the threat value in each region. But the danger
is that a region with no enemy units in it may be next to an area filled with
enemies. Thus, we can use relaxation techniques to spread the danger to
nearby regions [Press90].
As a final example, one pitfall is that you may be able to see a target that
is strategically important, but unreachable with your current units (such as an
island when you have no boats). Running a path finding algorithm from each
unit of yours to each potential target is too expensive, so you can use a
simple connected region extraction technique to label each movement
modalitys (air, land, water, etc.) contiguously reachable areas. If the label for
the land under a unit is different from the label under a target, it cant reach
the target. These regions can often be computed once at the beginning of the
game, and just stored for constant time look up.
These are just a few examples of analysis techniques. Some games may
require more complicated statistical analyses of terrain and troop
distributions, predictive extrapolations on the positions of troops that cannot
be seen any longer, clustering techniques for proper target definitions, etc.
Various computational geometry techniques such as Voronoi diagrams and
convex hull techniques are also applicable for computing distance related
metrics and cluster analysis [Preparata85]. The goals of the analysis are
simple, however: find the enemy & rank targets by importance.
Resource Allocation
The resource allocation module takes the strategic goals from the analysis
module, and allocates available troops to those goals. This module outputs
commands to the Tactical AI (or directly to the units). This system handles
matching forces against enemy forces, taking over terrain resources,
attacking and defending bases, etc. The actual decision on what troops to use
for what happens here.
The simple Bipartite Matching problem Matching our available forces to
58

our strategic goals can be thought of as a variant on the bipartite matching

problem [see Sedgewick92]. In that problem, you have resources that you
must maximally match to some set of tasks (see Figure 2). For us, the
resources are the groups of units (and the individual units) and the tasks are
the strategic goals, such as "attack/counter an enemy group" or "take over a
mineral mine". In the simplest version of the bipartite matching problem, you
construct a graph (a collection of nodes and direction connections between
them) in which each resource that is capable of addressing a task is
connected to that task, as in Figure 2.5.

Figure 2.5. Resource Allocation: Here we have four units (resources) and
five targets (goals).

Matching problem
Matching problem is a bit more involved for a few main reasons. First,
each task (strategic goal) can requirea different amount of resources (troop
strength composition) to accomplish. Second, each resource (group of our
units) can have a different ability to satisfy each goal (troop strength and
composition). Third, some tasks are more important than others. Also, if we
cannot commit enough resources to a task to fully achieve the goal of that
task, we do not want to commit any units (an insufficient force) to that task.
Thus, we are not concerned with just committing the maximum amount of
resources to tasks, but we also prefer to achieve the most important strategic
goals before the less important ones. Two more wrinkles are that a single
groups forces could be split between two tasks, and/or a single task could be
tackled by forces from a number of different groups.
There are many possible solutions to this problem. Our solutions tend to
be variants on network flow solutions [see Sedgewick92]. Not only are all of
the goals ranked in priority, but all of the matches between troops and goals
are ranked, too. The largest term in the matchs scores is the raw goal priority,
but the appropriateness of a particular unit type and the distance from each
59

unit to the goal (for example) also contribute. This helps a unit right next to
medium priority target choose that goal instead of a high priority one very far
away. We run a greedy network flow bipartite matching algorithm to
commitroops to particular goals. In an RTS such as Dark Reign, after the full
graph is matched as best as we can, we send all the troops on their way. For a
Turn-Based game, in which one decision can immediately affect our next
decision, as soon as one goal is satisfied (has enough troops) we send them
immediately, and then update some of the other matches based on the
results of our action.

60

UNIT III
3.1 GAME AI
At its most basic level, artificial intelligence consists of emulating the
behavior of other players or the entities (that is, all the elements of the game
that can act or be acted upon-from players to missiles to health pickups) they
represent. The key concept is that the behavior is simulated. In other words,
AI for games is more artificial and less intelligence. The system can be as
simple as a rules-based system or as complex as a system designed to
challenge a player as the commander of an opposing army.

3.2 THE IMPORTANCE OF AI DEVELOPMENT IN GAMES

3.2.1 The Purpose of AI in Games
AI can play multiple roles in gaming. It can be a general set of rules used to govern
the behavior of entities in the game world. You could also consider the pre-scripted
events that entities follow a type of AI. For example, in the game F.E.A.R*, the creepy
little girl who appears to frighten players and foreshadow future events is a prescripted event. What comes to mind for most people when they think of AI and games
is the computer-controlled players in multiplayer games. However, all of these are
different roles that AI can fulfill.

F.E.A.R.'s (Vivendi Universal*) use of scripted events is a type of AI

3.2.2 Basic Necessities for AI in Games

Depending on the nature of the role that the AI is supposed to fill, there can be very
little in the way of system needs. The more complex the system, the more
requirements an AI will have. Basic needs are nothing more than the processing time
needed to run the AI. More complex systems require some means of perceiving the
AIs environment, a record of player actions, and some means of evaluating the
success of previous decisions.
61

3.2.3 Decision-making
The core concept behind AI is decision making. To execute these choices, the
intelligent system needs to be able to affect the entities using the AI system. You can
organize this execution in either an AI push or an entity pull strategy.
AI push systems tend to isolate the AI system as a separate element of the game
architecture. Such a strategy often takes on the form a separate thread or threads in
which the AI spends its time calculating the best choices given the game options.
When the AI makes a decision, that decision is then broadcast to the entities involved.
This approach works best in real-time strategy games, where the AI is concerned with
the big picture.
Entity pull systems work best for games with simple entities. In these games, the
entities call on the AI system when the entity thinks, or updates itself. This
approach works very well in systems with large numbers of entities that do not need
to think very often, such as shooters. This system can also benefit from multithreading techniques, but it requires some extra planning
3.2.4 Basic Perceptions
For the AI to make meaningful decisions, it needs some way of perceiving its
environment. In simpler systems, this perception can be a simple check on the
position of the player entity. As systems become more demanding, entities need to
identify key features of the game world, such as viable paths to walk through, coverproviding terrain, and areas of conflict.
The challenge for designers and developers is to come up with a way to identify key
features important to the intelligence system. For example, cover can be
predetermined by the level designers or can be pre-computed when a map is loaded or
compiled. Some elements must be evaluated on the fly, such as conflict maps and
imminent threats.
3.2.5 Rules-Based Systems
The most basic form an intelligent system can take is that of a rules-based system.
This system stretches the term artificial intelligence. A set of preset behaviors is
used to determine the behavior of game entities. With a variety of actions, the overall
result can be a behavior system that is not obvious although there is very little actual
intelligence involved.
A good example of a rules-based system is a Black Jack dealer (either video
Black Jack or real Black Jack). The dealer has a simple rule that it follows: Always hit
when the cards add up to 17 or less. To the average player, the perception is that the
dealer is playing competitively. The player will imagine a more competent adversary

62

than the one he or she faces (unless the house advertises the rule that the dealers play
by).
The classic application of this system is Pac-Man. Four ghosts plagued the player.
Each ghost followed a simple rule set. One ghost was always to turn left, another was
always to turn right, one turned in a random direction, and the last turned toward the
player. Individually, the ghosts would be easy to figure out, and the player would be
able to handily avoid them. As a group, the pattern of their movement appears to be a
complex, coordinated search party hunting the player. In reality, the only one that
even checks the player's position is the last one.

Visual representation of the rule set governing Pac-Man ghosts, where arrows
represent the decisions that will be made.

As this example suggests, rules do not need to be hard-coded: They can be based
on perceived states (as the last ghost was) or on editable parameters of the entity.
Variables such as aggression, courage, range of sight, and rate of thinking can all lead
to more diverse entity behavior, even within a rules-based system. Rules-based
systems are the simplest structure for an AI. More complex intelligent systems are
built upon and governed by a series of conditional rules. In tactical games, rules
govern which tactics to use. In strategy games, rules govern build orders and how to
react to conflicts. Rules-based systems are the foundation of AI.
3.2.6 Finite State Machines as AI
A finite state machine (FMS) is a way of conceptualizing and implementing an entity
that has distinct states throughout its life. A statecan represent physical conditions
that the entity is in, or it can represent emotional states that the entity can exhibit. In
this example, emotional states are nothing like a true AIs emotional states but
predetermined behavior models that fit into the context of the game.
Here are common examples of states for an AI system for a game with stealth
elements:
63

Layout of the states in a typical FSM, where arrows represent the possible changes in
state

Idle. In this state, the entity is passively standing around or walking along a
set path. Perceptions are low. Player sounds are not often checked for. Only if
this entity is attacked or sees a player directly in front of it will its state
change to a higher level of awareness.

Aware. This entity is actively searching for intruders. It checks often for the
sounds of the player and sees farther and wider than an idle entity. This entity
will move to the Intrigued state if it notices something out of place (something
to check for), such as open doors, unconscious bodies, or spent bullet casings.
Intrigued. This entity is aware that something is up. To demonstrate this
behavior, the entity will abandon its normal post or path and move to areas of
interest, such as the aforementioned open doors or bodies. If a player is seen,
the entity goes to the Alert state.
Alert. In this state, the entity has become aware of the player and will go
through the actions of hunting down the player: moving into range of attack,
alerting fellow guards, sounding alarms, and finding cover. When the entity is
within range of the enemy, it switches to the Aggressive state.
Aggressive. This is the state where the enemy has engaged in combat with
the player. The entity attacks the player when it can and seeks cover between
rounds of attack (based on attack cool-downs or reloading). The entity only
leaves this state if the enemy is killed (return to normal), if the enemy moves
out of firing range (go back to the Alert stage), or if the entity dies (go to the
Dead state). If the entity becomes low on health, it may switch to the Fleeing
state, depending on the courage of the specific entity.
Fleeing. In this state, the entity tries to run from combat. Depending on the
game, there may be a secondary goal of finding health or leaving the play

64

area. When the entity finds health, it may return to the Alert state and resume
combat. An entity that leaves is merely deleted.
Dead. In some games, the state of death may not be completely idle. Death
or dying can have the entity cry out, alerting nearby entities, or go into a
knocked-out state, where it can later be revived by a medic (and returned to a
state of Alert).

There are at least two simple ways to implement an FMS within the entity system.
One is to have each state be a variable that can be checked (often through a massive
switch statement). The other is to use function pointers (in the C language) or virtual
functions (in the C++ and other object-oriented languages).
3.2.7 Adaptive AI
The previous sections discussed methods for designing intelligence systems that fit
into the predefined events of a game. For most games, this is adequate as long as the
designs were thorough and there is a clear understanding of the goals of the intelligent
entities. When a game calls for more variability and a better, more dynamic adversary
for the player, the AI may need to be able to grow and adapt on its own.
Adaptive AI is used commonly in fighting games and strategy games, in which the
mechanics are deep and the options for gameplay are innumerable. To provide a
constant challenge for the player without the player eventually figuring out the
optimal strategy to defeat the computer, the AI needs to be able to learn and adapt.
3.2.8 Prediction
The ability to effectively anticipate an opponents next move is crucial in an adaptive
system. Different methods can be used, such as past-pattern recognition (covered in a
future article) or random guess, to determine the next action to take.
One basic method for adaptation is to keep track of past decisions and evaluate their
success. The AI system keeps a record of choices a player has made in the past. Past
decisions must be evaluated in some manner. (e.g. in fighting games, the advantage
gained or lost-health lost or time advantage-can be the measure for success.)
Additional information about the situation can be gathered to give the decisions some
context, such as relative health, previous actions, and position in the level (people
play differently when their backs are to the wall).
This history can be evaluated to determine the success of previous actions and
whether a change in tactics is required. Until the list of past actions is built, general
tactics or random actions can be used to guide the actions of the entity. This system
can tie into rules-based systems and different states.

65

In a tactical game, past history can decide the best tactics to use against a player team,
such as defensive, offensive, berserk, or some balanced means of play. In a strategy
game, the optimal composition of units in an army can be discovered on a per-player
basis. In games where the AI is controlling supportive characters for the player, the
adaptive AI can better complement the player's natural style by learning the way the
player acts.
The rampant progress of technology makes nearly every game a new beginning.
Even though some basics of the game engine will probably stay the same during a
games development, constant feature and schedule revisions will make creating a
subsystem such as AI something like shooting at a quickly moving target. AI is very
dependent on concrete details of the game environment, which is the main reason why
its often added as one of the last subsystems. In fact, early tech demos rarely feature
it.
There are other reasons why AI is usually shifted to the end of the development
process: Customers value great AI, and bad AI behavior can lead to negative reviews
in the media. A game has to generate money in the end, however, and AI simply does
not have the highest priority from a marketing point of view. Humans are very visual
animals, and a beautiful sunset is much easier to sell than any particularly clever
reasoning capabilities of an opponent.
In addition, early milestone demonstrations for the publisher, press presentations,
and other hype-generating events do not promote inclusion of a globally/consistently
good AI, but instead focus on one or two absolutely unexpected but genius outcomes
of revolutionary new and complex AI procedures (did you spot the ironic tone?) that
provide the necessary wows. Although long-term fun with the game certainly is
important as well, market mechanisms will make it very difficult for AI to receive
equal ranking with features such as graphics and physics. Things might get even more
difficult if the games market should finally turn into a real mass market. Markets such
as persistent online game worlds, on the other hand, may increasingly promote a focus
on consistently good AI because players continuously evaluate these games, gaining
much more insight on mechanics, and they can continuously decide to pay or quit.
Surveys indicate that the percentage of CPU (central processing unit) cycles that
developers are allowed to burn on AI computations is steadily growing. This might be
because the speed of graphics cards has been increasing much faster than that of the
CPU, which frees up lots of resources. Anyway, these additional resources are much
needed for AI computations and open up many possibilities for more sophisticated AI.

66

3.3 Difference between Game AI and Real or Academic AI

Traditional research in AI seeks to create a real intelligence-albeit through artificial
means. Projects such as the Massachusetts Institute of Technologys (MIT) Kismet*
are trying to create an AI that can learn and interact socially as well as exhibit
emotions. As of this writing, MIT is working on creating an AI that has the faculties
of a young child, with promising results.
For the purposes of todays games, true AI is above and beyond the requirements
of a piece of entertainment software. Game AI does not need to be sentient or selfaware (in fact, it is best if it isnt); it does not have to learn about anything beyond the
scope of gameplay. The real goal of AI in games is to simulate intelligent behavior,
providing the player with a believable challenge-a challenge that the player can then
overcome.
There has been a lot of talk about what AI can do for games. The
consensus is that 'not much.' Game developers are people that talk in
concrete terms, whereas AI researchers often talk haze and philosophy. If you
ask them to be concrete, they then spew mathematics and algorithms. You
can observe this disconnect comparing
AI Wisdom and AI textbooks. AI
Wisdom is hands-on and practical, AI textbooks are
hypothetical and
theoretical. I will now refer to the fields as EI (engineered entertainment
intelligence) and AI (academic analysis of intelligence).

If you ask an AI researcher what he thinks of EI, he will respond that it's in-a-rut,
plugging algorithms around without truly understanding what they mean, and so on. If
you ask a game development practician what he thinks of AI textbooks, she will
respond that it's impractical, incomprehensive, mathematical and hazy. EI is
development, engineering intelligence-like artifacts. AI is analysis, research into
intelligence-like artifacts. EI is sensing, AI is intuition. EI is results, AI is insights. EI
is sales, AI is grants. EI is standardization, AI is anarchy. Few EI people manage to
get published in AI journals (too boring), few AI people manage to get published in EI
publications (too impractical).
It's important to realize that research and development are two sides of the same
brain, just like feeling and thinking, perceiving and judging, introversion and
extraversion, disorder and order, plasticity and stability, fun and work, yin and yang.
These are dichotomies, and Chris Lofting's web site has much on this topic. The
crucial insight is in the fact that the optimum lies in the balance and collaboration
between the two, not in either extreme. Of course, researchers often think that there is
too much development, and developers often think that there is too much research.
Every salesman peddles his own goods.

67

3.4 Determiinstic an Non deterministic AI

Game AI techniques generally come in two flavors: deterministic and
nondeterministic.
Deterministic

Deterministic behavior or performance is specified and predictable. Theres no

uncertainty. An example of deterministic behavior is a simple chasing algorithm. You
can explicitly code a nonplayer character to move toward some target point by
advancing along the x and y coordinate axes until the characters x and y coordinates
coincide with the target location.
Nondeterministic

Nondeterministic behavior is the opposite of deterministic behavior. Behavior has a

degree of uncertainty and is somewhat unpredictable (the degree of uncertainty
depends on the AI method employed and how well that method is understood). An
example of nondeterministic behavior is a nonplayer character learning to adapt to the
fighting tactics of a player. Such learning could use a neural network, a Bayesian
technique, or a genetic algorithm.
Deterministic AI techniques are the bread and butter of game AI. These
techniques are predictable, fast, and easy to implement, understand, test, and debug.
Although they have a lot going for them, deterministic methods place the burden of
anticipating all scenarios and coding all behavior explicitly on the developers
shoulders. Further, deterministic methods do not facilitate learning or evolving. And
after a little gameplay, deterministic behaviors tend to become predictable. This limits
a games play-life, so to speak.
Nondeterministic methods facilitate learning and unpredictable gameplay.
Further, developers dont have to explicitly code all behaviors in anticipation of all
possible scenarios. Nondeterministic methods also can learn and extrapolate on their
own, and they can promote so-called emergent behavior, or behavior that emerges
without explicit instructions. The flocking and neural network algorithms well
consider in this book are good examples of emergent behavior.
Developers traditionally have been a bit wary of AI that is nondeterministic,
although this is changing. Unpredictability is difficult to test and debughow can
you test all possible variations of player action to make sure the game doesnt do
something silly in some cases? Game developers face an ever-shortening development
cycle that makes developing and testing new technology to production-ready
standards extremely difficult. Such short development periods make it difficult for
developers to understand cutting-edge AI technologies fully and to see their
implications in a mass-market commercial game.

68

At least until recently, another factor that has limited game AI development is the
fact that developers have been focusing most of their attention on graphics quality. As
it turns out, such focus on developing better and faster graphics techniques, including
hardware acceleration, might now afford more resources to be allocated toward
developing better, more sophisticated AI. This fact, along with the pressure to produce
the next hit game, is encouraging game developers to more thoroughly explore
nondeterministic techniques.

69

UNIT- IV
4.1 Pathfinding

Knowing how to get from point A to point B is something many games require.
Whether youre designing a deep turn-based strategy tactics RPG or a simple puzzle
game, navigating your game world often requires more intelligence from your game
characters than merely pointing towards a goal and moving in a straight line.
You wouldnt want the units in your game to walk through walls. Nor would
players permit units to bump into walls and get stuck when a way around an obstacle
is clearly visible. Theres nothing more frustrating than a real time strategy games
units being unable to make their way to a target location.
This is where pathfinding algorithms come in. Pathfinding is the basic building
block of most game A.I. (artificial intelligence), and with a little help from the proper
code your game will be that much smarter for it.
To get to their goal, your game entities may have to walk around obstacles, find
their way though a dungeon maze or drive around the city streets. It isnt enough
to eventually arrive at the destination via some frustratingly circuitous path - the route
they take should be the shortest possible route.
Solving this problem isnt a trivial task, but it is worth the effort. Enemies that
choose wisely can often appear nearly intelligent. Your players will appreciate that
things just work and will be able to focus on strategy and tactics instead of
worrying about having the babysit broken AI.
The standard method that a majority of games use for pathfinding is called A*,
which is pronounced a star.
4.1.1 The Origins of Pathfinding AI: Dijkstras Algorithm
In the fifties, a mathematician named Edsger Dijkstra worked out a method for
calculating efficient routes that could be applied to any hypethetical situation
requiring an automated choice between many possible steps.
It is often used in the example of trucks going from stop to stop over long distances,
when shipments had to be picked up from various locations along the way. Choosing
the most efficient route would save money.
4.1.2 The Problem With Dijkstras Algorithm
Dijkstras algorithm works great, but has the drawback of being naively implemented
- it doesnt use any special heuristics for choosing the most likely candidates first, and
so the solution is only known after each and every possible location is tested.
70

A-star pathfinding to the rescue!

A-star is a faster and more efficient version of Dijkstras algorithm. WikiPedia
defines A-star as a best-first, graph search algorithm that finds the least-cost path
from a given initial node to one goal node.
4.1.3 A-Star algorithm
The A* algorithm is often used in video games to enable characters to navigate the
world. A* is the most popular choice for pathfinding, because its fairly flexible and
can be used in a wide range of contexts. In the following diagram, the pink square is
the starting point, the blue square is the goal
A* is like Dijkstras algorithm in that it can be used to find a shortest path. A* is
like Greedy Best-First-Search in that it can use a heuristic to guide itself. In the simple
case, it is as fast as Greedy Best-First-Search:

A* algorithm

In the example with a concave obstacle, A* finds a path as good as what Dijkstras
algorithm found:
71

Dijkstras algorithm
The secret to its success is that it combines the pieces of information that
Dijkstras algorithm uses (favoring vertices that are close to the starting
point) and information that Greedy Best-First-Search uses (favoring vertices that are
close to the goal). In the standard terminology used when talking about
A*, g(n) represents the exact cost of the path from the starting point to any vertex n,
and h(n) represents the heuristic estimated cost from vertex n to the goal. In the above
diagrams, the yellow (h) represents vertices far from the goal and teal ( g) represents
vertices far from the starting point. A* balances the two as it moves from the starting
point to the goal. Each time through the main loop, it examines the vertex n that has
the lowest f(n) = g(n) + h(n).
4.1.4 A* heuristics
The heuristic can be used to control A*s behavior.

At one extreme, if h(n) is 0, then only g(n) plays a role, and A* turns into
Dijkstras algorithm, which is guaranteed to find a shortest path.
If h(n) is always lower than (or equal to) the cost of moving from n to the
goal, then A* is guaranteed to find a shortest path. The lower h(n) is, the
more node A* expands, making it slower.
If h(n) is exactly equal to the cost of moving from n to the goal, then A* will
only follow the best path and never expand anything else, making it very

72

fast. Although you cant make this happen in all cases, you can make it
exact in some special cases. Its nice to know that given perfect
information, A* will behave perfectly.
If h(n) is sometimes greater than the cost of moving from n to the goal,
then A* is not guaranteed to find a shortest path, but it can run faster.

At the other extreme, if h(n) is very high relative to g(n), then only h(n) plays a role,
and A* turns into Greedy Best-First-Search.

So we have an interesting situation in that we can decide what we want to get out
of A*. At exactly the right point, well get shortest paths really quickly. If were too
low, then well continue to get shortest paths, but itll slow down. If were too high,
then we give up shortest paths, but A* will run faster.
In a game, this property of A* can be very useful. For example, you may find that
in some situations, you would rather have a good path than a perfect path. To shift
the balance between g(n) and h(n), you can modify either one.
4.1.5 Speed or Accuracy
A*s ability to vary its behavior based on the heuristic and cost functions can be
very useful in a game. The tradeoff between speed and accuracy can be exploited to
make your game faster. For most games, you dont really need the best path between
two points. You just need something thats close. What you need may depend on
whats going on in the game, or how fast the computer is.
Suppose your game has two types of terrain, Flat and Mountain, and the
movement costs are 1 for flat land and 3 for mountains, A* is going to search three
times as far along flat land as it does along mountainous land. This is because
its possible that there is a path along flat terrain that goes around the mountains. You
can speed up A*s search by using 1.5 as the heuristic distance between two map
spaces. A* will then compare 3 to 1.5, and it wont look as bad as comparing 3 to 1. It
is not as dissatisfied with mountainous terrain, so it wont spend as much time trying
to find a way around it. Alternatively, you can speed up up A*s search by decreasing
the amount it searches for paths around mountainsjust tell A* that the movement
cost on mountains is 2 instead of 3. Now it will search only twice as far along the flat
terrain as along mountainous terrain. Either approach gives up ideal paths to get
something quicker.
The choice between speed and accuracy does not have to be static. You can
choose dynamically based on the CPU speed, the fraction of time going into
pathfinding, the number of units on the map, the importance of the unit, the size of the
group, the difficulty level, or any other factor. One way to make the tradeoff dynamic

73

is to build a heuristic function that assumes the minimum cost to travel one grid space
is 1 and then build a cost function that scales:

If alpha is 0, then the modified cost function will always be 1. At this setting,
terrain costs are completely ignored, and A* works at the level of simple
passable/unpassable grid spaces. If alpha is 1, then the original cost function will be
used, and you get the full benefit of A*. You can set alpha anywhere in between.
You should also consider switching from the heuristic returning
the absolute minimum cost to returning the expected minimum cost. For example, if
most of your map is grasslands with a movement cost of 2 but some spaces on the
map are roads with a movement cost of 1, then you might consider having the
heuristic assume no roads, and return 2 * distance.
The choice between speed and accuracy does not have to be global. You can
choose some things dynamically based on the importance of having accuracy in some
region of the map. For example, it may be more important to choose a good path near
the current location, on the assumption that we might end up recalculating the path or
changing direction at some point, so why bother being accurate about the faraway part
of the path? Or perhaps its not so important to have the shortest path in a safe area of
the map, but when sneaking past an enemy village, safety and quickness are essential.
4.1.6 Scale
A* computes f(n) = g(n) + h(n). To add two values, those two values need to be at
the same scale. If g(n) is measured in hours and h(n) is measured in meters, then A* is
going to consider g or h too much or too little, and you either wont get as good paths
or you A* will run slower than it could.
4.1.7 Exact Heuristics
If your heuristic is exactly equal to the distance along the optimal path, youll see A*
expand very few nodes, as in the diagram shown in next topics. Whats happening
inside A* is that it is computing f(n) = g(n) + h(n) at every node. When h(n) exactly
matches g(n), the value of f(n) doesnt change along the path. All nodes not on the right
path will have a higher value of f than nodes that are on the right path. Since A*
doesnt consider higher-valued f nodes until it has considered lower-valued f nodes, it
never strays off the shortest path.
4.1.8 Precomputed exact heuristic

74

One way to construct an exact heuristic is to precompute the length of the shortest
path between every pair of points. This is not feasible for most game maps. However,
there are ways to approximate this heuristic:

Fit a coarse grid on top of the fine grid. Precompute the shortest path
between any pair of coarse grid locations.
Precompute the shortest path between any pair of waypoints. This is a
generalization of the coarse grid approach.

Then add in a heuristic h' that estimates the cost of going from any location to
nearby waypoints. (The latter too can be precomputed if desired.) The final heuristic
will be:
h(n) = h'(n, w1) + distance(w1, w2) + h'(w2, goal)

or if you want a better but more expensive heuristic, evaluate the above with all
pairs w1,w2 that are close to the node and the goal, respectively.
4.1.9 Linear exact heuristic
In a special circumstance, you can make the heuristic exact without precomputing
anything. If you have a map with no obstacles and no slow terrain, then the shortest
path from the starting point to the goal should be a straight line.
If youre using a simple heuristic (one which does not know about the obstacles
on the map), it should match the exact heuristic. If it doesnt, then you may have a
problem with scale or the type of heuristic you chose.
On a grid, there are well-known heuristic functions to use.
Use the distance heuristic that matches the allowed movement:

On a square grid that allows 4 directions of movement, use Manhattan

distance (L1).
On a square grid that allows 8 directions of movement, use Diagonal
distance (L).
On a square grid that allows any direction of movement, you might or
might not want Euclidean distance (L2). If A* is finding paths on the grid
but you are allowing movement not on the grid, you may want to
consider other representations of the map.

On a hexagon grid that allows 6 directions of movement, use Manhattan distance

adapted to hexagonal grids.

75

Multiply the distance in steps by the minimum cost for a step. For example, if youre
measuring in meters, the distance is 3 squares, and each square is 15 meters, then the
heuristic would return 3 15 = 45 meters. If youre measuring in time, the distance is
3 squares, and each square takes at least 4 minutes to cross, then the heuristic would
return 3 4 = 12 minutes. The units (meters, minutes, etc.) returned by the heuristic
should match the units used by the cost function.
4.1.10 Manhattan distance
The standard heuristic for a square grid is the Manhattan distance. Look at your cost
function and find the minimum cost D for moving from one space to an adjacent
space. In the simple case, you can set D to be 1. The heuristic on a square grid where
you can move in 4 directions should be D times the Manhattan distance:
function heuristic(node) =
dx = abs(node.x - goal.x)
dy = abs(node.y - goal.y)
return D * (dx + dy)

How do you pick D? Use a scale that matches your cost function. For the best paths,
and an admissible heuristic, set D to the lowest cost between adjacent squares. In
the absence of obstacles, and on terrain that has the minimum movement cost D,
moving one step closer to the goal should increase g by D and decrease h by D. When
you add the two, f (which is set to g + h) will stay the same; thats a sign that the
heuristic and cost function scales match. You can also give up optimal paths to make
A* run faster by increasing D, or by decreasing the ratio between the lowest and
highest edge costs.

(Note: the above image has a tie-breaker added to the heuristic.}

76

4.1.11 Diagonal distance

If your map allows diagonal movement you need a different heuristic. The Manhattan
distance for (4 east, 4 north) will be 8D. However, you could simply move (4
northeast) instead, so the heuristic should be 4D2, where D2 is the cost of moving
diagonally.

function heuristic(node) =
dx = abs(node.x - goal.x)
dy = abs(node.y - goal.y)
return D * (dx + dy) + (D2 - 2 * D) * min(dx, dy)

Here we compute the number of steps you take if you cant take a diagonal, then
subtract the steps you save by using the diagonal. There are min(dx, dy) diagonal steps,
and each one costs D2 but saves you 2D non-diagonal steps.
When D = 1 and D2 = 1, this is called the Chebyshev distance. When D = 1 and D2 =
sqrt(2), this is called the octile distance.
Patrick Lester has a different way of writing this heuristic, using an explicit cases
for dx > dy vs dx < dy. The above code has the same test but its hidden inside the call
to min.
4.1.12 Euclidean distance
If your units can move at any angle (instead of grid directions), then you should
probably use a straight line distance:
function heuristic(node) =
dx = abs(node.x - goal.x)

77

dy = abs(node.y - goal.y)
return D * sqrt(dx * dx + dy * dy)

However, if this is the case, then you may have trouble with using A* directly because
the cost function g will not match the heuristic function h. Since Euclidean distance is
shorter than Manhattan or diagonal distance, you will still get shortest paths, but A*
will take longer to run:

Euclidean distance, squared

Ive seen several A* web pages recommend that you avoid the expensive square root
in the Euclidean distance by just using distance-squared:
function heuristic(node) =
dx = abs(node.x - goal.x)
dy = abs(node.y - goal.y)
return D * (dx * dx + dy * dy)

Do not do this! This definitely runs into the scale problem. The scale of g and h need
to match, because youre adding them together to form f. When A* computes f(n) =
g(n) + h(n), the square of distance will be much higher than the cost g and you will end
up with an overestimating heuristic. For longer distances, this will approach the
extreme of g(n) not contributing to f(n), and A* will degrade into Greedy Best-FirstSearch:

78

To attempt to fix this you can scale the heuristic down. However, then you run into the
opposite problem: for shorter distances, the heuristic will be too small compared
to g(n)and A* will degrade into Dijkstras algorithm.
If, after profiling, you find the cost of the square root is significant, either use a fast
square root approximation with Euclidean distance or use the diagonal distance as an
approximation to Euclidean.
4.1.13 Multiple goals
If you want to search for any of several goals, construct a heuristic h'(x) that is the
minimum of h1(x), h2(x), h3(x), ... where h1, h2, h3 are heuristics to each of the nearby
spots.
If you want to search for spot near a single goal, ask A* search to find a path to the
center of the goal area. While processing nodes from the OPEN set, exit when you
pull a node that is near enough.

79

4.1.14 Breaking ties

In some grid maps there are many paths with the same length. For example, in flat
areas without variation in terrain, using a grid will lead to many equal-length paths.
A* might explore all the paths with the same f value, instead of just one.

Tiles in f values

The quick hack to work around this problem is to either adjust the g or h values. The
tie breaker needs to be deterministic with respect to the vertex (i.e., it shouldnt just be
a random number), and it needs to make the f values differ. Since A* sorts by f value,
making them different means only one of the equivalent f values will be explored.
One way to break ties is to nudge the scale of h slightly. If we scale it downwards,
then f will increase as we move towards the goal. Unfortunately, this means that A*
will prefer to expand vertices close to the starting point instead of vertices close to the
goal. We can instead scale h upwards slightly (even by 0.1%). A* will prefer to
expand vertices close to the goal.
heuristic *= (1.0 + p)

The factor p should be chosen so that p <(minimum cost of taking one step)/(expected
maximum path length). Assuming that you dont expect the paths to be more than
1000 steps long, you can choose p = 1/1000. (Note that this slightly breaks

80

admissibility of the heuristic but in games it almost never matters.) The result of
this tie-breaking nudge is that A* explores far less of the map than previously:

Tie-breaking scaling added to heuristic.

When there are obstacles of course it still has to explore to find a way
around them, but note that after the obstacle is passed, A* explores very
little:

81

Tie-breaking scaling added to heuristic, works nicely with obstacles.

Steven van Dijk suggests that a more straightforward way to do this would to pass h to
the comparison function. When the f values are equal, the comparison function would
break the tie by looking at h.
Another way to break ties is to add a deterministic random number to the heuristic or
edge costs. (One way to choose a deterministic random number is to compute a hash
of the coordinates.) This breaks more ties than adjusting h as above. Thanks to Cris
Fuhrman for suggesting this.
A different way to break ties is to prefer paths that are along the straight line from the
starting point to the goal:
dx1 = current.x - goal.x
dy1 = current.y - goal.y
dx2 = start.x - goal.x
dy2 = start.y - goal.y
cross = abs(dx1*dy2 - dx2*dy1)
heuristic += cross*0.001

This code computes the vector cross-product between the start to goal vector and the
current point to goal vector. When these vectors dont line up, the cross product will
be larger. The result is that this code will give some slight preference to a path that

82

lies along the straight line path from the start to the goal. When there are no obstacles,
A* not only explores less of the map, the path looks very nice as well:

Tie-breaking cross-product added to heuristic, produces pretty paths.

However, because this tie-breaker prefers paths along the straight line from the
starting point to the goal, weird things happen when going around obstacles (note that
the path is still optimal; it just looks strange):

83

Tie-breaking cross-product added to heuristic, less pretty with obstacles.

To interactively explore the improvement from this tie breaker, see James Macgills
A* applet [or try this mirror or this mirror]. Use Clear to clear the map, and choose
two points on opposite corners of the map. When you use the Classic A* method,
you will see the effect of ties. When you use the Fudge method, you will see the
effect of the above cross product added to the heuristic.
Yet another way to break ties is to carefully construct your A* priority queue so
that new insertions with a specific f value are always ranked better (lower)
than old insertions with the same f value.
And yet another way to break ties on grids is to minimize turns. The change in x,y
from the parent to the current node tells you what direction you were moving in. For
all edges being considered from current to neighbor, if the change in x,y is different
than the one from parent to current, then add a small penalty to the movement cost.
The above modifications to the heuristic are a band aid fix to an underlying
inefficiency. Ties occur when there are lots of paths that are equally good, leading to a
large number of nodes to explore. Consider ways to work smarter, not harder:

Alternate map representations can solve the problem by reducing the number
of nodes in the graph. Collapsing multiple nodes into one, or by remove all but
the important nodes. Rectangular Symmetry Reduction is a way to do this on

84

square grids; also look at framed quad trees. Hierarchical pathfinding uses a
high level graph with few nodes to find most of the path, then a low level
graph with more nodes to refine the path.
Some approaches leave the number of nodes alone but reduce the number of
nodes visited. Jump Point Search skips over large areas of nodes that would
contain lots of ties; its designed for square grids. Skip links add shortcut
edges that skip over areas of the map. The AlphA* algorithm adds some
depth-first searching to the usual breadth-first behavior of A*, so that it can
explore a single path instead of processing all of them simultaneously.
Fringe Search (PDF) solves the problem instead by making node processing
fast. Instead of keeping the OPEN set sorted and visiting nodes one at a time,
it processes nodes in batches, expanding only the nodes that have low fvalues. This is related to the HOT queues approach.
4.2 Flocking and Stearing
Flocking move in cohesive groups rather than independently, grazing in a
flock rather than walking aimlessly.
classic flocking has elegantly 3 simple rules:
Cohesion: have each unit steer toward the average position of its
neighbors
Alignment: have each unit steer so as to align itself to the average
heading of its neighbors.
Separation: have each unit steer to avoid hitting its neighbors
Treating the units as rigid bodies enables you to take care of orientation
rather than use particles to represent the units which you don't have to worry
about rotation.
The visibility arc is defined by 2 parameters: the arc radius (r) and the
angle ()
large radius will allow the unit to see m ore of the group and results in a m
ore cohesive flock, a smaller radius tends to increase the likelihood of
temporarily lose sight of their neighbors and results in a larger flock with
possible detached units which need to be re-join, navigating around
obstacles also can cause a flock to break up.
and angle measures the field of view of each unit, a view of 360
degree is easier to implement but the resulting flocking behavior might be
somewhat unrealistic, a m ore com m on field of view is having a distinct
blind spot behind each unit, the very narrow field of view will degenerate
a flock into units that one follows another and form a curved line.
Steer model: each rule makes a small contribution to the total steering force
and the net result is applied to the unit, require some tuning to make sure no
single rule dominates, tuning is done by trial and error.
We want the avoidance rule steering force contribution to be small when
the units are far away from each other, but we want the avoidance rule
85

steering force contribution to be relatively large when the units are

dangerously close to each other, the avoidance steering force is inversely
proportional to the separation distance.
For alignment we will consider the angle between a given unit's current
heading relative to the average hading of its neighbors, if the angle is small,
we want to make only a small adjustment to its heading, whereas if the angle
is large, a large adjustment is required.
The arrangement of the units in a flock will change constantly, each unit
must update its view of the world each time through the game loop, we must
cycle through all the units in acquire each unit's unique perspective, this
neighbor search can become computational expensive as the number of units
grow large.
field-of-view check:
if ( wideView )
{
inView = ((w.y > 0) | | (w.y < 0) && (fabs(w.x) > fabs(w.y) *
backViewAngle));
}
if ( limitedView )
{
inView = (w.y > 0);
}
if ( narrowView )
{
inView = ((w.y > 0) && (fabs(w.x) < fabs(w.y) * backViewAngle));
}
cohesion rule:
if ( doFlock && (n > 0) )
{
pAve = pAve / n;
v = unit[i].vVelocity;
v.normalize( );
u = pAve - units[i].vPosition;
u.normalize( );
w = vRotate2D(-units[i].fOrientation, u);
if ( w.x < 0 ) m = -1;
if ( w.x > 0 ) m = 1;
if ( fabs(v * u) < 1 )
{
fs.x += m * steeringForce * acos(v * u) / pi;
}
}
86

alignment rule:
if ( doFlock && (n > 0) )
{
vAve = vAve / n;
u = vAve;
u.normalize( );
v = units[i].vVelocity;
v.normalize( );
w = vRotate2D(-units[i].fOrientation, u);
if ( w.x < 0 ) m = -1;
if ( w.x > 0 ) m = 1;
if ( fabs(v * u) < 1 )
{
fs.x += m * steeringForce * acos(v * u) / pi;
}
}
separation rule:
for ( j = 1; j < maxNumUnits; j++ )
{
if ( inView )
{
if ( d.magnitude( ) <= units[i].fLength * radiusFactor )
{
pAve += units[j].vPosition; vAve += units[j].vVelocity;
}
elsif ( d.magnitude() <= units[i].fLength*separationFactor )
{
if ( w.x < 0 ) m = -1; if ( w.x > 0 ) m = 1;
fs.x += m * steeringForce * (units[i].fLength *
separationFactor) / d.magnitude( );
}
}
}
Obstacle avoidance:
Simple idealization of an obstacle: cycle
Outfit our units with virtual feelers, they will stick out in front of the units,
and if they hit something, this will be an indication to the units to turn.
The vector v, represent the feeler, the vector a is the difference between
the unit's and the obstacle's positions, project a onto v by taking their do
product and produce vector p, subtract vector p from a yields vector b, to test
whether v intersects the circle somewhere we need to test 2 conditions: the
magnitude of p must be less than the magnitude of v, and the magnitude of b
87

must be less than the radius r of the obstacle.

The steering force is a function of the prescribed maxim um steering force
times the ratio of the magnitude of v to the magnitude of a, this will make the
steering correction greater the closer the unit is to the obstacle.
Follow the leader
We won't explicitly designate any particular unit as a leader, but instead
we'll let some simple rules sort out who should be or could be a leader; if no
other units are directly in front of a given unit, it becomes a leader; if at least
one unit is in front of and within view of the current unit, the current unit can't
become a leader and it must follow the flocking rule.
Example: Let us consider an interactive simulation project of the boids
flocking model.
Steering behaviors formulated as rules:
Concerned primarily with at least five rules
3 original flocking rules
Separation
Cohesion
Alignment
2 additional steering rules
Obstacle Avoidance
Goal-Seeking (seek)
Separation
Pushes boids apart to keep them from crashing into each other by
maintaining distance from nearby flock mates.
Each boid considers its distance to other flock mates in its neighborhood and
applies a repulsive force in the opposite direction, scaled by the inverse of the
distance.

88

Cohesion
Keeps boids together as a group.
Each boid moves in the direction of the average position of its neighbors.
Compute the direction to the average position of local flock mates and
steer in that direction.

Alignment
Drives boids to head in the same direction with similar velocities (velocity
matching).
Calculate average velocity of flock mates in neighborhood and steer towards
that velocity.

89

Obstacle Avoidance
Allows the flock to avoid obstacles by steering away from approaching
objects.
Reynolds uses the method shown below:
Assume a cylindrical line of sight
Compute cylinder-sphere intersection and veer away from any objects in
path.

Goal Seeking
Drives the flock in the direction of a target/goal.
Each boid determines the direction to the goal and then steers in that
direction
(Note: this is basically the same as cohesion).

90

Force Ordering Scheme

Behaviors can be assigned priorities (in order of increasing priority):
Alignment
Cohesion
Goal-seeking
Separation
Obstacle Avoidance
Forces can be given priority (higher priority forces can cancel out lower
priority ones).

4.3 Rule based system

4.3.1 Introduction
In computer science, rule-based systems are used as a way to store and
manipulate knowledge to interpret information in a useful way. They are often
used in artificial intelligence applications and research.

A typical rule-based system has four basic components:

A list of rules or rule base, which is a specific type of knowledge base.
An inference engine or semantic reasoner, which infers information or takes
action based on the interaction of input and rule base. The interpreter executes a
production system program by performing the following match-resolve-act cycle.

Match: In this first phase, the left-hand sides of all productions are
matched against the contents of working memory. As a result a conflict
set is obtained, which consists of instantiations of all satisfied productions.
An instantiation of a production is an ordered list of working memory
elements that satisfies the left-hand side of the production.

91

Conflict-Resolution: In this second phase, one of the production

instantiations in the conflict set is chosen for execution. If no productions
are satisfied, the interpreter halts.
Act: In this third phase, the actions of the production selected in the
conflict-resolution phase are executed. These actions may change the
contents of working memory. At the end of this phase, execution returns
to the first phase.

Temporary working memory.

A user interface or other connection to the outside world through
which input and output signals are received and sent.

Using a set of assertions, which collectively form the working memory, and a
set of rules that specify how to act on the assertion set, a rule-based system
can be created. Rule-based systems are fairly simplistic, consisting of little
more than a set of if-then statements, but provide the basis for so-called
expert systems which are widely used in many fields. The concept of an
expert system is this: the knowledge of an expert is encoded into the rule set.
When exposed to the same data, the expert system AI will perform in a similar
manner to the expert.
Rule-based systems are a relatively simple model that can be adapted to
any number of problems. As with any AI, a rule-based system has its strengths
as well as limitations that must be considered before deciding if its the right
technique to use for a given problem. Overall, rule-based systems are really
only feasible for problems for which any and all knowledge in the problem
area can be written in the form of if-then rules and for which this problem area
is not large. If there are too many rules, the system can become difficult to
maintain and can suffer a performance hit.
To create a rule-based system for a given problem, you must have (or
create) the following:
A set of facts to represent the initial working memory. This should be anything
relevant to the beginning state of the system.
A set of rules. This should encompass any and all actions that should be taken
within the scope of a problem, but nothing irrelevant. The number of rules in
the system can affect its performance, so you dont want any that arent
needed.
A condition that determines that a solution has been found or that none
exists. This is necessary to terminate some rule-based systems that find
themselves in infinite loops otherwise.

92

4.3.2 Theory of Rule-Based Systems

The rule-based system itself uses a simple technique: It starts with a rulebase, which contains all of the appropriate knowledge encoded into If-Then
rules, and a working memory, which may or may not initially contain any data,
assertions or initially known information. The system examines all the rule
conditions (IF) and determines a subset, the conflict set, of the rules whose
conditions are satisfied based on the working memory. Of this conflict set, one
of those rules is triggered (fired). Which one is chosen is based on a conflict
resolution strategy. When the rule is fired, any actions specified in its THEN
clause are carried out. These actions can modify the working memory, the
rule-base itself, or do just about anything else the system programmer
decides to include. This loop of firing rules and performing actions continues
until one of two conditions are met: there are no more rules whose conditions
are satisfied or a rule is fired whose action specifies the program should
terminate.
Which rule is chosen to fire is a function of the conflict resolution strategy.
Which strategy is chosen can be determined by the problem or it may be a
matter of preference. In any case, it is vital as it controls which of the
applicable rules are fired and thus how the entire system behaves. There are
several different strategies, but here are a few of the most common:
First Applicable: If the rules are in a specified order, firing the first
applicable one allows control over the order in which rules fire. This is the
simplest strategy and has a potential for a large problem: that of an infinite
loop on the same rule. If the working memory remains the same, as does the
rule-base, then the conditions of the first rule have not changed and it will fire
again and again. To solve this, it is a common practice to suspend a fired rule
and prevent it from re-firing until the data in working memory, that satisfied
the rules conditions, has changed.
Random: Though it doesnt provide the predictability or control of the firstapplicable strategy, it does have its advantages. For one thing, its
unpredictability is an advantage in some circumstances (such as games for
example). A random strategy simply chooses a single random rule to fire from
the conflict set. Another possibility for a random strategy is a fuzzy rule-based
system in which each of the rules has a probability such that some rules are
more likely to fire than others.
Most Specific: This strategy is based on the number of conditions of the
rules. From the conflict set, the rule with the most conditions is chosen. This is
based on the assumption that if it has the most conditions then it has the
most relevance to the existing data.

93

Least Recently Used: Each of the rules is accompanied by a time or step

stamp, which marks the last time it was used. This maximizes the number of
individual rules that are fired at least once. If all rules are needed for the
solution of a given problem, this is a perfect strategy.
"Best" rule: For this to work, each rule is given a weight, which specifies
how much it should be considered over the alternatives. The rule with the
most preferable outcomes is chosen based on this weight.

4.3.3 Methods of Rule-Based Systems

(I)Forward-Chaining
Rule-based systems, as defined above, are adaptable to a variety of
problems. In some problems, information is provided with the rules and the AI
follows them to see where they lead. An example of this is a medical diagnosis
in which the problem is to diagnose the underlying disease based on a set of
symptoms (the working memory). A problem of this nature is solved using a
forward-chaining, data-driven, system that compares data in the working
memory against the conditions (IF parts) of the rules and determines which
rules to fire.

(II) Backward-Chaining
In other problems, a goal is specified and the AI must find a way to
achieve that specified goal. For example, if there is an epidemic of a certain
disease, this AI could presume a given individual had the disease and attempt
94

to determine if its diagnosis is correct based on available information. A

backward-chaining, goal-driven, system accomplishes this. To do this, the
system looks for the action in the THEN clause of the rules that matches the
specified goal. In other words, it looks for the rules that can produce this goal.
If a rule is found and fired, it takes each of that rules conditions as goals and
continues until either the available data satisfies all of the goals or there are
no more rules that match.

Of the two methods available, forward- or backward-chaining, the one to use

is determined by the problem itself. A comparison of conditions to actions in
the rule base can help determine which chaining method is preferred. If the
average rule has more conditions than conclusions, that is the typical
hypothesis or goal (the conclusions) can lead to many more questions (the
conditions), forward-chaining is favored. If the opposite holds true and the
average rule has more conclusions than conditions such that each fact may
fan out into a large number of new facts or actions, backward-chaining is
95

ideal.
If neither is dominant, the number of facts in the working memory may
help the decision. If all (relevant) facts are already known, and the purpose of
the system is to find where that information leads, forward-chaining should be
selected. If, on the other hand, few or no facts are known and the goal is to
find if one of many possible conclusions is true, use backward-chaining.
(III) Improving Efficiency of Forward Chaining
Forward-chaining systems, as powerful as they can be if well designed,
can become cumbersome if the problem is too large. As the rule-base and
working memory grow, the brute-force method of checking every rule
condition against every assertion in the working memory can become quite
computationally expensive. Specifically, the computational complexity if the
order of RA^C, where R is the number of rules, C is the approximate number
of conditions per rule, and A is the number of assertions in working memory.
With this exponential complexity, for a rule-base with any real rules, the
system will perform quite slowly.
There are ways to reduce this complexity, thus making a system of this
nature far more feasible for use with real problems. The most effective such
solution to this is the Rete algorithm. The Rete algorithm reduces the
complexity by reducing the number of comparisons between rule conditions
and assertions in the working memory. To accomplish this, the algorithm
stores a list of rules matched or partially matched by the current working
memory. Thus, it avoids unnecessary computations in re-checking the already
matched rules (they are already activated) or un-matched rules (their
conditions cannot be satisfied under the existing assertions in the working
memory). Only when the working memory changes does it re-check the rules,
and then only against the assertions added or removed from working memory.
All told, this method drops the complexity to O(RAC), linear rather than
exponential.
The Rete algorithm, however, requires additional memory to store the state of
the system from cycle to cycle. The additional memory can be considerable,
but may be justified for the increased speed efficiency. For large problems in
which speed is a factor, the Rete method is justified. For small problems, or
those in which speed is not an issue but memory is, the Rete method may not
be the best option. Another unfortunate shortcoming of the Rete method is
that it only works with forward-chaining.
4.4 Finite State Machines
A finite-state machine is a model used to represent and control execution
flow. It is perfect for implementing AI in games, producing great results
96

without a complex code. This tutorial describes the theory, implementation

and use of simple and stack-based finite-state machines.

A finite-state machine, or FSM for short, is a model of computation based on a

hypothetical machine made of one or more states. Only a single state can be active at
the same time, so the machine must transition from one state to another in order to
perform different actions.
FSMs are commonly used to organize and represent an execution flow, which is
useful to implement AI in games. The "brain" of an enemy, for instance, can be
implemented using a FSM: every state represents an action, such as attack or evade:

FSM representing brain of the enemy

An FSM can be represented by a graph, where the nodes are the states and the
edges are the transitions. Each edge has a label informing when the transition should
happen, like the player is near label in the figure above, which indicates that the
machine will transition from wander to attack if the player is near.
4.4.1 Planning States and Their Transitions
The implementation of a FSM begins with the states and transitions it will have.
Imagine the following FSM, representing the brain of an ant carrying leaves home

97

FSM representing the brain of Ant

The starting point is the find leaf state, which will remain active until the ant finds
the leaf. When that happens, the current state is transitioned to go home remains
active until the ant gets home. When the ant finally arrives home, the active state
becomes find leaf again, so the ant repeats its journey.
If the active state is find leaf and the mouse cursor approaches the ant, there is a
transition to the run away state. While that state is active, the ant will run away from
the mouse cursor. When the cursor is not a threat anymore, there is a transition back to
the find leaf state.
Since there are transitions connecting find leaf and run away, the ant will always
run away from the mouse cursor when it approaches as long as the ant is finding the
leaf. That will not happen if the active state is go home (check out the figure below).
In that case the ant will walk home fearlessly, only transitioning to the find leaf state
when it arrives home.

98

FSM representing the brain of an Ant. Notice the lack of transition between run away
and go home
4.4.2 Implementing a FSM
In FSM can be implemented and encapsulated in a single class, named FSM for
instance. The idea is to implement every state as a function or method, using a
property called active state in the class to determine which state is active:
public class FSM {
private var activeState :Function; // points to the currently active state
function
public function FSM() {
}
public function setState(state :Function) :void {
activeState = state;
}
public function update() :void {
if (activeState != null) {
activeState();
}
}
}

99

Since every state is a function, while an specific state is active the function
representing that state will be invoked every game update. The active state property is
a pointer to a function, so it will point to the active state's function.
The update() method of the FSM class must be invoked every game frame, so that
it can call the function pointed by the active State property. That call will update the
actions of the currently active state.
The setState() method will transition the FSM to a new state by pointing the
active State property to a new state function. The state function doesn't have to be a
member of FSM; it can belong to another class, which makes the FSM class more
generic and reusable.
4.4.3 Using a FSM
Using the FSM class already described, it's time to implement the "brain" of a
character. The previously explained ant will be used and controlled by an FSM. The
following is a representation of states and transitions, focusing on the code:

FSM of ant brain with focus on the code.

The ant is represented by the Ant class, which has a property named brain and a
method for each state. The brain property is an instance of the FSM class:
public class Ant

100

{
public var position :Vector3D;
public var velocity :Vector3D;
public var brain
:FSM;
public function Ant(posX :Number, posY :Number) {
position = new Vector3D(posX, posY);
velocity = new Vector3D( -1, -1);
brain
= new FSM();
// Tell the brain to start looking for the leaf.
brain.setState(findLeaf);
}
/**
* The "findLeaf" state.
* It makes the ant move towards the leaf.
*/
public function findLeaf() :void {
}
/**
* The "goHome" state.
* It makes the ant move towards its home.
*/
public function goHome() :void {
}
/**
* The "runAway" state.
* It makes the ant run away from the mouse cursor.
*/
public function runAway() :void {
}
public function update():void {
// Update the FSM controlling the "brain". It will invoke the currently
// active state function: findLeaf(), goHome() or runAway().
brain.update();
// Apply the velocity vector to the position, making the ant move.
moveBasedOnVelocity();
}

101

(...)
}

The Ant class also has a velocity and a position property, both used to calculate
the movement using Euler integration. The update() method is called every game
frame, so it will update the FSM.
To keep things simple, the code used to move the ant, such as
moveBasedOnVelocity(), will be omitted. More info on that can be found in the
Understanding Steering Behaviors series.
Below is the implementation of each state, starting with findLeaf(), the state
responsible for guiding the ant to the leaf position:
public function findLeaf() :void {
// Move the ant towards the leaf.
velocity
=
new
Vector3D(Game.instance.leaf.x
Game.instance.leaf.y - position.y);

position.x,

if (distance(Game.instance.leaf, this) <= 10) {

// The ant is extremelly close to the leaf, it's time
// to go home.
brain.setState(goHome);
}
if (distance(Game.mouse, this) <= MOUSE_THREAT_RADIUS) {
// Mouse cursor is threatening us. Let's run away!
// It will make the brain start calling runAway() from
// now on.
brain.setState(runAway);
}
}
The goHome() state, used to guide the ant home:
public function goHome() :void {
// Move the ant towards home
velocity
=
new
Vector3D(Game.instance.home.x
Game.instance.home.y - position.y);
if (distance(Game.instance.home, this) <= 10) {

102

position.x,

// The ant is home, let's find the leaf again.

brain.setState(findLeaf);
}
}
Finally, the runaway() state, used to make the ant flee the mouse cursor:
public function runAway() :void {
// Move the ant away from the mouse cursor
velocity = new Vector3D(position.x - Game.mouse.x,
Game.mouse.y);

position.y

// Is the mouse cursor still close?

if (distance(Game.mouse, this) > MOUSE_THREAT_RADIUS) {
// No, the mouse cursor has gone away. Let's go back looking for the leaf.
brain.setState(findLeaf);
}
}

The result is an ant controlled by a FSM "brain":

Ant controlled by a FSM. Move the mouse cursor to threaten the ant.

4.4.4 Improving the Flow: Stack-Based FSM

Imagine that the ant also needs to run away from the mouse cursor when it is
going home. The FSM can be updated to the following:

Ant FSM updated with new transitions.

103

It seems a trivial modification, the addition of a new transition, but it creates a

problem: if the current state is run away and the mouse cursor is not near anymore,
what state should the ant transition to: go home or find leaf?
The solution for that problem is a stack-based FSM. Unlike our existing FSM, a
stack-based FSM uses a stack to control states. The top of the stack contains the active
state; transitions are handled by pushing or popping states from the stack:

Stack-based FSM

The currently active state can decide what to do during a transition:

104

Transitions in a stack-based FSM: pop itself + push new; pop itself; push new.

It can pop itself from the stack and push another state, which means a full
transition (just like the simple FSM was doing). It can pop itself from the stack, which
means the current state is complete and the next state in the stack should become
active. Finally, it can just push a new state, which means the currently active state will
change for a while, but when it pops itself from the stack, the previously active state
will take over again.
4.4.5 Implementing a Stack-Based FSM
A stack-based FSM can be implemented using the same approach as before,
but this time using an array of function pointers to control the stack.
The active State property is no longer needed, since the top of the stack
already points to the currently active state:
public class StackFSM {
private var stack :Array;
public function StackFSM() {
this.stack = new Array();
}
public function update() :void {
var currentStateFunction :Function = getCurrentState();
if (currentStateFunction != null) {
currentStateFunction();
}
}
public function popState() :Function {
return stack.pop();
}
public function pushState(state :Function) :void {
if (getCurrentState() != state) {
stack.push(state);
}
}
public function getCurrentState() :Function {

105

return stack.length > 0 ? stack[stack.length - 1] : null;

}
}

The setState() method was replaced with two new methods: pushState() and
popState(); pushState() adds a new state to the top of the stack, while popState()
moves the state at the top of the stack. Both methods automatically transition the
machine to a new state, since they change the top of the stack.
4.4.6 Using a Stack-Based FSM
When using a stack-based FSM, it's important to note that each state is
responsible for popping itself from the stack. Usually a state removes itself
from the stack when it is no longer needed, like if attack() is active but the
target just died. Using the ant example, just a few changes are required to
adapt the code to use a stack-based FSM. The problem of not knowing the
state to transition to is now seamlessly solved thanks to the very nature of
stack-based FSM:
public class Ant {
(...)
public var brain :StackFSM;
public function Ant(posX :Number, posY :Number) {
(...)
brain = new StackFSM();
// Tell the brain to start looking for the leaf.
brain.pushState(findLeaf);
(...)
}
/**
* The "findLeaf" state.
* It makes the ant move towards the leaf.
*/
public function findLeaf() :void {
// Move the ant towards the leaf.
velocity
=
new
Vector3D(Game.instance.leaf.x
Game.instance.leaf.y - position.y);
if (distance(Game.instance.leaf, this) <= 10) {
106

position.x,

// The ant is extremelly close to the leaf, it's time

// to go home.
brain.popState(); // removes "findLeaf" from the stack.
brain.pushState(goHome); // push "goHome" state, making it the active
state.
}
if (distance(Game.mouse, this) <= MOUSE_THREAT_RADIUS) {
// Mouse cursor is threatening us. Let's run away!
// The "runAway" state is pushed on top of "findLeaf", which means
// the "findLeaf" state will be active again when "runAway" ends.
brain.pushState(runAway);
}
}
/**
* The "goHome" state.
* It makes the ant move towards its home.
*/
public function goHome() :void {
// Move the ant towards home
velocity
=
new
Vector3D(Game.instance.home.x
Game.instance.home.y - position.y);

position.x,

if (distance(Game.instance.home, this) <= 10) {

// The ant is home, let's find the leaf again.
brain.popState(); // removes "goHome" from the stack.
brain.pushState(findLeaf); // push "findLeaf" state, making it the active
state
}
if (distance(Game.mouse, this) <= MOUSE_THREAT_RADIUS) {
// Mouse cursor is threatening us. Let's run away!
// The "runAway" state is pushed on top of "goHome", which means
// the "goHome" state will be active again when "runAway" ends.
brain.pushState(runAway);
}
}
/**
* The "runAway" state.
* It makes the ant run away from the mouse cursor.
*/

107

public function runAway() :void {

// Move the ant away from the mouse cursor
velocity = new Vector3D(position.x Game.mouse.y);

Game.mouse.x,

position.y

// Is the mouse cursor still close?

if (distance(Game.mouse, this) > MOUSE_THREAT_RADIUS) {
// No, the mouse cursor has gone away. Let's go back to the previously
// active state.
brain.popState();
}
}
(...)
}

The result is an ant able to run away from the mouse cursor, transitioning back to the
previously active state before the threat:
Ant controlled by a stack-based FSM. Move the mouse cursor to threaten the
ant.

4.4.7 Conclusion
Finite-state machines are useful to implement AI logic in games. They can be
easily represented using a graph, which allows a developer to see the big picture,
tweaking and optimizing the final result. The implementation of a FSM using
functions or methods to represent states is simple, but powerful. Even more complex
results can be achieved using a stack-based FSM, which ensures a manageable and
concise execution flow without negatively impacting the code.
4.5 Patterning and waypoints
Artificial Intelligence can be a significant addition to many different types of
video games. Techniques of artificial intelligence make a more compelling experience
for the player by simulating human-like behaviors in their computerized opponents.
Our goal is to create computer-controlled characters that will:
* act in a way that is neither completely predictable nor purely random
* be responsive/adaptive to the player and to the environment (such as an
items/obstacles in the layout)
* present a level of challenge that is neither too easy nor too hard
For example, Let us discuss about of enemy movement in an airplane shoot-emup game, with emphasis on the idea of "waypoints": reference points in space used for
navigation.
108

The main idea is to create a Sprite object (small and invisible) that serves as a
Waypoint; each will contain two instance variables: ID and NextID. Each Enemy
sprite contains an instance variable called TargetID, which stores the ID of a
Waypoint, and each Enemy's angle of motion is constantly updated so that it is
moving towards the corresponding Waypoint. Once an Enemy reaches the desired
Waypoint, the Enemy updates its TargetID to the value stored in the Waypoint's
NextID variable, and the Enemy's journey continues. In the sections that follow, we
discuss the implementation details, as well as introduce additional techniques (each of
which is optional) to make the enemy movement seem more "realistic".
4.5.1 Initial Setup
Movement
First, we create a new project with window size and layout size of 800 by 600
pixels. Then, create:
* a Sprite object named "Player" with 8-Direction and Bound to Layout behaviors
* a Sprite object named "Enemy" with Bullet and Timer behaviors, and an instance
variable called TargetID with initial value 1.
* a Sprite object named "EnemyMissile" with the Bullet and Destroy Outside Layout
behaviors.
* a Sprite object named "Waypoint" with two instance variables: ID and NextID
Then create multiple instances of the Waypoint sprite in your layout and arrange
them as desired. Change their instance variable ID values to the numbers 1, 2, 3,... in
the order they should be visited by the enemies. For each Waypoint, change the value
of NextID to one more than ID, except for the last Waypoint, whose NextID value
should be set to 1 (assuming they will be travelling in a loop).
There are two events to keep move the enemies along the waypoints.
First, for each Enemy you need to select the Waypoint whose ID is equal to the
Enemy's TargetID value, and set the angle of motion towards that Waypoint
(the angle can be calculated using the function angle(Enemy.X, Enemy.Y,
Waypoint.X, Waypoint.Y)). Also, make sure that the "Set Angle" property of the
Enemy's Bullet behavior is set to "No", and rotate the Enemy to face the
Player (this will be important when the enemies spawn objects to shoot at the
player). Second, if an enemy collides with a Waypoint and that Waypoint's ID
is equal to the Enemy's TargetID, then the Enemy determines its next target
by setting its TargetID value to equal the NextID value of the Waypoint.
109

Firing
We've set up the code so that enemies are facing the player; next, we would like
them to fire at regular intervals. There are a number of way to achieve this result:
* We could set up the event: Every X Seconds -- Enemy Spawns EnemyMissile. This
causes the unrealistic (and hence undesirable) result that every Enemy on the screen
will fire at the Player at the exact same time.
* When new enemies are spawned, we start a Timer action on each to activate
Regularly, say, 1.5 seconds, and use the On Timer condition for an Enemy to spawn
an EnemyMissile. This is slightly better, since Enemies could fire at different times,
but an attentive player will still spot a mechanical regularity in the firing patterns.
* A better option is to modify the previous approach, and set a Timer with a randomly
calculated duration (say, random(1.0, 2.0)) to activate Once, and then after the On
Timer condition activates, set another Timer in the same way (random duration, type
Once). There are no patterns in the fire timing here, and thus it may feel more
realistic.
In addition, it may not be the case that all enemies have perfect aim, so to
incorporate this feature you can calculate and set the missile's angle of motion to be
the angle towards the player plus a small random value between -X and X. (For this to
work correctly, EnemyMissile objects need to have the Bullet property "Set Angle" set
to "No".)

110

4.5.2 Route Variations

After a while, players might see that all enemies appear to be moving along the
same path (even though the path may be complicated). Surely, intelligentenemies
would not act in such a way!
As our next improvement, we add a second instance variable to the Waypoint
objects, called NextID2. The way we will use this variable is as follows: when an
Enemy reaches its target Waypoint, we'll add a subevent with the condition to
Compare Two Values: random(0.0 , 1.0) < 0.5 (a.k.a. "flip a coin"). If this is true, then
the Enemy's new TargetID will be set to NextID, otherwise (using an Else condition),
the new Target ID will be set to NextID2.
This can be accomplished using the events below; this event would then replace the
previous event containing the condition Enemy On collision with Waypoint.
4.5.3 Waypoint Movement
Again, after a while the player might notice that even though the enemies may
follow different paths, the points at which enemies change direction are predictable.
To remedy this situation, we will also have make every Waypoint move as the
Enemy sprites are moving toward them. This can be easily accomplished by adding
the Sine behavior to Waypoints. The Sine behavior can change the value of a sprite's
property (such as horizontal position, vertical position, size, opacity, etc.) in a periodic
way. The type of value that is changing is displayed next to the "Movement" property
in the properties panel; we'll leave the default setting of "Horizontal". The
"Magnitude" property of the Sine behavior controls the amount of change; we'll leave
this at the default value of 50.
To obtain more interesting Waypoint movement, we can add a second Sine
behavior to the Waypoint object (which will appear under the behaviors heading as
"Sine2". If we change the Movement to "Vertical" and change the "Period Offset"
111

value to 1 (or in general, 1/4 the value of the period), this will cause the Waypoints to
move in a circle.
((Technical mathy note: by changing the period offset of a Sine function to 1/4
the value of the period, we're actually creating a Cosine function, and in general, the
equation of a circle can be parameterized by using one of Sine or Cosine for the X
coordinate, and using the other for the Y coordinate.))
4.5.4 Enemy Speed
Another realistic touch we could add to Enemy movement is to vary the speed.
Always moving at the same speed falls into the category of unrealistic patterns that
we wish to avoid. Instead, we will set up our Enemy objects to alternate between slow
and fast movement.
To accomplish this, we'll add a Sine behavior to the Enemy objects, and set the
Movement property to "Value Only". Then, in the event sheet, we will have each
Enemy set its Bullet speed to 150 + Enemy.Sine.Value, and with the Sine Magnitude
set to 50, this means that the speed will fluctuate between 150 - 50 = 100 and 150 +
50 = 200.

4.6 Fuzzy Logic and Fuzzy State Machine

Fuzzy set theory is a means of specifying how well an object satisfies a
vague description. For example, consider the proposition "Nate is tall." Is this
true, given that Nate is 5' 10"? Most people would hesitate to answer "true" or
"false," preferring to say, "sort of." Note that this is not a question of
uncertainty about the external world we are sure of Nate's height. Rather it
is a case of vagueness or uncertainty about the meaning of the linguistic term
"tall." So most authors I ?- say that fuzzy set theory is not a method for
uncertain reasoning at all.
Another way to think of this is as similarity to a prototype how close is
Nate to our prototype for tall person? We could express this in logical notation
by making TallPerson a constant and having a function Similarity that
compares things to it: Similarity(Nate, TallPerson) = SortOf Fuzzy set theory
takes a slightly different approach: it treats TallPerson as a fuzzy predicate and
says that the truth value of TallPerson(Nate) is a number between 0 and 1 ,
rather than being just True or False. The name "fuzzy set" derives from the
112

interpretation of the predicate as implicitly defining a set of its members, a

set that does not have sharp boundaries. Consider a kennel with 60
rottweilers and 40 retrievers. If we pick a dog at random, the probability of it
being a rottweiler is 0.6; this is uncertainty. The uncertainty can be resolved
by looking at the dog. But if a rottweiler and a retriever were to mate, then
the puppies would be half rottweiler, half retriever.
There is no uncertainty here, no additional evidence we can gather to
determine the breed of the puppies. Rather, this is a case of fuzziness at the
boundary between two breeds.

Fuzzy Logic
Fuzzy logic takes a complex sentence such as TallPerson(Nate) V
Smart(Nate) and determines its truth value as a function of the truth values of
its components. The rules for evaluating the fuzzy truth, T, of a complex
sentence are
Fuzzy logic is therefore a truth-functional system, and is thus subject to all the
usual problems.
It has the additional
problem that it is inconsistent
with normal propositional or first-order logic. We would like any logic to ensure
that standard equivalences such as True hold, but in fuzzy logic,
Despite these serious semantic difficulties, fuzzy logic has been very
successful in commercial applications, particularly in control systems for
products such as automatic transmissions,trains, video cameras, and electric
shavers. Elkan (1993) argues that these applications are successful because
they have small rule bases, have no chaining of inferences, and have tunable
parameters that can be adjusted to improve the system's performance (often
by learning techniques). The fact that they are implemented with fuzzy
operators is incidental to their success.
Elkan predicts that when more complex applications are tackled with fuzzy
logic, they will run into the same kinds of problems that plagued knowledgebased systems using certainty factors and other approaches.
4.7 Genetic Algorithm
Nature has a robust way of evolving successful organisms. The organisms
that are ill-suited for an environment die off, whereas the ones that are fit live
to reproduce. Offspring are similar to their parents, so each new generation
has organisms that are similar to the fit members of the previous generation.
113

If the environment changes slowly, the species can gradually evolve along
with it,but a sudden change in the environment is likely to wipe out a species.
Occasionally, random mutations occur, and although most of these mean a
quick death for the mutated individual, some mutations lead to new
successful species. The publication of Darwin's The Origin of Species on the
Basis of Natural Selection was a major turning point in the history of science.
It turns out that what's good for nature is also good for artificial systems.
Figure given below shows the GENETIC-ALGORITHM, which starts with a set of
one or more individuals and applies selection and reproduction operators to
"evolve" an individual that is successful, as measured by a fitness function.

There are several choices for what the individuals are. They can be entire
agent functions, in which case the fitness function is a performance measure
or reward function, and the analogy to natural selection is greatest. They can
be component functions of an agent, in which case the fitness function is the
critic. Or they can be anything at all that can be framed as an optimization
problem.
Since the evolutionary process learns an agent function based on
occasional rewards (offspring) as supplied by the selection function, it can be
seen as a form of reinforcement learning.
Unlike the algorithms described in the previous sections, however, no
attempt is made to learn the relationship between the rewards and the
actions taken by the agent or the states of the environment. GENETICALGORITHM simply searches directly in the space of individuals, with the goal
of finding one that maximizes the fitness function. The search is parallel
because each individual in the population can be seen as a separate search. It
is hill climbing because we are making small genetic changes to the
individuals and using the best resulting offspring. The key question is how to
allocate the searching resources: clearly, we should spend most of our time on
the most promising individuals, but if we ignore the low-scoring ones, we risk
getting stuck on a local maximum. It can be shown that, under certain
114

assumptions, the genetic algorithm allocates resources in an optimal way .

Before we can apply GENETIC-ALGORITHM to a problem, we need to
answer the following four questions:

What is the fitness function?

How is an individual represented?
How are individuals selected?
How do individuals reproduce?

The fitness function depends on the problem, but in any case, it is a function
that takes an individual as input and returns a real number as output. In the
"classic" genetic algorithm approach, an individual is represented as a string
over a finite alphabet. Each element of the string is called a gene. In real DNA,
the alphabet is AGTC (adenine, guanine, thymine, cytosine), but in genetic
algorithms, we usually use the binary alphabet (0,1). Some authors reserve
the term "genetic algorithm" for cases where the representation is a bit string,
and use the term evolutionary programming when the representation is more
complicated. Other authors make no distinction, or make a slightly different
one.
The selection strategy is usually randomized, with the probability of
selection proportional to fitness. That is, if individual X scores twice as high as
Y on the fitness function, then X is twice as likely to be selected for
reproduction than is Y. Usually, selection is done with replacement, so that a
very fit individual will get to reproduce several times.
Reproduction is accomplished by cross-over and mutation. First, all the
individuals that have been selected for reproduction are randomly paired.
Then for each pair, a cross-over point is randomly chosen. Think of the genes
of each parent as being numbered from 1 to N. The cross-over point is a
number in that range; let us say it is 10. That means that one offspring will
get genes 1 through 10 from the first parent, and the rest from the second
parent. The second offspring will get genes 1 through 10 from the second
parent, and the rest from the first. However, each gene can be altered by
random mutation to a different value, with small independent probability. The
diagram given below shows the process.

115

In (a), we have an initial population of 4 individuals. They are scored by the

fitness function in (b); the top individual scores an 8 and the bottom scores a
5. It works out that the top individual has a 32% chance of being chosen on
each selection. In (c), selection has given us two pairs of mates, and the
cross-over points (dotted lines) have been chosen. Notice that one individual
mates twice; one not at all. In (d), we see the new offspring, generated by
cross-over of their parents' genes. Finally, in (e), mutation has changed the
two bits surrounded by boxes. This gives us the population for the next
generation.
For
example,
suppose
we
are
trying to learn a
decision
list
representation
for
the
restaurant
waiting problem .
The fitness function in this case is simply the number of
examples that an individual is consistent with. The representation is the tricky
part. There are ten attributes in the problem, but not all of them are binary. It
turns out that we need 5 bits to represent each distinct attribute/value pair:

We also need one bit for each test to say if the outcome is Yes or No. Thus, if
we want to represent a k-DL with a length of up to t tests, we need a
representation with t(5k +1) bits. We can use the standard selection and
reproduction approaches. Mutation can flip an outcome or change an
attribute. Cross-over combines the head of one decision list with the tail of
another.

116

4.8 Artificial Neural Network

An artificial neuron network (ANN) is a computational model based on the
structure and functions of biological neural networks. Information that flows
through the network affects the structure of the ANN because a neural
network changes - or learns, in a sense - based on that input and output.
ANNs are considered nonlinear statistical data modeling tools where the
complex relationships between inputs and outputs are modeled or patterns
are found. ANN is also known as a neural network.
These biologically inspired methods of computing are thought to be the
next major advancement in the computing industry. Even simple animal
brains are capable of functions that are currently impossible for computers.
Computers do rote things well, like keeping ledgers or performing complex
math. But computers have trouble recognizing even simple patterns much
less generalizing those patterns of the past into actions of the future
Now, advances in biological research promise an initial understanding of
the natural thinking mechanism. This research shows that brains store
information as patterns. Some of these patterns are very complicated and
allow us the ability to recognize individual faces from many different angles.
This process of storing information as patterns, utilizing those patterns, and
then solving problems encompasses a new field in computing. This field, as
mentioned before, does not utilize traditional programming but involves the
creation of massively parallel networks and the training of those networks to
solve specific problems. This field also utilizes words very different from
traditional computing, words like behave, react, self-organize, learn,
generalize, and forget.
4.8.1 Analogy to the Brain
The exact workings of the human brain are still a mystery. Yet, some
aspects of this amazing processor are known. In particular, the most basic
element of the human brain is a specific type of cell which, unlike the rest of
the body, doesn't appear to regenerate. Because this type of cell is the only
part of the body that isn't slowly replaced, it is assumed that these cells are
what provides us with our abilities to remember, think, and apply previous
experiences to our every action. These cells, all 100 billion of them, are known
as neurons. Each of these neurons can connect with up to 200,000 other
neurons, although 1,000 to 10,000 is typical.The power of the human mind
comes from the sheer numbers of these basic components and the multiple
connections between them. It also comes from genetic programming and
learning.

117

The individual neurons are complicated. They have a myriad of parts, subsystems, and control mechanisms. They convey information via a host of
electrochemical pathways. There are over one hundred different classes of
neurons, depending on the classification method used. Together these
neurons and their connections form a process which is not binary, not stable,
and not synchronous. In short, it is nothing like the currently available
electronic computers, or even artificial neural networks.
These artificial neural networks try to replicate only the most basic
elements of this complicated, versatile, and powerful organism. They do it in a
primitive way. But for the software engineer who is trying to solve problems,
neural computing was never about replicating human brains. It is about
machines and a new way to solve problems.
4.8.2 Artificial Neurons and How They Work
The fundamental processing element of a neural network is a neuron. Th is

building block of human awareness encompasses a few general capabilities. Basically,

a biological neuron receives inputs from other sources, combines them in some way,
performs a generally nonlinear operation on the result, and then outputs the final
result.

A Simple Neuron

Within humans there are many variations on this basic type of neuron, further
complicating man's attempts at electrically replicating the process of thinking. Yet, all
natural neurons have the same four basic components. These components are known
by their biological names - dendrites, soma, axon, and synapses. Dendrites are hairlike extensions of the soma which act like input channels. These input channels
receive their input through the synapses of other neurons. The soma then processes

118

these incoming signals over time. The soma then turns that processed value into an
output which is sent out to other neurons through the axon and the synapses.
Recent experimental data has provided further evidence that biological neurons
are structurally more complex than the simplistic explanation above. They are
significantly more complex than the existing artificial neurons that are built into
today's artificial neural networks. As biology provides a better understanding of
neurons, and as technology advances, network designers can continue to improve
their systems by building upon man's understanding of the biological brain.
But currently, the goal of artificial neural networks is not the grandiose recreation
of the brain. On the contrary, neural network researchers are seeking an understanding
of nature's capabilities for which people can engineer solutions to problems that have
not been solved by traditional computing.
To do this, the basic unit of neural networks, the artificial neurons, simulate the
four basic functions of natural neurons. Figure given below shows a fundamental
representation of an artificial neuron.

Basic Artificial Neuron

In Basic Artificial Neuron, various inputs to the network are represented by the
mathematical symbol, x(n). Each of these inputs are multiplied by a connection
weight. These weights are represented by w(n). In the simplest case, these products
are simply summed, fed through a transfer function to generate a result, and then
output. This process lends itself to physical implementation on a large scale in a small
package. This electronic implementation is still possible with other network structures
which utilize different summing functions as well as different transfer functions.
119

UNIT V
The Future for AI in games
Artificial intelligence in games has matured significantly in the past decade.
Creating effective AI systems has now become as important for game developers as
creating solid gameplay and striking visuals. Studios have begun to assign dedicated
programming teams to AI development from the onset of a game's design cycle,
spending more time and resources on trying to build varied, capable, and consistent
non-player characters (NPCs). More developers are also using advances in AI to help
their games stand out in what has already become a very crowded marketplace,
spawning a slowly growing discussion in the industry about redefining game genres.
Think tanks and roundtables on advances in game AI have become prominent at the
annual Game Developers Conference (GDC), while smaller AI-dedicated conferences
such as the annual Paris Game AI Conference and developer-run online hubs such as
AiGameDev.com are garnering a big industry and community following. While
industry awareness about the significance of AI in games continues to grow,
GameSpot prompted Matthew Titelbaum from Monolith Games, Remco Straatman
from Guerrilla Games, and AiGameDev.com founder Alex J. Champandard to share
their thoughts on the future and growth of game AI.

The Halo franchise is recognised as a leader in the field of game AI.

Unlocking new possibilities
While faulty AI is easily recognised, an AI system that is doing its job often
goes unnoticed. No one stops halfway through a level to admire the
idiosyncrasies displayed by NPCs unless they are doing something completely
out of character--the more unremarkable, the better the AI system. While
achieving this result is still a priority for game developers, making games with
an AI system that stands out for being good is a relatively new concept: few
studios want to dedicate costly man-hours to chasing innovation in a highly
technical field that, for the most part, is likely to go unnoticed. However, there
are some exceptions. In 2007, AiGameDev.com launched its annual game AI
awards, nominated and voted by the site's community. The purpose of the
120

awards was to spotlight the games that showed promise in the field of AI,
either by trying something different or exhibiting technical proficiency. In
2009, the Best Combat AI and the overall Best Game AI awards were won by
the same studio--Guerrilla Games for Killzone 2. Remco Straatman, lead AI
programmer at Guerrilla, says a lot has changed in game AI in the last five to
10 years, with more developers trading low-level scripting for more advanced
NPC decision systems.
"In general, I think game AI has gone from the stage where it was an
achievement if it did not stand out negatively to the point where AI in most
big games is solid, and some titles are using innovative new ideas,"
Straatman says. "More development teams have also moved from simple
state machines to behaviour trees and using planners in NPC AI systems
describing knowledge of the world around the NPCs have improved with
better knowledge for navigation over changing terrain, and more knowledge
about strategic properties of the world such as cover. I also think advances in
animation systems with better ways to combine various animations and
physics have become available, which now allows for more realistic
movement and responses to being hit [in combat AI]. Most of these systems
were not around 10 years ago or simply could not run on the hardware
available."
Creating a solid game AI system involves successfully networking smaller
systems together. For example, a system that deals with the problem-solving
capabilities of individual NPCs goes hand in hand with a system that makes
sense of the gameworld and its parameters and helps NPCs make relevant
decisions. Thankfully, developers don't have to build these systems from
scratch: they use specific planners that generate increasingly complex
networks.
"At the moment [Guerrilla Games] is using a specific type of planner for
our NPCs called Hierarchical Task Network (HTN)," Straatman says. "This is
capable of generating more complex plans than what we had before Killzone
2. We also keep on improving things like the CPU performance, which means
we can support more NPCs in Killzone 3 than we could in Killzone 2. The
terrain-reasoning systems we generate have also evolved over our various
titles. We are now able to deal with much more dynamic terrain (like obstacles
moving around or changing shape) than ever before. Our data on where there
is cover has also become more detailed, something that allows NPCs to deal
with more complex environments such as multistory buildings, etc."

121

Killzone 2's lead AI programmer Remco Straatman believes the industry is still
struggling to make NPCs as human as possible.
Back when Straatman and Guerrilla began work on Killzone and
Shellshock, the teams goal was to make the AI system as capable of making
its own decisions as possible, realising this would make things all the more fun
for players. However, doing this in a consistent way proved to be a lot more
work than the team anticipated, particularly when dealing with combat AI.
While the goal of normal AI is to emulate the real-life behaviour of a particular
nature (for example, doctor, civilian, or shopkeeper), combat AI works very
differently. Firstly, its main objective is to be as entertaining as possible. In
some cases this means being efficient at killing players; in other cases, it's
more about making intentional mistakes and "overacting" by way of signalling
to players what is about to happen.
"Where normal AI tries to emulate an expert medical specialist or world
champion chess player, game combat AI is more like emulating an actor,"
Straatman says. "At the end of Killzone 2 we found ourselves looking at the
NPCs doing things that we did not expect, and this surprised us positively.
Reviews and forum feedback confirmed we had at least partly achieved the
vision we had so many years back, and people playing the game recognised
and appreciated it."
One of Killzone 2's most commended features in the field of AI was the
game's skirmish mode. Because this mode is more team-based and tactical
than the single-player campaign, the AI bots in this part of the game need to
do more than simply run around and kill one another. Guerrilla based the
skirmish AI in Killzone 2 on the real-time strategy model, building two levels of
AI on each individual bot. The first is a commander AI, which controls overall
strategic decisions; the second is a squad AI, which translates the commander
AI's orders into orders for the individual bots. The team then taught the bots
how to use the in-game badges as part of the order given to them by the
squad. For example, if an engineer bot is ordered to defend an area, he will
first build a turret at a tactical position before starting to patrol. While some
122

might argue that AI bots no longer play as important a role in multiplayer

games--given that most gamers now play online--Straatman says bots
improve gameplay and give players a chance to test out multiplayer
strategies before going up against other human players.
"They give people a testing ground for real multiplayer--getting to know
the maps and the game modes in a game against human players can be too
much to start with."
According to Straatman, the area that needs most improvement in the game
AI field is buddy AI. Because buddy AI systems often have contradictory
constraints, getting this system right is often a big challenge: the buddies
should be visible and close to the player but not get in his line of fire; they
should stay close and respond to the player movement but not move around
all the time; and so on. Buddy AI is also much closer in view to players than
enemy AI, making any errors easier to spot.
"Enemy NPCs know what other NPCs of the same faction are going to do
because they are all computer-controlled and can tell each other what they
will do next. However, players are much harder to predict--if you would look at
movement patterns of players, you will see they are quite strange at times.
This is made worse by the fact that player turn rates, movement speeds, and
acceleration are very high. The last point is the expectation of the player:
enemies are only supposed to shoot at you, whereas buddies are supposed to
fight and interact with you in a sensible way. We are working hard to make the
buddies work better, because we feel that they can add a lot to the player
experience when done right."

The AI director in the Left 4 Dead games is an example of how developers can
use AI to reach beyond traditional individual NPC behaviour.
Straatman believes the struggle to make NPCs as human as possible is
still very much at the top of the list for many AI programmers, with the future
set to change the way we think about in-game interaction.
"The ideal is always to immerse the player in the game: the NPCs should
123

feel like they are living and breathing creatures, and this illusion should not be
spoiled anywhere. Within the relatively limited interaction you have in a
game, it may be achievable to make the distinction very small. I think human
behaviour is so interesting, and yet subtle interactions such as conversations
are still out of reach of autonomous AI; games rely on clever scripting or
cutscenes to get that across. If we as a field will master these types of
interactions, more parts of the game can be interactive, and possibly whole
new game genres may become feasible."
"I think this will make games more approachable and immersive. If we are
able to maintain the immersion by having realistic behaviour in the interactive
parts of the game, you will get a seamless experience from cutscenes to
combat. I also think we are ready to use AI for more than just individual
NPCs--the director system in Left 4 Dead is one interesting first step in that
direction. We probably will see more combinations of AI systems that before
were limited to one type of game: RTS games will have unit AI that will come
closer to what you now see in first-person shooter games. MMOs could also
start using more elaborate AI, potentially even to command hordes of NPCs. I
hope we will see some brave studios try to create these new systems that are
now becoming possible."
Getting the most out of multiplayer
Alex J. Champandard, the brains behind AiGameDev.com, has his finger in
every game AI pie in town. Following his work as senior AI programmer for
Rockstar Games developing RAGE (Rockstar Advances Game Engine),
Champandard moved on to contract for Guerrilla Games Killzone 2, where he
developed the strategic AI for the multiplayer bots before starting up
AiGameDev.com, the online hub of the game AI community. In-between
running the site, Champandard continues his AI consulting work with studios
like 2K Czech and Crytek, as well as co-organising the aforementioned Paris
Game AI Conference.
Champandard says the last decade has brought a deeper understanding
on the part of game developers about how to best fix age-old AI problems.
"We've realised that just borrowing techniques from traditional artificial
intelligence doesn't work, and it requires a significant know-how to get
something fun and believable out of those algorithms," Champandard says. "A
large part of figuring this out has been to think about the impact on the player
and the in-game results. Thinking about creating NPCs as 'computational
behaviour' instead of 'game AI' sums it up perfectly: it's not about the
intelligence under the hood, it's about what the behaviour turns out like:
adaptive, credible, entertaining."
124

Champandard lists the Assassin's Creed franchise as exemplary candidates in

the game AI field.
This recent progress means game AI is no longer the weakest link in game
development, as was the case 10 years ago. While some studios see AI as
nothing more than a necessity, others are trying to innovate in the field.
Champandard says the best example of this is AI Directors in sandbox games.
"The entire concept of a sandbox game is impossible without AI. The idea
that you can do anything in the world and its inhabitants will react to you
would not be possible without AI to power those NPCs. I think the industry has
already discovered that you need AI Directors to make sandbox games really
fun. Otherwise you may end up with situations that emerge out of the
simulation that are just plain boring. Using AI programs that 'direct' the game
helps make sure you're seeing the best of what the game has to offer, as
planned by the designer. This kind of technology opens the doors to new
types of games, where the story is generated as you play. The progress is
slow, however, which means it may take a few years before this becomes
mainstream."
Champandard believes the future of game AI lies in more solid multiplayer
experiences, as AI systems slowly improve.
"There are more games providing bots as a way to improve the
multiplayer experience. Playing online can be very unpredictable if you're not
with your friends, and statistically players tend to prefer playing against bots
than random people. On the technology side, AI is now helping out in all the
other disciplines of game development, slowly revolutionising software
engineering, and improving techniques and algorithms that are applied. It's an
exciting time to be in the field."

125

Expanding the AI domain

Like Straatman, Monolith Games' Matthew Titelbaum believes future AI
systems will become more immersive and allow for a new kind of gaming
experience. However, quoting his experience working on the F.E.A.R.
franchise, Titelbaum does not believe that giving NPCs more human-like
behaviour is the way to achieve this. To him, it is not humanity that will
advance game AI, but rather, more unpredictable behaviour.
"Most games take the player on a journey from point A to point B. Along
that journey, the game presents the player with a series of puzzles to solve.
Some may be navigational (i.e. how do I get across that gap?), some may be
organisational (i.e. what order should I build things in?), but most of them rely
on some sort of other characters' intent on destroying or rescuing the player.
Without a series of interesting interactions with these characters, the journey
can become fairly tedious," Titelbaum says. "It used to be acceptable for AI to
have perfect knowledge of the environment.
Now we have stimulus and
sensor systems to more accurately model what an AI can conceivably know
about. Were also using planners, hierarchical state machines, and behaviour
trees to map this out. The bar keeps getting raised as new concepts from the
academic world steadily find their way into commercial releases."

F.E.A.R. 2: Project Origin made use of Monolith's goal-based action planner AI,
which helped produce context-sensitive behaviours in the game.
Like Guerrilla, Monolith employs several subsystems for both normal and
combat AI. The common link between the two is the goal-based action
planner, which shipped in F.E.A.R. and F.E.A.R. 2: Project Origin, which helped
produce the context-sensitive behaviours in the games. The first step came
from the teams level designers, who annotated each level with information
that the AI could understand. For example, marking a table as a good spot to
take cover and noting that the table must be flipped over before being used.
After this marking is completed, it is up to the AI to properly interpret the
126

annotations and integrate its own context: when the AI decides to go for cover
and it chooses that table, the annotation tells it to flip it over. These systems
work together to give the AI more knowledge about the world, and give
players the impression that NPCs see their movements and correctly judge
their intentions.
So if the goal of NPCs is to fool players into believing they think and act
just like other players, why does Titelbaum think that striving to make NPCs
correctly copy human behaviour in future AI systems is not a good idea?
Titelbaums answer is to expand the domain to which AI is applied in
games. For example, if AI can stand in for humans playing the game, this
means developers should be able to find ways to allow AI to stand in for
humans making the game.
"A good portion of level designers' time on F.E.A.R. was spent annotating
all the places AI could hide; it's possible, using terrain analysis techniques,
that much of that nitty-gritty work can be automated. Using automation
removes the need to redo hand-done work when there are major changes to
the level layout. The more parts of level creation we can automate, the more
time level designers have to create and polish rich experiences."

Games like Far Cry 2 push the boundaries of game AI.

Titelbaum believes that as games mature and players' objectives begin to
push past the 'kill everything in sight' trajectory, developers will create AI that
is able to engage in richer, more immersive behaviour that will allow NPCs to
express motivation and emotion not just through dialogue, but also through
interactions with the player, other NPCs, and the environment.

127