ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 93

◆
5 ◆

COGNITIVE ARCHITECTURE

Michael D. Byrne
Rice University

What Are Cognitive Architectures? . . . . . . . . . . . . . . . . 94 Contemporary Architectures . . . . . . . . . . . . . . . . . . . . 102

Relevance to Human-Computer Interaction . . . . . . . . 95
Brief Look at Past Systems in HCI . . . . . . . . . . . . . . . . 96
The Model Human Processor (MHP) and GOMS . . . . . . 97
Cognitive Complexity Theory (CCT) . . . . . . . . . . . . . . . 98
CAPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Soar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
LICAI/CoLiDeS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
EPIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
ACT-R 5.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
The Future of Cognitive Architectures in HCI . . . . . 110
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

93
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 94
94 • BYRNE
Designing interactive computer systems to be efficient and easy to use
is important so that people in our society may realize the potential ben-
efits of computer-based tools. . . . Although modern cognitive psychol-
ogy contains a wealth of knowledge of human behavior, it is not a sim-
ple matter to bring this knowledge to bear on the practical problems of
design—to build an applied psychology that includes theory, data, and
knowledge. (Card, Moran, & Newell, 1983, p. vii)
Integrating theory, data, and knowledge about cognitive psy-
chology and human performance in a way that is useful for guid-
ing design in HCI is still not a simple matter. However, there
have been significant advances since Card, Moran, and Newell
(1983) wrote the previous passage. One of the key advances is
the development of cognitive architectures, the subject of this
chapter. The chapter will first consider what it is to be a cogni-
tive architecture and why cognitive architectures are relevant for
HCI. In order to detail the present state of cognitive architec-
tures in HCI, it is important to consider some of the past uses of
cognitive architectures in HCI research. Then, three architec-
tures actively in use in the research community (LICAI/CoLiDeS,
EPIC, and ACT-R) and their application to HCI will be examined.
The chapter will conclude with a discussion of the future of cog-
nitive architectures in HCI.
WHAT ARE COGNITIVE ARCHITECTURES?
Most dictionaries will list several different definitions for the
word architecture. For example, dictionary.com lists among
them “a style and method of design and construction,” e.g.,
Byzantine architecture; “orderly arrangement of parts; struc-
ture,” e.g., the architecture of a novel; and one from computer
science: “the overall design or structure of a computer system.”
What, then, would something have to be to qualify as a “cogni-
tive architecture”? It is something much more in the latter
senses of the word architecture, an attempt to describe the
overall structure and arrangement of a very particular thing,
the human cognitive system. A cognitive architecture is a broad
theory of human cognition based on a wide selection of human
experimental data, and implemented as a running computer
simulation program. Young (Gray, Young, & Kirschenbaum, 1997;
Ritter & Young, 2001) defined a cognitive architecture as an em-
bodiment of “a scientific hypothesis about those aspects of hu-
man cognition that are relatively constant over time and rela-
tively independent of task.”
This idea has been a part of cognitive science since the early
days of cognitive psychology and artificial intelligence, as mani-
fested in the General Problem Solver or GPS (Newell & Simon,
1963), one of the first successful computational cognitive mod-
els. These theories have progressed a great deal since GPS and
are gradually becoming broader. One of the best descriptions of
the vision for this area is presented in Newell’s (1990) book Uni-
fied Theories of Cognition. In it, Newell argued that the time has
come for cognitive psychology to stop collecting disconnected
empirical phenomena and begin seriously considering theoreti-
cal unification in the form of computer simulation models. Cog-
nitive architectures are attempts to do just this.
Cognitive architectures are distinct from engineering ap-
proaches to artificial intelligence, which strive to construct in-
telligent computer systems by whatever technologies best serve
that purpose. Cognitive architectures are designed to simulate
human intelligence in a humanlike way (Newell, 1990). For ex-
ample, the chess program that defeated Kasparov, Deep Blue,
would not qualify as a cognitive architecture because it does not
solve the problem (chess) in a humanlike way. Deep Blue uses
massive search of the game space, while human experts gener-
ally look only a few moves ahead, but concentrate effectively
on quality moves.
Cognitive architectures differ from traditional research in
psychology in that work on cognitive architecture is integra-
tive—that is, architectures include mechanisms to support at-
tention, memory, problem solving, decision making, learning,
and so forth. Most theorizing in psychology follows a divide-
and-conquer strategy that tends to generate highly specific the-
ories of a very limited range of phenomena; this has changed lit-
tle since the 1970s (Newell, 1973). This limits the usefulness of
such theories for an applied domain such as HCI, where users
employ a wide range of cognitive capabilities in even simple
tasks. Instead of asking, “How can we describe this isolated phe-
nomenon?” people working with cognitive architectures can
ask, “How does this phenomenon fit in with what we already
know about other aspects of cognition?”
Another important feature of cognitive architectures is that
they specify only the human “virtual machine,” the fixed ar-
chitecture. A cognitive architecture alone cannot do anything.
Generally, the architecture has to be supplied with the knowl-
edge needed to perform a particular task. The combination
of an architecture and a particular set of knowledge is gener-
ally referred to as a “computational cognitive model,” or just
a “model.” In general, it is possible to construct more than one
model for any particular task. The specific knowledge incor-
porated into a particular model is determined by the modeler.
Because the relevant knowledge must be supplied to the ar-
chitecture, the knowledge engineering task facing modelers
attempting to model performance on complex tasks can be
formidable.
A third centrally important feature of cognitive architectures
is that they are software artifacts constructed by human pro-
grammers. This has a number of relevant ramifications. First, a
model of a task constructed in a cognitive architecture is
runnable and produces a sequence of behaviors. These behav-
ior sequences can be compared with the sequences produced
by human users to help assess the quality of a particular model.
They may also provide insight into alternate ways to perform a
task—that is, they may show possible strategies that are not ac-
tually utilized by the people performing the task. This can be
useful in guiding interface design as well. Another feature of
many architectures is that they enable the creation of quantita-
tive models. For instance, the model may say more than just
“click on button A and then menu B,” but may include the time
between the two clicks as well. Models based on cognitive ar-
chitectures can produce execution times, error rates, and even
learning curves. This is a major strength of cognitive architec-
tures as an approach to certain kinds of HCI problems and will
be discussed in more detail in the next section.
On the other hand, cognitive architectures are large software
systems, which are often considered difficult to construct and
maintain. Individual models are also essentially programs, writ-
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 95
ten in the “language” of the cognitive architecture. Thus, indi-
vidual modelers need to have solid programming skills.
Finally, cognitive architectures are not in wide use among
HCI practitioners. Right now, they exist primarily in academic re-
search laboratories. One of the barriers for practitioners is that
learning and using most cognitive architectures is itself gener-
ally a difficult task; however, this is gradually changing, and
some of the issues being addressed in this regard will be dis-
cussed in section 4. Furthermore, even if cognitive architectures
are not in wide use by practitioners, this does not mean that
they are irrelevant to practitioners. The next section highlights
why cognitive architectures are relevant to a wide HCI audience.
RELEVANCE TO HUMAN-COMPUTER
INTERACTION
For some readers, the relevance of models that produce quan-
titative predictions about human performance will be obvious.
For others, this may be less immediately clear. Cognitive archi-
tectures (a) are relevant to usability as an engineering discipline,
(b) have several HCI-relevant applications in computing systems,
and (c) serve an important role in HCI as a theoretical science.
At nearly all HCI-oriented conferences, and many online re-
sources, there are areas where corporations recruit HCI profes-
sionals. A common job title in these forums is “usability engi-
neer.” Implicit in this title is the view that usability is, at least in
part, an engineering enterprise. In addition, while people with
this job title are certainly involved in product design, there is a
sense in which most usability engineering would not be recog-
nized as engineering by people trained in more traditional engi-
neering disciplines, such as electrical or aerospace engineering.
In traditional engineering disciplines, design is generally guided
at least in part by quantitative theory. Engineers have at their dis-
posal hard theories of the domain in which they work, and these
theories allow them to derive quantitative predictions. Consider
an aerospace engineer designing a wing. Like a usability engi-
neer, the aerospace engineer will not start with nothing; a pre-
existing design often provides a starting point; however, when
the aerospace engineer decides to make a change in that design,
there is usually quantitative guidance about how the perfor-
mance of the wing will change because of the change in design.
This guidance, while quantitative, is not infallible, hence the
need for evaluation tools such as wind tunnels. This is similar to
the usability engineer’s usability test. Unlike usability testing,
however, the aerospace engineer has some quantitative idea
about what the outcome of the test will be, and this is not guided
simply by intuition and experience, but by a mathematical theory
of aerodynamics. In fact, this theory is now so advanced that few
wind tunnels are built anymore. Instead, they are being replaced
by computer simulations based on “computational fluid dynam-
ics,” an outcome of the application of computational techniques
to complex problems in aerodynamics. Simulations have not en-
tirely replaced wind tunnels, but the demand for wind tunnel
time has clearly been affected by this development.
For the most part, the usability engineer lacks the quantita-
tive tools available to the aerospace engineer. Every design must
5. Cognitive Architecture • 95
be subjected to its own wind tunnel (usability) test, and the en-
gineer has little guidance about what to expect other than from
intuition and experience with similar tests. While intuition and
experience can certainly be valuable guides, they often fall short
of more “hard” quantitative methods. Perhaps the engineer can
intuit that interface “X” will allow users to complete tasks faster
than with interface “Y,” but how much faster? Ten percent?
Twenty percent? Even small savings in execution times can add
up to large financial savings for organizations when one con-
siders the scale of the activity. The paradigm example is the tele-
phone operators studied by Gray, John, and Atwood (1993),
where even a second saved on an average call would save the
telephone company millions of dollars.
Computational models based on cognitive architectures
have the potential to provide detailed quantitative answers, and
for more than just execution times. Error rates, transfer of knowl-
edge, learning rates, and other kinds of performance measures
are all metrics than can often be provided by architecture-based
models. Even if such models are not always precisely accurate in
an absolute sense, they may still be useful in a comparative
sense. For example, if a usability engineer is comparing interface
A with interface B and the model at his or her disposal does not
accurately predict the absolute times to complete some set of
benchmark tasks, it may still accurately capture the ordinal dif-
ference between the two interfaces, which may be enough.
Additionally, there are certain circumstances when usability
tests are either impractical, prohibitively costly, or both. For
example, access to certain populations such as physicians or
astronauts may be difficult or expensive, so bringing them in for
repeated usability tests may not be feasible. While developing a
model of a pilot or an air traffic controller performing an expert
task with specialized systems may be difficult at first, rerunning
that model to assess a change made to the user interface should
be much more straightforward than performing a new usability
test for each iteration of the system. This is possible only with a
quantitatively realistic model of the human in the loop, one that
can produce things such as execution times and error rates.
Computational models can, in principle, act as surrogate users
in usability testing, even for special populations.
Of course, some of these measures can be obtained through
other methods such as GOMS (Goals, Operators, Methods, and
Selection rules) analysis (Card et al., 1983; John & Kieras, 1996)
or cognitive walkthrough (Polson, Lewis, Rieman, & Wharton,
1992); however, these techniques were originally grounded in
the same ideas as some prominent cognitive architectures and
are essentially abstractions of the relevant architectures for par-
ticular HCI purposes. In addition, architecture-based computa-
tional models provide things that GOMS models and cognitive
walkthroughs do not. First, models are executable and genera-
tive. A GOMS analysis, on the other hand, is a description of the
procedural knowledge the user must have and the sequence of
actions that must be performed to accomplish a specific task
instance, while the equivalent computational model actually
generates the behaviors, often in real time or faster. Equally im-
portantly, computational models have the capacity to be reac-
tive in real time. So while it may be possible to construct a
GOMS model that describes the knowledge necessary and the
time it will take an operator to classify a new object on an air
traffic controller’s screen, a paper-and-pencil GOMS model can-
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 96
96 • BYRNE
not actually execute the procedure in response to the appear-
ance of such an object. A running computational model, on the
other hand, can.
Because of this property, architecture-based computational
models have some other important uses beyond acting as virtual
users in usability tests. One such use is in intelligent tutoring
systems (ITSs). Consider the Lisp tutor (Anderson, Conrad, &
Corbett, 1989). This tutoring system contained an architecture-
based running computational model of the knowledge neces-
sary to implement the relevant Lisp functions, as well as a mod-
ule for assessing which pieces of this knowledge were mastered
by the student. Because the model was executable, it could pre-
dict what action the student would take if the student had cor-
rect knowledge of how to solve the problem. When the student
took a different action, this told the ITS that the student was
missing one or more relevant pieces of knowledge. The student
could then be given feedback about what knowledge was miss-
ing or incomplete, and problems that exercise this knowledge
could be selected by the ITS. It is possible to generate more ef-
fective educational experiences by identifying students’ knowl-
edge, as well as the gaps in that knowledge. Problems that con-
tain knowledge students have already mastered can be avoided,
so as not to bore students with things they already know. This
frees up students to concentrate on the material they have not
yet mastered, resulting in improved learning (Anderson et al.,
1989). While the Lisp tutor is an old research system, ITSs based
on the same underlying cognitive architecture with the same es-
sential methodology have been developed for more pressing
educational needs such as algebra and geometry and are now
sold commercially (see http://www.carnegielearning.com).
Another HCI-relevant application for high-fidelity cognitive
models is populating simulated worlds or situations. For exam-
ple, training an F-16 fighter pilot is expensive, even in a simula-
tor, because that trainee needs to face realistic opposition. Real-
istic opposition consists of other trained pilots, so training one
person requires taking several trained pilots away from their nor-
mal duties (e.g., flying airplanes on real missions). This is difficult
and expensive. If, however, the other pilots could be realistically
simulated, the trainee could face opposition that would have
useful training value without having to remove already trained
pilots from their duties. Many training situations such as this ex-
ist, where the only way to train someone is to involve multiple
human experts who must all be taken away from their regular
jobs. The need for expensive experts can potentially be elimi-
nated (or at least reduced), however, by using architecturally
based cognitive models in place of the human experts. The U.S.
military has already started to experiment with just such a sce-
nario ( Jones, Laird, Nielsen, Coulter, Kenny, & Koss, 1999). Having
realistic opponents is desirable in domains other than training as
well, such as video games. Besides things like texture-mapped
3-D graphics, one of the features often used to sell games is net-
work play. This enables players to engage opponents whose ca-
pabilities are more comparable to their own than typical com-
puter-generated opponents; however, even with network play,
it is not always possible for a game to find an appropriate oppo-
nent. If the computer-generated opponent were a more high-
fidelity simulation of a human in terms of cognitive and percep-
tual-motor capabilities, then video game players would have no
difficulty finding appropriate opponents without relying on net-
work play. While this might not be the most scientifically inter-
esting use of cognitive architectures, it seems inevitable that cog-
nitive architectures will be used in this way.
Cognitive architectures are also theoretically important to
HCI as an interdisciplinary field. Many people (including some
cognitive psychologists) find terms from cognitive psychology
such as “working memory” or “mental model” vague and ill de-
fined. A harsh evaluation of explanations relying on such terms
is found in Salthouse (1988, p. 3): “It is quite possible that in-
terpretations relying on such nebulous constructs are only mas-
querading ignorance in what is essentially vacuous terminol-
ogy.” Computational cognitive architectures, on the other hand,
require explicit specifications of the semantics of theoretical
terms. Even if the architectures are imperfect descriptions of
the human cognitive system, they are at a minimum well speci-
fied and therefore clearer in what they predict than strictly ver-
bal theories.
A second theoretical advantage of computational theories
such as cognitive architectures is that they provide a window
into how the theory actually works. As theories grow in size and
number of mechanisms, the interactions of those mechanisms
becomes increasingly difficult to predict analytically. Computer
simulations permit relatively rapid evaluations of complex
mechanisms and their interactions. (For an excellent discussion
of this topic, see Simon, 1996.) Another problem with theories
based solely on verbal descriptions is their internal coherence
can be very difficult to assess, while such assessment is much
more straightforward with computational models. Verbal theo-
ries can easily hide subtle (and not so subtle) inconsistencies
that make them poor scientific theories. Computational models,
on the other hand, force explanations to have a high level of in-
ternal coherence; theories that are not internally consistent are
typically impossible to implement on real machines.
Finally, HCI is an interdisciplinary field, and thus theories
that are fundamentally interdisciplinary in nature are appropri-
ate. Cognitive architectures are such theories, combining com-
putational methods and knowledge from the artificial intelli-
gence end of computer science with data and theories from
cognitive psychology. While cognitive psychology and computer
science are certainly not the only disciplines that participate in
HCI, they are two highly visible forces in the field. Psychologi-
cal theories that are manifested as executable programs should
be less alien to people with a computer science background
than more traditional psychological theories.
Thus, cognitive architectures are clearly relevant to HCI at a
number of levels. This fact has not gone unnoticed by the HCI
research community. In fact, cognitive architectures have a long
history in HCI, dating back to the original work of Card, Moran,
and Newell (1983).
BRIEF LOOK AT PAST SYSTEMS IN HCI
The total history of cognitive architectures and HCI would be far
too long to document in a single chapter; however, it is possible
to touch on some highlights. While not all of the systems de-
scribed in this section qualify as complete cognitive architec-
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 97
tures, they all share intellectual history with more current ar-
chitectures and influenced their development and use in HCI.
Finally, many of the concepts developed in these efforts are still
central parts of the ongoing research on cognitive architecture.
In addition, there is a natural starting point:
The Model Human Processor (MHP) and GOMS
The Psychology of Human-Computer Interaction (Card, Moran,
& Newell, 1983) is clearly a seminal work in HCI, one of the
defining academic works in the early days of the field. While that
work did not produce a running cognitive architecture, it was
clearly in the spirit of cognitive architectures and was quite in-
fluential in the development of current cognitive architectures.
Two particular pieces of that work are relevant here, the Model
Human Processor (MHP) and GOMS.
The MHP represents a synthesis of the literature on cognitive
psychology and human performance up to that time, and
sketches the framework around which a cognitive architecture
could be implemented. The MHP is a system with multiple
memories and multiple processors, and many of the properties
of those processors and memories are described in some detail.
(See Fig. 5.1.) Card, Moran, and Newell (1983) also specified the
interconnections of the processors and a number of general
operating principles. In this system, there are three processors:
FIGURE 5.1. Model Human Processor. Based on Card, Moran,
and Newell (1983).
5. Cognitive Architecture • 97
(a) one cognitive, (b) one perceptual, and (c) one motor. In
some cases, the system essentially behaves serially. For instance,
in order for the system to press a key in response to the ap-
pearance of a light, the perceptual processor must detect the
appearance of the light and transmit this information to the cog-
nitive processor. The cognitive processor’s job is to decide what
the appropriate response should be, and then transmit that to
the motor processor, which is responsible for actually executing
the appropriate motor command. In this situation, the proces-
sors act serially, one after another; however, in more complex
tasks such as transcription typing, all three processors often
work in parallel.
Besides the specification of the timing for each processor
and the connectivity of the processors, Card, Moran, and Newell
named some general operating principles ranging from very
general and qualitative to detailed and quantitative. For exam-
ple, Principle P9, the Problem Space Principle, stated:
The rational activity in which people engage to solve a problem can be
described in terms of (1) a set of states of knowledge, (2) operators for
changing one state into another, (3) constraints on applying operators,
and (4) control knowledge for deciding which operator to apply next.
(Card et al. 1983, p. 27)
This is a particularly general and somewhat vague principle.
In contrast, consider Principle P5, Fitts’ Law:
The time Tpos to move the hand to a target of size S which lies a dis-
tance D away is given by:
Tpos  IMlog2(D/S  .5)
where IM  100 [70  120] ms/bit. (Card et al. 1983, p. 27).
This is a very specific principle that quantitatively describes
hand movement behavior, which is highly relevant to, say, point-
ing with a mouse. Overall, the specification of the MHP is quite
thorough, and it lays out a basis for a cognitive architecture able
to do a wide variety of HCI-relevant tasks; however, Card et al.
(1983) did not implement the MHP as a running cognitive ar-
chitecture. This is likely for pedagogical reasons; it is not nec-
essary to have a complete, running cognitive architecture for
the general properties of that architecture to be useful for guid-
ing HCI researchers and practitioners. At the time, computa-
tional modeling was the domain of a very specialized few in cog-
nitive psychology.
Card et al. (1983) laid out another concept that has been
highly influential throughout HCI and particularly in the com-
munity of computational modelers. This is GOMS, which stands
for Goals, Operators, Methods, and Selection rules. GOMS is a
framework for task analysis that describes routine cognitive
skills in terms of the four listed components. Routine cognitive
skills are those where the user knows what the task is and how
to do the task without doing any problem solving. Text editing
with a familiar editor is the prototypical case of this, but clearly,
a great many tasks of interest in HCI could be classified as rou-
tine cognitive skills. Thus, the potential applicability of GOMS
is quite broad. Indeed, GOMS has been applied to a variety
of tasks; the website http://www.gomsmodel.org lists 143 GOMS-
related papers in its bibliography.
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 98
98 • BYRNE
What does a GOMS analysis provide? Essentially, a GOMS
analysis of a task describes the hierarchical procedural knowl-
edge a person must have to successfully complete that task.
Based on that, and the sequence of operators that must be ex-
ecuted, it is possible to make quantitative predictions about the
execution time for a particular task. Other analyses, such as pre-
dictions of error, functionality coverage, and learning time, are
also sometimes possible. Since the original formulation pre-
sented in Card et al. (1983), a number of different forms of
GOMS analysis have been developed, each with slightly differ-
ent strengths and weaknesses ( John & Kieras, 1996).
The core point as it relates to cognitive architectures is that
GOMS analysis is originally based on a production rule analysis
(Card, personal communication, 1999). Because this will come
up several times, a brief introduction to production systems is
warranted. Production rules are IF-THEN condition-action pairs,
and a set of production rules (or simply “productions,” or just
“rules”) and a computational engine that interprets those pro-
ductions is called a “production system.” In addition to produc-
tions, production systems contain some representation of the
current state. This representation typically consists of a set
of loosely structured data elements such as propositions or
attribute-value pairs. This set is called the “working memory”
or “declarative memory.” Because “working memory” is also
a psychological term with somewhat different meaning in
that literature, “declarative memory” will be used in all further
discussions.
The operation of a production system is cyclic. On each cy-
cle, the system first goes through a pattern-matching process.
The IF side of each production tests for the presence of a par-
ticular pattern in declarative memory. When the IF conditions of
a production are met, the production is said to fire, and the ac-
tions specified on the THEN side are executed. The actions can
be things such as pressing a button, or even some higher level
abstraction of action (e.g., “turn left”). Actions also include mod-
ifying the contents of declarative memory, which usually means
that a different production or productions will match on the
next cycle. At this abstract and purely symbolic level, production
systems are Turing complete and thus can compute anything
that is computable (Newell, 1990), thus, they should be flexible
enough to model the wide array of computations performed
by the human cognitive system.
This is relevant to cognitive architectures because most cog-
nitive architectures are (or contain) production systems. GOMS
was actually abstracted from production rule analysis. Card et al.
(1983) discovered that, for routine cognitive skills, the structure
of the productions was quite similar across tasks and a more
abstract representation was possible. This representation is the
original GOMS formulation. Thus, translating a GOMS analysis
into production rules, the language of most cognitive architec-
tures, is generally straightforward. Similarly, for routine cogni-
tive skills, it is often relatively simple to derive a GOMS analysis
from the set of productions used to model the task. Models
based on cognitive architectures can go well beyond routine
cognitive skills, but this connection has certainly influenced the
evolution of research on cognitive architecture and HCI. This
connection has also fed back into research and development of
GOMS techniques themselves, such as Natural GOMS Language
(NGOMSL; Kieras, 1988). NGOMSL allows the prediction of
learning time for the knowledge described in a GOMS model
based on a theory of transfer of training referred to as cogni-
tive complexity theory (CCT), which will be described in more
detail in the next section.
Cognitive Complexity Theory (CCT)
When someone has learned to perform a task with a particular
interface and must switch, doing the same task with a new in-
terface, how much better off will they be than someone just
learning to do the task with the new interface?—that is, how
much is the knowledge gained from using the old interface
“transferred” to using the new interface? This kind of question
has intrigued psychologists for at least a century, and having
some answers to this question has implications for training pro-
grams, user interface design, and many other areas. Cognitive
complexity theory (Bovair, Kieras, & Polson, 1990; Kieras & Pol-
son, 1985) is a psychological theory of transfer of training ap-
plied to HCI. Most relevant to the current discussion, this the-
ory is based on production rules. The major points of CCT are
as follows:
• Knowledge of the procedures that people need to execute
to perform routine tasks can be represented with production
rules. The relevant production rules can be generated based
on a GOMS analysis of the task to be modeled.
• The complexity of a task will be reflected in the number and
content of the production rules. When certain conventions
are adopted about the style of those production rules, com-
plexity is reflected almost entirely in the number of rules.
• The time it takes to execute a procedure can be predicted
with a production system that interprets those rules along
with a set of typical operator times, for example, the time it
takes to type a three-letter command. The production inter-
preter used in this work was not intended to be a general
cognitive architecture, but the production system framework
is certainly consistent with current architectures.
• The time it takes to learn a task is a function of the number of
new rules that the user must learn. “New” is clearly defined in
this context. If the user already has a production, and a new
task requires a rule that is similar (again, similarity is well de-
fined based on the production rule syntax), then the rule for
the new task need not be learned.
• Some predictions about errors and speedup with practice can
also be gleaned from the contents of the production rules.
Obviously, this was an ambitious agenda, and there are many
subtleties. For example, the notion of a “task” as the term was
used in the description of CCT actually includes more than just
the task at an abstract level. Consider a simple instance of a text-
editing task, deleting the word “redux” from the middle of a sen-
tence. The actual commands needed to accomplish this task
could be very different in different text editors, thus, modeling
the “delete word” task would require two different sets of pro-
ductions, one for each editor—that is, the necessary knowl-
edge, and thus the production rules for representing it, is ac-
tually a function both of the task from the user point of view
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 99
(e.g., “delete word”) and the interface provided to accomplish
the task. Transfer from one text editor to another therefore de-
pends a great deal on the particulars of each interface. CCT thus
predicts asymmetrical transfer: learning editor A after editor B
should not be the same as learning editor B after editor A.
CCT models, such as a GOMS analysis, omit modeling many
details of user behavior. In general, anything that falls outside
the domain of procedural knowledge (how-to-do-it knowledge)
is not modeled. This means that the model does not model mo-
tor actions such as keypresses, and instead has a “DoKeystroke”
primitive operator. Nor do CCT models model things such as
natural language comprehension, clearly a requirement in text
editing. CCT models also do not include any model of the per-
ceptual processes required by users—the model was simply
given information about the state of the display and did not
have to, for example, look to see if the cursor was over a partic-
ular character. This is the same scope as a typical GOMS model,
though a CCT model is more formalized and quantitative than
the GOMS models described by Card et al. (1983).
In spite of these limitations (or perhaps in part because
these limitations allowed the researchers to concentrate on the
most central aspects of the phenomena), CCT fared very well.
Numerous laboratory experiments provide empirical support
for many of the claims of CCT (see especially Bovair et al., 1990).
The CCT framework was developed and validated in greatest de-
tail to pre-GUI text editing, but it has also been applied to menu-
based systems (Polson, Muncher, & Engelbeck, 1986) and a con-
trol panel device (Kieras & Bovair, 1986). Singley and Anderson
(1989) provided a strikingly similar analysis of transfer of train-
ing, as well as supporting empirical results, lending credence
to the CCT analysis. CCT was certainly one of the most promi-
nent early successes of computational modeling in HCI.
CAPS
CAPS (Collaborative Activation-based Production System; Just &
Carpenter, 1992) is a cognitive architecture designed to model
individual differences in working memory (WM) capacity and
the effects of working memory load. This speciality is applica-
ble to a number of HCI situations. Certainly, some kinds of user
interfaces can create excessive working memory demands, for
example, phone-based interfaces. In phone-based interaction
(PBI), options do not remain on a screen or in any kind of avail-
able storage; rather, users are forced to remember the options
presented. This seems like a prime candidate for modeling with
a system designed to capture the effects of working memory de-
mand, and this is exactly what Huguenard, Lerch, Junker, Patz,
and Kass (1997) did. Their data showed that, contrary to guide-
line advice and most people’s intuition, restricting phone
menus to only a few (three) items each does not reduce error
rates. The CAPS-based model provided a clear theoretical
account of this phenomenon. The model showed that short
menus are not necessarily better in PBI because of two side ef-
fects of designing menu hierarchies with few options at each
level. First, for the same number of total items, this increases
menu depth, which creates working memory demand. Second,
with fewer items at each level, each individual item has to be
more general and therefore more vague, especially at the top
5. Cognitive Architecture • 99
levels of the hierarchy. This forces users to spend WM resources
on disambiguating menu items when they are in a situation
where WM demands outstrip supply.
Another application of CAPS that is HCI relevant is the ac-
count of postcompletion error provided by Byrne and Bovair
(1997). What is a postcompletion error? Anecdotal evidence and
intuition suggests that, when interacting with man-made arti-
facts, certain kinds of errors occur with greater frequency than
others. In particular, one entire family of errors seems anecdo-
tally common: errors that are made when some part of a task oc-
curs after the main goal of the task has been accomplished
(hence, “postcompletion”). Nearly everyone reports having
made an error of this type at one time or another. Here are two
prototypical examples:
1. Leaving the original on the glass of a photocopier. The main
goal one generally has when using a photocopier is “get
copies,” and this goal is satisfied before one remove the orig-
inal document. This error is less common now that many
photocopiers include document feeders; the more current
equivalent is leaving a document on the glass in a flatbed
scanner.
2. Leaving one’s bank card in an automated teller machine
(ATM). Again, the main goal is something on the order of
“get cash,” and in many ATMs, card removal occurs after the
cash is dispensed. This error was common enough in the
first generation of ATMs that many ATMs are now designed in
such a way that this error is now impossible to make.
Other postcompletion errors include leaving the gas cap off
after filling up the car’s gas tank, leaving change in vending ma-
chines, and more—most readers can probably think of several
others. While numerous HCI researchers were aware of this
class of error (e.g., Young, Barnard, Simon, & Whittington, 1989;
Polson et al., 1992), no previous account explained why this
type of error is persistent, yet not so frequent that it occurs
every time. The CAPS model provides just such an account, and
can serve as a useful example of the application of a cognitive ar-
chitecture to an HCI problem.
Like most other production systems, CAPS contains two
kinds of knowledge: (a) declarative memory and (b) produc-
tions. Declarative memory elements in CAPS also have associ-
ated with them an activation value, and elements below a
threshold level of activation cannot be matched by productions’
IF sides. Additionally, unlike most other production systems, the
THEN side of a CAPS production may request that the activation
of an element be incremented. For this to be truly useful in
modeling working memory, there is a limit to the total amount
of activation available across all elements. If the total activation
exceeds this limit, then all elements lose some activation to
bring the total back within the limit. This provides a mechanism
for simulating human working memory limitations.
In Byrne and Bovair’s (1997) postcompletion error model,
there is a production that increments the activation of subgoals
when the parent goal is active and unsatisfied. Therefore, to use
the photocopier example, the “get copies” subgoal supplies ac-
tivation to all the unfulfilled subgoals throughout the task; how-
ever, when the “get copies” goal is satisfied, the activation sup-
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 100
100 • BYRNE
ply to the subgoals stops. Because the goal to remove the orig-
inal is a subgoal of that goal, it loses its activation supply. Thus,
what the model predicts is that the postcompletion subgoals are
especially vulnerable to working memory load, and lower ca-
pacity individuals are more “at risk” than higher-capacity indi-
viduals. Byrne and Bovair conducted an experiment to test this
prediction, and the data supported the model.
This is a good demonstration of the power of cognitive
architectures. Byrne and Bovair neither designed nor imple-
mented the CAPS architecture, but were able to use the theory
to construct a model that made empirically testable predictions,
and those predictions were borne out. Though it seems unlikely
that CAPS will guide future HCI work (since its designers have
gone in a different direction), it provides an excellent example
case for a variety of reasons.
Soar
The development of Soar is generally credited to Allan Newell
(especially, Newell, 1990), and Soar has been used to model a
wide variety of human cognitive activity from syllogistic reason-
ing (Polk & Newell, 1995) to flying combat aircraft in simulated
war games ( Jones et al., 1999). Soar was Newell’s candidate
“unified theory of cognition” and was the first computational
theory to be offered as such.
While Soar is a production system like CAPS, it is possible to
think of Soar at a more abstract level. The guiding principle be-
hind the design of Soar is Principle P9 from the Model Human
Processor, the Problem Space Principle. Soar casts all cognitive
activity as occurring in a problem space, which consists of a num-
ber of states. States are transformed through the applications of
operators. Consider Soar playing a simple game like tic-tac-toe as
player “X.” The problem space is the set of all states of the tic-tac-
toe board—not a very large space. The operators available at any
given state of that space are placing an X at any of the available
open spaces on the board. Obviously, this is a simplified exam-
ple; the problem space and the available operators for flying an
F-16 in a simulated war game are radically more complex.
Soar’s operation is also cyclic, but the central cycle in Soar’s
operation is called a decision cycle. Essentially, on each decision
cycle, Soar answers the question, “What do I do next?” Soar
does this in two phases. First, all productions that match the
current contents of declarative memory fire. This usually causes
changes in declarative memory, so other productions may now
match. Those productions are allowed to fire, and this contin-
ues until no new productions fire. At this time, the decision pro-
cedure is executed, in which Soar examines a special kind of de-
clarative memory element, the preference. Preferences are
statements about possible actions, for example, “operator o3 is
better than o5 for the current operator,” or “s10 rejected for su-
pergoal state s7.” Soar examines the available preferences and
selects an action. Thus, each decision cycle may contain many
production cycles. When modeling human performance, the
convention is that each decision cycle lasts 50 ms, so produc-
tions in Soar are very low-level, encapsulating knowledge at a
small grain size.
Other than the ubiquitous application of the problem space
principle, Soar’s most defining characteristics come from two
mechanisms developed specifically in Soar: (a) universal sub-
goaling and (b) a general-purpose learning mechanism. Because
the latter depends on the former, universal subgoaling will be
described first. One of the features of Soar’s decision process
is that it is not guaranteed to have an unambiguous set of pref-
erences to work with. Alternately, there may be no preferences
listing an acceptable action. Perhaps the system does not know
any acceptable operators for the current state, or perhaps the
system lacks the knowledge of how to apply the best operator.
Whatever the reason, if the decision procedure is unable to se-
lect an action, an impasse is said to occur. Rather than halting or
entering some kind of failure state, Soar sets up a new state in
a new problem space with the goal of resolving the impasse. For
example, if multiple operators were proposed, the goal of the
new problem space is to choose between the proposed opera-
tors. In the course of resolving one impasse, Soar may encounter
another impasse and create another new problem space, and so
on. As long as the system is provided with some fairly generic
knowledge about resolving degenerate cases (e.g., if all else
fails, choose randomly between the two good operators), this
universal subgoaling allows Soar to continue even in cases
where there is little knowledge present in the system.
Learning in Soar is a by-product of universal subgoaling.
Whenever an impasse is resolved, Soar creates a new produc-
tion rule. This rule summarizes the processing that went on in
the substate. The resolution of an impasse makes a change to
the superstate (the state in which the impasse originally oc-
curred); this change is called a result. This result becomes the
action, or THEN, side of the new production. The condition, or
IF, side of the production is generated through a dependency
analysis by looking at any declarative memory item matched in
the course of determining this result. When Soar learns, it learns
only new production rules, and it only learns as the result of re-
solving impasses. It is important to realize that an impasse is not
equated with failure or an inability to proceed in the problem
solving, but may arise simply because, for example, there are
multiple good actions to take and Soar has to choose one of
them. Soar impasses regularly when problem solving, and thus
learning is pervasive in Soar.
Not surprisingly, Soar has been applied to a number of learn-
ing-oriented HCI situations. One of the best examples is the re-
cent work by Altmann (Altmann & John, 1999; Altmann, 2001).
Altmann collected data from an experienced programmer while
she worked at understanding and updating a large computer
program by examining a trace. These data included verbal pro-
tocols (e.g., thinking aloud) as well as a log of the actions taken
(keystrokes and scrolling). The programmer occasionally scrolled
back to find a piece of information that had previously been dis-
played. Altmann constructed a Soar model of her activity. This
model simply attempts to gather information about its environ-
ment; it is not a complete model of the complex knowledge of
an experienced programmer. The model attends to various
pieces of the display, attempts to comprehend what it sees, and
then issues commands. “Comprehension” in this model is man-
ifested as an attempt to retrieve information about the object
being comprehended.
When an item is attended, this creates a production that
notes that the object was seen at a particular time. Because
learning is pervasive, Soar creates many new rules like this, but
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 101
because of the dependency-based learning mechanism, these
new productions are quite specific to the context in which the
impasse originally occurred. Thus, the “index” into the model’s
rather extensive episodic memory consists of very specific cues,
usually found on the display. Seeing a particular variable name is
likely to trigger a memory for having previously seen that vari-
able name. Importantly, this memory is generated automatically,
without need for the model to deliberately set goals to remem-
ber particular items.
Altmann (2001) discussed some of the HCI ramifications for
this kind of always-on episodic memory trace in terms of display
clutter. While avoiding display clutter is hardly new advice, it is
generally argued that it should be avoided for visual reasons
(e.g., Tullis, 1983). What Altmann’s model shows, however, is
that clutter can also have serious implications for effective use of
episodic memory. Clutter can create enormous demands for re-
trieval. Since more objects will generally be attended on a clut-
tered display, the episodic trace will be large, lowering the pre-
dictive validity for any single cue. It is unlikely that kind of
analysis would have been generated if it had not been guided by
a cognitive architecture that provided the omnipresent learning
of Soar.
Soar has also been used to implement models of exploratory
learning, somewhat in the spirit of LICAI (described later).
There are two prominent models here, (a) one called IDXL (Rie-
man, Young, & Howes, 1996) and (b) a related model called
Task-Action Learner (Howes & Young, 1996). These models
both attempt to learn unfamiliar GUI interfaces. IDXL operates
in the domain of using software to construct data graphs, while
the Task-Action Leaner starts with even less knowledge and
learns basic GUI operations such as how to open a file. For
brevity, only IDXL will be described in detail.
IDXL goes through many of the same scanning processes as
LICAI, but must rely on very different mechanisms for evalua-
tion since Soar is fundamentally different from LICAI. IDXL
models evaluation of various display elements as search through
multiple problem spaces, one that is an internal search through
Soar’s internal knowledge and the other a search through the
display. As items are evaluated in the search, Soar learns pro-
ductions that summarize the products of each evaluation. At
first, search is broad and shallow, with each item receiving a
minimum of elaboration; however, that prior elaboration guides
the next round of elaboration, gradually allowing IDXL to focus
in on the “best” items. This model suggests a number of ways
in which interface designers could thus help learners acquire a
new the knowledge needed to utilize a new interface. Like the
LICAI work, the IDXL work highlights the need for good labels
to guide exploration. A more radical suggestion is based on one
of the more subtle behavior of users and IDXL. When explor-
ing and evaluating alternatives, long pauses often occur on par-
ticular menu items. During these long pauses, IDXL is attempt-
ing to determine the outcome of selecting the menu item being
considered. Thus, one suggestion for speeding up learning of a
new menu-driven GUI is to detect such pauses, and show (in
some easily undoable way) what the results of selecting that
item would be. For instance, if choosing that item brings up a di-
alog box for specifying certain options, that dialog box could
be shown in some grayed-out form, and would simply vanish if
the user moved off of that menu item. This would make the
5. Cognitive Architecture • 101
evaluation of the item much more certain and would be an ex-
cellent guide for novice users. This is not unlike ToolTips for
toolbar icons, but on a much larger scale.
A model that does an excellent job of highlighting the power
of cognitive architectures is NTD-Soar (Nelson, Lehman, &
John, 1994). NTD stands for “NASA Test Director,” who
. . . is responsible for coordinating many facets of the testing and prepa-
ration of the Space Shuttle before it is launched. He must complete a
checklist of launch procedures that, in its current form, consists of 3000
pages of looseleaf manuals . . . as well as graphical timetables describ-
ing the critical timing of particular launch events. To accomplish this, the
NTD talks extensively with other members of launch team over a two-
way radio. . . . In addition to maintaining a good understanding of the
status of the entire launch, the NTD is responsible for coordinating trou-
bleshooting attempts by managing the communication between mem-
bers of the launch team who have the necessary expertise. (p. 658)
Constructing a model that is even able to perform this task at
all is a significant accomplishment. Nelson, Lehman, and John
(1994) were able to not only build such a model, but this model
was able to produce a timeline of behavior that closely matched
the timeline produced by the actual NTD being modeled—that
is, the ultimate result was a quantitative model of human per-
formance, and an accurate one at that.
It is unlikely that such an effort could have been accom-
plished without the use of an integrated cognitive architecture.
The NTD model made extensive use of other Soar models. Nel-
son, Lehman, and John (1994) did not have to generate and im-
plement theory of natural language understanding to model the
communication between the NTD and others, or the NTD read-
ing the pages in the checklist, because one had already been
constructed in Soar (Lewis, 1993). They did not have to con-
struct a model of visual attention to manage the scanning and vi-
sual search of those 3000 pages, because such a model already
existed in Soar (Wiesmeyer, 1992). A great deal of knowledge
engineering still took place in order to understand and model
this complex task, but using an integrated architecture greatly
eased the task of the modelers.
While this modeling effort was not aimed at a particular HCI
problem, it is not difficult to see how it would be applicable to
one. If one wanted to replace the 3000-page checklist with
something like a personal digital assistant (PDA), how could the
PDA be evaluated? There are very few NTDs in the world, and it
is unlikely that they would be able to devote much time to par-
ticipatory design or usability testing. Because an appropriate
quantitative model of the NTD exists, however, it should be pos-
sible to give the model a simulated PDA and assess the impact of
that change on the model’s performance. Even if the model
does not perfectly capture the effects of the change, it is likely
that the model would identify problem areas and at least guide
the developers in using any time they have with an actual NTD.
Soar has also been used as the basis for simulated agents in
war games, as mentioned previously ( Jones et al., 1999). This
model (TacAir-Soar) participates in a virtual battle space in
which humans also participate. TacAir-Soar models take on a
variety of roles in this environment, from fighter pilots to heli-
copter crews to refueling planes. Their interactions are more
complex than simple scripted agents, and they can interact with
humans in the environment with English natural language. One
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 102
102 • BYRNE
of the major goals of the project is to make sure that TacAir-Soar
produces humanlike behavior, because this is critical to their
role, which is to serve as part of training scenarios for human
soldiers. In large-scale simulations with many entities, it is much
cheaper to use computer agents than to have humans fill every
role in the simulation. While agents (other than the ubiquitous
and generally disliked paper clip) have not widely penetrated
the common desktop interface, this remains an active HCI
research area, and future agents in other roles could also be
based on cognitive architecture rather than more engineering-
oriented AI models.
While the Soar architecture is still being used for applied AI
work such as TacAir-Soar, there have been essentially no new
HCI-oriented Soar efforts in the last several years. Nothing, in
principle, has precluded such efforts, but for the time being,
Soar is primarily an example of previous achievements using
cognitive architectures in HCI, as opposed to a source of new
insight.
CONTEMPORARY ARCHITECTURES
Current cognitive architectures are being actively developed,
updated, and applied to HCI-oriented tasks. Two of the three
most prominent systems (EPIC and ACT-R) are production sys-
tems, or tather are centrally built around production systems.
While all contain production rules, the level of granularity of an
individual production rule varies considerably from architec-
ture to architecture. Each one has a different history and
unique focus, but they share a certain amount of intellectual
history; in particular, they have all been influenced one way or
another by both the MHP and each other. At some level, they
may have more similarities than differences; whether this is
because they borrow from one another or because the science
is converging is still an open question. The third system is
somewhat different from these two production system models
and will be considered first.
LICAI/CoLiDeS
LICAI (Kitajima & Polson, 1997) is a primary example of an HCI-
oriented cognitive architecture not based on the production sys-
tem framework. With the exception of the Soar work on ex-
ploratory learning, all of the work discussed up to this point more
or less assumes that the modeled users are relatively skilled with
the specific interface being used; these approaches do a poor
job of modeling relatively raw novices. One of the main goals of
LICAI is to address this concern. The paradigm question ad-
dressed by LICAI is, “How do users explore a new interface?”
Unlike the other architectures discussed, LICAI’s central con-
trol mechanisms are not based on a production system. Instead,
LICAI is built around an architecture originally designed to
model human discourse comprehension: construction-integra-
tion (C-I; Kintsch, 1998). Like production systems, C-I’s opera-
tion is cyclic. What happens on those cycles, however, is some-
what different from what happens in a production system. Each
cycle is divided into two phases: (a) construction and (b) inte-
gration (hence, the name). In the construction phase, an initial
input (e.g., the contents of the current display) is fed into a
weakly constrained, rule-based process that generates a net-
work of propositions. Items in the network are linked based on
their argument overlap. For example, the goal of “graph data”
might be represented with the proposition (PERFORM GRAPH
DATA). Any proposition containing GRAPH or DATA would thus
be linked to that goal.
Once the construction phase is completed, the system is left
with a linked network of propositions. What follows is the inte-
gration phase, in which activation propagates through the net-
work in a neural, network-like fashion. Essentially, this phase is
a constraint-satisfaction phase, which is used to select one of
the propositions in the network as the “preferred” one. For ex-
ample, the system may need to select the next action to perform
while using an interface. Action representations will be added to
the network during the construction phase, and an action will
be selected during the integration phase. The action will be per-
formed, and the next cycle initiated. Various C-I models have
used this basic process to select things other than actions. The
original C-I system used these cycles to select between differ-
ent interpretations of sentences.
LICAI has three main types of cycles: (a) one to select ac-
tions, (b) one to generate goals, and (c) one to select goals. This
contrasts with how most HCI tasks have been modeled in pro-
duction system architectures; in such systems, the goals are usu-
ally included in the knowledge given to the system. This is not
true in LICAI; in fact, the knowledge given to LICAI by the mod-
elers is minimal. For the particular application of LICAI, which
was modeling users who knew how to use a Macintosh for other
tasks (e.g., word processing) and were now being asked to plot
some data using a Macintosh program called CricketGraph (one
group of users actually worked with Microsoft Excel), it included
some very basic knowledge about the Macintosh GUI and some
knowledge about graphing. Rather than supply the model with
the goal hierarchy, Kitajima and Polson (1997) gave the model
the same minimal instructions that were given to the subjects.
One of the major jobs of the LICAI model, then, was to generate
the appropriate goals as they arose while attempting to carry
out the instructions.
Again, this illustrates one of the benefits of using a cognitive
architecture to model HCI tasks. Kitajima and Polson (1997) did
not have to develop a theory of text comprehension for LICAI to
be able to comprehend the instructions given to subjects; since
LICAI is based on an architecture originally designed for text
comprehension, they essentially got that functionality gratis. Ad-
ditionally, they did not include just any text comprehension en-
gine, but one that makes empirically validated predictions about
how people represent the text they read. Thus, the claim that
the model started out with roughly the same knowledge as the
users is highly credible.
The actual behavior of the model is also revealing, as it ex-
hibits many of the same exploratory behaviors seen in the users.
First, the model pursues a general strategy that can be classi-
fied as label-following (Polson et al., 1992). The model, like the
users, had a strong tendency to examine anything on the screen
that had a label matching or nearly matching (e.g., a near syn-
onym) a key word in the task instructions. When the particular
subtask being pursued by the model contained well-labeled
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 103
steps, the users were rapid, which was predicted by the model.
While this prediction is not counter-intuitive, it is important
to note that LICAI is not programmed with this strategy. This
strategy naturally emerges through the normal operation of
construction-integration through the linkages created by shared
arguments. The perhaps less-intuitive result—modeled suc-
cessfully by LICAI—is the effect of the number of screen objects.
During exploration in this task, users were slower to make
choices if there were more objects on the screen, but only if
those items all had what were classified as “poor” labels. In the
presence of “good” labels (literal match or near synonym), the
number of objects on the screen did not affect decision times of
either the users or LICAI.
The programmers who implemented the programs operated
by the users put in several clever direct manipulation tricks. For
example, to change the properties of a graph axis, one double-
clicks on the axis and a dialog box specifying the properties of
that axis appears. Microsoft Excel has some functionality that is
most easily accessed by drag-and-drop. Franzke (1994) found
that in a majority of first encounters with these types of direct
manipulations, users required hints from the experimenter to
be able to continue, even after two minutes of exploration. LI-
CAI also fails at these interactions because no appropriate links
are formed between any kind of task goal and these unlabeled,
apparently static screen objects during the construction phase.
Thus, these screen objects tend to receive little activation dur-
ing the integration phase, and actions involving other objects
are always selected.
Overall, LICAI does an excellent job of capturing many of the
other empirical regularities in exploratory learning of a GUI in-
terface. This is an important issue for many interfaces—partic-
ularly, any interface that is aimed at a walk-up-and-use audience.
While currently common walk-up-and-use interfaces, such at
ATMs, provide simple enough functionality that this is not al-
ways enormously difficult, this is not the case for more sophis-
ticated systems, such as many information kiosks.
More recently, LICAI has been updated (and renamed to
CoLiDeS, for Comprehension-based Linked model of Deliberate
Search; Kitajima, Blackmon, & Polson, 2000) to handle interac-
tion with web pages. This involves goals that are considerably
less well elucidated and interfaces with a much wider range of
semantic content. In order to help deal with these complexities,
LICAI has been updated with a more robust attentional mecha-
nism and a much more sophisticated notion of semantic simi-
larity based on Latent Semantic Analysis (LSA; Landauer & Du-
mais, 1997). LSA is a technique for mathematically representing
the meaning of words. While the details of the mathematics are
complex, the underlying idea is straightforward. To perform
LSA, one needs a large corpus of documents. An enormous ma-
trix is created, showing which words appear in which contexts
(documents). This matrix is then reduced by a mathematical
technique called “singular value decomposition” (a cousin of
factor analysis) to a smaller dimensional space of, say, 300 by 300
dimensions. Each word can then be represented by a vector in
this space. The “meaning” of a passage is simply the sum of the
vectors of the individual words in the passage. The similarity of
two words (or two passages) can be quantified as the cosine
between the vectors representing the words (or passages), with
larger numbers signifying greater semantic similarity.
5. Cognitive Architecture • 103
While LSA is certainly surrounded by a host of mathematical
and theoretical issues (it should be noted that many similar
techniques that differ in a variety of subtle ways have also been
developed), the key result from an HCI point of view is that the
technique works. Similarity measures generated by LSA gener-
ally agree very well with human judges, even for complex lan-
guage understanding tasks such as grading essays. (In fact, an
LSA-based system for grading essays and similar tasks has been
turned into a successful commercial product; see http://www
.pearsonkt.com.)
Based on the LSA-augmented CoLiDeS, Blackmon and col-
leagues (2002, 2003, 2005) have developed a web usability as-
sessment and repair technique they call “Cognitive Walkthrough
for the Web” (CWW). CWW is based on the idea that when users
are exploring a large website, they go through a two-stage
process of parsing and understanding the page, and ultimately
generate descriptions of the available actions. Using those de-
scriptions, they judge which action is most similar to their goal,
and then select the link or widget that corresponds with that ac-
tion. This is somewhat more complex than simply applying LSA
to all of the labels on the page and comparing each one to the
goal; the page is parsed into “regions,” each of which is as-
sessed, and smaller bits of text (say, link labels) are elaborated
with a C-I-based process. It is those elaborations, rather than the
raw link labels, that are compared via LSA to the description of
the user’s goal.
Based on this, CWW identifies where users with a particular
goal are likely to have navigation problems and even generates
problem severity estimates. These predictions have been vali-
dated in a series of experiments (Blackmon, Kitajima, & Polson,
2003; Blackmon, Kitajima, & Polson, 2005), and tools which par-
tially automate the process are available on the web (see http://
autocww.colorado.edu/). This is a significant advance in HCI, as
the problem of predicting how people use semantic similarity to
guide search on the web is not a trivial one and, prior to tech-
niques like this, has generally required extensive iteration of
user tests.
EPIC
With the possible exception of the NTD model, the models dis-
cussed up to this point have been primarily “pure” cognitive
models—that is, the perception and action parts of the mod-
els have been handled in an indirect, abstract way. These mod-
els focus on the cognition involved, which is not surprising
given the generally cognitive background of these systems;
however, even the original formulation of the MHP included
processors for perception and motor control. Additionally, in
fact, user interfaces have also moved from having almost ex-
clusively cognitive demands (e.g., one had to remember or
problem-solve to generate command names) to relying much
more heavily on perceptual-motor capabilities. This is one of
the hallmarks of the GUI, the shift to visual processing and di-
rect manipulation rather than reliance on complex composi-
tion of commands.
Providing accurate quantitative models for this kind of activ-
ity, however, requires a system with detailed models of human
perceptual and motor capabilities. This is one of the major foci
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 104
104 • BYRNE
and contributions of EPIC (for executive process interactive
control). EPIC is the brainchild of Kieras and Meyer (see espe-
cially 1996, 1997; Kieras, Wood, & Meyer, 1997). The overall
structure of the processors and memories in EPIC is shown in
Fig. 5.2. This certainly bears some surface similarity to the MHP,
but EPIC is substantially more detailed. EPIC was explicitly de-
signed to pair high-fidelity models of perception and motor
mechanisms with a production system. The perceptual-motor
processors represent a new synthesis of the human perfor-
mance literature, while the production system is the same one
used in the CCT work discussed earlier.
All EPIC processors run in parallel with one another. There-
fore, while the Visual Processor is recognizing an object on the
screen, the Cognitive Processor can decide what word should
be spoken in response to some other input, while at the same
time the Manual Motor processor is pressing a key. The infor-
mation flow is typical of traditional psychological models, with
information coming in through the eyes and ears, and outputs
coming from the mouth and hands. More specifically, what is
modeled in each of the processors is primarily time course.
EPIC’s Visual Processor does not take raw pixels as input and
compute that those pixels represent a letter “A.” Instead, it de-
termines whether the object on the screen can be seen and at
what level of detail, as well as how long it will take for a repre-
sentation of that object to be delivered to EPIC’s declarative
memory once the letter becomes available to the Visual Proces-
FIGURE 5.2. Overall structure of the EPIC architecture. Based on Kieras and Meyer (1996).
sor. The appearance of the letter can actually cause a number
of different elements to be deposited into EPIC’s declarative
memory at different times. For example, information about the
letter’s color will be delivered before information about the let-
ter’s identity.
Similarly, on the motor side, EPIC does not simulate the com-
putation of the torques or forces needed to produce a particu-
lar hand movement. Instead, EPIC computes the time it will take
for a particular motor output to be produced after the Cognitive
Processor has requested it. This is complicated by the fact that
movements happen in phases. Most importantly, each move-
ment includes a preparation phase and an execution phase. The
time to prepare a movement is dependent on the number of
movement features that must be prepared for each movement
and the features of the last movement prepared. Features that
have been prepared for the previous movement can sometimes
be reused to save time. EPIC can make repeated identical move-
ments rapidly because no feature preparation time is necessary
if the movements are identical. If they are not identical, the
amount of savings is a function of how different the current
movement is from the previous one. After being prepared, a
movement is executed. The execution time for a movement cor-
responds roughly to the time it takes to physically execute the
movement; the execution time for aimed movements of the
hands or fingers are governed by Fitts’ Law. EPIC’s motor
processors can only prepare one movement at a time, and can
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 105
only execute one movement at a time, but may be preparing
one movement while executing another. Thus, in some tasks, it
may be possible to effectively pipeline movements in order to
generate very rapid sequences of movements.
EPIC’s Cognitive Processor is a production system, the same
one that was used for the earlier CCT work. One highly salient
feature of this system is that multiple rules can fire on a pro-
duction cycle. In fact, there is no upper bound on the number
of productions that can fire on a cycle. Productions in this sys-
tem are at a much higher grain size than productions in Soar,
which gives EPIC a parallel quality at all levels—that is, all the
processors work in parallel, and EPIC’s cognitive processor is
itself capable of parallel processing.
This allows EPIC particular leverage in multiple-task situa-
tions. When more than one task is being performed, the tasks
can execute in parallel; however, many of the perceptual-motor
processors are effectively serial. People only have one set of eyes
that can only be aimed at one place at a time, so if multiple tasks
are ongoing and they both require the eyes, there must be
something that arbitrates. In EPIC, this additional knowledge
about how to manage multiple tasks is termed “executive”
knowledge, and the productions that implement this knowl-
edge execute in parallel with the productions implementing the
task knowledge.
Why is all this machinery and extra knowledge necessary? Be-
cause the world of HCI is changing. The GUI forced designers
and analysts to consider more seriously the perceptual-motor
constraints, and the propagation of computers with user inter-
faces away from the desktop and into mobile phones, kiosks, au-
tomobiles, and many, many other places creates a huge demand
on people’s ability to multitask. Multiple-task issues have largely
gone unmodeled and have been outside the theoretical scope
of most psychological accounts in HCI, at least before EPIC.
While LICAI and Soar have not been adequately equipped
to deal with high-performance perception and action compo-
nents of many tasks, EPIC is not equipped to handle some of the
issues covered by other architectures. In particular, EPIC does
not include any learning mechanisms, so it would be difficult
to generate EPIC models for many of the domains Soar has ap-
proached successfully. This is not a fatal shortcoming, however,
as there are a wide variety of domains in which learning is not a
key component and where high-fidelity modeling of perception
and action, along with multiple tasking, are central.
Naturally, these are the kinds of domains to which EPIC has
been applied. One of the first major applications of EPIC was to
a deceptively simple dual-task paradigm known as the “psycho-
logical refractory period” (or PRP; see Meyer & Kieras, 1997).
In this task, laboratory subjects are typically confronted with
two choice reaction time tasks, something on the order of,
“Either a red light or a green light will appear. If it’s red, hit the
‘L’ key. If it’s green, hit the ‘J’ key.” This sounds simple, but the
empirical literature is rich and shows a variety of subtle effects,
for which EPIC provides the first unified account. Critically, what
the EPIC models of these experiments show is that people’s
low-level strategies for scheduling the tasks play a large role in
determining performance in this simple paradigm.
EPIC has been used to model several tasks with a more di-
rectly HCI-oriented flavor. One of those tasks is menu selection
(Kieras & Meyer, 1997; Hornof & Kieras, 1997, 1999), but for
5. Cognitive Architecture • 105
brevity, a detailed description of these models will be omitted.
Another application of EPIC that definitely merits mention is the
model of telephone assistance operators (TAOs), data originally
presented in Gray et al. (1993). When a telephone customer di-
als “0,” a TAO is the person who answers. The TAOs modeled
here sat at a “dumb terminal” style workstation and assisted cus-
tomers in completing telephone calls. In particular, TAOs de-
termine how calls should be billed, and this is done by speak-
ing to the customer. The detailed EPIC models (Kieras et al.,
1997) covered a subset of the possible billing types.
This provided a good test of EPIC because the task was per-
formed under time pressure, and seconds—actually, millisec-
onds—counted in task performance. Second, this task is multi-
modal. The TAO must speak, type, listen, and look at a display.
Third, very fine-grained performance data were available to help
guide model construction. By now, it should come as no sur-
prise to the reader that it was possible to construct an EPIC
model that did an excellent job of modeling the time course of
the TAO’s interaction with the customer; however, this model-
ing effort went beyond just that and provided some insight into
the knowledge engineering problem facing modelers utilizing
cognitive architectures as well.
Like other production system models, EPIC provides a cer-
tain amount of freedom to the modeler in model construction.
While the architecture used provides certain kinds of con-
straints, and these constraints are critical in doing good science
and affecting the final form of the model (Howes & Young,
1997), the modeler does have some leeway in writing the pro-
duction rules in the model. This is true even when the produc-
tion rule model is derived from another structured representa-
tion such as a GOMS model, which was the case in the TAO
model. In EPIC, it is possible to write a set of “aggressive” pro-
ductions that maximize the system’s ability to process things in
parallel, while it is also possible to write any number of less ag-
gressive sets representing more conservative strategies. EPIC
will produce a quantitative performance prediction regardless
of the strategy, but which kind of strategy should the modeler
choose? There is generally no a priori basis for such a decision,
and it is not clear that people can accurately self-report on such
low-level decisions.
Kieras et al. (1997) generated an elegant approach to this
problem, later termed “bracketing” (Kieras & Meyer, 2000). The
idea is this: construct two models, one of which is the maximally
aggressive version. At this end of the strategy spectrum, the
models contain very little in the way of cognition. The Cognitive
Processor does virtually no deliberation and spends most of its
cycles simply reading off perceptual inputs and immediately
generating the appropriate motor output. This represents the
“super-expert” whose performance is limited almost entirely by
the rate of information flow through the perceptual-motor sys-
tems. At the other end of the spectrum, a model incorporating
the slowest reasonable strategy is produced. The slowest rea-
sonable strategy is one where the basic task requirements are
met, but with no strategic effort made to optimize scheduling to
produce rapid performance. The idea is that observed perfor-
mance should fall somewhere in between these two extremes,
hence, the data should be bracketed by the two versions of the
model. Different users will tend to perform at different ends of
this range for different tasks, so this is an excellent way to ac-
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 106
106 • BYRNE
commodate some of the individual differences that are always
observed in real users.
What was discovered by employing this bracketing proce-
dure to the TAO models was surprising. Despite the fact that the
TAOs were under considerable time pressure and were ex-
tremely well practiced experts, their performance rarely ap-
proached the fastest possible model. In fact, their performance
most closely matched a version of the model termed the “hier-
archical motor-parallel model.” In this version of the model, eye,
hand, and vocal movements are executed in parallel with one
another when possible; furthermore, the motor processor is
used somewhat aggressively, preparing the next movement
while the current movement was in progress. The primary place
where EPIC could be faster but the data indicated the TAOs
were not was in the description of the task knowledge. It is pos-
sible to represent the knowledge for this task as one single, flat
GOMS method with no use of subgoals. On the other hand, the
EPIC productions could represent the full subgoal structure or
a more traditional GOMS model. Retaining the hierarchical rep-
resentation—thus incurring time costs for goal management—
provided the best fit to the TAOs performance. This provides
solid evidence for the psychological reality of the hierarchical
control structure inherent in GOMS analysis, since even well-
practiced experts in fairly regular domains do not abandon it for
some kind of faster knowledge structure.
The final EPIC model that will be considered is the model of
the task first presented in Ballas, Heitmeyer, and Perez (1992).
Again, this model first appeared in Kieras and Meyer (1997), but
a richer version of the model is described in more detail later,
in Kieras, Meyer, and Ballas (2001). The display used is a split
screen. On the right half of the display, the user was confronted
with a manual tracking task, which was performed using a joy-
stick. The left half of the display was a tactical task in which the
user had to classify objects as hostile or neutral based on their
behavior. There were two versions of the interface to the tactical
task, one a command-line style interface using a keypad and one
a direct-manipulation-style interface using a touch screen. The
performance measure of interest in this task is the time taken to
respond to events (such as the appearance of a new object or a
change in state of an object) on the tactical display.
This is again a task well suited to EPIC because the perceptual-
motor demands are extensive. This is not, however, what makes
this task so interesting. What is most interesting is the human
performance data: in some cases, the keypad interface was
faster than the touch screen interface, and in many cases, the
two yielded almost identical performance, while in some other
cases, the touch screen was faster. Thus, general claims about
the superiority of GUIs do not apply to this case and a more pre-
cise and detailed account is necessary.
EPIC provides just the tools necessary to do this. Two models
were constructed for each interface, again using the bracketing
approach. The results were revealing. In fact, the fastest possible
models showed no performance advantage for either interface.
The apparent direct-manipulation advantage of the touch screen
for initial target selection was almost perfectly offset by some
type-ahead advantages for the keypad. The reason for the in-
consistent results is that the users generally did not operate at the
speed of the fastest possible model; they tended to work some-
where in between the brackets for both interfaces. However, they
tended to work more toward the upper (slowest-reasonable)
bracket for the touch screen interface. This suggests an advan-
tage for the keypad interface, but the caveat is that the slowest
reasonable performance bound for the touch screen was faster
than the slowest possible for the keypad. Thus, any strategy
changes made by users in the course of doing the task, perhaps
as a dynamic response to changes in workload, could affect
which interface would be superior at any particular point in the
task. Thus, results about which interface is faster are likely to
be inconsistent—which is exactly what was found.
This kind of analysis would be impossible to conduct without
a clear, quantitative human performance model. Constructing
and applying such a model also suggested an alternative inter-
face that would almost certainly be faster than either, which is
one using a speech-driven interface. One of the major perfor-
mance bottlenecks in the task was the hands, and so in this case,
voice-based interaction should be faster. Again, this could only
be clearly identified with the kind of precise quantitative mod-
eling enabled by something like the EPIC architecture.
ACT-R 5.0
ACT-R 5.0 (Anderson et al., 2004) represents another approach
to a fully unified cognitive architecture, combining a very broad
model of cognition with rich perceptual-motor capabilities. ACT-R
5.0 is the most recent iteration of the ACT-R cognitive architec-
ture (introduced in Anderson, 1993). ACT-R has a long history
within cognitive psychology, as various versions of the theory
have been developed over the years. In general, the ACT family
of theories has been concerned with modeling the results of psy-
chology experiments, and this is certainly true of the current in-
carnation, ACT-R. Anderson and Lebiere (1998) show some of
this range, covering areas as diverse as list memory (chapter 7),
choice (chapter 8), and scientific reasoning (chapter 11).
ACT-R is, like EPIC and Soar, a production system with activ-
ity centered on the production cycle, which is also set at 50 ms
in duration; however, there are many differences between ACT-
R and the other architectures. First, ACT-R can only fire one pro-
duction rule per cycle. When multiple production rules match
on a cycle, an arbitration procedure called conflict resolution
comes into play. Second, ACT-R has a well-developed theory of
declarative memory. Unlike EPIC and Soar, declarative memory
elements in ACT-R are not simply symbols. Each declarative el-
ement in ACT-R also has an activation value associated with it,
which determines whether and how rapidly it may be accessed.
Third, ACT-R contains learning mechanisms, but is not a perva-
sive learning system in the same sense as Soar. These mecha-
nisms are based on a “rational analysis” (Anderson, 1990) of the
information needs of an adaptive cognitive system.
For example of rational analysis, consider conflict resolution.
Each production in ACT-R has associated with it several numeric
parameters, including numbers which represent the probabil-
ity that if the production fires, the goal will be reached and the
cost, in time, that will be incurred if the production fires. These
values are combined according to a formula that trades off prob-
ability of success vs. cost and produces an “expected gain” for
each production. The matching production with the highest
expected gain is the one that gets to fire when conflict resolu-
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 107

5. Cognitive Architecture • 107

tion is invoked. The expected gain values are noisy, so the sys-
tem’s behavior is somewhat stochastic, and the probability and
cost values are learned over time so that ACT-R can adapt to
changing environments.
Similarly, the activation of elements in declarative memory
is based on a Bayesian analysis of the probability that a declara-
tive memory element will be needed at a particular time. This
is a function of the general utility of that element, reflected in
what is termed its “base-level” activation, and that element’s as-
sociation with the current context. The more frequently and re-
cently an element has been accessed, the higher its base-level
activation will be, and thus the easier it is to retrieve. This value
changes over time according to the frequency and recency of
use, thus this value is learned. These kinds of mechanisms have
helped enable ACT-R to successfully model a wide range of cog-
nitive phenomena.
The current version of the theory, 5.0, incorporates several
important changes over previous versions of the theory. First,
the architecture is now a fully modular architecture, with fully
separate and independent modules for different kinds of infor-
mation processing (such as declarative memory and vision).
Second, ACT-R 5.0 no longer contains a “goal stack” for auto-
mated goal management; goals are now simply items in declar-
ative memory and subject to processes like learning and decay.
Third, the new modular architecture allows processing in ACT-R
to be mapped onto the human brain, with different modules FIGURE 5.3. Overall structure of the ACT-R 5.0 architecture.
identified with different brain regions. Finally, a production rule
learning mechanism has been fully incorporated, so learning is
now more pervasive in ACT-R. The 5.0 version of the theory sub- Among other things, Ehret developed an ACT-R/PM model of a
sumes all previous versions of the theory including ACT-R/PM, fairly simple, but subtle, experiment. In that experiment, sub-
which is now considered obsolete. jects were shown a target color and asked to click on a button
In ACT-R 5.0, the basic production system is augmented with that would yield that color. The buttons themselves had four
four EPIC-like peripheral modules, as well as a goal module and types: (a) blank, (b) arbitrary icon, (c) text label, and (d) color.
a declarative memory module, as depicted in Fig. 5.3. Like EPIC, In the color condition, the task was simple: users simply found
all of these modules run in parallel with one another, giving the color that matched the target and then clicked the button.
ACT-R the ability to overlap processing. The peripheral modules In the text label condition, the task was only slightly more diffi-
come from a variety of sources. ACT-R’s Vision Module is based cult: users could read the labels on the buttons and select the
on the ACT-R Visual Interface described in Anderson, Matessa, correct one because the description matched the color. In the
and Lebiere (1997). This is a feature-based attentional visual arbitrary icon condition, more or less random pictures appeared
system, but does not explicitly model eye movements. Recently, on each icon (e.g., a mailbox). Users had to either memorize the
the Vision Module has been extended to include an eye-move- picture-to-color mapping, which they had to discover by trial
ment model (Salvucci, 2001a) as well. The Motor Module is and error, or memorize the location of each color, since the but-
nearly identical to the Manual Motor Processor in EPIC and is tons did not change their functions over time. In the hardest
based directly on the specification found in Kieras and Meyer condition, the blank condition, users simply had to memorize
(1996), and the Speech Module is similarly derived from EPIC. the mapping between button location and color, which they had
The Audition Module is a hybrid of the auditory system found in to discover through trial and error.
EPIC and the attentional system in ACT-R’s Vision Module. Clearly, the conditions will produce different average re-
One other important property of ACT-R is that it is possible sponse times, and what Ehret (1999) found is that they also pro-
to have models interact with the same software as the human duced somewhat different learning curves over time. Ehret
users being modeled. The software development conditions are added an additional manipulation as well: after users performed
fairly restrictive, but if these conditions are met, then both the the task for some time, all labeling was removed. Not surpris-
user and the model are forced to use the same software. This re- ingly, the amount of disruption was different in the different con-
duces the number of degrees of freedom available to the mod- ditions, reflecting the amount of incidental location learning that
eler in that it becomes impossible to force any unpleasant mod- went on as subjects performed the task. The ACT-R model that
eling details into the model of the user interface, because there Ehret constructed did an excellent job of explaining the results.
is no model of the user interface. More will be said about this This model represented the screen with the built-in visual mech-
issue in the next section. anisms from ACT-R and learned the mappings between color and
ACT-R has been applied to numerous HCI domains. The first location via ACT-R’s standard associative learning mechanisms.
example comes from the dissertation work of Ehret (1999). The initial difference between the various conditions was repro-
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 108
108 • BYRNE
duced, as were the four learning curves. The model also suffered
disruptions similar to those suffered by the human users when
the labels were removed. This model is an excellent demonstra-
tion of the power of ACT-R, exercising both the perceptual-mo-
tor capabilities of the system as well as the graded learning in
ACT-R’s rational-analysis driven mechanisms.
Salvucci (2001b) described an ACT-R model that tests ACT-
R’s ability to handle multimodal, high-performance situations in
a very compelling task: this model drives an automobile driving
simulator. This is not a robotics project; the ACT-R model does
not actually turn the steering wheel or manipulate the pedals;
rather, it communicates with the automobile simulation soft-
ware. The model’s primary job is to maintain lane position as
the car drives down the road. Salvucci (2001b) added an addi-
tional task that makes it particularly interesting: the model di-
als telephone numbers on a variety of mobile phone interfaces.
Two factors were crossed: (a) whether the telephone was di-
aled manually via keypad vs. dialed by voice, and (b) whether
the full telephone number needed to be dialed vs. a shortened
“speed dial” system. The model was also validated by compari-
son with data from human users.
Both the model and the human users showed that dialing
while not driving is faster than dialing while driving, and that
steering performance can be disrupted by telephone dialing.
Not surprisingly, the most disruptive interface was the “full-
manual” interface, where the full phone numbers were dialed
on a keypad. This is due largely to the fact that dialing with the
keypad requires visual guidance, causing the model (and the
users) to take their eyes off the road. There was very little dis-
ruption associated with the voice interfaces, regardless of
whether full numbers or speed-dial was used.
This is a powerful illustration of the value of cognitive
architectures for a number of reasons. First, the basic driving
model could simply be reused for this task; it did not have to be
reimplemented. Second, the model provides an excellent
quantitative fit to the human data. Third, this is an excellent
example of a situation where testing human users can be diffi-
cult. Testing human drivers with interfaces that degrade driving
performance is dangerous, so simulators are generally used for
this kind of evaluation; however, maintaining a driving simula-
tor requires a great deal of space and is quite expensive. If
someone wanted to test another variant of the telephone in-
terface, it would be much faster and cheaper to give that inter-
face to Salvucci’s (2001b) model than it would be to recruit and
test human drivers.
One of the long-standing criticisms of cognitive architectures
in HCI is that they are theoretical tools designed to support re-
search and are too difficult for the average practitioner to use
in a more applied context. This criticism certainly holds some
truth (though how much may be open to debate); however,
some researchers are actively trying to address it. One of the
most interesting developments in the cognitive architecture
arena in the past few years has been the work of John and col-
leagues (2004) on something they term “CogTool” (see http://
www.cogtool.org). CogTool allows interface designers to mock
up an interface in HTML, demonstrate the tasks to be per-
formed in a GUI environment, and have an ACT-R based cogni-
tive model of the tasks automatically synthesized. Alternative
interfaces can be mocked up, and predicted performance com-
pared, giving the designer important information about how
changes in the interface may impact user performance. This is
not limited strictly to prediction of time of execution, either, as
the full model output—a complete trace of the predicted user
behavior—is also available for inspection.
This is an important advance, as it makes the power of com-
plete and detailed computational cognitive modeling available
to people with no experience with cognitive architectures. It
also has important ramifications for the methodology of model
building, since models generated by CogTool are not hand-
tweaked in an attempt to optimize fit to data; in fact, the intent
is to make predictions about nonexistent data. Some validation,
however, has been performed, and early explorations with Cog-
Tool have found the time predictions to be quite accurate. As
exciting a development as this is, it is important to note that
there are clearly limitations. The language of available actions
is based on the keystroke-level model (KLM) of Card, Moran,
and Newell (1983) and is thus somewhat limited. For example,
it does not provide a model for any kind of cognition more com-
plex than very simple decisions about where to point. For ex-
ample, if one wanted to model use of an ATM, CogTool would
not model the process of the user trying to retrieve a security
code from memory. (However, if the user was highly practiced
such that this retrieval is very rapid and always successful, then
CogTool could be appropriate.) Efforts are underway to make
automatic translation of slightly more complex GOMS-style
models into ACT-R models possible (St. Amant, Freed, & Ritter,
2005), but that still would not address the more complex cog-
nition required by many interfaces. Nonetheless, CogTool rep-
resents a significant advance in making architecture-based mod-
eling accessible to those who are not experts in the field.
The last application of ACT-R in HCI to be covered in de-
tail is based on Pirolli and Card’s (1999) theory of information
foraging. This theory is based on the idea that people forage
for information in much the same way that animals forage for
food. The literature on the latter type of foraging is quite rich,
including equations that describe when foragers will give up
on a particular patch of information and move on to the next
one. Foraging for food is often guided by proximal cues (e.g.,
scent) that indicate the presence of distal content (e.g., food);
analogously, information foragers may use proximal cues (e.g.,
visible link labels) to guide their search for information that
they cannot presently access. In the spirit of the analogy, Pirolli
and Card term these proximal cues “information scent.” That
such a theory should be applicable to the web should be fairly
obvious.
This theory has spawned a number of models based on
modified versions of ACT-R. The original model from Pirolli and
Card (1999) was called ACT-IF (for “information foraging”).
Later versions were SNIF-ACT 1.0 (Card et al., 2001) and SNIF-
ACT 2.0 (Pirolli & Fu, 2003), where SNIF stands for “scent-based
navigation and information foraging.” These models use a mod-
ified form of ACT-R’s spreading activation to compute informa-
tion scent, which drives the model’s choices as it browses
through web pages. SNIF-ACT 1.0 and later also uses an altered
version of ACT-R’s conflict resolution mechanism to determine
when to abandon the current website (or information “patch”)
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 109
and move on to a new site. SNIF-ACT 2.0 adds a further im-
provement, which is the use of a new measure to compute the
strengths of association (and therefore spreading activation)
between words. This new measure, termed “pointwise mutual
information” (PMI), is much easier to compute than LSA, but
generally produces similar results. SNIF-ACT 2.0 was used to
predict link selection frequency in a large (N  244) Web be-
havior study with impressive results, accounting for 72% of
the selection variance on one site and 90% of the variance on
a second site.
Obviously, this work is clearly related to CWW. In fact, the for-
aging work has also led to a tool to help automate the process of
web usability assessment, this one named “Bloodhound” (Chi
et al., 2003). Bloodhound takes a site and a goal description and
uses scent metrics combined with Monte Carlo methods to
compute usage patterns and identify navigation problems. The
two projects clearly have similar goals and similar outputs. What
is less clear is the extent to which the differences in the under-
lying cognitive architecture and differences in other aspects of
the techniques (such as LSA vs. PMI) actually lead to different
predictions about user behavior. As of this writing, the two sys-
tems have never both been applied to the same website. How-
ever appealing such a “bake-off ” might be, it is important not
to lose sight of the advance even the possibility of such a com-
parison represents: it is only recently that the field has had psy-
chologically informed and quantitative predictions about some-
thing as complex as Web behavior available for potential
comparison. Thus, this is a concrete example of cognitive ar-
chitectures paying off in HCI.
Other published and ongoing work applying ACT-R to HCI-
oriented problems is available (e.g., Taatgen & Lee, 2003; Pee-
bles & Cheng, 2003; Byrne, 2001; Schoelles & Gray, 2000), but
space considerations prohibit a more exhaustive review. The vi-
tality of the research effort suggests that ACT-R’s combination of
perceptual-motor modules and a strong theory of cognition will
continue to pay dividends as an HCI research tool.
In fact, this is not limited to ACT-R; overall, cognitive archi-
tectures are an exciting and active area of HCI research. The sys-
tems described here all take slightly different approaches and
focus on slightly different aspects of various HCI problems, but
there is clearly a great deal of cross-pollination. Lessons learned
and advancements made in one architecture often affect other
TABLE 5.1. Architecture Feature Comparison
LICAI/CoLiDeS
Original focus Text comprehension
Basic cycle Construction- integration
Symbolic or activation-based? Both
Architectural goal management Special cycle types, supports
Vague goals
Detailed perceptual-motor systems No
Learning mechanisms No
Large, integrated models No
Extensive natural language Yes
Support for learning system None
User community* None
*Outside of the researchers who have developed the system.
5. Cognitive Architecture • 109
systems, for example, the development of EPIC’s peripheral sys-
tems clearly impacted ACT-R.
Comparisons
An exhaustive and detailed comparison of the major cognitive
architectures is beyond the scope of this chapter; however, an
excellent comparison that includes a number of other architec-
tures can be found in Pew and Mavor (1998). Certainly, the three
production systems are related: (a) Soar, (b) EPIC, and (c) ACT-R.
A major difference between them is their original focus; they
were originally developed to model slightly different aspects of
human cognition. As they develop, however, there appears to
be more convergence than divergence. This is generally taken as
a good sign that the science is cumulating. Still, there are dif-
ferences, and certainly between the production systems and LI-
CAI/CoLiDeS. Many of the relevant comparisons are summa-
rized in Table 5.1.
Most of the entries on this table have been previously dis-
cussed, with the exception of the last two. Support for learning
will be discussed in the next section. The presence and size of
the user community has not been discussed, as it is not clear
what role (if any) such a community plays in the veridicality of
the predictions made by the system; however, it may be relevant
to researchers for other reasons, particularly those trying to
learn the system.
In addition, many of the values on this table are likely to
change in the future. For example, the 6.0 version of ACT-R will
be available by the time this chapter appears (though there are
few major theoretical changes planned), and planning for a ma-
jor revision for version 7.0 is already underway.
It is difficult to classify the value an architecture has on a par-
ticular attribute as an advantage or a disadvantage, because what
constitutes an advantage for modeling one phenomenon may
be a disadvantage for modeling others. For example, consider
learning in Soar. Certainly, when attempting to model the im-
provement of users over time with a particular interface, Soar’s
learning mechanism is critical; however, there are many appli-
cations for which modeling learning is not critical, and Soar’s
pervasive learning feature occasionally causes undesired side ef-
fects that can make model construction more difficult.
EPIC ACT-R 5.0
Multiple-task performance Memory and problem-solving
Production cycle (parallel) Production cycle (serial)
Symbolic Both
None None
Yes Yes
No Yes
No Soon
No No
None Extensive tutorial materials, summer school
None Growing, primarily psychology
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 110
110 • BYRNE
THE FUTURE OF COGNITIVE
ARCHITECTURES IN HCI
Beyond the incremental development and application of archi-
tectures such as EPIC and ACT-R, what will the future hold for
cognitive architectures in HCI? What challenges are faced, and
what is the ultimate promise? Indeed, a number of important
limitations for cognitive architectures currently exist. There are
questions they cannot yet address, and it is hard to see how they
even would address some questions. Other limitations are more
pragmatic than in principle, but these are relevant as well.
First, there is a wide array of HCI problems that are simply
outside the scope of current cognitive architectures. Right now,
these architectures focus on cognition and performance, but
not on other aspects of HCI, such as user preference, boredom,
aesthetics, fun, and so on. Another important challenge, though
one that might be overcome, is that these architectures gener-
ally have not been applied to social situations, such as those en-
countered in groupware or online communities (Olson & Ol-
son, this volume; Preece, this volume). In principle, it is possible
to implement a model of social interaction in a cognitive archi-
tecture; however, the knowledge engineering problem here
would certainly be a difficult one. How does one characterize
and implement knowledge about social situations with enough
precision to be implemented in a production system? It may ul-
timately be possible to do so, but it is unlikely that this will hap-
pen anytime soon.
One problem that will never entirely be resolved, no matter
how diligent the modelers are, is the knowledge engineering
problem. Every model constructed using a cognitive architec-
ture still needs knowledge about how to use the interface and
what the tasks are. By integrating across models, the knowledge
engineering demands when entering a new domain may be re-
duced (the NTD is a good example), but they will never be elim-
inated. This requirement will persist even if an architecture was
to contain a perfect a theory of human learning—and there is
still considerable work to be done to meet that goal.
Another barrier to the more widespread use of cognitive ar-
chitectures in HCI is that the architectures themselves are large
and complex pieces of software, and (ironically) little work has
been done to make them usable or approachable for novices.
For example, “EPIC is not packaged in a ‘user-friendly’ manner;
full-fledged Lisp programming expertise is required to use the
simulation package, and there is no introductory tutorial or
user’s manual” (Kieras & Meyer, 1997, p. 399). The situation is
slightly better for ACT-R: tutorial materials, documentation, and
examples for ACT-R are available, and most years there is a two-
week “summer school” for those interested in learning ACT-R
(see http://act.psy.cmu.edu).
Another limiting factor is implementation. In order for a cog-
nitive architecture to accurately model interaction with an in-
terface, it must be able to communicate with that interface. Be-
cause most user interfaces are “closed” pieces software with no
built-in support for supplying a cognitive model with the infor-
mation it needs for perception (e.g., what is on the screen
where) or accepting input from a model, this creates a technical
problem. Somehow, the interface and the model must be con-
nected. An excellent summary of this problem can be found in
Ritter, Baxter, Jones, and Young (2000). A number of different
approaches have been taken. In general, the EPIC solution to
this problem has been to reimplement the interface to be mod-
eled in Lisp (more recent versions of EPIC require C), so the
model and the interface can communicate via direction function
calls. The ACT-R solution is not entirely dissimilar. In general,
ACT-R has only been applied to relatively new experiments or in-
terfaces that were initially implemented in Lisp, and thus ACT-
R and the interface can communicate via function calls. In or-
der to facilitate the construction of models and reduce the
modeler’s degrees of freedom in implementing a custom inter-
face strictly for use by a model, ACT-R does provide some abili-
ties for automatically managing this communication when the
interface is built with the native GUI builder for Macintosh Com-
mon Lisp under MacOS and Allegro Common Lisp under Win-
dows. If the interface is implemented this way, both human
users and the models can interact with the same interface.
The most intriguing development along this line, however, is
recent work by St. Amant and colleagues (St. Amant & Riedl,
2001; St. Amant, Horton, & Ritter, in press). They have imple-
mented a system called SegMan, which directly parses the
screen bitmap on Windows systems—that is, given a Windows
display—any Windows display—SegMan can parse it and iden-
tify things such as text, scroll bars, GUI widgets, and the like. It
also has facilities for simulating mouse and keyboard events.
This is an intriguing development, because in principle, it
should be possible to connect this to an architecture such as
EPIC or ACT-R, which would enable the architecture to poten-
tially work with any Windows application in its native form.
While a number of technical details would have to be worked
out, this has the potential of fulfilling one of the visions held by
many in the cognitive architecture community: a high-fidelity
“virtual user” that could potentially use any application or even
combination of applications. Besides providing a wide array of
new domains to researchers, this could be of real interest to
practitioners as well because this opens the door for at least
some degree of automated usability testing. This idea is not a
new one (e.g., Byrne, Wood, Sukaviriya, Foley, & Kieras, 1994;
St. Amant, 2000), but technical and scientific issues have pre-
cluded its adoption on even a limited scale. This would not elim-
inate the need for human usability testing (see Ritter & Young,
2001, for a clear discussion of this point) for some of the reasons
listed above, but it could significantly change usability engi-
neering practice in the long run.
The architectures themselves will continue to be updated
and applied to more tasks and interfaces. As mentioned, a new
version of ACT-R (version 6.0) is currently under development,
and this new version has definitely been influenced by issues
raised by HCI concerns. New applications of EPIC result in new
mechanisms (e.g., similarity-based decay in verbal working
memory storage; Kieras, Meyer, Mueller, & Seymour, 1999) and
new movement styles (e.g., click-and-point, Hornof & Kieras,
1999). Applications such as the World Wide Web are likely to
drive these models into more semantically rich domain areas,
and tasks that involve greater amounts of problem solving are
also likely candidates for future modeling.
Despite their limitations, this is a particularly exciting time to
be involved in research on cognitive architectures in HCI. There
is a good synergy between the two areas, as cognitive architec-
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 111
tures are certainly useful to HCI, so HCI is also useful for cognitive
architectures. HCI is a complex and rich yet still tractable domain,
which makes it an ideal candidate for testing cognitive architec-
tures. HCI tasks are more realistic and require more integrated ca-
pabilities than typical cognitive psychology laboratory experi-
ments, and thus cognitive architectures are the best theoretical
tools available from psychology. Theories like EPIC-Soar (Chong
& Laird, 1997) and ACT-R are the first complete psychological
models that go from perception to cognition to action in detail.
This is a significant advance and holds a great deal of promise.
References
Altmann, E. M. (2001). Near-term memory in programming: A simulation-
based analysis. International Journal of Human-Computer Studies,
54(2), 189–210.
Altmann, E. M., & John, B. E. (1999). Episodic indexing: A model of
memory for attention events. Cognitive Science, 23(2), 117–156.
Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ:
Erlbaum.
Anderson, J. R. (1993). Rules of the mind. Hillsdale, NJ: Erlbaum.
Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., &
Quin, Y. (2004). An integrated theory of the mind. Psychological
Review, 111, 1036–1060.
Anderson, J. R., Conrad, F. G., & Corbett, A. T. (1989). Skill acquisition
and the LISP tutor. Cognitive Science, 13(4), 467–505.
Anderson, J. R., & Lebiere, C. (1998). The atomic components of
thought. Mahwah, NJ: Erlbaum.
Anderson, J. R., Matessa, M., & Lebiere, C. (1997). ACT-R: A theory of
higher level cognition and its relation to visual attention. Human-
Computer Interaction, 12(4), 439–462.
Ballas, J. A., Heitmeyer, C. L., & Perez, M. A. (1992). Evaluating two
aspects of direct manipulation in advanced cockpits. Proceedings
of ACM CHI 92 Conference on Human Factors in Computing Sys-
tems (pp. 127–134). New York: ACM.
Blackmon, M. H., Kitajima, M., & Polson, P. G. (2003). Repairing usabil-
ity problems identified by the Cognitive Walkthrough for the Web.
In Human factors in computing systems: Proceedings of CHI 2003
(pp. 497–504). New York: ACM.
Blackmon, M. H., Kitajima, M., & Polson, P. G. (2005). Tool for accurately
predicting website navigation problems, non-problems, problem
severity, and effectiveness of repairs. In Human factors in comput-
ing systems: Proceedings of CHI 2005 (pp. 31–40). New York: ACM.
Blackmon, M. H., Polson, P. G., Kitajima, M., & Lewis, C. (2002). Cogni-
tive walkthrough for the web. In Human factors in computing sys-
tems: Proceedings of CHI 2002 (pp. 463–470). New York: ACM.
Bovair, S., Kieras, D. E., & Polson, P. G. (1990). The acquisition and per-
formance of text-editing skill: A cognitive complexity analysis.
Human-Computer Interaction, 5(1), 1–48.
Byrne, M. D. (2001). ACT-R/PM and menu selection: Applying a cognitive
architecture to HCI. International Journal of Human-Computer
Studies, 55, 41–84.
Byrne, M. D., & Bovair, S. (1997). A working memory model of a com-
mon procedural error. Cognitive Science, 21(1), 31–61.
Byrne, M. D., Wood, S. D., Sukaviriya, P. N., Foley, J. D., & Kieras, D. E.
(1994). Automating interface evaluation. ACM CHI’94 Conference on
Human Factors in Computing Systems (pp. 232–237). New York:
ACM Press.
Card, S. K., Moran, T. P., & Newell, A. (1983). The psychology of human-
computer interaction. Hillsdale, NJ: Erlbaum.
5. Cognitive Architecture • 111
The need for truly quantitative engineering models will only
grow as user interfaces propagate into more and more places
and more and more tasks. Cognitive architectures, which al-
ready have a firmly established niche in HCI, seem the most
promising road toward such models. Thus, as the architectures
expand their range of application and their fidelity to the hu-
mans they are attempting to model, this niche is likely to ex-
pand. HCI is an excellent domain for testing cognitive architec-
tures as well, so this has been, and will continue to be, a fruitful
two-way street.
Card, S. K., Pirolli, P., Van Der Wege, M., Morrison, J. B., Reeder, R. W.,
Schraedley, P. K., et al. (2001). Information scent as a driver of Web
behavior graphs: Results of a protocol analysis method for Web
usability. In Human factors in computing systems: Proceedings of
CHI 2001 (pp. 498–505). New York: ACM.
Chi, E. H., Rosien, A., Supattanasiri, G., Williams, A., Royer, C., Chow, C., et al.
(2003). The Bloodhound project: Automating discovery of web usabil-
ity issues using the Infoscent simulator. In Human factors in comput-
ing systems: Proceedings of CHI 2003 (pp. 505–512). New York: ACM.
Chong, R. S., & Laird, J. E. (1997). Identifying dual-task executive
process knowledge using EPIC-Soar. In M. Shafto & P. Langley (Eds.),
Proceedings of the Nineteenth Annual Conference of the Cognitive
Science Society (pp. 107–112). Hillsdale, NJ: Erlbaum.
Ehret, B. D. (1999). Learning where to look: The acquisition of location
knowledge in display-based interaction. Unpublished doctoral dis-
sertation, George Mason University, Fairfax, VA.
Franzke, M. (1994). Exploration, acquisition, and retention of skill with
display-based systems. Unpublished doctoral dissertation, University
of Colorado, Boulder.
Gray, W. D., John, B. E., & Atwood, M. E. (1993). Project Ernestine: Vali-
dating a GOMS analysis for predicting and explaining real-world task
performance. Human-Computer Interaction, 8(3), 237–309.
Gray, W. D., Young, R. M., & Kirschenbaum, S. S. (1997). Introduction
to this special issue on cognitive architectures and human-computer
interaction. Human-Computer Interaction, 12, 301–309.
Hornof, A., & Kieras, D. E. (1997). Cognitive modeling reveals menu
search is both random and systematic. Proceedings of ACM CHI 97
Conference on Human Factors in Computing Systems (pp. 107–
114). New York: ACM.
Hornof, A., & Kieras, D. (1999). Cognitive modeling demonstrates how
people use anticipated location knowledge of menu items. Pro-
ceedings of ACM CHI 99 Conference on Human Factors in Com-
puting Systems (pp. 410–417). New York: ACM.
Howes, A., & Young, R. M. (1996). Learning consistent, interactive, and
meaningful task-action mappings: A computational model. Cogni-
tive Science, 20(3), 301–356.
Howes, A., & Young, R. M. (1997). The role of cognitive architecture in
modeling the user: Soar’s learning mechanism. Human-Computer
Interaction, 12(4), 311–343.
Huguenard, B. R., Lerch, F. J., Junker, B. W., Patz, R. J., & Kass, R. E.
(1997). Working memory failure in phone-based interaction. ACM
Transactions on Computer-Human Interaction, 4, 67–102.
John, B. E., & Kieras, D. E. (1996). The GOMS family of user interface
analysis techniques: Comparison and contrast. ACM Transactions
on Computer-Human Interaction, 3, 320–351.
John, B. E., Prevas, K., Savucci, D. D., & Koedinger, K. (2004). Predictive
human performance modeling made easy. In Human factors in
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 112
112 • BYRNE
computing systems: Proceedings of CHI 2004 (pp. 455–462). New
York: ACM.
Jones, R. M., Laird, J. E., Nielsen, P. E., Coulter, K. J., Kenny, P., & Koss,
F. V. (1999). Automated intelligent pilots for combat flight simulation.
AI Magazine, 20(1), 27–41.
Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehen-
sion: Individual differences in working memory. Psychological
Review, 99(1), 122–149.
Kieras, D. E., & Bovair, S. (1986). The acquisition of procedures from
text: A production-system analysis of transfer of training. Journal of
Memory & Language, 25(5), 507–524.
Kieras, D. E., & Meyer, D. E. (1996). The EPIC architecture: Principles
of operation. Retrieved July 4, 1996, from ftp://ftp.eecs.umich
.edu/people/kieras/EPICarch.ps.
Kieras, D. E., & Meyer, D. E. (1997). An overview of the EPIC architec-
ture for cognition and performance with application to human-com-
puter interaction. Human-Computer Interaction, 12(4), 391–438.
Kieras, D. E., & Meyer, D. E. (2000). The role of cognitive task analysis
in the application of predictive models of human performance. In
J. M. Schraagen & S. F. Chipman (Eds.), Cognitive task analysis
(pp. 237–260). Mahwah, NJ: Erlbaum.
Kieras, D. E., Meyer, D. E., Mueller, S., & Seymour, T. (1999). Insights
into working memory from the perspective of the EPIC architecture
for modeling skilled perceptual-motor and cognitive human perfor-
mance. In A. Miyake & P. Shah (Eds.), Models of working memory:
Mechanisms of active maintenance and executive control (pp.
183–223). New York: Cambridge University Press.
Kieras, D. E., Meyer, D. E., & Ballas, J. A. (2001). Towards demystification
of direct manipulation: cognitive modeling charts the gulf of execu-
tion, Proceedings of ACM CHI 01 Conference on Human Factors
in Computing Systems (pp. 128–135). New York: ACM.
Kieras, D., & Polson, P. G. (1985). An approach to the formal analysis of
user complexity. International Journal of Man-Machine Studies,
22(4), 365–394.
Kieras, D. E., Wood, S. D., & Meyer, D. E. (1997). Predictive engineer-
ing models based on the EPIC architecture for multimodal high-
performance human-computer interaction task. Transactions on
Computer-Human Interaction, 4(3), 230–275.
Kintsch, W. (1998). Comprehension: A paradigm for cognition. New
York: Cambridge University Press.
Kitajima, M., & Polson, P. G. (1997). A comprehension-based model of
exploration. Human-Computer Interaction, 12(4), 345–389.
Kitajima, M., Blackmon, M. H., & Polson, P. G. (2000). A comprehension-
based model of web navigation and its application to web usability
analysis. In S. McDonald & Y. Waern & G. Cockton (Eds.), People and
Computers XIV-Usability or Else! (Proceedings of HCI 2000) (pp.
357–373). New York: Springer.
Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem:
The latent semantic analysis theory of acquisition, induction, and
representation of knowledge. Psychological Review, 104(2), 211–
240.
Lewis, R. L. (1993). An architecturally-based theory of human sentence
comprehension. Unpublished doctoral dissertation, University of
Michigan, Ann Arbor.
Meyer, D. E., & Kieras, D. E. (1997). A computational theory of executive
cognitive processes and multiple-task performance: I. Basic mecha-
nisms. Psychological Review, 104(1), 3–65.
Nelson, G., Lehman, J. F., & John, B. E. (1994). Integrating cognitive
capabilities in a real-time task. In A. Ram & K. Eiselt (Eds.), Proceed-
ings of the Sixteenth Annual Conference of the Cognitive Science
Society (pp. 353–358). Hillsdale, NJ: Erlbaum.
Newell, A. (1973). You can’t play 20 questions with nature and win: Pro-
jective comments on the papers of this symposium. In W. G. Chase
(Ed.), Visual information processing (pp. 283–308). New York: Aca-
demic Press.
Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Har-
vard University Press.
Newell, A., & Simon, H. A. (1963). GPS, a program that simulates human
thought. In E. A. Feigenbaum & J. Feldman (Eds.), Computers and
thought (pp. 279–293). Cambridge, MA: MIT Press.
Peebles, D., & Cheng, P. C. H. (2003). Modeling the effect of task and
graphical representation on response latency in a graph reading
task. Human Factors, 45, 28–45.
Pew, R. W., & Mavor, A. S. (Eds.). (1998). Modeling human and organi-
zational behavior: Application to military simulations. Washing-
ton, DC: National Academy Press.
Pirolli, P., & Card, S. (1999). Information foraging. Psychological Review,
106(4), 643–675.
Pirolli, P., & Fu, W. (2003). SNIF-ACT: A model of information foraging
on the world wide web. In P. Brusilovsky, A. Corbett, & F. de Rosis
(Eds.), Proceedings of the ninth international conference on user
modeling (pp. 45–54). Johnstown, PA: Springer-Verlag.
Polk, T. A., & Newell, A. (1995). Deduction as verbal reasoning. Psycho-
logical Review, 102(3), 533–566.
Polson, P. G., Lewis, C., Rieman, J., & Wharton, C. (1992). Cognitive
walkthroughs: A method for theory-based evaluation of user inter-
faces. International Journal of Man-Machine Studies, 36(5),
741–773.
Polson, P. G., Muncher, E., & Engelbeck, G. (1986). A test of a common
elements theory of transfer. Proceedings of ACM CHI’86 Conference
on Human Factors in Computing Systems (pp. 78–83). New York:
ACM.
Rieman, J., Young, R. M., & Howes, A. (1996). A dual-space model of iter-
atively deepening exploratory learning. International Journal of
Human-Computer Studies, 44(6), 743–775.
Ritter, F. E., Baxter, G. D., Jones, G., & Young, R. M. (2000). Cognitive
models as users. ACM Transactions on Computer-Human Interac-
tion, 7, 141–173.
Ritter, F. E., & Young, R. M. (2001). Embodied models as simulated
users: Introduction to this special issue on using cognitive models
to improve interface design. International Journal of Human-
Computer Studies, 55, 1–14.
Salthouse, T. A. (1988). Initiating the formalization of theories of cogni-
tive aging. Psychology & Aging, 3(1), 3–16.
Salvucci, D. D. (2001a). An integrated model of eye movements and
visual encoding. Cognitive Systems Research, 1(4), 201–220.
Salvucci, D. D. (2001b). Predicting the effects of in-car interface use on
driver performance: An integrated model approach. International
Journal of Human-Computer Studies, 55, 85–107.
Schoelles, M. J., & Gray, W. D. (2000). Argus Prime: Modeling emer-
gent microstrategies in a complex simulated task environment. In
N. Taatgen & J. Aasman (Eds.), Proceedings of the Third Interna-
tional Conference on Cognitive Modeling (pp. 260–270). Veenen-
dal, NL: Universal Press.
Simon, H. A. (1996). The sciences of the artificial (3rd ed.). Cambridge,
MA: MIT Press.
Singley, M. K., & Anderson, J. R. (1989). The transfer of cognitive skill.
Cambridge, MA: Harvard University Press.
St. Amant, R. (2000, Summer). Interface agents as surrogate users. Intel-
ligence magazine, 11(2), 29–38.
St. Amant, R., Horton, T. E., & Ritter, F. E. (in press). Model-based eval-
uation of expert cell phone menu interaction. ACM Transactions
on Human-Computer Interaction.
St. Amant, R., & Riedl, M. O. (2001). A perception/action substrate for
cognitive modeling in HCI. International Journal of Human-
Computer Studies, 55, 15–39.
ch05_8166_Sears/Jacko_LEA 7/13/07 9:22 PM Page 113

5. Cognitive Architecture • 113

St. Amant, R., Freed, A. R., & Ritter, F. E. (2005). Specifying ACT-R mod-
els of user interaction with a GOMS language. Cognitive Systems
Research, 6, 71–88.
Taatgen, N. A., & Lee, F. J. (2003). Production compilation: A simple
mechanism to model skill acquisition. Human Factors, 45, 61–76.
Tullis, T. S. (1983). The formatting of alphanumeric displays: A review
and analysis. Human Factors, 25(6), 657–682.
Wiesmeyer, M. (1992). An operator-based model of covert visual atten-
tion. Unpublished doctoral dissertation, University of Michigan, Ann
Arbor.
Young, R. M., Barnard, P., Simon, T., & Whittington, J. (1989). How would
your favorite user model cope with these scenarios? ACM SIGCHI
Bulletin, 20, 51–55.