Cognitive models: Understanding their critical role as explanatory and predictive

hypothesis generators in cognition research

Jen Schellinck (jennifer_schellinck@carleton.ca)
Institute of Cognitive Science, 1125 Colonel By Drive Ottawa
Ottawa, ON K1S 5B6
Richard Webster (richard.j.webster@googlemail.com)
Department of Biology, 1125 Colonel By Drive Ottawa
Ottawa, ON K1S 5B6


Keywords: model literacy, explanatory models, predictive
models, model validity, cognitive tools, scientific method

McClelland (2009) argues that models are
a key factor in the advancement of cognitive
science, and will only become more important as
computing power increases and models
themselves become increasingly powerful.
Nonetheless, the role that cognitive models play
in increasing our understanding of cognitive
systems can at times be viewed with some
uncertainty and even skepticism. The issue of
multiple realizability is a frequent concern (see for
example Edelstein-Keshet, 2001), underscored by
Marr's tri-level hypothesis (Marr 1982, Dawson
1998), which emphasizes that two systems that
are functionally similar may not be similar at
other levels. Issues relating to over- simplification
and over-fitting of models have also been raised
(see for example Ahn, Busemeyer, Wagenmakers
and Stout, 2008).
To address current issues with, and
critiques of, model use, and to support cognitive
scientists seeking to create better cognitive
models and use these models more effectively, we
think it is important to provide a more detailed
picture of how models fit into the scientific
process and, in doing so, offer a more concrete
explanation of the appropriate functionality of
models within science. As a further step towards
clarifying the role of model in scientific
investigation we have also identified two
categories of models- explanatory and predictive-
that can provide criteria for evaluating models and
model use. By choosing appropriate model types
and following these criteria, cognitive modellers
will create more powerful models and use them
with greater success.
With respect to the role of models in the
scientific process, first, as McClelland (2009)
notes, models are tools that scientists use
(Facilitation) to assist in theorizing about and
synthesizing existing knowledge. Second and, we
argue, most critically, acting in their role as
cognitive tools models may go beyond the
facilitation of synthesizing existing knowledge
and drive the generation of new hypotheses
(Formulation). Once generated, these hypotheses
can then be tested through experimentation
(Foundation). Thus, models play an important role
in all components of the scientific process and can
be beneficially used at each stage. At the same
time, different types of models are required to
support different types of theoretical and
experimental activity, so good choices of both
model type and construction methods are
important considerations.
Particularly within the context of
hypothesis generation, we argue that there are two
model aspects that determine the uses for which a
particular model is best suited: level of abstraction
and level of scientific validity. These two features
of models- abstraction and validity- can be
thought of as existing as two axes on a graph, with
one axis being a continuum from general to
specific and one axis being a continuum from
validated to hypothetical. Models can then be
thought of as falling somewhere within this space.
Models that fall onto different parts of this model
features space lend themselves to different goals-
in some cases providing explanatory power and in
others providing predictive power.
To understand why this is the case it is
important to understand, for each category, how
404
the interplay of these factors supports and
generates the relevant functionality of the model-
either explanatory or predictive. As a result of its
levels of abstraction and validation, each type of
model has different suitabilities and capabilities.
A model designed to generate broadly applicable
scientific laws relating to cognition, or to shed
light on the general underlying mechanisms of
cognition, may perform poorly if used to predict
the behavior of a particular system in a particular
circumstance. Conversely, a model designed to
explore the possible underlying mechanisms for a
particular cognitive phenomenon may not lend
itself to generating possible universal laws that are
applicable to many phenomena. If models are
used in ways that do not cater to their particular
strengths, poor conclusions may be drawn. Over
time the cumulative effect is that confidence in
the relevance of models to the given area of
research is diminished.
Having categorized models as either
explanatory or predictive based on their different
functions and methodological roles, it may further
be asked: What sorts of errors may occur when
using these different types of models? The first
point to note in this regard is that, irrespective of
their category, models may lead to inaccurate
results for two primary reasons: (i) through a
factual mis-identification of similarities between
the model and the target phenomenon, or (ii)
through reasoning errors during use of the model.
Using a model of a particular type in a
functionally inappropriate manner (using a
predictive model to understand a system or an
explanatory model to make specific predictions)
will almost necessarily result in the occurrence of
at least one of these types of errors.
In examining the types of errors more
closely, we can classify the types of modelling
errors into three main categories: model
construction errors, model validation errors and
model communication errors. In order to avoid
modelling errors in any of these areas, the key is
to provide explicit descriptions of a number of
critical aspects of the model and the target system.
Clearly describing the model and the target
system in these ways makes it easier for both the
creator and the user of the model to identify gaps,
assumptions and inconsistencies in either the
model or the understanding of the target system,
while at the same time making the model more
useable and validatable by people other than the
creators of the model.
Thus, while we believe that modelling in
cognitive science is not fundamentally different
from modelling in other sciences, we also believe
that the complexity of the system being studied
demands sophisticated modelling, rigorous
modelling practices and clear communication
strategies amongst model creators and users.
Consequently, if it is indeed the case that
cognitive scientists and other researchers are
being driven towards the use of complex models
out of necessity in order to gain a clearer
understanding of natural systems that are
themselves complex, the solution to modelling
critiques is not avoidance of complex and
sophisticated models but rather an understanding
of the role that models play in the investigation of
complex cognitive phenomena and, coming from
this, the proper use and construction of these
models.
References
Ahn, W.-Y., Busemeyer, J. R., Wagenmakers, E.-
J. & Stout, J. C. (2008). Comparison of Decision
Learning Models Using the Generalization
Criterion Method. Cognitive Science, 32, 1376
1402. doi: 10.1080/03640210802352992

Dawson, M. R. W. (1998). Understanding
Cognitive Science. Oxford, UK: Blackwell.

Edelstein-Keshet, L. (2001). Mathematical models
of swarming and social aggregation. The 2001
International Symposium on Nonlinear Theory
and its Applications, (NOLTA 2001).

Marr, David (1982). Vision. A Computational
Investigation into the Human Representation and
Processing of Visual Information. San Francisco,
CA: W.H. Freeman and Company.

McClelland, J. L. (2009). The Place of Modeling
in Cognitive Science. Topics in Cognitive Science,
1, 1138. doi: 10.1111/j.1756-8765.2008.01003.x
405