See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/305755437

Metacognition and learning in adulthood

Research · August 2016

DOI: 10.13140/RG.2.1.2231.0649

CITATIONS READS
8 5,227

1 author:

Theo Dawson
Lectica, Inc.
45 PUBLICATIONS   1,030 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

DiscoTest Initiative View project

Thinking complexity of national leaders View project

All content following this page was uploaded by Theo Dawson on 01 August 2016.

The user has requested enhancement of the downloaded file.

Developmental Testing Service, LLC

Metacognition and learning in

adulthood

Prepared in response to tasking from ODNI/CHCO/IC Leadership Development Office

1
by Dr. Theo L. Dawson , Developmental Testing Service, LLC

Saturday, August 23, 2008

35 South Park Terrace, Northampton, MA 01060 T «Phone» F «Phone» «Email» http:devtestservice.com

Developmental Testing Service, LLC

Metacognition and learning in

adulthood

Contents

Metacognition 2
Developmental Testing Service, LLC

Metacognition and learning in

adulthood 1

What is metacognition?
Metacognition is thinking about thinking. Metacognitive skills are usually
conceptualized as an interrelated set of competencies for learning and thinking,
and include many of the skills required for active learning, critical thinking,
reflective judgment, problem solving, and decision-making. Adults whose
metacognitive skills are well developed are better problem-solvers, decision makers and critical thinkers,
are more able and more motivated to learn, and are more likely to be able to regulate their emotions
(even in difficult situations), handle complexity, and cope with conflict. Although metacognitive skills, once
they are well-learned, can become habits of mind that are applied in a wide variety of contexts, it is
important for even the most advanced adult learners to “flex their cognitive muscles” by consciously
applying appropriate metacognitive skills to new knowledge and in new situations.

According to Flavell (1979), who coined the term, metacognition is a regulatory system that includes (a)
knowledge, (b) experiences, (c) goals, and (d) strategies. Metacognitive knowledge is stored knowledge
or beliefs about (1) oneself and others as cognitive agents, (2) tasks, (3) actions or strategies, and (4)
how all these interact to affect the outcome of any intellectual undertaking. Metacognitive experiences are
conscious cognitive or affective experiences that concern any aspect of an intellectual undertaking.
Knowledge is considered to be metacognitive (as opposed to simply cognitive) if it is used in a strategic
manner to meet a goal. According to Sternberg {, 1986 #14899) it is "figuring out how to do a particular
task or set of tasks, and then making sure that the task or set of tasks are done correctly" (p. 24).

Some of the concepts commonly associated with the metacognition literature are shown in Table 1.

1
Theo L. Dawson, Ph.D. (UC Berkeley, 1998), is a respected cognitive developmental psychologist. Her work focuses
on the description of learning sequences—the actual pathways through which people learn complex concepts and
skills—and the design of developmental assessments of these skills. She has worked closely with members of the IC
since 2002, when she, along with her colleague Dr. Kurt Fischer (Harvard Graduate School of Education), conducted
a study of the sequences through which critical thinking, leadership reasoning, and decision making skills develop in
adulthood. Since then, her work with the IC has involved assisting with the design of leadership curricula, evaluating
the extent to which curricula support leader development, and helping to define skill levels for a range of leadership
and analyst standards.

Metacognition 3
Developmental Testing Service, LLC

Table 1: Metacognition concepts

Concept
Metacognitive knowledge
about persons
Metacognitive knowledge
about tasks
Metacognitive knowledge
about strategies
Level of conscious aware-
ness of metacognitive
knowledge
Limits of metacognitive
knowledge
Duration of metacognitive
experiences
Occurrence of metacogni-
tive experiences
Effects of metacognitive
experiences
Memory-monitoring, self-
regulation, meta-reasoning,
consciousness/awareness,
and auto-
consciousness/self-
awareness
Transfer
Cognition
Ill-structured problems
Description
Includes a person’s beliefs about intra-individual differences, inter-individual differ-
ences, and universals of cognition
The information available to apply to a cognitive activity and an individual’s knowl-
edge about the task demands of a given situation
Awareness of and beliefs about available strategies
Retrieval and construction of metacognitive knowledge can be either conscious or
unconscious
Can be accurate or inaccurate, may not be activated when needed, may not have
much influence when it is activated, and may not have a beneficial effect when it is
influential
Can be momentary or lengthy, as when we are consciously grappling with a chal-
lenging problem
Most likely to occur when one is engaged in intentional, reflective intellectual activi-
ties such as problem-solving and learning
Can lead one to establish new goals and to revise or abandon old ones, can cause
one to add to one’s existing metacognitive knowledge base, and can activate
strategies that would otherwise have remained inactivated
Other terms for metacognition
Using a metacognitive skill learned in one context to solve a problem in another
context.
A general term for thinking, sometimes distinguished from metacognition, which is
thinking about thinking.
Ill-structured problems are open-ended, with unclear goals, multiple solutions, and
no “right” answers—like many of the problems we face in the workplace

Double loop learning Learning how the way one defines and solves problems can be a source of prob-
lems (Argyris, 1991).

Models of metacognition
The term metacognition is both a general term for “thinking about thinking,” and a
term used by a particular group of researchers to describe their field. There are
numerous models of metacognition, far too many to describe here. To complicate
things further, metacognition is a central component of several skill sets that are
central to education and the workplace, including (1) reflective judgment, (2) critical thinking, (3) decision
making, and (4) problem solving. (Some researchers argue that these are components of metacognition.)
Table 1 provides brief descriptions of some of the more prominent metacognitive models.

Metacognition 4
Developmental Testing Service, LLC

Table 2: Models of metacognition

Finding Implication Reference
Self regu- (J. G. Borkowski & Thorpe, 1994;
lated learning Como, 1986; Ghatala, 1986;
(lower levels) Schloemer & Brenan, 2006; Barry J.
Zimmerman, 1990)
Note: Watch out for applications of
this model that focus exclusively on
teaching students more effective
ways to memorize facts.

Self-regulated learners approach educational tasks with

confidence, diligence, and resourcefulness; are aware
when they know a fact or possess a skill and when they do
not; proactively seek out information when needed and
take the necessary steps to master it; find a way to suc-
ceed even when they encounter obstructions; view learn-
ing as a systematic and controllable process; accept re-
sponsibility for their achievement outcomes; and monitor
the effectiveness of their learning methods or strategies.
Self-regulated learning strategies include self-evaluation,
organization and transformation, goal setting and planning,
information seeking, record keeping, self-monitoring, envi-
ronmental structuring, giving self-consequences, rehears-
ing and memorizing, seeking social assistance, and re-
viewing. (paraphrased from Zimmerman, 1990) In addition
to metacognition, motivation and behavior are considered
to be components of self-regulated learning.

Metacognition 5
Developmental Testing Service, LLC

Finding Implication Reference

Critical think- The Foundation for Critical Thinking
ing (all lev- (http://criticalthinking.org)
els) This model includes standards of
thought (clarity, accuracy, rele-
vance, logicalness, breadth, preci-
sion, significance, completeness,
fairness, and depth), which are ap-
plied to the elements of thought
(shown on the left), resulting in the
development of intellectual virtues.
This is an iterative cycle, which is
repeated as reasoning increases in
complexity over the course of de-
velopment.
Critical thinking and creative thought
are linked in this model (Paul &
Elder, 2006).

Reflective The reflective judgment model is a very well-researched (Kitchener & Fischer, 1990; Kitch-
judgment model of the development of thinking about thinking. ener & King, 1990; P. K. Wood,
1997)
Double loop Argyris brings metacognition to the workplace by highlight- {Argyris, 1991 #14906; Argyris,
learning (up- ing the importance of becoming conscious of the conse- 1982 #14625;
per levels) quences of one’s actions. He uses real-life examples from
the workplace to demonstrate methods for teaching upper
level managers to think effectively about their thinking.

Metacognition 6
Developmental Testing Service, LLC
Metacognition research
Table 3: Metacognition and learning
Finding
Metacognitive skills develop.
Adults often fail to monitor their
thinking.
Students who have been taught
metacognitive (self-regulated learn-
ing) skills learn better than students
who have not been taught these
skills.
Students with good metacognitive
skills are better critical thinkers,
problem-solvers, or decision makers
than students who are not.
People whose thinking is more
complex tend to have better meta-
cognitive skills.
In most people, the development of
cognitive complexity progresses at
different rates in different knowledge
domains, depending upon experi-
ence and learning in particular do-
mains.
Cognitive development involves
both knowledge acquisition and
(largely unconscious) knowledge
structuring. (If there is no knowledge
to organize, then there is no devel-
opment.)
Content knowledge is more easily
accessed in real-world situations if
students learn how new concepts
and procedures can function as
tools for solving relevant problems.
Both content knowledge and meta-
cognitive skills are essential for
learning.
Metacognitive training can increase
students’ self-confidence and sense
of personal responsibility for their
own development.
Metacognition
Implication
They can be learned.
Adults can benefit from metacogni-
tive training.
It is possible to produce better
learners by teaching metacognitive
skills.
It is possible to produce better criti-
cal thinkers, problem-solvers, and
decision makers by teaching meta-
cognitive skills.
Metacognition and cognitive com-
plexity are related.
Since cognitive complexity and
metacognition are related, we might
expect metacognition to be more
advanced in more developed knowl-
edge domains.
Because metacognitive skills in-
volve the conscious structuring of
knowledge, they are likely to be
more developed in areas of greater
knowledge.
Learning environments should in-
clude opportunities for students to
reflectively apply new concepts and
tools in real-world contexts.
Learning may be enhanced when
instruction (1) provides explicit con-
tent knowledge while (2) asking
students to use metacognitive skills
to operate on that knowledge.
Increased self-confidence and a
sense of increased personal re-
sponsibility may provide motivation
for learning.
Reference
(Baer, Hollenstein, Hofstetter,
Fuchs, & Reber-Wyss, 1994;
Brown, Bransford, Ferrara, & Cam-
pione, 1983; John H. Flavell, 1979;
Ruth Garner & Alexander, 1989)
(J. H. Flavell, 1981; Ruth Garner &
Alexander, 1989; Glenberg, Wilkin-
son, & Epstein, 1982)
(J. Borkowski, Carr, & Pressely,
1987; Bransford, Sherwood, Vye, &
Rieser, 1986; Carr, Kurtz, Schnei-
der, Turner, & Borkowski, 1989; R.
Garner, 1990; Hascher & Oser,
1995; Mace, Belfiore, & Hutchinson,
2001; Pressley & Ghatala, 1990; B.
J. Zimmerman & Schunk, 2001)
(Bransford et al., 1986; Ewell-
Kumar, 1999; Heath, 1983)
(Swanson & Hill, 1993; Vukman,
2005)
(Fischer & Pruyne; Fischer, Yan, &
Stewart)
(Bransford et al., 1986)
(Glaser, 1984)
(Bransford et al., 1986; Perkins,
1987)
(McCombs & Marzano, 1990;
Schunk, 1990)
7
Developmental Testing Service, LLC
Finding
Metacognitive training can increase
students’ motivation to learn.
Moving adult learners to a point of
acknowledging that old routines no
longer work as well as new,
instructed ones takes time and
many demonstrations of the superi-
ority of the new routines.
“Intelligent novices” can use general
metacognitive skills to figure out
how to obtain knowledge in an un-
familiar domain.
Students receiving intelligent novice
feedback acquire a deeper concep-
tual understanding of domain princi-
ples and demonstrate better transfer
and retention of skills over time than
students who do not receive such
feedback.
When students perceive an empha-
sis on mastery goals in their class-
room, they report using more meta-
cognitive learning strategies.
Use of concept maps helped adult
students develop thinking skills,
promoted growth in understanding
their learning processes, and fos-
tered understanding of knowledge
construction.
Repeated experiences of dyadic
discussions within the classroom
improved reasoning skills (over con-
trols).
Informal learning is enhanced in
managers who employ a wide range
of metacognitive strategies.
Students in problem based learning
classrooms have been found to
have higher levels of intrinsic goal
orientation, task value, use of elabo-
ration learning strategies, critical
thinking, metacognitive self-
regulation, effort regulation, and/or
peer learning compared with con-
trol-group students.
Metacognition
Implication Reference
Training in metacognitive skills may (Bandura, 1986; Hofer & Yu, 2003;
enhance students’ sense of self Sperling, Howard, Staley, & DuBois,
efficacy, thus increasing their moti- 2004)
vation to learn.
Metacognitive strategies should be (Ruth Garner & Alexander, 1989)
embedded in assignments and
classroom activities across the cur-
riculum at every level of instruction
Once adults have gained expertise (Bruer, 1993; Mathan & Koedinger,
and learned how to use a range of 2005); Garner, 1989 #14918}
metacognitive skills in one domain,
they can use some of their meta-
cognitive skills to more rapidly learn
in another domain.
Teachers or “intelligent tutors” can (Mathan & Koedinger, 2005)
support the use of existing meta-
cognitive strategies in new knowl-
edge areas by providing feedback
that reminds students to employ
metacognitive strategies they have
used in familiar knowledge areas.
Classrooms in which “covering the (Ames & AfIher, 1988)
content” is emphasized over under-
standing can deprive students of the
opportunity to learn and master
learning skills.
Concept mapping, used well, is a (Daley, 2002)
useful metacognitive skill.
Active engagement in thinking (Kuhn, Shaw, & Felton, 1997)
about a topic enhances the quality
of reasoning about that topic.
Training in the use of metacognitive (Enos, Kehrhahn, & Bell, 2003)
strategies may increase informal
learning in less metacognitively so-
phisticated managers.
Problem based learning environ- (Sungur & Tekkaya, 2006)
ments may enhance metacognitive
skills relative to conventional in-
structional environments.
8
Developmental Testing Service, LLC

Finding Implication Reference

Gifted learners have been found to
employ fewer metacognitive strate-
gies than less gifted students.
Gifted learners, because they learn (Dresel & Haugwitz, 2005)
easily, may not need to employ
metacognitive strategies to excel.
This could result in reasoning defi-
cits in later life.

Table 4: Metacognition and critical thinking

Concept Resources
Critical thinking is the ability to (1) identify and formulate important questions and The Foundation for Critical
problems; (2) gather and assess information; (3) test proposed conclusions Thinking
against relevant criteria and standards; (4) think within alternative systems of (http://criticalthinking.org)
thought, assessing their assumptions, implications and practical consequences;
and (5) communicate effectively, without appealing to logical fallacies or manipu-
lating others.

Table 5: Metacognition and reflective judgment

Description
Reflective judgment is metacogni-
tion.
Beliefs about learning significantly
impact the quality of learning strate-
gies and learning outcomes in gen-
eral.
Implications
Students whose reflective judgment skills
are more developed are likely to be better
learners.
Source
(Hofer, 2004)
(B. K. Hofer, 1999; Barbara K.
Hofer, 2000; Paulsen & Feld-
man, 2005; Schommer, 1990,
1993; Schommer, Crouse, &
Rhodes, 1992)(Bråten &
Stromso, 2005; Muis, 2007; P.
Wood & Kardash, 2002)

Metacognition 9
Developmental Testing Service, LLC

Figure 1: Model of metacognitive skills and conditions for their development and use

Metacognition 10
Developmental Testing Service, LLC

Table 6: Metacognition and problem solving

Finding Implications Source
Effective problem solving depends Students whose reasoning skills are more de- Bransford, 1986 #6747;
on the nature and organization of veloped are likely to be better learners. Chi, 1988 #982}
the knowledge available to individu-
als.
Psychological biases interfere with Psychological biases may be ameliorated (Duchon, Ashmos, &
problem-solving through metacognition Dunegan, 1991)

Table 7: Metacognition and psychological biases

Unconscious knowledge structuring takes place every time we learn something. These natural structuring
processes have limitations that (1) are ineffective in some situations or (2) produce cognitive biases.
Metacognition can be employed to counter these biases.

Note: Although it is clearly part of the subtext of much research in metacognition (especially critical
thinking), the subject of psychological bias is rarely raised explicitly.

Bias Implications Source

Overestimate People tend to overestimate the depth of their explanatory knowl- (Mills & Keil, 2004)
explanatory edge (how well they understand concepts), which can produce deci-
knowledge sion errors.

Table 8: Metacognition and mindfulness

Finding or comment Source
“Mindfulness begins by bringing awareness to current experience—observing and at- (Bishop et al., 2004)
tending to the changing field of thoughts, feelings, and sensations from moment to
moment—by regulating the focus of attention. This leads to a feeling of being very alert
to what is occurring in the here-and-now. It is often described as a feeling of being fully
present and alive in the moment.” (p. 232) “Mindfulness is further defined by an orienta-
tion to experience that …[involves] making a commitment to maintain an attitude of
curiosity about where the mind wanders.” (p. 233)
Mindfulness can be thought of as creating an optimally receptive state for new learning (Garland, 2007; Tho-
and experience, increasing the likelihood that appropriate metacognitive skils will be mas, 2006)
selected and employed.
Mindfulness practice requires the activation of metacognitive knowledge, monitoring, (Wells, 2005)
and control.

Metacognition 11
Developmental Testing Service, LLC

Figure 2: Metacognitive skills are an important component of interpersonal intelligence

Metacognition 12
Developmental Testing Service, LLC

Teaching and learning metacognition

Figure 3: Learning model for the workplace, emphasizing motivation, assessment, and metacognition (adapted from
http://www.bbi.school.nz/philosophy/learningtolearn.html)

Table 9: Teaching and learning metacognitive skills: The research

Finding Reference
Metacognitive skills can be taught. (J. Borkowski et al.,
1987; Bransford et al.,
1986; R. Garner,
1990; Hascher & Oser,
1995)
Metacognitive skills learned in one context are not automatically transferred to another (Ericsson, Chase, &
context. Faloon, 1980)

Metacognitive skills are learned and applied more effectively in supported active learn-
ing contexts than in direct instruction contexts.

An application of metacognition to diversity issues. Could be useful as a starting place (Byars-Winston &
for discussion. Fouad, 2006)

The role of formative assessment in self-regulated learning. (Nicol & MacFarlane-

Dick, 2006)

Metacognition 13
Developmental Testing Service, LLC

Table 10: Teaching and learning metacognitive skills: Activities

Activity Description Resource
Identifying cogni- When students monitor their learning, they can become aware of (Nickerson, Perkins, &
tive errors potential problems, including errors in encoding, operations, and Smith, 1985)
goals. Errors in encoding include missing important data or not
separating relevant from irrelevant data. Errors in operations in-
clude failing to select the right sub-skills to apply or failing to divide
a task into subparts. For example, some math students will jump
right to what they think is the final calculation to get the desired
answer. Errors in goal seeking include misrepresenting the task and
not understanding the criteria to apply. Problems with cognitive load
include being unable to handle the number of sub-skills necessary
to do a task, or not having enough automatic, internalized sub-
skills.
A good way to discover what kind of errors managers are making in
their thinking processes is to have them unpack their thinking by
explaining, step by step, how they are approaching a given task.
This not only allows the instructor to diagnose possible errors, it
provides managers with an opportunity to describe their thinking
processes, which, by itself, develops their metacognitive abilities.
Strategic learning Working in groups, have students generate questions about the (Simpson & Nist, 2000)
content being learned and attempt to answer them.
Have students produce written or verbal summaries of course texts.
Require students to create examples, make analogies, and explain
relationships between concepts. Then, engage students in the use
of organizing strategies like concept maps, network representa-
tions, and other graphic organizers.
Self-regulatory These questions are designed to follow instruction on a particular (McInerney, McInerney,
metacognitive topic and precede instruction on the next topic: (a) "What did I learn & Marsh, 1997)
questions about this topic?" (monitoring) (b) "With what did I have difficulty?"
(monitoring) (c) "What types of things can I do to deal with this diffi-
culty?" (problem solving-planning) (d) "What specific action(s) am I
going to take this week to solve any difficulties?" (planning)
Before assigning this task for the first time, instructors should work
through an example with the group.

Metacognition 14
Developmental Testing Service, LLC

Activity Description Resource

Cromley summa- Skills need to be taught in the context in which they will be used. (Cromley, 2000)
rizes the research For example, if students are learning to lead in a high security situa-
findings, then tion, they need to practice confronting the kinds of issues that come
translates them up in high security situations, not just "general" skills.
into practical ac- Reading skills are subject-specific—understanding what you read in
tivities for the a technical report does not guarantee that you will read well in an-
classroom. This is other subject area.
an excellent re-
source for adult Problem-solving skills in one subject area are different from those in
educators. other areas. Problem-solving skills need to be taught separately for
each subject.
Since problem-solving skills do not automatically transfer from one
subject area to another, instructors need to show students how to
transfer these skills and give them lots of practice.
Students need more and better mental models of the world in order
to learn and master new information and skills.
Thinking skills, such as inferring unstated facts, need to be taught
explicitly in the classroom; they do not develop on their own (except
in a very few students). These strategies need to be practiced over
and over again.
Most adult learners have a very limited number of strategies for
understanding new material or solving problems. Teaching them
more strategies can help them learn much better.
Learning lasts when the student understands the material, not just
memorizes it.
Information needs to be presented in small chunks so that working
memory can process it.
Students need immediate practice to move information from work-
ing memory to long-term memory.
It is impossible to remember without associating new information
with what you already know.
Authors provide (From the abstract) This paper proposes an expanded conception (Holton & Clarke, 2006)
an overview of of scaffolding with four key elements:
scaffolding and its (i) scaffolding agency – expert, reciprocal, and self-scaffolding;
effects on learn-
ing. (ii) scaffolding domain – conceptual and heuristic scaffolding;
(iii) the identification of self-scaffolding with metacognition; and
(iv) the identification of six zones of scaffolding activity; each zone
distinguished by the matter under construction and the relative posi-
tioning of the participant(s) in the act of scaffolding.

Metacognition 15
Developmental Testing Service, LLC

Figure 4: Model of formative assessment and self-regulated learning

Metacognition 16
Developmental Testing Service, LLC

Table 12: Metacognition fads

Fad Claim Comments Source

Learning Different people have Adjusting teaching style to accommo- (Cromley, 2000; Curry, 1990;
styles different learning styles date different learning styles has not Stahl, 2002)
(that can be measured been shown to impact learning. Moreo-
with survey-type instru- ver, catering to learning preferences
ments) and these have may limit versatility, having a negative
an impact on learning. effect on learning over time.

Emotional Bringing intelligence to
intelligence bear upon emotion.
“Much of the data necessary for demon- {Landy, 2005
strating the unique association between #14980}
EI and work-related behavior appears to
reside in proprietary databases, pre-
venting rigorous tests of the measure-
ment devices or of their unique predic-
tive value. For those reasons, any
claims for the value of EI in the work
setting cannot be made under the scien-
tific mantle ” (p. 411)

Resources

Table 11: Metacognition curricula

Resource Description Level Resources
IDEAL problem The IDEAL problem solving system is “based on evidence Pre- (Bransford, Hay-
solving: Identify, that successful problem-solvers actively attempt to (a) management nes, Stein, Lin, &
Define, Explore, identify problems that others may have overlooked; (b) NashvilleRead,
Act, and Look, develop at least two sets of contrasting goals for any prob- 1998)
and Learn. lem and define them explicitly; (c) explore strategies and
continually evaluate their relevance to their goals; (d) an-
ticipate the effects of strategies before acting on them; and
(e) look at the effects of their efforts and learn from them.”
(p. 3)
TV411—a na- Focuses, in part on helping adults manage their own learn- Pre- (D'Amico &
tional television ing using metacognitive skills. management Capehart, 2001)
series aimed at
adult learners

Curricula to en- A handbook offering a number of activities designed for Pre- (Carman &
hance students adult learners. Many of these are appropriate for pre- management Askov, 1994)
higher order managers, especially if they are contextualized (free from
cognitive skills Eric).
Cognitive Strat- An instructional approach that has been shown to enhance Pre- (Scheid, 1993)
egy Instruction learning by emphasizing the development of thinking skills management
(CSI) and processes. The objective is to enable all students to
become more strategic, self-reliant, flexible, and productive
learners.
Using critical A useful resource for students of critical thinking in the Pre- http://www.unisa
thinking to gain workplace. management net.unisa.edu.au
knowledge and /Resources/nursi
understanding ng/Critical%20thi
nking/Critical%2
0thinking.htm

Metacognition 17
Developmental Testing Service, LLC

Resource Description Level Resources

Designing meta- Several research-based suggestions for creating learning (Lin, 2001)
cognitive activi- activities with a strong metacognitive component
ties

Table 12: Metacognition measures

Resource Description Resources
Lectical Reflective An online assessment of reflective judgment skill (for man- http://devtestservice.com
Judgment Assessment agers). Provides reliable developmental scores in ¼ level
(LRJA01) increments, as well as personalized feedback, including
developmentally informed learning suggestions. Intended for
personal use, coaching, and course evaluations.
Reasoning About Cur- The RCI is an online assessment of the capacity to recog- http://www.umich.edu/~re
rent Issues Test (RCI) nize and endorse statements that reflect the attributes of f-
reflective thinking. It is not an assessment of reflective judg/reasoningaboutcurre
judgment skill, and it is not intended as an assessment of ntissuestest.html
individuals, but can be used to assess group trends.
International Critical An assessment of students’ knowledge about critical think- http://www.criticalthinking.
Thinking Basic Con- ing concepts—the extent to which they have learned these org/courses/critical_think
cepts and Understand- concepts as they are presented in Elder and Paul’s model. It g_test1.cfm
ing Online Test is not an assessment of critical thinking ability. Reliability
information is not provided.
International Critical A pen and paper assessment of critical thinking skill that can http://www.criticalthinking.
Thinking Test be adapted to any subject area. Scoring is done by instruc- org/assessment//ICAT-
tors, and is based on rubrics. info.cfm
Motivated Strategies Based on the idea of self-regulated learning (SRL), this self- (Pintrich, Smith, Garcia, &
for Learning Question- report instrument is “designed to assess motivation and use Mckeachie, 1993)
naire (MSLQ) of learning strategies by college students. The motivation
scales tap into three broad areas: (1) value (intrinsic and
extrinsic goal orientation, task value), (2) expectancy (con-
trol beliefs about learning, self-efficacy); and (3) affect (test
anxiety). The learning strategies section is comprised of
nine scales which can be distinguished as cognitive, meta-
cognitive, and resource management strategies. The cogni-
tive strategies scales include (a) rehearsal, (b) elaboration,
(c) organization, and (d) critical thinking. Metacognitive
strategies are assessed by one large scale that includes
planning, monitoring, and regulating strategies. Resource
management strategies include (a) managing time and
study environment; (b) effort management, (c) peer learning,
and (d) help-seeking.” (abstract)

Metacognition 18
Developmental Testing Service, LLC

Resource Description Resources

Structure of Observed Method for evaluating learning/reasoning outcomes that (Biggs & Collis, 1982)
Learning Outcomes involves 5 levels of learning. Is often used to scaffold the
(SOLO) development of scoring rubrics. Rubrics are high inference
and inter-rater agreement is difficult to maintain. Nonethe-
less, when inter-rater agreement is properly controlled, the
rubrics are reliable enough for course evaluations (not usu-
ally for individual evaluation).
1. Pre-structural
Learners acquire bits of unconnected information, which
have no organization and make no sense.
2. Uni-structural
Learners make simple and obvious connections, but show
little evidence that their significance has been grasped.
3. Multi-structural
Learners make a number of connections, but meta-
connections between them are missed, as is their signifi-
cance for the whole.
4. Relational
Learners appreciate the significance of the parts in relation
to the whole.
5. Extended abstract
Learners make connections both within and beyond the sub-
ject area, showing they are able to generalize and transfer
the principles and ideas.

Metacognition 19
Developmental Testing Service, LLC

References
Ames, C., & AfIher, J. ( 1988). Achievement goals in the classroom: Students' learning strategies and motivation
processes. Journal of Educational Psychology., 80, 260-267.
Argyris, C. (1991). Teaching smart people how to learn. Harvard Business Review, May-June.
Baer, M., Hollenstein, A., Hofstetter, M., Fuchs, M., & Reber-Wyss, M. (1994). How d expert and novice writers differ
in their knowledge of the writing process and its regulation (metacognition) from each other, and what are the
differences in metacognitive knowledge between writers of different ages? Paper presented at the Annual
Meeting of the American Educational Research Association.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall, Englewood
Cliffs, NJ.
Biggs, J., & Collis, K. (1982). Evaluating the quality of learning: The SOLO taxonomy (structure of the observed
learning outcome). New York: Academic Press.
Bishop, S. R., Lau, M., Shauna, S., Carlson, L., Anderson, N. D., Carmody, J., et al. (2004). Mindfulness: A proposed
operational definition. Clinical Psychology: Science and Practice, 7(3), 230-241.
Borkowski, J., Carr, M., & Pressely, M. (1987). "Spontaneous" strategy use: Perspectives from metacognitive theory.
Intelligence, 11, 61-75.
Borkowski, J. G., & Thorpe, P. K. (1994). Self-regulation and motivation: A life-span perspective on
underachievement at any point during the life span (B. P. D & A. E. L, Trans.). In B. J. Z. Dale H. Schunk (Ed.),
Self-regulation of learning and performance: Issues and educational applications. (pp. 45-73). U Notre Dame,
Notre Dame, IN, US: Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US.
Bransford, J. D., Haynes, A. F., Stein, B. S., Lin, X., & NashvilleRead, N. (1998). The IDEAL workplace: Strategies for
improving learning, problem solving, and creativity.
Bransford, J. D., Sherwood, R., Vye, N. J., & Rieser, J. (1986). Teaching thinking and problem solving. American
Psychologist, 41(10), 1078-1089.
Bråten, I., & Stromso, H. (2005). The relationship between epistemological beliefs, implicit theories of intelligence,
and self-regulated learning among Norwegian postsecondary students. British Journal of Educational
Psychology, 75, 539–565.
Brown, A. L., Bransford, J. D., Ferrara, R. A., & Campione, J. C. (1983). Learning, remembering, and understanding.
In J. H. Flavell & E. M. Markman (Eds.), Carmichael's manual of child psychology (Vol. 1, pp. 77-166 ). New
York: Wiley.
Bruer, J. (1993). Schools for Thought: A science of learning in the classroom. Cambridge: MIT Press.
Byars-Winston, A. M., & Fouad, N. A. (2006). Metacognition and multicultural competence: Expanding the culturally
appropriate career counseling model. Career Development Quarterly, 54, 187-201.
Carman, P. S., & Askov, E. N. (1994). Development of a curriculum to enhance adult learners higher order skills.
University Park, PA: Pennsylvania State Universityo. Document Number)
Carr, M., Kurtz, B. E., Schneider, W., Turner, L. A., & Borkowski, J. G. (1989). Strategy acquisition and transfer
among German and American children: Environmental influences on metacognitive development. Developmental
Psychology, 25, 765-771.
Como, L. (1986). The metacognitive control components of self-regulated learning. Contemporary Educational
Psychology, 11, 333-346.
Cromley, J. (2000). Learning to think: Learning to learn. Washington, DC: National Institute for Literacyo. Document
Number)
Curry, L. (1990). One critique of the research on learning styles. Educational Leadership, 48, 50-56.
D'Amico, D., & Capehart, M. A. (2001). Letting Learners Lead: Theories of Adult Learning and TV411. Focus on
Basics, 5(B), 35-41.
Daley, B. J. (2002). Facilitating learning with adult students through concept mapping. Journal of Continuing Higher
Education, 50(1), 21-32.
Dresel, M., & Haugwitz, M. (2005). The relationship between cognitive abilities and self-regulated learning: Evidence
for interactions with academic self-concept and gender. High ability studies, 16(2), 201-218.
Duchon, D., Ashmos, D., & Dunegan, K. J. (1991). Avoid decision making disaster by considering psychological bias.
Review of Business, 13.
Enos, M. D., Kehrhahn, M. T., & Bell, A. (2003). Informal learning and the transfer of learning: How managers
develop proficiency. Human Resource Development Quarterly, 14(4), 369-.
Ericsson, K., Chase, W., & Faloon, S. (1980). Acquisition of a memory skill. Science and Engineering Ethics, 208,
1181-1182.
Ewell- Kumar, A. (1999). The influence of metacognition on managerial hiring decision making: Implications for
management development Dissertation Abstracts International Section A: Humanities and Social Sciences,
59(10-A).
Fischer, K. W., & Pruyne, E. Reflective thinking in adulthood: Development, variation, and consolidation. In J. Demick
(Ed.), Handbook of Adult Development.

Metacognition 20
Developmental Testing Service, LLC

Fischer, K. W., Yan, Z., & Stewart, J. B. Adult cognitive development: Dynamics in the developmental web. In J.
Valsiner & K. Connoly (Eds.), Handbook of developmental psychology (pp. 491-516). Thousand Oaks, CA: Sage.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry.
American Psychologist, 34(10), 906-911.
Flavell, J. H. (1981). Cognitive monitoring. In W. P. Dickson (Ed.), Children's oral communication skills (pp. 35-60).
New York: Academic.
Garland, E. L. (2007). The meaning of mindfulness: A second-order cybernetics of stress, metacognition, and coping.
Complementary Health Practice Review, 12(1), 15-30.
Garner, R. (1990). When children and adults do not use learning strategies: Toward a theory of settings. Review of
Educational Research, 60, 517-529.
Garner, R., & Alexander, P. A. (1989). Metacognition: Answered and unanswered questions. Educational
Psychologist, 24(2), 143-158.
Ghatala, E. S. (1986). Strategy-monitoring training enables young learners to select effective strategies. Educational
Psychologist, 21, 43-54.
Glaser, R. (1984). Education and thinking: The role of knowledge. American Psychologist, 39, 93-104.
Glenberg, A. M., Wilkinson, A. C., & Epstein, W. (1982). The illusion of knowing: Failure in the self-assessment of
comprehension. Memory & Cognition, 10, 597-602.
Hascher, T. A., & Oser, F. (1995). Promoting autonomy in the workplace--A cognitive-developmental intervention.
Paper presented at the Annual Meeing of the American Educational Research Association.
Heath, S. B. (1983). Ways with words: Language, life, and work in communities and classrooms. Cambridge, MA:
Cambridge University Press.
Hofer, B. K. (2004). Epistemological understanding as a metacognitive process: Thinking aloud during online
searching. Educational Psychologist, 39(1), 43-55.
Hofer, B. K., & Yu, S. L. (2003). Teaching self-reglated learning through a "learning to learn" course. Teaching in
Psychology, 30(1), 30-33.
Holton, D., & Clarke, D. (2006). Scaffolding and metacognition. International Journal of Mathematical Education in
Science and Technology,, 37(2), 127-143.
Kitchener, K. S., & Fischer, K. W. (1990). A skill approach to the development of reflective thinking. Contributions to
Human Development, 21, 48-62.
Kitchener, K. S., & King, P. M. (1990). The reflective judgment model: ten years of research. In M. L. Commons, C.
Armon, L. Kohlberg, F. A. Richards, T. A. Grotzer & J. D. Sinnott (Eds.), Adult development (Vol. 2, pp. 62-78).
New York: Praeger.
Kuhn, D., Shaw, V., & Felton, M. (1997). Effects of dyadic interaction on argumentative reasoning. Cognition and
Instruction, 15(3), 287-315.
Lin, X. (2001). Designing metagognitive activities. Educational Technology, Research, & Development, 49, 23-40.
Mace, F. C., Belfiore, P. J., & Hutchinson, J. M. (2001). Operant theory and research on self regulation. In B. J.
Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement (pp. 39-65). Mahwah,
NJ: Eribaum.
Mathan, S., & Koedinger, K. R. (2005). Fostering the intelligent novice: Learning from errors with metacognitive
tutoring. Educational Psychologist, 40(4), 257-265.
McCombs, B. L., & Marzano, R. J. (1990). Putting the self into self-regulating learning: The self as agent in integrating
will and skill. Educational Psychologist, 25(51-69).
McInerney, V., McInerney, D. M., & Marsh, H. W. (1997). Effects of metacognitive strategy training within a
cooperative group learning context on computer achievement and anxiety: An aptitude treatment interaction
study. Journal of Educational Psychology., 89(4), 686-695.
Mills, C. M., & Keil, F. C. (2004). Knowing the limits of one's understanding: The development of an awareness of an
illusion of explanatory depth. Journal of Experimental Child Psychology, 87, 1-32.
Muis, K. R. (2007). The role of epistemic beliefs in self-regulated learning. Educational Psychologist, 42(3), 173-190.
Nickerson, R. S., Perkins, D. N., & Smith, E. E. (1985). The teaching of thinking. Hillsdale, NJ: Erlbaum.
Nicol, D. J., & MacFarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven
principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.
Paul, R., & Elder, L. (2006). Critical thinking: The nature of critical and creative thought. Journal of Developmental
Education, 30(2), 34-35.
Perkins, D. (1987). Knowledge as design: Teaching thinking through content. In J. B. R. Sternberg (Ed.), Teaching
thining skills: Theory and practice. New York: W. H. Freeman.
Pintrich, P. R., Smith, D. A. F., Garcia, T., & Mckeachie, W. J. (1993). Reliability and predictive validity of the
Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801-
813.
Pressley, M., & Ghatala, E. S. (1990). Self-regulated learning: Monitoring learning from text. Educational
Psychologist, 25, 9-33.
Schloemer, P., & Brenan, K. (2006). Journal of Education for Business, 81-87.

Metacognition 21
 Developmental Testing Service, LLC

Schunk, D. H. (1990). Goal setting and self-efficacy during self-regulated learning. Educational Psychologist, 25, 71-
86.
Simpson, M. L., & Nist, S. L. (2000). An update on strategic learning: It’s more than textbook reading strategies.
Journal of Adolescent and Adult Literacy, 43, 528-541.
Sperling, R. A., Howard, B. C., Staley, R., & DuBois, N. (2004). Metaognition and self-regulated learning constructs.
Educational Research and Evaluation, 10(2), 117-139.
Stahl, S. A. (2002). Different strokes for different folks? In L. Abbedutto (Ed.), Taking sides: Clashing on controversial
issuesin educational psychology (pp. 98-107). Guilford, CT: McGraw-Hill.
Sungur, S., & Tekkaya, C. (2006). Effects of problem-based learning and traditional instruction on self-regulated
learning. Journal of Educational Research, 99(5), 307-317.
Swanson, L., & Hill, G. (1993). Metacognitive aspects of moral reasoning and behavior. Adolescence, 28, 711-735.
Thomas, D. C. (2006). Domain and development of cultural intelligence: The importance of mindfulness. Group &
Organization Management, 31(1), 78-99.
Vukman, K. B. (2005). Developmental differences in metacognition and their connections with cognitive development
in adulthood. Journal of Adult Development, 12(4), 211-221.
Wells, A. (2005). Detached mindfulness In cognitive therapy: A metacognitive analysis and ten techniques. Journal of
Rational-Emotive and Cognitive-Behavior Therapy, 23(4), 337-355.
Wood, P., & Kardash, C. A. (2002). Critical elements in the design and analysis of studies of epistemology. In B. K.
Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing
(pp. 231–260). Maswah, NJ: Lawrence Erlbaum.
Wood, P. K. (1997). A secondary analysis of claims regarding the Reflective Judgment interview: Internal
consistency, sequentiality and intra-individual differences in ill-structured problem-solving. In Higher education:
Handbook of theory and research (Vol. 12, pp. 243-312). New York: Agathon.
Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational
Psychologist, 25(1), 3-17.
Zimmerman, B. J., & Schunk, D. H. (2001). Reflections on theories of self-regulated learning and academic
achievement. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement:
Theoretical perspectives (pp. (pp. 289-307). Eribaum: Mahwah, NJ.

Metacognition 22

View publication stats