ASSESSING THE EFFECTIVENESS OF CRITICAL-THINKING INSTRUCTION

Author(s): Diane F. Halpern

Source: The Journal of General Education , 1993, Vol. 42, No. 4 (1993), pp. 238-254
Published by: Penn State University Press

Stable URL: https://www.jstor.org/stable/27797197

REFERENCES
Linked references are available on JSTOR for this article:
https://www.jstor.org/stable/27797197?seq=1&cid=pdf-
reference#references_tab_contents
You may need to log in to JSTOR to access the linked references.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

Penn State University Press is collaborating with JSTOR to digitize, preserve and extend access
to The Journal of General Education

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
ASSESSING THE EFFECTIVENESS OF CRITICAL
THINKING INSTRUCTION

Diane F. Halpern

"The country is engaged in a national debate on what

students should know and be able to do and on how to
measure achievement toward those goals."
(National Council on Education Standards and Testing
1992, 7-8)

There has been a growing trend among colleges in the United

States and Canada to require all students to fulfill a requirement
in "critical thinking" as part of their general education program.
Critical thinking is a widely used term that includes skills in
applying, analyzing, synthesizing, and evaluating information
and the disposition to apply these skills (National Center for
Excellence in Critical Thinking Instruction 1991). The ability to
think critically is almost always listed as one of the desirable
outcomes of undergraduate education (Halpern 1988). Al
though there is considerable disagreement over who should
teach such courses, whether they should be "stand-alone" ge
neric courses or incorporated into specific content areas, and
what sorts of thinking skills students should be learning in these
courses, there is virtually no disagreement over the need to help
college students improve how they think (e.g., Perkins and Solo
mon 1989). The debate over thinking-skills instruction at the
postsecondary level is particularly timely because the goal of
increasing the number of college students who can think effec
tively and solve problems is one component of the National
Educational Goals advanced by the National Governors' Asso
ciation and President Bush.
Students from North America routinely rank below those
from other parts of the world in academic areas such as scien

JGE: THE JOURNAL OF GENERAL EDUCATION, Vol. 42, No. 4, 1993.

Copyright ? 1993 The Pennsylvania State University Press, University Park, PA.

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 ASSESSING THE EFFECTIVENESS 239

tifie knowledge and understanding, mathematical problem solv

ing, and general literacy (Lapointe, Mead, and Phillips 1989).
Economists and politicians argue that the poor performance of
North American students is a threat to our ability to remain a
world leader in science and technology. The underlying ratio
nale for college-level courses in critical thinking is the belief
that students will become better thinkers if they acquire and
use thinking skills such as the ability to synthesize and analyze
information, identify main ideas, cite evidence in support of a
conclusion, and use probabilities. In California, for example,
there is a statewide general education requirement in critical
thinking for all students in the 108 community colleges and 20
state universities. Thus, in California alone, somewhere near 1
million college students are taking course work that is designed
to improve how they think. Surprisingly, there has been rela
tively little concern with the question of whether it is possible
to improve thinking skills with instruction specifically designed
for that purpose.

Framing the Question

The purpose of this paper is to answer the question, "What is

the evidence that we can teach college students to improve how
they think?" The answer to this question should be of great
interest to thousands of college curriculum committees who are
engaged in periodic reviews or revisions of their general educa
tion program. If the evidence is positive, then they will have a
rationale for instituting or maintaining a critical-thinking re
quirement; similarly, negative information would call the entire
critical-thinking "movement" into question and could suggest
that scarce general education units might be put to a more
beneficial use.
The question about the effectiveness of critical-thinking in
struction is very difficult to answer because of the multiple
complexities involved in determining if it is possible to improve
thinking skills with instruction. Like every research question,
the answer that we provide is inextricably determined by the
way we phrase the question and the way we look for an answer.
Key elements in framing the question are considered here.

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
240 THE JOURNAL OF GENERAL EDUCATION

Identification of Outcome Variables

Any assessment of student gains in the ability to think critically
needs to be based upon an operational definition of critical
thinking. Although there is no absolute agreement on what
constitutes critical thinking, there is sufficient overlap in the
various definitions to allow an evaluator to move beyond the
definitional stage. Specific lists of thinking skills can be found
in several sources (e.g., Facione 1991; Halpern 1989, in press).
A example of the sort of thinking skills that are usually in
cluded in such lists is presented in table 1.
Courses that are designed to enhance the thinking abilities of
students will usually identify a subset of skills, such as those
listed in table 1, and design instruction to develop the selected
skills. Topics of instruction often include inductive and deduc
tive reasoning, argument analysis, decision making, problem
solving, likelihood and uncertainty, and hypothesis testing. An
assessment of whether students became more critical thinkers
because of their course work will have to be tied to the goals
that were specifically targeted in the course.

Design and Selection of Assessment Instruments

Like other assessments of educational outcomes, there is a
need for tests or other measures that are sensitive enough to
pick up subtle changes in students' thinking abilities and in the
disposition to think critically. Thus, a quality assessment will
need to address the dual problem of whether students can think
more critically and, if they can, whether they actually use their
critical-thinking skills without specific prompting. Some of the
indicators of improved thinking ability may be subtle, but sig
nificant, such as an increase in the willingness to suspend judg
ment or a more deliberate effort to consider information that
would disconfirm a favored conclusion. These are important
positive changes that are difficult to "pick up" with most stan
dard instruments.
Given our knowledge of cognitive development, it would be
unrealistic to expect a huge gain in the thinking abilities of
college students that can be attributable to one course that is a
quarter or semester in length. In addition, most general educa
tion requirements in critical thinking are at the lower-division,
usually freshman, level. Of course, it is hoped that critical

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 ASSESSING THE EFFECTIVENESS 241

Table 1 An Example of a Thinking Skills List

reading with a high level of comprehension

providing support for a conclusion
understanding principles of likelihood and uncertainty
using analogies
reasoning about ratios
recognizing the difference between correlation and cause
combinatorial reasoning
isolating and controlling variables
evaluating evidence (often with emphasis on avoiding fallacies)
planning a course of action
modeling
generating hypotheses
using retention (memory) strategies
making spatial representations
restructuring problems
using problem-solving heuristics
seeking patterns
incorporating anomalous data into a coherent framework
recognizing regression to the mean

thinking skills will continue to improve throughout the under

graduate years. If a critical-thinking assessment is conducted at
or near graduation, cognitive growth that is attributable to the
entire undergraduate educational experience can be measured,
but it would be extremely difficult to tease out the effect of a
single lower-division course from an assessment taken during
the senior year. Cognitive growth is a gradual and cumulative
process; there are no quick fixes. It is more realistic to expect
modest improvements in thinking abilities, a fact that makes
assessment all the more difficult.

Ecologically Valid Indicators

The goal of critical-thinking instruction is to produce students
who have become better thinkers in the real-world contexts
that extend beyond the usual in-class exam. The entire enter
prise is of little value if the students in critical-thinking classes
are not using what they learn in novel, out-of-class situations.
The thinking skills that are learned must be spontaneously prac
ticed when students are out of the classroom and removed from
the instructor, who may serve as a stimulus cue to use these
skills. Ideally, the students who have become better thinkers

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
242 THE JOURNAL OF GENERAL EDUCATION

will demonstrate critical-thinking skills in a wide variety of con

texts that range from more reasoned consumerism to improved
problem solving. Thus a good assessment will be based on
"simulated scenarios" that are similar to the situations that
students will encounter out of the classroom.

Multiple Comparison Groups

Any assessment of gains in the ability to think critically will
require the use of multiple comparison groups so that the asses
sor can separate maturational gains and general thinking im
provement that results from the college experience as a whole
from those gains that directly result from instruction in critical
thinking. The obvious choice of using students who have not
taken a critical-thinking course and comparing them to a group
of students who have taken such a course is flawed because
critical-thinking courses are required in the general education
program at most of the universities where they are offered.
Students who have managed to avoid or postpone a required
course probably differ from those who have taken such a
course in many other ways. For example, the "avoiders" may
have less commitment to completing their degree require
ments, or they may work more hours so that the required
course was more difficult to fit in their schedule, or they may
just be more artful at avoiding requirements. Any of these
differences would confound comparisons between students
who have completed a critical-thinking requirement and those
who have not. The best solution to the problem of selecting an
appropriate comparison group is to use multiple groups, each
of which provides a different perspective on the question.
Comparison groups could include students who enrolled
prior to the establishment of a required course in critical think
ing, transfer students, students at a comparable institution that
does not require a course in critical thinking, as well as those
students who successfully managed to avoid or postpone the
required course work. Students can also be used as their own
comparisons by using a precourse-postcourse design. Like all
of the other possible comparison groups, the repeated measure
ment of students who have taken a course in critical thinking
also has drawbacks associated with it. There is always the drop
out problem with students who have not completed the course

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 ASSESSING THE EFFECTIVENESS 243

differing in numerous ways from those who have. In addition,

the repeated testing of the same students will affect results in
unknown ways?students could become test weary and/or test
wise and thus score either spuriously low or high for reasons
that are unrelated to gains in their knowledge.
Although each of these comparison groups is imperfect in
some way, when considered together, they could provide a
substantial body of support for the effectiveness of critical
thinking instruction.

Time of Testing
An ideal measure will extend beyond the semester or quarter of
instruction so that the assessor can determine if any lasting gains
are obtained. Do students who show improvement in their abil
ity to think critically maintain these gains over time, or do they
revert to easier and less effective methods of thinking? For exam
ple, will students who recognize an argument that confuses corre
lation with cause continue to recognize and avoid this fallacy
throughout their adult years? When they hear a political candi
date take credit for a positive event that occurred during her or
his term of office, will college graduates who had critical
thinking instruction be more likely to look for a causal link
between the candidate's actions and the event than college
graduates who did have this sort of targeted instruction? A qual
ity assessment of critical thinking will examine the way critical
thinking skills are maintained over one's lifetime.

Identification of Classroom Strategies That Engender Critical

Thinking
What aspects of critical-thinking instruction are the most effec
tive? This is an important question for instructors in critical
thinking courses. A quality assessment would allow us to deter
mine if gains are attributable to written assignments, active
participation in problem solving, the development of a close
relationship with a caring instructor, a combination of these
variables, or some other yet undetermined variable. An assess
ment that attempts to identify the specific educational experi
ences that result in improved critical thinking would require a
more fine-grained analysis of the nature of the instruction. In
formation of this sort would allow professors to make informed

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
244 THE JOURNAL OF GENERAL EDUCATION

decisions about what to include and how to teach courses that

are designed to improve thinking skills and would also provide
guidance about what works so that professors in upper-division
general education courses will know how to reinforce thinking
skills that are developed in lower-division courses.
The research literature in cognitive and learning psychology
is massive and provides some direction for identifying effective
educational techniques. Perusal of the leading texts and journal
articles yields several guidelines for teaching higher-order think
ing skills, a term that is often used as a synonym for critical
thinking. Teaching strategies that require active involvement
by the learner, practice of thinking skills in multiple settings,
use of examples that are similar to the situations in which the
skills will be used, students' monitoring of their comprehension
and progress toward an identifiable goal, a rationale for learn
ing the skills, and use of intrinsic motivational techniques and
multiple learning strategies are most likely to yield optimal
results (Bigge and Shermis 1992; Mayer 1992; Meyers 1987;
Schoenfeld 1987). An ideal assessment would allow the asses
sor to determine which of these practices is most closely related
to the enhancement of student thinking skills.

Issues in Educational Measurement

There are also all the usual problems that are associated with
educational measurement such as construct and predictive valid
ity, reliability, and the extent to which the scale of measurement
is assessing a unitary construct. In general, multiple-choice tests
tend to have problems with validity, and open-ended verbal and
written examinations tend to have problems with reliability. The
current trend toward open-ended assessments such as portfo
lios, interviews, focus groups, and responses to hypothetical
situations all suffer from the problem of unknown reliability and
validity (Halpern, in press). Keep in mind that no assessment
can be valid for any purpose if it is not reliable.
These are only some of the relevant considerations that need
to be resolved in order to examine changes in the ability to
think critically. The only thing that is easy about this undertak
ing is that it is easy to see that assessing outcomes that result
from critical thinking is fraught with multiple measurement and
logistical problems and decisions.

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 ASSESSING THE EFFECTIVENESS 245

The Evidence

Despite all of the difficulties in assessing gains in critical think

ing, there is a diverse body of evidence showing that thinking
can be improved with instruction that is specifically designed
for that purpose. Although there has been some debate about
whether it is possible to produce long lasting enhancement in
the ability to think effectively (Block 1985; Glaser 1984;
Resnick 1983), we now have a considerable body of evidence
that thinking-skills courses have positive effects that are trans
ferable to a wide variety of situations. At least seven qualita
tively different forms of outcome evaluations for thinking
courses have been conducted, all of which are generally sup
portive of the idea that the ability to think critically can be
improved as a result of thinking-skills instruction. While none
of these constitute the "perfect measure," taken together they
provide a substantial body of support for the value of thinking
skills instruction.

Formal Evaluations of Programs Designed to Enhance

Thinking Skills
Many North Americans are surprised to learn that there has
been a massive nationwide effort in Venezuela to improve the
thinking skills of its citizens. In fact, for a short while, they had
an official high-level position in the government that was a
minister of thinking improvement. A major component of this
effort was the design and development of a program of critical
thinking instruction for use in their schools. Psychologists from
Venezuela and the United States devised 60 lesson plans deal
ing with topics such as ordering and classifying events, under
standing language, verbal reasoning, problem solving, decision
making, and inventive or creative thinking. Fifty-six of these
lessons were presented in several seventh-grade classes. Other
comparable classes were designated as nontreatment control
classes. Results obtained from hundreds of students showed
that the students who received specific thinking instruction out
performed control subjects on standardized tests of thinking
skills. But, this is not the most important part of the assess
ment. The students who participated in the thinking-skills in
struction also showed greater gains in orally presented argu

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
246 THE JOURNAL OF GENERAL EDUCATION

ments and in answering open-ended essay questions when

these responses were graded blind, that is, when the graders
did not know if the student who was speaking or writing had
been in the experimental or control group (Herrnstein, Nicker
son, de Sanchez, and Swets 1986; Schoenfeld 1987; Walsh
1981). This is a strong finding because the students showed
clearer thinking when they were discussing and writing about a
novel topic, which is undoubtedly one of the most important
goals of critical-thinking instruction. Blind evaluations in natu
ralistic settings offer the best evidence that we can help stu
dents improve how they think.
Although no other thinking-skills program has received the
extensive testing and scrutiny that the Venezuela project has
undergone, similar positive results have been reported for
other programs as well. In Chance's (1986) independent review
of several other programs, he concluded that good thinking is a
skill that can be taught and that thinking is best taught by direct
and systematic instruction. Critical thinking skills do not neces
sarily develop as a by-product of discipline-specific course
work. Specially designed courses that focus on generic thinking
skills using varied examples provide the ideal combination of
skills training and practice with transferring the skills. A broad
based, cross-disciplinary approach is most effective for critical
thinking instruction.

Student Self-Reports
A second approach to answering the question of whether college
students can learn to improve how they think is to ask the stu
dents who have completed a thinking-skills course. Although it
is important to consider student perceptions about their own
thinking abilities, there are obvious problems with student self
reports. Students may report that they have learned to think
better when, in fact, they have not or, conversely, that they have
not improved when they really have. Despite this problem, it is
comforting to know that the overwhelming majority of students
report that they have made substantial gains in their thinking
ability after completing a thinking-skills course (e.g., Block and
Taylor 1984; Dansereau et al. 1979; Wheeler 1979). Students
rate themselves higher on numerous self-report scales, including
the willingness to suspend judgment, ability to evaluate conflict

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 ASSESSING THE EFFECTIVENESS 247

ing claims, use of likelihood and uncertainty, and utilization of

numerous problem-solving heuristics such as working backward
from the goal, eliminating useless information, and evaluating
the credibility of an information source. In other words, the
consumers of critical-thinking courses believe in their effective
ness. Students who take these courses frequently endorse re
quired course work in critical thinking as a valuable component
of the general education package.

Gains in IQ Scores
One way of measuring thinking improvement is with gains in
IQ scores on standardized intelligence tests. The underlying
rationale is that people should become more intelligent when
they think better and that gains in intelligence will be reflected
in higher 10 scores. This is consistent with most people's intu
itive notion of what it means to be intelligent, that is to think
better, be more reflective, make better decisions, evaluate evi
dence more systematically, and read with greater comprehen
sion. Thus, there is considerable overlap between being intelli
gent and thinking critically, but they are not identical concepts.
One of the pioneers in this area is Moshe Rubinstein, who has
developed a long-running popular course at UCLA designed to
teach students the skills of critical thinking. An educator at a
midwestern university who has adopted Rubinstein's program
claimed that 82.4 percent of the students who have taken his
course scored higher on a test of intelligence than they did
before the course began (Rubinstein 1980). It is difficult to
know how to interpret claims such as this one. Sternberg (1986)
has also presented substantial additional support for the notion
that IQ scores can be increased as a result of thinking-skills
instruction. Although I am somewhat skeptical about any
claims to enhance intelligence that are attributable to a single
course, I offer this as additional evidence that thinking courses
can have positive effects.

Cognitive Growth and Development

Other evidence that thinking skills can be improved with instruc
tion specifically designed for this purpose was presented by Fox,
Marsh, and Crandall (1983). These researchers measured cogni
tive growth using Piagetian measures that are widely used in

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
248 THE JOURNAL OF GENERAL EDUCATION

studies of cognitive development. They found that college stu

dents who had been taught general problem-solving skills
showed significant gains in measures of cognitive growth and
development while a control group of college students who had
not received the problem-solving training did not show signifi
cant gains. This is an impressive improvement in light of the
finding that only 25 to 50 percent of first-year college students
possess the skills needed for abstract logical thought (McKinnon
and Renner 1971).
Piagetian measures of cognitive development offer a rich
possibility for future assessments of college students. If college
students need to be able to function at the cognitive develop
mental level of "formal operations," the level associated with
the ability to think about abstract concepts, then any instruc
tion that moves students to this level should increase the proba
bility of student success in upper-division courses.

Expert-Like Mental Representations

One of the most theoretically advanced means of exploring
course-related changes in thinking ability is to examine the
underlying structure of cognitive skills and knowledge. It is one
method of studying the way people think about and organize
information. This is related to the notion of the internal repre
sentation of knowledge or schemata that is so popular in con
temporary cognitive psychology. The idea behind this sort of
investigation is that when people become better thinkers about
a topic, their internal representations of the topic-related
knowledge will become similar to the internal representations
of experts in that field (Glaser 1992). In other words, novices
will come to think more like the experts.
In an attempt to improve thinking skills, Schoenfeld and
Herrmann (1982) taught college freshmen and sophomores
either general techniques of problem solving or "structural
programming" which involved "an orderly way to solve non
mathematical problems using the computer." Students who
received the general strategy instruction approached mathe
matical problems in ways that were similar to the approaches
used by professors of mathematics. The general strategy ap
proach demonstrated substantial improvement in problem
solving scores, while those in the computer course showed

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 ASSESSING THE EFFECTIVENESS 249

little improvement over the scores obtained before taking the

course. This sort of assessment is particularly useful for profes
sors who are interested in the more theoretical aspects of
critical thinking such as understanding how an individual's
information structures change as a result of instruction. It is a
valuable method, but not likely to be used by faculty outside
of the cognitive sciences.

A Cognitive Skills Approach

Another approach to assessing critical thinking is to begin with
a specification of the skills that are needed for effective think
ing. There are many lists of cognitive skills that contain entries
similar to those in table 1 (e.g., Halpern 1989). In a recent
study, Facione (1991) devised a multiple-choice test of selected
thinking skills which he administered to students who had com
pleted a general education college-level course in critical think
ing. He found that these students scored significantly higher
than comparable students who had not taken a course in critical
thinking. In fact, if he found that college students could not
learn these concepts, it would call into question the entire ratio
nale for higher education.
A similar, but more advanced test of critical-thinking skills
was developed by Lehman, Lempert, and Nisbett (1988) for
use with students in graduate school. They found that the think
ing skills that the students were taught in their graduate pro
grams generalized to everyday life events. Taken together,
these studies show that thinking skills can be identified, taught,
learned, and used appropriately in novel contexts. Tests such as
these that target individual thinking skills allow us to determine
what is known on a skill-by-skill basis. This information can be
used specifically to improve instruction of those skills that stu
dents are not exhibiting.

Measures of Spontaneous Transfer

The goal of any instruction designed to improve thinking is trans
fer of training. What I mean by transfer is use of critical-thinking
skills in a wide variety of contexts. The whole enterprise would
be of little value if these skills were only used in the classroom or
only on problems that are very similar to those presented in
class. Ideally, critical-thinking skills should be used to recognize

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 250 THE JOURNAL OF GENERAL EDUCATION

and resist unrealistic campaign promises, circular reasoning,

faulty probability estimates, weak arguments by analogy, or lan
guage designed to mislead whenever and wherever they are en
countered. The ability to transfer thinking skills is, in my opin
ion, the most important of the outcome measures.
Nisbett and his colleagues (Lehman and Nisbett, 1990) have
engaged in a series of studies in which they examined the spon
taneous transfer of thinking skills to real-world problems that
occur outside the classroom. In one such study, they phoned
students at home several months after completion of their
course work. Students were told that the caller was taking a
sports poll, and they were asked to comment on several contem
porary sports figures. The critical questions concerned whether
these students would recognize and use concepts like regres
sion to the mean and the need for an adequately large sample
size when they reasoned about a nonacademic topic. Results
were supportive of the idea that the students had learned and
spontaneously used these concepts.
There are numerous other examples that all point to the
same conclusion: better thinking can be learned with appropri
ate instruction (for example, see Cyert 1980; Halpern 1987a;
1987b; Lochhead and Clement 1979). There are multiple quali
tatively different types of evidence that show that thinking
skills can be improved with specific instruction that has been
designed for that purpose. The empirical literature shows that
when critical-thinking instruction is done well, students are
more disposed to think critically and become more able think
ers. Although we still know relatively little about the most
effective components of courses designed to improve student
thinking, critical-thinking instruction has proved itself to be a
valuable component of the general education curriculum.

Directions for Future Assessments

Although there is still much work to be done, the research litera

ture shows that the goal of helping college students think more
critically is obtainable. It is now time to go beyond debates about
whether improved thinking can be an outcome of higher educa
tion. It can. However, we still know very little about important

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 ASSESSING THE EFFECTIVENESS 251

issues in critical-thinking instruction, for example, which educa

tional experiences yield the greatest gains in critical thinking or
whether laboratory experiences develop thinking skills more
fully than standard lecture courses. The active, involved nature
of laboratories would be expected to foster gains in critical think
ing, but there is little empirical data on this question. Similarly,
do students develop thinking skills more effectively in courses
that require a great deal of writing and speaking, or are these
popular but unsubstantiated "myths" about learning in higher
education?
There is tremendous potential for a business-college partner
ship in the area of critical thinking. Virtually every business or
industry position that involves responsibility and action in the
face of uncertainty would benefit if the people filling that posi
tion obtained a high level of the ability to think critically. Infor
mation about the nature of the thinking skills that college gradu
ates can be expected to achieve by graduation would be very
useful to prospective employers who could use this information
to make better matches between prospective employees and
the cognitive demands of a position. Business and industry can
also be involved in identifying the critical-thinking skills that
they want their employees to have when they graduate from
college. They should be willing to spend some of the money
that is used for in-house training on general education critical
thinking courses if the business and industry leaders were as
sured that their funds were being spent to improve the thinking
abilities of their future employees.
We also know very little about the way thinking skills change
over the adult life span. This is an important question because
North American society is aging and, at the same time, tech
nologies are changing so rapidly that many adults can expect to
work at jobs that don't currently exist. Critical-thinking skills
offer the greatest chance of success for creating and adjusting
to change. The incorporation of critical-thinking courses in con
tinuing education course work, in addition to general education
programs, would offer this sort of learning experience to a
segment of society that usually does not enroll in lower-division
classes.
An alternative point of view for the many questions about
critical-thinking instruction is to begin with a survey of those

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
252 THE JOURNAL OF GENERAL EDUCATION

thinking skills that are most needed in contemporary society

(e.g., posing and solving problems such as improving and de
bugging a computer program) and then proceed with a diagnos
tic approach in which individual students are assessed with re
gard to those skills. The goal of a diagnostic approach is to
identify specific skills that an individual needs to strengthen.
Individualized instruction that is skill-specific would be em
ployed (perhaps with the use of modern technological aids such
as interactive video disks or computer programs), and the stu
dent is then retested to determine if the targeted skills have
been learned.
We also know very little about individual student variables
that contribute to the ability to think critically. For example,
are there course patterns and curricula that students take in
high school and in college that develop thinking skills better
than others? Do physical science majors, for example, have a
different thinking-skills profile than humanities majors, either
when they enter the major or when they graduate? Do individ
ual learning styles or preferred modes of thought predict the
attainment of specific thinking skills? There is an endless list of
questions that can be asked about the relationship among learn
ing experiences, individual abilities, and the development of
critical-thinking skills. Now that it is clear that higher education
can enhance the thinking skills of college students, more spe
cific relationships can be examined to provide direction for the
critical-thinking component of general education.

References

Bigge, M. L., and S. S. Shermis. Learning Theories for Teachers. 5th ed. New York:
Harper Collins, 1992.
Block, R. A. "Education and Thinking Skills Reconsidered." American Psychologists
40 (1985): 574-75.
Block, R. A., and S. V. Taylor. "Cognitive Skills: Enhancement and Assessment
Issues." Symposium on A Psychological Perspective on Teaching Thinking Skills
to College Students. Annual Meeting of the American Psychological Assn. To
ronto, August 1984.
Chance, P. Thinking in the Classroom: A Survey of Programs. New York: Teacher's
College Press, Columbia University, 1986.
Cyert, R. M. "Problem Solving and Education Policy." Problem Solving and Educa
tion: Issues in Teaching and Research. Ed. D. T. Tuma and F. Reif. Hillsdale:
Erlbaum, 1980. 3-8.

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 ASSESSING THE EFFECTIVENESS 253

Dansereau, D. F., . W. Collins, . A. McDonald, C. D. Holley, J. Garland, G.

Diekoff, and S. H. Evans. "Development and Evaluation of a Learning Strategy
Training Program." Journal of Educational Psychology 71 (1979): 64-73.
Facione, P. "Testing College Level Critical Thinking Skills." 11th Annual International
Conference on Critical Thinking and Educational Reform. Sonoma, August 1991.
Fox, L. S., G. Marsh, and J. C. Crandall, Jr. "The Effect of College Classroom
Experiences on Formal Operational Thinking." Annual Convention of the West
ern Psychological Assn. San Francisco, 30 April 1983.
Glaser, R. "Education and Thinking: The Role of Knowledge." American Psychologist
39 (1984): 93-104.
-. "Expert Knowledge and Processes of Thinking." The Enhancement of Think
ing Skills in the Sciences and Mathematics. Ed. D. F. Halpern. Hillsdale: Erlbaum,
1992. 63-75.
Halpern, D. F. "Analogies as a Critical Thinking Skill." Applications of Cognitive
Psychology: Computing and Education. Ed. D. Berger, . Pezdek, and W. Banks.
Hillsdale: Erlbaum, 1987a. 75-86.
-. "Thinking Across Disciplines: Methods and Strategies to Promote Higher
Order Thinking in Every Classroom." Thinking Skills Instruction: Concepts and
Techniques. Ed. M. Heiman and J. Slomianko. Washington: National Education
Assn., 1987b. 69-70.
-. "Assessing Student Outcomes for Psychology Majors." Teaching of Psychol
ogy 15 (1988): 181-86.
-. Thought and Knowledge: An Introduction to Critical Thinking. 2nd ed.
Hillsdale: Erlbaum, 1989.
-. "Targeting Outcomes: Covering Your Assessment Needs and Concerns."
Handbook for Enhancing Undergraduate Education in Psychology. Ed. T. V. Mc
Govern. American Psychological Assn., in press.
Herrnstein, R. J., R. S. Nickerson, M. de Sanchez, and J. A. Swets. "Teaching Think
ing Skills." American Psychologist 41 (1986): 1279-89.
Lapointe, A., . Mead, and G. Phillips. A World of Differences. Princeton: Educa
tional Testing Services, 1989.
Lehman, D., R. E. Lempart, and R. Nisbett. "The Effects of Graduate Training on
Reasoning." American Psychologist 43 (1988): 431-42.
Lehman, D., and R. Nisbett "A Longitudinal Study of the Effects of Undergraduate
Training on Reasoning." Developmental Psychology 26 (1990): 952-60.
Lochhead, J., and J. Clement, eds. Cognitive Process Instruction: Research on Teaching
Thinking Skills. Philadelphia: Franklin Institute, 1979.
Mayer, R. E. Thinking, Problem Solving, Cognition. 2nd Ed. New York: Freeman,
1992.
McKinnon, J. W., and J. W. Renner. "Are Colleges Concerned with Intellectual Devel
opment?" American Journal of Psychology 39 (1971): 1047-52.
Meyers, C. Teaching Students to Think Critically. San Francisco: Jossey-Bass, 1987.
National Center for Excellence in Critical Thinking Instruction. "Defining Critical
Thinking." Draft Report. Santa Rosa: Sonoma State University, 1991.
National Council on Education Standards and Testing. Raising Standards for American
Education. Washington: Author, 1992.
Perkins, D., and G. Soloman. "Are Cognitive Skills Context Bound?" Educational
Researcher 18 (1989): 16-25.
Resnick, L. B. "Mathematics and Science Learning: A New Conception." Science 220
(1983): 477-78.
Rubinstein, M. F. "A Decade of Experience in Teaching an Interdisciplinary Problem

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms
 254 THE JOURNAL OF GENERAL EDUCATION

Solving Course." Problem Solving and Education: Issues in Teaching and Re

search. Ed. D. J. Tuma and F. Reif. Hillsdale: Erlbaum, 1980. 25-38.
Schoenfeld, A. H., ed. Cognitive Science and Mathematics Education. Hillsdale: Erl
baum, 1987.
Schoenfeld, A. H., and D. J. Hermann. "Problem Perception and Knowledge Struc
ture in Expert and Novice Mathematical Problem Solvers." Journal of Experimen
tal Psychology: Learning, and Cognition 8 (1982): 484-94.
Sternberg, R. J. Intelligence Applied: Understanding and Increasing Your Intellectual
Skills. San Diego: Harcourt, 1986.
Walsh, J. "A Plenipotentiary for Human Intelligence." Science 214 (1981): 640-41.
Wheeler, D. D. "A Practicum in Thinking." A Practicum in Thinking. Ed. D. D.
Wheeler and W. N. Dember. Cincinnati: Depart, of Publication and Printing
Services, U of Cincinnati, 1979. 6-18.

This content downloaded from

37.210.165.200 on Mon, 12 Apr 2021 19:05:54 UTC
All use subject to https://about.jstor.org/terms