Accepted Manuscript

Title: Challenges in wide scale implementation efforts to

foster higher order thinking (HOT) in science education across
a whole school system
Author: Anat Zohar
PII: S1871-1871(13)00041-2
DOI: http://dx.doi.org/doi:10.1016/j.tsc.2013.06.002
Reference: TSC 207
To appear in: Thinking Skills and Creativity
Received date: 22-1-2013
Revised date: 12-6-2013
Accepted date: 15-6-2013
Please cite this article as: Zohar, A., Challenges in wide scale implementation efforts to
foster higher order thinking (HOT) in science education across a whole school system,
Thinking Skills and Creativity (2013), http://dx.doi.org/10.1016/j.tsc.2013.06.002
This is a PDF le of an unedited manuscript that has been accepted for publication.
As a service to our customers we are providing this early version of the manuscript.
The manuscript will undergo copyediting, typesetting, and reviewof the resulting proof
before it is published in its nal form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that
apply to the journal pertain.
Page 1 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Challenges in wide scale implementation efforts to foster higher order
thinking (HOT) in science education across a whole school system
Abstract
This study explores the challenges involved in scaling up projects and in
implementing policies across the whole school system in the area of teaching higher
order thinking (HOT) in Israeli science classrooms. Eight semi-structured individual
interviews were conducted with science education experts who hold leading positions
pertaining to learning and instruction on the state level of the following school subjects:
elementary and junior- high school science and technology; high-school physics; high
school chemistry; and high school biology. Some of the challenges that the interviews
revealed are common to many types of educational change processes. The interviews
also revealed several challenges which are more specific to the educational endeavor of
teaching HOT according to the infusion approach across large numbers of classrooms:
challenges involved in weaving HOT into multiple, varied, specific science contents;
challenges involved in planning a reasonable and coherent developmental sequence of
thinking goals; the fact that content goals tend to have priority over thinking goals and
thus to disperse of the latter in policy documents and in implementation processes; and
finally, the considerable challenges (pedagogical and organizational) involved in
developing educators sound and deep professional knowledge in the area of teaching
HOT and metacognition on a large, nation-wide scale. The data shows that wide-scale
implementation of thinking in Israeli science classrooms often develops as an
evolutionary rather than as a revolutionary process. The implications for designing large
scale implementation programs aimed at fostering students reasoning are discussed.
Key words: Higher order thinking, thinking strategies, large scale
implementation, from policy to practice, teachers' knowledge

Page 2 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
1. Introduction
1.1 Issues involved in teaching HOT on a large scale
So teaching for thinking and understanding [across the whole school
system] We have not yet entirely deciphered the code of how to do it
[L2]
As the introductory citation indicates, the challenge of teaching thinking on a
large, national scale is a huge one. There is nothing new in acknowledging that a large
gap often exists between educational policy and the way it is implemented. This gap is
especially large in the context of policies that address changes in the core of education,
i.e., changes in learning and instruction, such as the change involved in teaching
thinking. Unfortunately, it is very difficult to change the core of education on a large,
system-wide scale.. Large scale efforts to improve teaching and learning focus more on
structural and administrative characteristics of reform than they do on fundamental
changes in the instructional core. Innovations that require significant changes in the core
of educational practice are usually not only limited in their effects to a small scale, but
also do not usually last very long.

Innovations addressing the teaching of thinking are definitely at the very core of
learning and instruction. Without delving into the challenges involved in defining higher
order thinking (e.g., Resnick, 1987; Schraw & al., 2011), I refer here to the latter
concept in its widest sense encompassing issues such as thinking skills/strategies,
critical thinking, argumentation, use of evidence, scientific reasoning, scientific literacy,
inquiry, problem-based learning and problem solving. During the past 30 years there
have been a substantial, and rapidly growing number of empirical studies supporting
models and theories that address teaching thinking in science classrooms. Consequently,
educators are currently familiar with many good models that work quite well for
teaching science by emphasizing students' higher order thinking rather than merely
memorization of facts.
Most of the models for teaching thinking in science education classrooms were
studied within small scale projects. In addition, there have been some pioneering
attempts to scale up such projects to scores of teachers and classrooms (e.g., Adey &
Page 3 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Shayer, 1993; 1994; Blumenfeld, Fishman, Krajcik & Soloway, 2000; Osborne,
Erduran and Simon 2004; White & Frederiksen, 1998; Author, 2004). However, at the
turn of the 21
st
century, successful teaching of thinking on the level of small or even
large - scale projects is no longer sufficient. Policy documents from all over the world
highlight the need to teach 21
st
century skills. HOT is an important component of any
list of 21
st
century skills (Partnership for 21
st
Century Skills, retrieved July, 2011;
Pellegrino & Hilton, 2012). Resnick (2010) argues that scaling up the "thinking
curriculum" in a way that will foster proficiency for ALL students is currently a major
educational challenge:
"Today we are aiming for something new in the world: An elite standard for
everyone That is what the term 21st-century skills really means. The skills are
not new (some students have been successfully learning them in some schools
from the beginning of civilization). But the aspiration to successfully teach
knowledge-grounded reasoning competencies to everyone is still just thatan
aspiration. But the transformation of the institution of schooling that will be
needed to come close to making the aspirational goal a real achievement is huge
"(P. 184)
The goal of this paper is to examine the challenges involved in scaling- up
instruction of higher order thinking. The meaning of scaling up in this context is to take
ideas and practices educators are familiar with on the level of projects and to implement
them on a national level, i.e., across the state's whole school system. The paper
examines these challenges by studying the views of leaders who had been involved in
various large scale efforts to implement HOT in science instruction. Naturally, some of
the pertinent challenges are common to gaps between policy and practice in general, or
to scaling - up innovative, reform pedagogies in other areas (e.g., Blumenfeld, Fishman,
Krajcik, Marx, & Soloway, 2000; Dede, Honan and Peters, 2005; Elmore, 2004;
Fullan, 2007; Levin, 2008; Levin & Fullan, 2008; Lee & Krajcik, 2012). Yet, because of
the unique features of teaching higher order thinking, some of these challenges are
unique to efforts aiming at fostering students' thinking across hundreds or even
thousands of classrooms.
Since many of the challenges that will be described in the findings section
pertain to the development of teachers' knowledge, , the next section will discuss
relevant prior studies addressing teachers' knowledge in the context of teaching HOT.
Page 4 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
This will be followed by a section that will describe the educational context within
which the present study took place.
1.2 Teachers' knowledge in the context of teaching HOT
Instruction of HOT requires much more than adopting a new curriculum because
it requires a deep change in teaching practices. Like the teaching of other issues that
pertain to current educational reforms, it stretches and challenges teachers' capabilities.
In order to be able to respond to the unexpected events characterizing "thinking rich
classrooms", teachers must be able to teach in an intelligent, flexible and resourceful
way that cannot be scripted into a fixed set of technical instructional routines and skills
(Carpenter et. al., 2004; Loef-Frank et al., 1998). In order to teach thinking successfully
teachers need to replace the traditional view of teaching as transmission of information
and learning as passive absorption with more active, constructivist views of learning
and an intricate set of specific beliefs and knowledge about teaching. Let us take a
closer look at this knowledge.
1.2.1 Subject Matter Knowledge and Pedagogical Content Knowledge (PCK)
in the Context of teaching HOT
As many studies show, familiarity with whatever it is that one is supposed to
teach is a necessary condition for instruction. Another necessary condition for sound
instruction is familiarity with appropriate teaching methods. There is a large body of
literature that, following Lee Shulmans work, addressed various components of
teachers knowledge and distinguished (among other things) between subject matter
knowledge, general pedagogical knowledge and pedagogical content knowledge (PCK).
However, since the classic discourse in this area usually applies to teaching concepts
rather than to teaching thinking, the meaning of these components of teachers
knowledge is not straight forward when we try to apply it to the context of teaching
HOT. It therefore requires further clarification.
The term used in the literature for whatever it is that one is supposed to teach is
subject-matter knowledge (e.g., Cocharn & Jones, 1998; Shulman, 1986, 1987; Wilson
et al., 1987). But because of the unique nature of thinking strategies this concept is
confusing when the focus of our attention is on teaching thinking rather than on
teaching facts and concepts. Although according to Shulman subject matter knowledge
includes substantive knowledge (the explanatory structures or paradigms of the field)
Page 5 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
and syntactic knowledge (the methods and processes by which new knowledge in the
field is generated), content knowledge (the knowledge of specific facts and concepts) is
an essential component. When we focus on teaching thinking the traditional meaning of
content knowledge is not at the core of our educational agenda. Therefore, in order to
avoid confusion and to delineate the unique nature of teaching thinking, I prefer in this
context to substitute the term subject-matter knowledge with the term knowledge of
elements of thinking. This includes: a. knowledge of individual thinking strategies such
as making comparison, formulating justified arguments or drawing valid conclusions; b.
knowledge of genres of thinking such as argumentation, inquiry learning, problem
solving, critical thinking, scientific thinking or creative thinking (Schraw & al., 2011;
Ministry of education, 2009); c. knowledge of metacognition (for elaboration, see
below); and d. knowledge of a variety of additional issues which are important for a
successful "thinking classroom" such as thinking dispositions or habits of mind, and an
appropriate "culture of thinking" (Perkins et. al., 1993; Swartz et. al., 2008). It is
important to note that several previous studies show that in-service and pre-service
teachers initial knowledge of thinking strategies are often not sound enough for
purposes of instruction (e.g., Bransky et al., 1992; Brownell et al., 1993; Jungwirth,
1987, 1990, 1994; Paul et al., 1997; Zembal-Saul et al., 2002; Author, 2004).
A second component of teachers knowledge which is significant for the present
paper is pedagogical content knowledge (PCK). PCK is a blend of pedagogical
knowledge and subject-matter knowledge that is specific to each teaching topic (e.g.,
Adams & Krockover, 1997; Cocharn & Jones, 1998; Gess-Newsome & Lederman,
1999; Kennedy, 1990; Loughran et al., 2000; Shulman, 1986, 1987; Van Driel et al.,
1998; Wilson et al., 1987; Zeidler, 2002). In the context of teaching higher order
thinking, the classic conceptual distinction made in the literature between pedagogical
content knowledge and general pedagogical knowledge is fuzzy and unclear. Part of the
difficulty in aligning teachers knowledge in the context of teaching thinking with the
prevalent concepts used in the literature is related to the debate among scholars
regarding the question of whether thinking strategies are general or content specific.
Teaching thinking according to the infusion approach i.e., integrating the teaching of
thinking with the teaching of specific contents (Ennis, 1989; Swartz & al., 2008;
Abrami, 2008) assumes that thinking skills have some elements that are general and
other elements that are content specific (Perkins and Salomon, 1989). This notion
presents an innate difficulty in referring to the pedagogical knowledge teachers have in
Page 6 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
this field as either pedagogical content knowledge (that tends to be embedded in
specific subject-matters), or as general pedagogical knowledge (that tends to be
independent of specific subject-matters). It seems that because of the special nature of
the type of knowledge under consideration the existing constructs are problematic. I had
therefore suggested addressing teachers pedagogical knowledge in relation to
instruction of higher order thinking by using a special term: pedagogical knowledge in
the context of teaching higher order thinking (Author, , 2004; 2008 ). This term fits
well with the term knowledge of elements of thinking explained earlier, and
highlights the fact that pedagogical knowledge in this field has some unique
characteristics. At the same time this term does not imply a commitment to treat this
knowledge as either content-specific or general.

1.2.2 Teachers' knowledge in the context of metacogniton
A third component of relevant teachers knowledge pertains to
metacognition. One of the most widely used definitions of metacognition was proposed
by Flavell and his colleagues (Flavell, 1979; Flavell, Miller, & Miller, 2002),
distinguishing between two major components of metacognition: metacognitive
knowledge and metacognitive monitoring and self-regulation. The latter component is
also named in current literature as metacognitive skills (e.g., Veenman, Van Hout-
Wolters, & Afflerbach, 2006). Metacognitive knowledge includes three sub-categories:
knowledge about persons, tasks, and strategies. In the metacognitive skills branch of
metacognition, Flavell et al. (2002) elaborate on monitoring, self-regulation, and also
describe planning and evaluating. These sub-categories are also used in other prominent
frameworks. For example, Schraw and his colleagues describe the regulation component
of metacognition as comprising of processes of planning, monitoring, and evaluation
(Schraw, 1998; Schraw & Moshman, 1995). More recently, Whitebread and his
colleagues have proposed a framework that adopts Flavells definitions of
metacognitive knowledge and defines regulatory skills of planning, monitoring,
evaluating, and control (Whitebread et al., 2009). Many studies show that using
metacognition in the classroom may improve learning in general (see Veenman 2011 for
review), and learning of problem-solving, inquiry and higher order thinking in particular
(e.g., Chen & Klahr, 1999; Lin & Lehman, 1999; Ross, 1988; Schoenfeld, 1992; Toth,
Klahr, & Chen, 2000;White & Frederiksen, 1998, 2000; Author, and XXXA, 2008;
Author, and XXXB, 2008).
Page 7 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Yet, despite its strong effect on learning, recent studies highlight the fact that
teachers find enacting a pedagogy for metacognition difficult. It is not trivial for them to
take up research-based ideas in this field and to translate them into practical
recommendations (Leat and Lin, 2007). However, despite the prominent role of
metacognition in student success, only limited research has been conducted to explore
teachers' and pre-service teachers' metacognitive knowledge, their pedagogical
knowledge, and their ability to make progress in these types of knowledge following PD
(Abd-El Khalicka and Akerson, 2009; Kramersky and Mihcalsky, 2009; Wilson and
Bai, 2010). More specifically, a review of journal articles about metacognition revealed
that between 2000-2012 only three studies were conducted to explore science teachers'
learning of issues pertaining to the pedagogy of teaching metacognition in science
classrooms (Author and XXX, submitted).
One of the components of metacognition which is particularly significant
for teaching and learning thinking strategies is meta-strategic knowledge, or MSK for
short. MSK consists of general knowledge about thinking strategies, i.e., what is the
strategy and when, why and how it should be used. In order to apply MSK successfully
in the classroom, teachers need to have sound MSK, as well as a variety of specific
elements of pedagogical knowledge consisting of relevant teaching strategies such as:
modeling the use of a thinking strategy in a variety of specific contents; providing
opportunities for students to articulate the thinking strategies they apply during
thinking; to introduce the "language of thinking" into the classroom; to design and to
teach careful and thoughtful learning activities in which thinking goals are made
explicit; and to engage in long term and systematic planning of thinking activities across
several sections of the science curriculum. Research findings show= that science
teachers' initial knowledge regarding MSK is lacking and unsatisfactory for the purpose
of instruction and that teachers are not aware of the pertinent pedagogical knowledge
before they study about it in PD courses designed to address that knowledge (Author,
1999; 2006; 2008).

1.2.3 Teachers' knowledge pertaining to pedagogies of knowledge
construction
1.3 Educational Context: relevant facets of the Israeli educational
school system and a brief history of teaching HOT in science classrooms in Israel
Page 8 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
In order to understand the context of the processes discussed in this paper, some
relevant background information about the Israeli educational school system and about
pertinent efforts to teach thinking over the years is called for.
The Israeli educational system is centralized. With approximately 2 million
students (k-12th grade) and over 4000 schools there is basically one mandatory
curriculum prescribed by the Ministry of Education that covers a large percentage of
what is taught in most schools. At the end of high school students take the matriculation
exams that consist of exams in 7 mandatory core subjects: Language (Hebrew/Arabic),
English (as 2
nd
language), Mathematics, History, Bible, Literature and Civics.
Additional subjects are mandatory in elementary and junior high schools (Science and
technology, Geography, 2nd foreign language, etc.). In addition, many other subjects
are electives in high school (e.g., Biology, physics, chemistry, communication, arts,
computers, etc.). Each subject has a steering committee which consists of academics,
Ministry of Education officials and teachers. The steering committee is usually chaired
by a distinguished professor from the pertinent academic discipline who is interested in
contributing to educational practice in his or her field. For each subject there is also a
National Subjects Superintendent (NSS) who is responsible (in collaboration with the
steering committee) for policy making (i.e., for defining the goals of the curriculum)
and for the practical sides of instruction, including teachers PD and assessment in that
particular subject. NSSs work with a team of instructors who help to coordinate and to
lead the above activities in each subject. Instructors also provide teachers with
pedagogical support through PD courses, visits in classrooms and schools and by
meetings with small groups of teachers to discuss professional matters. As we shall see,
instructors have a prominent role in the implementation processes described in what
follows.
Teaching thinking is not a new goal in science education in Israel. In fact, it has
quite a long history. Although in the first half of the 20
th
century students had
sometimes studied science by inquiry and through observations that took place in field
excursions, science education during that period had been, for the most part, descriptive
and fact-based. The first comprehensive science education reform in this area took place
in the late 1960's, highlighting inquiry and higher order thinking. It centered on a policy
decision to change biology education into inquiry teaching by adopting the inquiry-
oriented Yellow version of BSCS (BSCS- Biological Science Curriculum Study,
1963), and adapting it to the context of the Israeli school system. This reform which
Page 9 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
had been extremely innovative at the time, assumed that the way to create a long-
lasting, sustainable educational change must be holistic (Tamir, 2006). It was assumed
that the route to success must pass through working simultaneously in three main areas:
curriculum materials, professional development, and assessment. The new assessment
methods that had been developed as part of that change process in Israel were
innovative and unique in the 1970s'. The biology team under the leadership of Pinchas
Tamir understood that a lack of consistency between the inquiry-goals of the new
curriculum and the assessment will put the reform effort at risk, because teachers will
continue to teach for the test in terms of content, rather than follow the inquiry-
oriented learning environment of the new curriculum. The matriculation exam was
therefore transformed in a radical way to reflect the goals of the new inquiry-based
curriculum. In effect, Tamir and his team changed the traditional matriculation
examination into an assessment that by todays terminology can be defined as
alternative assessment. The "new" matriculation examination consisted of multiple,
varied means of assessment designed to assess science knowledge as well as inquiry
skills: a written test, a school-based research project accompanied by an oral exam, a
school-based laboratory test, and a field test. It is remarkable to note that many of the
main facets of the inquiry-based biology curriculum and assessment are still practiced in
the biology national curriculum even today, more than 40 years after the reform had
been initiated. Consequently, biology teaching in Israel had been more thinking oriented
than physics or chemistry even in the early 2000's (Author and XXX, 2005).
The next noteworthy reform policy is the "Tomorrow 98" reform. Following the
nomination of a public committee, a comprehensive report was written in 1992 about
math, science and technology education. The report made a list of recommendations
aimed at the improvement of education in these areas with an eye to preparing students
for life in the 21
st
century (Ministry of Education and Culture, 1992). Among other
goals, the report stated that the development of students' thinking in science is an
important educational goal. The report was followed by a generous budget devoted to
the improvement of k-12 science education. Among the report's practical consequences
were the following: a. Two new curriculum documents (for grades 1-6 and for grades 7-
9) which included explicit HOT goals to be taught in science lessons. In those
documents lists of thinking skills are delineated in a special section of the curriculum
that is separate from the sections delineating science topics and concepts b. Several
Page 10 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
separate projects that created learning materials designed to integrate the teaching of
science topics with the teaching of thinking strategies and to promote project-based
learning.
Another important facet of these projects were PD courses aimed at developing
teachers' ability to use the new learning materials in a proficient way (Eylon & Bagno,
1997; Spector-Levy, Eylon & Schertz, 2008; r, 2004). Changes aimed at fostering HOT
and inquiry learning were also introduced to high school chemistry (See Avargil & al.,
this volume).
Finally, in January 2007 the Israeli Ministry of Education (MOE) adopted a new
general (i.e., not limited to science education) national educational policy: Pedagogical
Horizon (PH) Education for Thinking (Office of Pedagogical Affairs, MOE, 2007;
Author, 2008; Gallagher, Hipkins and Zohar, 2012). The novelty of the PH policy was
is in addressing the teaching of thinking as an explicit, major and universal educational
goal; and in planning practical means for wide-scale implementation throughout the
school system. The emphasis of the Pedagogical Horizon policy was on pedagogy
rather than on content: on how to rather than on what to teach. The policy adopted an
infusion approach to teaching Higher Order Thinking (HOT): thinking was integrated
into school curricula rather than taught as an independent subject. Therefore, the policy
advocated thinking within conceptually rich domains of knowledge. An ideal lesson
according to the PH policy consisted of both content goals and thinking goals each of
which were addressed in an explicit way. The lesson was rich in cognitively challenging
questions and tasks that made intense usage of thinking strategies such as
argumentation, problem solving, asking questions, comparing and contrasting, making
decisions, controlling variables, drawing conclusions and identifying assumptions. All
these thinking strategies were being used within the lessons subject matter and were
thus embedded in rich conceptual contents. The classroom learning environment
fostered discourse that wa rich in the language of thinking. Inquiry learning was
encouraged. Finally, lessons also fostered metacognitive thinking, including intensive
engagement with MSK. This means that meta-level elements of thinking strategies were
taught in an explicit way. Principles pertaining to the "what", "when" and "why" of
thinking strategies were explicitly addressed in class discussions, in active individual
and group-work and in learning materials (textbooks and computerized learning
materials. Many studies show that such explicit engagement with general meta-level
Page 11 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
features of thinking have a large positive effect on students' level of thinking (e.g.,
Abrami, 2008; Dwyer & al., 2012; Author and XXXA, 2008; Author and XXXB,
2008).
Following the model of the BSCS program that had taken place 4 decades earlier
the PH was implemented through working simultaneously on the development of three
areas: Curricula and learning materials, PD and assessment.
1.4. Research Question
In sum, this brief review shows that several system-wide substantial efforts to
promote HOT had taken place in science education in Israel over the years. These
efforts may provide a fruitful context to study system-wide change processes in the area
of teaching HOT. Numerous informal conversations conducted over the years with
some of the people who were deeply involved in leading such implementation processes
indicated that they had gained rich knowledge and insights through their experiences. A
set of interviews was thus designed with the overall goal of uncovering that knowledge
and insights in order to turn them into explicit common knowledge. The interviews
covered too many topics to be addressed in a single paper (see below for more details).
The present paper centers on one of these topics- the challenges involved in scaling up
instruction of HOT across the whole school system. Consequently, the present paper
explores the following research question: What do science education leaders view as the
main challenges in scaling up instruction of higher order thinking across the whole
school system?
2. Methodology
This study is based on a set of eight semi-structured individual interviews
conducted with science education leaders in Israel. These leaders were chosen because
of their prominent roles and first-hand experiences in leading activities and processes on
the national level in the following school science subjects: elementary general science,
junior high school general science, high school biology, high school physics, and high
school chemistry. Four interviewes were conducted with National Subject Supervisors
(NSS's), i.e., with people who have had leading positions in directing and supervising
the pertinent school science subjects within the Ministry of Education, Four additional
interviews were conducted with university science education professors who, in
Page 12 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
addition to their academic roles, had been active and highly influential in national level
science education committees in the Ministry and in university-based science education
projects. They served as chairs or members of steering committees who directed the
various school science subjects, as directors of large-scale science projects, and/or as
directors of teachers' national PD centers.
One of the university science education professors was involved intensively in
leading educational activities in two different school science subjects and was therefore
interviewed twice a separate interview took place for each of the school subjects s/he
was involved in. Consequently there were a total of eight interviews but only seven
different interviewees.
All the potential candidates that were asked to participate in this study consented
to do so (100% consent). The researcher had prior professional acquaintance with all
seven interviewees. In order to encourage free expression of ideas the interviewer
promised anonymity. Because the interviewees are public figures who are well known
in the educational arena in Israel all personal details and potentially identifying
information are omitted from the study. Therefore, additional information about the
interviewees cannot be provided here in order to preserve the promised anonymity.
Interviews asked about the present state of teaching for thinking and
understanding in the relevant school science subjects, about past and current change
processes related to wide-scale implementation of thinking and about challenges and
assessments related to these processes (for the full interview protocol please see
Appendix A). Interviews were between one and two hours long. Interviews were semi-
structured: Following a general presentation of the goal of the study and the definition
of higher order thinking in the interview, the researcher presented the same set of 10
questions to all interviewees. However, following their diverse areas of expertise and
personal experiences, each interviewer was allowed to elaborate on different sections of
the interview protocol. Different follow-up questions were thus presented to each of the
interviewees. The interviews were transcribed and analyzed using content analysis, i.e.,
"a process of qualitative data reduction and sense - making effort that takes a volume of
qualitative material and attempts to identify core consistencies and meanings" (Patton,
2002, p. 453). The interview transcripts were read several times and several short ideas
emerging from the text were written for each section of the interview, using a word
Page 13 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
processor table. Sections consisting of ideas that had a similar meaning were tassembled
together to create patterns.
Each pattern was then read and re-read to create codes and sub-codes. Sections
consisting of the same codes and sub-codes were assembled together to create groups of
citations. Finally, each of the patterns chosen for the analysis presented in the present
paper (for elaboration on the choice process see below) were analyzed by trying to
interpret the meaning of the citations grouped under the same sub-code and code. The
analysis thus focused on trying to make sense of citations grouped in each pertinent
pattern using a narrative approach.
3. Results
As mentioned earlier, due to the interviewees' high expertise, the interviews were
extremely rich in ideas and insights and the text contained too many significant patterns
to be included in one paper. In order to reduce the number of patterns for the present
analysis, it was decided to limit the topic of the present paper to the challenges involved
in large- scale implementation of teaching HOT. Within this wide topic, two major
groups of patterns were identified: (a) patterns addressing challenges in large-scale
educational change in general (i.e., challenges that are visible in many areas of
educational change, see section 3.1); and, (b) patterns addressing challenges pertaining
to pedagogical issues that are more specific to large-scale implementation of HOT (see
section 3.2). In what follows the first group of patterns is described in short and the
second group is described in detail.
3.1 Challenges which are visible in many areas of educational change
The first group of patterns addressed issues involved in large-scale educational
changes in general (i.e., changes that are not specific to the implementation of HOT).
Although the term HOT appeared in the text describing these patterns, this term could
have theoretically been replaced with terms describing other educational goals without
any loss of meaning. The ideas expressed in these patterns can be found in numerous
previous studies describing educational change processes in varied contexts. These
patterns were judged as less central to the present study and are thus summarized and
described briefly in Figure 1.
--------------------------------
Page 14 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Insert figure 1 about here
--------------------------------

. The data presented in Figure 1 confirms that successful large scale
implementation of teaching HOT encounters the same general system- wide challenges
as many other large scale educational change processes in numerous areas that appear in
the literature concerning educational change (e.g., Elmore, 2004; Fullan, 2007)

3.2 Challenges that are unique to large scale implementation of HOT
The second group of patterns addressed pedagogical issues which are more
specific to the challenges involved in large scale implementation of HOT (see Figure 2).
These patterns were judged as central to the present study and therefore they are
described in detail, using a narrative approach. Since the borderline between the two
groups of patterns is not always clear-cut, my rule of thumb was that boarder - line
patterns were judged according to whether or not I estimated that the challenges
described in the patterns contributed in a significant way to the emerging discussion
about large scale implementation of HOT.
In the case of patterns reflecting assessment however, an exception had been made
to this rule of thumb. Although issues pertaining to assessment are central to large scale
implementation of HOT they were not described here in detail (but instead were
described briefly in Figure 1). Since space restrictions made it necessary to leave
something out, I chose not elaborate on assessment in this particular paper because: (a)
addressing this significant topic required an elaborate theoretical and data-based
analysis that was beyond the scope of this paper; (b) Since assessment was the focus of
several of the other papers in this Special Issue I assumed that the relative contribution
of an additional detailed discussion of assessment would be smaller than that of other
topics which were not as prominent in the other papers.
The second group of patterns are described in detail in the following sections and
summarized briefly in Figure 2.
Page 15 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
--------------------------------
Insert figure 2 about here
--------------------------------
3.2.1 The challenge of teaching thinking in varied, specific science contents
Integrating the teaching of content and thinking goals to create a coherent
curriculum is a considerable challenge even in small scale projects. However, the
teaching of HOT strategies in varied, yet specific science contents across the whole
science curriculum is a much larger challenge. The underlying theoretical framework is
that instruction needs to integrate these two dimensions (i.e., content and thinking)
according to the infusion approach to teaching HOT (e.g., Ennis, 1989; Swartz et. al.,
2008). Accordingly, teaching thinking should be woven into the teaching of
conceptually rich science content. Moreover, the theoretical approach is that thinking
goals should be conceived of as explicit, distinct educational goals that should be
discussed explicitly in the classroom while engaging with authentic, rich science topics.
L1 formulated this theoretical approach by saying: "We are talking about direct
teaching of thinking, about teaching according to the infusion approach - content
knowledge together with procedural knowledge". By the time this study took place (the
interviews took place during the summer of 2012), this approach is no longer an
innovation in Israeli science classrooms, because, as L2 states: "Explicit reference to the
need for infusing thinking skills has been on the agendasince 1996". Yet,
implementing this approach across the whole school system encounters difficulties on
many levels:
"This whole business of anchoring the thinking in the science topics was not
entirely clear, nor was it emphasized There were a lot of discussions around this issue
but there was no clear policy and no clear indication of how to actually carry it out.
Eh In the context of the new syllabus [i.e., the syllabus written in 2009- 2010] the
notion was that thinking skills should be combined [with the science topics] in a
systematic way. But ... at least at the beginning it was not clear where [i.e., in what
science topics] to combine what [thinking skills] and how to perforn it in a spiral way
This whole issue of science content and thinking skills, and of teaching them in an
integrated way, this is the discourse you hear on the policy level. You would like to
Page 16 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
construct it in a way that the thinking skills will support content knowledge and the
content knowledge will support the thinking skills. But nobody knows exactly how to
do it... [L2).
"I think that in the [biology] curriculum content goals appear separately. Actually,
I think it's the same in the junior high school science curriculum. Yes, there was an
effort to connect the thinking skills into the blueprint of what needs to be taught [in
terms of content], but there is no indication of how to integrate them" (L4)
"So these needs [i.e., the needs to combine the teaching of content and thinking]
came up, but people did not really know how to handle them" (L6]
These excerpts confirm that although the underlying theoretical framework was
clearly that instruction needs to integrate the content and thinking dimensions, in effect,
over the years content goals and HOT goals were detailed in curriculum documents in
two separate lists. There was a list of content goals and a separate list of thinking goals,
with no indication for how to combine them. Moreover, it seems that this lack of
indication was not incidental. Rather, it reflected gaps in pedagogical knowledge even
among academic experts. It seems that this type of experts' pedagogical knowledge has
been developed over the years so that they knew more about it in recent years
compared to several years ago. Yet, it seems that there is still a long way to go because
even at the time the interviews took place experts did not feel they had a clear and
systematic method for how to integrate the teaching of thinking and of science content:
"The fact that we now have a better methodology [for integrating the teaching of
thinking with science content] than we used to, does not mean that we now have a good
methodology. I don't think it [i.e., a good methodology] really exists. It's something that
people are currently developing, and will continue to develop in the future [L6]
In conclusion , it seems that the challenge of integrating the teaching of thinking
in multiple, varied science content had not yet been resolved even in the minds of the
experts who were leading the implementation processes.
3.2.2 Planning a reasonable and coherent developmental sequence
Matching the thinking goals of a small-scale project to students' age and
developmental stage is a challenging yet manageable task. This may be done based on a
Page 17 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
combination of research-based recommendations and educators' experience and on their
intuition regarding the type of learning which is appropriate for students of various ages
and cultural backgrounds. Planning a whole curriculum encompassing varied ages and
school subjects, however, is a much more complex endeavor. In considering the "when"
dimension of teaching thinking, Alexander & al. (2011) briefly summarize the vast
developmental psychology literature indicating that age can have significant effects on
mental processing. Yet, Alexander and her colleagues emphasize that the level of
thinking competence depends upon a variety of additional reasons, such as the amount
of scaffolding provided to students during instruction, the learners' prior experiences
with HOT, their level of subject-matter expertise and more. One particular post-
piagetian model that deserves mentioning in this context is Fischer's Dynamic Skill
Theory which postulates that a simple age-related stage theory is too simple to explain
the vast variability observed in human psychological traits. Instead of viewing stages of
cognitive development fixed like the steps of a ladder, this theory assumes a more
dynamic metaphor for development that of a constructive web. Unlike the steps in a
ladder, the strands in a web are not fixed in a determined order but are the joint product
of the web-builder's constructive activity and the supportive context in which it is built.
Therefore, the support (e.g., types of learning experiences and amount of engagement)
that the educational system may provide to develop children's thinking abilities in
younger ages may have detrimental effect on the types of thinking tasks that would be
appropriate for them in later ages (Fischer and Bidell, 2006). Under these complex
conditions it is therefore not surprising that the issue of creating a coherent
developmental trajectory of thinking goals across the curriculum stood out as a huge
challenge for the experts who participated in the interviews.
One major challenge is that it is not easy to determine the levels of difficulty of
science thinking tasks. This challenge came up both in elementary and high school
science. L1 expressed this challenge when describing the work she did with elementary
teachers on asking/posing questions:
"We have this exercise in which we invite teachers to ask questions on a [given]
text. Sometimes the questions are shallow and low-level, but sometimes we are
beginning to receive complex, high-level questions which are really challenging and
demanding. So how can we create a reasonable progression?" [L1]
Page 18 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Similarly, L5 expressed the challenge involved in defining the meaning of a new
high school physics curriculum that will be more demanding in terms of its' levels of
thinking: "When you declare a new program with a higher level of thinking, you need to
define what does a higher level of thinking mean.". L5 emphasizes that over the years
there was no explicit theory for "why a certain task is simpler or more complex, or what
does a student need to do in terms of her thinking in order to provide an answer [to a
given task]". This is because in physics, the complexity of determining the thinking
level of a task is affected by the interaction of several confounding factors such as the
depth and quantity of mathematical understanding required for the task, the difficulty
involved in analyzing the visual information entailed in the task, and the degree to
which non-explicit assumptions are involved. In biology, L3 expressed a similar
concern regarding the level of inquiry projects in biology. She stated that it is perfectly
OK with her if her high school students choose to investigate the effects of salt
concentration on seed germination for their inquiry projects. But such a research project
could also apply to elementary school, and there needs to be a clear difference between
the requirements for these two age groups. For instance, in high school, she would like
students to get into the micro, cellular level rather than stay on the general, macro level.
However, L3 reports that she finds that defining those requirements is not a simple task.
Consequently, it is not easy to explain it to teachers in a clear way.
Another, related, challenge is involved in creating a systematic sequence of
thinking strategies that will be taught across ages and subjects. L2 describes how in the
1996 junior high school syllabus thinking goals were addressed explicitly in the first
time as a list of thinking skills:
"People always talked about the need to do it, but this was the first time that the
syllabus had a page with a list of thinking skills, for better or for worse..[15 years
later] I still dont think we have anything much better than that. I remember we said
that, at least in the syllabus, we should address thinking skills in an orderly manner,
rather than only state that they should be included" [L2].
When the junior high school science syllabus had been updated six years later,
the issue of how to address thinking skills in a systematic way was brought up once
again: "But em at least at the beginning, it was not clear where to address which
thinking strategies and how to do it in a spiral way" [L2]. By the end of the first
Page 19 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
decade of the 21
st
century, this challenge has not yet been resolved. By now, states L2, it
has been widely accepted by the experts who plan the curriculum that the teaching of
thinking skills must be integrated into the science content. However, when the experts
attempted to actually execute this idea in the 2009 updated syllabus, they once again
encountered a huge challenge:
"You must think [simultaneously] about several content areas as well as about
quite a large repertoire of thinking strategies you need to teach during those three years
[grades 7-9]. The coordination and the spiral, and repeating the same thinking
strategy in several different contents [which is significant for students' ability to achieve
transfer], and the various levels of thinking, and the adaptation to a variety of learners-
this is a huge challenge even for experts. We certainly can't expect teachers do be able
to do it on their own" [L2].
Interestingly, at the time of the interviews (which took place 16 years after the
1996 syllabus first included an explicit reference to thinking skills), the development of
a draft of a detailed sequence of science inquiry thinking skills for grades 1-9 has been
in progress. L1 and L8 both described this draft document stating that it is being
constructedt in a spiral way, detailing for each grade level 3-4 thinking strategies which
needs to be taught in an explicit, focused way. In addition, the document attempts to
map the curriculum and prevalent textbooks, suggesting content areas and activities in
which it is appropriate to engage in specific thinking strategies. However, since at the
time of the interviews this document was still only a draft, a deeper discussion of this
effort was precluded.

3.2.3 The priority of content goals over thinking goals in policy documents and in
their implementation
The fact that on the pedagogical level it was not entirely clear even to academic
experts what are the best methodologies for integrating HOT goals into the teaching of
science contents, was perhaps one of the factors contributing to the ambiguous
messages sent by policy makers. In the earlier years, it seemed that the policy
documents themselves suffered from inconsistency in this area:
Page 20 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
The process of trying to write a syllabus for junior high school took many years,
and it [i.e., the inclusion of thinking skills] went back and forth. Thinking skills were
put in, but then they were taken out again.(L2)
However, even after the message in the policy documents had become more
coherent so that curriculum documents emphasized HOT skills as explicit and clear
goals, policy makers as well as teachers still devoted much more resources to teaching
content goals then to teaching thinking goals. When describing the process by which the
schools were instructed by the NSS office how to teach the prescribed curriculum, L8
(who is a practitionaire working in the Ministry) said the following:
"We took out the most important science topics from the curriculum of each
school grade. We counted the number of hours allocated to each topic in the curriculum
and practically divided the school year in terms of what should be taught at each part of
the year. Now as you can see here [pointing to a document written for teachers and
found on-line] - it's all content. There is nothing here about skills. It's true that the
Introduction to the curriculum says that "skills will be studied in an integrated way"
[reading from the introduction]. But there is no policy document showing that [the NSS
office] supported the teaching of thinking in any structured or explicit way" [L8].
L8 then described how following low achievements in the TIMSS and PISA
international tests the Minister of Education and the General Director decided to
allocate more resources to improve science achievements. She stated that the goal of
that program was defined as improving "science knowledge and skills". Despite this
definition, said L8, the field work that followed the policy statement centered almost
exclusively on improving students' knowledge:
"We focused more on content, on what to teach, which topics, principles,
concepts, phenomena and scientific processes [and] much less on the skills, even
though the whole policy move was oriented towards raising achievements, i.e.,
improving knowledge and skills. During the first two years we were working almost
exclusively on the content After two years we said, OK, the teachers are already
teaching the required science topics, we now have to start working on the science skills"
[L8]

Page 21 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
This report was corroborated by an academic who had been extensively involved
in developing learning kits for teachers (developed as part of implementing that same
policy):

"The [recommended] time lines [for teaching] were arranged around contents.
Now you need to teach energy, now this and now that. Developing the teaching and
learning kits followed a similar pattern. The time-line for their development was so tight
that it was not possible to treat the skills in an appropriate way. I mean, more work was
done on the conceptual dimension. The treatment of the skill dimension was much less
serious from now on, in the next several years, there will be an opportunity to do this
because the learning materials will begin to integrate thinking skills in a more serious
way [L2].

These excerpts show that a similar picture was portrayed by an academic and a
practitioner: although policy documents called for the integration of HOT skills and
science contents, work on implementing this policy across the school system centered
initially on the science content, shying away from the HOT skills. This indicates that the
content goals were perceived as more important and tangible. Only after two years of
extensive work on implementing the new science contents, attention turned to
integrating the thinking into the science contents. During those first several years, a
substantial budget was allocated to raising science scores. This budget was, however,
temporary. By the time this paper is being written, the special budget had already been
reduced in order to attend to newer educational policies in other areas. The big question
is whether by the time the system would be ready to treat the thinking skills seriously,
enough of this special budget will still be available for the substantial processes required
for the implementation of HOT across the school system.
3.2.4 The challenge of human capital: the intricate nature of large scale
professional development (PD) in the context of teaching HOT
Like many recent scholars worldwide (e.g., Barber & Mourshed, 2007), all the
interviewees in the present study viewed teachers as a crucial element in successful
implementation of HOT in science classrooms. They brought numerous examples of
teachers' participation in committees that engaged in policy making, in planning
curricula and in planning the implementation process. These examples affirm that
Page 22 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
representation of teachers in various national committees responsible for designing
these processes has become a common practice. Yet, similar to the findings of the
previous studies discussed in the literature review , the interviewees see the knowledge
teachers need in order to become proficient in teaching HOT as extremely challenging
and complex. The development of that knowledge is in essence a pedagogical process
which is not easy to achieve even on a small scale project. When this pedagogical
challenge encounters the constraints inherent to large scale implementations, it becomes
even more demanding. The interview data point to two main organizational constraints
characteristic of large scale implementation that affected the feasibility of the deep and
long-term learning teachers need: (a) the amount of time required for deep learning of
the relevant complex knowledge relative to the small number of hours allocated to that
learning in teachers' workshops (see section 3.5.1); This problem was corroborated by
shortage of resources for school-based support for teachers; and, (b) a shortage in a
stable group of highly trained and professional instructors who could lead the work with
teachers (see section 3.5.2)
3.2.5 Conflict between scope and complexity of knowledge goals for teachers and
duration of PD workshops
The enormity of the pedagogical challenge involved in developing the needed
teachers' knowledge conflicted with the organizational structure that was available for
teachers' PD: the administrative infra-structure for PD simply did not allow enough time
to address this complex pedagogical challenge in a satisfactory manner. This idea was
expressed in the interviews in many different ways. L1 who was intensively engaged in
leading teachers' PD workshops elaborated on this issue:
"Getting to know the [thinking] strategies is not enough. You must also learn to
understand their significance, to know how to adapt them to your own classroom's
needs, to experiment with them to be able to lead reflection processes with students
and with other teachers, to evaluate students' work, to be able to give them feedback.
All these elements which are entailed in teaching HOT are not part of the culture we are
used to for example, what feedback will a teacher write to her student when she is
grading a test? There is a lot of work here, you know How do you work with
teachers on acquiring all these tools? Its really really not that simple and we don't
have the time to do it".[L1]
Page 23 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
In elementary school, an agreement with the teacher union dictates that teachers'
participation in PD would be limited to 30 hours per year. Some of these 30 hours must
be devoted to content knowledge. The remaining hours are viewed as insufficient for
meaningful learning of the complex knowledge teachers need in order to make the
transition to a thinking rich classroom:
"Throughout their professional lives they are used to teaching declarative
knowledge. The curriculum always addresses declarative knowledge. When they write
lessons plans, it's always about declarative knowledge. What do I have to teach
tomorrow? Take a look at teachers' lesson plans and you will see: The respiratory
system, or this or that system Their head is tuned in this direction. Suddenly they are
hearing from us: "No. Not only declarative knowledge. From now on you also need to
address procedural knowledge". This is a paradigm shift. If a science teacher attends
[a workshop] for thirty hours of PD thirty hours means only seven or eight meetings,
and part of the time is devoted to science content. You can't expect a meaningful
learning process in thirty hours What also seems to me extremely important in
assimilating a culture of thinking is to work on habits of mind and on thinking
dispositions so that they would become part of the classrooms culture. I believe it is
extremely important to do so rather than to just be satisfied only with constructing
distinct thinking strategies [L1].
L1 then continue to describe the complex demands of inquiry learning. The
knowledge teachers need to be able to lead inquiry learning is not only complex in itself
compared to the duration of the workshops, but it becomes even more difficult to
address it appropriately in the face of competing and rapidly changing policies:
It's also important to work on the development of complex thinking processes
[such as inquiry] and to give them tools to engage in them.eh but we don't have
enough time to do all that- You can't do all that in thirty hours. And there are always
new policies. This year the NSS declared that inquiry learning is a priority.[quiet for a
while].But along with inquiry they had also decided to promote the subject of health.
You can't do it all at once. Ehh.. in many of the courses this year we worked with
teachers on how to carry out a complete inquiry process from beginning to end. I mean,
starting with encountering a phenomenon, asking questions this is a very difficult
process. We try to make them aware of the complete sequence of inquiry teaching. But
Page 24 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
this is not enough. We are missing the ability to support them once they are back in their
classrooms. They come to the workshop and they study but I don't know exactly what
they absorbed. The fact that they nodded and told me how exciting this is I don't
knowI am afraid there is a large gap here." [L1]
3.2.6 The challenge of securing a highly-proficient, system-wide, stable
infrastructure of instructors
The second significant challenge in this section pertains to the problem of
securing a highly-proficient, system-wide, stable infrastructure of instructors who can
assist in PD workshops and provide school-based pedagogical support for teachers. The
roles of these professionals include teaching in teachers' PD workshops; participation in
teams that develop model learning materials and assessments; and visiting classrooms to
provide school based feedback and support for teachers. This level of work obviously
requires knowledge that needs to be even more proficient than the knowledge teachers
need for classroom instruction. It therefore takes quite a long time to prepare the high-
quality practitioners needed to fulfill these roles. In all 4 areas (general science, physics,
biology and chemistry) the interviewees reported that they see instructors as an
extremely important link in the implementation process and that considerable resources
were invested in their PD:

"Yes. A second strategy which I really believe in pertains to instructors. Those
who went through the PD courses" (L5)

"Implementation, in all areas, but also specifically in the area of teaching HOT,
takes place on several levels. The first level is that of the NSS and her instructors...
They form a support group. A group that brings up new ideas andtries them out in
experimental pilot projects. This is the group of people with whom I do much of my
work I have 22 instructors. Plans are made together with the instructors who form
the "implementation fan". Then they meet with groups of teachers in the various
districts and disseminate the decisions and plans that we made. Usually we first try it in
pilot projects and then disseminate it to all teachers. [L3]
Page 25 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t

Leading teachers and instructors are usually chosen from teachers who are much
above average in terms of their pedagogical abilities, i.e., they are known as "star"
teachers. Yet, ample resources are often devoted to their long term PD:

"In order to actually implement this policy document [i.e., about teaching HOT],
we first needed to train instructors. We had a complete training system. We trained a
whole group of instructors in fostering HOT" [L1]

"Some groups of instructors met regularly for 6-7 years, other groups met only for
3-4 years.[L2] ".

Following such long term PD workshops, instructors were indeed capable to
support teachers in developing the ability to teach for thinking and understanding:

"Wherever we had projects that supported teachers in their field work we found
really interesting things For instance, [in a Ph.d study conducted under the
supervision of L2], we followed the development of teachers with whom we had
worked for several years. We found a huge development. Then we had quite a few
courses for leading teachers where we also saw a substantial development. It's not true
that you cannot help teachers make progress in this area. But we came to the conclusion
that it's very difficult to do that on a large scale"[L2]

In sum, the basic rationale for building the infra-structure of instructors was to
support the ability of scaling up. The assumption was that this group of professionals
would be able to work with relatively large numbers of teachers as leaders of PD
workshops and as mentors of school-based instruction. Experience as well as a formal
assessment study indicated that it was possible to help this group of excellent teachers
Page 26 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
make a huge development in their pedagogical knowledge in the context of teaching
thinking. Why then, the pessimistic conclusion regarding the scalability of teachers'
development in this area? The answer came up in many of the interviews when the
interviewees described examples of how various organizational obstacles blocked the
flow of knowledge from instructors to teachers: First, it took several years to prepare
high-quality practitioners who had enough knowledge and experience to guide teachers
in the complex PD processes that needed to take place. Then, there were several
organizational reasons that made this group of experts unstable with large turn-over,
including the following: a. Frequent change of policies required frequent transitions to
other areas of teacher support. Therefore, often after making the long-term investment
in the development of instructor's knowledge in the context of teaching HOT they were
then assigned to work with teachers on other areas (such as ICT or content goals) and
the investment in the development of their knowledge in the context of teaching HOT
was lost; b. Inherent difficulties involved in working with teacher combined with low
salaries for instructors (relative to the time they needed to invest in carrying out their
role), caused a rapid turn-over of instructors. Their less than optimal working conditions
encouraged many of these excellent people to leave their position after a relative short
period, thereby contributing to a disrupted flow of knowledge between policy
documents and classrooms.
3.2.7. Challenges pertaining to metacognitive knowledge
Another important yet complex knowledge element that was found to be difficult
to address during the PD workshops in a satisfactory manner is metacognitive
knowledge. As explained earlier, metacognition is "thinking about thinking" . This
means that in order to be able to engage in metacognitive teaching about HOT, teachers
must first be familiar with the pertinent cognitive processes i.e., with the strategies
involved in the thinking that needs to take place in their classrooms. In addition,
teachers must also acquire the special pedagogical knowledge that pertains to
metacognitive teaching. L1 described how addressing the various components of
metacognition during the PD workshops presented additional challenges:
"Teachers must first experience metacognition in their own learning, they need to
experience learning processes that include metacognition. Then we need to help them
construct two types of meta-level knowledge: First, the meta- strategic knowledge of
Page 27 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
the thinking strategies which they are not familiar with. Second, the meta-strategic
knowledge necessary for teaching the pedagogy of how to teach metacognition. If
a teacher is not familiar with the "thinking map" [i.e., with the meta-strategic
knowledge of a thinking strategy] it will be very difficult for her to construct a
metacognitive teaching strategy because these two things go together. [quiet]. And
we don't have enough time to do that. We have only 30 hours and we can't fit this in
in a meaningful way"[L1]
Over the years L1 had substantial experience managing numerous elementary
science teachers' workshops. Based on that experience she described how she had
repeatedly witnessed teachers' gaps in knowledge of thinking strategies. She also noted
that the workshops' limited number of hours hindered teachers' ability to acquire the
necessary pedagogical knowledge for teaching metacognition. L1 had been aware of
these limitations. Yet, she sadly stated that the PD workshops she was leading could not
adequately address these knowledge gaps. The problem from her point of view, i.e.
from the point of view of a highly qualified professional who planned the curriculum for
teachers' PD, was that the complexity of the metacognitive knowledge teachers needed,
the fact that it must be built on a prior solid familiarity with thinking strategies , and the
fact that the duration of the PD was limited, contributed to the difficulty of addressing
metacognition in an appropriate way. She then continued to describe how in some of the
courses, they did try to address metacognitive teaching. Yet, she was not happy with the
result. She viewed metacognitive teaching as so challenging that she was skeptical as to
the effect that a limited duration of engaging with the relevant knowledge could have on
teachers' practice. The fact that there was no budget for classroom supervision and
mentoring mades her even more skeptical as to whether the workshop indeed made it
possible for teachers to be able to actually apply metacognition in their classrooms.
A similar gap was also reported in the case of the ability of high school chemistry
teachers to apply metacognitive teaching in their classrooms:
"Metacognition appears in our instructional unit as "Time to Think".., i.e., the
unit includes a section which shows students how to think, a kind of reflection But
not all teachers can work with this section. I just gave a lecture in a workshop for
chemistry teachers and I was astounded to discover that many teachers, even though it
appears in the textbook, do not really understand it. It's not only that they do not teach
Page 28 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
the metacognition in the relatively simple way it appears in the instructional unit, but
they dont even know how to use it for monitoring their own thinking and for
monitoring their students' responses.even though it appears in the textbook. Well, we
know that the fact that it is written in the textbook does not yet mean that it is present in
the classroom" [L7].

4. Summary and discussion
This study illuminates the challenges involved in large scale implementation of
HOT in science education as they are viewed by a group of leaders who were engaged
in implementation processes in an intensive way. However, since this paper focuses on
challenges, it may give a misrepresentative image. At the outset of the discussion
section it is important to emphasize which conclusions should NOT be drawn from this
study. Focusing on the challenges does not mean that the processes we are studying
were not effective or that large scale implementation efforts in this area are doomed to
failure. The analysis presented here is NOT an evaluation study of the various attempts
to teach HOT in science education in Israel. Although a comprehensive evaluation study
of these efforts had not yet taken place, studies and reports examining sections of these
efforts show considerable effects and progress (see for example Avargil & al., this
volume, Office of Pedagogical Affairs, MOE, 2009; Spector-Levy, Eylon & Schertz,
2008;Gallagher, Hipkins and Zohar 2012). Despite these positive effects, however, the
implementation processes under consideration are far from being completed. Although
considerable developments in the desired direction have been taking place, it is clear
that if the aim is for thinking- rich instruction to become a routine in all science lessons,
there is still a long way to go. The gradual nature of these processes therefore should
allow us to recognize progress but still be concerned by the challenges hindering further
improvements.
Indeed, the data indicate that the route to thinking rich instruction in all
classrooms is neither short nor smooth. The challenges described in this study stress the
fact that introducing HOT to science classrooms does not entail "all or nothing"
processes because implementation processes of new pedagogies are often evolutionary
rather than revolutionary (Dede, 2006; Cohen, 2010)).Cohen (2010) explains that the
answers provided by educators regarding the feasibility of sweeping pedagogical
Page 29 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
changes are not satisfying. He argues that the question regarding the likelihood of
pedagogical change and its nature needs to be asked from a broad historical perspective,
rather than from a limited view of one isolated link in the chain of time. If we consider
the magnitude of the pedagogical change required to make the transformation to a
thinking-rich curriculum, we must consider the possibility that we are only at the
beginning of a long journey, and that learning from implementation in the field will be
slow. In Cohen's opinion, it is likely that those who strive to promote instruction that he
calls adventurous, are in effect trying to bring about the beginning of a great, slow
change in the perception of knowledge, learning and instruction. This future change,
however, is still in its infancy and needs many more years to materialize. Cohen argues
that the early stages of such huge system-wide changes are characterized by examining
alternatives, inventing new patterns of which only few will survive, and developing
ideologies and strategies for change. Thus, complex pedagogical changes that involve
deep changes in teachers' knowledge and beliefs regarding learning and instruction, are
more correctly viewed as long-term, slow, evolutionary processes, rather than
revolutionary processes. According to this view, it is important to study the challenges
involved in such long-term change processes because understanding them may be
informative for improving future system-wide implementation efforts.
Part of the challenges encountered by the leaders who were interviewed for this
study are common to many types of educational change processes (see figure 1). In
addition, this study revealed several meaningful challenges that are specific to the
educational effort of applying HOT according to the infusion approach across large
numbers of classrooms (see figure 2): challenges involved in weaving HOT into
multiple, varied, specific science contents; challenges involved in planning a reasonable
and coherent developmental sequence of thinking goals; the fact that content goals tend
to have priority over thinking goals and therefore to disperse of the latter in policy
documents and in implementation processes; the challenges involved in supporting
teachers' ability to teach metacognition; and finally, the huge challenges involved in
developing educators sound and deep professional knowledge in the area of teaching
HOT.. Although challenges related to PD are by no means unique to the area of
teaching HOT, the specific nature of these challenges in this area is deeply rooted in the
unique nature of teaching thinking. These challenges are aggravated by lack of adequate
organizational support.
Page 30 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Several reasons make metacognition an exceptionally difficult topic to implement
in system-wide PD workshops which almost always suffer from a shortage of hours: the
fact that metacognitive knowledge is itself quite complex, that chronologically speaking
it can only be addressed after participants had acquired a certain degree of knowledge
concerning the cognitive level of using thinking strategies, and the fact that
metacognitive teaching requires particular teaching strategies. Previous studies show
that when a small-scale project addresses metacognitive teaching in a focused way, it
can indeed be effective (Author,, 2006; Veenman, 2011). But when moving to a system-
wide scale, the challenges involved in metacognitive teaching (relative to the many
other important goals that teachers' workshops need to address during a limited period)
make it an especially difficult issue to address in an appropriate way.
More generally, the data indicate that the interviewees viewed the teaching of
HOT according to the infusion approach as requiring complex teachers' knowledge.
They believed it requires a high level of content knowledge, sound knowledge of
thinking strategies, knowledge of complex thinking processes such as problem- based
and inquiry learning, knowledge about the culture of thinking, knowledge about
thinking dispositions, and acquaintance with a variety of pedagogical tools that are
specific to the teaching of HOT.
Due to the magnitude of the shift required in teachers' knowledge while they
make the transition to teaching thinking, one of the interviewees stated that it merits the
label of a "paradigm shift". Addressing this paradigm shift in an appropriate way
requires long-term learning with highly trained instructors. Organizational constrains
which are typical to large scale implementation made it difficult to support the required
long- term teachers learning and therefore hindered the development of teachers
knowledge. These organizational constrains were most apparent in the number of hours
devoted to teachers' learning and in the limited support for instructors which hindered
the feasibility of creating a stable infrastructure of highly professional instructors.
These findings have several practical and research-oriented implications. The
challenges brought up in the interviews point to the directions that future
implementation efforts and future research should center on (in addition to the activities
already conducted in the implementation processes described earlier). First, several of
the findings show gaps in experts' knowledge, indicating that even the experts do not yet
Page 31 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
have a coherent systematic model and practical plan for weaving HOT into the topics of
the science curriculum over the span of the school years. Consequently, the leaders of
the implementation need to invest time as well as Research and Development funds in
order to: a. conceptualize theory-driven models for weaving HOT systematically into
multiple, varied, specific science contents and, b. for planning a reasonable and coherent
developmental sequence for teaching thinking goals across different ages and student
populations. Subsequently, various such models may be tried out and evaluated
empirically so that decision making as to the optimal model of implementation in
various educational contexts and for different student population would be evidence-
based. At the moment very little research has been conducted around these issues.
Second, leaders and teachers have to learn how to view thinking goals in a way
that would render them as important and tangible as content goals.
Third, the findings have important implications concerning metacognition.
Although ample research conducted in laboratory or small- scale studies in schools
show that metacognition has tremendous effects in improving students reasoning,
hardly any research has been conducted on the effect of metacognitive teaching across
the whole school system. The difficulties shown here concerning teachers' knowledge
and PD in the area of metacognition point to a dangerous potential hazard in scaling up
the teaching of metacognition. More research is needed in order to understand what
elements of metacognitive teaching may be more amenable for relatively short-term PD
and thus less sensitive to the "dilution" (i.e., loss of focus and meaning) which may take
place in large scale implementation. Such research is crucial for a conceptualization of
just what components of metacognition can we hope to teach in a sound way across a
large number of classrooms and just how to conduct wide scale PD in the area of
metacognition.
Finally, and perhaps most importantly, the findings have important implications for PD.
in the context of teaching HOT. The mismatch between the scope of the required PD
according to pedagogical considerations and the administrative and organizational
support for its execution seems to be a major impediment in the ability to apply policies
in this area in a successful way. Recommendations for overcoming this impediment
center on increasing the weight of PD as a determining factor in any implementation
process in this area. The details of the recommendations depend upon the political level
Page 32 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
supporting the relevant educational policy. If the educational policy advocating
thinking-rich instruction is supported by a stable (in terms of longevity), high-level
political entity that sees education for thinking as a primary goal and has the power to
assign sufficient organizational and financial resources to the implementation process,
then a systematic plan of implementation can be based upon the pedagogical needs for a
high quality and lengthy PD. In this case, pedagogy may drive the planning of
administrative and organizational structures and of an appropriate budget that can
support the complex pedagogical needs. Examples of such comprehensive and long
term support for PD may be seen in the successful large scale efforts of pedagogical
reforms that took place in locations such as Finland or Ontario (Darling-Hammond,
2010; Fullan, 2007). In addition, under such favorable conditions, rational and
systematic models for taking innovations to scale are in place (Lee and Krajcik, 2012)
In many cases however, conditions are less favorable and high level political
support in policies advocating instruction of HOT does not exist. In the case of the
Israeli efforts to foster policies advocating thinking rich instruction in science education,
the thinking curriculum had been embraced by the highest levels within the Ministry of
Education, but it had never been a primary goal of a high level and stable (in terms of
longevity) political entity. I suspect that efforts to implement the thinking curricula in
other countries often operate under similar, less than optimal, conditions. In such cases
it is unlikely that the optimal organizational and administrative conditions needed for
profound PD will be met. Consequently, a gap between the desired and actual teacher
knowledge in the context of teaching thinking will be created and at least some of the
pedagogical goals of the thinking curriculum would have to be compromised.
Since this gap in teacher knowledge is likely to affect the implementation process
of teaching HOT in a fundamental way, it is important to recognize it and take it into
consideration when planning large scale efforts in this area. Some possible directions
for addressing this problem include the following: a. Along with the recommendations
of numerous previous reports (e.g., Darling-Hammond, 2010; Elmore, 2004; Fullan,
2007) , this study calls to give PD priority over other goals; In particular, the findings
point to the need to improve the structural and organizational infra- structure necessary
for supporting deep and long term teacher learning. ; b. An implementation plan cannot
be deeper than the depth of the PD it can support. When a realistic assessment of the
organizational infra-structure for PD processes indicates that PD would necessarily be
Page 33 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
limited, the thinking goals of the large scale implementation should be re-considered
and adjusted. In such cases, it may be worthwhile to consider more modest thinking
goals for the whole enterprise which may be achievable in a relatively sound way rather
than to stick to more ambitious goals which may be in danger of superficial
implementation. For example, an evaluation study of a policy that advocated inquiry
learning in elementary schools in Israel in the 1990s revealed that inquiry learning had
become quite common in elementary schools so that most schools exhibited some form
and scope of inquiry learning. Yet, that study also revealed that in most schools inquiry
learning was limited to shallow, surface elements of inquiry. The emphasis was on the
product, i.e., on a written paper rather than on the process of inquiry learning. In
many cases the product was the result of cut and paste processes that students were
involved in, often with addition of nice graphics, rather than the result of the deep
thinking processes that are fundamental to any deep inquiry process (e.g., asking fruitful
questions, planning solutions, integrating information from multiple sources, processing
data, drawing valid conclusions, considering evidence, formulating arguments, etc.).
According to the study, the main reasons for the shallowness of students inquiry
learning were teachers limited conceptions and understanding of inquiry and limited
pedagogical knowledge of how to lead deep inquiry processes in the classroom (Gordon
& al., 2003). When approximately 10 years later a new thinking curriculum was
considered for elementary schools (as part of the Pedagogical Horizons policy, see
Gallagher, Hipkins and Z0har, 2012), the findings of that study were taken into account.
Although theoretical considerations gave top priority to the goal of implementing
inquiry learning, a practical assessment of the available resources and support led to the
conclusion that the deep and long term PD that was judged to be crucial for sound
implementation of inquiry learning among elementary school children will not be
feasible. This practical consideration led to the formulation of a much more modest
thinking curriculum that centered on e three thinking strategies (posing questions,
making comparisons and argumentation) rather than on a full inquiry process.
In terms of research, the gaps that often exist between the pedagogical needs of
PD in this area and the available resources for such PD open the gate to a new set of
questions: if only a limited number of hours of PD devoted to HOT are available each
year, what are the most important things to work on? What elements of teachers
knowledge can we hope to teach meaningfully with limited resources for PD? Which
Page 34 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
elements of teacher knowledge that may seem significant based on findings from small
scale projects, are the most resistant to the "diluting" phenomena inherent to scaling up
and therefore should be recommended for wide implementations? Future studies that
will address such questions may enable evidence- based decision making when planning
large scale PD processes with less than optimal conditions.
In summary, these ideas suggest an alternative strategic direction for taking
teaching thinking to scale. This paper emphasizes the magnitude of the challenges involved in
scaling up the thinking curriculum. Obviously, whenever there is a sound and strong
support for large scale implementation in this area, ambitious and revolutionary goals
can and should be aspired for. However, whenever there is only a limited amount of
support in a policy aspiring to take HOT to scale, the magnitude of the challenges
involved suggest that it may make more sense to lower expectations by aspiring for
more modest goals rather than expecting to replicate the effects of small-scale
successful projects, This approach suggests that the huge goal of implementing
teaching thinking across a whole system may need to be implemented in small steps,
according to an evolutionary rather than a revolutionary model. It means that achieving
our far-reaching, ultimategoal will take time (Cohen, 1988) and will involve a
succession of interim sub-goals. This "small- steps", evolutionary model for scaling up
the thinking curriculum opens the door for many new empirical and theoretical
questions that may be investigated in future studies.
.
Finally, a note regarding the main limitation of this study is in place. Since the
study was conducted in one specific educational context with only seven interviewees,
generalizations based on its findings are limited. In particular, it should be remembered
that the study was conducted in an educational system which is quite centralized. The
findings therefore may be less relevant to decentralized educational systems where wide
- scale implementation processes of the thinking curriculum may take on considerably
different routes. Future studies may thus also ask what are the similarities and
differences between wide scale implementation of HOT in centralized and less
centralized educational systems.
Page 35 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Figure 1: A summary of category A patterns
----------------------------------------------------------------------------------
1. Vision and leadership are needed to advance HOT goals. The driving force for leading implementation processes
should be a long-term vision rather than efforts to respond to the many frequent, ad hoc problems emerging from the field, injuring
the developing of the vision.
2. Public support of chief administrators in the process is crucial. They need to state publicly that the educational
goal of developing students' HOT is a prominent one and to post it high on their list of educational agendas.
3. A large gap exists between statements and policy documents on the one hand and what actually takes place in
practice on the other hand.
4. Other policies and canges matter. Seemingly unrelated policy decisions, organizational structures and various
changes from the past may have a strong impact on the implementation of HOT. For example, processes that took place in Junior
high schools, resulted in improved physics science content knowledge. This seemingly unrelated process affected the feasibility of
teaching thinking high school physics.
5. Bureaucracy and Budget constrains limit the implementation process. It is most crucial in covering extra
teaching time (because more hours are needed to implement the change successfully), in supporting teachers' PD(through release
time or other financial incentives for devoting the time), and in providing instructors to support teachers' classroom practice.
6. When the implementation encounters difficulties, when it becomes wide-scale, or simply as time goes by, the
message and requirements of the implementation may need to be simplified and compromised. There is a danger that the essence of
the change will be diluted. The implementation may then attend to surface, "technical" elements of the educational enterprise
rather than to its deep content.
7. Several years after the initiation of the change process, its main messages become diluted and need to be re-
emphasized. Sustainability requires "boosting up" of the main messages after a certain number of years to preserve the original
stamina and coherence of ideas.
8. The implementation requires organizational structures, among which are the following: an "implementation fan"
of growing expertise among instructors and leading teachers, teachers' centers, incentives for teachers to participate in PD,
appropriate technical infrastructure (such as labs and lab technicians to support inquiry learning).
9. Frequent political and organizational changes result in frequent policy changes that damage the implementation
process.
10. Academic expertise is an invaluable resource in planning, developing and implementing the change process.
Relationships between academics, practitioners and policy makers are important.
11. Seemingly disconnected processes that take place in the school system may bring up unexpected opportunities that
can be hitched to advance activities that support instruction of HOT. For example: budgets allocated to raising test scores were
intended for promoting science content knowledge. However, large chunks of that budget were used by various agents to develop
learning materials and PD that support HOT skills.
12. Assessment is a key factor. The Ministry of Education's requirements for improved quantitative outcomes for
students and teachers is an inhibiting factor. Standardized tests (national and international) inhibit and contradict the development of
HOT. The goal of improving International test scores may bring in useful budgets and activities, but test preparation may interfere
with teaching HOT. National assessments (e.g., the matriculation exam) need to be changed in order to reflect thinking goals.

Page 36 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Figure 2: A summary of category B patterns
1. The challenge of teaching thinking in multiple, varied, specific science
contents
2. Planning a reasonable and coherent developmental sequence
3. The priority of content goals over thinking goals in policy
documents and in their implementation
4. The challenge of human capital: the intricate nature of large scale
PD in the context of teaching HOT
5. Conflict between scope and complexity of knowledge goals for teachers and
duration of PD workshops
6. The challenge of securing a highly-proficient, system-wide, stable
infrastructure of instructors
7. Challenges pertaining to metacognitive knowledge
Page 37 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Appendix A- Interview Protocol
1. Can you portray an updated picture of your subject in terms of teaching for thinking and
understanding? What exists on the level of policy (ask for reference to relevant policy/curricular
documents)? What actually takes place on the level of schools and classrooms practice?
2. What implementation means were operated in the school system in order to turn the policy into
practice? [if relevant add:] What strategies do you employ in order to implement the policy?
3. Do you think that in your subject the current state of teaching thinking is the desired state? If
not- what do you think is missing?
4. Please try to observe the last 10-15 years retrospectively: How did we reach the current state of
teaching thinking? Please tell me about important events over the years that influenced teaching
thinking in [name of relevant school science subject]
5. [If did not mention the Pedagogical Horizon policy intuitively) : Did the Pedagogical Horizon
policy affect teaching thinking in [name of relevant school science subject]? [If yes] What
was the effect? [If no] Why do you think there was not effect?
6. What do you see as the main obstacles/barriers in implementing teaching thinking across the
various school subjects and content domains?
7. What are you predictions regarding the future developments of teaching for thinking and
understanding in [name of relevant school science subject]?
8. Are there any evaluation studies of implementing teaching thinking in [name of relevant school
science subject]? If so, can you please tell me about them? Can you please send me the full
study/reference ?
9. Did academics (i.e., university or college professors) contribute to the implementation process?
If so, what was the contribution?
10. What are the challenges you see in integrating higher order thinking in science content in an
authentic way? Please bring example(s)
Page 38 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
References
Abrami, P. C., Bernard, R. M., Borokhovski, A. W., Surkes, M. A., Tamim,
R.,&Zhang,D. (2008). Instructional interventions affecting critical thinking
skills and dispositions: Astage 1 meta-analysis. Reviewof Educational Research,
78, 11021134.

Adams, P. E., and Krockover, G. H. (1997). Beginning science teacher cognition and its origins
in the preservice secondary science teacher program. Journal of Research in Science
Teaching, 34, 633-65.
Adey, P.S & Shayer, M. (1993). An exploration of long-term far-transfer effects
following an extended intervention program in the high school science
curriculum. Cognition and Instruction, 11(1), 1-29
Adey, P.S & Shayer, M. J. (1994). Really Raising Standarts. London: Routledge
Alexander, P. A., Dinsmore, D. L., Fox, E., Grossnickle, E. M., Loughlin, S. M.,
Maggioni, L., Parkinson, M. M. and Winters, F. (2011). Higher order thinking
and knowledge. In: Schraw, G. and Robinson, D. R. (Eds.). Assessment of
Higher Order Thinking Skills, p. 47-88. Charlotte, NC: Information Age
Publisher.
Barber, M., & Mourshed, M. (2007). How the world's best performing school
systems came out on top. London: Mckinsey and Company.
BSCS (1963). Biological Science: Inquiry Into Life. New York: Harcot Brace
Publishers.
Bransky, J., Hadass, R., & Lubezky, A. (1992). Reasoning fallacies in preservice
elementary school teachers. Research in Science & Technological Education,
10 (1), 83-92.
Blumenfeld, P., fishman, B. J., Krajcik, J., Marx, R. W., & Soloway, E. (2000).
Creating Usable Innovations in Systemic Reform: Scaling Up Technology-
Embedded Project-Based Science in Urban Schools. Educational
Psychologist, 35, 149-164.
Brownell, G., Jadallah, E., & Brownell, N. (1993). Formal reasoning ability in
preservice elementary education students: matched to the technology
education task at hand? Journal of Research on Computing in Education, 25
(4), 439-446.
Carpenter, T. P., Lynn-Blanton, M., Cobb, P., Loef-Frank, M., Kaput, J. and McClain,
K. (2004). Scaling up innovative practices in mathematics and science.
Research report. NCISLA (National center for improving learning and
Page 39 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
achievement in mathematics and science), Wisconsin Center for Education
Research, School of education, University of Wisconsin-Madison.
Chen, Z., & Klahr, D. (1999). All other thing being equal: Children's acquisition of
the control of variables strategy. Child Development, 70, 1098-1120.
Cocharn, K. F. & Jones, A. L. (1998). The subject matter knowledge of preservice
science teachers. In: Fraser, B. & Tobin, K. (eds). International Handbook of
Science Education. (pp. 707-718). Dordrecht, The Netherlands: Kluwer
Cohen, D.K. (1988). Teaching practice: Plus que ca change ... In P. Jackson (Ed.),
Contributing to Educational Change: Perspectives on Research and Practice
(pp.27-84). Berkeley, CA: McCutchan
Darling-Hammond, L. (2010). The flat world and education: how Americas
commitment to equity will determine our future. New York and London:
Teachers College Press
Dede, C., Honan, J. P., & Peters, L. C. (2005). Lessons learned from technology-
based educational innovation. San Francisco: Jossey-Bass

Dede, C. (2006). Evolving innovations beyond ideal settings to challenging contexts
of practice. In K. Sawyer (Ed.), The Cambridge Handbook of the Learning
Sciences (pp. 551566). New York: Cambridge University Press.
Dwyer, C. P., Hogan, M. J. and Stewart, I. (2012). An evaluation of argument
mapping as a method of enhancing critical thinking performance in e-learning
environments. Metacognition Learning, 7, 219244
Elmore, R. C. (2004). School reform from the inside out: policy, practice, and
performance. Harvard Educational Press: CA Mass
Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed
research. Educational Researcher, 18(3), 4-10.
Eylon, B. & Bagno, E. (1997). Professional development of physics teachers through
long-term in-service programs: the Israeli experience. American Institute of
Physics (AIP) Conference Proceedings, 399, pp. 299-326.
Fischer, K. W. and Bidell, T. R. (2006) Dynamic Development of Action
and Thought. In: Lerner, R. M. (ed.). Handbook of Child Psychology
(6
th
edition), ch. 7, pages 313-399. Hoboken, NJ: John Willey & Sons.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of
cognitive development inquiry. American Psychologist, 34, 906-911.
Page 40 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Flavell, J. H., Miller, P. H., & Miller, S. A. (2002). Cognitive Development (4
th

edition). Upper Saddle River, New Jersey: Prentice Hall.
Fullan, M., 2007. The New Meaning of Educational Change, NY and London:
Teachers College Press, 4
th
edition.
Gallagher, C., Hipkins, R. and Zohar, A. (2012). Positioning thinking within national
curriculum and assessment systems: Perspectives from Israel, New Zealand and
Northern Ireland. Thinking Skills and Creativity, 7, 134-143.
Gess-Newsome, J., & Lederman, N. G. (eds.). (1999). Examining Pedagogical
content knowledge. Dordecht, Netherlands: Kluwer.
Gordon, D., Levin, M. R., Keinan, A. and Bar-On, N. (2003). Inquiry Learning in
Elementary Schools- A Research Report. Beer Sheva: Ben Gurion University
and Kay Teachers College.
Jungwirth, E. (1987). Avoidance of logical fallacies: a neglected aspect of science
education and science -teacher education. Research in Science and Technological
Education, 5, 43-58.
Jungwirth, E. (1990). Science teachers sponatanous, latent or non-attentace to the
validity of conclusions in reported situations. Research in Science and
Technological Education , 8, 103-115.
Jungwirth, E. (1994). Science- teachers as uncritical consumers of invalid
conclusions: incompetence or just poor performance? Paper presented at the
1994 NARST annual conference , Anaheim: CA
Lee, O. & Krajcik, J. (2012). Large-Scale Interventions in Science Education for
Diverse Student Groups in Varied Educational Settings. Journal of Research
in Science Teaching, 49(3), 271-280.
Levin, B. (2008). How to Change 5000 Schools: A Practical and Positive
Approach for Leading Change at Every Level. Cambridge, MA: Harvard
Education Press.
Levin, B. and Fullan, M. (2008). Learning about system renewal. Educational
Management Administration and Leadership, 36, 289-303.
Lin, X., & Lehman, J. D. (1999). Supporting learning of variable control in a
computer-based biology environment: effects of prompting college students
reflect on their own thinking. Journal of Research in Science Teaching,
36(7), pp. 837-858.
LoefFrank, M., Carpenter, T., Fennema, E., Ansel, E., Behrend, J.(1998).
Understanding teachers' self-sustaning' generative change in the context of
professional development. Teaching and teacher education, 14(1),67-80.
Page 41 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Loughran, J., Gunstone, R., Berry, A., Milroy, P., & Mulhall, P. (2000).
Documenting science teachers pedagogical content knowledge through
PaP-eRs. Paper presented at the annual meeting of the American
Educational Research Association (AERA), New Orleans: April
Office of Pedagogical Affairs, Israel Ministry of Education (MOE), (2007).
Pedagogical Horizon for learning. (in Hebrew).Retrieved from:
http://cms.education.gov.il/EducationCMS/Units/Mazkirut_Pedagogit/OfekPeda
gogi/ HinuchLechashiva/Hinoch_Le_Chashiva.htm (in Hebrew)
Office of Pedagogical Affairs, Israel Ministry of Education (MOE), (2009). Teaching
thinking report (Pedagogical Horizon): 2006-2009. Jerusalem: Israel
Ministry of Education. (in Hebrew).
Osborne, J. Erduran, S. and Simon, S. (2004). Enhancing the quality of argument in
school science. Journal of Research in Science Teaching, 41, 994-1020.
Partnership for 21
st
Century Skills, 2011. Framework for 21
st
Century Learning,
http://www.p21.org/overview/skills-framework, retrieved July 2012
Patton, M. Q. (2002). Qualitative Research & Evaluation Methods (3
rd
Edition).
London and New Delhi: Sage Publications.
Paul, R. W., Elder, L., & Bartel, T. (1997). Teachers of teachers: Examining
preparation for critical thinking. Paper presented at the Annual Meeting of the
American Educational Research Association. Chicago: March 14-18.
Pellegrino, J. W., & Hilton, M. L. (Eds.). (2012). Education for Life and Work:
Developing Transferable Knowledge and Skills in the 21st Century.
Committee on Defining Deeper Learning and 21st Century Skills; Center for
Education; Division on Behavioral and Social Sciences and Education;
National Research Council. Washington D.C: National Academic Press

Perkins, D. N. and Salomon, G. (1989). Are cognitive skills context-bound?
Educational Researcher, 18, pp. 16-25.
Perkins, D. N., Jay, E., & Tishman, S. (1993). Beyond abilities: a dispositional theory
of thinking. Merrill-Palmer Quarterly, 39(1), pp. 1-21.
Resnick, L. (1987). Education and Learning to Think. Washington D.C.: National
Academy Press.
Resnick, L. (2010). Nested Learning Systems for the Thinking Curriculum.
Educational Researcher, 39, 183- 197.
Page 42 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Schraw, G. (1998) Promoting general metacognitive awareness, Instructinal Science,
26, 113-125.
Schraw, G. & Moshman, D. (1995). Metacognitive Theories. Educational Psychology
Review, 7(4), pp. 351-371.
Schraw, G., McCrudden, M. T., Lehman, S. and Hoffman, B. (2011). An overview of
thinking skills. In: Schraw, G. and Robinson, D. R. (Eds.). Assessment of Higher
Order Thinking Skills, p. 1-18. Charlotte, NC: Information Age Publisher.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching.
Educational Researcher, 15, 4-14.
Shulman, L. S. (1987). Knowledge and Teaching: foundation of the new Reform.
Harvard Educational Review, 57(1), pp. 1-22.
Spector-Levy, O., Eylon, B. & Schertz, Z. (2008). Teaching Communication skills in
science: Tracing teacher change. Teaching and Teacher Education 24, 462477

Swartz, R. J., Costa, A. L., Beyer, B. K., Reagan, R. and Kallick, B. (2008). Thinking-Based
Learning. New York and London: Teachers College Press.
Tamir, P. (2006). Inquiry in Science Education and its Reflection in the Israeli
Biology Teaching. In Zohar, A. (Ed.), Learning by Inquiry: An Ongoing
Challenge. Jerusalem: Magnes, pp. 1556. (in Hebrew)
Van Driel, J. H., Verloop, N., & de Voss, W. (1998). Developing science teachers
pedagogical content knowledge. Journal of Research in Science Teaching,
35, 673-695.
Veenman, M. V. J., Van Hout-Wolters., B & Afflerbach, P. (2006). Metacognition
and learning: conceptual and methodological considerations. Metacognition and
Learning, 1, 3-14.
Veenman, M. V. J. (2011). Learning to self-monitor and self-regulate. In: Mayer, R.
E. and Alexander, P. A. (Eds.). Handbook of Research on Learning and
Instruction, ch. 10. p. 197-218 . New York and London: Routledge.
White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling and metacognition:
making science accessible to all students. Cognition and Instruction, 16(1), 3-
118.
Whitebread, D., Coltman, P., Pasternak, D., Sangster, C., Grau, V., Bingham, S., and
Demetriou, D. (2009). The development of two observational tools for assessing
metacognition and self-regulated learning in young children. Metacognition and
Learning, 4(1), 63-85.
Page 43 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Zeidler, D. (2002). Dancing with maggots and saints: visions for subject-matter
knowledge, pedagogical knowledge and pedagogical content knowledge in
science teacher education reform. Journal of Science Teacher Education, 13,
27-42.
Zembal-Saul, C., Munford, D., Crawford, B, Friedrichsen, P. & Land, S. (2002).
Scaffolding preservice science teachers' evidence-based arguments during an
investigation of natural selection. Research in Science Education, 32, 437-463.
Page 44 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
Challenges in wide scale implementation efforts to foster higher order
thinking (HOT) in science education

Anat Zohar
School of Education
The Hebrew University
Jerusalem, Israel,
Anat.zohar1@mail.huji.ac.il
Page 45 of 45
A
c
c
e
p
t
e
d

M
a
n
u
s
c
r
i
p
t
I explore challenges involved in scaling up instruction of higher
order thinking
Weaving thinking into multiple, varied, specific science contents is a huge
challenge
Content goals tend to take priority over thinking goals
Organizational issues hinder large scale, deep teachers' professional
development
Scaling up thinking is often an evolutionary rather than a revolutionary
process