This article was downloaded by: [University of Nebraska, Lincoln]

On: 09 April 2015, At: 00:43

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Research in Science & Technological

Education
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/crst20

Development of three-tier heat,

temperature and internal energy
diagnostic test
a a
Deniz Gurcay & Etna Gulbas
a
Department of Secondary Science and Mathematics Education,
Physics Education, Faculty of Education, Hacettepe University,
Turkey
Published online: 02 Apr 2015.

Click for updates

To cite this article: Deniz Gurcay & Etna Gulbas (2015): Development of three-tier heat,
temperature and internal energy diagnostic test, Research in Science & Technological Education,
DOI: 10.1080/02635143.2015.1018154

To link to this article: http://dx.doi.org/10.1080/02635143.2015.1018154

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
 Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015
 Research in Science & Technological Education, 2015
http://dx.doi.org/10.1080/02635143.2015.1018154

Development of three-tier heat, temperature and internal energy

diagnostic test
Deniz Gurcay* and Etna Gulbas

Department of Secondary Science and Mathematics Education, Physics Education, Faculty

of Education, Hacettepe University, Turkey
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

Background: Misconceptions are major obstacles to learning physics, and the

concepts of heat and temperature are some of the common misconceptions that
are encountered in daily life. Therefore, it is important to develop valid and reli-
able tools to determine students’ misconceptions about basic thermodynamics
concepts. Three-tier tests are effective assessment tools to determine misconcep-
tions in physics. Although a limited number of three-tier tests about heat and
temperature are discussed in the literature, no reports discuss three-tier tests that
simultaneously consider heat, temperature and internal energy.
Purpose: The aim of this study is to develop a valid and reliable three-tier test
to determine students’ misconceptions about heat, temperature and internal
energy.
Sample: The sample consists of 462 11th-grade Anatolian high school students.
Of the participants, 46.8% were female and 53.2% were male.
Design and methods: This research takes the form of a survey study. Initially, a
multiple-choice test was developed. To each multiple-choice question was added
one open-ended question asking the students to explain their answers. This test
was then administered to 259 high school students and the data were analyzed
both quantitatively and qualitatively. The students’ answers for each open-ended
question were analyzed and used to create the choices for the second-tier ques-
tions of the test. Depending on those results, a three-tier Heat, Temperature and
Internal Energy Diagnostic Test (HTIEDT) was developed by adding a
second-tier and certainty response index to each item. This three-tier test was
administered to the sample of 462 high school students.
Results: The Cronbach alpha reliability for the test was estimated for correct and
misconception scores as .75 and .68, respectively. The results of the study sug-
gested that HTIEDT could be used as a valid and reliable test in determining
misconceptions about heat, temperature and internal energy concepts.
Keywords: three-tier test; heat; temperature; internal energy; misconceptions

Introduction
Because concepts of science are abstract and reworded by mathematics, it is difficult
for students to understand these concepts (Wessel 1999). Despite their different
approaches towards learning, most cognitive theoreticians believe that the learning
process is affected by certain pre-existing conceptions and beliefs that students
already have regarding the facts of science lessons (Aydın and Uşak 2003). Alterna-
tive conceptions which students possess before they come to the learning environ-
ment take shape as a result of their personal experiences and their interactions with
*Corresponding author. Email: denizg@hacettepe.edu.tr

© 2015 Taylor & Francis

 2 D. Gurcay and E. Gulbas

their physical and social environments (Özmen 2004). Several scientists state that
this incomplete conceptual understanding must be considered because students build
new knowledge on existing knowledge (Libarkin and Kurdziel 2001). Nevertheless,
during scientific learning, ideas that are based on students’ preconceptions and seem
logical to them may sometimes be incompatible with the scientific informations
adopted by the scientific community (Osborne 1982). Students’ conceptions that are
different from those of the scientific community and resistant to change are called
misconceptions (Helm 1980). However, in science education literature such cases
are now widely referred to as ‘alternative conceptions’ or ‘preconceptions’ (Libarkin
and Kurdziel 2001).
A misconception is not an incorrect answer that comes from a lack of knowledge
or from a mistake. Misconception describes a different scientific definition of a con-
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

cept which substitutes a concept in the human mind (Eryılmaz and Sürmeli 2002).
Hammer (1996) revealed that misconceptions are resistant to change and hinder a stu-
dent’s ability to understand scientific concepts and form new cognitive structures;
thus, the misconception must be corrected. To help students learn scientific concepts,
a meaningful relationship should be established between the new concept and the
knowledge already possessed by the student; furthermore, any contradictions should
be resolved (Aydoğan, Güneş, and Gülçiçek 2003). This goal can only be achieved by
eliciting existing conceptual background and verifying those concepts (Yağbasan and
Gülçiçek 2003). Likewise, Taber (2000a) emphasizes that teachers need to know the
preconceptions of students who come to the learning environment. Consequently, tak-
ing into consideration the challenges leading to students’ misconceptions and using
appropriate learning strategies, students are enabled to perform conceptual restructur-
ing (Özmen 2004). Many researchers use strategies of conceptual change in modify-
ing students’ misconceptions. Yılmaz and Eryılmaz (2010, 341) indicated that as a
result of these strategies, various possible outcomes, such as total or partial restructur-
ing of student’s pre-existing knowledge, reassignment of concepts within the sche-
mata, and knowledge enrichment are feasible. However, teachers must have access to
useful teaching tools to enable students to understand new scientific concepts and
to determine the efficacy of in-class learning (Treagust 2006). Therefore, it is crucial
to develop efficient assessment tools that bring out students’ conceptual background.
The problem of misconceptions is a major obstacle to learning and is frequently
encountered in physics because of its abstract nature (Aydoğan, Güneş, and Gülçiçek
2003; Gönen and Akgün 2005). Several studies on student misconceptions about
physics have been conducted over the past 30 years or more (Eryılmaz 2010). In
physics education research, misconceptions in thermodynamics were studied, in
addition to those in mechanics, electricity and optics. The basic concepts of thermo-
dynamics: ‘heat’ and ‘temperature’ are included in the curricula of primary and sec-
ondary education, and in the science departments of universities in Turkey, as well
as in several other countries around the world. Various studies are available on stu-
dents’ misconceptions about heat and temperature (Erickson 1979, 1980; Appleton
1984; Thomaz et al. 1995; Harrison, Grayson, and Treagust 1999; Carlton 2000;
Eryılmaz and Sürmeli 2002; Aydoğan, Güneş, and Gülçiçek 2003; Gönen and
Akgün 2005; Yeşilyurt 2006; Eryılmaz 2010; Gönen and Kocakaya 2010;
Tanahoung, Chitaree, and Soankwan 2010; Alwan 2011). Furthermore, numerous
studies deal with misconceptions about heat and temperature (Wiser 1986; Wiser
and Kipman 1988; Wiser, Kipman, and Halkiadakis 1988; Başer 1996, 2006a,
2006b; Başer and Çataloğlu 2005; Başer and Geban 2007).
 Research in Science & Technological Education 3

Some studies investigating students’ misconceptions about heat and temperature

reported that students explained these concepts using the material properties of
objects (Erickson 1979, 1980; Clough and Driver 1985; Thomaz et al. 1995).
According to these studies, students regarded heat as a type of matter that is inside
objects and that can be transported within objects and moved from one object to
another. Temperature, however, is perceived as a property of the material of which
the object is composed. Since physical concepts such as heat and temperature are
explained by students based on the material properties or behaviour of objects, these
concepts are called substance-based concepts (Reiner et al. 2000). Some studies
reveal that students fail to distinguish between the concepts of ‘heat and tempera-
ture’ and ‘heat and internal energy’ (Warren 1972; Harrison, Grayson, and Treagust
1999). Some students consider temperature to be a measure of heat (Thomaz et al.
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

1995), and internal energy to be the amount of heat an object possesses (Warren
1972).
Several factors play a part in why students have difficulty learning the concepts.
Differences between the scientific language and everyday language, and the termino-
logical language used in school textbooks are among the factors that make learning
difficult (Doige and Day 2012). There are various conflicting definitions of the con-
cepts of heat and temperature in textbooks (de Berg 2008). For example, some text-
books include statements arguing that heat ‘is a form of energy’ or ‘is internal
energy’. These different statements made about the concept of heat make it more
difficult for students to understand these concepts (Sozbilir 2003). In addition to
them, different terminologies used in connection with the concept of heat are another
factor preventing proper understanding of this concept. Many studies argue that it is
not correct to use the concept of heat as a noun (Doige and Day 2012). For example,
it is quite wrong to use statements that deal with the concept of heat as a noun such
as ‘heat flow’, ‘heat transfer’, ‘heat loss’, etc. because these statements lead to the
misconception that an object may contain heat (Doige and Day 2012). As a result, it
is not possible to avoid misconceptions about the concepts of heat and temperature
and eliminate learning difficulties without reaching a consensus about the concepts
of heat and temperature in scientific definitions and eliminating the contradictions in
the definitions made. Therefore, first, the difference between the concepts of heat
and temperature should be analyzed thoroughly. Although various definitions of heat
and temperature have been proposed in the course of history, the first person who
made the distinction between these concepts was Joseph Black (Lehrman 1973).
Joseph Black expressed the relationship between these concepts using the equation
ΔH = msΔt thereby revealing the difference between these concepts (Lehrman 1973;
de Berg 2008).
To prevent students from developing or retaining misconceptions regarding the
concepts of heat, temperature and internal energy and erase such misconceptions, it
is important to teach students the correct scientific definitions of these concepts. The
definition of the concept of temperature may be stated as follows: ‘Temperature is a
measure of the average kinetic energy of the particles’ (Taber 2000b, 323). On the
other hand, the relationship between the concepts of internal energy and heat was
given by Warren (1972) as follows: ‘Internal energy is the sum total of the kinetic
and potential energies of the particles constituting matter. Heat, however, is energy
that is transferred because of a difference in temperature’ (Warren 1972, 41). The
internal energy of an object can change because of work applied to it or because of
 4 D. Gurcay and E. Gulbas

heat (Warren 1972). Work and heat express the process of energy transfer; therefore,
it is wrong to talk about an object having heat (Warren 1972).
Interviews (Osborne and Gilbert 1980), multiple-choice tests (Wiser 1986;
Beichner 1994; Başer 1996), two-tier tests (Franklin 1992; Tan et al. 2002; Tan
et al. 2005) and three-tier tests (Eryılmaz 2010; Eryılmaz and Sürmeli 2002; Peşman
and Eryılmaz 2010; Arslan, Cigdemoglu, and Moseley 2012) are some of the com-
monly employed tools to determine misconceptions in physics. Of these, interviews
can be conducted with students varying broadly in age and enable teachers to ana-
lyze the reasons for students’ answers. Moreover, it is more advantageous in terms
of the flexibility and profoundness of the study. However, the interview process,
reporting process and analysis are time consuming (Osborne and Gilbert 1980).
Another tool employed to determine students’ misconceptions is the multiple-choice
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

test, which is more advantageous than interviews because they can be administered
to a broader sample of students and scoring is more objective (Beichner 1994).
However, Rollnick and Mahooana (1999) stated that, very often, students give the
incorrect reasons for their correct answers, and multiple-choice tests do not provide
a deep understanding of the students’ ideas on the topic. Thus, interviews and multi-
ple-choice tests each have their own advantages and disadvantages. Beichner (1994)
claimed that using interviews and multiple-choice tests together minimizes each
method’s disadvantages. Tamir (1971) proposed the use of multiple-choice test items
that included responses with known student alternative conceptions, and that also
required students to justify their choice by explaining their answer. Thus, the use of
justifications when answering multiple-choice questions can be a sensitive and effec-
tive way to assess learning within a student population (see Treagust 2006). The use
of justifications in tests led to the development of two-tier tests. In a two-tier test,
the first tier of items comprises a multiple-choice question, and the second tier elicits
the reasoning behind the response (Treagust 2006). In this way, the reasons for the
students’ responses in two-tier tests may be understood (Eryılmaz 2010). However,
the test results overestimated the percentage of misconceptions (Griffard and
Wandersee 2001), and lack of knowledge was not differentiated from misconcep-
tions (Eryılmaz 2010; Eryılmaz and Sürmeli 2002). These are some negative aspects
of two-tier tests. Hasan, Bagayoko, and Kelley (1999) proposed that an index (the
certainty-of-response index) be used, where the student is asked to indicate the cer-
tainty of their response. This index could be used in conjunction with diagnostic
tests to differentiate between lack of knowledge and misconceptions. They empha-
sized that students who got a low index score for an incorrect answer exhibited lack
of knowledge, whereas a high index score for an incorrect answer indicated a mis-
conception (Hasan, Bagayoko, and Kelley 1999). Moreover, if the respondent gave
a correct answer and got a high index score, it should be considered as a sign of the
certainty of the respondent’s correct answer (Hasan, Bagayoko, and Kelley 1999).
Thus, three-tier tests were developed to eliminate the problem of two-tier tests,
wherein differentiating between students’ misconceptions and lack of knowledge is
difficult. The first two tiers of a three-tier test are the same as those of a two-tier test.
However, in a three-tier test – in addition to the first two tiers – the students are
asked if they are confident about their answers in the third tier. According to Peşman
and Eryılmaz (2010), one of the advantages of three-tier tests is the possibility to
estimate the percentages of false positives and false negatives, which can be utilized
to determine the validity of the test. Hestenes and Halloun (1995) revealed that false
positives and false negatives are linked to the content validity of tests. False
 Research in Science & Technological Education 5

positives indicate that incorrect explanations are provided for correct answers,
whereas false negatives indicate that correct explanations are provided for incorrect
answers (Hestenes and Halloun 1995).
To summarize, three-tier tests were developed to assess students’ misconcep-
tions, understand their reasoning, and differentiate between their lack of knowledge
and their misconceptions. However, the number of studies in which three-tier tests
are employed is limited (Caleon and Subramaniam 2010; Peşman and Eryılmaz
2010; Arslan, Cigdemoglu, and Moseley 2012). Furthermore, although a limited
number of three-tier tests (Eryılmaz and Sürmeli 2002; Eryılmaz 2010) were devel-
oped for heat and temperature, no reports exist of three-tier tests for heat, tempera-
ture and internal energy all together. Therefore, it is vital to develop a three-tier test
that simultaneously considers the concepts of heat, temperature and internal energy.
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

By using such a test before and after teaching, it will be possible to reach clearer
information about whether students have misconceptions about this subject or not, if
there are, then teachers can engage in efforts aimed at eliminating these misconcep-
tions (Treagust 1988). Within the scope of this study, we aimed to develop a three-
tier ‘Heat, Temperature and Internal Energy’ Diagnostic Test (HTIEDT) to determine
the misconceptions of 11th-grade students about heat, temperature and internal
energy. Answers will be sought to the research question below in accordance with
the purpose of the study.

(a) Is HTIEDT a valid and reliable test that can be used in diagnosing miscon-
ceptions of students in the 11th grade?

Method
In the process of developing HTIEDT, a multiple-choice test was initially developed
and administered, and then a three-tier HTIEDT was developed and administered.
Three-tier diagnostic tests are criterion-referenced tests and are designed to evalu-
ate students’ misconceptions or understandings of scientific concepts. Moreover,
criterion-referenced assessment should be based on a set of recognized performance
standards. Additionally, from the perspective of evidence-centred design,
assessment should begin by asking what complex of knowledge, skills, or other attri-
butes should be assessed, presumably because they are tied to explicit or implicit
objectives of instruction or are otherwise valued by society. Next, what behaviors or
performances should reveal those constructs, and what tasks or situations should elicit
those behaviors? (Messick 1994, 16).
Therefore, at the beginning, we established the expected performance standards of
the students for conceptual understanding (see Table 2). The goal of this study was
to develop a three-tier diagnostic test to identify the misconceptions given in Table 1,
which we identified based on a review of the relevant literature.

Development of multiple-choice test

To develop multiple-choice tests, we first reviewed the related literature and exam-
ined in detail all conceptual questions within the literature. Based on the existing
literature, we created a list of misconceptions related to heat, temperature and
internal energy. The test questions were designed by focusing on three main
misconceptions (see Table 1).
 Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

Table 1. Choices indicating a misconception according to three-tier HTIEDT.

Misconceptions Items Item choices Explanation of misconceptions
1. Heat of an object H·I I.1.1.A, I.1.2.A, I.1.3.A; I.1.1.B, I.1.2.A, A student who, in the first-tier question, says the heat of the large or
depends on its size. I.1.3.A small object is greater, and who explains, in the second-tier question,
H.II II.1.1.A, II.1.2.A, II.1.3.A; II.1.1.B, the answer to the previous question by saying that heat depends on the
II.1.2.A, II.1.3.A size of an object, and who, in the third-tier question, says he or she is
H.III III.1.1.A, III.1.2.A, III.1.3.A; III.1.1.B, sure of their answers has a misconception.
III.1.2.A, III.1.3.A
H.IV IV.1.1.A, IV.1.2.A, IV.1.3.A; IV.1.1.B,
IV.1.2.A, IV.1.3.A
2. Temperature of an T.I I.2.1.A, I.2.2.A, I.2.3.A; I.2.1.B, I.2.2.A, A student who, in the first-tier question, says that the temperature of
D. Gurcay and E. Gulbas

object depends on its I.2.3.A the large or small object is greater, and who, in the second-tier
size. T.II II.2.1.A, II.2.2.A, II.2.3.A; II.2.1.B, question, explains the answer to the previous question by saying that
II.2.2.A, II.2.3.A temperature depends on the size of the object, and who, in the third-
T.III III.2.1.A, III.2.2.A, III.2.3.A; III.2.1.B, tier question, says he or she is sure of their answers has a
III.2.2.A, III.2.3.A misconception.
T.IV IV.2.1.A, IV.2.2.A, IV.2.3.A; IV.2.1.B,
IV.2.2.A, IV.2.3.A
3. Internal energy is the IE.I I.3.1.A, I.3.2.D, I.3.3.A; I.3.1.B, I.3.2.D, A student who, in the first-tier question, says the internal energy of the
amount of heat that an I.3.3.A; I.3.1.C, I.3.2.C, I.3.3.A large or small object is greater, and who, in the second-tier questions,
object has. IE.II II.3.1.A, II3.2.D, II.3.3.A; II.3.1.B, explains the answer to the previous question by saying the heat of the
II3.2.D, II.3.3.A;II.3.1.C, II3.2.C, object is greater, and who, in the third-tier question, says he or she is
II.3.3.A sure of their answers OR a student who, in the first-tier question, says
IE.III III.3.1.A, III.3.2.D, III.3.3.A; III.3.1.B, the internal energies of the objects are equal, and who, in the second-
III.3.2.D, III.3.3.A;III.3.1.C, III.3.2.C, tier question, explains the answer to the previous question by saying
III.3.3.A the heat of the objects are equal, and who, in the third-tier question,
IE.IV IV.3.1.A, IV.3.2.D, IV.3.3.A; IV.3.1.B, says he or she is sure of their answers has a misconception.
IV.3.2.D, IV.3.3.A;IV.3.1.C, IV.3.2.C,
IV.3.3.A
 Research in Science & Technological Education 7

Eryılmaz (2010) revealed that misconception tests such as the Force Concept
Inventory (Hestenes, Wells, and Swackhamer 1992) and the Thermal Concept
Evaluation (Yeo and Zadnik 2001) have limitations, as a score related to a specific
misconception cannot be estimated because of the lack of various questions that
assess the same misconception in those tests. In both tests, some of the misconcep-
tions were determined by one or two questions whereas others were determined by
several questions (Eryılmaz 2010). Therefore, in the present study, we focus on only
three misconceptions to make it easier to estimate the scores for each misconception
and analyze students’ misconceptions in depth.
In developing the test, we began by writing four main questions related to a sam-
ple case. Only one of these questions was adapted from the literature (Harrison,
Grayson, and Treagust 1999); the remaining three questions were created by the
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

authors. A total of 12 multiple-choice questions based on the sample case in main

questions were prepared. An open-ended question was added after each multiple-
choice question that asked students to explain why they chose their answer. Next,
the test was presented to two lecturers from two different universities and a physics
teacher to be assessed for face validity, content validity, content accuracy and
concordance. After the assessment, the above-mentioned specialists made some sug-
gestions about the suitability of the questions to determine the content validity and
misconceptions, and the requested amendments were made accordingly. Following
this process, the test was administered to 10 11th- and 12th-grade students who had
learnt the subject beforehand to verify whether the questions were understandable
and clear. The required changes were made according to the feedback received from
the students and the final version of the multiple-choice test about heat, temperature
and internal energy was completed.

Development of three-tier HTIEDT

The multiple-choice test was administered to 259 Anatolian high school 11th- and
12th-grade students and the answers obtained from the students were processed
using Microsoft Excel. The correct scores and misconception scores were obtained
using a correct answer key and misconception answer key. We conducted factor
analysis to determine the construct-related validity of the test according to the cor-
rect and misconception scores. When the results were analyzed, it was established
that the test consisted of three factors. For both score types, the questions of the first
factor were related to heat, those of the second factor were related to temperature
and those of the third factor were related to internal energy. The Cronbach alpha
reliability coefficient for the multiple-choice test for the correct scores was estimated
as .67. Because the study sample was large, the facility index p and the item dis-
crimination index D of the test items were analyzed by the 27% upper–lower group
method (Crocker and Algina 1986, 314). Analyzing the multiple-choice test based
on the correct scores gave an average facility index .31 for the multiple-choice and
an average discrimination index .47. Based on the correct and misconception scores
for questions related to heat, the average facility index was .09 and .62, respectively.
These results indicate that students had difficulty formulating correct answers for the
questions related to heat. Moreover, they suggest that the false answers to the heat-
related questions were inspired by misconceptions. Based on the correct scores for
questions related to heat, the average discrimination index was .14, whereas for the
other questions in the test this was greater than .40. The explanation for the low
 8 D. Gurcay and E. Gulbas

discrimination index for questions related to heat is that most students did not have
a correct conceptual understanding of heat but had misconceptions in this area.
Because the reliability of the correct scores of the test was .67 and the results of fac-
tor analysis provided positive proof of the construct-related validity, we decided that
all questions in the test should be used. Moreover, the student answers to the open-
ended questions in the multiple-choice test were analyzed and categorized depending
on their similarities. The categorized answers were gathered under basic headings
and used to form the choices for the questions of the second tier of the HTIEDT. In
the second-tier questions, the reasons for the students’ answers for the first-tier
questions were asked. In the third tier, the students were asked whether they were
confident about their answers for the first two tiers. The HTIEDT was again pre-
sented to the specialists to be assessed for content validity, content accuracy and
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

concordance, and requested amendments were made accordingly. Finally, the test
was administered to five 12th-grade students to test the understandability and clarity
of the questions. All necessary changes were made based on the feedback received
from the students and the final version of the three-tier HTIEDT was completed.
The test consisted of four main questions, which translated into 12 first-tier ques-
tions, 12 second-tier questions and 12 third-tier questions for the HTIEDT. The total
score of the three-tier HTIEDT ranged from 0 to 12. A sample question taken from
the three-tier HTIEDT is given in Appendix 1.

Population and sample

The population of the study comprised 11th-grade students from Anatolian high
schools, in one of the biggest districts of Ankara, Turkey. Students in Turkey enter
the secondary school on the basis of student selection exams that are conducted fol-
lowing primary school. Based on their scores, they are enrolled in science high
schools, Anatolian high schools or private high schools. Students whose scores do
not qualify them to enrol in these schools attend state high schools. The schools
included in this study were selected by convenience sampling. The sample com-
prises 462 11th-grade Anatolian high school students; 46.8% of the participants were
female and 53.2% were male. The participants were 17 to 18 years old. In addition,
information about the students’ physics scores in the school report belonging to the
previous semester was collected. Physics scores of 1.3% of the students were 1,
while 3.7% of the students had 2, 18.4% had 3, 48% had 4 and 28.6% had 5 as
physics scores in their school reports. Academic report score is calculated using a
1–5 scale.

Data analysis
The three-tier HTIEDT was administered to 11th-grade students in the academic year
2011–2012. The test took approximately 15–20 minutes, and the answers were
entered into a Microsoft Excel spreadsheet. As in the pilot study, the items were
analyzed by exploratory factor analysis to determine whether the items were loaded
correctly. Seven different scores were obtained using the correct answer key and the
misconception answer key of the three-tier HTIEDT: correct scores–1, correct
scores–2, correct scores–3, misconception scores–1, misconception scores–2,
misconception scores–3 and confidence levels. Those scores were estimated using
codes presented in the literature (Peşman and Eryılmaz 2010). Table 3, demonstrates
 Research in Science & Technological Education 9

the coding and estimation of correct scores–1, correct scores–2, correct scores–3,
misconception scores–1, misconception scores–2 and misconception scores–3.
Moreover, for the confidence levels, the student answers were analyzed only
according to the question in the third tier. If the student’s answer to the third-tier
question was ‘I am sure’, it was coded as 1; otherwise it was coded as 0. The stu-
dent scores for the third tier were summed and the confidence levels were estimated.
Furthermore, to determine the content validity of the test, the percentages of false
positives and false negatives were calculated. If a student’s answer to a first-tier
question was correct, and to the second-tier question was incorrect, and if the third-
tier answer was ‘I am sure’, then the student’s answer to the three-tier question was
a false positive. If a student’s answer to the first-tier question was incorrect, and to
the second-tier question was correct, and if the third-tier answer was ‘I am sure’,
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

then the student’s answer to the three-tier question was a false negative (Peşman and
Eryılmaz 2010).
Misconceptions measured in the three-tier HTIEDT are given in Table 1. Table 2
shows in detail how students’ correct conceptual understanding can be understood.

Results
This study examines both construct-related validity and content validity of the
HTIEDT. Additionally, we present descriptive statistics for the HTIEDT, the percent-
age of correct answers and that of misconceptions related to the HTIEDT. The data
were analyzed in three different ways to confirm the validity of the HTIEDT.
Initially, exploratory factor analysis was employed to identify the factor structure of
the test. Furthermore, the correlation between correct scores–2 and confidence
levels, and that between misconception scores–2 and confidence levels were
analyzed with regards to construct-related validity. The percentages of false positives
and false negatives were calculated for content validity.
To confirm construct-related validity of HTIEDT, factor analysis was conducted
for both correct and misconception scores depending on the answers given to the
three-tier questions. According to the results of factor analysis for correct scores–3,
the three-tier questions that formed the test can be categorized under three factors:
the first factor was questions about heat, the second factor was questions about tem-
perature and the third factor was questions about internal energy. The total variance
explained by these three factors was 71.09%. When factor analysis was applied to
misconception scores–3, we observed that the three-tier questions could again be
categorized under three factors. Similarly, the questions related to heat were in the
first factor, those related to temperature were in the second factor and those related
to internal energy were in the third factor. The factors that formed the test explained
67.22% of the total variance. In conclusion, the results indicate that all three-tier
questions in the test were loaded under logical factors.
Cataloglu (2002) stated that a positive correlation between the student scores for
a test and confidence levels could be evidence of the validity of the test. In this case,
we expected that students who got higher scores on the test were more confident
with their answers than those who got lower scores. To analyze the validity of
HTIEDT, we estimated the Pearson product-moment correlation coefficient between
correct scores–2 and confidence levels. A positive relation between correct scores–2
and confidence levels was observed; however, this relation was not significant
(r = .051, p > .05). Because the HTIEDT is a test used to determine both students’
 10 D. Gurcay and E. Gulbas

Table 2. Choices indicating correct understandings according to three-tier HTIEDT.

Explanation of
Correct understanding Items Item choices understanding
1. Heat is the energy I, II, III and IV 1.1. D, 1.2. E, 1.3. A A student who, in the
transferred due to first-tier question, says
temperature the heat of objects
difference. cannot be compared,
and who, in the second-
tier question, explains
(since objects cannot
have heat) by saying
that heat is energy that
is transferred due to
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

temperature difference,
and who, in the third-
tier, says he or she is
sure of their answers has
the correct conceptual
understanding of the
concept of heat.
2. If two objects that I, II, III and IV 2.1.C, 2.2.B, 2.3.A A student who, in the
were made of the first-tier question, says
same material but temperatures of objects
different in size are equal, and who, in
were left in the the second-tier question,
same environment explains this by saying
long enough, their that temperature is
temperature would independent of the size
be the same. of the object, and who,
in the third-tier question,
says that he or she is
sure of their answers has
a correct conceptual
understanding of the
concept of temperature.
3. Internal energy of I 3.1.A, 3.2.A, 3.3.A A student who, in the
an object depends II 3.1.A, 3.2.A, 3.3.A first-tier question, says
on its size. III 3.1.B, 3.2.A, 3.3.A internal energy of the
IV 3.1.A, 3.2.A, 3.3.A large object is greater,
and who, in the second-
tier question, explains
this by saying that
internal energy depends
on the size of the object,
and who, in the third-
tier question, says he or
she is sure of their
answers has a correct
conceptual
understanding of the
concept of internal
energy.

conceptual understanding and their misconceptions, the Pearson product-moment

correlation coefficient was also estimated between the misconception scores–2 and
confident levels. The results of the analysis indicated a positive and statistically
 Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

Table 3. Scoring based on correct and misconception answers.

Scoring based on correct answers Scoring based on misconception answers
Correct Correct Correct Misconception Misconception Misconception
Scores scores–1 scores–2 scores–3 scores–1 scores–2 scores–3

Scores One-tier Two-tier Three-tier One-tier Two-tier Three-tier

based on questions questions questions questions questions questions
Coding If the answer of If the answer of the If the answers of the If the student If the student checked If the student checked
the student to the student to the question student to the checked an option the option that the option that
question in the in the first tier and to question in the first that indicated the indicated the indicated the
first tier was the question in the tier and to the misconception in the misconception in the misconception in the
correct, it was second tier were question in the second first tier, it was first tier and the option first tier and the option
coded as 1; correct, it was coded tier were correct and coded as 1; that gives the reason that gives the reason
otherwise it was as 1; otherwise it was that to the question in otherwise it was related to the related to the
coded as 0. coded as 0. the third tier was ‘I coded as 0. misconception in the misconception in the
am sure’, it was second tier together, it second tier together,
coded as 1; otherwise was coded as 1; and the answer to the
it was coded as 0. otherwise it was coded question in the third
as 0. tier was ‘I am sure’, it
was coded as 1;
otherwise it was coded
as 0.
Total The scores of the The scores of the The scores of the The scores of the The scores of the The scores of the
scores student from student from each student from each student from each student from each student from each
each one-tier two-tier question were three-tier question one-tier question two-tier question were three-tier question
question were summed and correct were summed and were summed and summed and were summed and
summed and scores–2 were correct scores–3 were misconception misconception scores– misconception scores–
correct scores–1 estimated. estimated. scores–1 were 2 were estimated. 3 were estimated.
were estimated. estimated.
Research in Science & Technological Education
11
 12 D. Gurcay and E. Gulbas

significant relation (r = .108, p < .05) between misconception scores–2 and

confidence levels. We thus conclude from these results that students with more
misconceptions tended to be more confident about their answers. In other words,
students were more confident about their misconception-based answers than their
correct answers.
Regarding content validity in diagnostic tests, Hestenes and Halloun (1995) sug-
gested estimating the percentages of false positives and false negatives. Furthermore,
they stated that the percentage of false positives and that of false negatives should be
less than 10%. According to them, a percentage of false negatives below 10% sug-
gests that the questions were explicit and coherent for students who had sufficient
knowledge. In the present study, the percentage of false negatives was 3%. We thus
conclude that the percentage of students who gave incorrect answers to first-tier ques-
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

tions and correct answers to second-tier questions was not high. In other words, the
questions were explicit and coherent. Hestenes and Halloun (1995) further revealed
that it is difficult to minimize the percentage of false positives according to the per-
centage of the false negatives, because students might correctly answer first-tier ques-
tions without begin able to explain their reasons for choosing their specific answer.
Thus, they concluded that the students would inevitably give guessed answers. The
present study had 7% false positives, which is consistent with the criterion presented
by Hestenes and Halloun (1995) and proves the content validity of HTIEDT.
The results of item analysis according to correct scores–3 and misconception
scores–3 is presented in Table 4. The facility index p and the item discrimination
index D of the test items were analyzed by the 27% upper–lower group method.
The average facility index of the test according to correct scores–3 was .14, and that
of the questions according to correct scores–3 was .05 for heat, .21 for temperature
and .16 for internal energy. The average facility index of the questions according to
misconception scores–3 was .28 for heat, .22 for temperature, .17 for internal
energy. These findings indicate that the students had difficulty primarily in answer-
ing questions related to heat. Furthermore, they show that students tend to give more
misconception answers to heat-related questions compared with the other questions.
On the other hand, the average discrimination index of the questions according to

Table 4. Item analysis of HTIEDT according to the 27% upper–lower group method.
Correct scores–3 Misconception scores–3
Facility index Discrimination index Facility index Discrimination index
Items (p) (D) Items (p) (D)
I.1 .04 .09 H·I .28 .57
II.1 .05 .10 H.II .27 .54
III.1 .05 .10 H.III .28 .56
IV.1 .05 .10 H.IV .30 .59
I.2 .25 .50 T.I .20 .41
II.2 .25 .50 T.II .20 .41
III.2 .15 .30 T.III .24 .49
IV.2 .21 .42 T.IV .22 .44
I.3 .15 .30 IE.I .17 .34
II.3 .17 .34 IE.II .16 .33
III.3 .15 .30 IE.III .18 .35
IV.3 .16 .32 IE.IV .18 .37
Mean .14 .28 Mean .22 .45
 Research in Science & Technological Education 13

correct scores–3 was .10 for heat, .43 for temperature and .32 for internal energy,
and that of the questions according to misconception scores–3 was .56 for heat, .44
for temperature, .35 for internal energy.
Analyzing the HTIEDT according to correct scores–3 revealed that the average
discrimination index of questions related to heat was very low (.10), whereas,
according to misconception scores–3, the average discrimination index was very
high (.56). Analyzing the questions related to temperature and internal energy
according to correct scores–3 and misconception scores–3 showed that the discrim-
ination indices of those questions were good. The low facility indices and discrim-
ination indices for questions related to heat indicate that no significant difference
exists between students who got high correct scores and students who got low
correct scores from the test. That is, some of the students who got high correct
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

scores from the test also had misconceptions about heat. Therefore, the reason for
the low discrimination indices for questions is attributed to the misconceptions stu-
dents had related to heat rather than to the structure of the test questions. HTIEDT is
thus a valid test, particularly for determining student misconceptions. However, low
facility indices and low discrimination indices for questions related to heat decrease
the validity of HTIEDT for being employed as an achievement test. Eryılmaz (2010)
also observed that the percentage of correct answers to questions about heat was
quite low. On the other hand, the results of the exploratory factor analysis for correct
scores–3 and misconception scores–3 indicated that the test items were categorized
under expected factors. These results indicate that the three-tier HTIEDT could be
used to diagnose misconceptions. Although problems are mentioned about the valid-
ity and reliability of three-tier diagnostic tests in the literature, it is emphasized that
these tests are quite effective in identifying misconceptions (Eryılmaz 2010).
However, the average facility index for heat-related questions for the three-tier
HTIEDT was .05 according to correct scores–3 and the average discrimination index
was .10 but, according to misconception scores–3, the average facility index was .28
and the average discrimination index was .56. Furthermore, considering that the
factor analysis gave the expected results, the heat-related items could be used to
diagnose misconceptions.
Table 5 presents the descriptive statistics of HTIEDT with regards to correct
scores–3 and misconception scores–3. The Cronbach alpha reliability of the test was
estimated as .75 for correct scores–3 and .68 for misconception scores–3. The score
range for each test is between 0 and 12. The average for correct scores–3 for the
three-tier HTIEDT was 1.01. This very low mean score indicates that most students
had a low level of understanding about heat, temperature and internal energy. The
averages for correct scores–3 for heat-, temperature- and internal energy-related ques-
tions were 0.10, 0.51 and 0.40, respectively, which indicates that students had diffi-
culty mainly in answering questions related to heat. Similarly, the results indicated
that the skewness for correct scores–3 was 1.79, which indicates that students had a
low level of understanding of heat, temperature and internal energy. Skewness defines
whether the distribution is even or not. If the value of skewness is greater than one
and positive, this indicates that scores are concentrated in lower scores while higher
scores are very few (Tabachnick and Fidell 2007). On the other hand, the skewness
value belonging to correct scores-3, indicates that very few students have higher
levels of understanding of the concepts of heat, temperature and internal energy. On
the other hand, the average for misconception scores–3 for the three-tier HTIEDT
was 2.53, which is relatively high compared with that of correct scores–3. That is, the
 14 D. Gurcay and E. Gulbas

Table 5. Descriptive statistics for correct scores–3 and misconception scores–3.

Correct scores–3 Misconception scores–3
Number of students 462 462
Number of items 12 12
Maximum 7.0 9.0
Minimum 0 0
Mean 1.01 2.53
Standard error of the mean .079 .11
Median 0 2.0
Mode 0 0
Range 7.0 9.0
Standard deviation 1.70 2.29
Variance 2.88 5.23
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

Skewness 1.79 .63

Kurtosis 2.48 −.37
Reliability (Cronbach alpha) .75 .68

students had misconceptions rather than a correct conceptual understanding of heat,

temperature and internal energy. The averages for misconception scores–3 for
heat-, temperature- and internal energy-related questions were 1.07, 0.96 and 0.50,
respectively. According to these results, the average for misconception scores–3 for
heat-related questions is higher than for temperature- and internal energy-related
questions. This result indicates that students had more misconceptions related to heat
than to temperature and internal energy. According to misconception scores–3, skew-
ness was estimated as .63 and kurtosis −.37. This indicates that misconception
scores–3 was close to normal distribution.
Figure 1 shows the percentages of correct answers for the one-, two-, and
three-tier questions of HTIEDT. The average percentage of students who gave
correct answers to the first tier of the test was 29%, and that of students who gave
correct answers to both questions in the first two tiers was 16%. The average per-
centage of students who gave correct answers to all three questions in the three tiers
was 8%. The difference between the average percentages of correct answers for one-
and second-tier questions was approximately 13%. Of that difference, 7% came from
the percentage of false positives and 6% came from inconsistent answers. The differ-
ence between the average percentages of correct answers for the two- and three-tier
questions was approximately 8%. That is, the students stated that, although they
gave correct answers to two-tier questions in the test, they were not confident about
their answers; this indicates a lack of knowledge. Moreover, the average percentage

Figure 1. Percentages of correct answers for one-tier, two-tier, and three-tier questions of
HTIEDT.
 Research in Science & Technological Education 15

of students who gave correct answers to all three-tier test items was 8%. This result
shows that the students had a very low level of understanding of heat, temperature
and internal energy.
Figure 2 shows the percentage of misconception answers for the one-, two- and
three-tier questions of HTIEDT. The average percentage of students who gave mis-
conception answers to the questions in the first tier of the test was 51%, and that of
students who gave misconception answers to both questions in the one- and second-
tier tests was 36%. The average percentage of students who gave misconception
answers to all questions in the three-tier test was 21%. The difference between the
average percentages of misconceptions in the one- and three-tier questions was 15%.
Of this difference, 3% resulted from the percentage of false negatives, and 12% from
inconsistent answers. The difference between the average percentages of the miscon-
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

ceptions in the two- and three-tier questions was 15%. That is, the students stated that,
although they gave misconception answers to two-tier questions in the test, they were
not confident about their answers; this indicates a lack of knowledge. Moreover, the
average percentage of students who gave misconception answers to all three-tier test
items was 21%. This result shows that the students’ level of misconceptions about
heat, temperature and internal energy is higher than their level of understanding.
Examination of the three-tier questions of HTIEDT revealed that the students
gave correct answers mainly to questions related to temperature. Thirteen per cent of
students stated that, if two objects composed of the same material but differing in
size were left in the same environment for a sufficiently long time, their tempera-
tures would be the same. The questions that the students found most difficult were
those related to heat. When students were asked to compare the heat of two objects
composed of the same material but differing in size and that were left in the same
environment for a sufficiently long time, 3% of the students stated that the heat of
the objects could not be compared because the objects do not have heat. In questions
related to internal energy, 10% of the students stated that the larger of the two
objects composed of the same material and left in the same environment for a
sufficiently long time had more internal energy.
Analyzing the students’ misconception scores for the three-tier HTIEDT showed
that students had misconceptions mainly for three-tier questions related to heat: 27%
of students stated that the heat of two objects composed of the same material and
left in the same environment for a long time depends on the size of the objects.

Figure 2. Percentages of misconception answers for one-tier, two-tier and three-tier

questions of HTIEDT.
 16 D. Gurcay and E. Gulbas

Similarly, 24% of students stated that the temperature of two objects composed of
the same material and left in the same environment for a long time depends on the
size of the objects. The students had the least misconceptions in the three-tier ques-
tions related to internal energy. Accordingly, 13% of the students stated that the
internal energy of two objects composed of the same material and left in the same
environment for a long time depends on the amount of heat each object has. In fact,
this misconception about internal energy is based on students’ misconception about
heat, because many students think objects have heat.

Discussion and suggestions

This study aimed to develop a valid and reliable three-tier HTIEDT that could iden-
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

tify students’ misconceptions about heat, temperature and internal energy. The
results obtained from the item analysis and the test statistics proved that the three-
tier HTIEDT that was developed in this study is a valid, effective and reliable tool
with which misconceptions can be diagnosed. Furthermore, we observed that stu-
dents mainly provided correct answers to temperature-related questions and had
more misconceptions with regard to heat-related questions compared with tempera-
ture- and internal energy-related questions.
There are some advantages in developing a three-tier test to determine miscon-
ceptions. For example, three-tier tests are practical and manageable for teachers and
researchers because they are easy to administer to exhaustive samplings and are not
time consuming. Multiple-choice tests have similar advantages. However, multiple-
choice tests and two-tier tests are less efficient in that it is not possible to differenti-
ate between students’ lack of knowledge and their misconceptions. This situation
makes the three-tier test better than multiple-choice or two-tier tests for determining
misconceptions.
To analyze the construct-related validity of HTIEDT, the correlations between
correct scores–2 and confidence levels, and between misconception scores–2 and
confidence levels were examined. To determine the factor structure of the HTIEDT,
exploratory factor analysis was conducted for correct scores–3 and misconception
scores–3. Additionally, to analyze the content validity of HTIEDT, the percentages
of false positives and false negatives were estimated.
Contrary to expectations, although a positive but statistically insignificant rela-
tion was observed between correct scores–2 and confidence levels, a positive and
statistically significant relation was observed between misconception scores–2 and
confidence levels. That is, students were more confident about their misconception
answers than about their correct answers. Actually, students are given correct con-
cepts at schools and attention is drawn to misconceptions by teachers. This situation
indicates that misconceptions resist cognitive structures and are difficult to change.
Several studies indicate that the confidence students have in their incorrect answers
constitutes a resisting misconception (Odom and Barrow 2007). Therefore, it is
important to analyze the correlation between misconception scores–2 and confidence
levels to ensure the construct-related validity. The results of the exploratory factor
analysis indicate that the questions about heat, temperature and internal energy were
unexpected factors. One might think that the positive but statistically insignificant
relation between correct scores–2 and confidence levels would decrease the validity
of the HTIEDT for determining the conceptual understanding level of students.
However, the results of exploratory analysis provide proof to the contrary. When the
 Research in Science & Technological Education 17

content validity of the HTIEDT was analyzed, we observed that the percentages of
false positives and false negatives were consistent with the results of Hestenes and
Halloun (1995). Moreover, we observed that, upon shifting from first-tier questions
to third-tier questions, the percentages of both correct answers and misconception
answers gradually decreased. Some of the students who gave correct answers or
misconception answers to first-tier questions did not give the correct reason for
choosing that specific answer and the reason behind the misconception in the second
tier. Some other students stated that they were not confident about their answers in
the first and second tiers. Therefore, shifting from first-tier questions to third-tier
questions, a decrease was observed in the percentages of correct and misconception
answers. According to Eryılmaz and Sürmeli (2002), three-tier tests are more effec-
tive than multiple-choice tests and two-tier tests for assessing misconceptions. If the
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

misconceptions of students were assessed with three-tier tests, it would be easier to

decide whether the students’ answers were incorrect because of their misconceptions
or their lack of knowledge on the subject. This is because two-tier tests overestimate
the percentages of misconception and correct scores (Griffard and Wandersee 2001).
The results of the present study further support this conclusion. Thus, the HTIEDT
is a valid and effective test for diagnosing both students’ misconceptions and con-
ceptual understandings about heat, temperature and internal energy.
The Cronbach alpha reliability coefficient was .75 for correct scores–3 and .68
for misconception scores–3. The value of the reliability coefficient of the tests
developed to determine misconceptions (Peşman and Eryılmaz 2010; Arslan,
Cigdemoglu, and Moseley 2012) was consistent with the results of this study.
In conclusion, the HTIEDT is a valid and reliable test that may be used to deter-
mine misconceptions that high-school students may have about heat, temperature
and internal energy, and to make a distinction between students’ misconceptions and
their lack of knowledge. Both teachers and researchers can administer HTIEDT to
the students before and/or after teaching and can determine their pre-knowledge,
lack of knowledge and misconceptions about heat, temperature and internal energy.
Because Caleon and Subramaniam (2010) emphasized that diagnostic tests could be
employed to monitor students’ gradual conceptual development, administering the
HTIEDT as a pre-test or post-test should provide information on the efficiency of
instruction. However, as Hasan, Bagayoko, and Kelley (1999) stated, although lack
of knowledge and misconceptions provide data about the students’ understandings
of a subject, different methodologies should be employed to eliminate the problems
related to lack of knowledge and misconceptions.

References
Alwan, A. A. 2011. “Misconception of Heat and Temperature among Physics Students.”
Procedia Social and Behavioral Sciences 12: 600–614.
Appleton, K. 1984. Children’s Ideas about Hot and Cold. Learning in Science Project (Pri-
mary). Working Paper No. 127. Hamilton, New Zealand: Waikato University, Science
Education Research Unit. ED 252 407.
Arslan, H. O., C. Cigdemoglu, and C. Moseley. 2012. “A Three-tier Diagnostic Test to
Assess Pre-service Teachers’ Misconceptions about Global Warming, Greenhouse Effect,
Ozone Layer Depletion, and Acid Rain.” International Journal of Science Education
34 (11): 1667–1686.
Aydın, H., and M. Uşak. 2003. “The Importance of the Investigation of Alternative Concep-
tions in Science Classes: A Theoretical Approach.” Pamukkale University Journal of
Education 1 (13): 121–135.
 18 D. Gurcay and E. Gulbas

Aydoğan, S., B. Güneş, and Ç. Gülçiçek. 2003. “The Misconceptions about Heat and
Temperature.” Gazi University Journal of Gazi Educational Faculty 23 (2): 111–124.
Baser, M. 1996. “Effect of Conceptual Instruction on Understanding of Heat and Temperature
Concepts and Science Attitude.” Unpublished master’s thesis. Ankara, Turkey: Middle
East Technical University.
Başer, M. 2006a. “Effect of Conceptual Change Oriented Instruction on Students’
Understanding of Heat and Temperature Concepts.” Journal of Maltese Education
Research 4 (1): 64–79.
Başer, M. 2006b. “Fostering Conceptual Change by Cognitive Conflict Based Instruction on
Students’ Understanding of Heat and Temperature Concepts.” Eurasia Journal of Mathe-
matics, Science and Technology Education 2 (2): 96–114.
Başer, M., and E. Çataloğlu. 2005. “Effect of Conceptual Change Oriented Instruction on
Remediation of Students’ Misconceptions Related to Heat and Temperature Concepts.”
Hacettepe University Journal of Education 29: 43–52.
Başer, M., and Ö. Geban. 2007. “Effectiveness of Conceptual Change Instruction on Under-
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

standing of Heat and Temperature Concepts.” Research in Science & Technological

Education 25 (1): 115–133.
Beichner, R. J. 1994. “Testing Student Interpretation of Kinematics Graphs.” American
Journal of Physics 62 (8): 750–762.
de Berg, K. C. 2008. “The Concepts of Heat and Temperature: The Problem of Determining
the Content for the Construction of an Historical Case Study Which is Sensitive to Nature
of Science Issues and Teaching–Learning Issues.” Science & Education 17 (1): 75–114.
Caleon, I., and R. Subramaniam. 2010. “Development and Application of a Three-tier
Diagnostic Test to Assess Students’ Understanding of Waves.” International Journal of
Science Education 32 (7): 939–961.
Carlton, K. 2000. “Teaching about Heat and Temperature.” Physics Education 35 (2): 101–105.
Cataloglu, E. 2002. “Development and Validation of an Achievement Test in Introductory
Quantum Mechanics: The Quantum Mechanics Visualization Instrument.” Accessed
August 8, 2012. https://etda.libraries.psu.edu/paper/5937/1204-ABD
Clough, E. E., and R. Driver. 1985. “Secondary Students’ Conceptions of the Conduction of Heat:
Bringing Together Scientific and Personal Views.” Physics Education 20 (4): 176–182.
Crocker, L., and J. Algina. 1986. Introduction to Classical and Modern Test Theory. Orlando,
FL: Holt, Rinehart and Winston.
Doige, C. A., and T. Day. 2012. “A Typology of Undergraduate Textbook Definitions of
‘Heat’ across Science Disciplines.” International Journal of Science Education 34 (5):
677–700.
Erickson, G. L. 1979. “Children’s Conceptions of Heat and Temperature.” Science Education
63 (2): 221–230.
Erickson, G. L. 1980. “Children View Points of Heat: A Second Look.” Science Education
64 (3): 323–336.
Eryılmaz, A. 2010. “Development and Application of Three-tier Heat and Temperature Test:
Sample of Bachelor and Graduate Student.” Eurasian Journal of Educational Research
40: 53–76.
Eryılmaz, A., and E. Sürmeli. 2002. “Üç-Aşamalı Sorularla Öğrencilerin Isı ve Sıcaklık
Konularındaki Kavram Yanılgılarının Ölçülmesi [Identifying Students’ Misconception on
Heat and Temperature through Three-Tier Questions].” Paper presented at the 5th
National Conference on Science and Mathematics Education. Accessed February 7, 2011.
http://www.fedu.metu.edu.tr/ufbmek-5/bkitabi/PDF/Fizik/Bildiri/t110d.pdf
Franklin, B. J. 1992. The Development, Validation and Application of a Two-tier Diagnostic
Instrument to Detect Misconceptions in the Areas of Force, Heat, Light and Electricity.
Unpublished doctoral dissertation, Louisiana State University and Agricultural and
Mechanical College.
Gönen, S., and A. Akgün. 2005. “The Investigation of Applicability of Worksheet Was
Developed about Relationship between Heat and Temperature Concepts.” Electronic
Journal of Social Sciences 3 (11): 92–106.
Gönen, S., and S. Kocakaya. 2010. “A Cross-age Study: A Cross-age Study on the
Understanding of Heat and Temperature.” Eurasian Journal of Physics and Chemistry
Education 2 (1): 1–15.
 Research in Science & Technological Education 19

Griffard, P. B., and J. H. Wandersee. 2001. “The Two-tier Instrument on Photosynthesis:

What Does It Diagnose?” International Journal of Science Education 23 (10):
1039–1052.
Hammer, D. 1996. “More than Misconceptions: Multiple Perspectives on Student Knowledge
and Reasoning, and an Appropriate Role for Education Research.” American Journal of
Physics 64: 1316–1325.
Harrison, A. G., D. J. Grayson, and D. F. Treagust. 1999. “Investigating a Grade 11 Student’s
Evolving Conceptions of Heat and Temperature.” Journal of Research in Science
Teaching 36 (1): 55–87.
Hasan, S., D. Bagayoko, and E. L. Kelley. 1999. “Misconceptions and the Certainty of
Response Index (CRI).” Physics Education 34 (5): 294–299.
Helm, H. 1980. “Misconceptions in Physics amongst South African Students.” Physics
Education 15 (2): 92.
Hestenes, D., and I. Halloun. 1995. “Interpreting the Force Concept Inventory.” Physics
Teacher 33: 502–506.
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

Hestenes, D., M. Wells, and G. Swackhamer. 1992. “Force Concept Inventory.” Physics
Teacher 30: 141–158.
Lehrman, R. L. 1973. “Energy is Not the Ability to Do Work.” The Physics Teacher 11 (1):
15–18.
Libarkin, J. C., and J. P. Kurdziel. 2001. “Research Methodologies in Science Education-
Assessing Students’ Alternative Conceptions.” Journal of Geoscience Education 49 (4):
378–383.
Messick, S. 1994. “The Interplay of Evidence and Consequences in the Validation of Perfor-
mance Assessments.” Educational Researcher 23 (2): 13–23.
Odom, A. L., and L. H. Barrow. 2007. “High School Biology Students’ Knowledge and
Certainty about Diffusion and Osmosis Concepts.” School Science and Mathematics 107
(3): 94–101.
Osborne, R. J. 1982. “Science Education: Where Do We Start?” The Australian Science
Teachers’ Journal 28: 21–30.
Osborne, R. J., and J. K. Gilbert. 1980. “A Technique for Exploring Students’ Views of the
World.” Physics Education 15: 376–379.
Özmen, H. 2004. “Some Student Misconceptions in Chemistry: A Literature Review of
Chemical Bonding.” Journal of Science Education and Technology 13 (2): 147–159.
Peşman, H., and A. Eryılmaz. 2010. “Development of a Three-tier Test to Assess Misconcep-
tions about Simple Electric Circuits.” The Journal of Educational Research 103: 208–222.
Reiner, M., J. D. Slotta, M. T. H. Chi, and L. B. Resnick. 2000. “Naive Physics Reasoning: A
Commitment to Substance-based Conceptions.” Cognition and Instruction 18 (1): 1–34.
Rollnick, M., and P. P. Mahooana. 1999. “A Quick and Effective Way of Diagnosing Student
Difficulties: Two-tier from Simple Multiple Choice Questions.” South African Journal of
Chemistry 52 (4): 161–164.
Sozbilir, M. 2003. “A Review of Selected Literature on Students’ Misconceptions of Heat
and Temperature.” Boğaziçi University Journal of Education 20 (1): 25–41.
Tabachnick, B. G., and L. S. Fidell. 2007. Using Multivariate Statistics. Boston, MA:
Pearson Education Inc.
Taber, K. S. 2000a. “Chemistry Lessons for Universities?: A Review of Constructivist Ideas.”
University Chemistry Education 4 (2): 63–72.
Taber, K. S. 2000b. “Finding the Optimum Level of Simplification: The Case of Teaching
about Heat and Temperature.” Physics Education 35 (5): 320–325.
Tamir, P. 1971. “An Alternative Approach to the Construction of Multiple-choice Test Items.”
Journal of Biological Education 5 (6): 305–307.
Tan, K. C. D., N. K. Goh, L. S. Chia, and D. F. Treagust. 2002. “Development and Applica-
tion of a Two-tier Multiple Choice Diagnostic Instrument to Assess High School
Students’ Understanding of Inorganic Chemistry Qualitative Analysis.” Journal of
Research in Science Teaching 39 (4): 283–301.
Tan, K. C. D., K. S. Taber, N. K. Goh, and L. S. Chia. 2005. “The Ionisation Energy
Diagnostic Instrument: A Two-tier Multiple-choice Instrument to Determine High School
Students’ Understanding of Ionisation Energy.” Chemistry Education Research and
Practice 6 (4): 180–197.
 20 D. Gurcay and E. Gulbas

Tanahoung, C., R. Chitaree, and C. Soankwan. 2010. “Probing Thai Freshmen Science
Students’ Conceptions of Heat and Temperature Using Open-ended Questions: A Case
Study.” Eurasian Journal of Physics and Chemistry Education 2 (2): 82–94.
Thomaz, M. F., I. M. Malaquias, M. C. Valente, and M. J. Antunes. 1995. “An Attempt to
Overcome Alternative Conceptions Related to Heat and Temperature.” Physics Education
30: 19–26.
Treagust, D. F. 1988. “Development and Use of Diagnostic Tests to Evaluate Students’
Misconceptions in Science.” International Journal of Science Education 10 (2): 159–169.
Treagust, D. F. 2006. “Diagnostic Assessment in Science as a Means to Improving Teaching,
Learning and Retention.” UniServe Science–Symposium Proceedings: Assessment in
science teaching and learning, 1–9. Sydney, Australia: Uniserve Science, Sydney.
Warren, J. W. 1972. “The Teaching of the Concept of Heat.” Physics Education 7: 41–44.
Wessel, W. 1999. “Knowledge Construction in High School Physics: A Study Student Tea-
cher Interaction.” Accessed February, 2011. http://www.saskschoolboards.ca/old/Re
searchAndDevelopment/ResearchReports/Instruction/99-04.htm
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

Wiser, M. 1986. The Differentiation of Heat and Temperature: An Evaluation of the Effect of
Microcomputer Teaching on Students’ Misconceptions.” Technical Report 87–5.
Cambridge, MA: Educational Technology Center. ED 291 592.
Wiser, M., and D. Kipman. 1988. The Differentiation of Heat and Temperature: An Evalua-
tion of the Effect of Microcomputer Models on Students’ Misconceptions. Technical
Report 88–20. Cambridge, MA: Educational Technology Center.
Wiser, M., D. Kipman, and L. Halkiadakis. 1988. Can Models Foster Conceptual Change?
The Case of Heat and Temperature. Technical Report 88–7. Cambridge, MA: Educational
Technology Center.
Yağbasan, R., and Ç. Gülçiçek. 2003. “Describing the Characteristics of Misconceptions in
Science Teaching.” Pamukkale University Journal of Education 1 (13): 102–120.
Yeo, S., and M. Zadnik. 2001. “Introductory Termal Concept Evaluation: Assessing Students’
Understanding.” The Physics Teacher 39: 496–504.
Yeşilyurt, M. 2006. “High School Students’ Views about Heat and Temperature Concepts.”
International Journal of Environmental and Science Education 1 (1): 1–24.
Yılmaz, S., and A. Eryılmaz. 2010. “Integrating Gender and Group Differences into Bridging
Strategy.” Journal of Science Education and Technology 19 (4): 341–355.

Appendix 1. Three example questions from three-tier HTIEDT

A baker bakes two different types of bread from the same bread dough, with one loaf being
bigger than the other. The baker bakes the big and small loaves in a working bakery oven
under the same conditions and for the same time. (The loaves do not physically touch each
other. The baking time of the loaves is equal.)
Answer the questions below according to the information given above.
(1.1) Compare the instantaneous heat of the big and small loaves of bread after coming
out of the oven.

(A) Big loaf of bread has higher heat.

(B) Small loaf of bread has higher heat.
(C) Their heat is equal.
(D) Their heat cannot be compared.
(E) Other. Please write……………….

(1.2) Which one below is the reason for your answer to the previous question?

(A) Heat depends on the size of the loaf of bread.

(B) Heat does not depend on the size of the loaf of bread.
(C) The oven has put equal heat in both loaves of breads.
(D) Heat is the energy transferred due to temperature difference.
 Research in Science & Technological Education 21

(E) The loaf of bread that has higher temperature has higher heat.
(F) Heat cannot be measured.
(G) Other. Please write……………….

(1.3) Are you sure about the answers of the previous two questions?

(A) I am sure.
(B) I am not sure.

(2.1) Compare the instantaneous temperature of the big and small loaves of bread after
coming out of the oven.

(A) Big loaf of bread has higher temperature.

(B) Small loaf of bread has higher temperature.
Downloaded by [University of Nebraska, Lincoln] at 00:43 09 April 2015

(C) Their temperature is equal.

(D) Their temperature cannot be compared.
(E) Other. Please write……………….

(2.2) Which one below is the reason for your answer to the previous question?

(A) Temperature depends on the size of the loaf of bread.

(B) Temperature does not depend on the size of the loaf of bread.
(C) The oven has put equal heat in both loaves of breads.
(D) Temperature is average energy.
(E) The loaf of bread that has higher heat has higher temperature.
(F) The temperature of the loaf of bread cannot be mentioned.
(G) Other. Please write……………….

(2.3) Are you sure about the answers of the previous two questions?

(A) I am sure.
(B) I am not sure.

(3.1) Compare the instantaneous internal energy of the big and small loaves after coming
out of the oven.

(A) Big loaf of bread has higher internal energy.

(B) Small loaf of bread has higher internal energy.
(C) Their internal energy is equal.
(D) Their internal energy cannot be compared.
(E) Other. Please write……………….

(3.2) Which one below is the reason for your answer to the previous question?

(A) Internal energy depends on the size of the loaf of bread.

(B) Internal energy does not depend on the size of the loaf of bread.
(C) The oven has put equal heat in both loaves of bread.
(D) The loaf of bread that has higher heat has higher internal energy.
(E) The loaf of bread that has higher temperature has internal energy.
(F) Internal energy cannot be measured.
(G) Other. Please write……………….

(3.3) Are you sure about the answers of the previous two questions?

(A) I am sure.
(B) I am not sure.