Thinking Skills and Creativity 43 (2022) 100999

Contents lists available at ScienceDirect

Thinking Skills and Creativity

journal homepage: www.elsevier.com/locate/tsc

Exploring the structure of students’ scientific higher order

thinking in science education
He Sun a, Yueguang Xie a, *, Jari Lavonen b
a
Faculty of Information Science and Technology, Northeast Normal University, 130117 JingYue Avenue, ChangChun, JiLin, China
b
Faculty of Educational Sciences, University of Helsinki, Finland

A R T I C L E I N F O A B S T R A C T

Keywords: Science education research has long focused on scientific higher-order thinking (S-HOT) and
Higher-order thinking observed that students’ HOT plays an important role in the science learning process. However, the
Factor structure existing research has tended to overlook the Comprehensive nature of scientific higher-order
Science education
thinking (S-HOT) and stopped short of defining its unique features. In response, this study pre­
Educational technology
sents a novel model for S-HOT that synthesises five competencies. To confirm the structure of S-
HOT, the study developed a new questionnaire to measure students’ S-HOT. The study’s partic­
ipants included 1,153 students from 15 junior high schools in China. Exploratory and confir­
matory factor analyses indicated that the developed S-HOT survey had satisfactory validity and
reliability. Structural equation modelling was used to confirm the cause-and-effect relationships
between creative thinking, critical thinking, metacognition, science self-efficacy, and scientific
reasoning in the S-HOT structure. The results showed that the average score of male students was
significantly higher than that of female students in creative thinking. It was also revealed that
there was a correlation between the five competencies: metacognition had a significant path to
scientific reasoning, science self-efficacy had a significant path to critical and creative thinking,
and scientific reasoning directly affected science self-efficacy.

1. Introduction

One of the most important issues for science education reform and innovations has been the integration of the use of educational
technology (Lavonen et al., 2004; Osborne, 2013). In this situation, a higher level of students’ skills is putting forward for re­
quirements. Some international tests such as the TIMSS, PISA, and NAEP seek to assess students’ learning in science with such
educational technology.
Students are required to develop skills to handle the rapidly changing world that awaits them, as well as take part in formal science
learning curricula (Nesbitt-Hawes, 2005). In particular, all students must develop their higher-order thinking (HOT) skills, not just
high-achieving students ((Tal et al., 2000); Zohar & Dori, 2003). FitzPatrick and Schulz (2015) found that HOT in science is an
essential thinking skill for all students. In China, the junior high school science subjects include Physics, Chemistry, Biology, and
Geography. Of these, Physics and Chemistry are known as Material sciences, Biology forms part of the life sciences, and Geography is
an earth, cosmos, and space science. The ‘Core Literacy for Chinese Student Development’ guide focuses on growing students’
low-level thinking to HOT, and HOT represents the higher pursuit of disciplinary education values (Xie, 2017). Hence, fostering HOT in

* Corresponding author.
E-mail address: xyg6367@126.com (Y. Xie).

https://doi.org/10.1016/j.tsc.2022.100999
Received 3 August 2021; Received in revised form 31 December 2021; Accepted 3 January 2022
Available online 5 January 2022
1871-1871/© 2022 Elsevier Ltd. All rights reserved.
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

the sciences is an increasing concern in education institutions, especially in the rich technology class environment, as is ensuring that
educators adopt appropriate approaches to facilitate students’ thinking development.
The nature of scientific HOT (S-HOT) is becoming increasingly obvious with the deepening of the related research. Recently,
several studies have proposed that the development of S-HOT in science education should be designed based on Bloom’s taxonomy
(Istiyono, Dwandaru, Setiawan & Megawati, 2020; Ong, Hart & Chen, 2016; Ramos, Dolipas & Villamor, 2013; (Saido et al., 2017);
Setyasih, Hartono & Prasetyo, 2019; Subia, Marcos, Valdez, Pascual & Liangco, 2020; Takko et al., 2020; WALİD, SAJİDAN, RAMLİ &
KUSUMAH, 2019). Students’ thinking skills enable them to work on problems involving operational analysis, evaluation, and crea­
tivity, which are features that are used to design S-HOT problems and teaching strategies. The idea is to observe students’ HOT levels in
science education, then enhance them. However, the results have been limited and have not fully reflected the comprehensive
characteristics of students’ S-HOT. As influential as Bloom’s taxonomy has been on educational practice, it has been subject to severe
criticism (Kreitzer & Madaus, 1994). One of the most common criticisms is that the taxonomy oversimplifies the nature of thought and
its relationship to learning (Furst, 1994). However, the model of HOT is multidimensional and more constructivist in nature
(Mota-Valtierra, Rodríguez-Reséndiz & Herrera-Ruiz, 2019; Ortiz-Echeverri, Rodríguez-Reséndiz & Garduño-Aparicio, 2018).
This characterisation of HOT is often overlooked when considering the school curriculum (Resnick, 1987), and Bloom’s taxonomy
simply does not support this structure (Marzano & Kendall, 2007). For example, educators who were trained in the structure of Bloom’s
taxonomy were consistently unable to recognise questions at higher levels of the taxonomy as more difficult than questions at lower
levels ((Fairbrothers et al., 1975); (Kirsch and Poole, 1972); Michael, Stanley & Bolton, 1957); therefore, the hierarchical structure of
Bloom’s taxonomy did not hold together well from either the logical or empirical perspective (Marzano & Kendall, 2007). Anderson
and Krathwohl (2001) developed a new taxonomy for categorising knowledge and cognitive processes, which was based on Bloom’s
taxonomy but critiqued the order of the levels. To date, however, the science education research community has failed to meet the need
to develop valid and reliable models that can contribute to developing students’ HOT within science education (Songer & Ruiz-Primo,
2012). We may still know little about the specific nature of S-HOT in science education, however, there is no doubt that science
educators need such an S-HOT model to improve their teaching and develop students’ thinking skills as information technology is
gradually integrated into instruction. The difficulties of developing a model of S-HOT in practice represent past experiences, and this is
attributed to a variety of causes, including teachers’ lack of understanding of the expression of HOT in science education, the vagueness
of the S-HOT concept, and the difficulties inherent in investigating specific methods to enhance students’ S-HOT.
Over the years, several science education researchers have focused on the complexity and importance of HOT in science. In much of
this research, the authors describe critical thinking, creative thinking, scientific process, and reasoning thinking as comprising S-HOT,
including complex reasoning and scientific processes, before expanding the exploration of learning models (Miri, David & Uri, 2007;
Nesbitt-Hawes, 2005; Osborne, 2013; Ramos et al., 2013; Zohar & Dori, 2003). Various contributions have collected data by using a
framework of the characteristics of critical thinking to analyse students’ performance in science education. It is well-known that
critical thinking or other single skills cannot represent all the skills students must develop, especially HOT. HOT is defined as not only a
set of comprehensive abilities but also through various other descriptions, such as the ability to devise complex, nonalgorithmic,
multiple solutions; make nuanced judgments and interpretations; and multiple other criteria involving uncertainty, self-regulation,
imposing meaning, and effort. Hofer and Pintrich (1997) noted that metacognition is becoming recognised as an important compo­
nent of HOT, and other researchers have adopted the common perspective of ‘HOT about cognition’ (Flavell, 1979; Nelson, Stuart,
Howard & Crowley, 1999).
When HOT skills are required to solve complex problems, individuals begin to employ multiple skills to reach their goal (Magno,
2010). In the US science standards (National Research Council, 2012b), higher-order reasoning has been considered to require the use
of a set of scientific practices, and performances of higher-order reasoning provide students with the ability to conduct suitable sci­
entific reasoning (Osborne, 2013). In science education, science self-efficacy influences students’ science-related activities as they
experience success in science (Bandura, 1997; (Britner and Pajares, 2006) ; Zeldin & Pajares, 2000). Traditionally, metacognitive skills
have been considered central to the development of scientific reasoning by providing the epistemological basis, conceptual motivation,
and cognitive control for individuals to deploy underlying competencies in science learning tasks (Hogan & Maglienti, 2001; Kuhn,
2000; Kuhn & Pearsall, 1998; Kuhn & Pease, 2006; Pintrich, Marx & Boyle, 1993; Schauble, Klopfer & Raghavan, 1991). The concept of
metacognition has generally taken the viewpoint of comprising ‘higher-order cognition about cognition’ and disengages higher-order
from lower-order skills in human consciousness (Flavell, 1979; Nelson et al., 1999).
However, the existing science scholarship has tended to overlook the comprehensive nature of S-HOT and stopped short by gen­
eralising the unique features of S-HOT in student science learning. Another limitation associated with the HOT literature is that it has
focused on only a few characteristics within the theoretical frameworks. Therefore, the present research aims 1) to gain a deeper
insight into S-HOT in science education by explaining its constitutive concepts (scientific reasoning, critical thinking, creative
thinking, science self-efficacy, and metacognition); and 2) examine whether and how these abilities relate to science learning. The
study’s significance is twofold: (1) First, the structure of S-HOT was identified during the data analysis, and causal relationships in the
ability to form S-HOT structures in the context of science learning were revealed; and (2) the study’s results may contribute to further
research on, and the practice of, using appropriate teaching methods in science education.

2. Literature review

The correlations between scientific reasoning, critical thinking, creative thinking, science self-efficacy, and metacognition were
reviewed based on the existing research evidence.

2
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

2.1. Critical and creative thinking in science

Some previous empirical studies in the field of science education have demonstrated a strong correlation between creative and
critical thinking (Bailin, 2002; Chen, 2021; Forawi, 2016; Magno, 2010; Romero, 2017). Especially in a science enquiry situation, both
creative and critical thinking are needed in different phases of enquiry. Within this process, creative thinking, which is imaginative and
‘the artist’s way of thinking’, puts more emphasis on brainstorming and generating ideas and alternatives. Critical thinking, which is
mainly inquisitive and ‘the detective’s way of thinking’, emphasises the analytical and systematic dimensions (understanding and
decomposing the problem, etc.) (Vincent-Lancrin et al., 2019).
Critical thinking in science includes developing research questions, forming hypotheses, designing experiments or investigations,
analysing and interpreting data, drawing conclusions, and forming scientific explanations ((Osborne et al., 2003); Zohar & Dori, 2003).
The concept of creative thinking in science is related to scientific thinking, exploratory processes, and problem-solving (Craft, 2001;
Sternberg, 2006), and its process involves fluency, flexibility, originality, and elaboration (Munandar, 2012).

2.2. Self-efficacy in science

Self-efficacy refers to individuals’ beliefs about their abilities to excel (or perform) in academic activities (A. Bandura, 1986). In
science tasks or activities, students’ science self-efficacy influences their science-related choices, their engagement or perseverance
when they meet difficulties, and the confidence they have that they will succeed (Ballen, Wieman, Salehi, Searle & Zamudio, 2017;
Ferrell & Barbera, 2015; Linnenbrink & Pintrich, 2003). The PISA Scientific Literacy 2015 Framework defined students’ science related
self-efficacy beliefs as the strength of their beliefs that they will be able to solve tasks (OECD, 2015).
When students have more self-efficacy, they are more likely to solve questions with critical thinking; therefore, self-efficacy has a
positive influence on critical thinking (Phan, 2009) and is also related to increased creative thinking (Liu, Pan, Luo, Wang & Pang,
2017).

2.3. The use of metacognition

Metacognition is regarded as an individual’s ability to think about their current cognitive processes (Flavell, 1976). Metacognition
is called ‘cognition about cognition’ and refers to individuals’ knowledge about, and awareness and control of, their cognitive activities
(Nelson, 1990). This type of cognition is classified into three components: metacognitive knowledge, metacognitive experience, and
metacognitive monitoring and control (Flavell, 1979). The most frequently used metacognition in science education is regulation and
control (L Rusyati, Widodo & Ha, 2020).
Metacognition has a positive relationship with critical thinking and science reasoning (Amsel et al., 2008; Zimmerman, 2000,
2007), as well as a strong correlation with self-efficacy and creative thinking (Coutinho & Neuman, 2008; Puente-Díaz, Cav­
azos-Arroyo & Vargas-Barrera, 2021; Schmidt & Ford, 2003). However, Priest (2006) and Kaufman, Evans and Baer (2010) found that
the correlations between creative thinking and metacognitive monitoring and control were not significant in the art, writing, and
musical fields. Hong, Neil and Peng (2016) observed that when individuals’ metacognitive monitoring and control were controlled,
they had no significant effect on creative thinking. Some scholars have also revealed that good critical thinkers engage in more
metacognitive activities, especially high-level planning and high-level evaluation strategies (Ku & Ho, 2010), and that metacognition
affects critical thinking (Magno, 2010).

Fig. 1. S-HOT research framework.

3
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

2.4. Scientific reasoning

In early studies, the concept of scientific reasoning was derived from Piaget’s ‘formal reasoning’. In science education, scientific
reasoning skills include the ability to systematically explore a problem, formulate and test hypotheses, manipulate and isolate vari­
ables, and observe and evaluate the consequences (Bao, 2009). The scientific reasoning model is used to assess student skills based on
three fundamental practices: hypothesising, experimenting, and evidence evaluation (Osborne, 2013).
Science reasoning and critical thinking play important roles in science education, and some empirical studies have proposed that
there is a positive significant link between scientific reasoning and critical thinking (Quitadamo & Kurtz, 2007). However, the rela­
tionship between these skills remains to be clarified. There is a positive and significant correlation between reasoning and self-efficacy
because an individual’s reasoning ability not only influences what tasks they can successfully complete but also their self-efficacy
(Lawson, Banks & Logvin, 2007).
Critical thinking and scientific reasoning are similar but different constructs that include various types of higher-order cognitive
processes, metacognitive strategies, and dispositions involved in making meaning from information (Dowd, Thompson, Schiff &
Reynolds, 2018). From a science education perspective, the present study’s framework was based on the previous theoretical and
empirical evidence on the five competencies that comprise S-HOT. The hypothetical model is shown in Fig. 1.

3. Methods

In this study, we adopted a mixed research method: (1) a new questionnaire was developed to measure S-HOT; (2) quantitative
analysis was employed to explore and confirm the survey structure; (3) SEM analysis was used to ensure the associations between the
five skills shown in Fig. 1. The participants, research instruments, and data analysis are elaborated in the following.

3.1. Participants

The study’s participants included altogether 1153 students (552 female) from 15 junior high schools in Suzhou City, Shenzhen City,
Yinchuan City, Ningbo City, Foshan City, Harbin City, Changchun City, and Shandong City in China. The students ranged in age from
14 to 16 years old. According to the study’s purpose, we collected data from 378 students (190 female) from junior high schools in
Suzhou, Shenzhen, Changchun, Yinchuan for exploratory analyses. Thereafter, data from 675 students (362 female) were collected
from junior high schools in Foshan, Ningbo, Shandong, and Changchun for confirmatory analyses. Data collection took place in
classrooms over a period of three months. In the exploratory factor analysis (EFA) part of the study, the students used answer sheets to
respond to the questions. In the confirmatory factor analysis (CFA) part of the study, the students used computers to answer questions
online. Testing was conducted by three teachers in each classroom under the supervision of the project manager. All participants
voluntarily agreed to the conditions of participation in the study, and the data were kept anonymous and confidential. All the
participating students completed the survey.

3.2. Research instruments

We referred to the Scale of Higher-Order Thinking ((Yulian and Yueguang, 2017)) and developed a novel questionnaire to measure
students’ S-HOT based on the S-HOT model. A five-point Likert scale from ‘very often’ to ‘never’ was used to gather information on
students’ actual S-HOT.

3.2.1. Critical and creative thinking in science

The California Critical Thinking Disposition Inventory (CCTDI) (Facione, Facione & Giancarlo, 1996) and the Torrance Test of
Creative Thinking (TTCT) (Torrance, Orlow & Safter, 1990) were referenced to design the critical and creative thinking questions. The
questions were designed to incorporate the higher-order features within scientific literacy. The critical thinking and creative thinking
in science scale items included 5–6 items for each of the two scales (e.g. ‘I like to make conjectures or hypotheses about scientific
problems and test them’ and ‘I can come up with a different solution or express my own unique opinion’).

3.2.2. Science self-efficacy

According to Bandura, self-efficacy is influenced by four principal sources: mastery experiences, vicarious experiences, emotional
arousal, and verbal persuasion (A. Bandura, 1986). We referred to the PISA 2015 test’s definition of the concept of science self-efficacy.
The science self-efficacy scale items included four items (e.g. ‘I am aware of my ability to successfully complete a learning task or solve
a scientific problem’ and ‘When I encounter difficulties in solving a problem, I will be brave enough to make some effort and try to
overcome them’).

3.2.3. Metacognitive skills

Metacognitive skills cannot be directly measured; instead, they must be inferred from students’ behaviours (Veenman, 2007). The
metacognitive skills that were measured included students’ monitoring of their problem-solving process, improving their
problem-solving solutions, analysing why they could not solve the problem, and developing learning plans. This scale included five
items (e.g. ‘I will develop a study plan based on the learning objectives’ and ‘When my learning outcomes do not reach expectations, I
analyse the reasons why and develop a plan for improvement’).

4
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

3.2.4. Scientific reasoning

The scientific reasoning scale items pertained to four dimensions: generating investigation questions; refining the questions so that
they can be investigated (e.g. ‘I can recognise the authenticity of the information I obtain from the Internet’); using prior knowledge to
inform the design (e.g. ‘I use my scientific knowledge and investigative skills to solve real-life problems’); determining if the design is
an appropriate match for the question, including variables and procedures; and ensuring that the conclusions are supported by the data
(e.g. ‘I gathered information from a variety of sources and thought it over before making a decision’) (Koslowski, 1996).

3.3. Data analysis

The final version of the S-HOT survey was developed by reducing the number of items. To ensure that the survey was valid, reliable,
and structurally sound, the following steps were undertaken.
An exploratory factor analysis was used to reduce the large number of original variables to a smaller number of factors. In the EFA,
the significant Bartlett’s test (Bartlett, 1954) and the value of Kaiser-Meyer-Olkin measure exceeding 0.60 (Kaiser, 1974) were first
employed for structure detection. A Cronbach’s alpha value of 0.7 or above is considered to be evidence of internal consistency (Brink,
1998). A parallel analysis was also conducted for factor extraction (Henson & Roberts, 2006). Thereafter, principal component analysis
with varimax rotation was conducted, and the varimax rotation was used to create orthogonal factors (Dunteman, 1989; Smith, 2002).
Since orthogonal rotation maximises the sum of the variances of the squared loadings (squared correlations between variables and
factors), this method was used to obtain the commonality factor of each scale and reduce redundancies. This method allowed the
number of variables to be compressed and for fewer variables to be used to explain most of the variables in the original data, elim­
inating redundant information. It also eliminated the covariance between the original variables and overcame the resulting arithmetic
instability. The acceptable level of factor loading was set at 0.5 and above, and items with many cross-factor loadings were omitted
(Bentler, 1990).
In the CFA, the validity and reliability of the questionnaire were explored to clarify its resulting structures. Maximum likelihood
estimation was used to evaluate the measurement model. The goodness of fit structure model included the root mean squared error of
approximation (RMSEA), comparative fit index (CFI) and Tucker Lewis index (TLI). The RMSEA was below 0.5 (MacCallum, Browne &
Sugawara, 1996), and the GFI, CFI, and TLI were all above 0.95, indicating a perfect fit (Hu & Bentler, 1999). In addition, to explore the
correlations between critical thinking, creative thinking, metacognition, science self-efficacy, and scientific reasoning, participants’
final questionnaire responses were subjected to a series of structural equation modelling (SEM) analyses, which was the main method
used to assess the causal relationships between the variables (Suhr, 2006).

Fig. 2. Results of parallel and raw data analysis.

5
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

4. Results and discussion

4.1. EFA results

The reliability (Cronbach’s Alpha) coefficients for the factors were 0.846, 0.816, 0.805, 0.725, 0.772, respectively, and the overall
alpha was 0.929, suggesting that the scales are measuring students’ S-HOT. The Kaiser-Meyer-Olkin measure was within a very
reasonable range; KMO = 0.944. Bartlett’s test of sphericity also supported the use of a factor analytic approach (χ 2 = 3614.110, df =
253, p < 0.001). The Bartlett’s test and Kaiser-Meyer-Olkin measure indicated that the data were factorable. The number of factors
according to the percentile of parallel analysis was five (Fig. 2). Table 1 shows that five factors were extracted by using rotated factor
analysis. A total of 23 items were retained in the final version of the S-HOT survey. The five factors jointly accounted for 61.74% of the
variance observed. Each factor was able to account for 13.71%, 13.38%, 12.87%, 11.2%, and 10.58% of the variance for Factors 1, 2, 3,
4, and 5, respectively.

4.2. CFA results

4.2.1. CFA for the S-HOT survey

AMOS 22.0 was used to determine the CFA factor loadings of each item of the S-HOT survey (Table 2). The fitness of the items for
the S-HOT survey (chi-square per degree of freedom = 397.755, RMSEA = 0.045, CFI = 0.960, TLI = 0.953) indicated a sufficient fit
and helped confirm the questionnaire’s structure. At first, the measurement did not fit the data. Overall, the loading factors of the CRI5
item were < 0.5, and it was, therefore, omitted from the final version. The loading factors of the remaining 22 items were all > 0.5, and
they were, therefore, considered important ((Stevens, 2012)).

4.2.2. Correlation analyses for the S-HOT

Table 3 shows the 675 (male = 313, female = 362) students’ average item scores and standard deviations on the S-HOT survey. The
students’ levels of different S-HOT competencies were ordered as follows (from high to low): critical thinking, science self-efficacy,
scientific reasoning, metacognition, creative thinking. The results showed that most students’ performance tended to be average,
with only their creative thinking skills being generally lower. A t-test revealed no significant difference in critical thinking, meta­
cognition, science self-efficacy, or scientific reasoning between the male and female students. However, the male students scored
significantly higher in creative thinking than the female students (3.2 vs. 2.9; t = 3.8, p < 0.00).
The factor covariances in Table 4 show the correlations between the S-HOT skills. The table lists the standardized estimated co­
efficients of the correlations of the S-HOT. Moderate correlations (0.547–0.663) were observed between critical thinking, creative

Table 1
Results of the EFA and reliability of the S-HOT survey.
Indicators Factor 1 Factor 2 Factor 3 Factor 4 Factor 5

Factor 1: Creative thinking, alpha = 0.836

CRE1 0.758
CRE2 0.744
CRE3 0.705
CRE4 0.644
CRE5 0.638
Factor 2: Science self-efficacy, alpha = 0.816
SSE1 0.688
SSE2 0.642
SSE3 0.602
SSE4 0.522
Factor 3: Metacognition, alpha = 0.805
ME1 0.791
ME2 0.742
ME3 0.640
ME4 0.550
ME5 0.501
Factor 4: Critical thinking, alpha = 0.725
CRI1 0.628
CRI2 0.624
CRI3 0.621
CRI4 0.576
CRI5 0.507
Factor 5: Scientific reasoning, alpha = 0.772
SR1 0.726
SR2 0.661
SR3 0.544
SR4 0.530

Note. N = 378; CRE = Creative thinking; SSE = Science self-efficacy; ME = Metacognition; CRI = Critical thinking; SR = Scientific reasoning.

6
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

Table 2
Results of the CFA.
Indicators Factor 1 Factor 2 Factor 3 Factor 4 Factor 5

CRE1 0.7
CRE2 0.661
CRE3 0.620
CRE4 0.616
CRE5 0.601
SSE1 0.762
SSE2 0.680
SSE3 0.610
SSE4 0.608
ME1 0.831
ME2 0.804
ME3 0.779
ME4 0.753
ME5 0.686
CRI1 0.648
CRI2 0.637
CRI3 0.661
CRI4 0.725
CRI5 0.437
SR1 0.677
SR2 0.652
SR3 0.636
SR4 0.542

Note. N = 675; CRE = creative thinking; SSE = science self-efficacy; ME = metacognition; CRI = critical thinking; SR = scientific reasoning.

Table 3
Summary statistics of male and female students’ S-HOT.
Scale Male studentsN = 313 Female studentsN = 362 All participantsN = 675
M SD M SD M SD

Creative thinking 3.2 0.74 2.9 0.68 3.09 0.72

Critical thinking 3.68 0.75 3.67 0.69 3.68 0.72
Metacognition 3.56 0.83 3.61 0.79 3.59 0.81
Science self-efficacy 3.75 0.73 3.76 0.69 3.76 0.71
Scientific reasoning 3.73 0.73 3.7 0.71 3.72 0.72

M: Mean, SD: standard deviation.

Table 4
Correlations amongst S-HOT competencies. Std. estimate for all participants (N = 675).
Creative thinking Metacognition Critical thinking Science self-efficacy Science reasoning

Creative thinking 1.00

Metacognition 0.718** 1.00
Critical thinking 0.716** 0.547** 1.00
Science self-efficacy 0.866** 0.842** 0.66** 1.00
Scientific reasoning 0.837** 0.66** 0.663** 0.768** 1.00
**
p < 0.001.

thinking, metacognition, science self-efficacy, and scientific reasoning. A stronger correlation was detected between creative thinking
and metacognition (0.718) and critical thinking (0.716), and between science reasoning and creative thinking (0.768). Strong cor­
relations were identified between science self-efficacy and creative thinking (0.866) and metacognition (0.842), and between science
self-efficacy and metacognition (0.842).
The data from the CFA was used to test the structural equation model. Results for the measurement model showed an acceptable
model fit χ 2 = 419.472, p < 0.000, df =179, RMSEA = 0.036, CFI = 0.958, and TLI = 0.51. However, there was no main effect for the
relation between metacognition and creative thinking (β = 0.051; p = 0.18) or critical thinking (β = 0.059; p = 0.102). After deleting
the insignificant paths, the final S-HOT model was developed (Fig. 2), and the goodness of the S-HOT model fit improved (χ 2 =
422.868, p < 0.000, df =181, RMSEA = 0.045, CFI = 0.958, and TLI = 0.951). Finally, metacognition (β = 0.658; p < 0.001) directly
affected scientific reasoning. Meanwhile, science self-efficacy (β = 0.82; p < 0.001) (β = 0.621; p < 0.001) directly affected critical
thinking and creative thinking, and scientific reasoning (β = 0.327; p < 0.001) directly affected science self-efficacy (Fig. 3).
The results reveal that the different components of metacognition had different effects on the creative thinking process (Jia, Li &
Cao, 2019). In science education, S-HOT is even more evident in the learning activities process in the classroom. The process of creative

7
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

Fig. 3. Structural equation modelling of S-HOT.

thinking involves ‘self-monitoring’ (Sternberg, 1985). In this model, creative and critical thinking were both affected by self-efficacy,
with higher trust strengthening the relation. This means that the relations between self-efficacy and the creative thinking and critical
thinking measures became stronger as self-efficacy increased.
The effect of science self-efficacy on the critical thinking of the junior high school students in this study was stronger than that of
science self-efficacy on creative thinking. Students’ confidence can help them use critical thinking when solving scientific problems.
Moreover, the effect of metacognition on scientific reasoning was stronger than that of scientific reasoning on science self-efficacy. This
means that individuals’ metacognition has a positive moderating effect on their scientific reasoning, thereafter enhancing their sci­
entific self-efficacy. When individuals’ science self-efficacy is increased, creative and critical thinking also increase. As there are
positive correlations between metacognition and scientific self-efficacy, and scientific reasoning and critical and creative thinking,
these abilities also contribute to each other.

5. Conclusion

This study contributes to the advancement of the structure of students’ S-HOT in science education. First, the study highlighted the
lack of a defined S-HOT structure as an essential issue that must be solved. To respond to this problem, our research posited creative
thinking, critical thinking, metacognition, science self-efficacy, scientific reasoning as indicators of S-HOT and explored their inter­
action with each other. To facilitate this, an S-HOT survey was composed, based on previous scales and interventions, to measure
students’ performance and help confirm the structure of S-HOT. This S-HOT survey can help to determine students’ level of S-HOT.
Second and relatedly, the results revealed that there is a correlation between the five competencies. We highlighted the unique
features of the S-HOT model within science learning that can be used to design rational interventions; therefore, the study also offers a
significant contribution by proposing better approaches to classroom instruction.
This study’s findings allow us to make various recommendations for science education; for example, the necessity to improve the
creative thinking skills of all students, especially female students. The results show that it is most effective to enhance metacognition
and science self-efficacy and provide students with continuous encouragement. In the science learning process, suitable guidance
facilitates students’ experience of doing and truth-seeking, especially in the context of scientific reasoning. In science enquiry learning
practice, teachers should focus on increasing students’ science self-efficacy to promote creative and critical thinking. In addition, we
must increase the moderating role of metacognition to promote the development of scientific reasoning in order to subsequently
promote students’ overall S-HOT.
This study has some limitations. First, more research is needed to examine the specific skills assessment of S-HOT during the
different stages. Second, the extent of the investigation was limited. Future studies should examine students’ S-HOT while taking into
account their differing economic status, the quality of schools, and students’ academic performance. Further research also can apply
this model to allow students to practice solving complex problems and explore the design of instruction in problem-solving. Suitable
teaching approaches and methods will be explored in future that can facilitate the learning of different types of knowledge and further
S-HOT development.

8
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

CRrediT authorship contribution statement

He Sun (First Author): Writing- Original draft preparation, Conceptualization, Methodology, Investigation, Data curation, Soft­
ware, Validation. Yueguang Xie (Corresponding Author):Supervision, Writing- Original draft preparation, Project Administration. Jari
Lavonen: Writing- Reviewing and Editing

Fundings

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This
work is supported by The National Social Science Fund of China under grant number BCA190074

CRediT authorship contribution statement

He Sun: Conceptualization, Methodology, Investigation, Data curation, Software, Validation. Yueguang Xie: Supervision, Project
administration. Jari Lavonen: Writing – review & editing.

References

Amsel, E., Klaczynski, P. A., Johnston, A., Bench, S., Close, J., Sadler, E., et al. (2008). A dual-process account of the development of scientific reasoning: The nature
and development of metacognitive intercession skills. Cognitive Development, 23(4), 452–471. https://doi.org/10.1016/j.cogdev.2008.09.002
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives. Longman.
Bailin, S. (2002). Critical thinking and science education. Science & Education, 11(4), 361–375.
Ballen, C.J., .Wieman, C., Salehi, S., Searle, J.B., .& Zamudio, K.R. (.2017). Enhancing diversity in undergraduate science: Self-efficacy drives performance gains with
active learning. CBE—Life Sciences Education,16(4), ar56.
Bandura, A. (1986a). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
Bandura, A. (1986b). The explanatory and predictive scope of self-efficacy theory. Journal of social and clinical psychology, 4(3), 359–373.
Bandura, A. (1997). The anatomy of stages of change. American journal of health promotion: AJHP, 12(1), 8–10.
Bao, L. (2009). Learning and scientific reasoning. Education forum, 323.
Bartlett, M. S. (1954). A note on multiplying factors for various chi square approximations. Journal of the Royal Statistical Society, 16, 296–298.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246.
Brink, P. J., & Wood, M. J. (1998). Advanced design in nursing research. Newbury Park: Sage Publications.
Britner, S. L., & Pajares, F. (2006). Sources of science self-efficacy beliefs of middle school students. Journal of Research in Science Teaching: The Official Journal of the
National Association for Research in Science Teaching, 43(5), 485–499.
Chen, X. M. (2021). Integration of creative thinking and critical thinking to improve geosciences education. The Geography Teacher, 18(1), 19–23. https://doi.org/
10.1080/19338341.2021.1875256
Coutinho, S. A., & Neuman, G. (2008). A model of metacognition, achievement goal orientation, learning style and self-efficacy. Learning Environments Research, 11(2),
131–151. https://doi.org/10.1007/s10984-008-9042-7
Craft, A., Jeffrey, B., & Leibling, M. ((2001).Eds.). Creativity in education. A&C Black.
Dowd, J.E., .Thompson, R.J., .Jr., Schiff, L.A., .& Reynolds, J.A. (.2018). Understanding the complex relationship between critical thinking and science reasoning
among undergraduate thesis writers. CBE life sciences education, 17(1), doi:10.1187/cbe.17-03-0052.
Dunteman, G.H. (.1989). Principal components analysis (No. 69). Sage.
Facione, P. A., Facione, C. F., & Giancarlo, C. A. (1996). The California critical thinking disposition inventory test manual. Millbrae, CA: California Academic Press.
Fairbrothers, D. E., Mabry, T. J., Scogin, R. L., & Turner, B. L. (1975). The bases of angiosperm phylogeny: Chemotaxonomy. Annals of the Missouri Botanical Garden,
765–800.
Ferrell, B., & Barbera, J. (2015). Analysis of students’ self-efficacy, interest, and effort beliefs in general chemistry. Chemistry Education Research and Practice, 16(2),
318–337.
FitzPatrick, B., & Schulz, H. (2015). Do curriculum outcomes and assessment activities in science encourage higher order thinking? Canadian Journal of Science,
Mathematics and Technology Education,, 15(2), 136–154.
Flavell, J.H. (.1976). “Metacognitive aspects of problem solving” in the nature of intelligence. ed. L. B. Resnick (Hillsdale, NJ: Erlbaum), 231–236.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906–911. https://doi.org/
10.1037/0003-066X.34.10.906
Forawi, S. A. (2016). Standard-based science education and critical thinking. Thinking skills and creativity, 20, 52–62.
Furst, E. (1994). Bloom’s Taxonomy: Philosophical and Educational Issues. In L. Anderson, & L. Sosniak (Eds.), Bloom’s taxonomy: A forty-year retrospective (pp.
28–40). Chicago: The National Society for the Study of Education.
Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved proactive. Educational
and Psychological Measurement, 66(3), 393–416.
Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of
educational research, 67(1), 88–140.
Hogan, K., & Maglienti, M. (2001). Comparing the epistemological underpinnings of students’ and scientists’ reasoning about conclusions. Journal of Research in
Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 38(6), 663–687.
Hong, E., Neil, H. F. O., & Peng, Y. (2016). Effects of explicit instructions, metacognition, and motivation on creative performance. Creativity Research Journal, 28,
33–45. https://doi.org/10.1080/10400419.2016.1125252
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation
modelling: A multidisciplinary journal, 6(1), 1–55.
Istiyono, E., Dwandaru, W. S. B., Setiawan, R., & Megawati, I. (2020). Developing of computerized adaptive testing to measure physics higher order thinking skills of
senior high school students and its feasibility of use. European Journal of Educational Research, 9(1), 91–101.
Jari Lavonen, K. J., & Meisalo, Veijo (2004). Integration of ICT to science education through professional development of teachers. Computers and Advanced Technology
in Education, 564–569.
Jia, X., Li, W., & Cao, L. (2019). The role of metacognitive components in creative thinking. Frontiers in psychology, 10, 2404. https://doi.org/10.3389/
fpsyg.2019.02404
Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39, 13–36. https://doi.org/10.1007/BF02291575
Kaufman, J. C., Evans, M. L., & Baer, J. (2010). The American idol effect: Are students good judges of their creativity across domains? Empirical Studies of the Arts, 28,
3–17. https://doi.org/10.2190/EM.28.1.b

9
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

Kirsch, J. A. W., & Poole, W. E. (1972). Taxonomy and distribution of the grey kangaroos, Macropus giganteus Shaw and Macropus fuliginous (Desmarest), and their
subspecies (Marsupialia: Macropodidae). Australian Journal of Zoology, 20(3), 315–339.
Koslowski, B. (1996). Theory and evidence: The development of scientific reasoning. Cambridge, MA: The MIT Press.
Kreitzer, A., & Madaus, G. (1994). Empirical investigations of the hierarchical structure of the taxonomy. Teachers College Record, 95(6), 64–81.
Ku, K. Y. L., & Ho, I. T. (2010). Metacognitive strategies that enhance critical thinking. Metacognition and Learning, 5(3), 251–267. https://doi.org/10.1007/s11409-
010-9060-6
Kuhn, D. (2000). Metacognitive development. Current directions in psychological science, 9(5), 178–181.
Kuhn, D., & Pearsall, S. (1998). Relations between metastrategic knowledge and strategic performance. Cognitive Development, 13(2), 227–247.
Kuhn, D., & Pease, M. (2006). Do children and adults learn differently? Journal of cognition and development, 7(3), 279–293.
L Rusyati, N. Y. R., Widodo, A., & Ha, M. (2020). A review of research trends on meta-cognitive in science education within the past decade. International Conference on
Mathematics and Science Education (ICMScE), 2020, 1806. https://doi.org/10.1088/1742-6596/1806/1/012136
Lawson, A. E., Banks, D. L., & Logvin, M. (2007). Self-efficacy, reasoning ability, and achievement in college Biology. Journal of Research in Science Teaching, 44(5),
706–724. https://doi.org/10.1002/tea.20172
Linnenbrink, E. A., & Pintrich, P. R. (2003). The role of self-efficacy beliefs in student engagement and learning in the classroom. Reading & Writing Quarterly, 19(2),
119–137. https://doi.org/10.1080/10573560308223
Liu, W., Pan, Y., Luo, X., Wang, L., & Pang, W. (2017). Active procrastination and creative ideation: The mediating role of creative self-efficacy. Personality and
Individual Differences, 119, 227–229. https://doi.org/10.1016/j.paid.2017.07.033
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modelling. Psychological
methods, 1(2), 130.
Magno, C. (2010). The role of metacognitive skills in developing critical thinking. Metacognition and Learning, 5(2), 137–156. https://doi.org/10.1007/s11409-010-
9054-4
Marzano, R.T., .& Kendall, J.S. (.2007). The new taxonomy of educational objectives the United States of America.
Michael, W.B., .Stanley, J.C., .& Bolton, D.L. (.1957). Book review: Taxonomy of educational objectives, the classification of educational goals, handbook I: Cognitive
domain.
Miri, B., David, B. C., & Uri, Z. (2007). Purposely teaching for the promotion of higher-order thinking skills: A case of critical thinking. Research in science education,, 37
(4), 353–369.
Mota-Valtierra, G., Rodríguez-Reséndiz, J., & Herrera-Ruiz, G. (2019). Constructivism-based methodology for teaching artificial intelligence topics focused on
sustainable development. Sustainability, 11(17), 4642.
Munandar. (2012). Pengembangan kreativitas anak berbakat. Jakarta: Rineka Cipta.
Nelson, T. O. (1990). Metamemory: A theoretical framework and new findings. Psychology of Learning and Motivation., 26, 125–173.
Nelson, T. O., Stuart, R. B., Howard, C., & Crowley, M. (1999). Metacognition and clinical psychology: A preliminary framework for research and practice. Clinical
Psychology & Psychotherapy: An International Journal of Theory & Practice, 6(2), 73–79.
Nesbitt-Hawes, P. J. (2005). Higher order thinking skills in a science classroom computer simulation. Queensland University of Technology. Doctoral dissertation.
OECD. (2015). PISA 2015 released field trial cognitive items.
Ong, K. K. A., Hart, C. E., & Chen, P. K. (2016). Promoting higher-order thinking through teacher questioning: A case study of a Singapore science classroom. New
Waves-Educational Research and Development Journal, 19(1), 1–19.
Ortiz-Echeverri, C. J., Rodríguez-Reséndiz, J., & Garduño-Aparicio, M. (2018). An approach to STFT and CWT learning through music hands-on labs. Computer
Applications in Engineering Education, 26(6), 2026–2035.
Osborne, J. (2013). The 21st century challenge for science education: Assessing scientific reasoning. Thinking Skills and Creativity, 10, 265–279. https://doi.org/
10.1016/j.tsc.2013.07.006
Osborne, J., Collins, S., Ratcliffe, M., Millar, R., & Duschl, R. (2003). What “ideas-about-science” should be taught in school science? A Delphi study of the expert
community. Journal of research in science teaching, 40(7), 692–720.
Phan, H. P. (2009). Relations between goals, self-efficacy, critical thinking and deep processing strategies: A path analysis. Educational Psychology, 29(7), 777–799.
https://doi.org/10.1080/01443410903289423
Pintrich, P. R., Marx, R. W., & Boyle, R. A. (1993). Beyond cold conceptual change: The role of motivational beliefs and classroom contextual factors in the process of
conceptual change. Review of Educational research, 63(2), 167–199.
Priest, T. (2006). Self-evaluation, creativity, and musical achievement. Psychol. Music, 34, 47–61. https://doi.org/10.1177/0305735606059104
Puente-Díaz, R., Cavazos-Arroyo, J., & Vargas-Barrera, F. (2021). Metacognitive feelings as a source of information in the evaluation and selection of creative ideas.
Thinking Skills and Creativity, 39. https://doi.org/10.1016/j.tsc.2020.100767
Quitadamo, I. J., & Kurtz, M. J. (2007). Learning to improve: Using writing to increase critical thinking performance in general education Biology. CBE—Life Sciences
Education,, 6(2), 140–154. https://doi.org/10.1187/cbe.06-11-0203
Ramos, J. L. S., Dolipas, B. B., & Villamor, B. B. (2013). Higher order thinking skills and academic performance in physics of college students: A regression analysis.
International Journal of Innovative Interdisciplinary Research, 4(1), 48–60.
Resnick, L.B. (.1987).& Science National Research Council (US). Committee on research in mathematics. Education and learning to think.
Romero, L. (2017). Molière: Traditions in criticism, 1900-1970. The University of North Carolina Press.
Saido, G. A., Siraj, S., Nordin, A. B., & Al-Amedy, O. S. (2017). Teaching strategies for promoting higher order thinking skills: A case of secondary science teachers.
MOJEM: Malaysian Online Journal of Educational Management, 3(4), 16–30.
Schauble, L., Klopfer, L. E., & Raghavan, K. (1991). Students’ transition from an engineering model to a science model of experimentation. Journal of research in science
teaching, 28(9), 859–882.
Schmidt, A. M., & Ford, J. K. (2003). Learning within a learner control training environment: The interactive effects of goal orientation and metacognitive instruction
on learning outcomes. Personnel psychology, 56(2), 405–429.
Setyasih, A. D., Hartono, H., & Prasetyo, A. P. B. (2019). Efforts to increase scientific literacy and thinking process by higher order thinking skills. Journal of Primary
Education, 8(2), 144–151.
Smith, L.I. (.2002). A tutorial on principal components analysis.
Songer, N.B., .& Ruiz-Primo, M.A. (.2012). Assessment and science education: Our essential new priority?.
Sternberg, R. J. (1985). Implicit theories of intelligence, creativity, and wisdom. Journal Personality and Social Psychology, 49, 607–627. https://doi.org/10.1037/
0022-3514.49.3.607
Sternberg, R. J. (2006). The nature of creativity. Creativity research journal, 18(1), 87.
Stevens, J. P. (2012). Applied multivariate statistics for the social sciences. Routledge.
Subia, G., Marcos, M., Valdez, A., Pascual, L., & Liangco, M. (2020). Cognitive levels as measure of higher-order thinking skills in senior high school mathematics of
science, technology, engineering and mathematics (STEM) graduates. Technology Reports of Kansai University, 62(03).
Suhr, D. (2006). The basics of structural equation modeling. Irvine, CA: SAS User Group of the Western Region of the United States (WUSS). Presented.
Takko, M., Jamaluddin, R., Kadir, S. A., Ismail, N., Abdullah, A., & Khamis, A. (2020). Enhancing higher-order thinking skills among home science students: the effect
of cooperative learning student teams-achievement divisions (STAD) module. International Journal of Learning, Teaching and Educational Research,, 19(7), 204–224.
Tal, R. T., Dori, Y. J., & Lazarowitz, R. (2000). A project-based alternative assessment system. Studies in Educational Evaluation, 26(2), 171–191.
Torrance, E. P., Orlow, E. B., & Safter, H. T. (1990). Torrance tests of creative thinking. Bensenville, IL: Scholastic Testing Service.
Veenman, M. V. (2007). The assessment and instruction of self-regulation in computer-based environments: A discussion. Metacognition and Learning, 2(2–3),
177–183.

10
H. Sun et al. Thinking Skills and Creativity 43 (2022) 100999

Vincent-Lancrin, S., González-Sancho, C., Bouckaert, M., de Luca, F., Fernández-Barrerra, M., Jacotin, G., et al. (2019). Fostering students’ creativity and critical thinking:
What it means in school, educational research and innovation. OECD Publishing. https://doi.org/10.1787/62212c37-en
WALİD, A., SAJİDAN, S., RAMLİ, M., & KUSUMAH, R. G. T. (2019). Construction of the assessment concept to measure students’ high order thinking skills. Journal for
the Education of Gifted Young Scientists, 7(2), 237–251.
Xie Yueguang, Y. X. (2017). Explanation and instruction suggestions for senior high information technology curriculum information consciousness literacy. Curriculum
Teaching Material and Method, 37, 85–90. https://doi.org/10.19877/j.cnki.kcjcjf.2017.12.014
Yulian, Jiang, & Yueguang, Xie (2017). Research on high-order thinking structure measurement model based on ESEM. Modern Distance Education Research, (3),
94–104.
Zeldin, A. L., & Pajares, F. (2000). Against the odds: Self-efficacy beliefs of women in mathematical, scientific, and technological careers. American educational research
journal, 37(1), 215–246.
Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. Handbook of self-regulation (pp. 13–39). Academic Press.
Zimmerman, C. (2007). The development of scientific thinking skills in elementary and middle school. Developmental review, 27(2), 172–223.
Zohar, A., & Dori, Y. J. (2003). Higher order thinking skills and low-achieving students: Are they mutually exclusive? The journal of the learning sciences, 12(2),
145–181.

11