PsyCh Journal (2020)

DOI: 10.1002/pchj.328

Set-shifting ability is specifically linked to high-school science

and math achievement in Chinese adolescents

Jingguang Li ,1 Yajun Zhao,2 Shan Zhou,1 Yuling Pu,1 Hongyu He,1 and Ming Zhao1
1
College of Education, Dali University, Dali, China, 2School of Sociology and Psychology, Southwest
Minzu University, Chengdu, China

Abstract: Previous meta-analytic studies have revealed that set-shifting ability contributes to achievement in math and reading equally
during childhood, but less is known about whether set-shifting ability plays an equally important role in achievement in different school
subjects during high-school learning. To fill this research gap, the current study examined the association between set-shifting ability and
academic achievement among 10th-grade Chinese adolescents who attended high school (N = 221). Meanwhile, we further explored the
possible mechanisms underlying this association by testing the mediating role of math abilities (i.e., arithmetic and algebraic ability). Set-
shifting ability was measured through a task-switching task in which participants continuously change task rules based on color and shape
sets. Academic achievement was indexed by curriculum-based examination scores. Based on exploratory factor analyses, the achievement
scores could be classified into two categories: (a) science and math achievement, including math, physics, chemistry, biology, and geogra-
phy scores; and (b) humanities achievement, comprising Chinese, English, history, and politics scores. Arithmetic ability and algebraic
ability were measured by participants’ ability to perform calculations and solve algebraic equations, respectively. Correlation and regres-
sion analyses found that set-shifting ability was related to science and math achievement, but not to humanities achievement. Further-
more, the association between set-shifting ability and science and math achievement is partly mediated by algebraic ability. Together,
these results suggest that set-shifting ability plays a domain-specific role in achievement. That is, set-shifting ability is specifically linked
to science and math achievement in Chinese adolescents, and this specific link may be partly mediated by algebraic ability.

Keywords: achievement; algebraic ability; math; science; set-shifting

Correspondence Dr. Jingguang Li, Dali University, Hongsheng Street #2, Dali, Yunnan 671000, China. Email: jingguang.li.
k@gmail.com
Received 14 October 2018. Accepted 24 October 2019.

Identifying the individual psychological factors that influ-
ence academic achievement is a critical task for psycho-
logical science that may help researchers and teachers
develop pedagogical methods that improve students’
learning outcomes. Previous studies have established
numerous individual factors that predict academic
achievement, such as general intelligence (Rohde &
Thompson, 2007), working memory (Jacob & Parkinson,
2015), Big-Five personalities (Poropat, 2009), and self-
regulation (Duckworth, Peterson, Matthews, & Kelly,
2007; Duckworth & Seligman, 2005). In the current study,
we examine the role of a specific cognitive function – set-
shifting ability – in academic achievement.
Set-shifting ability and academic achievement
Set-shifting (also known as “cognitive flexibility” and “task
switching”) is one of the executive functions, which com-
prise cognitive processes underlying the cognitive control
of thought and behavior (Miyake et al., 2000). Set-shifting
refers to “changing perspectives or approaches to a prob-
lem, flexibly adjusting to new demands, rules, or priorities
(as in switching between tasks)” (Diamond, 2013). In set-
shifting tasks, such as the well-known Wisconsin Card
Sorting Task (Berg, 1948), participants develop rules in
working memory and then switch to perform different tasks
based on different rules (Garon, Bryson, & Smith, 2008).

© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
2 Set-shifting and academic achievement
Students often have to shift between different mental sets
during learning and testing; therefore, a number of previous
studies have examined the role of set-shifting ability in
predicting academic achievement. Several studies have
reported a lack of an association between set-shifting abil-
ity and academic performance (e.g., Cragg, Keeble, Rich-
ardson, Roome, & Gilmore, 2017; Espy et al., 2004; Lee,
Ng, & Ng, 2009; Monette, Bigras, & Guay, 2011) whereas
other studies have reported positive correlations between
set-shifting ability and academic performance (e.g., Bull,
Espy, & Wiebe, 2008; Bull & Scerif, 2001; Cantin,
Gnaedinger, Gallaway, Hesson-McInnis, & Hund, 2016;
Clark, Pritchard, & Woodward, 2010; Filippetti & Richaud,
2017; Gerst, Cirino, Fletcher, & Yoshida, 2017). Critically,
two meta-analyses have confirmed the positive association
between set-shifting ability and academic achievement.
First, Yeniad, Malda, Mesman, van IJzendoorn, and Pieper
(2013) reported that shifting was positively and equally cor-
related achievement in both math, k = 18, r = .26, 95%
confidence interval (CI) [.15, .35], and reading, k = 16,
r = .21, 95% CI [.11, .31]. Second, Jacob and Parkinson
(2015) reported similar meta-analysis results, but with a
slightly higher estimation of effect sizes. That is, set-
shifting ability was positively and equally correlated with
achievement in both math, k = 17, r = .34, 95% CI [.24,
.44], and reading, k = 11, r = .32, 95% CI [.30, .54]. In
summary, set-shifting ability may be positively correlated
with academic achievement, and the association does not
differ between math and reading.
An important question regarding the association between
set-shifting ability and academic achievement is whether
this link is domain-general or domain-specific. That is,
does set-shifting ability play an equally important role in
all school subjects (referred to as the domain-general
hypothesis) or does it have a stronger impact on certain
subjects than on others (referred to as the domain-specific
hypothesis)? Critically, these two contradictory hypotheses
have different educational implications. If the domain-
specific hypothesis is correct, educators in different school
subjects should develop different pedagogical methods
based on the specific influence of set-shifting ability on
achievement in certain subjects. For example, if set-shifting
ability has higher predictive power for achievement in math
than in other subjects, math teachers should pay special
attention to students’ constraints in set-shifting ability. In
addition, it is appropriate to design intervention programs
to improve math achievement by training set-shifting
© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
ability. If the domain-general hypothesis is correct, educa-
tors should not differentiate the role of set-shifting in learn-
ing among different school subjects. The extant literature
appears to support a domain-general view on the relation-
ship between set-shifting ability and academic achievement.
That is, two meta-analyses—the most compelling method
for aggregating empirical data—have both revealed that
set-shifting ability equally correlated with math and reading
achievement (Jacob & Parkinson, 2015; Yeniad et al.,
2013). However, we argue that it is necessary to test the
generality of the domain-general hypothesis in the follow-
ing two ways.
First, students generally learn many more subjects than
math and reading, such as humanities and science. Differ-
ent subjects generally have different cognitive demands;
thus, it is unclear whether set-shifting ability contributes
equally to achievement in all school subjects. Conse-
quently, it is necessary to examine the association by
expanding to a broader category of school subjects.
Second, and more important, most previous studies have
examined the association between set-shifting ability and
academic achievement in samples of children. Seventeen of
18 of Yeniad et al.’s (2013) meta-analyses included empiri-
cal studies that tested this association for children from kin-
dergarten and primary school. In Jacob and Parkinson’s
(2015) meta-analyses, 80% of all included studies tested
the association in children from kindergarten and primary
school. We argue that the results based on children might
differ from those for older participants (e.g., high-school
students). First, the developmental trajectories of set-
shifting ability across childhood and adolescence might be
different (Lee, Bull, & Ho, 2013). Moreover, even for the
same subject, knowledge and skill learning differ in differ-
ent learning phases. For example, high-school math may
require much more cognitive processing (including set-
shifting) than may kindergarten math because the former is
much more difficult than is the latter. Thus, it is necessary
to replicate the findings for children with older participants
such as high-school students.
In brief, to evaluate the domain-general versus domain-
specific hypotheses, it is necessary to conduct investiga-
tions with diverse school subjects and populations other
than children (e.g., adolescents). Only four studies have
tested the relationship between set-shifting ability, as mea-
sured by the card-sorting paradigm, and academic achieve-
ment in adolescent or adult populations (Au et al., 2011;
Boschloo, Krabbendam, Aben, de Groot, & Jolles, 2014;
PsyCh Journal
Cragg et al., 2017; Latzman, Elkovitch, Young, & Clark,
2010). Au et al. (2011) observed that set-shifting ability
significantly contributed to overall academic rank in 12- to
17-year-old Chinese Hong Kong adolescents, but they did
not examine different school subjects in detail. Cragg et al.
(2017) observed a lack of correlation between set-shifting
ability and performance on standard math achievement test
in both adolescents (11–14 years of age) and young adults
(college students) in the United Kingdom. Because these
two studies did not examine the correlation with different
school subjects, they do not provide enough evidence to
examine the domain-general versus domain-specific
hypotheses. However, the other two studies have examined
the association between set-shifting ability and academic
achievement with different school subjects. First, Latzman
et al. (2010) found that sorting task performance was sig-
nificantly correlated with achievement in reading, math,
social studies, and science in male adolescent students
(11–16 years of age). However, after controlling for general
intelligence, sorting task performance only predicted read-
ing and science achievement, but not math and social stud-
ies achievement. Thus, Latzman et al.’s data support the
domain-specific hypothesis. Second, Boschloo et al. (2014)
observed almost no correlation between sorting task perfor-
mance and academic achievement in Dutch, r = .01,
English, r = −.06, and math, r = −.02, in adolescents
(12–18 year olds) who participated in a pre-university edu-
cational track. However, as suggested by Boschloo et al.
themselves, the lack of correlation might be due to the
high-performing nature of the sample (top 20% of Dutch
students), which may limit the range of correlation.
The current study
Based on the previous literature review, more evidence is
needed to resolve the domain-general versus domain-specific
debate with diverse school subjects and populations other than
children. Moreover, in the current study, we aimed to alter
past research designs in three vital ways.
First, both Latzman et al. (2010) and Boschloo et al.
(2014) tested the association between set-shifting and aca-
demic achievement in Western participants. Therefore, it is
worth examining this association in other populations. For
example, a previous study has demonstrated that other
components of executive function (e.g., working memory)
may have different influences on academic achievement
© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
3
across participants from the United States and China (Lan,
Legare, Ponitz, Li, & Morrison, 2011). Chinese adoles-
cents’ school learning differs from that of their Western
counterparts in two critical aspects. First, cross-country
comparison studies consistently have shown that Chinese
adolescents generally outperform Western adolescents in
science and math achievement (see Hau & Ho, 2010).
Thus, the observed association between set-shifting ability
and academic achievement in Western students may be dif-
ferent in another higher performing group, Chinese adoles-
cents (e.g., Boschloo et al., 2014). Second, there are
significant differences in the goals, contents, and strategies
of learning, teaching, and testing between the Chinese and
Western educational systems for adolescents (e.g., Silver,
2009; Zhu & Fan, 2006). Consequently, learning the same
school subject may require different cognitive demands
(including the use of shifting ability) between Chinese and
Western adolescents. For example, Chinese students rely
more heavily on memorization in school learning relative
to their Western counterparts (Kember, 2016). Thus, set-
shifting ability may play a less important role in Chinese
students’ learning outcomes, given that memory ability
may make a greater contribution to academic achievement.
In summary, concerning these two differences
(i.e., relatively higher performance and potentially different
cognitive demands for learning in Chinese adolescents),
set-shifting ability might have a different impact on aca-
demic achievement across the Chinese and Western educa-
tional contexts. Consequently, the current study aims to test
the domain-general versus domain-specific debate with
Chinese adolescents.
Second, all previous studies in adolescent populations
used the card-sorting task paradigm to measure set-shifting
ability. In the extant literature, the card-sorting paradigm
has often been used when studying individual differences
(e.g., neuropsychology) whereas the task-switching para-
digm has often been used in the field of experimental psy-
chology. When analyzing the correlation between set-
shifting ability and academic achievement, most researchers
have used the card-sorting paradigm. However, many
researchers have argued that the card-sorting task is not a
pure task for measuring set-shifting ability (e.g., Anderson,
Damasio, Jones, & Tranel, 1991; Buchsbaum, Greer,
Chang, & Berman, 2005; Miyake et al., 2000). Lange et al.
(2018) recently noted a critical difference between these
task paradigms is rule uncertainty. That is, the card-sorting
paradigm requires participants to seek undefined and
4 Set-shifting and academic achievement
changing task rules whereas the task rules are predefined in
the task-switching paradigm. For example, manipulating
the rule uncertainty significantly affects the task perfor-
mance in card-sorting tasks (Lange et al., 2018). Therefore,
the card-sorting task may not only measure individuals’ set-
shifting ability but also measure individuals’ ability to cope
with abstract uncertain rules such as abstract reasoning
ability (Boschloo et al., 2014; Delis, Squire, Bihrle, &
Massman, 1992; Latzman & Markon, 2010). Therefore, the
current study aims to examine the relationship between set-
shifting ability and academic achievement by using a rela-
tively purer measure of set-shifting ability: the task-
switching paradigm.
Third, less is known about the possible mechanisms
underlying the association between set-shifting ability and
academic achievement. We hypothesize that two categories
of domain-specific cognitive abilities—basic mathematical
abilities (i.e., the abilities to solve arithmetic and algebraic
problems) and reading abilities—might be candidate medi-
ators of the association. First, previous studies have rev-
ealed that set-shifting ability is positively correlated with
both basic mathematical ability and reading ability (Yeniad
et al., 2013). Second, science and math subjects (i.e., math,
physics, chemistry, biology, and geography) rely heavily on
the ability to solve mathematical problems, including both
arithmetic and algebra, whereas humanities subjects (lan-
guage, history, and politics) rely heavily on the ability to
read and understand. Therefore, the current study also aims
to test whether basic mathematical ability (including arith-
metic and algebraic ability) mediates the association
between set-shifting ability and math and science achieve-
ment and whether reading ability mediates the association
between set-shifting ability and humanities achievement.
To achieve these goals, we examined the association
between set-shifting ability and academic achievement in
different subjects among Chinese 10th-grade students. Set-
shifting ability was assessed by a task-switching task in
which participants had to continuously switch from color
discrimination to shape discrimination. Academic achieve-
ment was assessed from examination scores from all sub-
jects. Meanwhile, algebraic ability was indexed by
participants’ fluency in solving algebraic equations. Arith-
metic ability was measured by participants’ fluency in solv-
ing simple arithmetic problems (i.e., addition, subtraction,
and multiplication). Correlation and regression analyses
were used to test the association between set-shifting ability
and academic achievement, and mediation analyses were
© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
used to test the mediating role of arithmetic and algebraic
ability in this association.
Methods
Participants and procedures
Participants were 221 10th-grade students who were rec-
ruited from four classes of one local public high school in
Dali, China. The mean age of the participants was
16.1 years (SD = 0.5); 65.8% of the participants were girls.
The Medical Ethics Committee of Dali University approved
the study, and we obtained written informed consent from
all participants and their parents.
Participants completed several paper-based tests in their
own classroom with all their own classmates: a computa-
tion fluency test, an algebraic ability test, self-reported
reading ability, and a short form of Raven’s Advanced Pro-
gressive Matrices (APM; Bors & Stokes, 1998). In addi-
tion, they completed the task-switching task individually on
a separate day than the paper-based tasks. Three students
did not participate in the task-switching task due to school
absence.
Measures
Task-switching task
In this study, we used unmodified versions of the task-
switching task from the well-validated Executive Abilities:
Measures and Instruments for Neurobehavioral Evaluation
and Research (NIH-EXAMINER; https://memory.ucsf.edu/
examiner) test battery to measure set-shifting ability
(Kramer et al., 2014). In this task, participants match a fig-
ure on the top of the screen to one of two figures in the
lower corners of the screen. In the beginning of each trial,
a visually presented verbal cue indicates the dimension on
which to match (color vs. shape). In task-homogeneous
blocks, participants performed either color matching or
shape matching. In task-heterogeneous blocks, participants
alternated randomly between the two matching tasks. There
were two homogenous blocks (one for color matching, one
for shape matching) that each consisted of 20 trials. There
was only one heterogeneous block that consisted of 64 trials
(32 trials for color matching, 32 trials for shape matching).
To comprehensively evaluate participants’ performances,
in accordance with the NIH-EXAMINER (Kramer et al.,
2014) and the NIH toolbox for cognition assessment
(Weintraub et al., 2013), we used the single score of set-
PsyCh Journal
shifting ability directly generated by the NIH-EXAMINER
computerized program. This score combined both accuracy,
Cronbach’s α = .58, and reaction time (RT), Cronbach’s
α = .96, performance in the heterogeneous block (correct
rate = 92.98%). This scoring method generates accuracy
and RT subscales that range from 0 and 5, and the final
composite score ranges from 0 to 10. The accuracy score is
the correct rate multiplied by 5. The RT score is based on
the median RT with log (Base 10) transform and exclusion
of outliers (low cutoff = 400 ms, high cutoff = 2800 ms).
Specifically, RT score = 5 – {5 * [log (RT) – log (400)]/
[log (2800) – log (400)]}. The total set-shifting score is the
sum of the accuracy and RT scores. Note that the single
shifting score was correlated with traditional shift cost indi-
cators of the set-shifting paradigm, measured by the perfor-
mance differences between the task-heterogeneous and the
task-homogeneous blocks, with RT cost, r = .78, p < .01;
with accuracy cost, r = .33, p < .01, demonstrating the
validity of the single shifting measure.
Academic achievement
Academic achievement was indexed by the scores of the
final examination for the first semester, which reflect stu-
dents’ learning progress in the first half of the 10th-grade
year. The exams were administered by the school teachers,
and all participants took the same examination. Test sub-
jects included Chinese (M = 94.38, SD = 8.94), math
(M = 94.51, SD = 19.10), English (M = 87.90, SD = 15.03),
physics (M = 76.33, SD = 13.10), chemistry (M = 73.80,
SD = 12.78), biology (M = 64.12, SD = 12.58), geography
(M = 61.58, SD = 10.82), history (M = 67.09, SD = 8.50),
and politics (M = 62.52, SD = 6.62). The full scores were
150 for Chinese, math, and English and 100 for the
remaining subjects.
To further explore the underlying structure of the
achievement data, we performed exploratory factor analy-
sis. The Kaiser-Meyer-Olkin measure of sampling ade-
quacy was 0.86, indicating the possibility of common
factor(s). Bartlett’s tests of sphericity were statistically sig-
nificant, χ2(36) = 778.30, p < .001, indicating adequate
sampling and significant correlations among the observed
variables. In summary, these analyses indicated that the
data set was suitable for factor analysis. Using the principal
components analysis (PCA), we found that the first three
eigenvalues for the data set were 4.20, 1.26, and 0.85,
respectively. To determine the number of factors, we
© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
5
performed a parallel analysis, which compares the eigen-
values of real data to those of the permutations of the real
data (O’connor, 2000). Specifically, the first three eigen-
values for simulation data (times of permutation = 1,000)
were 1.32, 1.21, and 1.13, respectively, and for real data,
they were 4.20, 1.26, and 0.85, respectively. The point
where the eigenvalues of real data dropped below the simu-
lation data came after the second factor. Thus, the data
demonstrated a two-factor model. Finally, the PCA with a
Promax rotation revealed the following factor structure
accounting for 50% of the total eigenvalues: Factor
1 (which refers to science and math) included math, chem-
istry, physics, geography, and biology, eigenvalue = 4.20;
Factor 2 (which refers to humanities subjects) included
items related to politics, history, English, and Chinese,
eigenvalue = 1.26. These two standardized factor scores
have been used to index science and math achievement in
and humanities achievement in subsequent analyses,
respectively.
Arithmetic test
The Calculation Fluency Test (CFT) was used to index the
participants’ arithmetic ability (Douglas & LeFevre, 2018).
The CFT consists of three subtests involving two-term
arithmetic problems on three separate A4 pages, and each
subtest contains 60 problems. The first subtest consists of
double-digit addition problems (e.g., 52 + 19), the second
subtest consists of double-digit subtraction problems
(e.g., 84–47), and the third subtest consists of single-digit
multiplication problems (e.g., 67 × 4). For each test, partic-
ipants are given 1 min and are instructed to solve the prob-
lems quickly and accurately. Given that all the arithmetic
questions were easy for high-school students, the CFT was
considered to be a fluency-based test. On average, partici-
pants completed 21.95 (SD = 5.00), 17.03 (SD = 3.96), and
16.25 (SD = 3.92) problems in the addition, subtraction,
and multiplication subtests, respectively. In this study,
Cronbach’s αs for the addition, subtraction, and multiplica-
tion subtests were .95, .97, and .97, respectively. To sim-
plify the subsequent analyses, the sum of the three subtests
was used as a measure of arithmetic calculation ability
(e.g., Douglas & LeFevre, 2018).
Algebraic test
Following the CFT test format, we developed one algebraic
equation-solving test to assess participants’ algebraic
6 Set-shifting and academic achievement

ability. In this test, participants solved pure linear algebra
equations with only one unknown (e.g., “X−12 = 74; X =?
”). The test included 60 questions on two A4 sheets
(30 questions on each sheet). Participants were given 2 min
and instructed to solve the problems quickly and accurately.
Before the formal test, participants solved three sample
practice problems. Because all the algebraic questions were
easy for high-school students, this test was also considered
a fluency-based test. In this study, Cronbach’s α for this test
was .89.
Reading ability
We used a single-item self-reported measure to index read-
ing ability. Participants were asked to indicate “How would
you compare your reading skill to that of others in your
grade in this school” with a 5-point Likert scale of 1 (far
worth than peers) to 5 (far better than peers). This single-
item measure was revised from an overall rating item
(i.e., “How would you compare your reading skill to that of
others in your elementary classes?”) from the Adult Read-
ing History Questionnaire (Lefly & Pennington, 2000).
Raven’s APM
Raven’s APM, in which participants must identify a miss-
ing figure required to complete a larger pattern, is a widely
used tool to measure nonverbal general intelligence. To
shorten the test time, an abbreviated form of Raven’s APM
was used. This test involves only 12 selected items from
the original 36-item Raven’s APM, but it has been shown
to retain the difficulties and validity of the original test
(Bors & Stokes, 1998). Given the limited number of items,
Cronbach’s α of Raven’s APM scores for this study was

Figure 1. The histograms (middle panel), scatterplots of correlations (left panel), and correlation coefficients (right panel) for major study variables.
Shifting = shifting ability; Science_Math = Science and math achievement; Humanities = humanities achievement; Arithmetic = arithmetic performance;
Algebraic = algebraic performance; Reading = self-reported reading ability; IQ = Raven performance. *p < .05. **p < .01. ***p < .001. [Color figure can
be viewed at wileyonlinelibrary.com]

© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
PsyCh Journal 7

Table 1 not perform separate analyses for each gender in the

Hierarchical Regression Model to Predict Science and Math Achievement
remainder of this study. In summary, stronger set-shifting
B (SE) β 4R2 ability is associated with higher science and math achieve-
Step 1 6.70%** ment, but not higher humanities academic achievement.
Age .05 (.12) .03
Gender .26 (.13) .13 Furthermore, to test whether the correlation between the
General intelligence .09 (.03) .20 set-shifting ability and science and math achievement held
Step 2 11.07%***
when controlling for the possible third variables (i.e., age,
Age .09 (.11) .05
Gender .26 (.13) .13 gender, and general intelligence), we performed a hierarchi-
General intelligence .07 (.03) .17 cal regression analysis. The regression model included age,
Set-shifting ability .32 (.10) .20
gender, and general intelligence—measured by Raven’s
**p < .01. ***p < .001.
APM—in Step 1 and set-shifting ability in Step 2. Regres-
only .57. Although the reliability score was concerning, we sion analyses revealed that the set-shifting ability explained
still included Raven’s APM scores in the subsequent ana- additional variance, ΔR2 = 4.37%, in science and math
lyses for research purposes. achievement beyond the variance explained by age, gender,
and general intelligence (Table 1). Therefore, set-shifting
ability has incremental power for predicting science and
Data analysis
math achievement beyond age, gender, and general
All statistical analyses were performed using R 3.4.3
intelligence.
(R Foundation for Statistical Computing, Vienna, Austria).
Even after establishing the aforementioned positive asso-
Specifically, the factor analysis and mediation analyses
ciation between set-shifting ability and science and math
were performed using the “psych” package (Revelle, 2017).
achievement, a further question remains unknown: whether
set-shifting ability is specifically associated with achieve-
Results ment in particular science and math subjects or is generally
associated with achievement in all science and math sub-
The histograms, scatterplots of correlations, and correlation jects. Thus, we performed further correlational analyses
coefficients for major study variables are presented in and found that set-shifting ability is positively correlated
Figure 1. Critically, set-shifting ability is positively associ- with achievement in all science and math subjects: math,
ated with adolescents’ science and math achievement, r = .19, p < .01; physics, r = .21, p < .05; chemistry,
r = .24, p < .01. However, no statistically significant corre- r = .21, p < .05; biology, r = .17, p = .01; geography,
lation was found between set-shifting ability and adoles- r = .20, p < .01. In contrast, set-shifting ability is not statis-
cents’ humanities achievement, r = .11, p = .09. Critically, tically correlated with achievement in all humanities sub-
the difference between these two correlation coefficients jects: English, r = .04, p = .56; politics, r = −.02, p = .78;
was also statistically significant, Steiger’s Z = 2.61, p < .01. Chinese, r = .12, p = .08; history, r = .02, p = .72. Overall,
Next, we performed separate tests for boys (n = 76) and the association between set-shifting ability and science and
girls (n = 145), and revealed a similar pattern of associa- math subject achievement is not restricted to particular
tions. That is, the association between set-shifting ability subjects.
and science and math achievement was statistically signifi- A further research question addresses how set-shifting
cant for both boys, r = .35, p < .01, and girls, r = .16, ability is linked to science and math achievement. Given
p < .01. By contrast, the association between set-shifting the important role of basic mathematical ability
ability and humanities achievement did not reach statistical (i.e., abilities in solving arithmetic and algebraic problems)
significance in either boys, r = .21, p = .08, or girls, in science and math subjects, we tested the possible mediat-
r = .05, p = .55. The differences between these two types ing role of basic mathematical ability in the association
of correlation coefficients were also statistically significant between set-shifting ability and science and math achieve-
for both boys, Steiger’s Z = 1.85, p < .05, and girls, ment. Arithmetic and algebraic ability were indexed by par-
Steiger’s Z = 1.79, p < .05, at the one-tailed significance ticipants’ ability to solve arithmetic problems
level. Given that similar correlations were observed in both (i.e., addition, subtraction, and multiplication) and algebraic
boys and girls and the small sample size of boys, we did problems (i.e., linear algebraic equations) (Note that the

© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
8 Set-shifting and academic achievement
Figure 2. Algebraic ability mediates the association between set-shifting
ability and science and math achievement. Path a represents the effect of
set-shifting ability on algebraic ability, Path b represents the effect of alge-
braic ability on science and math achievement, Path c represents the effect
of set-shifting ability on science and math achievement, and Path c’ repre-
sents the effect of set-shifting ability on science and math achievement,
after adjusting for algebraic ability. The indirect effects are equivalent to
a × b or c – c’. Standardized regression coefficients are displayed for all
paths.
correlation between math achievement and mathematical
abilities was not high: arithmetic ability, r = .12, p > .05;
algebraic ability, r = .37, p < .001; thus, math achievement
and these two math abilities are indeed separate constructs
even though they all involve solving of math problems.)
After including arithmetic ability as an intermediate vari-
able, the associations between set-shifting ability and sci-
ence and math achievement were almost unchanged, from
.24 to .23; indirect effect = .02, 95% CI [−.01, .05], 5,000
times bootstrapping. By contrast, after including algebraic
ability as an intermediate variable, the associations between
set-shifting ability and science and math achievement were
weaker, but still significant (Figure 2). The 5,000 bootstrap
simulations further revealed that algebraic ability partly
mediated the associations between set-shifting ability and
science and math achievement, indirect effect = .06, 95%
CI [.01, .12]. Furthermore, we tested whether this mediat-
ing role of algebraic ability holds for each science and
math subject. Thus, we performed five similar mediation
tests, but with math, physics, chemistry, biology, and
Table 2
Summary of the Effect of Set-Shifting Ability on Science and Math Achievement Through Algebraic Ability (5,000 Bootstraps)
Dependent variables Total effect
Math .19**
Physics .21**
Chemistry .22***
Biology .17*
Geography .20**
Note. Standardized regression coefficients are displayed for all paths. CI = confidence interval.
*p < .05. **p < .01. ***p < .001.
© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
geography achievement as dependent variables, respec-
tively. As illustrated in Table 2, a similar pattern of media-
tion effects could be observed for all of these school
subjects, although the mediation relationships were only
marginally statistically significant for biology and
geography.
Finally, even no significant correlation was found between
set-shifting ability and humanities achievement, we tested
whether self-reported reading ability could mediate this rela-
tionship. Although self-reported reading ability positively cor-
related with humanities achievement, r = .23, p < .001, no
mediation effect was observed, indirect effect = .00, 95%
CI [−.01, .01], 5,000 times bootstrapping.
In summary, the study established that set-shifting ability
is specifically linked to science and math achievement, and
further revealed that algebraic ability partly mediates the
positive association between set-shifting ability and science
and math achievement.
Discussion
In this study, we assessed the association between set-
shifting ability and academic achievement among Chinese
adolescents. Set-shifting ability was positively correlated
with science and math achievement, but not with humani-
ties achievement. Furthermore, the association between set-
shifting ability and science and math achievement was
mediated by algebraic ability, but not by arithmetic ability.
Together, these results suggest that set-shifting ability may
specifically boost adolescents’ science and math learning,
which may be caused partly by the mediating role of alge-
braic ability.
These findings differ from those in previous studies on
children that reported a domain-general link between set-
shifting ability and academic achievement (Jacob &
Parkinson, 2015; Yeniad et al., 2013). However, our results
Indirect effect
Direct effect Point estimate 95% CI
.13* .06 [.01, .12]
.15* .06 [.01, .11]
.15* .07 [.01, .13]
.14* .03 [.00, .07]
.18* .02 [.00, .06]
PsyCh Journal
are partly consistent with Latzman et al. (2010), who also
found that set-shifting ability had different impacts on dif-
ferent school subjects among U.S. male adolescents. How-
ever, the pattern of influence was somewhat different from
that in our study: Latzman et al. (2010) found that set-
shifting predicts achievement in reading and science, but
not math and social studies. There are several possible rea-
sons of the discrepancies in results between our study and
the extant literature (including both studies on children and
adolescents), such as (a) differences in age (children
vs. adolescent), (b) differences in ethnicity (Western
vs. Chinese), (c) different measures of set-shifting ability (task
switching vs. the card-sorting paradigm), and (d) differences
in the content and difficulty of each country’s academic
achievement test. Currently, there is not enough information
to determine which is the major reason underlying the dis-
crepancies. Nevertheless, our study provides the first evidence
that not all school subjects are equally influenced by set-
shifting ability in a Chinese context. Moreover, together with
Latzman et al.’s study (2010), our investigation provides addi-
tional evidence supporting the hypothesis that set-shifting abil-
ity has a domain-specific influence on different school
subjects in the adolescent population.
Furthermore, what are the underlying cognitive mecha-
nisms of the observed specific association between set-shifting
ability and science and math achievement in Chinese adoles-
cents? First, set-shifting ability may directly contribute to sci-
ence and math academic achievement. High-school science
and math tasks require students to switch frequently between
different conceptual representations (i.e., goals, rules, or strate-
gies for problem-solving). Thus, stronger set-shifting ability
would help students gain an additional advantage in solving
science and math problems. Admittedly, humanities subjects
may also require set-shifting processing; however, perfor-
mance in humanities subjects may rely heavily on semantic
memory capacity and writing ability. For example, Chinese
high-school students have to memorize a large amount of
facts and opinions for Chinese, history, and politics courses.
Moreover, when completing essay questions in humanities
subjects, writing ability is indispensable. Together, the differ-
ences in the cognitive demands of certain school subjects
determine whether set-shifting ability contributes to academic
achievement in a specific subject.
Second, our study suggests that set-shifting ability may
indirectly contribute to science and math academic achieve-
ment, through the mediating role of algebraic ability. When
solving algebraic equations, students need to continuously
© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
9
switch among different computation rules. For example,
when solving the equation 2X + 3 = 5, students typically
perform the following three mental operations:
(a) transformation, 2X = 5–3; (b) subtraction, 5–3 = 2; and
(c) division, 2/2 = 1. Students with stronger set-shifting
ability could have an additional advantage when switching
between these mental operations. Furthermore, the ability
to solve algebraic equations is widely used in high-school
science and math courses. For example, much of the core
knowledge within high-school science courses is expressed
as algebraic knowledge, such as Newton’s second law of
physics, the balancing of chemical equations in chemistry,
the photosynthesis equation in biology, and the map scale
in geography. Together, stronger set-shifting ability is
linked to better algebraic performance, which in turn leads
to an advantage in science and math performance.
The current study invites a broad investigation to explore
how set-shifting ability promotes science and math learning.
First, given the partial mediation relationship found in the cur-
rent study, algebraic ability is definitely not the only mediator
of this association. For example, several studies have found a
positive link between set-shifting ability and creativity
(Madjar & Shalley, 2008; Nusbaum & Silvia, 2011; Pan &
Yu, 2016), which implies that creativity might also be a medi-
ator. Future studies are needed to find other new mediators,
which may help to elucidate the mechanism underlying the
association between set-shifting and achievement. Second,
set-shifting may also directly influence participants’ ability to
solve science and math problems. A direction for future in-
depth investigations is to find the specific areas of science and
math that are closely related to set-shifting ability.
Furthermore, a critical question is whether the associa-
tion between set-shifting ability and science and math
achievement is practically significant. In our study, set-
shifting ability only explained a portion (~4%) of the total
variance of math achievement. However, we argue that such
an effect size is still meaningful for two reasons. First,
because academic achievement is influenced by many other
cognitive (e.g., general intelligence and working memory)
and noncognitive factors (motivation and personality traits),
4% accountability from a single cognitive factor is not
small. For example, meta-analytic studies have demon-
strated that the correlations between academic achievement
and other well-known contribution factors are close to or
even less than 4%, including grit, r = .18 (Credé, Tynan, &
Harms, 2017), growth mind-set, r = .10 (Sisk, Burgoyne,
Sun, Butler, & Macnamara, 2018), number sense, r = .20
10
(Chen & Li, 2014), and inhibition control, r = .27) (Allan,
Hume, Allan, Farrington, & Lonigan, 2014). Second,
because of the attenuating effect of measurement error
(i.e., imperfect reliability), the real effect size of the associa-
tion is underestimated (Schmidt & Hunter, 1999).
Consequently, the finding of a link between set-shifting
ability and science and math achievement may shed light
on the pedagogy of these courses. First, educators may
improve science and math learning by developing training
programs on set-shifting ability. Currently, researchers have
begun to improve math learning outcomes by cognitive
training in another executive function—updating
(e.g., Kroesbergen, Van’t Noordende, & Kolkman, 2014;
St Clair-Thompson, Stevens, Hunt, & Bolder, 2010). The
current study suggests that set-shifting ability may be
another promising candidate for cognitive training to
improve math learning. Second, high-school teachers may
teach students to master strategies (or simply improve their
awareness) to be flexible in solving science and math prob-
lems, which may further allow them to perform better in
learning and testing in science and math courses.
Finally, the current investigation has several limitations
that deserve consideration. First, we did not provide evi-
dence supporting a causal relationship between set-shifting
ability and academic performance. Future studies may con-
sider using training or longitudinal designs to explore the
causal direction of this association. Second, we based our
study solely on 10th-grade Chinese adolescents from Main-
land China, which constrains the generalizability of the cur-
rent findings to other Chinese populations and to other
Eastern and Western adolescents. Thus, there is a need to
replicate the findings detected in our study with other sam-
ples. Third, we did not consecutively examine the role of
two other executive functions (i.e., updating and inhibition)
in the relationship between set-shifting ability and aca-
demic performance. Some authors have argued that
updating and inhibition act as parts of the basic cognitive
processes of set-shifting (Bull & Lee, 2014; Diamond,
2013); therefore, future studies could investigate how set-
shifting ability, jointly with other executive functions, pre-
dicts high-school science and math achievement.
Conclusion
Previous meta-analytic studies (Jacob & Parkinson, 2015;
Yeniad et al., 2013) have revealed that set-shifting ability
© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
Set-shifting and academic achievement
equally contributes to achievement in math and reading
during childhood, but less is known about whether set-
shifting ability plays an equally important role in achieve-
ment in different school subjects during high-school learn-
ing. In this study, we provided the first evidence that
Chinese high-school students’ set-shifting ability is specifi-
cally linked to their science and math achievement, but not
to humanities achievement. Moreover, the association
between set-shifting ability and science and math achieve-
ment is mediated by algebraic ability. Together, these
results suggest that set-shifting ability may specifically
boost Chinese adolescents’ science and math learning,
which may be caused partly by the mediating role of alge-
braic ability. Future studies should explore the causality,
underlying mechanisms, and generalizability of these
relationships.
Disclosure of conflict of interest
The authors declare that there are no conflicts of interest.
Acknowledgments
The authors thank Yongsheng Wang, Hongbin Yang, Jin
Huang, Shuai Yin, and Rui Jiang for their assistance in
data collection. This work was supported by National Natu-
ral Science Foundation of China Grant 31500884, the Fund
for Educational Sciences from the Philosophy and Social
Sciences Projects of Yunnan Province Grant AFSZ18030,
and the Innovation Team of Dali University Grant
SKPY2019303.
References
Allan, N. P., Hume, L. E., Allan, D. M., Farrington, A. L., &
Lonigan, C. J. (2014). Relations between inhibitory control and
the development of academic skills in preschool and kindergar-
ten: A meta-analysis. Developmental Psychology, 50(10),
2368–2379. https://dx.doi.org/10.1037/a0037493
Anderson, S. W., Damasio, H., Jones, R. D., & Tranel, D. (1991).
Wisconsin Card Sorting Test performance as a measure of
frontal lobe damage. Journal of Clinical and Experimental
Neuropsychology, 13(6), 909–922. https://doi.org/10.1080/
01688639108405107
Au, A. M., Lam, R., Teng, Y., Lau, K. M., Lai, M. K.,
Chan, C. C., & Leung, P. (2011). The relevance of executive
functioning to academic performance in Hong Kong adoles-
cents. International Journal on Disability and Human Develop-
ment, 10(3), 179–185. https://doi.org/10.1515/ijdhd.2011.031
PsyCh Journal
Berg, E. A. (1948). A simple objective technique for measuring
flexibility in thinking. The Journal of General Psychology, 39
(1), 15–22. https://doi.org/10.1080/00221309.1948.9918159
Bors, D. A., & Stokes, T. L. (1998). Raven’s advanced progres-
sive matrices: Norms for first-year university students and the
development of a short form. Educational and Psychological
Measurement, 58(3), 382–398. https://doi.org/10.1177/
0013164498058003002
Boschloo, A., Krabbendam, L., Aben, A., De Groot, R., &
Jolles, J. (2014). Sorting Test, Tower Test, and BRIEF-SR do
not predict school performance of healthy adolescents in
preuniversity education. Frontiers in Psychology, 5, 287.
https://doi.org/10.3389/fpsyg.2014.00287
Buchsbaum, B. R., Greer, S., Chang, W. L., & Berman, K. F.
(2005). Meta-analysis of neuroimaging studies of the Wis-
consin card-sorting task and component processes. Human
Brain Mapping, 25(1), 35–45. https://doi.org/10.1002/hbm.
20128
Bull, R., Espy, K. A., & Wiebe, S. A. (2008). Short-term memory,
working memory, and executive functioning in preschoolers:
Longitudinal predictors of mathematical achievement at age
7 years. Developmental Neuropsychology, 33(3), 205–228.
https://doi.org/10.1080/87565640801982312
Bull, R., & Lee, K. (2014). Executive functioning and mathemat-
ics achievement. Child Development Perspectives, 8(1), 36–41.
https://doi.org/10.1111/cdep.12059
Bull, R., & Scerif, G. (2001). Executive functioning as a predictor
of children’s mathematics ability: Inhibition, switching, and
working memory. Developmental Neuropsychology, 19(3),
273–293. https://doi.org/10.1207/S15326942DN1903_3
Cantin, R. H., Gnaedinger, E. K., Gallaway, K. C., Hesson-
McInnis, M. S., & Hund, A. M. (2016). Executive functioning
predicts reading, mathematics, and theory of mind during the
elementary years. Journal of Experimental Child Psychology,
146, 66–78. https://doi.org/10.1016/j.jecp.2016.01.014
Chen, Q., & Li, J. (2014). Association between individual differ-
ences in non-symbolic number acuity and math performance:
A meta-analysis. Acta Psychologica, 148, 163–172. https://doi.
org/10.1016/j.actpsy.2014.01.016
Clark, C. A., Pritchard, V. E., & Woodward, L. J. (2010). Pre-
school executive functioning abilities predict early mathematics
achievement. Developmental Psychology, 46(5), 1176–1191.
https://doi.org/10.1037/a0019672
Cragg, L., Keeble, S., Richardson, S., Roome, H. E., &
Gilmore, C. (2017). Direct and indirect influences of executive
functions on mathematics achievement. Cognition, 162, 12–26.
https://doi.org/10.1016/j.cognition.2017.01.014
Credé, M., Tynan, M. C., & Harms, P. D. (2017). Much ado
about grit: A meta-analytic synthesis of the grit literature. Jour-
nal of Personality and Social Psychology, 113(3), 492–511.
https://doi.org/10.1037/pspp0000102
Delis, D. C., Squire, L. R., Bihrle, A., & Massman, P. (1992).
Componential analysis of problem-solving ability: Performance
of patients with frontal lobe damage and amnesic patients on a
new sorting test. Neuropsychologia, 30(8), 683–697. https://
doi.org/10.1016/0028-3932(92)90039-O
Diamond, A. (2013). Executive functions. Annual Review of Psy-
chology, 64, 135–168. https://doi.org/10.1146/annurev-psych-
113011-143750
© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
11
Douglas, H. P., & LeFevre, J. A. (2018). Exploring the influence
of basic cognitive skills on the relation between math perfor-
mance and math anxiety. Journal of Numerical Cognition, 3(3),
642–666. https://doi.org/10.5964/jnc.v3i3.113
Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R.
(2007). Grit: Perseverance and passion for long-term goals.
Journal of Personality and Social Psychology, 92(6),
1087–1101. https://doi.org/10.1037/0022-3514.92.6.1087
Duckworth, A. L., & Seligman, M. E. (2005). Self-discipline out-
does IQ in predicting academic performance of adolescents.
Psychological Science, 16(12), 939–944. https://doi.org/10.
1111/j.1467-9280.2005.01641.x
Espy, K. A., McDiarmid, M. M., Cwik, M. F., Stalets, M. M.,
Hamby, A., & Senn, T. E. (2004). The contribution of execu-
tive functions to emergent mathematic skills in preschool chil-
dren. Developmental Neuropsychology, 26(1), 465–486. https://
doi.org/10.1207/s15326942dn2601_6
Filippetti, A. V., & Richaud, M. C. (2017). A structural equation
modeling of executive functions, IQ and mathematical skills in
primary students: Differential effects on number production,
mental calculus and arithmetical problems. Child Neuropsy-
chology, 23(7), 864–888. https://doi.org/10.1080/09297049.
2016.1199665
Garon, N., Bryson, S. E., & Smith, I. M. (2008). Executive func-
tion in preschoolers: A review using an integrative framework.
Psychological Bulletin, 134(1), 31–60. https://doi.org/10.1037/
0033-2909.134.1.31
Gerst, E. H., Cirino, P. T., Fletcher, J. M., & Yoshida, H. (2017).
Cognitive and behavioral rating measures of executive function
as predictors of academic outcomes in children. Child Neuro-
psychology, 23(4), 381–407. https://doi.org/10.1080/09297049.
2015.1120860
Hau, K. T., & Ho, I. T. (2010). Chinese students’ motivation and
achievement. In M. H. Bond (Ed.), The Oxford handbook of
Chinese psychology (pp. 187–204). Oxford: Oxford University
Press.
Jacob, R., & Parkinson, J. (2015). The potential for school-based
interventions that target executive function to improve aca-
demic achievement: A review. Review of Educational Research,
85(4), 512–552. https://doi.org/10.3102/0034654314561338
Kember, D. (2016). Why do Chinese students out-perform those
from the West? Do approaches to learning contribute to the
explanation? Cogent Education, 3(1), 1248187. https://doi.org/
10.1080/2331186X.2016.1248187
Kramer, J. H., Mungas, D., Possin, K. L., Rankin, K. P.,
Boxer, A. L., Rosen, H. J., … Widmeyer, M. (2014). NIH
EXAMINER: Conceptualization and development of an execu-
tive function battery. Journal of the International Neuropsycho-
logical Society, 20(1), 11–19. https://doi.org/10.1017/
S1355617713001094
Kroesbergen, E. H., Van’t Noordende, J. E., & Kolkman, M. E.
(2014). Training working memory in kindergarten children:
Effects on working memory and early numeracy. Child Neuro-
psychology, 20(1), 23–37. https://doi.org/10.1080/09297049.
2012.736483
Lan, X., Legare, C. H., Ponitz, C. C., Li, S., & Morrison, F. J.
(2011). Investigating the links between the subcomponents of
executive function and academic achievement: A cross-cultural
analysis of Chinese and American preschoolers. Journal of
12
Experimental Child Psychology, 108(3), 677–692. https://doi.
org/10.1016/j.jecp.2010.11.001
Lange, F., Kip, A., Klein, T., Mueller, D., Seer, C., & Kopp, B.
(2018). Effects of rule uncertainty on cognitive flexibility in a
card-sorting paradigm. Acta Psychologica, 190, 53–64. https://
doi.org/10.1016/j.actpsy.2018.07.002
Latzman, R. D., Elkovitch, N., Young, J., & Clark, L. A. (2010).
The contribution of executive functioning to academic achieve-
ment among male adolescents. Journal of Clinical and Experi-
mental Neuropsychology, 32(5), 455–462. https://doi.org/10.
1080/13803390903164363
Latzman, R. D., & Markon, K. E. (2010). The factor structure and
age-related factorial invariance of the Delis-Kaplan Executive
Function System (D-KEFS). Assessment, 17(2), 172–184.
https://doi.org/10.1177/1073191109356254
Lee, K., Bull, R., & Ho, R. M. H. (2013). Developmental changes
in executive functioning. Child Development, 84(6),
1933–1953. https://doi.org/10.1111/cdev.12096
Lee, K., Ng, E. L., & Ng, S. F. (2009). The contributions of work-
ing memory and executive functioning to problem representa-
tion and solution generation in algebraic word problems.
Journal of Educational Psychology, 101(2), 373–387. https://
doi.org/10.1037/a0013843
Lefly, D. L., & Pennington, B. F. (2000). Reliability and validity of the
adult reading history questionnaire. Journal of Learning Disabilities,
33(3), 286–296. https://doi.org/10.1177/002221940003300306
Madjar, N., & Shalley, C. E. (2008). Multiple tasks’ and multiple
goals’ effect on creativity: Forced incubation or just a distrac-
tion? Journal of Management, 34(4), 786–805. https://doi.org/
10.1177/0149206308318611
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H.,
Howerter, A., & Wager, T. D. (2000). The unity and diversity
of executive functions and their contributions to complex “fron-
tal lobe” tasks: A latent variable analysis. Cognitive Psychol-
ogy, 41(1), 49–100. https://doi.org/10.1006/cogp.1999.0734
Monette, S., Bigras, M., & Guay, M. C. (2011). The role of the
executive functions in school achievement at the end of Grade
1. Journal of Experimental Child Psychology, 109(2), 158–173.
https://doi.org/10.1016/j.jecp.2011.01.008
Nusbaum, E. C., & Silvia, P. J. (2011). Are intelligence and crea-
tivity really so different? Fluid intelligence, executive pro-
cesses, and strategy use in divergent thinking. Intelligence, 39
(1), 36–45. https://doi.org/10.1016/j.intell.2010.11.002
O’connor, B. P. (2000). SPSS and SAS programs for determin-
ing the number of components using parallel analysis and
© 2020 The Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd
Set-shifting and academic achievement
Velicer’s MAP test. Behavior Research Methods, Instru-
ments, & Computers, 32(3), 396–402. https://doi.org/10.
3758/BF03200807
Pan, X., & Yu, H. (2016). Different effects of cognitive shifting
and intelligence on creativity. The Journal of Creative Behav-
ior, 52(3), 212–225. https://doi.org/10.1002/jocb.144
Poropat, A. E. (2009). A meta-analysis of the five-factor model of
personality and academic performance. Psychological Bulletin,
135(2), 322–338. https://doi.org/10.1037/a0014996
Revelle, W. R. (2017). psych: Procedures for personality and psy-
chological research. Software. Retrieved from https://cran.r-
project.org/package=psych
Rohde, T. E., & Thompson, L. A. (2007). Predicting academic
achievement with cognitive ability. Intelligence, 35(1), 83–92.
https://doi.org/10.1016/j.intell.2006.05.004
Schmidt, F. L., & Hunter, J. E. (1999). Theory testing and mea-
surement error. Intelligence, 27, 183–198. https://doi.org/10.
1016/S0160-2896(99)00024-0
Silver, E. A. (2009). Cross-national comparisons of mathematics
curriculum materials: What might we learn? ZDM, 41(6),
827–832. https://doi.org/10.1007/s11858-009-0209-1
Sisk, V. F., Burgoyne, A. P., Sun, J., Butler, J. L., &
Macnamara, B. N. (2018). To what extent and under which cir-
cumstances are growth mind-sets important to academic
achievement? Two meta-analyses. Psychological Science, 29
(4), 549–571. https://doi.org/10.1177/0956797617739704
St Clair-Thompson, H., Stevens, R., Hunt, A., & Bolder, E.
(2010). Improving children’s working memory and classroom
performance. Educational Psychology, 30(2), 203–219. https://
doi.org/10.1080/01443410903509259
Weintraub, S., Dikmen, S. S., Heaton, R. K., Tulsky, D. S.,
Zelazo, P. D., Bauer, P. J., & Gershon, R. C. (2013). Cognition
assessment using the NIH Toolbox. Neurology, 80(11 Suppl.
3), S54–S64. https://doi.org/10.1212/WNL.0b013e3182872ded
Yeniad, N., Malda, M., Mesman, J., van IJzendoorn, M. H., &
Pieper, S. (2013). Shifting ability predicts math and reading
performance in children: A meta-analytical study. Learning and
Individual Differences, 23, 1–9. https://doi.org/10.1016/j.lindif.
2012.10.004
Zhu, Y., & Fan, L. (2006). Focus on the representation of prob-
lem types in intended curriculum: A comparison of selected
mathematics textbooks from Mainland China and the United
States. International Journal of Science and Mathematics Edu-
cation, 4(4), 609–626. https://doi.org/10.1007/s10763-006-
9036-9