Cognitive Motivational Engagement in ICT Mediates The Effect of ICT Use On Academic Achievements

Computers & Education 204 (2023) 104871
Contents lists available at ScienceDirect
Computers & Education

journal homepage: www.elsevier.com/locate/compedu
Cognitive-motivational engagement in ICT mediates the effect of

ICT use on academic achievements: Evidence from 52 countries
Sandy C. Li a, *, Jinxin Zhu b
a
Department of Educational Studies, Hong Kong Baptist University, Hong Kong Special Administrative Region of China
b
Department of Curriculum and Instruction, The Education University of Hong Kong, Hong Kong Special Administrative Region of China
A R T I C L E I N F O A B S T R A C T
Keywords: Despite huge investment of information and communication technology (ICT) in schools across
PISA 2018 the world, recent studies showed its non-positive association with academic achievement. While
ICT use educators or policymakers may doubt the contribution of ICT in education, few studies examined
Academic achievement
the indirect effect of ICT use on academic achievement via ICT-related dispositions, such as
Cognitive-motivational engagement in ICT
Multilevel structural equation modelling
students’ perceived autonomy, competence, and interest in ICT (cognitive-motivational engagement
in ICT). Guided by self-determination theory (SDT) and related literature, we hypothesized a
three-level structural equation model to examine the interplay between ICT use, cognitive-
motivational engagement in ICT, and academic achievement based on 15-year-old secondary
students from 52 countries in the PISA 2018. The results showed cognitive-motivational
engagement in ICT exerted a positive mediating effect between ICT use and academic achieve
ment at the student and school levels, indicating ICT use exerted an indirect positive effect on
academic achievement, and satisfaction of students’ basic psychological needs in a specific area
such as ICT is likely to promote their optimal functioning in other areas such as learning. Hence,
students should be provided more opportunities to orchestrate technologies in formal and
informal contexts helps develop their self-determination, discernment, and self-regulation in ICT
use, and thereby creating the conditions in which better learning outcomes are more likely to
occur. These findings also deepen our understanding of SDT and provide a guiding framework for
educators to design pedagogies that can better address students’ ICT-related psychological needs.
Finally, this study shows that a quadratic term should be modeled when examining the rela
tionship between ICT use and academic achievement.
1. Introduction
Despite the goodwill that ICT infusion can help transform traditional classroom learning, promote educational outcomes, and
bridge achievement gaps (Looi et al., 2020; Reyes, 2021), recent studies showed ICT is not positively (either negatively or inverted
U-shaped) associated with students’ academic achievement (Bhutoria & Aljabri, 2022; Kong et al., 2022; Odell et al., 2020; Ünal et al.,
2022). While educators or policymakers may doubt the contribution of ICT in education, few studies examine the indirect effect of ICT
use on students’ academic achievement via ICT-related dispositions, such as their perceived autonomy, competence, and relatedness in
* Corresponding author. AAB814, 8/F, Academic and Administration Building Hong Kong Baptist University, 15 Baptist University Road, Kowloon
Tong, Hong Kong Special Administrative Region of China.
E-mail address: sandyli@hkbu.edu.hk (S.C. Li).
https://doi.org/10.1016/j.compedu.2023.104871
Received 16 September 2022; Received in revised form 7 July 2023; Accepted 10 July 2023
Available online 17 July 2023
0360-1315/© 2023 Elsevier Ltd. All rights reserved.
S.C. Li and J. Zhu Computers & Education 204 (2023) 104871
ICT (cognitive-motivational engagement in ICT; Kunina-Habenicht and Goldhammer (2020). Guided by the self-determination theory
(SDT) (Ryan & Deci, 2017) and related literature (e.g. Zhu and Chiu, 2019; Chiu (2022); Kunina-Habenicht and Goldhammer (2020)),
we hypothesized and tested that students’ cognitive-motivational engagement in ICT predicts their academic achievement and that the
former mediates the relationship between ICT use and academic achievement with a three-level non-linear structural equation model
(SEM) on the PISA 2018 database. As past studies showed that satisfactions of students’ psychological needs for autonomy, compe
tence, and relatedness are important factors for fostering intrinsic motivation and self-regulation for optimal functioning and
fulfillment across different contexts, including personal growth and learning (Conesa et al., 2022; Jeno et al., 2018; Ryan & Deci,
2017). Along the same vein, we argue that the satisfaction of students’ ICT-related autonomy, competence, and relatedness has a
positive effect on motivating students to tackle the challenges they encounter beyond the formal learning settings with ICT. These
informal learning experiences are likely to foster students’ self-efficacy and self-regulatory capacities for learning, thereby better
academic achievement. Moreover, this study examines whether the direct, non-positive effects of ICT use on academic achievement
could be found across different countries. If these non-positive effects of ICT use are prevalent phenomena, a deeper understanding of
the mediating role of cognitive-motivational engagement in ICT will help inform future ICT implementation in schools, and will
provide researchers and educators a guiding framework for reshaping ICT pedagogies in a way that students’ ICT-related psychological
needs, and thereby their self-determination in ICT use can be better addressed.
2. Conceptual framework and literature review
2.1. Self-determination theory in learning settings
According to SDT (Ryan & Deci, 2000), autonomy (control of their own lives), competence (mastery over challenges), and relatedness
(sense of cohesiveness and attachment to other people) are the three universal basic psychological needs of human beings. Satisfaction
of these needs is essential for individuals’ optimal functioning and growth (Deci, 1971; Ryan & Deci, 2000). Moreover, these psy
chological needs can be seen as an “ecological phenomenon” and are generalizable across different settings (Malian & Nevin, 2002, p.
73), transcending from personal growth and well-being (Deci, 1971; Ryan & Deci, 2000) to workplace (Manganelli, Thibault-Landry,
Forest, & Carpentier, 2018a, 2018b), and to education settings (Conesa et al., 2022). Specifically, in the learning setting, if students’
basic psychological needs are satisfied, they are more likely to involve in activities for learning, growth, and fulfillment (Deci, 1971),
and thereby better academic achievement (Zhou et al., 2021). Similar results were reported in the online learning settings (Chiu, 2022;
Mehrvarz, Heidari, Farrokhnia, & Noroozi, 2021; Yang et al., 2018). According to SDT, autonomy and competence are regarded as the
pivotal factors for intrinsic motivation, whereas relatedness is considered as the contributing factor for transcending external values
and behaviours into self-regulation (Jeno et al., 2018; Ryan & Deci, 2017). Hence, we argue that the satisfaction of students’
ICT-related basic psychological needs (i.e., autonomy, competence, and relatedness in ICT use; cognitive-motivational engagement in ICT,
Kunina-Habenicht and Goldhammer (2020) enhances their intrinsic motivation and self-regulation, with which students will be
self-driven to use ICT to address the challenges they are facing in their daily lives and informal learning contexts (Mehrvarz et al., 2021;
Odell et al., 2020). These informal learning experiences may have a positive effect beyond ICT-related behaviours or achievements,
likely building students’ self-efficacy, self-directedness, and self-regulatory capacities for learning in and across the formal and
informal settings, and thereby better academic achievement (Conesa et al., 2022; Mehrvarz et al., 2021; Yang et al., 2018).
Moreover, psychological need satisfaction (or level of perceived autonomy, competence, and relatedness) is a result of past
experience and varies from person to person, depending on the social conditions with which the person is interacting (Deci, 1985; Ryan
& Deci, 2000). In the ICT setting, students may develop their cognitive-motivational engagement in ICT via participation in varied ICT
activities (e.g., ICT use at school in general, ICT use outside school for school work, and ICT use for leisure (Li & Petersen, 2022). Other
background variables such as family socioeconomic status and ICT availability at school may also associate with their
cognitive-motivational engagement in ICT. Students with better family socioeconomic status and easier access to ICT at school may
have more chances of interaction with ICT and are more likely to have a higher level of ICT autonomy, competence, and interest (Zhu
and Mok, 2020). In addition, these past ICT experience and the background variables may contribute to students’ academic
achievement directly (will be discussed in later sections).
2.2. Cognitive-motivational engagement in ICT and academic achievement
As discussed above, the satisfaction of students’ ICT-related basic psychological needs enhances their intrinsic motivation to use
ICT to solve problems arisen from their daily lives and informal learning contexts. These learning experiences may help promote
students’ self-efficacy and self-regulatory capacities for learning across different contexts, and thereby students’ academic achieve
ment (Mehrvarz et al., 2021; Odell et al., 2020) Specifically, grounded on the SDT (Ryan & Deci, 2000), cognitive-motivational
engagement in ICT is conceptualized as the cognitive-motivational factors (e.g., perceived autonomy in ICT use, perceived competence
in ICT use, and relatedness in ICT use) facilitating the development of ICT skills in a self-regulated and intrinsically motivated way
(Zylka et al., 2015). Perceived ICT autonomy refers to students’ need to feel in charge of their own behaviours and actions related to
ICT use. Perceived ICT competence refers to students’ need to gain mastery of the necessary ICT skills to achieve their goals. ICT
relatedness refers to students’ need to develop interest, affectionate experience, and intrinsic motivation in ICT use.
In the present study, we hypothesized that students’ cognitive-motivational engagement in ICT predicts their academic achieve
ment. However, past studies showed inconclusive results regarding the relationship between three aspects of cognitive-motivational
engagement in ICT and academic achievement. For instance, Ma and Qin (2021) found that students’ perceived autonomy in ICT use
2
was positively associated with their academic achievement, which is consistent across 16 countries (regions) in PISA 2018 study;
however, students’ ICT-related interest and perceived competence in ICT use was associated with academic achievement positively in
some countries and negatively in others. Also, employing multilevel analysis for the summarized effects across countries using PISA
2015 database, Hu et al. (2018) found that ICT-related interest and autonomy in ICT use had positive associations with academic
achievement at the student and school levels; however, the association between competence in ICT and academic achievement was
significant at the student level but not at the school level. Similar mixed results were found in analyses conducted by (Courtney et al.,
2022) and Park and Weng (2020).
In the abovementioned studies, different facets of cognitive-motivational engagement in ICT were treated as separate variables in
their analysis. As these facets are moderately or highly correlated with each other, treating them as separate variables might yield a
statistical illusion with some of them as negative suppressors (which are positively related to the outcome variable but show a negative
regression coefficient) (Chen & Krauss, 2014). Hence, they should be treated as one construct; however, it remains unknown how
cognitive-motivational engagement in ICT as a single construct interacts with students’ academic achievement and their ICT use. We
will address this gap in this study.
2.3. ICT use, cognitive-motivational engagement in ICT, and academic achievement
According to SDT, ICT use in varied settings (e.g., ICT use at school in general, ICT use outside school for school work, and ICT use
for leisure) provides students with ample opportunities to orchestrate technologies to satisfy their basic psychological needs related to
ICT, and thereby support of their learning in both formal and informal contexts (Deci, 1985; Ryan & Deci, 2000). In other words,
cognitive-motivational engagement in ICT may mediate the relationship between ICT use and academic achievement.
For instance, Rohatgi et al. (2016) identified an indirect effect of ICT use on students’ computer and information literacy
achievements via students’ ICT competence based on the Norwegian data obtained from the International Computer and Information
Literacy Study. Legrain et al. (2015) conducted a quasi-experimental study and found that students’ self-determination in physical
education (PE) mediated the relationship between integrating ICT in PE (compared to traditional teaching) and their achievements in
cognitive and motor skills in gymnastics exercises. In another study on biology instructions, using a mobile application (compared to
traditional textbooks) predicted higher intrinsic motivation and self-determination in learning biology which, in turn, predicted better
biology achievement (Jeno et al., 2019). However, the above studies tested the SDT in domain-specific settings (e.g., ICT, PE, and
biology), but did not investigate whether ICT use and self-determination in ICT can exert effects beyond the ICT setting, contributing to
academic achievement. This study will address this gap.
2.4. The potential non-linear effect of ICT use on academic achievement
Past studies on the linear relationship between ICT use and academic achievement yielded mixed results, with some showing
positive results. For instance, based on PISA 2015, Hu et al. (2018) and Gómez-Fernández and Mediavilla (2021) revealed that stu
dents’ ICT use outside school for leisure correlated positively with their academic achievement. However, other studies reported
negative results. For instance, Zhang and Liu (2016) examined the five waves of PISA studies from 2000 to 2012 and identified the
trend that students’ ICT use was negatively correlated with their science and mathematics achievements. Likewise, ICT use at school
(Hu et al., 2018), ICT use outside school for schoolwork (Zhang & Liu, 2016), and ICT use outside school for leisure (Gómez-Fernández
& Mediavilla, 2021; Navarro-Martinez & Peña-Acuña, 2022; Odell et al., 2020; Park & Weng, 2020) were found to correlate negatively
with students’ academic achievement.
However, the ambivalent results might arise from imposing the assumption of a linear relationship between students’ ICT use and
their academic achievement (Rodrigues and Biagi (2017). By examining the academic achievement scores of three groups of students
with high, moderate, and low ICT use in PISA 2015, they found that moderate ICT use, rather than high or low use, positively predicted
academic scores, and suggested that the relationship between students’ ICT use and their academic achievement might follow an
inverted U-shaped curve. A similar non-linear relationship was found between ICT use and reading performance in the Dutch sample
(Gubbels et al., 2020). Nonetheless, in the studies conducted by Gubbels et al. (2020) and Rodrigues and Biagi (2017), they did not
model the multilevel structure. As the structure of PISA data was intrinsically hierarchical, treating single-level data as independent
observations may yield biased results. Based on the data of a single participating economy (Hong Kong) in PISA 2018, this inverted-U
shaped relationship was also identified between ICT use outside school for schoolwork and students’ academic achievement; yet, the
association between ICT use at school and academic achievement was linear and negative (Zhu and Li (2022). However, whether the
potential non-linear relationships between ICT use and academic achievement could be found across different countries remains an
open question. This study will address this gap using a three-level quadratic regression analysis.
2.5. ICT availability at school and academic achievement
Building the ICT infrastructure provides students with chances to engage in ICT-related activities and efficient access to learning
resources and hence may contribute to student academic achievement. For instance, Zhang and Liu (2016) found that the number of
Internet-connected computers available to students exerts a positive influence on school average academic achievement across the four
cycles of PISA studies from 2000 to 2009. However, the influence was found negative based on PISA 2012 database (Zhang and Liu
(2016)) and non-significant based on the PISA 2015 database ((Hu et al., 2018). In PISA 2018, it was observed that countries or
economies having higher reading performance tended to have more Internet-connected school computers available to students (OECD,
3
2020). Nonetheless, few studies examined the influence at the school and country levels simultaneously. Also, it remains unknown how
ICT availability at school, such as the proportion of Internet-connected school computers available to students, competes with
cognitive-motivational engagement in ICT in predicting students’ academic achievement after controlling for students’ socioeconomic
status and gender.
2.6. Socioeconomic status and academic achievement
Students from families with more human social, cultural, and financial capitals (socioeconomic status (Avvisati, 2020); usually have
more learning resources and hence more learning opportunities and better learning achievement (Zhu and Li, 2022). For instance,
students’ academic achievement were found to correlate strongly with their socioeconomic status (Chiao & Chiu, 2018; OECD, 2019b).
In the present study, to account for its influence on students’ academic achievement, students’ socioeconomic status was used as a
control variable in our multilevel analysis.
2.7. Gender and academic achievement
Past studies repeatedly reported gender disparity in academic achievement (e.g., OECD, 2015; 2019b). In general, girls out
performed boys in reading while the opposite was true in mathematics (OECD, 2015). The gap observed in science achievement was
narrower than that in reading and mathematics. Although the gaps were found to be tightening in PISA 2018, gender disparity in
achievements remains a matter of considerable concern in many countries (OECD, 2019b). In the present study, we account for
gender’s influence on students’ academic achievement by treating it as a control variable in our analysis.
2.8. The present study
As discussed above, students’ ICT use may affect their academic achievement indirectly via cognitive-motivational engagement in
ICT. Providing more opportunities for students to use ICT may help foster their cognitive-motivational engagement in ICT. Hence,
students may be more self-motivated to challenge themselves with ICT use in problem solving (Odell et al., 2020) and likely in ac
ademic learning (Jeno et al., 2017), and hence have better academic achievement (Conesa et al., 2022; Yang et al., 2018).
Apart from the direct effects, this study examines the nonlinear effects of ICT use on academic achievement with a sample from 52
countries or economies in PISA 2018 database. Specifically, we tested two different multilevel structural equation models with random
intercepts at the school and country levels: the quadratic model and the linear model. ‘Quadratic’ refers to the incorporation of a
quadratic term and a linear term of ICT use as predictors for academic achievement and cognitive-motivational engagement in ICT. To
unfold the ambivalence in the relationship between ICT use and students’ academic achievement across counties, the fitting indices of
and the proportion of variance of academic achievement explained by the two models were compared.
Specifically, the research questions of this study were as follows:
After controlling the effect of gender at the student and school levels, and the effect of students’ socioeconomic status at the student,
school, and country levels,
1. Whether and how did cognitive-motivational engagement in ICT mediate the effect of ICT use on academic achievement?
2. How did ICT use directly affect academic achievement?
3. How did ICT availability at school directly affect academic achievement?
3. Methods
3.1. Data source and participants
The data source for the present study was derived from the PISA study orchestrated by Organization for Economic Co-operation and
Development (OECD) in 2018. In every three-year cycle, PISA assesses Grade-9 (15-year-old) students’ performance in applying their
knowledge and skills in three key learning domains: reading, mathematics, and science, with a stronger focus on a selected domain in
each three-year cycle to address challenges that emerge from their everyday-life experiences. In the present study, we utilized the data
derived from students’ cognitive test scores in Science, Mathematics, and Reading, Student Context Questionnaire, School Context
Questionnaire, and the ICT Familiarity Questionnaire. As not all countries or economies chose to take part in the ICT Familiarity
survey, there were altogether 52 countries or economies with 11,403 schools and 250,163 students involved in our analysis. It is
The key reasons for choosing PISA 2018 database were twofold. Firstly, it contains variables probing students’ ICT use in a wide
range of contexts which were not included in previous PISA studies. These include subject-related ICT use during lessons, subject-
related ICT use outside lessons, ICT use outside school for schoolwork, ICT use, ICT use at school in general and ICT use outside
school for leisure, where the first two were unique to PISA 2018. These variables enabled us to have a better discernment of students’
use of ICT for learning and its impact on their academic achievement. Secondly, as one of the focuses of the present study was to
examine the role of cognitive-motivational engagement in ICT in affecting students’ academic achievement, the PISA 2018 database
contains the necessary variables, such as students’ perceived autonomy in ICT use, perceived competence in ICT use and relatedness in
ICT use for building the construct.
4
3.2. Measures
In PISA studies, many questionnaire items were deliberately designed to be combined and scaled to form meaningful indices for
measuring latent traits that cannot be observed directly. In some PISA studies, these indices were further combined to form higher-
order latent variables (Burns et al., 2020; Kunina-Habenicht & Goldhammer, 2020; Park & Weng, 2020).
The present analysis involved two higher-order latent variables: students’ academic achievement and students’ cognitive-
motivational engagement in ICT derived from their respective manifest variables, as well as indices related to students’ ICT use in
different contexts, ICT availability at school, students’ socioeconomic status, gender, final adjusted student weight, and final adjusted
school weight. As PISA adopted a stratified two-stage sample design, student weights and school weights must be integrated into the
analysis to warrant that each sampled student properly represents the correct number of students in the entire PISA population.
The construct of cognitive-motivational engagement in ICT was incorporated in the ICT Familiarity Questionnaire of PISA 2015;
PISA 2018. The measurement invariance of this construct was verified by Ma and Qin (2021), showing students’ observed scores on the
construct can be compared meaningfully.
3.2.1. Students’ academic achievement

In PISA studies, science literacy (SCIE) was assessed based on how well students demonstrated their conceptual and procedural
knowledge about science, while mathematics literacy (MATH) and reading literacy (READ) were related respectively to the extent to
which students were able to apply mathematics to solve real-world problems, and how well students comprehended, used, and re
flected on written texts. Students were not required to complete the entire assessments but different sections of the assessments with
overlapping items. To reduce measurement errors, PISA generated 10 plausible values for each student’s science, mathematics, and
reading scores from a probability distribution representing the range of the student’s ability (OECD, 2019a), for instance, PV1SCIE –
PV10SCIE for science, PV1MATH – PV10MATH for mathematics, and PV1READ – PV10READ for reading (OECD, 2019c). In the
present study, we defined students’ academic achievement as a latent variable comprising the scores of science (SCIE), mathematics
(MATH), and reading (READ) as manifest variables.
3.2.2. Students’ ICT use in various contexts (HOMESCH, USESCH, & ENTUSE)
In PISA 2018, students’ ICT use was differentiated by the location and purpose of technology use. It involved 3 major scales,
namely, ICT use outside school for schoolwork (HOMESCH), ICT use at school in general (USESCH), and ICT use outside school for
leisure (ENTUSE).
HOMESCH was derived from 11 items with a 5-point Likert scale ranging from never to everyday, asking students how often they
use digital devices outside school for specific activities like browsing the Internet for schoolwork, using social networks for
communication with teachers, doing homework on a computer, etc. The reliability of HOMESCH ranges from 0.87 to 0.97 across all the
participating countries and economies (OECD, 2019c).
USESCH comprised 10 items with the same scale used for HOMESCH, asking students how often they used digital devices at school
for specific activities, such as using email, browsing Internet for schoolwork, and using learning apps. USESCH was derived based on
students’ responses on these items using IRT scaling (OECD, 2019c). The reliability of USESCH ranges from 0.79 to 0.97 across all the
participating countries and economies (OECD, 2019c).
ENTUSE was derived from 12 items with the same scale used for USESCH, asking students how often they use digital devices outside
school for leisure activities e.g., playing one-player games, playing collaborative online games, chatting online, playing online games
via social networks, downloading music, films, games, or software from the Internet. The reliability of ENTUSE ranges from 0.78 to
0.94 across all the participating countries and economies (OECD, 2019c).
3.2.3. Cognitive-motivational engagement in ICT

In the studies conducted by Meng et al. (2019), and Kunina-Habenicht and Goldhammer (2020) based on the Chinese, German and
Swiss samples of PISA 2015, cognitive-motivational engagement in ICT was defined as a latent construct comprising students’
perceived autonomy in ICT use (AUTICT), students’ perceived ICT competence (COMPICT), students’ interest in ICT use (INTICT).
AUTICT was derived from 5 items based on IRT scaling, probing students’ perceived autonomy in using digital media and digital
devices, e.g. “If I need a new application, I choose it by myself,” etc. A 4-point Likert scale ranging from “strongly disagree” to “strongly
agree” was adopted for each item (OECD, 2019c).
COMPICT was derived from 5 items, with the same scale used for AUTICT, probing students’ perceived competence in using digital
media and digital devices, e.g. “I feel comfortable using digital devices that I am less familiar with,” etc. (OECD, 2019c).
INTICT was derived from 5 items based on IRT scaling, with the same scale used for AUTICT, probing students’ interest in using
digital media and digital devices, e.g. “I forget about time when I’m using digital devices,” etc. (OECD, 2019c).
3.2.4. ICT availability at school (RATCMP1 & RATCMP2)

ICT availability at school was measured by the number of available computers per student for educational purposes (RATCMP1)
and the proportion of Internet-connected computers at school available to students (RATCMP2) derived from School Context
Questionnaire.
3.2.5. Socioeconomic status (ESCS) and gender (GENDER)

In PISA, students’ economic, social, and cultural status (ESCS) is a composite variable comprising three manifest variables which
5
include highest parental occupation, parental education, and home possessions including books. As there is no direct income measure
available in the data, PISA uses household items as a proxy for family wealth (OECD, 2019c).
3.3. Multilevel structural equation modelling (MSEM)
In the multilevel SEM analysis, each variable was partitioned separately into three components: student (within-school) level
component, the school (between-school) level component, and the country (between-country) level component, except for RATCMP1
and RATCMP2. Since RATCMP1 and RATCMP2 are related to ICT availability at school, they are, by nature, school-level variables.
Thus, they were partitioned into two components only: the school (between-school) level components and the country (between-
country) level components. The school-level components of RATCMP1 and RATCMP2 were computed by centering the variables at the
weighted average of individual countries while the country (between-country) level components refer to the weighted country average
of individual countries or economies.
3.3.1. The quadratic model

Based on the conceptual framework depicted in Fig. 1, we explored the non-linear relationship between ICT use and academic
achievement, and the mediation of cognitive-motivational engagement in ICT between ICT use and academic achievement by con
structing a three-level quadratic structural equation model with random-intercept at the school and country levels.
At the student level, after controlling for the effects of GENDER on SCIE, MATH, and READ, and the effect of ESCS on academic
achievement as shown in Fig. 1, a quadratic relationship was prescribed between ICT use in different contexts and academic
achievement, in which a quadratic term (HOMESCH^2, USESCH^2, and ENTUSE^2) and a linear term (HOMESCH, USESCH, and
ENTUSE) of each type of ICT use were prescribed as the predictors for academic achievement. A linear relationship was prescribed
between ICT use in different contexts and cognitive-motivational engagement in ICT, in which only the linear term (HOMESCH,
USESCH, and ENTUSE) of each type of ICT use was used as the predictor for academic achievement. In addition, cognitive-
motivational engagement in ICT was also prescribed as a predictor for academic achievement. Through these structural relation
ships, the role of cognitive-motivational engagement in ICT in mediating the indirect effects of ICT use on acdemic achievements was
examined.
At the school level, the structural relationships were similar to that prescribed at the student level, with the student-level com
ponents being substituted by their corresponding school-level components. In addition, RATCMP1, RATCMP2 was also prescribed as a
predictor for academic achievement and cognitive-motivational engagement in ICT.
Likewise, at the country level, the structural relationships were basically the same as the prescribed at the school level, with the
school-level components were substituted by their corresponding country-level components. Nonetheless, in our preliminary analysis,
collinearity between predictor variables at the country level was found. A strong correlation was found between RATCMP2 and
ENTUSE^2 (− 0.770), between RATCMP2 and ENTUSE (0.592), between HOMESCH^2 and USESCH^2 (0.825), and between HOMESCH
and USESCH (0.683) (Appendix 2). As collinearity between predictor variables may reduce their statistical significance in the
regression analysis, USESCH, USESCH^2, ENTUSE, and ENTUSE^2 were removed from the model. RATCMP2 was kept as the predictor
for academic achievement as it has a strong correlation with SCIE, MATH, and READ, ranging from 0.65 to 0.67 (Appendix 2).
Fig. 1. A conceptual framework depicting the structural relationships among students’ academic achievement, ICT use, cognitive-motivational
engagement in ICT, and ICT availability at school, with gender and student’s socioeconomic status as control variables at the student, school,
and country levels. (Note: ‘ICT availability at school’ is included as variables at the school and country levels. Dashed and solid lines represent
respectively a quadratic and a linear relationship.)
6
3.3.2. The linear model

To compare the relative strength of the quadratic models in accounting for the variance in academic achievement, a linear model
was constructed by constraining the coefficient of the quadratic term of each type of ICT use appeared in the quadratic model to zero.
3.3.3. The baseline model

For comparison, a baseline model was also constructed by constraining all the coefficients of the independent variables such as
different types of ICT use, cognitive-motivational engagement in ICT, and the proportion of Internet-connected computers available to
students at school to zero except for the two control variables: GENDER and ESCS.
3.3.4. Analysis
The analysis was conducted using Mplus. Mplus ran 10 analyses with the 10 sets of plausible values (the nth set of plausible values
of the three academic achievement scores used in one analysis), and summarized the results with the average estimates and adjusted
the standard errors.
4. Results
4.1. Model comparison
The ICCs for achievement in science, mathematics, and reading were.217, 0.219, and 0.2 32 respectively at the school level, and
0.202, 0.232, and 0.189 respectively at the country level. Hence, a multi-level analysis should be conducted to avoid potentially biased
results if the nested structure was overlooked. The results given in Tables 1 and 2 indicate the quadratic model has relatively smaller
AIC, BIC, and chi-square values, indicating the quadratic model has a better data fit as compared to the linear and baseline models. As
depicted in Table 3, the quadratic and linear models respectively account for 11.8% and 9.7% of the variance (R2) of academic
achievement, indicating the quadratic model has a 22% improvement in R2 of academic achievement, thereby showing a better data fit
as compared to the linear model at the student level. Nonetheless, at the school and country levels, the R2 of academic achievement
explained by the quadratic and linear models are close to one another (Table 3).
4.2. The student-level quadratic model
4.2.1. ICT use in different contexts as predictors of academic achievement

As shown in Table 4, the estimates of the unstandardized coefficients of HOMESCH^2, USESCH^2, and ENTUSE^2 are negative and
significant, indicating the relationship between ICT use and academic achievement follows an inverted U-shaped curve as depicted in
Fig. 2(a)-(c).
4.2.2. Cognitive-motivational engagement in ICT as a predictor of academic achievement

As depicted in Table 4, the estimate of the unstandardized coefficient of cognitive-motivational engagement in ICT is positive and
significant, showing that the cognitive-motivational engagement in ICT is a predictor of academic achievement at the student level and
that they follow a positive linear relationship.
4.2.3. ICT use in different contexts as predictors of cognitive-motivational engagement in ICT

As given in Table 4, the unstandardized coefficients of HOMESCH, USESCH, and ENTUSE are positive and significant, indicating
ICT use in different contexts exert a direct positive effect on cognitive-motivational engagement in ICT. Among the three different types
of ICT use, ICT use for leisure has the strongest effect on cognitive-motivational engagement in ICT.
4.3. The school-level quadratic model
4.3.1. ICT use in different contexts as predictors of academic achievement

As shown in Table 4, only the unstandardized coefficients of USESCH, USESCH^2, ENTUSE, and ENTUSE^2 are significant, with the
quadratic terms being negative, showing the relationships between USESCH and academic achievement, and between ENTUSE and
academic achievement follow an inverted U-shaped curve (Fig. 3(a)-(c)). Nonetheless, taking a close examination of the 2.5–97.5
percentile range of USESCH and ENTUSE, academic achievement decreases monotonically as USESCH and ENTUSE increase.
Table 1
Fitting indices of the quadratic, modified quadratic, linear, and baseline models.
Model χ2 (dfa) CFI TLI RMSEA SRMR AIC BIC
Quadratic 6642.208 (160) .957 .938 .012 .036 3555205.5 3556236.7

Linear 7927.227 (167) .948 .929 .013 .048 3559942.1 3560901.1
Baseline 18347.301 (192) .879 .856 .019 .104 3604880.3 3605581.5
a
Degrees of freedom.
7
Table 2
Chi-square difference test between SEM models.
Model comparison Sattora-Bentler Scaled χ2 Difference (TRd) Difference in Degrees of Freedom (Δdf = dfB-dfA) Two-tailed p-value for TRd and Δdf
A B
Quadratic Baseline 11705.0930 32 .0000

Linear Baseline 10420.0740 25 .0000
Quadratic Linear 1285.0190 7 .0000
Table 3
R-square estimates of academic achievement at the student, school, and country levels.
R2 estimate of academic achievement (standard error)
Student-level School-level Country-level

Quadratic 0.118 (0.007) 0.697 (0.022) 0.562 (0.065)
Linear 0.097 (0.007) 0.698 (0.023) 0.566 (0.065)
Baseline 0.033 (0.005) 0.585 (0.026) 0.549 (0.088)
Table 4
Unstandardized estimates of the quadratic models at the student, school, and country levels.
Variable Student School Country
Measurement part
cognitive-motivational engagement in ICT →AUTICT 1 1 1
→COMPICT 0.836*** 1.119*** 1.552***
→INTICT 1.064*** 1.061*** 1.099***
academic achievement →SCIE 1 1 1
→MATH 0.888*** 0.996*** 1.086***
→READ 0.986*** 1.001*** 0.943***
Regression part
cognitive-motivational engagement in ICT →academic achievement 0.184*** 1.568*** 0.671
HOMESCH − 0.054*** − 0.023 − 0.243**
HOMESCH^2 − 0.025*** − 0.031 0.179
USESCH − 0.109*** − 0.139*** /
USESCH^2 − 0.018*** − 0.080** /
ENTUSE 0.029*** − 0.468*** /
ENTUSE^2 − 0.020*** − 0.048** /
RATCMP1 / − 0.016** 0.146
RATCMP2 / 0.048 1.000*
ESCS 0.150*** 0.513*** 0.290**
HOMESCH →cognitive-motivational engagement in ICT 0.053*** − 0.004 0.004
USESCH 0.052*** − 0.024* /
ENTUSE 0.238*** 0.316*** /
RATCMP1 / 0.001 0.047
RATCMP2 / 0.006 0.123
ESCS 0.062*** 0.105*** 0.069
GENDER 0.152*** − 0.054 − 0.643
GENDER →SCIE 0.024* − 0.062 − 5.919
GENDER →MATH 0.095*** 0.036 − 3.537
GENDER →READ − 0.226*** − 0.386*** − 5.818*
Note: (1) ^2 represents the quadratic term of the associated variable; (2)/stands for not applicable; (3) ***p < .001, **p < .01, *p < .05.

As given in Table 4, the unstandardized coefficient of cognitive-motivational engagement in ICT is positive and significant, indi
cating cognitive-motivational engagement in ICT exerts a positive linear effect on academic achievement at the school level. It is also
noteworthy that the standardized coefficient of cognitive-motivational engagement given in Appendix 5 is 0.559, showing the effect of
cognitive-motivational engagement on academic achievement is of a large effect size (Cohen, 1988).
4.3.3. ICT use in different contexts as predictors of cognitive-motivational engagement in ICT

As given in Table 4, ENTUSE exerts a strong positive effect on cognitive-motivational engagement in ICT. This strong effect can also
be reflected by strong correlation between ENTUSE and the manifest variables of cognitive-motivational engagement in ICT (AUTICT,
INTICT, and COMPICT), which ranges from 0.77 to 0.82 (Appendix 1).
Although USESCH exerts a weak negative effect on cognitive-motivational engagement in ICT, and HOMESCH has no significant
effect, both variables have a moderate positive correlation with the manifest variables of cognitive-motivational engagement in ICT
(AUTICT, INTICT, and COMPICT), ranging from 0.142 to 0.433 (Appendix 1). The change sign between correlation and regression
8
Fig. 2. The quadratic relationships between ICT use in various contexts and academic achievement ((a)–(c)) at the student level. (Note: The two
vertical dotted lines on the left and right of the y-axis represent respectively 2.5 percentile and 97.5 percentile of the variable on the x-axis).
Fig. 3. Relationships between ICT use in various contexts and academic achievement ((a)–(c)) at the school level. (Note: (1) In 3(a) the coefficients
of HOMESCH and HOMESCH^2 are insignificant; (2) the two vertical dotted lines on the left and right of the y-axis of each graph represent
respectively 2.5 percentile and 97.5 percentile of the variable on the x-axis).
coefficient is due to the strong correlation of ENTUSE with cognitive-motivational engagement in ICT that it suppresses the effect of
other predictor variables, USESCH and HOMESCH. This effect is coined as the suppressor effect by Falk and Miller (1992).
4.3.4. ICT availability at school as a predictor of academic achievement

As shown in Table 4, RATCMP1 exerts a significant but weak negative effect on academic achievement, while RATCMP2 has no
significant effect at all. However, the correlation between RATCMP1 and the manifest variables of academic achievement: SCIE,
Fig. 4. The relationship between ICT uses outside school for schoolwork and academic achievement at the country level (Note: (1) the estimate of
the coefficient of HOMESCH^2 is insignificant; (2) the two vertical dotted lines on the left and right of the y-axis represent respectively 2.5 percentile
and 97.5 percentile of the variable on the x-axis).
9
MATH, and READ, are − 0.036, − 0.055, and − 0.051, respectively (Appendix 1), which are rather weak. Thus, the effect of RATCMP1
on academic achievement at the school level is minimal.
4.4. The country-level quadratic model
4.4.1. ICT use outside school for schoolwork as a predictor of academic achievement
The unstandardized coefficient of HOMESCH^2 is insignificant, showing that the relationship between HOMESCH and academic
achievement follows a straight line with a negative slope (Fig. 4).
4.4.2. ICT availability at school as a predictor of academic achievement

As shown in Table 4, RATCMP2 has a significant effect on academic achievement. Although the estimate of the coefficient of
RATCMP1 is insignificant, it has a strong positive correlation with SCIE, MATH, and READ, which ranges from 0.52 to 0.67 (Ap
pendix 2). The insignificance in the relationship between RATCMP1 and academic achievement is likely due to the suppressor effect
caused by RATCMP2.

Although the relationship between cognitive-motivational engagement in ICT and academic achievement is not significant, SCIE,
MATH, and READ have a positive correlation with AUTICT, COMPICT, and INTICT, which ranges from .16 to 0.54 (Appendix 2). The
standardized coefficient of cognitive-motivational engagement in ICT (0.204) (Appendix 6) which is comparable to that of RATCMP2
(0.340), showing cognitive-motivational engagement in ICT has a moderate positive effect on academic achievement. The statistical
insignificance may be due to the small sample size as the number of countries involved in the analysis is only 52.
5. Discussion and implications
5.1. The mediating role of cognitive-motivational engagement in ICT in the relationship between ICT use and academic achievement
The results of this study reveal that cognitive-motivational engagement in ICT exerts a positive mediating role in the effect of ICT
use on academic achievement at the student and school levels. ICT use outside school for schoolwork (at the student level), ICT use at
school in general (at the student level), and ICT use outside school for leisure (at the student and school levels) exert a positive direct
effect on cognitive-motivational engagement in ICT, which, in turn, exerts a positive effect on academic achievement. ICT use helps
promote students’ self-determination in technology use, thereby impacting their academic achievement. For instance, while extensive
use of ICT in browsing the Internet for schoolwork may distract students from learning, it may also provide good opportunities for
students to develop their autonomy and discernment in navigating through the vast information space, which may eventually lead to
more self-regulatory use of ICT for learning. The positive effect of ICT use on cognitive-motivational engagement in ICT is even more
prominent in the context of ICT use for leisure as shown in the results at the student and school levels (Tables A4 and A5). Even when IT
use is for leisure, such as playing video games, it could engage students in informal exploratory learning and promote problem-solving
strategies.
The relationship between cognitive-motivational engagement in ICT and academic achievement at the country level is not sig
nificant. Nonetheless, students’ perceived autonomy in ICT use, students’ perceived ICT competence, and students’ interest in ICT use
are strongly correlated to academic achievement at the country level. The insignificance in the regression analysis at the country level
may be caused by Type II errors, as the number of countries involved in the analysis is small. All in all, the results indicate students’
cognitive-motivational engagement in ICT is seemingly playing a pivotal role in promoting academic achievement.
As students’ cognitive-motivational engagement in ICT is conceptualized as their self-determination in ICT use, we argue that a
strong sense of autonomy, competence, and relatedness in ICT use is likely to promote their optimal functioning, growth, and aspi
rations in different contexts, including learning. Specifically, a higher level of cognitive-motivational engagement in ICT may enhance
students’ propensity to use ICT for problem solving and learning in and across the formal and informal contexts. These learning ex
periences may help develop students’ self-efficacy and self-regulatory capacities for learning, thereby leading to better academic
achievement. As elucidated by Skinner et al. (2008), students are predisposed to search for and succeed in environments where they
are empowered to exercise their autonomy and competence in learning. Thus, the results of the present study provide a guiding
framework for teachers to redesign their ICT pedagogies in a way that students’ ICT-related psychological needs can be better
addressed, and thereby fostering their self-determination in ICT use. To further deepen our understanding of the role of
cognitive-motivational engagement in ICT in learning, a more detailed examination of how students’ cognitive-motivational
engagement in ICT may actually affect their problem solving and self-regulation in learning is thus necessary.
5.2. The direct, non-positive effects of ICT use on academic achievement
As discussed above, the non-linear effects of ICT use on academic achievement are most prominent at the student level in which ICT
use outside school for schoolwork, ICT use at school, and ICT use outside school for leisure all follow an inverted U-shaped relationship
with academic achievement. At the school level, an inverted U-shaped relationship was also identified between ICT use at school for
schoolwork and academic achievement, and between ICT use for leisure and academic achievement. Nonetheless, by taking a close
examination of the 2.5–97.5 percentile range of ICT use at school, it is evident that academic achievement, as the dependent variable,
10
decreases monotonically as ICT use at school increases. Similar negative relationships were also found between ICT use outside school
for leisure and academic achievement at the school level, and between ICT use outside school for schoolwork and academic
achievement at the country level.
These inverted quadratic or negative relationships between ICT use and academic achievement can be interpreted in different
ways. As suggested by Rodrigues and Biagi (2017), the inverted U-shaped relationship indicates students who are close to the
bottom-end of the ICT use spectrum may be inhibited from their use. Thus, increasing their ICT use could have a beneficial effect on
their school performance. Thus, excessive, or minimal use of ICT may hinder learning while moderate use of ICT could bring the most
benefits in terms of academic achievement.
However, increasing ICT use is not likely to generate any benefits for those who are already using ICT intensively. These inverted U-
shaped relationships hint at the fact that while ICT use in different contexts could promote the development of different generic
learning skills, they can also displace activities that are conducive to learning. Extensive use of ICT has the potential to develop be
haviours such as multitasking or rapid task switching which may distract students from learning. Thus, excessive, or minimal use of ICT
may hinder learning while moderate use of ICT could bring the most benefits in terms of academic achievement.
On the other hand, the negative relationship between ICT use at school and academic achievement found at the school level may
suggest that schools, in general, lacked the capacity to unleash the full potential of technology and to change their ways of teaching.
Teachers might use technology only for a limited range of pedagogical purposes which might not lead to effective learning. More
research into effective integration of ICT in education is thus needed. Another possible explanation for the negative relationship
between ICT use at school and academic achievement is that ICT was often used for remedial purposes by low-achieving students in
schools (Zhang & Liu, 2016). Moreover, as discussed above, the effect of ICT use for leisure on academic achievement follows an
inverted U-shaped curve at the student level and is negative at the school level. In contrast, ICT use for leisure also exerts a positive
indirect effect on academic achievements via cognitive-motivational engagement in ICT. ICT use for leisure is indeed a double-edged
sword that may affect student development. Adopting either a completely prohibitive or a laissez-faire approach to ICT use for leisure
may not be in the best interest of students’ growth and development. For parents, perhaps, the best approach is perhaps to broaden the
scope of students’ leisure activities, set aside a dedicated family time at home, and develop in students the capacity to self-regulate
their ICT use.
One of the challenges in interpreting these relationships is that the PISA items for ICT use are designed for probing the quantity
rather than the quality of use. For instance, the item, ‘using computer to do schoolwork’ could lead to quite different learning out
comes, depending on teachers’ pedagogical intentions. Thus, to have a better discernment of the impact of ICT use on academic
achievement, it is necessary that the items should encompass a pedagogical dimension that probes into the effectiveness of its use.
5.3. The effect of ICT availability at school on academic achievement at the country level
At the country level, the proportion of Internet-connected computers at school available to students exerts a strong positive effect
on academic achievement, while the number of available computers per student for educational purposes also has a strong positive
correlation with science, mathematics, and reading scores. These results echo the findings derived from the work conducted by Zhang
and Liu (2016) on the five waves of PISA studies. Thus, the provision of a robust ICT infrastructure and the assurance of students’
access to online ICT resources remains pivotal to ICT implementation in schools.
6. Limitations
One of the limitations of this study is that the number of countries and economies involved in the analysis is only 52 as not all
participating countries or economies have chosen to complete the ICT Familiarity Questionnaire. This small sample size may affect the
analysis of the country-level models in a way that Type II errors are more likely to occur. Another limitation of this study is the cross-
sectional data. Hence, the findings of this study do not indicate causal effects. We employed the self-determination theory to provide
the theoretical support for the causal effects of ICT use on cognitive-motivational engagement in ICT and their effects on academic
achievement, to partially address this limitation. Guided by the theory, we tested some specific directional relationships according to
the data characteristics. For instance, as PISA data measured past ICT use and the current ICT-related cognitive motivational factors at
one time point, we tested the potential effect of ICT use on cognitive-motivational engagement in ICT rather than the other way
around. Future studies can address this limitation by conducting a longitudinal study. For instance, students who have more chances to
use ICT are more likely to develop their cognitive-motivational engagement in ICT, and these students, in turn, may be more likely to
involve in ICT activities. This hunch can be tested with a cross-lagged panel model using a longitudinal data set. The third limitation of
this study is aggregating the results at the school and the country levels. ICT strategies may differ across schools and, especially, across
countries. However, this study employed a multilevel analysis and hence the between-school and between-country differences cannot
be tested. To partially address this limitation, we included ESCS as a control variable at both the school and the country levels. Future
studies can address this limitation by testing the moderation effect of school-level and country-level factors (e.g., ESCS, access to ICT at
home).
7. Conclusion
Guided by SDT, this study tested whether and showed that cognitive-motivational engagement in ICT exerted a positive mediating
effect between ICT use and academic achievement at the student and school levels. Theoretically, this study deepened our
11
understanding of SDT that satisfaction of students’ basic psychological needs in a specific area such as ICT is likely to promote their
optimal functioning in other areas such as learning. According to SDT, the satisfaction of ICT-related psychological needs may have a
positive effect on ICT-related behaviours. However, our study suggests that this positive effect may go beyond the situations or settings
related to ICT. Students who have a strong sense of autonomy, competence, and relatedness in ICT use may also be likely to involve
activities for optimal functions and growth in a non-ICT situation, such as learning. A higher level of cognitive-motivational
engagement in ICT may lead to more self-regulatory use of ICT for learning, thereby leading to better learning outcomes.
Methodologically, this study shows that a quadratic term should be modeled when examining the relationship between ICT use and
academic achievement. While some past studies showed ICT use is non-positively associated with students’ academic achievement,
recent studies with limited samples show a non-linear relationship between students’ ICT use and their academic achievement. Based
on a sample from 52 countries, this study shows excessive, or minimal use of ICT may hinder learning, while moderate use of ICT could
bring the most benefits to students in terms of academic achievement. At the school level, students’ academic achievement decreases
monotonically as ICT use at school and ICT use outside school for leisure increase. At the country level, the proportion of Internet-
connected school computers available to students exerted a strong positive direct effect on academic achievement, indicating build
ing a robust ICT infrastructure and ensuring students’ access to online ICT resources at schools are conducive to promoting students’
performance.
Practically, the finding of the positive indirect effects of ICT use on academic achievement suggests that providing students with
more opportunities to orchestrate technologies in formal and informal contexts helps develop their self-determination, discernment,
and self-regulation in ICT use, and thereby creating the conditions in which better learning outcomes are more likely to occur. The
finding also provides a guiding framework for teachers to design pedagogies through which students’ ICT-related psychological needs
can be better addressed.
Credit author statement
Sandy C. Li: Conceptualization, Formal analysis, Writing – original draft. Jinxin Zhu: Methodology, Writing – review & editing.
Declaration of competing interest
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The
authors have no financial/personal interest or personal relationships with other people or organizations that could affect our objec
tivity or inappropriately influence our work.
Data availability
Data will be made available on request.
Appendix 1
Table A1
Correlation between manifest variables of cognitive-motivational engagement in ICT, ICT use in different contexts, ICT availability at school, and the
manifest variables of academic achievement at the school level
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11)
AUTICT (1) 1
INTICT (2) .846 1
COMPICT (3) .911 .935 1
HOMESCH (4) .433 .369 .431 1
USESCH (5) .207 .142 .199 .581 1
ENTUSE (6) .792 .767 .818 .531 .311 1
RATCMP1 (7) .036 .005 .025 .037 .077 .020 1
RATCMP2 (8) .068 .131 .126 .082 .044 .092 .493 1
SCIE (9) .402 .449 .418 .045 − .138 .126 ¡.036 .128 1
MATH (10) .394 .428 .403 .062 − .105 .125 ¡.055 .115 .955 1
READ (11) .398 .479 .430 .039 − .156 .130 ¡.051 .133 .970 .944 1
12
Appendix 2
Table A2
Correlation between ICT use in different contexts, ICT availability at school, and the manifest variables of academic achievement at the country level
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11)
RATCMP1 (1) 1
RATCMP2 (2) .493 1
HOMESCH (3) − .088 − .090 1
HOMESCH^2 (4) − .025 .025 − .293 1
USESCH (5) .251 .109 .683 .-255 1
USESCH^2 (6) .014 .089 - .389 .825 − .343 1
ENTUSE (7) .185 .592 .363 − .255 .293 − .091 1
ENTUSE^2 (8) − .366 − .770 .018 .072 − .202 .031 − .534 1
SCIE (9) .565 .667 − .279 .098 - .032 .155 .395 - .499 1
MATH (10) .554 .645 - .230 .101 - .016 .145 .457 - .498 .959 1
READ (11) .520 .668 − .243 .006 .012 .060 .376 − .512 .974 .924 1
Appendix 3
Table A3
Correlation between manifest variables of cognitive-motivational engagement in ICT and academic achievement at the country level
(1) (2) (3) (4) (5) (6)
AUTICT (1) 1
COMPICT (2) .477 1
INTICT (3) .654 .664 1
SCIE (4) .522 .225 .426 1
MATH (5) .538 .158 .353 .959 1
READ (6) .537 .334 .472 .973 .922 1
Appendix 4
Table A4
Standardized student-level estimates of the quadratic, linear, and baseline models
Variable Quadratic Linear Baseline
Measurement part
cognitive-motivational engagement in ICT →AUTICT 0.755*** 0.755*** 0.746***
→COMPICT 0.614*** 0.614*** 0.599***
→INTICT 0.812*** 0.812*** 0.841***
academic achievement →SCIE 0.901*** 0.902*** 0.902***
→MATH 0.812*** 0.812*** 0.813***
→READ 0.893*** 0.891*** 0.890***
Regression part
cognitive-motivational engagement in ICT →academic achievement 0.191*** 0.192*** /
HOMESCH − 0.074*** − 0.096*** /
HOMESCH^2 − 0.07*** / /
USESCH − 0.143*** − 0.150*** /
USESCH^2 − 0.042*** / /
ENTUSE 0.043*** 0.010 /
ENTUSE^2 − 0.089*** / /
RATCMP1 / / /
RATCMP2 / / /
ESCS 0.173*** 0.177*** 0.181***
HOMESCH →cognitive-motivational engagement in ICT 0.070*** 0.070*** /
USESCH 0.065*** 0.065*** /
ENTUSE 0.344*** 0.344*** /
RATCMP1 / / /
RATCMP2 / / /
ESCS 0.068*** 0.068*** 0.106***
GENDER 0.099*** 0.099*** 0.146***
(continued on next page)
13
Table A4 (continued )
GENDER →SCIE 0.014* 0.004 0.021**

GENDER →MATH 0.059*** 0.049*** 0.064***
GENDER →READ − 0.138*** − 0.148*** − 0.132***
Appendix 5
Table A5
Standardized school-level estimates of the quadratic, linear, and baseline models.
Measurement part
→COMPICT 0.912*** 0.914*** 0.863***
→INTICT 0.990*** 0.989*** 0.998***
→MATH 0.967*** 0.968*** 0.966***
→READ 0.974*** 0.972*** 0.966***
Regression part
HOMESCH − 0.022 − 0.044 /
HOMESCH^2 − 0.037 / /
USESCH − 0.137*** − 0.133*** /
USESCH^2 − 0.074** / /
ENTUSE − 0.440*** − 0.470*** /
ENTUSE^2 − 0.065** / /
RATCMP1 − 0.033** − 0.033** /
RATCMP2 0.019 0.020 /
ESCS 0.570*** 0.571*** 0.765***
HOMESCH →cognitive-motivational engagement in ICT − 0.010 − 0.010 /
USESCH − 0.066* − 0.066* /
ENTUSE 0.832*** 0.832*** /
RATCMP1 0.007 0.007 /
RATCMP2 0.007 0.006 /
ESCS 0.328*** 0.329*** 0.615***
GENDER − 0.053 − 0.054 0.035
GENDER →SCIE − 0.022 − 0.039 − 0.096***
GENDER →MATH 0.012 − 0.004 − 0.059*
GENDER →READ − 0.132*** − 0.145*** − 0.203***
Appendix 6
Table A6
Standardized country-level estimates of the quadratic, linear, and baseline models
Measurement part
→COMPICT 0.873*** 0.876*** 0.899***
→INTICT 0.704*** 0.703*** 0.718***
→MATH 0.936*** 0.928*** 0.926***
→READ 0.920*** 0.912*** 0.897***
Regression part
cognitive-motivational engagement in ICT →academic achievement 0.204 0.173 /
HOMESCH − 0.183* − 0.224*** /
HOMESCH^2 0.089 – /
USESCH 0.146 0.166 /
USESCH^2 0.340* 0.342* /
ENTUSE 0.410** 0.421** 0.741***
ENTUSE^2 0.009 0.006 /
14
RATCMP1 0.155 0.154 /

RATCMP2 0.138 0.139 /
ESCS 0.321 0.319 0.418***
HOMESCH →cognitive-motivational engagement in ICT 0.009 0.006 /
USESCH / / /
ENTUSE / / /
RATCMP1 0.155 0.154 /
RATCMP2 0.138 0.139 /
ESCS 0.321 0.319 0.418***
GENDER − 0.099 − 0.097 − 0.23
GENDER →SCIE − 0.265 − 0.31* − 0.392**
GENDER →MATH − 0.142 − 0.195 − 0.272
GENDER →READ − 0.265* − 0.302** − 0.380***
Appendix 7
Table A7
Unstandardized student-level estimates of the quadratic, linear, and baseline models
Measurement part
→COMPICT 0.836*** 0.837*** 0.824***
→INTICT 1.064*** 1.064*** 1.116***
→MATH 0.888*** 0.887*** 0.887***
→READ 0.986*** 0.985*** 0.982***
Regression part
HOMESCH − 0.054*** − 0.070*** /
HOMESCH^2 − 0.025*** / /
USESCH − 0.109*** − 0.115*** /
USESCH^2 − 0.018*** / /
ENTUSE 0.029*** 0.007 /
ENTUSE^2 − 0.020*** / /
RATCMP1 / / /
RATCMP2 / / /
ESCS 0.150*** 0.155*** 0.158***
HOMESCH →cognitive-motivational engagement in ICT 0.053*** 0.053*** /
USESCH 0.052*** 0.052*** /
ENTUSE 0.238*** 0.238*** /
RATCMP1 / / /
RATCMP2 / / /
ESCS 0.062*** 0.062*** 0.095***
GENDER 0.152*** 0.152*** 0.225***
GENDER →SCIE 0.024* 0.006 0.034**
GENDER →MATH 0.095*** 0.080*** 0.104***
GENDER →READ − 0.226*** − 0.244*** − 0.216***
Appendix 8
Table A8
Unstandardized school-level estimates of the quadratic, linear, and baseline models
Measurement part
→COMPICT 1.119*** 1.122*** 1.152***
→INTICT 1.061*** 1.061*** 1.114***
15
→MATH 0.996*** 0.994*** 0.995***

→READ 1.001*** 1.001*** 1.000***
Regression part
HOMESCH − 0.023 − 0.047 /
HOMESCH^2 − 0.031 / /
USESCH − 0.139*** − 0.140*** /
USESCH^2 − 0.080** / /
ENTUSE − 0.468*** − 0.515*** /
ENTUSE^2 − 0.048** / /
RATCMP1 − 0.016** − 0.017** /
RATCMP2 0.048 0.053 /
ESCS 0.513*** 0.529*** 0.696***
HOMESCH →cognitive-motivational engagement in ICT − 0.004 − 0.004 /
USESCH − 0.024* − 0.024* /
ENTUSE 0.316*** 0.316*** /
RATCMP1 0.001 0.001 /
RATCMP2 0.006 0.006 /
ESCS 0.105*** 0.105*** 0.151***
GENDER − 0.054 − 0.055 0.027
GENDER →SCIE − 0.062 − 0.116 − 0.283**
GENDER →MATH 0.036 − 0.013 − 0.176*
GENDER →READ − 0.386*** − 0.440*** − 0.607***
Appendix 9
Table A9
Unstandardized country-level estimates of the quadratic, linear, and baseline models
Measurement part
→COMPICT 1.552*** 1.558*** 1.645***
→INTICT 1.099*** 1.097*** 1.138***
→MATH 1.086*** 1.079*** 1.100***
→READ 0.943*** 0.947*** 0.951***
Regression part
cognitive-motivational engagement in ICT →academic achievement 0.671 0.581 /
HOMESCH − 0.243** − 0.303*** /
HOMESCH^2 0.179 / /
USESCH 0.146 0.170 /
USESCH^2 1.000* 1.026* /
ENTUSE 0.290** 0.303*** 0.594***
ENTUSE^2 0.004 0.002 /
RATCMP1 1 1 1
RATCMP2 1.552*** 1.558*** 1.645***
ESCS 1.099*** 1.097*** 1.138***
HOMESCH →cognitive-motivational engagement in ICT 1 1 1
USESCH / / /
ENTUSE / / /
RATCMP1 0.047 0.047 /
RATCMP2 0.123 0.124 /
ESCS 0.069 0.069 0.090**
GENDER − 0.643 − 0.627 − 1.499
GENDER →SCIE − 5.919 − 7.088* − 10.318*
GENDER →MATH − 3.537 − 4.893 − 7.844
GENDER →READ − 5.818* − 6.781** − 9.751**
16
References
Avvisati, F. (2020). The measure of socio-economic status in PISA: A review and some suggested improvements. Large-scale Assess Educ, 8(1), 1–37. https://doi.org/
10.1186/s40536-020-00086-x
Bhutoria, A., & Aljabri, N. (2022). Patterns of cognitive returns to information and communication technology (ICT) use of 15-year-olds: Global evidence from a
hierarchical linear modeling approach using PISA 2018. Computers & Education, 181, Article 104447. https://doi.org/10.1016/j.compedu.2022.104447
Burns, E. C., Martin, A. J., & Collie, R. J. (2020). Supporting and thwarting interpersonal dynamics and student achievement: A multi-level examination of PISA 2015.
International Journal of Research and Method in Education, 1–15. https://doi.org/10.1080/1743727X.2020.1757639
Chen, P. Y., & Krauss, A. D. (2014). Suppression effect. In M. S. Lewis-Beck, A. Bryman, & T. F. Liao (Eds.), The SAGE encyclopedia of social science research methods.
SAGE Publications. https://doi.org/10.4135/9781412950589.n989.
Chiao, C., & Chiu, C.-H. (2018). The mediating effect of ICT usage on the relationship between students’ socioeconomic status and achievement. The Asia-Pacific
Education Researcher, 27(2), 109–121. https://doi.org/10.1007/s40299-018-0370-9
Chiu, T. K. F. (2022). Applying the self-determination theory (SDT) to explain student engagement in online learning during the COVID-19 pandemic. Journal of
Research on Technology in Education, 54(sup1), S14–S30. https://doi.org/10.1080/15391523.2021.1891998
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
Conesa, P. J., Onandia-Hinchado, I., Duñabeitia, J. A., & Moreno, M.Á. (2022). Basic psychological needs in the classroom: A literature review in elementary and
middle school students. Learning and Motivation, 79, Article 101819. https://doi.org/10.1016/j.lmot.2022.101819
Courtney, M., Karakus, M., Ersozlu, Z., & Nurumov, K. (2022). The influence of ICT use and related attitudes on students’ math and science performance: Multilevel
analyses of the last decade’s PISA surveys. Large-scale Assessments in Education, 10(1), 8. https://doi.org/10.1186/s40536-022-00128-6
Deci, E. L. (1971). Effects of externally mediated rewards on intrinsic motivation. Journal of Personality and Social Psychology, 18(1), 105–115. https://doi.org/
10.1037/h0030644
Deci, E. L. (1985). Intrinsic motivation and self-determination in human behavior. Plenum Press.
Falk, R., & Miller, N. (1992). A primer for soft modeling. Akron, OH: The University of Akron Press.
Gómez-Fernández, N., & Mediavilla, M. (2021). Exploring the relationship between information and communication technologies (ICT) and academic performance: A
multilevel analysis for Spain. Socio-Economic Planning Sciences, 77, Article 101009. https://doi.org/10.1016/j.seps.2021.101009
Gubbels, J., Swart, N. M., & Groen, M. A. (2020). Everything in moderation: ICT and reading performance of Dutch 15-year-olds. Large-scale Assessments in Education, 8
(1), 1–17. https://doi.org/10.1186/s40536-020-0079-0
Hu, X., Gong, Y., Lai, C., & Leung, F. K. S. (2018). The relationship between ICT and student literacy in mathematics, reading, and science across 44 countries: A
multilevel analysis. Computers & Education, 125, 1–13. https://doi.org/10.1016/j.compedu.2018.05.021
Jeno, L. M., Adachi, P. J. C., Grytnes, J.-A., Vandvik, V., & Deci, E. L. (2019). The effects of m-learning on motivation, achievement and well-being: A self-
determination theory approach. British Journal of Educational Technology, 50(2), 669–683. https://doi.org/10.1111/bjet.12657
Jeno, L. M., Danielsen, A. G., & Raaheim, A. (2018). A prospective investigation of students’ academic achievement and dropout in higher education: A self-
determination theory approach. Educational Psychology, 38(9), 1163–1184. https://doi.org/10.1080/01443410.2018.1502412
Jeno, L. M., Grytnes, J.-A., & Vandvik, V. (2017). The effect of a mobile-application tool on biology students’ motivation and achievement in species identification: A
self-determination theory perspective. Computers & Education, 107, 1–12. https://doi.org/10.1016/j.compedu.2016.12.011
Kong, Y., Seo, Y. S., & Zhai, L. (2022). ICT and digital reading achievement: A cross-national comparison using PISA 2018 data. International Journal of Educational
Research, 111, Article 101912. https://doi.org/10.1016/j.ijer.2021.101912
Kunina-Habenicht, O., & Goldhammer, F. (2020). ICT engagement: A new construct and its assessment in PISA 2015. Large-scale Assessments in Education, 8(1), 1–21.
https://doi.org/10.1186/s40536-020-00084-z
Legrain, P., Gillet, N., Gernigon, C., & Lafreniere, M.-A. (2015). Integration of information and communication technology and pupils’ motivation in a physical
education setting. Journal of Teaching in Physical Education, 34(3), 384–401. https://doi.org/10.1123/jtpe.2014-0013
Li, S. C., & Petersen, K. B. (2022). Does ICT Matter? Unfolding the Complex Multilevel Structural Relationship between Technology Use and Academic Achievements
in PISA 2015. Educational Technology & Society, 25(4), 43–55. https://drive.google.com/file/d/1RTUkHdEJYugJKQc_C1moWdApPoicxzW9/view.
Looi, C.-K., Zhang, H., Gao, Y., & Wu, L. (2020). ICT in education and implications for the belt and road initiative. Singapore: Springer Singapore Pte. Limited.
Malian, I., & Nevin, A. (2002). A Review of self-determination literature: Implications for practitioners. Remedial and Special Education, 23(2), 68–74. https://doi.org/
10.1177/074193250202300202, 10.1016/j.chb.2015.07.008.
Manganelli, L., Thibault-Landry, A., Forest, J., & Carpentier, J. (2018). Self-determination theory can help you generate performance and well-being in the workplace:
A review of the literature. Advances in Developing Human Resources, 20(2), 227–240. https://doi.org/10.1177/1523422318757210
Ma, Y., & Qin, X. (2021). Measurement invariance of information, communication and technology (ICT) engagement and its relationship with student academic
literacy: Evidence from PISA 2018. Studies In Educational Evaluation, 68, Article 100982. https://doi.org/10.1016/j.stueduc.2021.100982
Mehrvarz, M., Heidari, E., Farrokhnia, M., & Noroozi, O. (2021). The mediating role of digital informal learning in the relationship between students’ digital
competence and their academic performance. Computers & Education, 167, Article 104184. https://doi.org/10.1016/j.compedu.2021.104184
Meng, L., Qiu, C., & Boyd-Wilson, B. (2019). Measurement invariance of the ICT engagement construct and its association with students’ performance in China and
Germany: Evidence from PISA 2015 data. British Journal of Educational Technology, 50(6), 3233–3251. https://doi.org/10.1111/bjet.12729
Navarro-Martinez, O., & Peña-Acuña, B. (2022). Technology usage and academic performance in the Pisa 2018 report. Journal of New Approaches in Educational
Research, 11(1), 130–145. https://doi.org/10.7821/naer.2022.1.735
Odell, B., Cutumisu, M., & Gierl, M. (2020). A scoping review of the relationship between students’ ICT and performance in mathematics and science in the PISA data.
Social Psychology of Education, 23(6), 1449–1481. https://doi.org/10.1007/s11218-020-09591-x
OECD. (2015). The ABC of gender equality in education. PISA. OECD Publishing. https://doi.org/10.1787/9789264229945-en
OECD. (2019a). PISA 2018 assessment and analytical framework. PISA, OECD Publishing. https://doi.org/10.1787/b25efab8-en
OECD. (2019b). PISA 2018 results (Volume II). PISA: OECD Publishing. https://doi.org/10.1787/b5fd1b8f-en
OECD. (2019c). Scaling outcomes. In PISA 2018 technical report. PISA, OECD Publishing. https://www.oecd.org/pisa/data/pisa2018technicalreport/PISA2018-
TecReport-Ch-12-Scaling-Outcomes.pdf.
OECD. (2020). PISA 2018 results, ume V. PISA, OECD Publishing. https://doi.org/10.1787/ca768d40-en
Park, S., & Weng, W. (2020). The relationship between ICT-related factors and student academic achievement and the moderating effect of country economic indexes
across 39 countries: Using multilevel structural equation modelling. Educational Technology & Society, 23(3), 1–15. https://drive.google.com/file/d/1-
6OxvF1EiTJr51lWiMfDDpXl-Exnh0CT/view?usp=sharing.
Reyes, V. C. (2021). Clipped wings: School leaders’ identities in ICT-enriched education landscapes – a narrative inquiry. Educational Management Administration &
Leadership, 49(5), 807–823. https://doi.org/10.1177/1741143220918257
Rodrigues, M., & Biagi, F. (2017). Digital technologies and learning outcomes of students from low socio-economic background: An Analysis of PISA 2015. J. R. C. European
Commission. https://doi.org/10.2760/415251
Rohatgi, A., Scherer, R., & Hatlevik, O. E. (2016). The role of ICT self-efficacy for students’ ICT use and their achievement in a computer and information literacy test.
Computers & Education, 102, 103–116. https://doi.org/10.1016/j.compedu.2016.08.001
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55
(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68
Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. New York: Guilford Publications.
Skinner, E., Furrer, C., Marchand, G., & Kindermann, T. (2008). Engagement and disaffection in the classroom: Part of a larger motivational dynamic? Journal of
Educational Psychology, 100(4), 765–781. https://doi.org/10.1037/a0012840
17
Ünal, E., Uzun, A. M., & Kilis, S. (2022). Does ICT involvement really matter? An investigation of Turkey’s case in PISA 2018. Education and Information Technologies,
27(8), 11443–11465. https://doi.org/10.1007/s10639-022-11067-8
Yang, D., Lavonen, J. M., & Niemi, H. (2018). Online learning engagement: Critical factors and research evidence from literature. Themes in eLearning, 11(1), 1–18.
http://earthlab.uoi.gr/tel/index.php/themeselearn/article/view/5/1.
Zhang, D., & Liu, L. (2016). How does ICT Use influence students’ achievements in math and science over time? Evidence from PISA 2000 to 2012. Eurasia Journal of
Mathematics, Science and Technology Education, 12(9), 2431–2449. https://doi.org/10.12973/eurasia.2016.1297a
Zhou, J., Huebner, E. S., & Tian, L. (2021). The reciprocal relations among basic psychological need satisfaction at school, positivity and academic achievement in
Chinese early adolescents. Learning and Instruction, 71, Article 101370. https://doi.org/10.1016/j.learninstruc.2020.101370
Zhu, J., & Chiu, M. M. (2019). Early home numeracy activities and later mathematics achievement: Early numeracy, interest, and self-efficacy as mediators.
Educational Studies in Mathematics, 102(2), 173–191. https://doi.org/10.1007/s10649-019-09906-6
Zhu, J., & Li, S. C. (2022). The non-linear relationships between ICT use and academic achievement of secondary students in Hong Kong. Computers & Education, 187,
Article 104546. https://doi.org/10.1016/j.compedu.2022.104546
Zhu, J., & Mok, M. M. C. (2020). Predictors of students’ participation in internet or computer tutoring for additional instruction and its effect on academic
achievement. Journal of Computer Assisted Learning, 36(5), 729–740. https://doi.org/10.1111/jcal.12440
Zylka, J., Christoph, G., Kroehne, U., Hartig, J., & Goldhammer, F. (2015). Moving beyond cognitive elements of ICT literacy: First evidence on the structure of ICT
engagement. Computers in Human Behavior, 53, 149–160. https://doi.org/10.1016/j.chb.2015.07.008
18

Cognitive Motivational Engagement in ICT Mediates The Effect of ICT Use On Academic Achievements

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cognitive Motivational Engagement in ICT Mediates The Effect of ICT Use On Academic Achievements

Uploaded by

Copyright:

Available Formats

Computers & Education 204 (2023) 104871

Contents lists available at ScienceDirect

Computers & Education

Cognitive-motivational engagement in ICT mediates the effect of

2. Conceptual framework and literature review

2.1. Self-determination theory in learning settings

2.2. Cognitive-motivational engagement in ICT and academic achievement

2.3. ICT use, cognitive-motivational engagement in ICT, and academic achievement

2.4. The potential non-linear effect of ICT use on academic achievement

2.5. ICT availability at school and academic achievement

2.6. Socioeconomic status and academic achievement

2.7. Gender and academic achievement

2.8. The present study

3.1. Data source and participants

3.2.1. Students’ academic achievement

3.2.3. Cognitive-motivational engagement in ICT

3.2.4. ICT availability at school (RATCMP1 & RATCMP2)

3.2.5. Socioeconomic status (ESCS) and gender (GENDER)

3.3. Multilevel structural equation modelling (MSEM)

3.3.1. The quadratic model

3.3.2. The linear model

3.3.3. The baseline model

4.1. Model comparison

4.2. The student-level quadratic model

4.2.1. ICT use in different contexts as predictors of academic achievement

4.2.2. Cognitive-motivational engagement in ICT as a predictor of academic achievement

4.2.3. ICT use in different contexts as predictors of cognitive-motivational engagement in ICT

4.3. The school-level quadratic model

4.3.1. ICT use in different contexts as predictors of academic achievement

Quadratic 6642.208 (160) .957 .938 .012 .036 3555205.5 3556236.7

Quadratic Baseline 11705.0930 32 .0000

Student-level School-level Country-level

4.3.2. Cognitive-motivational engagement in ICT as a predictor of academic achievement

4.3.3. ICT use in different contexts as predictors of cognitive-motivational engagement in ICT

4.3.4. ICT availability at school as a predictor of academic achievement

4.4. The country-level quadratic model

4.4.2. ICT availability at school as a predictor of academic achievement

4.4.3. Cognitive-motivational engagement in ICT as a predictor of academic achievement

5. Discussion and implications

5.2. The direct, non-positive effects of ICT use on academic achievement

Credit author statement

Declaration of competing interest

Data will be made available on request.

(1) (2) (3) (4) (5) (6)

Variable Quadratic Linear Baseline

GENDER →SCIE 0.014* 0.004 0.021**

Variable Quadratic Linear Baseline

Variable Quadratic Linear Baseline

RATCMP1 0.155 0.154 /

Variable Quadratic Linear Baseline

Variable Quadratic Linear Baseline

→MATH 0.996*** 0.994*** 0.995***

Variable Quadratic Linear Baseline

You might also like

→MATH 0.996* 0.994* 0.995***