Computers in Human Behavior 60 (2016) 198e211

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier.com/locate/comphumbeh

Modeling the continuance usage intention of online learning

environments
han*, Buket Akkoyunlu
€ khan Dag
Go
Hacettepe University, Faculty of Education, Department of Computer Education and Instructional Technology, 06800, Beytepe, Ankara, Turkey

a r t i c l e i n f o a b s t r a c t

Article history: This study focused on the continuance usage intention toward online learning environments and also
Received 3 October 2015 proposed and empirically tested an integrated model to better understand the determinants of students’
Received in revised form continuance intention to use these environments. For this purpose, variables which may have an effect
14 February 2016
on the continuance usage intention were examined and an “online continuance usage intention model”
Accepted 16 February 2016
was revealed. When the variables of the hypothetical model were determined, Technology Continuance
Available online xxx
Theory, Information Systems Success Model, Cognitive Model and Information Systems Expectation
Confirmation Model have been practiced on. Empirical data from 467 public university students who had
Keywords:
Continuance usage
used an online learning environment for the first time were tested against the proposed research model
Information systems by using path analysis. The results indicated that, confirmation of the usage of online learning envi-
Information quality ronments could be explained by information quality, system quality and service quality variables. 63% of
Online learning environments the variance of the satisfaction variable was explained by information quality, system quality, service
Service quality quality, confirmation, utilitarian value, outcome expectations and perceived value. Research results
System quality confirmed the propounded constructs of Information Systems Success Model and Information Systems
Expectation Confirmation Model. In line with the obtained findings and results, some of the various
suggestions were provided for the next studies and implementations.
© 2016 Elsevier Ltd. All rights reserved.

1. Introduction
Continuous expansion of online learning, with the help of unique
and interesting learning platform designs and activities, has led to
enhancements in student learning processes. Interaction and
communication patterns within these environments are different
from face-to-face learning environments (Symeonides & Childs,
2015); students learn in a diverse environment through varied
modes of participation. On the other hand, various types of
communication styles, such as studenteinstructor communication,
studentestudent communication and student-communities’
communication, are considered about these learning environments
(Hung & Chou, 2015; Sharma, Joshi, & Sharma, 2016) and learners
have discussions with teachers/instructors and other learners
through the use of synchronous and asynchronous communication
€kçearslan & Alper, 2015; Shukor, Tasira, & Van der Meijden,
tools (Go
2015). To understand online behaviour, researchers are concentrated
* Corresponding author.
E-mail addresses: gokhandaghan@gmail.com, gokhand@hacettepe.edu.tr
han), buket@hacettepe.edu.tr (B. Akkoyunlu).
(G. Dag
on conventional media behaviour and also new aspects which are
applicable to virtual environments (Esteban-Millat, Martínez-Lo  pez,
Huertas-García, Meseguer, & Rodríguez-Ardura, 2014). There are
various studies in the literature of this area which emphasise that
active student participation in effectively designed online courses is
important for the success of these learning environments (Bourelle,
Bourelle, Knutson, & Spong, 2016; Harasim, Hiltz, Teles, & Turoff,
1995; Hranstinski, 2009; Mandernach, Gonzales, & Garnett, 2006;
Masters & Oberprieler, 2004; McKavanagh, Kanes, Beven,
Cunningham, & Choy, 2002; Sutton, 2001). There are also a number
of studies on the effectiveness of synchronous or asynchronous tools
used in environments where adequate participation and interaction
are ensured (Cheng, Pare , Collimore, & Joordens, 2011; Shana, 2009).
However, in order to ensure effective participation in online learning
environments, students must spend adequate time in the environ-
ment, participate actively and interact with both the teacher and the
other students. You and Kang (2014) emphasized that unsuccessful
online learners do not allocate enough time and effort within these
learning environments. To enhance online learning, it is necessary to
enhance online learner participation in online learning environ-
€
ments (Hranstinski, 2009). Oncü and Çakır (2011) mentioned that
improving cooperation and communication among online learners

http://dx.doi.org/10.1016/j.chb.2016.02.066
0747-5632/© 2016 Elsevier Ltd. All rights reserved.
 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211
was one of the indicators of successful online learning environments.
Furthermore, limitedness of learner participation in online learning
environments and failure to obtain expected benefits from the uti-
lisation of such environments have become frequently experienced
problems (Hew, Cheung, & Ng, 2009; Hewitt, 2005). In the study
conducted by Shaw in 2012, students’ participation in online learning
environments was classified as replies with solutions, asks questions,
browses other’s solutions and questions and no action. No action stu-
dents did not interact with others in the environment, while browses
other’s solutions and questions students could effectively use such
environments while remaining passive listeners. In other words,
these environments are made use of and adopted for a certain period
of time, however there is no continuance or sustainability in usage
behaviour. Bento and Schuster (2003) classified student participation
in online learning environments into four groups: active learners,
social participants, missing in action and witness learners. Pala and
Erdem (2011) talked about missing in action as “students who
communicate poorly and rarely interact with the content”, while
stating that witness learners were similar to the missing in action
students but with the single difference of being good readers. No
matter what kind of classifications are made, students with low levels
of participation should be encouraged for continuance and students
should be motivated to make use of these environments. In summary,
ensuring continuance usage is a variable that is as important as the
participation variable and should therefore be considered for further
research. Furthermore, Seddon (1997) and Lee (2010) suggested that
ensuring continuance usage would lead to achievement.
Studies on acceptance, adoption and usage of technological
developments start from the point where technology is introduced
as an innovation and try to analyse its usage and adoption during
this time. However, none of the theories or models in the accep-
tance and adoption literature could predict the continuance in
diverse learning environments. Moreover, the variables that these
theories were based on also varied. Although continuance has been
explained in certain studies as an extension of the adoption process
(Jasperson, Carter, & Zmud, 2005; Karahanna, Straub, & Chervany,
1999; Venkatesh & Davis, 2000), this type of thinking has been
widely criticised by recent theories (Bhattacherjee, 2001) and it has
been suggested that long-term usage should be predicted and
analysed through different constructs. In light of the opinion that a
precise continuance may not be obtained in usage of an accepted
technology, Bhattacherjee (2001) and Lee (2010) named this situ-
ation as “acceptance-discontinuance anomaly” and considered it as
a phenomenon that should be investigated.
Studying whether the various dynamics considered in online
learning environments design are reflected in the usage of the
environment is quite essential in terms of the proportion of time
and efforts spent over the results. While there are variables
affecting the adoption and usage of a technology, there may also be
certain variables which affect its long-term usage. Identification of
these variables and testing of a potential model built upon them
would fill a gap in the literature of a model or theory for predicting
the continuance usage of online learning environments and for
designing successful online learning environments. It is clear that
these environments can provide expected benefits, but only when
continuance usage is ensured. Furthermore, the primary condition
for ensuring effective learner participation in online learning en-
vironments is continuance usage. With respect to varied levels of
interaction and participation, ensuring continuance in students’
usage of online learning environments is considered to be essential
in terms of maintaining the purposefulness of learning experiences
within these environments. Therefore, in the light of the literature
and the complex relationship patterns among these variables, it is
believed that determining those variables that could affect
continuance usage will contribute to more effective online learning
199
environment design and to a certain extent, benefit teachers and
students. In this respect, the purpose of this study is to identify the
variables that could affect the continuance usage intention of on-
line learning environments and present an “online continuance
usage intention model”.
1.1. Theoretical framework
1.1.1. Continuance
Certain diffusion and acceptance researchers such as Davis
(1989), Venkatesh, Morris, Davis, and Davis (2003) have conduct-
ed studies and developed theories about the acceptance, adoption
and utilisation of technological innovations for many years. How-
ever, none of these theories and models explain continuance usage.
Some of the studies in the literature (Doherty, 2011; Handal,
Cavanagh, Wood, & Petocz, 2011; Kukulska-Hulme, 2012; Kurt,
2012; Lee, Yoon, & Lee, 2009; Vanderlinde & van Braak, 2011; Wang
& Wang, 2009) examined students’ and teachers’ adoption and
acceptance of these environments, however they did not interpret
the continuance or come to conclusions about the longitudinal
effect of time. Furthermore, the variables on which these theories
were based are quite different from the variables considered within
the context of recent theories regarding continuance.
Some recent studies have found that in varied learning envi-
ronments the main focus should be on the continuance usage
behaviour rather than short-term usage (Bhattacherjee, 2001;
Bhattacherjee, Perols, & Sanford, 2008; Ifinedo, 2006; Jasperson
et al., 2005; Terzis, Moridis, & Economides, 2013; Thong, Hong, &
Tam, 2006). Studies have also shown that interiorised usage could
not be predicted by studies using concepts such as short-term us-
age, acceptance and adoption (Jasperson et al., 2005; Shih, 2008);
and therefore, studies on ensuring continuance usage have gained
importance. In the light of this gap in the literature, varied and
enhanced theories have been developed with respect to the pre-
diction of continuance usage of technological innovations. Within
these theories, the tracking of the usage process does not cease
upon understanding usage behaviour and the continuance usage of
the technology could be predicted.
Limayem and Cheung (2008), in their study on the continuance
usage of internet-based learning technologies, integrated a habit
variable into Bhattacherjee’s (2001) Information Systems Contin-
uance Model and tested this extended theory with 303 university
students. The study concluded that habit, continuance intention,
satisfaction and prior behaviour were variables that could affect
continuance usage of internet-based learning technologies and the
continuance usage variance was predicted at the rate of 23%.
In a study on the continuance of virtual communities (Jin,
Cheung, Lee, & Chen, 2007), a research model was constructed
based on the Information Systems Continuance Model and Infor-
mation Adoption Model and tested with 240 students enrolled at a
state university in China. According to the findings of the study,
satisfaction and information usefulness were the two major vari-
ables that could affect continuance intention. It was observed that
while the satisfaction variable could be predicted with the infor-
mation usefulness and source credibility variables, the information
usefulness variable could be predicted with the information quality
and source credibility variables.
In a study where continuance participation behaviour in Face-
book was analysed within a social and behavioural approach (Al-
Debei, Al-Lozi, & Papazafeiropoulou, 2013), it was found that
continuance participation intention was directly effective on
continuance participation behaviour, while attitude and subjective
norms were indirectly effective. It was also found that perceived
behavioural control had both a direct effect on continuance
participation behaviour and an indirect effect on continuance
200 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211
participation intention. In another study on continuance intention
towards Twitter (Liu, Cheung, & Lee, 2010), technology gratification
and content gratification were found to have significant effects on
user satisfaction and user satisfaction was found to have a signifi-
cant effect on continuance intention.
As studies on diverse technologies or learning environments
have illustrated, continuance variables that were used with varied
theories have changed and this change has resulted in diverse
conclusions with respect to continuance variables. However, look-
ing at the recent studies with a holistic perspective, it has been
observed that constructs which affect the long-term utilisation of
technological innovations are as important as adoption. Another
interesting point has been the variety of the target variables, which
there have been attempts to predict. The continuance intention
variable was the topic of certain studies (Barnes, 2011; Chang, 2013;
Guo, Xiao, Van Toorn, Lai, & Seo, 2016; Harden, Ryan, & Prybutok,
2012; Jin, Lee, & Cheung, 2010; Kang & Lee, 2010; Lin, 2011; Lin,
Wu, & Tsai, 2005; Venkatesh, Thong, Chan, Hu, & Brown, 2011),
while continuance usage was studied in some others (Zhou, 2011)
and both variables were studied together in further studies
(Bhattacherjee et al., 2008; Limayem & Cheung, 2008, 2011;
Limayem, Hirt, & Cheung, 2007). In addition, the concept of sus-
tainability has started to be used in recent studies (Butler, 2001;
Cheung & Lee, 2009; Hasim & Salman, 2010; Ramayah, Ahmad, &
Lo, 2010). Taking into account the acceptance-discontinuance
anomaly brought forward by Bhattacherjee (2001) and under-
standing this as a phenomenon to be investigated, this study is
based upon the continuance concept.
Some of the frequently used theories or models for exploring
continuance usage intentions within technological innovations are
Information Systems Expectation Confirmation Model
(Bhattacherjee, 2001), Cognitive Model (Oliver, 1980) and Tech-
nology Continuance Theory (Liao, Palvia, & Chen, 2009). Moreover,
this research made use of the Information Systems Success Model
developed by DeLone and McLean (2003).
1.1.2. Information systems expectation confirmation model
This model was developed by Bhattacherjee in 2001 in order to
reveal the variables that affect the continuance usage intentions of
individuals with respect to information technologies. The roots of
this model -known as the Information Systems Continuance
Model-date back to the institutive research done in earlier years in
the marketing field. Inspired by studies such as Oliver’s Cognitive
Model (1980), in which the variables that effected repurchasing of
the same product by individuals were examined, this model was
presented for information systems in 2001. This is an important
model as it brings forward the differences between usage and long-
term usage. The fact that the target variable is an intrinsic variable
like intention has been criticised in recent studies (Limayem &
Cheung, 2008, 2011) and it is suggested that direct estimation of
the continuance usage and its integration into the models must be
examined. Bhattacherjee (2001) gives the operational definitions of
some variables that contribute to the model as below:
Information System Continuance Intention: Intention of the users
regarding continuance usage of the information system. It is the
target variable attempting to be predicted in the model.
Perceived Usefulness: Perceptions regarding what the users will
gain by using the information system.
Satisfaction: Emotions regarding users’ previous usage of the
information system.
Confirmation: Perceptions regarding the harmony between the
users’ expectations of the information system and the system’s
actual performance.
The central variable of Bhattacherjee’s (2001) Information Sys-
tems Expectation Confirmation is satisfaction. This variable takes
an important place in the model. According to the model, expec-
tations are important determiners of satisfaction. Taking into
consideration that expectations may change over time, it is claimed
that expectations related to users’ experiences are significant and
powerful predictors of satisfaction.
1.1.3. Cognitive model
Cognitive model, developed in 1980 by Oliver, can be stated as
the precursor to the other models and theories studied in this
literature. This model appeared before the Expectation Confirma-
tion Model was developed and states that confirmation behaviour
of the users affects satisfaction and satisfaction predicts long-term
usage intention. In this model, the attitude variable involves a
general evaluation of the technology used, whereas the satisfaction
variable comprises a judgment based on performance. In this
context, two variables which affect usage and long-term usage are
included in the same model and this makes the Cognitive Model as
important as others.
1.1.4. Technology continuance theory
Liao, et al. (2009) Technology Continuance Theory is an inte-
grated theory created by combining Technology Acceptance Model,
Expectation Confirmation Model and Cognitive Model for predict-
ing long-term usage of technological innovations. This theory
tested the other three models in terms of their explanatory power
and brought together six variables that had contributed consider-
ably to the general explanatory percentage of these theories and
models. The theory is based on the following variables: confirma-
tion, satisfaction, perceived usefulness, perceived ease of use, attitude
and information system continuance intention. This theory is the
latest of the theories and models that can be seen in the current
literature and is distinct from other theories and models as it brings
together a prior variable such as attitude and a posterior variable
such as satisfaction. In addition, it has proved the existence of a
predictor relationship between the two.
1.1.5. Information systems success model (DeLone & McLean, 1992,
2003; Petter, DeLone, & McLean, 2013)
The literature contains many models that attempt to determine
the success of information systems and present how constructs
affect this success. The most frequently used one of these models is
Information Systems Success Model. In this model, variables that
estimate the information system’s success in different conditions
and their relationships were examined. In the first model, intro-
duced in 1992, system quality and information quality were found
to have an effect on usage and satisfaction and it was concluded
that usage and satisfaction also effected one another. This model
was revised in 2003 after 11 years (DeLone & McLean, 2003) and a
service quality variable was added. Apart from satisfaction and
usage, usage intention was also taken into consideration and the
model became more dynamic. Ten years later, in 2013, another
update was made (Petter et al., 2013) and many other variables
were added to the model. Whereas task characteristics, project and
organisational characteristics and user and social characteristics
constitute the model’s independent variable, technology concepts
(such as usage, user satisfaction or net benefits) constitute the
dependent variable. Although it has been 22 years since the initial
conception of the model, system quality, information quality, usage
and satisfaction variables still maintain an appropriate level of
explanation percentages. Therefore, these variables are among the
important central variables in the final form of the model. One year
before the current form of the model was published, a research
under the name of The past, present, and future of IS Success had
been published (Petter, DeLone, & McLean, 2012). Researchers
attribute the most important reason for adding new variables and
 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211
redeveloping their models to the changing nature of information
systems. The changes that come with the information age are re-
flected in the information systems mentioned previously, thus,
changes to the concept of “success” become inevitable over time.
Researchers draw attention to four different themes for the future
of these studies: recognising the importance of “information” in the
information system, developing adaptive research processes and
models to measure Information Systems Success, disseminating
practical approaches to measure Information Systems Success and
preventing silos in Information Systems Success research. They
offered these themes as calls for action in the future of Information
Systems Success research.
Although the “Information Systems Success Model” has been
developed in an organisational context and its efficiency in
economical environments has been tested, it has also been used in
estimating the success, usage and continuance of many learning
systems in an educational context. In the literature, many studies
that integrate this model with different models or theories in an
educational context can be found (Alsabawy, Cater-Steel, & Soar,
2013; Balaban, Mu, & Divjak, 2013; Bhuasiri, Xaymoungkhoun, Zo,
Rho, & Ciganek, 2012; Chen, 2010; Li, Duan, Fu, & Alford, 2011; Lin
& Wang, 2012; Wang & Chiu, 2011).
1.1.6. Continuance in online learning environments
It is stated that studies on continuance of information system
can be classified into three distinct, yet overlapping groups (Larsen,
Sørebø, & Sørebø, 2009; Sørebø, Halvari, Gulli, & Kristiansen, 2009).
The first of these groups contain researches in which adoption is
taken and used as an independent variable in estimating the
continuance of information systems (e.g. Chiu, Hsu, Sun, Lin, & Sun,
2005; Limayem & Cheung, 2008; Lin et al., 2005; Roca, Chiu, &
Martínez, 2006). The second group contains researches that can
better explain the separation of the variables in the Information
Systems Continuance Model and their usage over time (e.g. Cheung
& Limayem, 2005; Chiu, Sun, Sun, & Ju, 2007; Kim & Malhotra,
2005). The third group contains researches that enrich the Infor-
mation Systems Continuance Model by integrating it with different
standpoints and thus can reach to higher explanation percentages
(e.g. Chiu, Chiu et al., 2007; Hsu & Chiu, 2004; Hsu, Chiu, & Ju, 2004;
Liao, Chen, & Yen, 2007; Liu, Liao, & Pratt, 2009; Roca & Gagne ,
2008). It can be said that this research belongs to the third group.
In this context, an integrated model has been created by deter-
mining variables which have significant effects on continuance
usage behaviour by the means of the models and theories from the
literature and modeling the continuance usage behaviour for online
learning environments was the aim of this study.
There is a few study about the continuance usage intention of
online learning environments. Jin et al. (2010) proposed a model to
predict continuance of online communities but this study based on
the information systems continuance, utilitarian and hedonic values
and affective commitment. Another study investigates online
learners’ continuance intention (Guo et al., 2016) but this study based
on an integrated flow framework. Some of the other studies were
focused on personal utilitarian motivations, ignored social factors or
hedonic factors (Chen, 2007; Cheung & Lee, 2007; Jin, Cheung, Lee, &
Chen, 2009). Our study is different from the prior studies within the
context of Information Systems Success Model, Technology Contin-
uance Model, utilitarian value, outcome expectations, perceived
value and perceived usability. A more comprehensive online learning
continuance model is developed and tested in this study.
1.2. Research model and hypotheses
In this research, the variables that have been hypothesised in the
model and the relationships between these anticipated variables
201
have been determined as a result of a literature review of the
models and theories that have taken place in this study’s theoret-
ical framework. The research model can be seen in Fig. 1 below.
When Fig. 1 is examined, it can be seen that information quality,
system quality and service quality have a possible effect on satis-
faction and confirmation. These three independent variables and
relationships are derived from DeLone and McLean’s (2003) Infor-
mation Systems Success Model. Satisfaction, confirmation and
continuance intention are taken from Information Systems
Continuance Model (Bhattacherjee, 2001), Cognitive Model (Oliver,
1980) and Technology Continuance Model (Liao et al., 2009). As
satisfaction and continuance intention are the main variables of the
Information Systems Continuance Model, the relationship between
these two variables are especially unchanged. In addition to these
constructs, four other exogenous variables and their anticipated
relationships with significant and high predictive power came from
the literature and are listed below.

Utilitarian value / (Chiu, Chiu, et al., 2007; Chiu, Sun, et al.,
2007; Kim & Oh, 2011)

Outcome expectations / (Hsu et al., 2004)

Perceived value / (Chang, 2013)

Perceived usability / (Chiu et al., 2005)
Although the role of information quality, system quality, service
quality, confirmation, perceived usability, perceived value, utili-
tarian value and outcome expectations have been investigated
within different Information Systems contexts, very few studies
have examined their role within the online learning setting. This
paper argues that these variables impact continuance intention.
The model shown in Fig. 1 was then used to clarify the principal
determinants of continuance intention in the online learning
setting by testing the following hypotheses.
H1. There is a positive relationship between information quality
and satisfaction.
H2. There is a positive relationship between system quality and
satisfaction.
H3. There is a positive relationship between service quality and
satisfaction.
H4. There is a positive relationship between information quality
and confirmation.
H5. There is a positive relationship between system quality and
confirmation.
H6. There is a positive relationship between service quality and
confirmation.
H7. There is a positive relationship between information quality
and perceived value.
H8. There is a positive relationship between system quality and
perceived value.
H9. There is a positive relationship between confirmation and
satisfaction.
H10. There is a positive relationship between outcome expec-
tations and satisfaction.
H11. There is a positive relationship between utilitarian value
and satisfaction.
H12. There is a positive relationship between perceived value
and satisfaction.
H13. There is a positive relationship between utilitarian value
and continuance intention.
H14. There is a positive relationship between perceived value
and continuance intention.
H15. There is a positive relationship between perceived usability
and continuance intention.
H16. There is a positive relationship between satisfaction and
continuance intention.
202 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211

Fig. 1. The research model.

2. Method
2.1. Study group
The study group of this research consisted of 467 undergraduate
students enrolled in a public university in the 2013/2014 fall se-
mester. The common characteristic of these students was that they
were all taking a Basic Information and Communication Technology
Usage course through an online learning portal and all of the re-
spondents were students of Faculty of Education. Because of this,
the study group was selected with the purposive sampling method.
The main goal of this method is to focus on the same characteristics
of online learning population which will best enable us to under-
stand the continuance usage. Students’ demographic information is
presented in Table 1.
As shown in Table 1, 387 of the students were female (82.9%)
while 80 of them were male (18.1%). In addition, 365 were first
year students (f ¼ 365; 78.2%), while 102 were second year stu-
dents (f ¼ 102; 21.8%). The major of most students was Eleman-
tary School Primary Education (22.7%), while the least common
major was Chemistry Education (1.7%). All of the students were
taking the previously mentioned online learning course for the
first time.
Technology Usage course is given. This course, taken in the online
learning environment, aims to develop the basic computer skills of
all the students in the faculty. Bringing forward an international
curriculum, introducing computer competence to students and
preparing students for evaluations made for the European Computer
Driving Licence certificate are among the purposes of this online
course. It is hoped that students will gain advantages in online
learning owing to this course being given in all faculties of the uni-
versity. The Basic Information and Communication Technologies
Usage course is structured into seven modules with the titles shown
below and included some of main acquisitions and sub acquisitions.

Concepts of information and communication technology (ICT)

Computer usage and managing files

Word processing

Spreadsheets

Databases

Presentation

Web browsing and communication
2.3. Implementation process of the research

The research was conducted in the fall semester of the 2013/

2.2. Online learning environment used in the research 2014 academic year. All the students taking the course were
brought together at the beginning of the semester and were
The learning environment used in the research is an online informed about the purpose, importance and scope of the course;
platform on which a Basic Information and Communication how it would be conducted; the online platform it would use and

Table 1
Demographic information of the study group.

First class Second class Total

Female Male Female Male

Computer Education and Instructional Technology 3 8 e 1 12

Psychological Counseling and Guidance 31 e 3 1 35
Science Education (Elementary School) 59 8 6 1 74
Mathematics Education (Elementary School) 26 4 e e 30
Preschool Education (Elementary School) 51 1 3 1 56
Primary Education (Elementary School) 78 16 9 3 106
Biology Education (Secondary School) 7 e 7 1 15
Physics Education (Secondary School) 4 2 8 1 15
Chemistry Education (Secondary School) 4 2 e 2 8
Mathematics Education (Secondary School) 7 1 1 2 11
German Language Education 15 7 14 1 37
French Language Education 3 1 16 7 27
English Language Education 23 4 9 5 41
Total 311 54 76 26 467
 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211
how it would be accessed; potential problems that could be faced in
accessing the online environment; and matters related to the
evaluation of the course. After this information was given, they
were told about this research and informed that participation in the
research was voluntary. The students were given a username and a
password to access the online environment. The students used
these details to access the online course and start interacting with
the content and learning modules.
The scale to be used in the research was transferred to the
electronic platform and sent to the students’ e-mail addresses. An
electronic web link of the scale was added to the home page of the
online learning platform and students were asked to fill it out. Data
collection processes continued until the end of the semester.
2.4. Data collection tool
In order to estimate the constructs that were necessary for the
research, related literature was reviewed and after the constructs
were decided, the data collection tool began to be shaped in
accordance with the online learning platform. In order to create an
item pool, items whose content and construct validity were tested
and developed for different technologies and platforms were
selected as the baseline and they were made suitable for online
learning environments. For instance, from the research in which
Chang (2013) tried to estimate the continuance intention regarding
e-learning platforms, the item of information quality was trans-
formed from “e-learning system presents the information appro-
priately” to “online learning platform presents the information
appropriately,” when structuring the data collection tool’s item
pool. Thus, the scale consisted of 10 constructs and 41 items were
given the final form. In order to maintain the cultural adaptation
and linguistic equivalence of the items mentioned above, they were
translated to Turkish by two experts, one of whom is in the
Department of English Translation and Interpreting in the same
university’s Literature Faculty, and another who works in the En-
glish Linguistics Department. Completed translations were evalu-
ated with an holistic view and a draft Turkish form was prepared. It
was detected that there were differences between the translations
and that some items on the final Turkish form and some scale items
were expressed in a clearer and more appropriate way. Then, the
draft form in Turkish was translated into English by a different
language expert. It was seen that original items and translated
items overlapped substantially. The items that did not overlap were
discussed with the translator and the final Turkish form was sha-
ped. Therefore, the translation and re-translation process regarding
the cultural adaptation of the articles was finalised. Each scale item
consists of a 7-point Likert scale (1: Definitely Disagree, 7: Defi-
nitely Agree). The constructs that the scale aims to estimate, item
numbers of the constructs and from which researches the items are
taken are shown on Table 2. Items of the data collection tool are
presented in Appendix 1.
2.5. Data analysis
Research data were analysed according to path analysis method.
At the beginning of the data analysis (before applying statistical
procedures) data abnormalities such as missing data and outliers
were investigated. Incomplete instruments were not included in
the study. Before the data were analysed, assumptions of multi-
variate statistical methods had been tested, and it was concluded
that the data were in accordance with the path analysis. For this
purpose, we checked the homogeneity of covariance matrices with
the Box’s M test of Equality of Covariance Matrices. Conducting the
Box's M test, no significant differences could be observed (Box's
M ¼ 535.884, F ¼ 1.855, p ¼ .034) and the differences among groups
203
were not significant. As suggested in the literature (Field, 2005;
Meyers, Gamst, & Guarino, 2006; Pallant, 2005), we used
a ¼ .001 for the Box's M test which is an indication of heterogeneity.
In order to check multicollinearity, we used the variance infla-
tion factor (VIF) and tolerance values. All VIF values ranged from
1.006 to 1.054 (VIF values must be less than 5 [Hair, Ringle, &
Sarstedt, 2011]) and tolerance values ranged from .949 to 994.
Findings showed that the VIF and tolerance values were
satisfactory.
Normal distribution of variables were checked by skewness and
kurtosis values. In addition, histograms and probability plots of the
variables revealed that the data were normally distributed. In
addition, scatter plot matrix of the variables showed multivariate
normality.
3. Findings
This section includes the findings of the research. First of all, the
findings regarding verification of the research model, then good-
ness of fit indices of the model and lastly, psychometric properties
of the estimations are given.
3.1. Model verification
Descriptive statistics of the data acquired after the data collec-
tion tool used in the research was applied to the respondent stu-
dents are presented in Table 3.
Mean, standard deviation, standard error of the mean and vari-
ance values of 10 constructs that are included in the research are seen
in Table 3. At least 1 and at most 7 points can be given to the scale
items and in this context, data length varies among a range of 6 points.
A hypothetical research model was tested as a result of the data
collected in the research and significance levels of the potential
relationships were determined. The final form of the model after
the estimation is presented in Fig. 2.
Fig. 2 includes estimated path coefficients acquired after testing
the research model and percentages of the explained variances for
endogenous variables. When the path coefficients of the model are
examined, it can be seen that all hypotheses were confirmed (H1-
H16). Direct and indirect relationships can be seen in Fig. 2 and
Table 4 also shows the direct, indirect and total effects of the model.
When Table 4 is examined, it can be seen that 41% (R2 ¼ .41) of
the confirmation of online learning environment usage is able to
be predicted by information quality, system quality and service
quality. Effects of these variables on the confirmation are statis-
tically significant and direct. It can be said that the strongest of the
effects mentioned above is information quality (b ¼ .488) which is
followed by system quality (b ¼ .270) and service quality
(b ¼ .187).
Of the variance seen in the satisfaction regarding the usage of
the online learning environment, 63% (R2 ¼ .63) is predicted by
seven exogenous variables (information quality, system quality,
service quality, confirmation, utilitarian value, outcome expecta-
tions and perceived value). Direct effects of all these variables are
statistically significant. When taking the aforementioned effects
into consideration, it can be said that the strongest predictor effect
is provided by the confirmation (b ¼ .468). In addition, indirect
effects of the information quality, system quality and service quality
variables that come from DeLone and McLean’s (2003) Information
Systems Success Model on the confirmation variable are concluded
to be statistically significant. However, total effect of any variable is
not as strong as the direct effect of the confirmation.
Of the variance seen in the perceived value, 28% (R2 ¼ .28) is
predicted with the information quality and system quality. It can be
asserted that the direct effect of the information quality on the
204 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211

Table 2
Constructs of the scale, item numbers of each construct and the researches of original items.

Constructs of the scale Item numbers Original items’ researches

Information Quality 6 Chang, 2013

System Quality 5 Chang, 2013
Service Quality 4 Chang, 2013
Utilitarian Value 4 Kim & Oh, 2011; Chiu, Chiu, et al., 2007;
Chiu, Sun, et al., 2007
Perceived Usability 6 Chiu et al., 2005
Confirmation 3 Kang et al., 2009; Liao et al., 2009
Perceived Value 3 Chang, 2013
Outcome Expectations 4 Hsu et al., 2004
Satisfaction 3 Chiu et al., 2005; Liao et al., 2009
Continuance Intention 3 Chiu et al., 2005; Liao et al., 2009

Table 3
Descriptive statistics of the research data.

Constructs Mean Standard deviation Standard error of the mean

Information Quality 3.7869 1.85196 .8570

System Quality 3.9670 1.77571 .8217
Service Quality 3.9427 1.76520 .8168
Utilitarian Value 4.1249 1.81378 .8393
Perceived Usability 4.1167 1.77898 .8232
Confirmation 3.8958 1.89472 .8768
Perceived Value 3.9714 1.86250 .8619
Outcome Expectations 3.8533 1.85951 .8605
Satisfaction 3.8587 1.87865 .8693
Continuance Intention 3.9065 1.81895 .8417
N: 467; Missing value: 0; Minimum: 1; Maximum: 7; Range: 6

perceived value (b ¼ .220) is stronger than the effect of the system
quality (b ¼ .137). There is no exogenous variable that has any in-
direct effect on the perceived value.
Of the variance seen in the continuance intention in the position
of target variable of the research model, 58% (R2 ¼ .58) was able to
be predicted by other exogenous variables. At the rate of this
explanation, there are both direct and indirect effects of the utili-
tarian value and perceived value variables, but only indirect effects
of perceived usability and satisfaction. When the research model is
examined, it can be said that there are only indirect effects of the
information quality, system quality, service quality, outcome ex-
pectations and satisfaction on the continuance intention. If all the
effects mentioned above are taken into consideration, it is clear that
the strongest effect is provided by satisfaction (b ¼ .508). Direct
effect of this variable alone is stronger than the direct, indirect and
total effects of many other variables. The fact that continuance
intention is predicted by all the other variables directly or indirectly
makes it the target variable of the model. Thus, this variable is the
most important variable of the research model in terms of its
predictor power since the variable that has the strongest predictor
effect on this variable is satisfaction. Similarly, the variable that has
the strongest predictor effect on the satisfaction variable is confir-
mation. These two findings confirm the relationships between the
satisfaction, confirmation and continuance intention variables

Fig. 2. Analysis of the research model.

 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211 205

Table 4
Identification of the research model: Direct, indirect and total effects.

Variables Effects

Dependent Variable (endogenous) Independent Variable (exogenous) Direct effect Indirect effect Total effect
**
Confirmation (R ¼ .41) 2
Information Quality .488 e .488**
System Quality .270** e .270**
Service Quality .187* e .187*
Satisfaction (R2 ¼ .63) Information Quality .269** .095** .364**
System Quality .250** .058* .308**
Service Quality .136* .029* .165*
Confirmation .468** e .468**
Utilitarian Value .190** e .190**
Outcome Expectations .088* e .088*
Perceived Value .169** e .169**
Perceived Value (R2 ¼ .28) Information Quality .220** e .220**
System Quality .137* e .137*
Continuance Intention (R2 ¼ .58) Information Quality e .107* .107*
System Quality e .116** .116**
Service Quality e .119** .119**
Utilitarian Value .159** .018* .177**
Perceived Value .258** .097* .355**
Perceived Usability .328** e .328**
Confirmation e .159** .159**
Outcome Expectations e .045* .045*
Satisfaction .508** e .508**
* **
: p < .05; p < .01 (two way).

presented in DeLone and McLean’s (2003) Information Systems 3.3. Psychometric properties of measurements in the research model
Success Model.
When the direct, indirect and total effects of the research model Certain psychometric properties should be reported in order to
are examined, it is seen that all the effects are significant as well as evaluate the measurement models, which were dealt with the
positive. There is no exogenous variable in the model that affects estimated model. Average explained variance values for each
the endogenous variables negatively. construct, correlations between the constructs, standardised factor
loadings related to the items of the constructs included in the
measurement models, item mean scores, standard errors and t-
3.2. Goodness of fit indices of the research model statistics of each item could be listed among these properties.
Table 6 presents the average variance explained (AVE) results
When goodness of fit indices of the research model are exam- related to the constructs of the research model and correlations
ined, it is observed that the c2/df ratio is 1.79, and this is equal to between the constructs.
perfect fit (p < .05). Even if the fact that this ratio is less than 3 Table 6 shows the AVE values belonging to the constructs
means a perfect fit, that c2 can easily be affected by sample size and studied in the research model as well as the correlations of these
regarding that at larger values like .05 it generally takes a signifi- constructs. Values observed in the diagonal axis of the correlation
cant value (Schumacker & Lomax, 2004). Along with this ratio, the matrix for each construct are equal to the square root of AVEs. Other
other indices should also be reported. Reported results of other off-diagonal values are the correlations between constructs.
goodness of fit indices are shown in Table 5. According to the results of the measurements, despite the fact
As examined in Table 5, it can be seen that the model has a good that the goodness of fit indices for the research model indicate a
fit. Just goodness of fit index (GFI) value remained below the good fit, findings about the reliability and validity of the model
acceptable fit limit but considering the other goodness of fit indices, should also be reported. In this respect, convergent validity and
it was decided that the model was an adequate fit. Moreover, that discriminant validity of the measurements were analysed. These
the adjusted goodness of fit index (AGFI) is .86 may ensure that two types of validities indicate the construct validity of the mea-
lowness of GFI can be ignored because, AGFI is the corrected version surements performed.
of GFI for the number of parameter estimations (Çokluk, Convergent validity is a type of construct validity, which in-
Şekercioglu, & Büyüko€ ztürk, 2010). Generally, when one of these dicates the convergence between the measurements that are per-
two goodness of fit values adjusts well, it leads us to interpret that formed with the aim of measuring a certain construct and the
the model has an overall good fit. alternative measurements that aim to measure the same construct.
As Fornell and Larcker (1981) suggested, convergent validity could
be obtained when AVE value and factor loading values of each item
Table 5 are greater than 0.50. According to Table 6, AVE values were
Goodness of fit indices of the research model. observed to change between 0.59 and 0.88. Factor loading values of
each scale item are presented in Table 7. Looking at the fact that the
Goodness of fit Indices Model Value Good fit Acceptable fit
loading values changed between 0.650 and 0.911, the research
RMSEA .028 0  RMSEA .05 .05 < RMSEA .08 model could be considered to have convergent validity.
NFI .95 .95  NFI 1.00 .90  NFI <.95
NNFI .96 .97  NNFI 1.00 .95  NNFI <.97
Discriminant validity requires the potential relationships be-
CFI .95 .97  CFI 1.00 .95  CFI <.97 tween the scales that measure different constructs to be poor.
S-RMR .039 0  S-RMR  .05 .05 < S-RMR .10 Carless (2004) mentioned that in order to ensure discriminant
GFI .88 .95  GFI 1.00 .90  GFI <.95 validity, correlations between the measurement of a construct and
AGFI .86 .90  AGFI 1.00 .85  AGFI <.90
the measurements of other constructs could be presented as
206 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211

Table 6
Correlations of latent variables and AVEs.

Constructs AVE*** 1 2 3 4 5 6 7 8 9 10

1. Information Quality .61 .78

2. System Quality .71 .44** .84
3. Service Quality .79 .41** .47** .89
4. Utilitarian Value .59 .22* .24* .26* .77
5. Perceived Usability .74 .28* .32** .20* .18* .86
6. Confirmation .88 .47** .48** .34** .31** .21* .94
7. Perceived Value .66 .39** .21* .26* .32** .12* .28** .81
8. Outcome Expectations .69 .31** .26* .22* .15* .17* .20* .18* .83
9. Satisfaction .85 .46** .40** .36** .31** .20* .41** .24* .20* .92
10. Continuance Intention .88 .33** .30** .25* .32** .28* .42** .29* .24* .35** .94
n ¼ 467; *p < .05; **p < .01; ***
AVE: Average Variance Explained

evidence. Pituch and Lee (2006) reported that AVE values could be ensured when the square root of the AVE values for each construct
used in order to obtain the discriminant validity at a adequate level. is greater than the correlations between that construct and other
Fornell and Larcker (1981) stated that this type of validity could be constructs. In Table 6, it was observed that the square roots of the

Table 7
Psychometric properties of the measurements.

Scale item Standardised factor loadings Mean (scale range: 1 to 7) Standard error t- statistic

Information Quality (IQ)

IQ1 .878 3.9257 .029 41.127
IQ2 .812 3.5687 .049 32.102
IQ3 .797 3.8663 .053 29.910
IQ4 .826 4.1245 .037 32.288
IQ5 .844 3.1127 .044 38.812
IQ6 .876 4.1235 .032 40.126
System Quality (SyQ)
SyQ1 .7894 3.5468 .057 17.799
SyQ2 .7912 4.1237 .046 18.812
SyQ3 .8116 3.9577 .039 20.007
SyQ4 .8227 4.2566 .027 21.120
SyQ5 .8159 3.9502 .034 20.014
Service Quality (SeQ)
SeQ1 .7681 4.1132 .045 13.546
SeQ2 .7952 3.5879 .039 15.529
SeQ3 .8059 3.9595 .034 20.156
SeQ4 .8435 4.1102 .028 22.277
Utilitarian Value (UV)
UV1 .6979 3.9333 .051 30.135
UV2 .7257 4.2781 .043 33.256
UV3 .7691 4.1633 .040 34.419
Perceived Usability (PU)
PU1 .8131 4.2268 .034 29.255
PU2 .7982 4.3666 .039 23.726
PU3 .7539 3.8625 .042 22.642
PU4 .7192 3.7563 .051 20.136
PU5 .8034 4.2136 .037 24.456
PU6 .7138 4.2744 .051 20.009
Confirmation (C)
C1 .8891 3.6579 .026 44.425
C2 .8264 4.1129 .042 35.259
C3 .8669 3.9166 .031 39.124
Perceived Value (PV)
PV1 .8443 3.5972 .049 25.546
PV2 .8034 4.0994 .053 22.125
PV3 .8913 4.2176 .036 34.476
Outcome Expectations (OE)
OE1 .766 3.9126 .026 16.624
OE2 .709 4.0863 .030 12.273
OE3 .697 3.7799 .037 10.268
OE4 .650 3.6344 .041 9.456
Satisfaction (S)
S1 .911 3.8843 .019 78.845
S2 .876 3.7279 .032 69.947
S3 .907 3.9639 .024 74.426
Continuance Intention (CI)
CI1 .903 4.0873 .028 64.492
CI2 .890 3.8891 .033 58.826
CI3 .846 3.7431 .037 49.927
 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211
AVE values were aligned on the diagonal axis of the correlation
matrix and these values changed between 0.77 and 0.94. For
instance, the square root of the AVE value for the satisfaction var-
iable was 0.92; and the correlations between the satisfaction vari-
able and other variables was not greater than this value. Table 6
displays that the correlations changed between 0.20 and 0.46.
Since the same observations were made for all other constructs, the
research model could be considered to have discriminant validity.
Due to the fact that both discriminant and convergent validities of
the research model were ensured, the model could be considered to
have construct validity.
Certain psychometric properties of items measured in the
research model such as standardised factor loadings, means, stan-
dard errors and t values are presented in Table 7.
Table 7 shows the standardised factor loading values, means,
standard errors and t values of the items related to the constructs
studied within the research model. The fact that the factor loading
values changed between 0.650 and 0.911 indicates that the model
has convergent validity; and all of the t values were observed to be
greater than 1.96 at the significance level of p ¼ .05.
These findings indicate that the research model is valid. With
respect to the reliability of the model, composite reliability co-
efficients were calculated for each construct. These coefficients
indicate the construct reliability of the model. Table 8 shows the
composite reliability coefficients of the constructs included in the
research model.
As Table 8 indicates, composite reliability coefficients changed
between 0.71 and 0.89. Due to the fact that these values are greater
than the critical value of 0.60, which was determined by Bagozzi
and Yi (1988), the construct reliability could be considered
ensured. Since both construct validity and construct reliability were
ensured, the research model and the findings obtained could be
considered reliable and valid.
4. Results, discussion and recommendations
This section summarises the findings obtained as a result of the
research, discusses them with relevance to the literature, and
shows results that are different or consistent with results of some
other studies. Based on these results, some recommendations are
made for future studies and the implementation process.
4.1. Results and discussion
This study aimed to determine variables affecting the contin-
uance usage intention of online learning environments and to
develop an online continuance usage intention model. As a result
of testing the hypothetical model, it was observed that informa-
tion quality, system quality and service quality affect the satis-
faction of online learning environment usage. These findings
Table 8
Composite reliability coefficients of the constructs.
Construct Composite reliability coefficient
Information Quality .73
System Quality .80
Service Quality .83
Utilitarian Value .71
Perceived Usability .82
Confirmation .87
Perceived Value .77
Outcome Expectations .79
Satisfaction .86
Continuance Intention .89
207
support the relationships suggested in DeLone and McLean’s
(2003) Information Systems Success Model. There are various
studies in the literature which aimed to test these three re-
lationships. In a research by Chiu, Chiu, et al. (2007) and Chiu, Sun,
et al. (2007), it was attempted to predict web-based learning
satisfaction through the quality variables; the impact of informa-
tion quality and system quality on satisfaction was proved to be
significant while the impact of service quality was proved to be
insignificant. In another research on the success of knowledge
management systems, it was deduced that the impact of infor-
mation quality, system quality, and service quality on satisfaction
is significant (Halawi, McCarthy, & Aronson, 2007). In a research
conducted by Petter, DeLone and McLean in 2008, relationships
set forth in DeLone and McLean’s (2003) Information Systems
Success Model were researched through qualitative literature re-
view, and it was proved that in individual-based studies, infor-
mation and system quality had a predictor effect on satisfaction. In
this research, it was also proved that service quality had a varia-
tional predictor effect. Some researches were supportive of this
potential impact (Chang, 2013; Halawi et al., 2007; Leclercq, 2007),
while some were not (Chiu, Chiu, et al., 2007; Chiu, Sun, et al.,
2007; Marble, 2003). In another research conducted by
Sabherwal, Jeyaraj, and Chowa (2006), a meta-analysis process
was carried out and the research model was tested at the end of
this process. In this research, the effect of service quality on
satisfaction was proved to be significant but the other relation-
ships were not analysed. With reference to these researches, it can
be stated that these three relationships set forth in DeLone and
McLean's (2003) Information Systems Success Model were
confirmed in some researches conducted within the educational
context but sometimes they gave insignificant results. In this
research, the effects of information quality, system quality and
service quality on satisfaction were considered significant and it
could be said that these three predictor variables had an indirect
effect on continuance intention.
In this research, information quality, system quality and service
quality variables obtained from DeLone and McLean's (2003) In-
formation Systems Success Model, were associated with the
confirmation variable of Information Systems Expectation Confir-
mation Model (Bhattacherjee, 2001). These strong predictor vari-
ables of the two models were combined in the same model with a
view to measure the continuance intention that can be provided in
the usage of online learning environments. It could be stated that
predicting 41% of the variance of the confirmation belonging to
these three variables and making their relationships significant
would make the model significant and contribute to the existing
knowledge in the literature.
This research indicated that information quality, system quality
and service quality variables had significant effects on confirmation
and satisfaction; students gave importance to the dimensions dis-
cussed by the variables of this research in the online learning
environment; it had an effect on the continuance that could be
provided in usage. Furthermore, the fact that most of the students
were first year students and that this was the first time they had
experienced an online lesson could have led to a positive effect.
In this study, confirmation predicts satisfaction and satisfaction
predicts continuance intention; this phenomenon validates the
construct of Information Systems Expectation Confirmation Model
(Bhattacherjee, 2001) once again. While 41% of continuance
intention variance and 33% of satisfaction variance were explained
in the original model, it was considered important that the unified
research model would reach 58% of continuance intention variance
and 53% of satisfaction variance. These percentages were rather
good for the social sciences. We could say that other variables had
an important effect on the 20% increase in continuance intention
208 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211
variance and 17% increase in satisfaction variance. The amount of
explained variance in the continuance intention variable could vary
over a wide range in the studies. While the research conducted by
Lin, Chen, and Fang (2011) addressing negative critical incidents
perspective on continuance intention related to e-learning has
reached such a great prediction percentage as 92%, in another
research where Information Systems Expectation Confirmation
Model (Bhattacherjee, 2001) was expanded by habit and prior
behaviour variables, only 23% of prediction percentage has been
achieved (Limayem & Cheung, 2008). In this research, where
continuance usage intention for the online learning environment
was discussed, it could be stated that the considerable amount of
58% percentage was very important.
Of the variance seen in the satisfaction variable, 63% was pre-
dicted by seven external variables (information quality, system
quality, service quality, confirmation, utilitarian value, outcome
expectations and perceived value). In addition, the indirect impact
of information quality and system quality on perceived value and
the indirect impact of information quality, system quality and
service quality on confirmation were significant. Regarding all the
relationships, it could be argued that the confirmation variable
caused the strongest effect on satisfaction. This finding was
consistent with the relationship set forth in Information Systems
Expectation Confirmation Model (Bhattacherjee, 2001). It could be
said that this relationship was very strong and reflected many
other results within the literature. This relationship seemed to be
statistically significant and was confirmed by many researches in
the literature (Hong, Thong, & Tam, 2006; Larsen et al., 2009;
Recker, 2010; Roca et al., 2006). In a study on social networking
services, this relationship was found insignificant (Kang, Hong, &
Lee, 2009). Considering the characteristics of the study group,
the fact that the significant effect of the confirmation on the
satisfaction variable could be understood. Regarding the predic-
tion of an important percentage of confirmation variable and the
overall consistency of the model, it can be stated that Information
Systems Expectation Confirmation Model (Bhattacherjee, 2001)
could meet the expectations with regard to the significance of
central relationships.
In this study, the target variable of the research model is
continuance intention. A significant part of the variance observed
in this variable was predicted by other variables. In particular, the
effect of satisfaction on continuance intention was highly regarded,
because this was the major relationship of Information Systems
Expectation Confirmation Model (Bhattacherjee, 2001). Signifi-
cance of this relationship was consistent with the literature. In
researches on mobile Internet usage (Thong et al., 2006; Zhou,
2011), the continuance of the e-learning environment (Chang,
2013; Chiu et al., 2005; Larsen et al., 2009; Lin et al., 2011), the
continuance usage of the web (Chen, Yen, & Hwang, 2012) and the
continuance usage of blogs (Shiau & Chau, 2012), it could be seen
that this relationship was significant. In a research conducted with
teachers (Sørebø et al., 2009), the effect of satisfaction on contin-
uance intention turned out to be insignificant.
In this research, the effects of perceived value and utilitarian
value on satisfaction and continuance intention were found as
significant. We could also say that the relationships between
outcome expectations and satisfaction, perceived usability and
continuance intention were significant. Regarding all these find-
ings, we could see that four endogenous variables of confirmation,
satisfaction, perceived value and continuance intention achieved
these prediction percentages respectively: 41%, 63%, 28% and 58%.
Confirming that the model created in this research was valid, reli-
able, and consistent with goodness of fit indices indicate that it
could be used to determine continuance usage intention of online
learning environments.
4.2. Recommendations
4.2.1. Recommendations for future studies
1. In this research, the predictor effect of attitude variables has not
been examined. It is suggested that the research model be
expanded by adding the attitude variable to a larger model and
be reexamined. At this stage, it is useful to take the relationships
found in the literature (confirmationeattitudeecontinuance
intention relationships in Technology Continuance Theory and
Cognitive Model) into account.
2. Different variables, which are considered to be effective in
achieving greater explanation percentages in continuance
intention, should also be considered. As the unpredicted per-
centage of the continuance intention is still unclear, this topic
should be discussed.
3. The online learning environment used in this research was a
one-semester course. Therefore, the continuance usage was not
possible to measure directly. In long-term and longitudinal re-
searches, more accurate results might be achieved. If any
continuance usage index can be developed with consideration
to the students, more accurate and generalisable results will be
gained. In this case, as pointed out and stressed in the literature,
adding the continuance directly to the models and measuring
the new models is possible.
4. In the online learning environment referred in this research,
only studentecontent interaction is enabled. It is necessary to
determine whether the same constructs are significant in en-
vironments where all the students and teachers are together
and all kinds of interactions can be observed. It is really
important to determine the guidance needs of students in these
environments and determine what kind of constructs produce
which effects on different students. Comparing the potential
findings of this research with the results of it can make the
generalisation more reliable.
5. It should not be ignored that certain variables used in this
research can change over time with user experiences. Therefore,
it is necessary to take measurements periodically and if possible,
to conduct longitudinal studies. Thus the model, in which cross-
sectional significance was tested and confirmed, will be revised
in terms of longitudinal consistency.
4.2.2. Recommendations for implementation process
1. The learning environment for this research was fully voluntary.
There were no obligations for students related to the usage of
the online learning environment. As the research results cannot
be expanded to mandatory online learning environments, it is
suggested that future researches aim to learn what kind of
constructs produce significant findings in these environments in
terms of theory and practice. As a result of such research, the
constructs which have an effect on the usage of online learning
environments, both voluntary and mandatory, can be compared
and some implications made.
2. Regarding the finding that a quality service that meets the ex-
pectations of students ensures the continuance of the online
learning environment, particular attention should be paid to the
information provided and the overall system design of these
environments.
3. Increasing the interaction between satisfaction and attitude
variables could be recommended to ensure the continuance
usage of online learning environments. However, examining
this relationship will make a different contribution to the liter-
ature from the previous studies. Prior and posterior reciprocal
 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211 209

effects of these two variables can produce different findings (continued )

from the ones that can be produced separately. It can be said
Construct Item Item
that a possible effect may bring a different perspective to number
Technology Continuance Theory and Cognitive Model.
18 Overall, the use of online learning environment
4. In an online learning environment used by students with
delivers me good value.
different user experiences, different constructs can show pre- Perceived 19 Learning to operate the online learning environment is
dictive features to ensure continuance usage. Therefore, stu- Usability easy for me.
dents’ online learning experiences, habits and previous usage of 20 It is easy for me to become skillful at using the online
online learning environments must be taken into account. For learning environment.
21 Using the online learning environment can improve
learning environments to be formed which take these things my learning performance.
into account, important tasks fall to teaching staff and designers 22 Using the online learning environment can simplify
of these environments. the learning process.
5. In the literature, the importance of ensuring the continuance 23 Using the online learning environment fits well with
the way I learn.
usage for instructors and instructional designers is researched
24 The setup of the online learning environment is
and discussed. Continuance usage should also be evaluated from compatible with the way I learn.
students’ perspectives. In this view, phenomenological re- Confirmation 25 My experience with using the online learning
searches can be conducted. environment was better than I expected.
6. The effects on continuance usage of online learning environ- 26 The service level provided by the online learning
environment was better than I expected.
ments should be taken into consideration while designing on- 27 Overall, most of my expectations from using the online
line courses. learning environment were confirmed.
7. In this research, only an online platform that enables stu- Perceived 28 I use the online learning environment to keep up to
dentecontent interaction is referred as a learning environment. Value date.
29 The online learning environment satisfies my needs
It is asserted that technological innovations such as mobile
for knowledge, learning and development.
technologies and learning management systems, that are 30 The online learning environment plays a crucial role
becoming increasingly common in educational contexts, and for me.
their continuance usage should be evaluated in the future Outcome Expectations
studies. 31 If I use the online learning environment, I will increase
my effectiveness on the task.
32 If I use the online learning environment, I will gather
complete and timely information.
Appendix 1. Data collection tool 33 If I use the online learning environment, my peers will
perceive me as competent.
34 If I use environment, I will increase my sense of
the accomplishment.
online
learning
Construct Item Item
Satisfaction 35 I am satisfied with the performance of the online
number
learning environment.
Information 1 Online learning environment provides relevant 36 I am pleased with the experience of using the online
Quality information for my homework. learning environment.
2 Online learning environment presents the information 37 My decision to use the online learning environment
in an appropriate format. was a wise one.
3 The information content in the online learning Continuance 38 I intend to continue using the online learning
environment is very good. Intention environment in the future.
4 The information from the online learning environment 39 I will continue using the online learning environment
is up-to-date enough for my purposes. in the future.
5 The reliability of output information from online 40 I will regularly use the online learning environment in
learning environment is high. the future.
6 Online learning environment provides the information
I need in time.
System 7 Steps to complete a task in the online learning
Quality environment follow a logic sequence. References
8 Performing an operation in the online learning
environment always leads to a predicted result. Al-Debei, M. M., Al-Lozi, E., & Papazafeiropoulou, A. (2013). Why people keep
9 The organisation of information on the online learning coming back to facebook: explaining and predicting continuance participation
environment screens is clear. from an extended theory of planned behaviour perspective. Decision Support
10 Online learning environment has natural and Systems, 55(1), 43e54. http://dx.doi.org/10.1016/j.dss.2012.12.032.
Alsabawy, A. Y., Cater-Steel, A., & Soar, J. (2013). IT infrastructure services as a
predictable screen changes.
requirement for e-learning system success. Computers & Education, 69,
11 Online learning environment responds quickly during
431e451. http://dx.doi.org/10.1016/j.compedu.2013.07.035. November 2013.
the busiest hours of the day.
Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models.
Service 12 Overall, the online learning environment have Journal of the Academy of Marketing Science, 16(1), 74e94. http://dx.doi.org/
Quality excellent quality. 10.1177/009207038801600107.
13 The service quality provided by this online learning Balaban, I., Mu, E., & Divjak, B. (2013). Development of an electronic portfolio sys-
environment matches with my expectations. tem success model: an information systems approach. Computers & Education,
14 Online learning environment gives a fast service for 60(1), 396e411. http://dx.doi.org/10.1016/j.compedu.2012.06.013.
users. Barnes, S. J. (2011). Understanding use continuance in virtual worlds: empirical test
15 The operation hours of the online learning of a research model. Information & Management, 48(8), 313e319. http://
environment are convenient to its users. dx.doi.org/10.1016/j.im.2011.08.004.
Utilitarian 16 Compared to the effort I need to put in, the use of Bento, R., & Schuster, C. (2003). Participation: the online challenge. In A. Aggarwal
Value online learning environment is beneficial to me. (Ed.), Web based education: Learning from experience (pp. 156e164). Hershey,
17 Compared to the time I need to spend, the use of Pennsylvania: Idea Group Publishing.
Bhattacherjee, A. (2001). Understanding information systems continuance: an
online learning environment is worthwhile to me.
expectation-confirmation model. MIS Quarterly, 25(3), 351e370. http://
(continued on next page) dx.doi.org/10.2307/3250921.
210 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211
Bhattacherjee, A., Perols, J., & Sanford, C. (2008). Information technology continu-
ance: a theoretical extension and empirical test. Journal of Computer Information
Systems, 49(1), 17e26.
Bhuasiri, W., Xaymoungkhoun, O., Zo, H., Rho, J. J., & Ciganek, A. P. (2012). Critical
success factors for e-learning in developing countries: a comparative analysis
between ICT experts and faculty. Computers & Education, 58(2), 843e855.
http://dx.doi.org/10.1016/j.compedu.2011.10.010.
Bourelle, A., Bourelle, T., Knutson, A. V., & Spong, S. (2016). Sites of multimodal
literacy: comparing student learning in online and face-to-face environments.
Computers and Composition, 39, 55e70. http://dx.doi.org/10.1016/
j.compcom.2015.11.003.
Butler, B. S. (2001). Membership size, communication activity, and sustainability: a
resource-based model of online social structures. Information Systems Research,
12(4), 346e362. http://dx.doi.org/10.1287/isre.12.4.346.9703.
Carless, S. A. (2004). Discriminant validity. In M. S. Lewis-Beck, A. Bryman, &
T. F. Liao (Eds.), Encyclopedia of social science research methods (p. 273). Thou-
sand Oaks, CA: SAGE Publications. http://dx.doi.org/10.4135/
9781412950589.n248.
Chang, C.-C. (2013). Exploring the determinants of e-learning systems continuance
intention in academic libraries. Library Management, 34(1/2), 40e55. http://
dx.doi.org/10.1108/01435121311298261.
Chen, I. Y. L. (2007). The factors influencing members’ continuance intentions in
professional virtual communities e a longitudinal study. Journal of Information
Science, 33(4), 451e467. http://dx.doi.org/10.1177/0165551506075323.
Chen, H.-J. (2010). Linking employees’ e-learning system use to their overall job
outcomes: an empirical study based on the IS success model. Computers &
Education, 55(4), 1628e1639. http://dx.doi.org/10.1016/j.compedu.2010.07.005.
Cheng, C. K., Pare , D. E., Collimore, L.-M., & Joordens, S. (2011). Assessing the
effectiveness of a voluntary online discussion forum on improving students’
course performance. Computers & Education, 56(1), 253e261. http://dx.doi.org/
10.1016/j.compedu.2010.07.024.
Chen, S.-C., Yen, D. C., & Hwang, M. I. (2012). Factors influencing the continuance
intention to the usage of Web 2.0: an empirical study. Computers in Human
Behavior, 28(3), 933e941. http://dx.doi.org/10.1016/j.chb.2011.12.014.
Cheung, C. M. K., & Lee, M. K. O. (2007). What drives members to continue sharing
knowledge in a virtual professional community? The role of knowledge self-
efficacy and satisfaction. In Z. Zhang, & J. Siekmann (Eds.), Knowledge science,
engineering and management: Second international conference KSEM 2007 pro-
ceedings (pp. 472e484). Germany: Springer-Verlag Berlin. http://dx.doi.org/
10.1007/978-3-540-76719-0_46.
Cheung, C. M. K., & Lee, M. K. O. (2009). Understanding the sustainability of a virtual
community: model development and empirical test. Journal of Information
Science, 35(3), 279e298. http://dx.doi.org/10.1177/0165551508099088.
Cheung, C. M. K., & Limayem, M. (2005). The role of habit in information systems
continuance: examining the evolving relationship between intention and us-
age. In Proceedings of the international conference on information systems. Las
Vegas, USA. Association for Information Systems.
Chiu, C. M., Chiu, C. S., & Chang, H. C. (2007). Examining the integrated influence of
fairness and quality on learners’ satisfaction and web-based learning continu-
ance intention. Information Systems Journal, 17(3), 271e287. http://dx.doi.org/
10.1111/j.1365-2575.2007.00238.x.
Chiu, C.-M., Hsu, M.-H., Sun, S.-Y., Lin, T.-C., & Sun, P.-C. (2005). Usability, quality,
value and e-learning continuance decisions. Computers & Education, 45(4),
399e416. http://dx.doi.org/10.1016/j.compedu.2004.06.001.
Chiu, C.-M., Sun, S.-Y., Sun, P.-C., & Ju, T. L. (2007). An empirical analysis of the
antecedents of web-based learning continuance. Computers & Education, 49(4),
1224e1245. http://dx.doi.org/10.1016/j.compedu.2006.01.010.
Çokluk, O.,€ Şekerciog lu, G., & Büyüko €ztürk, Ş. (2010). Sosyal bilimler için çok
degişkenli istatistik: SPSS ve LISREL uygulamaları (1. Baskı). Ankara: Pegem A
Yayıncılık.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance
of information technology. MIS Quarterly, 13(3), 319e340. http://dx.doi.org/
10.2307/249008.
DeLone, W. H., & McLean, E. R. (1992). Information systems success: the quest for
the dependent variable. Information Systems Research, 3(1), 60e95. http://
dx.doi.org/10.1287/isre.3.1.60.
DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean Model of infor-
mation systems success: a ten-year update. Journal of Management Information
Systems, 19(4), 9e30.
Doherty, I. (2011). Evaluating the impact of educational technology professional
development upon adoption of Web 2.0 tools in teaching. Australasian Journal of
Educational Technology, 27(3), 381e396.
Esteban-Millat, I., Martínez-Lo pez, F. J., Huertas-García, R., Meseguer, A., & Rodrí-
guez-Ardura, I. (2014). Modelling students' flow experiences in an online
learning environment. Computers & Education, 71, 111e123. http://dx.doi.org/
10.1016/j.compedu.2013.09.012. February 2014.
Field, A. (2005). Discovering statistics using SPSS: And sex and drugs and rock 'n' roll
(2nd ed.). London: Sage Publications.
Fornell, C., & Larcker, D. F. (1981). Structural equation models with unobservable
variables and measurement error. Journal of Marketing Research, 18(1), 39e50.
Go€kçearslan, Ş., & Alper, A. (2015). The effect of locus of control on learners' sense of
community and academic success in the context of online learning commu-
nities. The Internet and Higher Education, 27, 64e73. http://dx.doi.org/10.1016/
j.iheduc.2015.06.003. October 2015.
Guo, Z., Xiao, L., Van Toorn, C., Lai, Y., & Seo, C. (2016). Promoting online learners’
continuance intention: an integrated flow framework. Information & Manage-
ment, 53(2), 279e295. http://dx.doi.org/10.1016/j.im.2015.10.010.
Hair, J. F., Ringle, C. M., & Sarstedt, M. (2011). PLS-SEM: indeed a silver bullet. Journal
of Marketing Theory and Practice, 19(2), 139e151. http://dx.doi.org/10.2753/
MTP1069-6679190202.
Halawi, L. A., McCarthy, R. V., & Aronson, J. E. (2007). An empirical investigation of
knowledge management systems success. Journal of Computer Information
Systems, 48(2), 121e135.
Handal, B., Cavanagh, M., Wood, L., & Petocz, P. (2011). Factors leading to the
adoption of a learning technology: the case of graphics calculators. Australasian
Journal of Educational Technology, 27(2), 343e360.
Harasim, L., Hiltz, S. R., Teles, L., & Turoff, M. (1995). Learning networks: A field guide
to teaching and learning online. Cambridge, MA: MIT Press.
Harden, G., Ryan, S. D., & Prybutok, V. R. (2012). Social networking site continuance:
the paradox of negative consequences and positive growth. Informing Science:
the International Journal of an Emerging Transdiscipline, 15, 207e227.
Hasim, M. S., & Salman, A. (2010). Factors affecting sustainability of Internet usage
among youth. The Electronic Library, 28(2), 300e313. http://dx.doi.org/10.1108/
02640471011033657.
Hew, K. F., Cheung, W. S., & Ng, C. S. L. (2009). Student contribution in asynchronous
online discussion: a review of the research and empirical exploration. Instruc-
tional Science, 38(6), 571e606. http://dx.doi.org/10.1007/s11251-008-9087-0.
Hewitt, J. (2005). Toward an understanding of how threads die in synchronous
computer conferences. Journal of the Learning Sciences, 14(4), 567e589. http://
dx.doi.org/10.1207/s15327809jls1404_4.
Hong, S.-J., Thong, J. Y. L., & Tam, K. Y. (2006). Understanding continued information
technology usage behavior: a comparison of three models in the context of
mobile internet. Decision Support Systems, 42(3), 1819e1834. http://dx.doi.org/
10.1016/j.dss.2006.03.009.
Hranstinski, S. (2009). A theory of online learning as online participation. Computers
& Education, 52(1), 78e82. http://dx.doi.org/10.1016/j.compedu.2008.06.009.
Hsu, M. H., & Chiu, C. M. (2004). Predicting electronic service continuance with a
decomposed theory of planned behaviour. Behaviour & Information Technology,
23(5), 359e373. http://dx.doi.org/10.1080/01449290410001669969.
Hsu, M. H., Chiu, C. M., & Ju, T. L. (2004). Determinants of continued use of the
WWW: an integration of two theoretical models. Industrial Management & Data
Systems, 104(9), 766e775. http://dx.doi.org/10.1108/02635570410567757.
Hung, M.-L., & Chou, C. (2015). Students' perceptions of instructors' roles in blended
and online learning environments: a comparative study. Computers & Educa-
tion, 81, 315e325. http://dx.doi.org/10.1016/j.compedu.2014.10.022. February
2015.
Ifinedo, P. (2006). Acceptance and continuance intention of web-based learning
technologies (WLT) use among university students in a Baltic country. The
Electronic Journal of Information Systems in Developing Countries, 23(6), 1e20.
Jasperson, J. S., Carter, P. E., & Zmud, R. W. (2005). A comprehensive conceptuali-
zation of post-adoptive behaviours associated with IT enabled work systems.
MIS Quarterly, 29(3), 525e557.
Jin, X.-L., Cheung, C. M. K., Lee, M. K. O., & Chen, H.-P. (2007). Understanding the
sustainability of virtual communities in China. In European conference on in-
formation systems (ECIS) 2007 proceedings (pp. 310e321). http://aisel.aisnet.org/
ecis2007/7.
Jin, X.-L., Cheung, C. M. K., Lee, M. K. O., & Chen, H.-P. (2009). How to keep members
using the information in a computer-supported social network. Computers in
Human Behavior, 25(5), 1172e1181. http://dx.doi.org/10.1016/j.chb.2009.04.008.
Jin, X.-L., Lee, M. K. O., & Cheung, C. M. K. (2010). Predicting continuance in online
communities: model development and empirical test. Behaviour & Information
Technology, 29(4), 383e394. http://dx.doi.org/10.1080/01449290903398190.
Kang, Y. S., Hong, S., & Lee, H. (2009). Exploring continued online service usage
behavior: the roles of self-image congruity and regret. Computers in Human
Behavior, 25(1), 111e122. http://dx.doi.org/10.1016/j.chb.2008.07.009.
Kang, Y. S., & Lee, H. (2010). Understanding the role of an IT artifact in online service
continuance: an extended perspective of user satisfaction. Computers in Human
Behavior, 26(3), 353e364. http://dx.doi.org/10.1016/j.chb.2009.11.006.
Karahanna, E., Straub, D. W., & Chervany, N. L. (1999). Information technology adop-
tion across time: a cross-sectional comparison of pre-adoption and post-adoption
beliefs. MIS Quarterly, 23(2), 183e213. http://dx.doi.org/10.2307/249751.
Kim, S. S., & Malhotra, N. K. (2005). A longitudinal model of continued IS use: an
integrated view of four mechanisms underlying postadoption phenomena.
Management Science, 51(5), 741e755. http://dx.doi.org/10.1287/
mnsc.1040.0326.
Kim, B., & Oh, J. (2011). The difference of determinants of acceptance and contin-
uance of mobile data services: a value perspective. Expert Systems with Appli-
cations, 38(3), 1798e1804. http://dx.doi.org/10.1016/j.eswa.2010.07.107.
Kukulska-Hulme, A. (2012). How should the higher education workforce adapt to
advancements in technology for teaching and learning? The Internet and Higher
Education, 15(4), 247e254. http://dx.doi.org/10.1016/j.iheduc.2011.12.002.
Kurt, S. (2012). How do teachers prioritize the adoption of technology in the
classroom? Teachers and Teaching, 18(2), 217e231. http://dx.doi.org/10.1080/
13540602.2012.632271.
Larsen, T. J., Sørebø, A. M., & Sørebø, Ø. (2009). The role of task-technology fit as
users’ motivation to continue information system use. Computers in Human
Behavior, 25(3), 778e784. http://dx.doi.org/10.1016/j.chb.2009.02.006.
Leclercq, A. (2007). The perceptual evaluation of information systems using the
construct of user satisfaction: case study of a large French group. The DATABASE
for Advances in Information Systems, 38(2), 27e60. http://dx.doi.org/10.1145/
 G. Daghan, B. Akkoyunlu / Computers in Human Behavior 60 (2016) 198e211
1240616.1240621.
Lee, M.-C. (2010). Explaining and predicting users’ continuance intention toward e-
learning: an extension of the expectationeconfirmation model. Computers &
Education, 54(2), 506e516. http://dx.doi.org/10.1016/j.compedu.2009.09.002.
Lee, B.-C., Yoon, J.-O., & Lee, I. (2009). Learners' acceptance of e-learning in South
Korea: theories and results. Computers & Education, 53(4), 1320e1329. http://
dx.doi.org/10.1016/j.compedu.2009.06.014.
Liao, C., Chen, J. L., & Yen, D. C. (2007). Theory of planning behavior (TPB) and
customer satisfaction in the continued use of e-services: an integrated model.
Computers in Human Behavior, 23(6), 2804e2822. http://dx.doi.org/10.1016/
j.chb.2006.05.006.
Liao, C., Palvia, P., & Chen, J.-L. (2009). Information technology adoption behavior
life cycle: toward a Technology Continuance Theory (TCT). International Journal
of Information Management, 29(4), 309e320. http://dx.doi.org/10.1016/
j.ijinfomgt.2009.03.004.
Li, Y., Duan, Y., Fu, Z., & Alford, P. (2011). An empirical study on behavioural intention
to reuse e-learning systems in rural China. British Journal of Educational Tech-
nology, 43(6), 933e948. http://dx.doi.org/10.1111/j.1467-8535.2011.01261.x.
Limayem, M., & Cheung, C. M. K. (2008). Understanding information systems
continuance: the case of Internet-based learning technologies. Information &
Management, 45(4), 227e232. http://dx.doi.org/10.1016/j.im.2008.02.005.
Limayem, M., & Cheung, C. M. K. (2011). Predicting the continued use of Internet-
based learning technologies: the role of habit. Behaviour & Information Tech-
nology, 30(1), 91e99. http://dx.doi.org/10.1080/0144929X.2010.490956.
Limayem, M., Hirt, S. G., & Cheung, C. M. K. (2007). How habit limits the predictive
power of intention: the case of information systems continuance. MIS Quarterly,
31(4), 705e737.
Lin, K.-M. (2011). e-Learning continuance intention: moderating effects of user e-
learning experience. Computers & Education, 56(2), 515e526. http://dx.doi.org/
10.1016/j.compedu.2010.09.017.
Lin, K.-M., Chen, N.-S., & Fang, K. (2011). Understanding e-learning continuance
intention: a negative critical incidents perspective. Behaviour & Information
Technology, 30(1), 77e89. http://dx.doi.org/10.1080/01449291003752948.
Lin, W.-S., & Wang, C.-H. (2012). Antecedences to continued intentions of adopting
e-learning system in blended learning instruction: a contingency framework
based on models of information system success and task-technology fit. Com-
puters & Education, 58(1), 88e99. http://dx.doi.org/10.1016/
j.compedu.2011.07.008.
Lin, C. S., Wu, S., & Tsai, R. J. (2005). Integrating perceived playfulness into
expectationeconfirmation model for web portal context. Information & Man-
agement, 42(5), 683e693. http://dx.doi.org/10.1016/j.im.2004.04.003.
Liu, I. L. B., Cheung, C. M. K., & Lee, M. K. O. (2010). Understanding Twitter usage:
what drive people continue to tweet. In Pacific Asia conference on information
systems (PACIS), Taipei, Taiwan (pp. 928e939).
Liu, S.-H., Liao, H.-L., & Pratt, J. A. (2009). Impact of media richness and flow on e-
learning technology acceptance. Computers & Education, 52(3), 599e607. http://
dx.doi.org/10.1016/j.compedu.2008.11.002.
Mandernach, B. J., Gonzales, R. M., & Garnett, A. L. (2006). An examination of online
instructor presence via threaded discussion participation. Journal of Online
Learning and Teaching, 2(4), 248e260. Retrieved from http://jolt.merlot.org/
index.html.
Marble, R. P. (2003). A system implementation study: management commitment to
project management. Information & Management, 41(1), 111e123. http://
dx.doi.org/10.1016/S0378-7206(03)00031-4.
Masters, K., & Oberprieler, G. (2004). Encouraging equitable online participation
through curriculum articulation. Computers & Education, 42(4), 319e332. http://
dx.doi.org/10.1016/j.compedu.2003.09.001.
McKavanagh, C., Kanes, C., Beven, F., Cunningham, A., & Choy, S. (2002). Evaluation
of web-based flexible learning (pp. 20e21). Adelaide: National Centre for Voca-
tional Education Research.
Meyers, L. S., Gamst, G., & Guarino, A. (2006). Applied multivariate research: Design
and interpretation. Thousand Oaks: Sage Publications.
Oliver, R. L. (1980). A cognitive model for the antecedents and consequences of
satisfaction. Journal of Marketing Research, 17(4), 460e469.
€
Oncü, S., & Çakır, H. (2011). Research in online learning environments: priorities and
methodologies. Computers & Education, 57(1), 1098e1108. http://dx.doi.org/
10.1016/j.compedu.2010.12.009.
Pala, F. K., & Erdem, M. (2011). O €g
renci yo€ netimli çevrimiçi tartışmalara katılımın
yo €netim sorumlulug u ve o €grenme stilleri açısından incelenmesi. Hacettepe
Üniversitesi Eg itim Fakültesi Dergisi, 41, 360e371.
Pallant, J. (2005). SPSS survival manual: A step by step guide to data analysis using
SPSS for windows (Version 12) (2nd ed.). New South Wales: Allen and Unwin.
Petter, S., DeLone, W. H., & McLean, E. R. (2008). Measuring information systems
success: models, dimensions, measures, and interrelationships. European Jour-
nal of Information Systems, 17(3), 236e263. http://dx.doi.org/10.1057/
ejis.2008.15.
Petter, S., DeLone, W. H., & McLean, E. R. (2012). The past, present, and future of “IS
Success” [Special issue] Journal of the Association for Information Systems, 13(5),
341e362.
Petter, S., DeLone, W. H., & McLean, E. R. (2013). Information systems success: the
quest for the independent variables. Journal of Management Information Sys-
tems, 29(4), 7e61. http://dx.doi.org/10.2753/MIS0742-1222290401.
Pituch, K. A., & Lee, Y.-K. (2006). The influence of system characteristics on e-
learning use. Computers & Education, 47(2), 222e244. http://dx.doi.org/10.1016/
211
j.compedu.2004.10.007.
Ramayah, T., Ahmad, N. H., & Lo, M.-C. (2010). The role of quality factors in intention
to continue using an e-learning system in Malaysia. Procedia Social and
Behavioral Sciences, 2(2), 5422e5426. http://dx.doi.org/10.1016/
j.sbspro.2010.03.885.
Recker, J. (2010). Explaining usage of process modeling grammars: comparing three
theoretical models in the study of two grammars. Information & Management,
47(5e6), 316e324. http://dx.doi.org/10.1016/j.im.2010.06.006.
Roca, J. C., Chiu, C. M., & Martínez, F. J. (2006). Understanding e-learning continu-
ance intention: an extension of the Technology Acceptance Model. International
Journal of HumaneComputer Studies, 64(8), 683e696. http://dx.doi.org/10.1016/
j.ijhcs.2006.01.003.
Roca, J. C., & Gagne, M. (2008). Understanding e-learning continuance intention in
the workplace: a self-determination theory perspective. Computers in Human
Behavior, 24(4), 1585e1604. http://dx.doi.org/10.1016/j.chb.2007.06.001.
Sabherwal, R., Jeyaraj, A., & Chowa, C. (2006). Information systems success: indi-
vidual and organizational determinants. Management Science, 52(12),
1849e1864. http://dx.doi.org/10.1287/mnsc.1060.0583.
Schumacker, R. E., & Lomax, R. G. (2004). A beginners’ guide to structural equation
modeling. Mahwah, NJ: Lawrance Erlbaum Associates.
Seddon, P. B. (1997). A respecification and extension of the DeLone and McLean
model of IS success. Information Systems Research, 8(3), 240e253. http://
dx.doi.org/10.1287/isre.8.3.240.
Shana, Z. (2009). Learning with technology: using discussion forums to augment a
traditional-style class. Educational Technology & Society, 12(3), 214e228.
Sharma, S. K., Joshi, A., & Sharma, H. (2016). A multi-analytical approach to predict
the Facebook usage in higher education. Computers in Human Behavior, 55(Part
A), 340e353. http://dx.doi.org/10.1016/j.chb.2015.09.020.
Shaw, R.-S. (2012). A study of the relationships among learning styles, participation
types, and performance in programming language learning supported by online
forums. Computers & Education, 58(1), 111e120. http://dx.doi.org/10.1016/
j.compedu.2011.08.013.
Shiau, W.-L., & Chau, P. Y. K. (2012). Understanding blog continuance: a model
comparison approach. Industrial Management & Data Systems, 112(4), 663e682.
http://dx.doi.org/10.1108/02635571211225530.
Shih, H.-P. (2008). Continued use of a Chinese online portal: an empirical study.
Behaviour & Information Technology, 27(3), 201e209. http://dx.doi.org/10.1080/
01449290600802403.
Shukor, N. A., Tasira, Z., & Van der Meijden, H. (2015). An examination of online
learning effectiveness using data mining. Procedia - Social and Behavioral Sci-
ences, 172, 555e562. http://dx.doi.org/10.1016/j.sbspro.2015.01.402.
Sørebø, Ø., Halvari, H., Gulli, V. F., & Kristiansen, R. (2009). The role of self-
determination theory in explaining teachers’ motivation to continue to use e-
learning technology. Computers & Education, 53(4), 1177e1187. http://
dx.doi.org/10.1016/j.compedu.2009.06.001.
Sutton, L. A. (2001). The principle of vicarious interaction in computer-mediated
communications. International Journal of Educational Telecommunications, 7(3),
223e242. Retrieved from http://www.editlib.org/j/IJET.
Symeonides, R., & Childs, C. (2015). The personal experience of online learning: an
interpretative phenomenological analysis. Computers in Human Behavior, 51,
539e545. http://dx.doi.org/10.1016/j.chb.2015.05.015. October 2015.
Terzis, V., Moridis, C. N., & Economides, A. A. (2013). Continuance acceptance of
computer based assessment through the integration of user's expectations and
perceptions. Computers & Education, 62, 50e61. http://dx.doi.org/10.1016/
j.compedu.2012.10.018. March 2003.
Thong, J. Y. L., Hong, S.-J., & Tam, K. Y. (2006). The effects of post-adoption beliefs on
the expectation-confirmation model for information technology continuance.
International Journal of Human-Computer Studies, 64(9), 799e810. http://
dx.doi.org/10.1016/j.ijhcs.2006.05.001.
Vanderlinde, R., & van Braak, J. (2011). A new ICT curriculum for primary education
in flanders: defining and predicting teachers' perceptions of innovation attri-
butes. Educational Technology & Society, 14(2), 124e135.
Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology
acceptance model: four longitudinal field studies. Management Science, 46(2),
186e204. http://dx.doi.org/10.1287/mnsc.46.2.186.11926.
Venkatesh, V., Morris, M. G., Davis, F. D., & Davis, G. B. (2003). User acceptance of
information technology: toward a unified view. MIS Quarterly, 27(3), 425e478.
Venkatesh, V., Thong, J. Y. L., Chan, F. K. Y., Hu, P. J.-H., & Brown, S. A. (2011).
Extending the two-stage information systems continuance model: incorpo-
rating UTAUT predictors and the role of context. Information Systems Journal,
21(6), 527e555. http://dx.doi.org/10.1111/j.1365-2575.2011.00373.x.
Wang, H. C., & Chiu, Y. F. (2011). Assessing e-learning 2.0 system success. Computers &
Education, 57(2), 1790e1800. http://dx.doi.org/10.1016/j.compedu.2011.03.009.
Wang, W.-T., & Wang, C.-C. (2009). An empirical study of instructor adoption of
web-based learning systems. Computers & Education, 53(3), 761e774. http://
dx.doi.org/10.1016/j.compedu.2009.02.021.
You, J. W., & Kang, M. (2014). The role of academic emotions in the relationship
between perceived academic control and self-regulated learning in online
learning. Computers & Education, 77, 125e133. http://dx.doi.org/10.1016/j.com-
pedu.2014.04.018. August 2014.
Zhou, T. (2011). Understanding mobile Internet continuance usage from the per-
spectives of UTAUT and flow. Information Development, 27(3), 207e218. http://
dx.doi.org/10.1177/0266666911414596.