The role of emotions and task signicance in Virtual Education

Gwen Noteborn , Katerina Bohle Carbonell, Amber Dailey-Hebert, Wim Gijselaers

a b s t r a c t a r t i c l e i n f o
Article history:
Accepted 21 March 2012
Available online 29 March 2012
Keywords:
Academic emotions
Virtual learning environments
Second life
Simulation
Academic performance
Task signicance
This paper analyzed the role of emotions in a virtual world (Second Life) through students' level of enjoyment
and boredom and their inuence on students' achievement level. The virtual world was an educational tool
used to fully immerse students in the content of the course. In addition to supporting prior research on the
importance of task value on academic enjoyment, the current research provides a new perspective on the
relationship between academic emotions and academic success, particularly for virtual worlds. A regression
analysis was conducted to measure the relationship of task value and emotions on two types of academic
performance: Individual exam scores and team scores on their Second Life assignment. Pekrun's Academic
Emotions Questionnaire (AEQ) was used to measure two academic emotions: boredom and enjoyment. Both
academic emotions were measured on an individual level. Results from this study show that task value was
positively related to enjoyment and negatively related to boredom, yet it was unrelated to academic
performance. While enjoyment had a positive relationship to examperformance, boredom also had a positive
relationship to the team assignment conducted in the virtual world. The possibility that students might have
answered the AEQ relating to the theoretical aspects of the course instead of the practical aspects of the
Second Life Assignment, may be one possible explanation for this result.
2012 Elsevier Inc. All rights reserved.
Tell me, and I will forget. Show me, and I may remember. Involve me,
and I will understand. Confucius, 450 B.C.
1. Introduction
With the advent of emergent technologies, simulations can mimic
reality and provide opportunities to deliver learning experiences in a
realistic setting (Galvo, Martins, & Gomes, 2000). Simulations are
used in learning environments and offer a time-efcient way to reach
the desired level of performance, while simultaneously increasing the
performance level reached (Parush, Hamm, & Shtub, 2002). Simula-
tions not only copy the physical aspects of an environment, (such as
weather disturbances for pilot training), but can also incorporate
intangible aspects of that environment (such as stress or a breakdown
in collaboration between team members). In both cases, participant
actions determine the learning experience and, thus, the path of
simulation.
Virtual worlds represent a specic type of simulation, which
differs from other forms of simulation in the following ways: First,
virtual worlds have a persistent character (Cannon-Bowers & Bowers,
2008). This implies that action in the virtual world continues, even if
not all players are currently online. A second characteristic of virtual
worlds is that multiple learners, who are geographically dispersed,
can interact within the same environment (Cannon-Bowers &
Bowers, 2008). While research on synthetic learning environ-
ments has been conducted, further research on the usefulness of
virtual words in education is needed. As in virtual worlds, syn-
thetic learning environments provide a simulated learning experi-
ence through technology-enabled learning environments (Cannon-
Bowers & Bowers, 2009). The research in this broad eld of
technology-based learning environments, has shown that authen-
ticity of those environments leads to increased learning perfor-
mance (Honebein, Duffy, & Fishman, 1993; Petraglia, 1998). Next to
authenticity, the level of engagement and the immersion opportuni-
ties created in such environments are often cited as a possible
explanation to their effectiveness (Cannon-Bowers & Bowers, 2008;
Kozlowski & Bell, 2007). However, while these ndings are promis-
ing, they do not clearly depict how the emotional context relates to
the learning outcome (Cannon-Bowers & Bowers, 2008; Pekrun,
2005).
Research on emotional experiences in classroom settings has
grown in recent years, yet this strand of research remains sparse
regarding online learning environments. Studies have shown that
emotional experiences inuence a student's motivation, learning
strategies and achievement; and that such emotional experiences are
inuenced by personality and classroom characteristics (Goetz,
Pekrun, Hall, & Haag, 2006; Pekrun, Elliot, & Maier, 2009; Pekrun,
Goetz, Daniels, Stupnisky, & Perry, 2010; Pekrun, Goetz, Titz, & Perry,
2002). Jrvenoja and Jarvel (2005) and Wosnitza and Volet (2005)
indicate the important role of social emotions in technological
environments and found the nature of emotions to be based on
student's perceived control over the learning activity. Interestingly,
Artino (2009) demonstrates that negative emotional experiences
Internet and Higher Education 15 (2012) 176183
Corresponding author.
E-mail address: gcm.noteborn@maastrichtuniversity.nl (G. Noteborn).
1096-7516/$ see front matter 2012 Elsevier Inc. All rights reserved.
doi:10.1016/j.iheduc.2012.03.002
Contents lists available at SciVerse ScienceDirect
Internet and Higher Education
increase students' metacognitive activities, and reduce their satisfac-
tion level and motivation to continue with the course. These ndings
conrm aspects of the research on emotional experiences in ofine
course environments, yet point towards the different impacts of
boredom on learning activities in online learning environments.
The discussion above highlights the lack of research into emo-
tional experience in virtual worlds. While Pekrun has established a
benchmark related to the types of emotional experiences in class-
room settings, research on emotional experiences in online learning
environments requires further inquiry and research. To date, it
remains unclear how emotional experiences inuence performance
in virtual worlds. Therefore, the aim of this study is to analyze the
relationship between emotional experiences and student perfor-
mance in virtual worlds, and to discuss their implications.
2. Conceptual framework
2.1. Virtual worlds as an educational simulation tool
Simulation can be a concrete form of reality or an abstraction
(Sauv, Renaud, Kaufman, & Marquis, 2007). Virtual worlds (such as
Second Life) provide for an abstract form of reality, where learners
are not physically present but instead use technology to interact with
people and objects in the virtual world. The value of Second Life lies in
its capability to provide an authentic replication of reality (Kozlowski
& Bell, 2007; Sauv et al., 2007). Second Life is a system that attempts
to provide realistic environments, which incorporate a working
representation of reality where users log in via a web-based appli-
cation in a world that mimics reality. Communication takes place
synchronously (in real time) and is instantaneous, via text messaging
or voice chat; a concept being referred to as immediacy (Wood,
2010). Immediacy offers the opportunity for learners to gain direct
feedback or assistance from fellow learners or teachers. The realistic
setting results from various elements, such as an in world economy
including a currency system (Linden Dollars), resulting in an actual
economy with market data and the representation of numerous
multinational companies, national governments and other institutes.
In the case of Second Life, the setting and thereby the degree of
reality, are simplied (Galvo et al., 2000). Instructional designers
limit the various elements in Second Life to narrow overexposure for
students. Although the environment might be simplied, what is
being simulated is not limited to the physical aspect of the system,
(e.g., driving a car or ying an airplane) but can also incorporate the
underlying structure of the task or problem (e.g., the aspect of
competition or social interaction). A special feature of Second Life in
comparison to other virtual worlds is that it allows for multi-user
online role-playing. Furthermore, users can collaboratively create and
use in-world artifacts such as text, images, and three-dimensional
models, increasing a sense of community among players (Wood,
2010). This creates a persistent character, which implies that actions
continue and the world evolves irrelevant of the presence of specic
players who can enter the world at any desired time. By means of this
persistence Second Life mimics reality to an even further extent than
other virtual worlds allowing for complex interactions amongst
players and with the environment.
Research shows the importance of immersion (offered by Second
Life), as it enables players to gain an understanding of their future
work environment after graduation and allows players to experience
how elements of their future professional environment interact with
each other (Belei, Noteborn, & De Ruyter, 2011; Kozlowski & Bell,
2007). Through the authentic environment and unique characteristics
of Second Life, the gap between learning and transfer environ-
ment narrows and leads to increased student performance (Cannon-
Bowers & Bowers, 2008; Kozlowski & Bell, 2007). While it is
suggested that students are immersed while working in a virtual
world (Cannon-Bowers & Bowers, 2008), it remains unclear whether
emotion inuences the learning outcome of students in these
environments (Cannon-Bowers & Bowers, 2009).
2.2. Task value
An important predictor of learning is the perceived task value
students attach to the learning tasks. Task value is the degree to
which the quality of the task contributes to the probability of
potential participants selecting it or not (Eccles, 2005). Thus a task
is valued high if a learner perceives that it will advance his/her
understanding of the relevant domain. Eccles based this on the
expectancy-value theory of achievement and argues that task value is
composed of four value elements: Attainment value, intrinsic or
interest value, utility value and costs of engaging (Durik, Vida, &
Eccles, 2006). Attainment value refers to a person's value attached to
performing well on the given task. Intrinsic or interest value is the
perceived enjoyment a person can take out of this task. Utility value is
the degree to which the task contributes to an individual's long term
goal; whereas costs of engaging relate to the opportunity costs,
energy and other forms of costs the individual has to invest in while
engaging in the task (Eccles, 2005).
Research has shown task value to be related to academic success
(Durik et al., 2006; Simons, Dewitte, & Lens, 2004). This relationship
is mainly grounded in the utility component of task value (Simons
et al., 2004). Therefore, to attribute a high utility value, students
must comprehend the rationale for engaging in a specic task and
understand the added value it offers to their professional de-
velopment (Debnath, Tandon, & Pointer, 2007). A study conducted
by Hulleman, Durik, Schweigert, and Harackiewicz (2008) used a
classroom setting and a high school sports camp to demonstrate how
task value positively relates to performance. Furthermore, Artino, La
Rochelle, and Durning (2010) showed task value to be positively
related to enjoyment and negatively correlated to boredom. How-
ever, based on this research, there was no indication that task value
related to academic performance. Studies investigating task value in
technology-based learning are still rare. For example, Chiu and Wang
(2008) have investigated perceived task value for professionals with
regard to continuing a web-based course. In other studies, task value
has been shown to positively relate to satisfaction of online learners
(Artino, 2009) and to their cognitive and metacognitive learning
activities (Artino & Stephens, 2007). Nevertheless, Cannon-Bowers
and Bowers (2009) call for more research on the impact of task value
in technology based learning environments, and further inquiry is
needed to expand the foundation of knowledge on this topic.
2.3. Role of emotions for achievements
Emotions are dened as subjective experiences which are depen-
dent on the context in which they arise (Linnenbrink, 2006). Goetz,
Cronjaeger, Frenzel, Ldtke, and Hall (2010) have shown that the
emotions experienced by students, such as boredom and enjoyment,
signicantly differ across domains. Research has shown that positive
emotions, such as enjoyment, facilitate the recovery of positive material
(Isen, 1990), whereas negative emotions reduce performance (Beier &
Kanfer, 2009). The amount of positive or negative emotions students
experience, while engaged in learning tasks, inuences their goal and
thereby their learning process (Bruinsma, 2004; Kay, 2008).
Emotions are experienced in various situations. Achievement
emotion is one specic type of emotion (Pekrun, 2006). Achievement
emotions are directly tied to the achievement activity itself (e.g.
writing an exam) or the outcome associated with the activity (e.g. the
exam grade) (Pekrun, 2006). According to Pekrun, Goetz, Frenzel,
Barchfeld, and Perry (2011), achievement emotions inuence student
learning and performance, yet are induced by feelings of control over
activities that students perceive as important; in other words, those
activities that have a high task value. Control-value theory states that
177 G. Noteborn et al. / Internet and Higher Education 15 (2012) 176183
the source of student's emotion is their subjective control and their
subjective values (Pekrun, 2000). Subjective control refers to the
perceived control a student believes to have over one's self. Sub-
jective value is the importance a student attaches to the achievement
outcome and activities. Subjective value and control inuence the
valence of emotions and their activating effect on students' level of
motivation (Pekrun et al., 2002). Those two characteristics, valence
and activation, are used to classify emotions as positive or negative
and as engaging or disengaging students towards the activity.
Positive-activating emotions, such as enjoyment, have been found to
positively relate to the use of deep-level cognitive learning strategies;
whereas negative-deactivating emotions, such as boredom, function
as a motivational barrier, interfering with students' ability to learn in
academic settings. Pekrun et al. (2010) analyzed the degree of task
achievement control on boredom. Their results have shown that
students, who perceive little control over (or assign little value to)
the relevance of their learning activity, experience higher levels of
boredom.
Recent research on emotional experiences in online courses has
shown that boredom and frustration negatively inuence students'
level of satisfaction, and motivation to continue with the course
(Artino, 2009). Yet interestingly, while boredom had a negative
impact on metacognitive learning strategies, frustration positively
inuenced them. Findings by Wosnitza and Volet (2005) provide a
possible explanation for this phenomenon. Frustration in online
learning environments can be directed at the task, one's self and the
technology. Artino (2009) speculates that students experienced self-
related frustration, which motivated them to use more metacognitive
control strategies to aid their understanding indicating a possible
correlation between frustration and motivation toward metacogni-
tive control strategies.
As described above, task value is based on the expectancy-value
model (Eccles, 2005). Therefore, the value students attach to a task is
inuenced by the degree to which they believe the task applies to their
career. Pekrun (2006) and Pekrun et al. (2011) also dene achieve-
ment emotions as having an inuence on academic achievement,
which are induced by feelings of control over activities that students
perceive as important. This is based on the control-value theory which
states that students' perceived level of control, and value related to the
activity and outcome, inuence their performance level. While Artino
et al. (2010) have shown that task value inuences positive and
negative emotions in ofine courses, this relationship has not yet been
analyzed in virtual learning environments.
Fig. 1 summarizes the relationships analyzed in this study.
These relationships connect the concept of task value to performance
via emotional experience. As mentioned above, task value is said
to inuence achievement, as the value students attach to a task
determines, in part, their level of performance. This relationship is
mediated by the emotions students experience towards the course.
Emotional experiences are related to the control students perceive to
have over a task and its outcome. Thus, a student perceiving high task
value, but low control over the learning activity, will not achieve a
high outcome. This is due to the experience of negative emotions
towards the task, reducing the performance level. As a result of this
information and the conceptual framework previously shared, the
following research question is explored:
To what extent does the level of task value relate to student's
achievement outcome and is this relationship mediated by the emotions
students experience while engaged in course-related activities?
3. Method
3.1. Setting
We conducted the study in a third year undergraduate Brand
Management course offered at a large European university. The Brand
Management course was an elective course that was conducted
over eight weeks within the international business degree program.
Students were randomly assigned to small groups where they
discussed relevant literature. In addition to this, students formed
small teams within those groups and collaborated on a practical
assignment referred to as the Second Life assignment. Student
teams developed, promoted and eventually sold a product within the
virtual environment of Second Life. Three product types, in the fast
moving consumer goods section, were dened and student teams
were equally distributed amongst these categories. By creating
predened categories, competition between student teams heavily
increased and provided a closer simulation of reality, with regard to
competition between companies. As in real life, people who worked
for a company were also buyers. Therefore, towards the end of the
course, students were individually given 20 Linden Dollars to buy
products created by fellow students of their course. During this
buying activity, the students had to indicate why they chose to buy a
certain product.
3.2. Sample
The sample included 155 international business students who
followed the course. The study participants were randomly assigned
into groups for the literature discussion. These groups consisted of 13
students per group, with a total of 12 groups in the course. For the
Second Life assignment, students formed teams (of four to ve
students, from each of the 12 pre-dened groups); which resulted in
33 teams. This team size was chosen as it yields the highest group
performance outcome (Lou, Abrami, & D'Apollonia, 2001).
With 139 students who responded to the survey, a response rate
of 81 % was achieved. Thirty teams were included in the team
analysis, with incomplete data for three teams. While 86 % of the
students who lled out the survey followed a major or minor in
marketing. Their mean age was 21.45 years (SD=2.44). 53 % of the
sample was female.
3.3. Procedure
Data collection for academic emotions and task value was
completed after students had the opportunity to spend their 20
Linden Dollars, but before they had to complete the nal exam. In this
way, the performance of the exam, or the grade they received, did not
bias their emotional experience. In addition, by choosing this time
frame, students were still immersed in the course and had a better
recollection of their emotional experiences and task value beliefs.
Students received a link to the survey via an announcement in their
online learning platform and in Second Life. The introduction text to
the survey clearly stated that the data gathered would be condential.
Professors informed students about the survey in the small groups
and a reminder was sent two days following. Performance data was
collected upon conclusion and completion of the course.
3.4. Measures
3.4.1. Academic emotions
Enjoyment and boredom were measured with the enjoyment and
boredom scales of the Achievement Emotions Questionnaire (AEQ),
shown to have good reliability and validity in a variety of educational
settings (Pekrun, Goetz, & Frenzel, 2005). The AEQ, a self-report
questionnaire with good psychometric qualities, also proved useful in
assessing linkages between academic emotions and students' learn-
ing and academic achievement. Two scales of this questionnaire were
used, namely the enjoyment scale (10 items) and boredom scale
(11 items). Example items for the enjoyment scale include I look
forward to studying for this course, I enjoy dealing with this course
material and Reecting on my progress in this course work makes
178 G. Noteborn et al. / Internet and Higher Education 15 (2012) 176183
me happy. Examples of items for the boredom scale include
Studying this course bores me, This course material is so boring
that I nd myself daydreaming and I would rather put off this
boring course work till tomorrow. The words this course were
replaced with the name of the specic course, however, no other
adaptions were made. A 7-point Likert scale was used, ranging from 1
Strongly Disagree to 7 Strongly Agree.
3.4.2. Task value
Task value was measured through the validated task value scale
(6 items) of the Motivated Strategies for Learning Questionnaire
(MSLQ) (Pintrich, Smith, Garcia, & McKeachie, 1991). The MSLQ is an
instrument measuring students' motivation for studying at the
university. Sample items for the task value scale include, It is very
important for me to learn the course material in this class, I think
the course material in this class is useful for me to learn and
Understanding the subject matter of this course is very important to
me. The items were not adapted in any way. As for the emotion
scales, a 7-point Likert scale was used ranging from 1 Strongly
Disagree to 7 Strongly Agree.
3.4.3. Academic performance
Study performance was measured by two indicators: exam grade
and Second Life grade. Both performance indicators were measured
on a 10-point scale, with 5.5 as the passing rate. The outcome variable
exam grade was measured through standardized test scores on the
nal written exam. For the written exam, students had three hours to
answer 5 essay questions. They were not allowed to use any course
materials and the exam was administered in the university's su-
pervised assessment center. The second performance measurement,
the Second Life grade, was based on a group performance indicator,
comprising sales revenue, peer feedback, argumentation for chosen
branding strategies, creativity and innovativeness. At the end of the
course, evaluation of the Second Life project was based on student
presentations which shared their chosen branding strategy. The
following stakeholder groups were involved in the evaluation
process: Four university professors who lectured for the course and
an expert jury. The expert jury consisted of four full professors in
marketing and two brand managers of a multinational company
operating in the fast moving consumer goods market. They used an
evaluation form to provide the grade and a justication for it. The
nal Second Life grade was equally weighted between the averages of
the four juries. In using these two performance indicators, a
difference was made between applying the theoretical concepts on
a hypothetical written case and between applying the theoretical
concepts to a real life assignment in a virtual setting. The nal grade
for the course was composed of 40 % exam grade, 30 % Second Life
team grade and 30 % individual participation during small group
sessions.
3.5. Analysis
Two regression analyses were conducted, with one performance
indicator measured on an individual level and the other performance
indicator on a team level. To conduct the team level analysis,
calculations of the mean team values for enjoyment, boredom and
task value were used. While the variables were measured on an
individual level, inter-rater reliability coefcients (IRR) were still
calculated to assess whether they could be aggregated on a team
level. The procedures, as outlined in LeBreton and Senter (2007),
were followed.
First, we calculated the internal reliability of the individual scales
to conduct a descriptive analysis. Following this, the BaronKenny 4-
step mediation model was applied (Baron & Kenny, 1986). The
mediation model contained three regression analyses, and a test to
assess full or partial mediation. The rst model measured whether
task value inuenced the outcome variable. In the second model, task
value was regressed against the two mediators, enjoyment and
boredom. The third model analyzed whether the mediators would
inuence the nal outcome variable while controlling for task value.
And nally, the third model analyzed whether the effect of task value
on performance was zero when the mediator variables were added.
The same steps were conducted on a team level. On the individual
analysis, the exam grade was the dependent variable. For the team
analysis, the same regression steps were followed, but the scores for
the variables were aggregated on a team level. Here, the Second Life
grade was the dependent variable, as this performance indicator was
given per team (not on an individual basis).
4. Results
The results of Table 1 show the mean, standard deviation,
Cronbach's alpha and correlation values on the individual level.
The Cronbach's alpha for the three instruments were above .90
indicating high reliability of the chosen instruments. The mean of task
value, enjoyment and boredom were 5.92 (SD=.80), 4.97 (SD=.96)
and 2.53 (SD=1.15) respectively. The average exam grade was 6.21
(SD=1.16) and the average team grade for the Second Life assign-
ment was 7.63 (SD=.75), and both grades were measured on a 10-
point scale.
The correlation table shows that task value was positively correlated
with enjoyment (.61, pb.001) and negatively with boredom (.46,
Task Value
Boredom
Enjoyment
Grade
Fig. 1. Mediation model of emotions mediating the relationship between task value
and student performance.
Table 1
Mean, SD, Cronbach alpha and correlation analysis on the individual level.
Variable Mean SD Alpha Enjoyment Boredom Exam grade Second Life grade
Task value 5.91 .80 .90 .61

.46

.14 .10
Enjoyment 4.97 .96 .91 .64

.20

.05
Boredom 2.53 1.15 .96 .08 .23

Exam grade 6.21 1.16 .11

Second Life grade 7.63 .75
Note: N=139 students.
p-valueb.05 (two-tailed).
p-valueb.01 (two-tailed).
p-valueb.001 (two-tailed).
179 G. Noteborn et al. / Internet and Higher Education 15 (2012) 176183
pb.001) and was not correlated with any of the performance indicators.
Enjoyment was negatively correlated with boredom (.64, pb.001)
and slightly positively correlated with the exam grade (.20, pb.05), but
not with the Second Life grade. Boredomwas positively correlated with
the Second Life grade (.23, pb.01), but was not correlated with the
exam grade. The performance indicators were not correlated with each
other.
Table 2 presents the result of the mediation analysis on the
individual level. The results of model 1 show that task value did not
inuence the exam grade. In the second model task value signi-
cantly inuenced the measured emotions: there was a positive effect
on enjoyment (=.61, pb.001) and a negative effect on boredom
(=.46, pb.001). In the third model, enjoyment inuenced the
exam grade (=.23, pb.10), but boredom and task value did not
signicantly inuence the exam grade. Consequently, since the rst
step in the mediation analysis provided a non-signicant result, the
mediation model could not be supported.
In order to analyze the impact of academic emotions on the
performance in the Second Life assignment itself, the correlation and
regression analyses were conducted on a team level. Tables 3 and 4
show the results on a team level. The individual scores for task value,
boredom and enjoyment were aggregated on a team level. The
performance indicator, the grade for the Second Life assignment, was
given per team.
Table 3 presents the descriptive and correlation analysis with the
team, as a unit of analysis. The mean of the inter-rater reliability
coefcients for task value and enjoyment was above .70, indicating the
values can be aggregated at the team level. The mean reliability
coefcient for boredom was .66, slightly below the cut-off point for
aggregating values at the team level. Although the interrater reliability
coefcient for team boredom is slightly below the acceptance level of
.70, the results of this analysis provide aninsight inthe relationbetween
boredom and academic performance. The values were aggregated, as
the construct itself was measured, at an individual level.
The mean team values for task value, enjoyment and boredom
were 5.92 (SD=.46), 4.96 (SD=.61) and 2.52 (SD=.75). The mean
team grade for exams was 6.023 (SD=1.22) and the Second Life
grade was 7.6 (SD=.72).
On the team level, task value was positively and signicantly
correlated with enjoyment (.79, pb.001) and negatively with bore-
dom (.64, pb.001). It was not correlated with any of the per-
formance indicators. Team enjoyment was negatively correlated with
team boredom (.74, pb.001), but was not correlated with either of
the performance indicators. Boredom was not signicantly correlated
with the performance indicators, and the team exam grade and
Second Life grade were not signicantly correlated with each other.
Table 4 presents the result of the mediation analysis on the team
level. The rst model in the mediation analysis shows that team task
value did not inuence the Second Life grade. In the second model,
task value positively enjoyment (=. 79, pb.001) and negatively
inuenced boredom (=.64, pb.001). The third model shows
that, of all the predictors, only boredom signicantly inuenced the
Second Life grade (=.60, pb.05). As task value did not inuence the
outcome variable, the mediation model could not be conrmed. The
regression analysis on the team level indicated a slightly different
result than of the individual analysis. The impact of enjoyment was
non-signicant, whereas the impact of boredom on the Second Life
grade was signicant and positive (=. 60, pb.05).
Table 2
Summary of hierarchical regression analysis for task value predicting individual performance, mediated by perceived emotions.
Variable Exam grade Enjoyment Boredom
B SE B SE B SE
Step 1
Task value .20 .13 .14
Model summary R
2
=.01, p=.12
Step 2
Task value .73 .08 .61

.66 .11 .46

Model summary R
2
=.37, pb.001 R
2
=.21, pb.001
Step 3
Task value .05 .16

.03

Enjoyment .29 .16 .23

Boredom .09 .12 .09

Model summary R
2
=.02, p=.13
Notes: N=139 students. The p-value for enjoyment in model 3 is .08, thus approaching signicance.

p-valueb.10.
p-valueb.05.
p-valueb.01.
p-valueb.001
Table 3
Mean, SD, IRR, and correlation analysis on the team level.
Variable Mean SD Interrater reliability
(IRR) score
Enjoyment Boredom Exam grade Second Life grade
Task value 5.92 .46 .82 .79

.64

.20 .16
Enjoyment 4.96 .61 .77 .74

.31

.06

Boredom 2.52 .75 .66 .23 .33

Exam grade 6.02 1.22 .11
Second Life grade 7.60 .72
Note: N=30 teams.
p-valueb.05 (two-tailed).
p-valueb.01 (two-tailed).
p-valueb.001 (two-tailed).
180 G. Noteborn et al. / Internet and Higher Education 15 (2012) 176183
5. Discussion
Fig. 2 summarizes results of the individual regression analysis, and
Fig. 3 summarizes results of the teamlevel analysis. Both gures show
that task value does not inuence the nal achievement outcome.
Hence, in the study, the emotions students experienced did not act
as a mediator between task value and achievement outcome.
The second step in the analysis shows that task value does
inuence the emotions experienced by students, which can be seen
in Figs. 2 and 3. Perceived task value is based on a number of per-
ceptions, such as the derived value for students' long-term goals, the
performance they want to achieve and the cost related to performing
the task (Eccles, 2005). The result of the regression analysis dem-
onstrates that students perceive the task to be valuable and, through
this, experience positive and activating emotions towards engaging in
the task. It should be mentioned though that, while task value and
control-value theory mention the importance of perceived control, the
present study did not investigate student's perceived level of control
over the achievement outcome and activity.
The regression results of model 3 in the mediation analysis paint an
interesting picture. Fig. 2 shows that, on the individual level, enjoyment
inuences the performance on the exam grade; demonstrating that
positive and activating emotions lead to higher performance.
Boredom, both individual and aggregated on a team level, in-
uences the Second Life grade. However the interrater reliability
coefcient for team boredom is slightly below the acceptable 0.7 level
and thus, interpreting the behavior of the boredom factor needs to be
done with caution. It might be that aggregating this score at the team
level can provide over- or underestimates of the coefcients. Our
research shows that individual boredom is correlated with the
Second Life grade. This indicates that there is a relationship between
individual boredom and the team assignment. A possible explanation
could be that boredom measures individual boredom with theoretical
course elements (e.g., studying for class), whereas to perform well on
the team assignments students need to engage in more practical
assignments. Even more, students reporting boredom in relation to
the theoretical course elements might put more emphasis on the
practical aspects (the Second Life assignment) of the course, which in
turn has an inuence on their grade. A second option is that students
attribute unequal weights to course elements between boredom and
enjoyment. In other words, while students refer to all course ele-
ments when relating to enjoyment, they only relate to the theoretical
course element when reporting boredom. Several reasons for this
result may exist; however, further research is needed to investigate
this area.
6. Limitations and future research
There are important limitations that should be considered when
interpreting the results. First, the reader should consider that the
learning activity was obligatory within the course and, thus, reduced
the students' perceived level of control over the task; resulting in the
potential to bias their responses, as they are obliged to execute this
task. Through this, a ceiling effect for enjoyment and a oor effect for
boredom could be observed. Secondly, the construct of boredom
measured in this study demonstrates that boredom can relate to
many different aspects of the learning activities.
Thirdly, several of the statistically signicant relationships were
only signicant at the pb.10 level. A more conclusive result may have
been achieved with a larger sample.
Future research should investigate whether the proposed expla-
nation provided for the ndings on the inuence of boredom on the
.60*
Task Value
Boredom
Enjoyment
Grade
.79***
.64***
Fig. 3. Results of the team mediation model of emotions mediating the relationship
between task value and student Second Life grade.
Task Value
Boredom
Enjoyment
Grade
.61**
.46**
.23
Fig. 2. Results of the individual mediation model of emotions mediating the
relationship between task value and student exam grade.
Table 4
Summary of hierarchical regression analysis for task value predicting team performance, mediated by perceived emotions.
Variable Second Life grade Enjoyment Boredom
B SE B SE B SE
Step 1
Task value .25 .29 .16
Model summary R
2
=.01, p=.402
Step 2
Task value 1.04 .16 .79

1.04

.23

.64

Model summary R
2
=.60, pb.001 R
2
=.39, pb.001
Step 3
Task value .30

.45 .19
Enjoyment .63 .39 .53
Boredom .58 .26 .60

Model summary R
2
=.10, p=.13
Note: N=30 teams.

p-valueb.10.
p-valueb.05.
p-valueb.01.
p-valueb.001.
181 G. Noteborn et al. / Internet and Higher Education 15 (2012) 176183
Second Life grade holds. Using an academic emotion scale that
specically addresses the feelings experienced, in relation to studying
theory and applying theory, could be applied. A more nuanced
boredom scale, emphasizing on the simulation aspect, for online
courses could help to clarify this relationship. Also attention should
be paid to the evaluation of emotions experienced in relation to the
course and the weight attributed to specic course elements. In
addition, further research into emotions experienced at a team level
should provide more insights for coaching students in team level
assignments. Research should pay attention towards distinguishing
emotions experienced during the performance of technical tasks
in online learning environments and tasks related to acquiring and
applying knowledge. The theory behind task value and emotions
suggests that those two factors should interact with each other and
inuence student's achievement, further research is needed to de-
termine the relationship.
7. Conclusion
Research to date (Pekrun, 2005; Pekrun et al., 2011) points toward
the importance of emotions in online learning environments, yet does
not clearly state the source neither the inuence on achievement
(Pekrun, 2005). While prior studies in classroom settings have
demonstrated the importance of emotions for students' achievement
(Pekrun et al., 2009; Pekrun et al., 2002) this relationship was only
recently investigated in online learning environments, yet not in
virtual worlds, which are based on simulation and distinguish itself
by persistence and immediacy. The ability of virtual worlds to
immerse students in the learning activity, and through this, create
an emotional experience, was an important factor to consider in
learning (Cannon-Bowers & Bowers, 2009).
The present study analyzed the role of emotions in virtual worlds
through students' level of enjoyment and boredom and their in-
uence on students' achievement level. In addition to supporting
prior research on the importance of task value on academic
enjoyment (Artino et al., 2010; Kirschner, Strijbos, Kreijns, & Beers,
2004), the current research provides a new perspective on the
relationship between academic emotions and academic success,
particularly for virtual worlds. The study conrmed prior ndings
about the importance of task value for emotional experiences, which
subsequently inuenced students' performance and thus their level of
learning.
When referring to our results, individual enjoyment, as hypoth-
esized, had a positive inuence on the exam grade, but boredom also
had a positive impact on the Second Life grade. The present study
indicated that students may experience high levels of boredom, yet
still achieved high levels of academic performance on practical
assignments. If measuring emotions within a course, future research
should clearly make a distinction between the course as such, and
individual course elements which may contribute to achievement. In
this case, a distinction should be made between the emotions
experienced when executing the practical assignment in Second Life
in comparison to studying the theoretical aspects of the course.
Developing a new AEQ especially for online simulation assignments
could be a solution. More research is needed in this area.
References
Artino, A. R. (2009). Online learning: Are subjective perceptions of instructional
context related to academic success? The Internet and Higher Education, 12(34),
117125. doi:10.1016/j.iheduc.2009.07.003.
Artino, A. R., La Rochelle, J. S., & Durning, S. J. (2010). Secondyear medical students'
motivational beliefs, emotions, and achievement. Medical education, 44(12),
12031212 Wiley Online Library.
Artino, A. R., & Stephens, J. M. (2007). Motivation and self-regulation in online courses: A
comparative analysis of undergraduate and graduate students. Annual meeting of the
Association for Educational Communications and Technology, Anaheim, CA.
Baron, R. M., & Kenny, D. A. (1986). The moderatormediator variable distinction in
social psychological research: Conceptual, strategic, and statistical considerations.
Journal of Personality and Social Psychology, 51(6), 1173 American Psychological
Association.
Beier, M. E., & Kanfer, R. (2009). Motivation in training and development: A phase
perspective. Learning, training, and development in organizations (pp. 65). : Psychology
Press.
Belei, N., Noteborn, G. C. M., & De Ruyter, K. (2011). It's a brand new world: Teaching
brand management in virtual environments. June 2011. Journal of Brand
Management, 18(8), 611623.
Bruinsma, M. (2004). Motivation, cognitive processing and achievement in higher education.
Learning and Instruction, 14(6), 549568. doi:10.1016/j.learninstruc.2004.09.001.
Cannon-Bowers, J. A., & Bowers, C. A. (2008). Synthetic learning environments. Handbook
of research on educational communications and technology (pp. 318327). : Routledge.
Cannon-Bowers, J. A., & Bowers, C. A. (2009). Synthetic learning environments: On
developing a science of simulation, games, and virtual worlds for training. In S. W.
J. Kozlowski, & E. Salas (Eds.), Learning, training, and development in organizations
(pp. 229262). New York: Routledge.
Chiu, C., & Wang, E. (2008). Understanding Web-based learning continuance intention:
The role of subjective task value. Information Management, 45(3), 194201. doi:
10.1016/j.im.2008.02.003.
Debnath, S. C., Tandon, S., & Pointer, L. V. (2007). Designing business school courses to
promote student motivation: An application of the job characteristics model. Journal
of Management Education, 31(6), 812831. doi:10.1177/1052562906290914.
Durik, A. M., Vida, M., & Eccles, J. S. (2006). Task values and ability beliefs as predictors
of high school literacy choices: A developmental analysis. Journal of Educational
Psychology, 98(2), 382393. doi:10.1037/0022-0663.98.2.382.
Eccles, J. S. (2005). Subjective task value and the Eccles et al. model of achievement-
related choices. In A. J. Elliot, & C. S. Dweck (Eds.), Handbook of competence and
motivation (pp. 105121). New York: The Guilford Press.
Galvo, J. R., Martins, P. G., & Gomes, M. R. (2000). Modeling reality with simulation
games for a cooperative learning. Proceedings of the 32nd conference on Winter
simulation (pp. 16921698). San Diego, CA, USA: Society for Computer Simulation
International Retrieved from http://dl.acm.org/citation.cfm?id=510378.510626.
Goetz, T., Cronjaeger, H., Frenzel, A. C., Ldtke, O., & Hall, N. C. (2010). Academic self-
concept and emotion relations: Domain specicity and age effects. Contemporary
Educational Psychology, 35(1), 4458. doi:10.1016/j.cedpsych.2009.10.001.
Goetz, T., Pekrun, R., Hall, N., & Haag, L. (2006). Academic emotions from a social-
cognitive perspective: antecedents and domain specicity of students' affect in the
context of Latin instruction. The British Journal of Educational Psychology, 76(Pt 2),
289308. doi:10.1348/000709905X42860.
Honebein, P. C., Duffy, T. M., & Fishman, B. J. (1993). Constructivism and the design of
learning environments: Context and authentic activities for learning. In T. M. Duffy,
J. Lowyck, & D. H. Jonassen (Eds.), Designing environments for constructive learning,
105. (pp. 87108)New-York: Springer Verlag.
Hulleman, C. S., Durik, A. M., Schweigert, S. B., & Harackiewicz, J. M. (2008). Task values,
achievement goals, and interest: An integrative analysis. Journal of Educational
Psychology, 100(2), 398416. doi:10.1037/0022-0663.100.2.398.
Isen, A. M. (1990). The inuence of positive and negative affect on cognitive
organization: Some implications for development. Psychological and biological
approaches to emotion, 7594.
Jrvenoja, H., & Jarvel, S. (2005). How students describe the sources of their emotional
and motivational experiences during the learning process: A qualitative approach.
Learning and Instruction, 15(5), 465480. doi:10.1016/j.learninstruc.2005.07.012.
Kay, R. (2008). Exploring the relationship between emotions and the acquisition
of computer knowledge. Computers in Education, 50(4), 12691283. doi:10.1016/
j.compedu.2006.12.002.
Kirschner, P. A., Strijbos, J. -W., Kreijns, K., & Beers, P. (2004). Designing electronic
collaborative learning environments. Educational Technology Research and Devel-
opment, 52(3), 4766. doi:10.1007/BF02504675%.
Kozlowski, S. W. J., & Bell, B. (2007). A theory-based approach for designing
distributed learning systems. In S. M. Fiore, & E. Salas (Eds.), Toward a science
of distributed learning (pp. 1539). Washington, DC: American Psychological
Association.
LeBreton, J. M., & Senter, J. L. (2007). Answers to 20 questions about interrater
reliability and interrater agreement. Organizational Research Methods, 11(4),
815852. doi:10.1177/1094428106296642.
Linnenbrink, E. A. (2006). Emotion research in education: theoretical and methodo-
logical perspectives on the integration of affect, motivation, and cognition.
Educational Psychology Review, 18(4), 307314. doi:10.1007/s10648-006-9028-x.
Lou, Y., Abrami, P. C., & D'Apollonia, S. (2001). Small group and individual learning with
technology: A meta-analysis. Sage Publications. Review of Educational Research,
71(3), 449521. doi:10.3102/00346543071003449.
Parush, A., Hamm, H., & Shtub, A. (2002). Learning histories in simulation-based
teaching: The effects on self-learning and transfer. Computers in Education, 39(4),
319332. doi:10.1016/S0360-1315(02)00043-X.
Pekrun, R. (2000). A social-cognitive, control-value theory of achievement emotions.
Elsevier. Advances in Psychology, 131, 143163. doi:10.1016/S0166-4115(00)80010-2.
Pekrun, R. (2005). Progress and open problems in educational emotional research.
Learning and Instruction, 15, 497506. doi:10.1016/j.learninstruc.2005.07.014.
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions,
corollaries, and implications for educational research and practice. Educational
Psychology Review, 18(4), 315341. doi:10.1007/s10648-006-9029-9.
Pekrun, R., Elliot, AndrewJ., & Maier, M. A. (2009). Achievement goals and achievement
emotions: Testing a model of their joint relations with academic performance.
Journal of Educational Psychology, 101(1), 115135. doi:10.1037/a0013383.
182 G. Noteborn et al. / Internet and Higher Education 15 (2012) 176183
Pekrun, R., Goetz, T., Daniels, L. M., Stupnisky, R. H., & Perry, R. P. (2010). Boredom in
achievement settings: Exploring controlvalue antecedents and performance
outcomes of a neglected emotion. Journal of Educational Psychology, 102(3),
531549. doi:10.1037/a0019243.
Pekrun, R., Goetz, T., & Frenzel, A. C. (2005). Academic Emotion Questionnaire-
Mathematics (AEQ-M): User's manual. Munich, Germany.
Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring
emotions in students' learning and performance: The Achievement Emotions
Questionnaire (AEQ). Contemporary Educational Psychology, 36(1), 3648. doi:
10.1016/j.cedpsych.2010.10.002.
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students' self-
regulated learning and achievement: A program of qualitative and quantitative
research. Educational Psychologist, 37(2), 91105 Routledge.
Petraglia, J. (1998). Reality by design: The rhetoric and technology of authenticity in
education. Mahwah, NJ: Lawrence Erlbaum Associates.
Pintrich, P. R., Smith, D., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the
motivated strategies learning questionnaire (MSLQ). : The University of Michigan
(NCRIPTAL).
Sauv, L., Renaud, L., Kaufman, D., & Marquis, J. S. (2007). Distinguishing between
games and simulation: A systematic review. Journal of Educational Technology and
Society, 10(3), 247 Citeseer.
Simons, J., Dewitte, S., & Lens, W. (2004). The role of different types of instrumentality
in motivation, study strategies, and performance: Know why you learn, so you'll
know what you learn! The British Journal of Educational Psychology, 74(Pt 3),
343360. doi:10.1348/0007099041552314.
Wood, N. T. (2010). Real lesson in virtual worlds: using virtual worlds to educate and
train business students. In C. Wankel, M. Marovich, & J. Stanaityte (Eds.), Cutting-
edge social media approaches to business education: Teaching with LinkedIn,
Facebook, Twitter, Second Life, and Blogs. : Information Age Pub Inc.
Wosnitza, M., & Volet, S. (2005). Origin, direction and impact of emotions in
social online learning. Learning and Instruction, 15(5), 449464. doi:10.1016/j.
learninstruc.2005.07.009.
Gwen Noteborn Obtained her MSc in Strategic Marketing and MA in Conict
Management and works as a Researcher at the Department of Educational Research
and Development at Maastricht University. Her research focuses on the effectiveness of
online tools in education. Gwen has been teaching the course Brand Management for
three consecutive years and is responsible for the implementation of Second Life
within the course Brand Management. For her work in the Brand Management course
she received the educational Prize for an outstanding contribution to education.
Katerina Bohle Carbonell Obtained her MSc in Management of Learning at Maastricht
University and is a researcher at the Educational Research and Development at
Maastricht University. Katerina developed her passion on online learning when writing
her Master Thesis in this eld and is now a PhD candidate at the Education Department
of the Faculty of Health, Medicine and Life Sciences at Maastricht University.
Dr. Amber Dailey-Hebert Is a post-doc researcher in the department of Educational
Research and Development, Maastricht University School of Business and Economics
(the Netherlands). She received her Ph.D. in Education from Cornell University,
investigating how instructor practice informs student learning outcomes. She has also
served as an online course developer, instructor, and mentor for online course
facilitation and instruction, is the Associate Editor of InSight: A Journal of Scholarly
Teaching, and chairs the Research Committee for the Association of Continuing and
Higher Education. Dailey-Hebert has been published on topics related to leading
organizational change, faculty evaluation models online, and faculty professionaliza-
tion, and is currently conducting research in integrative eLearning andragogies and
higher education. She has served as the Associate Dean of the School for Online
Learning, Adult Education Department Chair and Program Coordinator, and as the
Founding Director of the Center for Excellence in Teaching and Learning. Dr. Dailey-
Hebert was honored as the Ebadi Scholar of the Year in 2007 and as the Outstanding
Graduate Faculty member at Park University, has been recognized nationally with the
Outstanding Innovation Recognition Award from the Professional and Organizational
Development Network in Higher Education, and published the book Service-
eLearning: Educating for Citizenship in 2008.
Wim Gijselaers Is a professor in the eld of professional learning and head of the
department of Educational Research and Development (ERD), Maastricht University,
the Netherlands. His research addresses effects of social and cognitive processes on
professional development. Next, he researches how student learning can be improved
through course and program innovation. His current research projects deal with how
visualization tools can guide engineering teams in new product development, how
talented people can become experts in a professional domain, and how drop-out in
professional education can be reduced through curriculum interventions. He is chief-
editor of the Springer book series Innovation and Change in Professional Education
and associate editor of the Springer book series Advances in Business Education and
Technology. His work has been published in many international refereed journals and
edited volumes.
183 G. Noteborn et al. / Internet and Higher Education 15 (2012) 176183