Gamification
Gamer Style: Performance Factors in Gamified Simulation
Surabhi Gupta Tim Coles
University of Western Australia contact@timcoles.info
Perth, Australia Cedric Dumas
guptas09@student.uwa.edu.au Ecole des Mines de Nantes,
cedric.dumas@mines-nantes.fr
ABSTRACT
Serious games and gamified simulations are increasingly
being used to aid instruction in technical disciplines
including the medical field. Assessments of player
performance are important in understanding user profiles in
order to establish serious games as a reliable, consistent
method for increasing skills and competence in all trainees.
In this study we used questionnaires, game characteristic
metrics and EEG analysis to explore players’ performance
in a bronchoscopy simulator. We found that players who
performed better were younger, made fewer errors, were
quicker and differed in spectral profile during game play.
Our findings, while speculative, have implications for
training regimes in which gamified simulations are
employed. We make suggestions for game design and for
tailoring training regimes to suit individual learning styles
to enhance knowledge acquisition and retention.
Author Keywords
Interactive learning environment; serious games; cognitive
load; electroencephalogram; EEG.
ACM Classification Keywords
I.2.1 Computing methodologies: games.
INTRODUCTION
In an increasingly digital world, more disciplines are
turning to serious games and gamified simulation (where
game elements are used in non-game concepts [8]) to aid
education. In healthcare, these are particularly useful, as
they allow trainees to practice invasive procedures before
performing them in a clinical setting. The effectiveness of
serious games and simulations in obtaining training
outcomes is largely unevaluated due to a lack of assessment
metrics [2]. One of the most important aspects of
assessment is user performance. Characterization of
players’ activity will allow for tailoring of training regimes
to facilitate the best possible learning outcomes.
Publication rights licensed to ACM. ACM acknowledges
that this contribution was authored or co-authored by an
employee, contractor or affiliate of a national government.
As such, the Government retains a nonexclusive, royalty-
free right to publish or reproduce this article, or to allow
others to do so, for Government purposes only.
CHI'16, May 07 - 12, 2016, San Jose, CA, USA
Copyright is held by the owner/author(s). Publication rights
licensed to ACM.
ACM 978-1-4503-3362-7/16/05…$15.00
DOI: http://dx.doi.org/10.1145/2858036.2858461
2014
#chi4good, CHI 2016, San Jose, CA, USA
Simon J McBride
DanaKai Bradford
CSIRO, Brisbane, Australia
Simon.McBride@csiro.au
Dana.Bradford@csiro.au
The ability of serious games to impart learning can be
measured in several ways (reviewed in [2]). Three types of
primary assessment have been described specifically for
serious games [23], comprising completion assessment, in-
process assessment and teacher assessment. Game
completion signifies that comprehension and learning have
occurred, as the player must interact with the material in the
game in order to progress. It is possible to learn how to beat
a game without learning the material, thus on its own,
completion assessment is an insufficient measure of game
performance. In-process assessment complements
completion assessment by recording actions of the player
including time taken to complete the lesson and number of
errors made. Together these assessments provide a strong
indication of knowledge acquisition through serious games.
Teacher assessment adds less tangible factors gained
through observation of the student in action. Other factors
associated with enhanced learning include engagement and
‘flow’ – where the user experiences high levels of
enjoyment; and psycho-physiological states. Common tools
for subjective assessment of serious games include the
Game Engagement [6] and EGameFlow questionnaires
[11]. Psycho-physiological methods include facial
electromyography, to measure facial contractions associated
with positive and negative valence; cardiovascular activity,
to measure heart beat and rate to detect attention and
arousal; galvanic skin response, to measure activation of the
sympathetic nervous system in response to arousal; and
electroencephalography (EEG), to measure cortical activity
associated with game play [2].
In this study, we explored whether there were distinct
features affecting players’ performance in a gamified
simulation that have the potential to impact training
outcomes. We looked at player demographics including
age, self-reported explicit knowledge, game characteristics
including in-process and completion assessment, and
spectral analysis using the EPOC Emotiv EEG headset. We
hypothesized that there would be a different spectral profile
depending on player performance as measured through
level achievement, and that this difference would be evident
in game play characteristics. Our aim was to determine the
factors associated with performance so that training could
be modified to produce the best possible outcomes for all
trainees. We reflect on potential improvements to the game
design and postulate how training regimes could be tailored
to accommodate player differences in knowledge
acquisition through game play.
Gamification
Related works
Cortical mapping
Cortical mapping refers to the recording of activity in the
surface areas (cortex) of the brain. Electrical activity occurs
at different frequencies and is thought to indicate brain
behaviors associated with the different lobes of the brain,
frontal, parietal, temporal and occipital. The frontal cortex
regulates high order processing and motor functions, the
parietal lobe is involved with somatosensory functions, and
the temporal and occipital lobes control audio and visual
functions respectively. Alpha signals (7-13Hz) are typical
of a relaxed waking state [18]. Beta frequencies are quite
fast at 13-25Hz and signal intense mental activity. They are
more prominent over frontal and parietal regions and there
is some evidence to suggest that beta activity mediates
motor activity, with faster response times predicted by
higher beta activity [27]. The slow delta rhythms (0-4Hz)
are often large in amplitude but tend to be related to deep
sleep, especially rapid eye movement (REM) sleep. Theta
activity (4-7Hz) is also associated with sleep, however
several studies have found increased frontal theta activity
correlated with working memory tasks [7, 13, 15, 24, 31].
Many of the studies employing EEG based assessment of
video game play in the healthcare field are based on a
seminal study exploring EEG mapping of cortical activity
during a working memory task [13]. This study
convincingly demonstrated increases in theta activity across
the frontal midline area of the brain. This is the area
involved in problem solving, planning and aspects of
working memory including attention. Where theta activity
was increased, alpha signals decreased. This led the
researchers to postulate that frontal midline theta activity
was related to mental effort, and that attenuation of alpha
signals freed cortical resources for increased task difficulty,
as described in earlier research [26]. Their findings are
further supported by their observation that alpha activity
increases with practice, suggesting that fewer cortical
resources are required following knowledge acquisition.
A review of functional magnetic resonance imaging studies
reported that activation of the parietal lobes is consistently
associated with memory retrieval [33]. This review
complements previous EEG studies showing a role for the
parietal lobe in event recollection [26]. Recently, theta
activity in the temporal lobe was demonstrated to be
associated with autobiographical memories with detailed
visual imagery [12].
Video game assessment using EEG
As video games are thought to rely on working memory for
the acquisition of knowledge to allow game advancement
through level progression, frontal midline theta activity has
been explored in clinical and recreational populations.
Mental effort related EEG activity has been used to
elucidate whether video games trigger abnormal neural
responses in adolescents subject to seizures compared to
healthy aged matched participants [24]. Here it was found
2015
#chi4good, CHI 2016, San Jose, CA, USA
that both epileptic and healthy control subjects show task
related increases in frontal midline theta activity with
associated decreases in alpha activity, further supporting a
relationship between increased theta and decreased alpha
activity in mental effort. Due to a concern that youths spend
long hours playing video games, one study looked at spatio-
spectral mapping during extended video game play,
specifically exploring frontal midline theta activity
associated with mental effort [15]. Length of game play was
found to be correlated with increasing frontal theta wave
activity, suggesting that extended game play requires more,
rather than less, cognitive input. Furthermore, their results
suggest that theta activity is higher during strategy games
than competitive games. As serious games usually involve
some form of strategy, it would be expected that serious
game play would result in increased theta activity.
Accordingly, increases in frontal midline theta activity and
associated decrease in alpha activity have been found in
serious game progression [7, 31]. These studies have all
concentrated on the relationship between theta and alpha
activity, but have not conducted full cortical mapping of all
frequencies during serious games.
Serious games and gamified simulations for healthcare
Serious games and gamified simulations are employed in a
variety of fields, but appear to be particularly effective in
healthcare for both patients and practitioners. A 2013
review of serious games for health found over 100 games
[34], some of which were found to influence positive health
behavior (reviewed in [2]). A 2012 review of serious games
for training medical professionals found 30 available
games, 17 of which were designed specifically for training
and 13 commercial games that facilitated the development
of skills useful for medical staff [14]. Most of the games
designed specifically for training were developed for team
training in acute care and triage, while two were developed
for specific surgeries (coronary bypass and knee
restructure). Commercial games facilitated psychomotor
skills useful for performing laparoscopies. Since this
review, more serious training games have been developed
for specific procedures (for example [19]) including two
training simulators designed by CSIRO, one for
colonoscopies and one for bronchoscopies.
METHODOLOGY
This study used questionnaires, game play characteristics
and spectral analysis to explore features of players’
performance in a gamified simulation that have the
potential to impact training outcomes. Approval was
obtained from the CSIRO Animal Food and Health
Sciences Human Research Ethics Committee: Low Risk
Review Panel (23/2013).
Participants
Participants (N=15) were recruited from staff at the
Australian eHealth Research Centre through email
invitation. Inclusion criteria included minimum age of 22
(the corpus callosum, the tract joining the hemispheres, and
the last tract to fully develop in the human brain, is mature
Gamification
in the majority of people by age 22 [20]); limited
knowledge of lung anatomy and bronchoscopies; and
normal or corrected-to-normal vision. Males were
overrepresented (n=12, 80%), which is consistent with the
proportion of men (82%) who work as respiratory and sleep
medicine clinicians (thoracic departments) in Australia
[16]. Most participants were in their 30s or 40s (M=38.4,
range 22-50, one participant did not disclose age), with post
graduate qualifications (n=11, 73%). Participants answered
a brief questionnaire on self-ratings of information
technology (IT) skills and knowledge of anatomy (score out
of 10 where 1=no knowledge and 10=familiar working
knowledge); whether they wore glasses; and had used
virtual simulators or bronchial simulators (yes/no).
Equipment
Bronchoscopy simulator
The gamified simulation used was a bronchoscopy
simulator. A bronchoscopy is a highly invasive medical
procedure in which a thin tube tipped with a light source
and camera is passed through the mouth and trachea to
perform an endoscopic examination of the lungs
Bronchoscopies are used to directly observe the branches
and nodes (bronchi) of the lung for a number of pathologies
and conditions. Due to the complicated morphology of
bronchial branching, extensive training is required to ensure
the entirety of the lung anatomy is traversed. Such training
is beneficial for patient care and reduces the time spent
performing this invasive procedure. Historically, trainees
have learned this procedure through books, training videos
or observing experts undertake a procedure. With rapidly
changing technologies, these methods are time consuming
and do not instill high confidence levels in the trainees.
CSIRO’s low fidelity, ‘part task’ bronchoscopy simulator
(Figure 1) was designed for novice trainees when learning
the Ikeda nomenclature [17] for lung bronchial tree naming
and navigation to enhance knowledge of lung anatomy and
improve efficacy of bronchoscopic navigation [9]. The
simulator includes a number of game elements including
increasingly difficult levels, task achievement to advance
through levels, return to start for error. Game metrics
include level times and number of errors. To ensure the user
experience mimics a real bronchoscopy, visual game
elements are kept to a minimum (see [29]).
Three simulator levels provided users with increasing levels
of difficulty by reducing the number of navigational cues
visible on the screen. Figure 1A shows the start point for
the simulation, a low fidelity depiction of a bronchoscope
camera after passing the trachea. Figure 1B depicts the
bronchoscope image after entering (traversing) the right
main bronchus, with the apical (RB1) and posterior (RB2),
sublobar lung segments visible. Cues visible during
traversal when the simulator is in level one (the least
difficult mode) (Figure 1B) include overarching
navigational information provided around the diameter of
the trachea, bronchi names floating in the bronchi lumen
2016
#chi4good, CHI 2016, San Jose, CA, USA
(RB1 and RB2). The target bronchi, next in the traversal
order, is underlined. Visited bronchi, those the user has
already traversed are also indicated for the user. In
transitioning to level two (Figure 1C), names of individual
bronchi are removed, including target and visited bronchi,
and the trainee is guided only by the overarching
navigational information. In level three (Figure 1D), no
information is provided and the trainee must navigate the
bronchial tree in the correct order without being guided by
directional or ordinal cues.
To advance the simulated bronchoscope further into the
lung the user moved the mouse pointer onto and left-clicked
target bronchi. Bronchoscope retraction was achieved by
pressing the space bar.
EPOC EMOTIV EEG headset
While traditional EEG devices are complex, expensive and
require specialist technicians, the Emotiv headset is a
relatively cheap, compact device that is easy to set up and
comparatively comfortable. The Emotiv headset was
designed as a brain-computer interface for gaming, however
it has been employed for a number of research activities
including the assessment of psychological and cognitive
states (reviewed in [22]).
Participants used the simulator running on a desktop PC
activated via a mouse and keyboard while wearing the
Emotiv wireless EEG headset (http://emotiv.com/eeg/). The
headset recorded 14 channels at 128Hz across frontal (left
AF3, F3, F7, FC5; right AF4, F4, F8, FC6), temporal (left
T7; right T8), parietal (left P7; right P8) and occipital (left
O1; right O2) lobes, with bi-polar reference nodes (CMS,
DRI) using the international 10-20 system (Figure 1).
Procedure
Prospective participants attended a brief overview of the
project where consent forms, participant information sheets
and interview questionnaires were completed. Eligible
participants were appointed scheduled times for
participation in the study, consisting of one Tuesday and
one Friday session (four days apart). Due to the number of
participants, experiments spanned three weeks.
The first session, on Tuesday, comprised fitting the Emotiv
EEG headset and describing the controls of the
Bronchoscopy simulator. Spectral data was recorded from
the start of the game play until all three levels were
completed with three consecutive error-free traversals or 20
failed attempts were made in any of the three levels. The
second session, on the following Friday, consisted of a
refresher of the controls and then a retrial of the highest
level completed in their first session and again spectral data
was recorded. Participants had the option of removal of the
Emotiv EEG headset every 15 minutes if the headset felt
uncomfortable. In order to explore features of performance,
high performing participants (those who achieved level
three in session 1) were separated from low performing
participants (those who achieved level two in Session 1).
Gamification #chi4good, CHI 2016, San Jose, CA, USA

Figure 1. The bronchoscopy simulator facilitates learning lung anatomy and nomenclature in bronchoscopy trainees, providing less
navigational information with each level. The simulator is using a computer with keyboard and mouse while wearing Emotiv
headsets, that recorded EEG activity from 14 points across frontal (left AF3, F3, F7, FC5; right AF4, F4, F8, FC6), temporal (left
T7; right T8), parietal (left P7; right P8) and occipital (left O1; right O2) lobes, with two bi-polar reference nodes (CMS, DRI).

Data acquisition and analysis RESULTS

Emotiv’s Test Bench software was used to record the EEG
signals. The EEG signal data from each participant was
coded for game completion characteristics and and exported
for spectral analysis. No data was lost due to connectivity
issues. Data analysis was performed using SPSS V20 and
MathWorks Matlab. Game completion features were
analysed in SPSS V20 using independent t-tests. Where
underlying assumptions of normality were not met, non-
parametric (Mann-Whitney) tests were used. The four step
spectral analysis was undertaken in MATLAB using the
package EEGlab. Initially, movement artifacts were
removed 33], then an infinite impulse response filter was
used to remove the DC offset (4200uV). The signal
received from the EEG, in voltage, changes against time for
each of the channels. The signal was converted to frequency
using fast Fourier transformation (FFT) and a hamming
window to emphasize the middle of the signal over the
edge. From log values of the powers generated in the FFT,
average bandwidths were extracted corresponding to alpha
(7–13Hz), beta (13–25Hz), delta (0–4Hz) and theta (4–
7Hz).
Model Analysis – Generalised Estimating Equations
The method of generalised estimating equations [21] was
used to compare between lobes, hemispheres, frequency
bandwidths and levels whilst accounting for multiple
measurements from the same participant. We used the
exchangeable correlation (Wald Chi square). To safeguard
against misspecification, robust estimators were used.
Participants reported themselves to be in the upper levels of
IT literacy, with low levels of anatomical knowledge, little
previous experience with virtual simulators, and no
experience with bronchial simulators (Table 1). Participants
were divided into two groups based on performance in the
first session. These are referred to as Group L2 (those who
reached level two in Session 1) and Group L3 (those who
reached level three in Session 1). These groups were then
compared on game characteristics and in the spectral
analysis. Session 2 data was not available for one
participant due to attrition.
Game characteristics differed between groups
Level achievement is independent of self reported skill base
In the first session, all participants completed level one and
seven (48%) went on to complete level two. Of these seven,
all completed level three as well, and all but one did so in
the minimum of three attempts (three consecutive error-free
attempts are required to pass the level). These participants
constituted Group L3. Based on self-ratings (1= no
knowledge, 10= functional working knowledge, Group L3
did not have greater knowledge of anatomy (M=4.13) than
Group L2 (M=3.71) (t(13)=0.323, p=0.75(ns) two-tailed).
Self reported skills in IT were also not related to level
achievement, Group L3 reported similar IT self ratings
(M=7.71) to Group L2 (M=8.88) (t(13)=1.47, p=0.16 (ns)
two-tailed). Of the Group L3 participants, one was familiar
with virtual simulators, compared to three in Group L2.

2017
 Gamification #chi4good, CHI 2016, San Jose, CA, USA

Group L2 Group L3 traversal attempts or average length of completed traversals.

Attained level two Attained level three Group L2 had a similar number of error-free traversals
(53%) compared to Group L3 (61%) (Table 2), and all
(n=8*) (n=7)
participants completed level one after few errors (M=7.9
Gender Male=6; Female=2 Male=6; Female=1 traversals including the three consecutive error-free
44.3 (35–50) 32.4 (22–45) traversals required to move to the next level). Level one
Agea game play however, was not indicative of success in more
sd 5.65 sd 7.56
difficult levels. One participant, for example, was the sole
Glasses Yes=5 Yes=3 participant to complete level one in the minimum of three
PostGrad Yes=6 (75%) Yes=5 (71%) consecutive error free traversals, but only completed two
non-consecutive error free traversals in 20 attempts at level
a
Skills two, and hence did not proceed to level three. Level two
-IT 8.9 (6–10/10) 7.7 (6–10/10) posed a greater challenge for more than half the
participants, with only seven participants transitioning to
-anatomy 4.13 (1–10/10) 3.7 (2–8/10) level three during Session 1.
Simulator Of the eight participants in Group L2, five did not complete
-virtual Yes=3 (38%) Yes=1 (14%) a single error-free traversal of level two, and no participants
completed two consecutive error-free traversals during
-bronchial Yes=0 Yes=0 Session 1. In fact, only 4% of level two traversals were
* Session 2 spectral data unavailable for one participant error-free compared to 48% in Group L3. This suggests
a. Mean (range); sd standard deviation that, for Group L2, the step from level one to level two (i.e.
Table 1: Participant characteristics. removal of bronchi names) was too great a reduction in
navigational information for successful traversal of the
complete bronchial tree. Due to the number of errors made,
Level achievement is age dependent median level two completion times in Session 1 differed
While there were no differences in any of the explicit significantly between Group L2 (median 1328.31s) and
knowledge measures, participants with different levels of Group L3 (median 732.96s) (Mann–Whitney U=9.00, n1=8,
achievement did differ in age. Across all participants, age n2=7, p<0.028, two tailed). On average, those in Group L2
distribution was approximately normal, with an average of (M=1366.67s) took 1.5 times as long to complete level two
38.6 (range 22–49). Participants in Group L3 were, on (i.e. navigate 20 unsuccessful attempts) as it took those in
average, significantly younger (M=32.43 years) than those Group L3 (M=881.76) to complete level two (ie three
in Group L2 (M=44.29) (t(13)=3.078, p=0.01). Overall, consecutive error-free traversals) (Table 2).
lower aged participants performed at a higher level.
Groups differed in number of errors across sessions
Differences in duration of traversals and level completion In the second session, participants began from the highest
Each time an error was made, the participant had to restart level they had achieved in Session 1. For the most part, the
the level; hence number of errors is equal to number of findings from Session 1 were replicated in Session 2. Error-
traversals minus one. Three consecutive error-free free traversals were rare in Group L2 (12%) compared to
traversals of the level in under 20 attempts were required to Group L3 (75%). Despite performing poorly compared to
move to the next level, without time constraints. Of the Group L3, Group L2 did show some improvement across
three levels in the bronchoscopy simulator, level one was sessions. Participants in Group L2 had significantly fewer
the most time consuming per attempt and level two required errors in Session 2 (M=15.57) than in Session 1 (M=20)
the greatest number of attempts and hence the longest time (D=4.43, t(6)=9.0, p=0.001). This shows that some
for level completion, with level three conducted most navigational learning had occurred during game play, but
efficiently. The median time for completed traversal on average, it was insufficient for level completion.
(navigation of the level without error) for all participants
For participants in Group L3, there was no significant
significantly reduced between Session 1 (median=127s) and
difference in error rate between Session 1 (M=3.57) and
Session 2 (median=113.51s) (Mann–Whitney U=1166.0,
Session 2 (M=4) (D=-0.43, t(6)= 0.62, p=0.55, ns). Given
n1=15, n2=14, p<0.0001, two tailed). This reduction in time
that a minimum of three error free traversals is required for
to complete level traversals indicates that all participants
level completion, this mean represents a ceiling effect for
had effectively acquired and retained some knowledge of
this group. Six of the seven participants in this group (86%)
lung anatomy during the gamified simulation.
completed level three in Session 2 in the minimum three
When the groups were separated, marked differences in error-free traversals. For these participants, a single session
level completion only became apparent after the first level. was sufficient to acquire and retain enough navigational and
Level one was an equal playing field, with no significant bronchial information to traverse the level four days later
differences in number of traversals, average length of without error.

2018
Gamification #chi4good, CHI 2016, San Jose, CA, USA

Group L2 Group L3 three, more graduated steps in level difficulty may be

required. For Group L2 overall, additional sessions would
Session 1
have been required to complete all levels in the
Level 1 error- 36 of 37 attempts 32 of 52 bronchoscopy simulator.
free traversals (53%) attempts (61%) Summary of game characteristics
Mean 8.375 7.43 Participants were divided into two groups based on level
traversals achievement in the first session. On average, those in Group
L2 were older, had significant differences in error rates
Mean level 915.50s 776.29s between sessions, were consistently slower at individual
completion traversals and were significantly slower at whole level
time completion than those in Group L3. As these findings have
Level 2 error- 7 of 160 attempts 31 of 65 implications for training mature aged staff with new serious
free traversals (4%) attempts (48%) game and simulator technologies, we were interested to
explore the neural correlates of game play and the
Mean 20 9.29 differences between the groups.
traversals
Spectral analysis differed between groups
Mean level 2 1366.67s 881.76s In order to explore the spectral profile associated with
completion performance evident in game characteristics, we conducted
time several comparisons to explore activity in different brain
Mean level 3 n/a 376.63s regions between the groups, levels and sessions. These are
completion summarized in Table 3, and the four analyses are detailed in
time the following pages.
Interhemispheric differences only found in Group L3
Session 2 (Re-navigate highest level achieved)
We first wanted to determine if one side of the brain was
Group L2 level 2 Group L3 level 3 more active during game play than the other. Neuroimaging
studies have found that the left hemisphere is more active
Error free 13 of 109 attempts 21 of 28
during verbal or text related tasks, while the right
traversals (12%) attempts (75%)
hemisphere is dominant for visuo-spatial tasks [30]. The
Mean 15.57 4 bronschoscopy simulator has both textual and spatial
traversals guidance cues so we were interested to see if there were
interhemispheric differences associated with game play. To
Mean level 1115.28s 453.26s
explore this, we compared activation in left and right
completion
hemispheres for each frequency. No significant differences
Table 2: Summary of traversal metrics for Group L2 and were found for delta or theta activation. For alpha
Group L3 in Session 1 and Session 2. frequencies, activation of the left hemisphere was
significantly greater than the right (Wald Chi square
(1)=65.716, p<0.001, β=1.584, CI 95% (1.201,1.967)). We
Knowledge retention described earlier that attenuation of alpha activity could
The within-group differences in number of errors and the indicate freeing of cortical resources [28]. Therefore, the
time to complete the level between sessions gives us insight reduction of alpha activity in the right hemisphere could be
into how well lung anatomy knowledge acquired in Session freeing resources for visuo-spatial attention. If this is the
1 was retained in Session 2. Reaching level three in Session case, we would also expect to see increased activation in
1 demonstrated that knowledge acquisition was high for frequencies indicating intense mental activity. This was
Group L3. For this group, knowledge retention was also supported by significantly higher beta frequencies in the
high, evidenced by similar level completion times and right hemisphere (Wald Chi square (1)=9.823, p=0.002,
traversals required to complete level three in Session 1 β=0.816, CI 95%=(0.306,1.326)). Beta activation is thought
(M=376.63s, M=3.57, respectively) and repeat this level in to be associated with increased concentration in goal
Session 2 (M=453.26s, M=4, respectively). For those who oriented tasks and has been shown to be correlated with
only reached level two in Session 1 (Group L2), the decreased response times [27]. Together these findings
acquisition of some navigational knowledge across sessions suggest that activity in the right hemisphere was modulated
was seen in a reduction in both number of traversals (M=20 to attend to visuo-spatial tasks by reducing alpha activity to
down to M=15.57) and a concordant reduction in mean allow for an increase in beta activity.
level completion times from 1366.67s to 1115.28s,
however, there were still three participants who did not
complete the level (20 unsuccessful attempts). For these

2019
Gamification #chi4good, CHI 2016, San Jose, CA, USA

Brain region Comparison Rationale Knowledge sought

Left and right The left hemisphere is more active during verbal or Are hemispheric differences
hemisphere in text related tasks, while the right hemisphere is apparent in game play?
both sessions. dominant for visuo-spatial tasks [30].

Frontal midline Several studies have correlated frontal midline theta Is level complexity reflected in
theta Session 1 activity with task complexity [13, 15, 24, 31]. frontal midline theta activity?

Posterior theta The parietal and temporal lobes are associated with Do groups differ in memory
in both groups recall of events [33] and autobiographical memories recall?
[12], respectively.
Fronto-parietal Beta activity across the fronto-parietal regions is Does beta activity vary across
beta across thought to facilitate motor behavior, and is sessions?
sessions correlated with decreased response times [27].

Table 3: Summary of comparisons undertaken in the EEG spectral analysis (see Fig. 1 for electrode composition).

This shows that there are hemispheric differences in game While this trend did not reach significance, it reflects the
play, but only for Group L3. No significant complexity of level two compared to level one and level
interhemispheric differences were found in Group L2 three. This trend is supported by our findings in the game
(Wald Chi square (1)=0.400, p=0.527, β=0.245, CI 95%=(- characteristics where an increased number of errors
0.513,1.002)). (traversals), and hence longest level completion times, were
recorded for level two.
To determine if this activation pattern was consistent across
sessions for Group L3 participants we compared
hemispheric activation in Sessions 1 and 2.We found a
significant interaction between Session and Hemisphere for
alpha activity (Wald Chi square (1)=7.255, p=0.007, β=-
1.182, CI 95% (-2.043,-0.322)), where alpha activity was
most attenuated in the right (rather than the left) hemisphere
in Session 2 (compared to Session 1). This is consistent
with an increased requirement for visuo-spatial cortical
resources to navigate the level on spatial cues alone in the
absence of nomenclature guides.
Within group differences in frontal midline theta activity
The frontal midline area of the brain is involved in problem
solving and planning, and particularly planning of motor
activity. It is also the area of the brain that directs attention
to relevant information while suppressing inappropriate
responses. We described earlier previous research that
found increasing frontal midline theta activity associated
with increased task complexity [13], and this finding has
been replicated in recreational video games [15, 24] and
simulation tasks [31]. We were therefore interested to see if
activity increased in this area of the brain during navigation
of the bronchoscopy simulator. The frontal midline area
corresponds to electrodes F3 and F4 (Figure 1, Table 3).
We compared activity in this area for each group to gauge
increasing task complexity across the levels of the game in
Session 1. For Group L2, no differences in frontal midline Figure 2: A trend toward an increase in frontal midline theta
theta activity were found between level one and level two. activity in both left (F3) and right (F4) hemispheres (reflecting
For Group L3, a trend toward increased frontal midline increased complexity) was found in level two for Group L3
theta activity was observed during level two of the three (those who completed all three levels in Session 1).
levels played (Figure 2).

2020
Gamification
Groups differ in posterior theta activity
The learning of tasks relies on working memory, which
involves a network of brain activity including the frontal
midline areas previously mentioned. This network extends
into parietal and temporo-parietal cortices [11]. We would
therefore expect to see some increased activation in the
parietal lobes during game play. In this study we found
increased theta activity in the parietal lobes in both groups.
What was most intriguing was a significant interaction
between Session and Level achieved (Wald Chi square
=9.296, p=0.002, β=4.164 CI 95% (1.487,6.842)) (Figure
3). For participants in Group L2, average theta activity in
the parietal lobe for Session 1 was significantly higher than
in Session 2 (Wald Chi square = 4.823 p=0.028, β=2.266 CI
95% (0.244, 4.823)). Conversely, for participants in Group
L3, average theta activity in the parietal lobe for Session 2
was significantly higher than in Session 1 (Wald Chi square
= 4.500 p=0.034, β=-1.898, CI 95% (-3.652,-0.144)).
Activity in the parietal lobe is particularly associated with
episodic memory retrieval, including recall of personal
events and experiences [33]. This activity could also be
explained by the role of the parietal lobe in visuomotor
transformations for actions of the hand [10].
During our analysis it was revealed that in addition to
increased theta activity in the parietal lobe in Session 1,
those in Group L2 also had significantly higher theta
activity in the temporal lobe in Session 1 than in Session 2
(Wald chi square = 4.515, p=0.034, β=3.835, CI 95%
(0.298,7.372)). Temporal lobe theta activation has been
shown to be associated with recall of autobiographical
memories [12]. This could indicate the groups do differ in
memory recall.
These findings are particularly interesting. Taken together
one speculative explanation could be that during Session 1,
participants in Group L2 drew on previous autobiographical
schema to assist in learning the material in the gamified
simulation, and hence parietal and temporal lobes were
activated; whereas participants in Group L3 relied more
heavily on recall in Session 2, potentially of the material
gleaned in Session 1. These variations in the neural
approach to Session 1 could explain the performance
differences between the groups.
2021
#chi4good, CHI 2016, San Jose, CA, USA
Frontal-parietal beta activity increased in Group L3
One last area of interest for us was the fronto-parietal
region. The frontal lobe houses the motor cortex while the
parietal lobe houses the somatosensory cortex. This region
therefore co-ordinates motor responses to perceived stimuli.
Beta activity across the fronto-parietal regions is thought to
facilitate motor behavior, and as noted earlier, is correlated
with decreased response times (faster response times are
predicted by higher beta activity [27]). As Group L3 had
consistently fast level traversal and completion times, and a
high level of accuracy in Session 2 (Table 2), we were
interested to see if there were neural correlates for this
behavior. For Group L3 a significant interaction between
Session and Frequency was found for both frontal (Wald
Chi square=5.429, p=0.020, β=-0.669, CI 95% (-1.233,-
0.106)) and parietal (Wald chi square = 11.328 p=0.001,
β=-2.006, CI 95% = (-3.178, -0.833)) lobes, where beta
activation in Session 2 was significantly higher than in
Session 1. This finding, together with increased right
hemisphere activity (see above) is in accordance with
previous studies [27]. Group L2 also showed a significant
interaction between level and frequency for beta activation
(Wald chi square = 3.969, p=0.046, β=1.501, CI 95% =
(0.024, 2.977)), with increased beta activity in Session 1.
However, the increase was limited to the parietal lobes, and
therefore did not involve the motor cortex. This shows that
beta activity does vary across sessions and may underpin
speed and accuracy.
Summary of spectral profiles
For Group L3, theta activation was higher in Session 1 than
in Session 2, and frontal midline theta activity was highest
during level two in Session 1. During the Session 2, Group
L3 had greater activation in the right hemisphere, and
increased beta activity in the frontal and parietal lobes. The
spectral profile of Group L3 suggests that they relied most
on visuo-spatial cues, and expended the most mental effort
on level two in Session 1. The increased activity in Session
2 suggests they drew on memory to coordinate a high level
of accuracy. For Group L2, theta and beta activation was
higher in Session 1 in temporal and parietal lobes, which
may suggest an attempt to draw on existing schema. While
more studies would be required to elucidate less speculative
interpretations, these spectral profiles clearly demonstrate
group differences in processing game features.
Gamification #chi4good, CHI 2016, San Jose, CA, USA

Figure 3: Spectral profile measured in log power for those participants who reached level two (Group L2) and those who reached
level three (Group L3) for Sessions 1 and 2. The profiles differed within groups between sessions and between groups. Asterisks
represent significantly higher activity within groups between sessions. A significant interaction between level and session was found
where Group L2 had greater activation in Session 1 compared to Session 2, and Group L3 had greater activation in Session 2
compared to Session 1 (0–4Hz delta, 4–7Hz theta, 7–13Hz alpha, 13–25Hz beta).
DISCUSSION research supports recent evidence of age-related slowing of
In this study we found that age influenced player reaction times within a video game scenario [32], and is in
performance, with younger participants demonstrating accord with traditional studies of age related declines in
greater knowledge acquisition through level progression task performance (for example [6]). As such, our findings
and completion assessment. On average, older participants have significant real-world implications. A study of the
made more errors, were consistently slower at individual cardiothoracic workforce in Australia found that only 1.9%
traversals and were significantly slower at whole level of cardiothoracic surgeons was aged less than 35 years [1].
completion than younger participants. Our EEG analysis This places trainee surgeons in the age overlap between
revealed differing spectral profiles that may account for Group L2 and Group L3, where the greatest variation in
variations in player performance. knowledge acquisition through gamified simulation is likely
One interpretation of spectral profiles is that performance in to be observed.
Group L3 was characterized by a reliance on visuo-spatial In 2015, the age of 35 is pivotal in our digital era as it
cues and potentially recall of Session 1 material to facilitate distinguishes digital natives, those born after 1980, from
completion of Session 2. In contrast Group L2 may have digital immigrants [25]. Digital natives have been raised,
attempted to draw on existing autobiographical schema to and are now being born, in a sea of technology, surrounded
facilitate game progress. Differences, however, could also by all the gadgets of the digital age. Their education is
be due to procedural or recognition memory. provided in a completely different language, and as a result
To our knowledge, this is the only assessment of knowledge they think differently, and process information in ways that
acquisition and spectral mapping during gamified are functionally different to digital immigrants [25]. We
simulation play involving a bronchoscopy simulator, and found it interesting, but not altogether surprising, that the
the first study to demonstrate age related differences in age difference we noted in level achievement hinged on this
medical training game characteristics and performance. Our digital divide. While there were a couple of digital
immigrants in Group L3, the group was predominantly

2022
Gamification
composed of digital natives, whereas all Group L2
participants were digital immigrants in the age range 35–50.
The three participants who did not complete level two were
at the upper end of this range. As serious games provide a
digital form of knowledge acquisition, in the ‘language’ of
digital natives, it is possible that digital immigrants will
struggle to acquire knowledge through this mechanism.
In support of this assertion, completion assessment [23],
(whether a player successfully completes the game) is often
used as a simple indicator that the player sufficiently
understands the subject matter. In our study, the eight
participants in Group L2 did not complete the game in the
two sessions provided, and three of these did not even
complete level two, indicating that learning of the game
material did not occur during the study paradigm.
While digital immigrants may have difficulty learning
through serious games, preliminary evidence suggests that
serious games do enhance training outcomes for digital
natives. A study of three serious games conducted with
college students (born after 1980) demonstrated
significantly better test scores for students who learned
using serious games compared to those who learned
through traditional instruction [4].
We have not found any studies that compare serious game
or simulation performance in digital natives and digital
immigrants. Our findings suggest that knowledge
acquisition through existing serious games (and particularly
those designed by digital natives) could be more suited to
digital natives. One of the key findings in a review of
performance assessment in serious games is the importance
of scheduling of contents provided to the player [2].
Therefore, the design of serious games needs to take into
account personalization of player profiles, including age,
and structure the knowledge space accordingly.
Adaptations to improve knowledge acquisition
Game design
All participants were able to navigate level one of the
bronchoscopy simulator with ease, as would be expected
given the number of navigational and nomenclature
guidance cues. More than 50% of the attempted traversals
were successful (error-free) and, on average, it took less
than 15 minutes for participants to complete the level with
three consecutive error-free traversals. The difficulty for
Group L2 arose in the transition to level two. Here the
successful traversal rate dropped to less than 5%. There
were simply insufficient guidance cues to navigate the
bronchial tree in the correct order without error. For
participants who obtain low success rates in the
bronchoscopy simulator, additional steps would be
beneficial. Instead of removing all nomenclature for level
two, removing nomenclature from half the bronchial lumen
as an intermediary step, or a third at a time for additional
steps, may improve performance in this transition. Results
from Group L3 suggest that once level two is mastered,
2023
#chi4good, CHI 2016, San Jose, CA, USA
level three is completed with ease, and that this information
is retained at least in the short term. Therefore, this minor
change to the game design may have the potential to
significantly improve game performance for digital
immigrants.
Training regimes
The findings from this study indicate that simply extending
timeframes for training regimes would not be helpful for
those players who had trouble transitioning to level two. In
this study there were no time limits, the only limitation was
20 attempts at each level. Therefore, more training sessions,
rather than longer training sessions, would be most
beneficial. Alternatively, relaxing allowable error, by
increasing number of attempts, may facilitate learning.
Future work
The spectral analysis conducted in this study demonstrated
that Group L2 had a different spectral profile to Group L3.
Whether this profile is related to the divide between digital
immigrants and natives would need to be explored in
mutually exclusive groups. Neural differences in
information processing are already being postulated for
such functions as attention and memory [25], which both
play a role in learning. Extending studies to knowledge
acquisition during serious games has the potential to
provide greater insights into the applicability of serious
games in industries dominated by mature aged workers.
CONCLUSION
In this study, we found there were distinct features affecting
players’ performance in a gamified simulation that have the
potential to impact training outcomes, particularly for older
participants. Our hypothesis that there would be a different
spectral profile depending on player performance, as
measured through level achievement, was confirmed and
we found that this difference was evident in game play
characteristics, including number of errors and time to
complete traversals and therefore levels. We speculated that
the effect of these factors on performance could be
mitigated by minor step-wise changes to the game design
and extending training regimes in order to produce the best
possible outcomes for all trainees. Our novel findings are
relevant for disciplines considering training through
gamified simulation and serious games. Inclusion of
features to accommodate age related profiles could allow all
players to execute serious games in gamer style.
ACKNOWLEDGMENTS
We would like to thank all the participants who generously
gave their time to this project. Special thanks to John
Gardner for contributing to study design and Shlomo
Berkovsky and CHI reviewers for insightful comments on
the manuscript.
Gamification
REFERENCES
1. Australian Medical Workforce Advisory
Committee 2001. The Cardiothoracic Surgery
Workforce in Australia. Sydney.
2. Francesco Bellotti, Bill Kapralos, Kiju Lee, Pablo
Moreno-Ger, and Riccardo Berta. 2013.
Assessment in and of Serious Games: An
Overview. Advances in Human-Computer
Interaction. Article ID 136864.
3. Selim R. Benbadis, Diego A. Rielo, EEG Artifacts.
Retrieved 11th August from: emedicine.medscape.
com/article/1140247-overview.
4. Richard Blunt. 2009. Do serious games work?
Results from three studies, eLearn Magazine, no.
12.
5. Jeanne H. Brockmyer, Christine M. Fox, Kathleen
A. Curtiss, Evan McBroom, Kimberly M. Burkhart,
and Jacquelyn N. Pidruzny. 2009. The development
of the Game Engagement Questionnaire: a measure
of engagement in video game-playing. Journal of
Experimental Social Psychology, 45, 4: 624–634.
6. Margaret Crossley, Merrill Hiscock. 1992. Age-
related differences in concurrent-task performance
of normal adults: Evidence for a decline in
processing resources. Psychology and Aging, Vol
7, 4: 499-506.
7. Lotfi Derbali and Claude Frasson. 2010. Player’s
motivation and EEG Waves Patterns in a Serious
Game Environment. In Intelligent Tutoring Systems
Part II, Lecture Notes in Computer Science, 6095:
297-299.
8. Sebastian Deterding, Dan Dixon, Rilla Khaled, and
Lennart Nacke. 2011. From game design elements
to gamefulness: defining "gamification". In
Proceedings of the 15th International Academic
MindTrek Conference: Envisioning Future Media
Environments (MindTrek '11). ACM, New York,
NY, USA.
9. Cedric Dumas and Tim Coles. 2013. Bronchoscopy
Training. Australian eHealth Research Centre
Colloquium, 27 March, Brisbane.
10. Leonardo Fogassi, Giuseppe Luppino.
(2005).Motor functions of the parietal lobe. Current
Opinion in Neurobiology, 15:626-631.
11. Fong-Ling Fu, Rong-Chang Su, and Sheng-Chin
Yu. 2009. EGameFlow: a scale to measure learners'
enjoyment of e-learning games. Computers and
Education, 52, 1: 101–112.
12. Lluis Fuentemilla, Gareth Barnes, Emrah Düzel,
Brian Levine. 2014. Theta oscillations orchestrate
medial temporal lobe and neocortex in
remembering autobiographical memories,
NeuroImage, 85, 2: 730-737.
13. Alan Gevins, Mary E. Smith, Linda K. McEvoy,
and Daphne Yu. 1997. High-resolution EEG
mapping of cortical activation related to working
2024
#chi4good, CHI 2016, San Jose, CA, USA
memory: effects of task difficulty, type of
processing, and practice. Cereb Cortex, 7: 374-85.
14. Maurits Graafland, Jan M. Schraagen and Marlies
P. Schijven. 2012. Systematic review of serious
games for medical education and surgical skills
training. British Journal of Surgery; 99: 1322–
1330.
15. Eric J. He, Han Yuan, Lijing Yang, Cameron P.
Sheikholeslami, Bin He. 2008. EEG spatio-spectral
mapping during video game play. In Information
Technology and Applications in Biomedicine, 346-
348.
16. Health Workforce 2025 - Medical Specialties,
Volume 3, November 2012, Health Workforce
Australia.
17. Shigeto Ikeda, 1974. Atlas of flexible
Bronchofiberoscopy. Igaku Shoin.
18. Eric Kandel, James Schwartz, Thomas Jessell.
2000. Principles of Neural Science. McGraw-Hill.
19. Daniel Katz, Samuel DeMaria, Prabal Khanal,
Kanav Kahol. 2014. Serious Gaming to Improve
the Safety of Central Venous Catheter Placement.
International Journal of Gaming and Computer-
Mediated Simulations, 6: 4.
20. Catherine Lebel and Christian Beaulieu. 2011.
Longitudinal development of human brain wiring
continues from childhood into adulthood. The
Journal of Neuroscience, 31, 30: 10937-10947.
21. Kung-Yee Liang, Scott L. Zeger. 1986.
Longitudinal data analysis using generalized linear
models. Biometrika.13-22.
22. Timothy McMahan, Ian Parberry, Thomas D.
Parsons. 2015. Modality specific assessment of
video game player’s experience using the Emotiv.
Entertainment Computing, 7: 1-6. ISSN 1875-9521,
http://dx.doi.org/10.1016/j.entcom.2015.03.001.
23. David Michael and Sande Chen. 2005. Proof of
learning: assessment in serious games.
http://www.gamasutra.com/view/feature/2433/proof
_of_learning_assessment_in_.php.
24. Emiliana Pellouchoud, Michael E. Smith, Linda
McEvoy, and Alan Gevins. 1999. Mental effort-
related EEG modulation during video-game play:
comparison between juvenile subjects with epilepsy
and normal control subjects. Epilepsia, 40, 4: 38-
43.
25. Marc Prensky. 2001. Digital natives, digital
immigrants. On the Horizon. NCB University
Press, 9, 5.
26. Michael D. Rugg, 1995. Event-related potential
studies of human memory. In The Cognitive
Neurosciences (Gazzaniga, M.S., ed.), 789–801,
MIT Press.
27. Daniel Senkowski, Sophie Molholm, Manuel
Gomez-Ramirez and John J. Foxe. 2006.
Oscillatory Beta Activity Predicts Response Speed
during a Multisensory Audiovisual Reaction Time
Gamification
Task: A High-Density Electrical Mapping Study.
Cerebral Cortex 16:1556-1565.
28. Charles Shagass 1972. Electrical activity of the
brain. In: Greenfield NS, Sternbach RA,
editors. Handbook of Psychophysiology. New York:
Rinehart & Winston, 263–328.
29. Aidan Sliney, David Murphy. 2008. JDoc: A
Serious Game for Medical Learning, in Advances
in Computer-Human Interaction, 2008 First
International Conference on, 131-136.
30. Edward E. Smith and John Jonides, 1997. Working
memory: A view from neuroimaging. Cognitive
Psychology, 33: 5–42.
31. Michael Smith, Alan Gevins, Halle Brown, Arati
Karnik, Robert Du. (2001) Monitoring Task
Loading with Multivariate EEG Measures during
2025
#chi4good, CHI 2016, San Jose, CA, USA
Complex Forms of Human-Computer Interaction.
Human Factors: The Journal of the Human Factors
and Ergonomics Society. Fall vol. 43, 3:366-380.
32. Joseph Thompson, Mark Blair, Andrew Henrey.
2014. Over the Hill at 24: Persistent Age-Related
Cognitive-Motor Decline in Reaction Times in an
Ecologically Valid Video Game Task Begins in
Early Adulthood. PLoS ONE 9, 4: e94215.
33. Anthony D. Wagner, Benjamin J. Shannon, Itamar
Kahn, and Randy L. Buckner. 2005. Parietal lobe
contributions to episodic memory retrieval. Trends
in Cognitive Sciences 9: 9.
34. Voravika Wattanasoontorn, Ruben García, Imma
Boada, Mateu Sbert. 2013. Serious games for
health. Entertainment Computing 12, 4, 4:231–247.