Professional Documents
Culture Documents
Computers in Human Behavior
Computers in Human Behavior
a r t i c l e i n f o a b s t r a c t
Article history: Driving can be dangerous, especially for young and inexperienced drivers. To help address the issue of
Received 28 July 2014 inexperience, a gamified logbook smartphone application was designed and developed for learner
Received in revised form drivers in Queensland, Australia. The application aims to make it easy for learner drivers to record their
1 August 2016
mandatory practice sessions while the added gamification aims to encourage learners to undertake a
Accepted 31 August 2016
wider range of practice. Previous research reported on a lab-based study of a gamified version and a non-
Available online xxx
gamified version of this application. This paper presents an updated design of the application and in-
vestigates the effect of the application when tested in the field. Results are provided from a within-
Keywords:
Gamification
groups field study undertaken with 25 learner drivers over a four-week period, during which the ef-
Gameful fect of the gamification on behavior change, perceived motivation and user experience was studied.
Game design Although results suggest that the gamified logbook was perceived as more enjoyable and motivating
User experience than the non-gamified version, no significant change in behavior was found. This encourages discussion
Learner driver on the effectiveness of gamification to encourage behavior change and the feasibility of using gamifi-
Motivation cation in this particular context.
Behavior change © 2016 Published by Elsevier Ltd.
Smartphone
1. Introduction totaled 18.8% of the state's fatalities (DTMR, 2013), despite only
comprising 12.9% of the licensed population (DTMR, 2014a). In
Younger drivers are at a greater risk of death and injury from order to address this issue in Australia, state governments have
road crashes when compared to older, more experienced drivers. employed a range of strategies. In Queensland, Australia, learner
Road crashes are the leading cause of death for persons aged 16e25 drivers are required to undertake 100 h of supervised driving
years (e.g., OECD & ECMT, 2006; World Health Organization, 2014) practice. Although undertaking 100 h of mandatory practice can be
and the second most common cause of disability for male and fe- seen as encouraging a greater amount of driving experience, it
male adolescents alike (World Health Organization, 2014). In doesn't necessarily encourage a greater variety of practice (Scott-
Australia in 2013, young drivers aged 17e25 years contributed Parker, Bates, Watson, King, & Hyde, 2011). All drivers should be
21.3% of the fatally-injured drivers (BITRE, 2013). In the Australian able to handle themselves in a variety of different road conditions,
state of Queensland in 2014, 25.1% of all road user fatalities, and therefore, it is important that learner drivers undertake a wide
34.7% of all road user hospitalized casualties, arose from a crash range of practice while learning to drive with a supervisor.
involving a driver aged 17e24 years. These drivers themselves This paper presents the design of a gamified logbook application
that has been updated to address usability and playability issues
found in a previous lab-based study (Fitz-Walter, Wyeth,
Tjondronegoro, & Scott-Parker, 2013). A within-groups study was
* Corresponding author. S-Block, QUT Gardens Point, 2 George St, Queensland
undertaken in the field using the update design, with 25 learner
4001, Australia.
E-mail addresses: z.fitz-walter@qut.edu.au (Z. Fitz-Walter), dm.johnson@qut. drivers recruited as participants. The effect of the updated gamifi-
edu.au (D. Johnson), peta.wyeth@qut.edu.au (P. Wyeth), dian@qut.edu.au cation design on behavior change, motivation and user experience
(D. Tjondronegoro), bscottpa@usc.edu.au (B. Scott-Parker).
http://dx.doi.org/10.1016/j.chb.2016.08.050
0747-5632/© 2016 Published by Elsevier Ltd.
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050
2 Z. Fitz-Walter et al. / Computers in Human Behavior xxx (2016) 1e10
was studied. The results of the study suggest that the gamified (Fitz-Walter et al., 2013). The results of this study suggested that
logbook was more enjoyable and more motivating than a non- there was potential for a gamified logbook to be more motivating
gamified version. However, no significant effect on behavior than a non-gamified logbook. However, this study was a
change was measured. laboratory-based study using drivers who had recently graduated
from learner to provisional (intermediate/restricted) licensure.
2. Previous research Further research in the field is needed with learner drivers in order
to properly evaluate the effect of a gamified logbook application.
2.1. Learning to drive
2.3. Evaluating the effect of gamification
As of July 2007, all learner drivers in the state of Queensland,
Australia under the age of 25, must undertake a minimum of 100 h A popular construct investigated by empirical gamification
of supervised driving experience (including 10 h of driving at night) studies has been behavior change (e.g., Chiu et al., 2009; De Oliveira
before they can apply for a Provisional (intermediate) license et al., 2010; Fujiki et al., 2008; Landers & Callan, 2011; Li, Grossman,
(DTMR, 2014b). In Queensland, practice hours are logged manually & Fitzmaurice, 2012). Behavior change has often been measured in
in a large (approx. 16 cm 22.5 cm) logbook. The information previous studies by recording a participant's time and frequency
required to be logged for each driving session includes the date, the spent on a gamified activity. This usage data is generally obtained
time at the start and end of the session, the driving duration (mi- automatically by the gamified system using various sensors. For
nutes), the car's odometer at the start and end of the session (ki- example, Chiu et al. (2009) created a system that recorded daily
lometers), the license number of the supervising driver, the State in intake of water, and De Oliveria et al. (2010) created a system that
which the supervising driver is licensed, the car number plate, and automatically recorded when medication was taken. Gamified us-
if the person is a registered driving instructor. Once complete, the age data was often compared with usage data from either a pre-test
logbook is submitted to the state licensing authority, the Depart- or control group. A number of these studies in various domains
ment of Transport and Main Roads, and audited for accuracy prior have reported that the addition of game elements led to an increase
to permitting the young novice driver to undertake their practical in desired behavior during the study (e.g., Cafazzo et al., 2012; Chiu
driving assessment. In a previous study (Fitz-Walter et al., 2013) it et al., 2009; De Oliveira et al., 2010).
was proposed that a transition to a smartphone logbook application However, behavior change is only one part of a user's experi-
would make this process easier and may be more appropriate for ence. Most software applications also have usability goals, such as
the technologically-oriented youth of today. A smartphone appli- efficiency, learnability, good utility, and ease of use. Other hedo-
cation could also streamline the logging process by using sensors nistic user experience goals exist as well, such as entertainment,
on the smartphone (e.g., location, time and date) and could backup fun, and emotional fulfilment (Rogers, Sharp, & Preece, 2011). Some
data to a server in case the smartphone is broken, lost, or stolen. empirical gamification studies have explored the effect of game
Given that a breadth of experience is an important part of driver elements on usability (e.g., ease of use). Interestingly, some studies
safety, it is imperative that learner drivers undertake a wide range have reported potential problems; for example, adding game ele-
of practice while learning to drive. Although novice drivers repre- ments to non-game contexts has led to usability issues (Guin, Baker,
sent those facing the greatest risk of death and injury on the road, Mechling, & Ruylea, 2012), confused users (Montola, Nummenmaa,
the learning-to-drive stage itself is one the safest periods of any Lucero, Boberg, & Korhonen, 2009), and cheating (Singer &
driving career. This is because learner drivers are required to be Schneider, 2012). This suggests that in addition to the effect of
supervised by an experienced driver throughout this period. In the gamification on behavior change, it is worthwhile exploring the
local context of Queensland, Australia, recent research suggests that effect of gamification on usability and user experience constructs as
learner drivers should aim to spread the driving practice over the well. Therefore, exploring the use of gamification in this study also
learning period, be encouraged to continue to practice beyond the allows for further contributions to be made to this area of research.
mandated one hundred hours, and drive in a variety of different
circumstances that become progressively more challenging in na- 3. Gamified learner logbook design
ture (Scott-Parker et al., 2011). In order to motivate such driving
practice, we employed the use of gamification. This section briefly describes the first version of the gamified
logbook evaluated in a previous study and then describes the
2.2. An overview of gamification updated design of the logbook used for the field study.
Gamification is a design strategy where game design elements 3.1. First version
are used in non-game contexts to encourage behavior change. A
game design element in this sense could include anything from A learner logbook smartphone application was developed. This
game aesthetics, such as graphics and progress bars, to complete application allowed learner drivers to record their driving practice
games with goals, rules, levels, quests, and achievements using a smartphone device and view a summary of all previous
(Deterding, Dixon, Kahled, & Nacke, 2011). Results from previous practice sessions undertaken. The application made it easy to re-
studies have generally indicated that adding game elements can cord practice by automatically capturing some of the information
lead to positive behavior change (e.g., Cafazzo, Casselman, required (e.g., start time and location, end time and location,
Hamming, Katzman, & Palmert, 2012; Chiu et al., 2009; De weather, time of day) using smartphone sensors. A gamification
Oliveira, Cherubini, & Oliver, 2010; Farzan et al., 2008; Flatla, experience was subsequently designed for the logbook application
Gutwin, Nacke, Bateman, & Mandryk, 2011; Froehlich et al., 2009; by using a gamification design framework proposed in earlier
Fujiki et al., 2008; Gustafsson, Katzeff, & Bång, 2009; Landers & research (Fitz-Walter et al., 2013). The activity of undertaking a
Callan, 2011; Liu, Alexandrova, & Nakajima, 2011; Thom, Millen, & virtual road trip around Australia formed the basis for the gamifi-
DiMicco, 2012). Gamification could also be explored further as a cation experience. A road trip is something often associated with
way to encourage learner drivers to undertake diverse practice. the primary target demographic, where it can be a coming of age
This theory is supported by a previous lab-based study that activity that young people may often undertake when they attain
compared a gamified and non-gamified learner logbook application their license (Tourism Australia, 2014). Using a road trip theme also
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050
Z. Fitz-Walter et al. / Computers in Human Behavior xxx (2016) 1e10 3
provides a clear goal for the user e to make it all the way around non-gamified logbook. However, participants did no prefer one
Australia e as well as provide smaller sub-goals, such as visiting version over the other. The results also suggested that the proposed
each state in Australia, and also visiting particular towns and gamification design added a level of complexity that created
famous tourist attractions within these states. This gamification confusion for some participants. Also, some participants noted that
design added an extrinsic fantasy layer on top of the process of they could potentially cheat by adding practice manually to receive
learning to drive in an attempt to encourage more regular, diverse additional coins.
and extended driving practice.
An iterative design process was used where a number of pro- 3.2. Second version
totypes were created and used to gather feedback from a wide-
range of different people, including game designers and re- Using the results of this first study, an updated version of the
searchers, learner drivers, driving supervisors, and experts in the gamification design is proposed. The overall design has been
field of learner driver research. Rather than being simply reward- simplified while still retaining the same key functionality of the
based alone (e.g., drive for 10 h and receive points or a badge) the original application; a logbook, a record practice function, and a
road trip design aimed to provide an experience where interesting settings view. The updated logbook provides an overview of the
choices could be made by the player (e.g., which route should the total amount of practice recorded using the application, as well as
player take? which car should the player choose?). A gamification details of each individual practice session (see Fig. 2 e left). Further
design document was created (similar to a game design document) practice details are provided when a learner taps the total number
that outlined the story, mechanics, goals, rules, challenges, actions, of minutes logged (see Fig. 2 e right).
feedback, player strategies, and aesthetics of the design. Using the Learners can press the record button at any time to begin
gamification design document, a digital prototype was created (see recording a new practice session. After entering the current
Fig. 1) that could run on iOS devices (iPhone, iPod Touch and iPad). odometer reading of the car (see Fig. 3 e left) the application starts
In this prototype, when learner drivers recorded real-life practice to automatically record practice (see Fig. 3 e middle). Once learners
they received virtual coins which could be used to buy fuel and finish practicing, tapping the stop button allows them to finalize
items to help them progress along their road trip. Bonus coins were the session by entering the vehicle details, odometer reading, and
received when players undertook recording different driving skills supervisor (see Fig. 3 e right). All other information is calculated
(e.g., reverse parallel parking), drove in different contexts (e.g., automatically using smartphone sensors and the Internet.
unsealed road), or practiced regularly (e.g., twice a week), in order The updated gamification design still encourages learners to
to encourage these desired behaviors. undertake a virtual road trip around Australia. However, rather
A combined usability and playability study was undertaken in than have the learner collect coins to buy fuel to proceed further
order to evaluate the gamified logbook. The design of the gamifi- along their road trip, the design has been simplified. The learner's
cation and outcome of this study are detailed in the previous virtual car now drives 1 km for every kilometer the learner drives in
publication (Fitz-Walter et al., 2013). To briefly summarize, 12 real life. The road trip overview page shows how close learners are
recent learner drivers were recruited to participate in a combined to reaching the next city on the road trip (see Fig. 4). This view also
usability and playability study testing both a gamified and non- provides links to a summary of the road trip, a list of friends and
gamified version of the logbook application. The purpose of this their progress, achievements unlocked, and coins available to
study was to identify potential usability and playability issues, as spend.
well as compare the experience of the two different versions of the Coins are still awarded when learners undertook diverse and
logbook application. The findings of the study suggested that the regular practice. These bonus coins can be used to upgrade to a
gamified logbook was more enjoyable and motivating than the better car. New cars provide various bonuses that help learners
Fig. 1. Screenshots from the first road trip logbook prototype for iOS.
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050
4 Z. Fitz-Walter et al. / Computers in Human Behavior xxx (2016) 1e10
complete the road trip (see Fig. 5 e left). Additionally, learners can have the application are progressing on their own virtual road trip.
choose their next destination city for the journey in order to pro- This aims to encourage friendly competition and create shared
vide a greater sense of autonomy, as well as allow for some varying experiences (see Fig. 6 e left). Learners can also unlock driving
strategy. Learners can often choose between taking a quick route achievements along the way for completing various milestones
that is more difficult to travel without an upgraded vehicle, or a (e.g., undertaking 200 h of practice) which provides extra goals for
slower route that is easier to travel with any vehicle (see Fig. 5 e those interested (see Fig. 6 e right).
right). After practice is recorded, a virtual road trip screen is displayed
A friends list allows learners to see how their friends who also (see Fig. 7 e left). This shows the learner's car travelling to the next
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050
Z. Fitz-Walter et al. / Computers in Human Behavior xxx (2016) 1e10 5
destination on the road trip. This aims to provide a greater sense of information is provided to the learner that instructs them on how
progression, along with a breakdown of bonus coins they received they could receive coins on the next trip. This aims to provide useful
to give strategic feedback. If they did not receive any coins, further feedback to the learner. If the learner unlocks an achievement after
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050
6 Z. Fitz-Walter et al. / Computers in Human Behavior xxx (2016) 1e10
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050
Z. Fitz-Walter et al. / Computers in Human Behavior xxx (2016) 1e10 7
practicing then this is displayed using an alert (see Fig. 7 e right). 0.94). A number of additional questions were administered after
the gamified version was used to gather feedback on the experience
4. Field study of the gamification (e.g., “I found the game elements in this appli-
cation enjoyable”). Short, semi-structured interviews were also
4.1. Study overview undertaken to gather data to augment usage and questionnaire
findings, and to probe participants further about their experiences.
A field study was undertaken in order to evaluate the effec-
tiveness of the updated gamification design. The aim of the eval- 4.3. Study method
uation was to investigate the effect the updated gamification design
had on the user experience, perceived motivation, and behavior of Participants were recruited via a Facebook event page created
the learner. The evaluation also aimed to identify any potential is- for the study and by snowball-sampling techniques. Participants
sues with the gamification design. Three hypotheses were formed: had to meet two criteria for recruitment: (1) the participant had to
be currently learning to drive in Queensland, Australia, and (2) they
1. We predict that participants will have a more positive user had to own and use an iOS mobile device (iPhone, iPod touch or
experience when using the gamified version compared o using iPad) on which to use the application. Participants received a $50
the non-gamified version. gift voucher to thank them for their participation in the field study.
2. We predict that participants will feel more motivated when A 30-min introduction session took place with the participant
using the gamified version compared to using the non-gamified either in person or over the phone before the field study. Partici-
version pants completed an online questionnaire to provide demographic
3. We predict that the gamified version will have a positive effect information and were interviewed regarding their current practice
on behavior when compared to the non-gamified version. habits and experience using the physical logbook. Participants
received one of the two interventions to install, along with an
In addition to these hypotheses, two research questions were introduction to the assigned application. Participants were asked to
also investigated: use the application whenever they undertook driving practice over
the following two weeks. At the end of the two-week period an
1. Does the addition of game elements negatively affect the ease of online questionnaire was administered and a short, semi-
use of the gamified application compared to the non-gamified structured interview took place. Participants were then instructed
application? to delete the application and were given a link to download the
2. Does the addition of game elements lead to any unintended other version of the application. They were asked to use the alter-
consequences compared to the non-gamified application? native version for two weeks. The same procedure was followed for
the second intervention with the addition of a number of further
The field study ran over four weeks during the months of July questions which asked participants to compare their experience of
and August 2013. Twenty-five participants undertook the study both versions of the application (e.g., “Of the two applications, which
(male ¼ 11, 44%; female ¼ 14, 56%). Their ages ranged from 16 to 28 was more motivating to use? Why/Why not?”).
years old, with an average age of 19.44 years (SD ¼ 3.43). All par-
ticipants held a Queensland Learners license during the period of 5. Results
the study. On average, the participants had held their licenses for 22
months (SD ¼ 19.331). During the study all participants tried both a To test the hypotheses, and to explore the first research ques-
gamified version and a non-gamified version of the application for tion, a series of Wilcoxon signed-rank tests were conducted
two weeks each, distributed in a random, counterbalanced order. comparing the non-gamified and gamified versions of the appli-
Driving practice was automatically captured by the device, and a cation. To control the familywise experimental error rate we
questionnaire and interview were administered at the end of each applied a Bonferroni adjustment resulting in a required significance
two-week period to measure the effect of each application. Data level of 0.0055 for the nine Wilcoxon signed-rank tests (Field,
from four participants was unable to be captured due to technical 2013). To explore research the second research question the inter-
limitations of their devices, therefore only 21 sets of data were used view results were coded and common themes identified.
for behavior change comparisons.
5.1. User experience results
4.2. Study measures
The gamified version of the application elicited a statistically
Driving behavior data was automatically sent to the researcher significant median increase in enjoyment (z ¼ 4.037, p < 0.0001,
each time a participant completed a practice session. In addition to gamified median ¼ 6, non-gamified median ¼ 5) with 21 partici-
the amount of practice undertaken, the skills the participant pants reporting a positive difference in enjoyment in response to
attempted in each practice session were also captured. The ques- the gamified version, 3 participants reporting no difference, and 1
tionnaire administered to the participant after each two-week participant reporting a negative difference. There were no signifi-
period adopted validated multi-item measurement scales from cant median increases in reported concentration, perceived control,
Koufaris (2002) for measuring concentration, enjoyment, perceived challenge, perceived ease of use, and perceived usefulness when
control, challenge, perceived ease of use, and perceived usefulness. using the gamified version compared to the non-gamified version.
These scales used 7-point, Likert-type responses. Additionally, in
order to measure participants' feelings of engagement and moti- 5.2. Motivation results
vation a two-item scale was developed (“While using the logbook
application I found it engaging”, “While using the logbook application I The gamified version of the application elicited a statistically
found it motivating”). To assess the reliability of the additional scales significant median increase in reported motivation (z ¼ 2.804,
developed for the questionnaire, Cronbach's alpha was calculated p ¼ 0.005, gamified median ¼ 6.5, non-gamified median ¼ 5.5)
for both the gamified and non-gamified responses and all scales with 16 participants reporting a positive difference in motivation in
were found to have acceptable reliability (alpha between 0.7 and response to the gamified version, 6 participants reporting no
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050
8 Z. Fitz-Walter et al. / Computers in Human Behavior xxx (2016) 1e10
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050
Z. Fitz-Walter et al. / Computers in Human Behavior xxx (2016) 1e10 9
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050
10 Z. Fitz-Walter et al. / Computers in Human Behavior xxx (2016) 1e10
Please cite this article in press as: Fitz-Walter, Z., et al., Driven to drive? Investigating the effect of gamification on learner driver behavior,
perceived motivation and user experience, Computers in Human Behavior (2016), http://dx.doi.org/10.1016/j.chb.2016.08.050