E Games 6

Computers & Education 86 (2015) 192e211
Contents lists available at ScienceDirect
Computers & Education

journal homepage: www.elsevier.com/locate/compedu
An application of adaptive games-based learning based on learning

style to teach SQL
Mario Soflano a, *, Thomas M. Connolly b, Thomas Hainey c
a
E260, Paisley Campus, University of the West of Scotland, UK
b
c
a r t i c l e i n f o a b s t r a c t
Article history: The fact that each student has a different way of learning and processing information has long been
Received 16 July 2014 recognised by educationalists. In the classroom, the benefits derived from delivering learning content in
Received in revised form ways that match the student's learning style have also been identified. As new modes of delivery of
21 March 2015
learning content such as computer-assisted learning systems (e.g. eLearning) have become increasingly
Accepted 24 March 2015
Available online 4 April 2015
popular, research into these has also identified the benefits of tailoring learning content to learning
styles. However, in games-based learning (GBL), the adaptation based on learning style to enhance the
educational experience has not been well researched. For the purpose of this research, a game with three
Keywords:
Games-based learning game modes has been developed: 1) non-adaptivity mode; 2) a mode that customises the game ac-
Adaptivity cording to the student's learning style identified by using a learning style questionnaire; and 3) a mode
Learning style that has an in-game adaptive system that dynamically and continuously adapts its content according to
SQL the student's interactions in the game.
Role-playing games This paper discusses the term adaptivity in a GBL context and presents the results of an experimental
study investigating the differences in learning effectiveness of the different game modes compared to a
paper-based learning. The study was performed with 120 Higher Education students learning the
database language SQL (Structured Query Language). The results show that the game developed,
regardless of mode, produced better learning outcomes than those who learned from a textbook while
adaptive GBL was better in terms of allowing learners to complete the tasks faster than the other two
game versions.
© 2015 Elsevier Ltd. All rights reserved.
1. Introduction
The fact that each student has a different way of learning and processing information has long been recognised by educationalists (Felder
& Brent, 2005; Kolb, 1984). In the classroom, the benefits derived from delivering learning content in ways that match the student's learning
style have also been identified (Price, 2004; Smith & Renzulli, 1984). As new modes of delivery of learning content such as computer-assisted
learning systems (e.g. eLearning) have become increasingly popular, research into these has also identified the benefits of tailoring learning
content to learning styles (Miller, 2005).
However, as Connolly and Stansfield (2006) have suggested, eLearning simply replicates the traditional education system (classroom
style) and may be overly focussed on method of delivery, i.e. delivering materials over the web rather than on actual teaching and learning,
and indeed motivating and engaging the students in the learning process. In contrast, games, particularly video games, appear to be able to
engage people over extensive periods of time and also motivate them to re-play the game repeatedly until they have mastered it (Kirriemuir
& McFarlane, 2004). Therefore, some educationalists (for example, Prensky, 2006) have considered games to be a potential platform in
supporting student learning and have turned their attention to what is now called games-based learning (GBL).
* Corresponding author.
E-mail addresses: mario.soflano@uws.ac.uk (M. Soflano), thomas.connolly@uws.ac.uk (T.M. Connolly), thomas.hainey@uws.ac.uk (T. Hainey).
http://dx.doi.org/10.1016/j.compedu.2015.03.015
0360-1315/© 2015 Elsevier Ltd. All rights reserved.
M. Soflano et al. / Computers & Education 86 (2015) 192e211 193
While many GBL applications have been developed in the last two decades, there remains a lack of empirical evidence to support the use
of GBL for learning purposes (Connolly, Boyle, MacArthur, Hainey, & Boyle, 2012). Given that there appears to be genuine advantages for
learning outcomes to be derived from the adaptation of teaching materials to learning styles in the classroom and remotely through
eLearning, it may also be possible that GBL applications that are adapted to the individual's learning style would improve learning
outcomes.
Kirriemuir and McFarlane (2004) have suggested that games, unlike classroom learning or eLearning, provide a different type of
engagement as they demand constant interaction and generate a ‘flow’ that could assist in engaging students. It is therefore possible for
students to adopt different leaning styles in GBL than they adopt in other learning settings.
In the next section, adaptivity is discussed followed by a short review of learning style theories and a review of previous empirical work
in adaptive GBL particularly those based on learning styles. Section 3 presents the game that has been created for this research study and
Section 4 discusses the research methodology used to evaluate the game. Section 5 discusses the evaluation of the adaptive GBL application.
The evaluation includes analysis of the difference in learning effectiveness and completion times between experimental groups. Section 6
provides conclusions from the study and discusses future research directions.
2. Previous research
2.1. Definition of adaptivity and adaptability
In computing, there are two types of adaptation process: adaptability and adaptivity (Jameson, 2003). Adaptability refers to the ability of
the user to ‘adapt’ to the system by explicitly customising the system according to their preferences (Bontcheva, 2002). On the other hand,
adaptivity, which is usually used in the context of a user-adaptive system, refers to the ability of the system to identify the user's preferences
or characteristics and customise the system accordingly; that is, the user implicitly influences the adaptation process (Mulwa, Lawless,
Sharp, Arnedillo-Sanchez, & Wade, 2010).
In modern computer systems, adaptability is usually implemented by providing options that allow the user to customise the system
according to their preferences. For example, in eLearning a student can choose a font size and font style associated with the learning
materials. Conversely, adaptivity does not explicitly require input from the users and it is usually hidden from them. The users simply see the
result of the customisation process provided by the system.
Although both types can exist in a computer system, each type has differences in terms of its usage. Nowadays, adaptability through user
customisation is widely adopted and exists in many computer systems. Whilst it is certainly useful in some circumstances, adaptability
requires direct manipulation from the user that can result in an increase in the user's cognitive load, especially if there are many options the
user needs to choose from (Oppermann, Rashev, & Kinshuk, 1997). Cognitive load theory (Sweller, 1994) suggests that learning may be
enhanced if instructional materials are designed to reduce the levels of extraneous cognitive load.
Conversely, adaptivity can capture the interactions between the user and the computer system and the adaptivity analyses the historical
interactions before making an automatic adjustment. Adaptivity is considered to be less intrusive compared to adaptability as it does not
require the user to make any changes and, as a result, the interaction between the user and the system can be maintained. This is useful
especially in a system that has a considerable amount of elements that the user would need to manipulate. In addition, adaptivity can be
used for a behavioural pattern analysis of the user's interaction with the system. Such an analysis may be used to create different user
experiences. The disadvantage of adaptivity is that the user does not have direct control in customising the system.
According to Ismailovic, Haladjian, Kohler, Pagano, and Brugge (2012), adaptivity involves players monitoring, characterisation and
assessment and adaptive intervention. The purpose of adaptivity is to support, encourage and motivate the player by considering the
player's strength and weaknesses (Budiharto, Rachmawati, Ricky, & Chyntia, 2013; Gasimov, Tan, Phang, & Sutanto, 2010; Ming-Chun &
Shyan-Ming, 2013). Adaptivity has also been used to make games more challenging, engaging and fun as well as to increase replayability
(Ibanez & Delgado-Mata, 2011; Li, Yao, Li, & Zhang, 2014; Sweetser & Wyeth, 2005). However, as noted in the comprehensive survey by
Karpinskyj, Zambetta, and Cavedon (2014), there is still a lack of empirical evidence related to the effect of adaptivity in player experience.
Doukianou, Petridis, and Dunwell (2014) has also noted the limited empirical evidence in games-based learning particularly to assess
human attributes and behaviours.
For this particular research, adaptivity is used to refer to the system's ability to automatically customise certain elements of the system
based on a series of the student's interactions with the system.
According to Charles et al. (2005) and Melis and Monthienvichienchai (2004) adaptation can be incorporated into games through:
1) A player's character: all actions undertaken by the character have implications; for example, if the character is wounded, the movement
of the character is slower.
2) Non-player character (NPC): the player can access this feedback by ‘talking’ to the NPC. Besides providing feedback, the conversation
itself may be used to alter the story based on the selections the player has made in the conversation.
3) The game environment: adaptation through the game environment can be categorised into: customisation, contextualisation and per-
sonalisation. In the context of GBL, customisation is related to the functionalities of the learning system; for example, when the student
reaches a particular level, a new control will appear. Contextualisation means that the content that is going to be delivered is adaptable
according to the student's performance, learning history or response to certain missions. Personalisation relates more to the student's
preference, for example, the font size.
4) Feedback/scaffolding: can be used to inform the student about their status in the game and to help the student to achieve the game and
educational objectives (Jackson, Krajcik, & Soloway, 1998).
In this paper, adaptivity was implemented through the conversation system where the adaptive system analyses the preferences of the
players and adapts the type of presentation on the next conversation between the player and NPC. The adaptivity mechanism of this
research is discussed further in Section 3.3.
194 M. Soflano et al. / Computers & Education 86 (2015) 192e211
2.2. Learning styles
While many learning styles have been proposed, one of the learning styles that has been used in many computer-assisted learning
applications with its reliability and validity being confirmed is the Felder-Silverman learning style. The Felder-Silverman learning style
model (1988) was developed by Richard Felder and Linda Silverman. The model consists of four main elements as shown in Table 2 (Graf,
Viola, Leo, & Kinshuk, 2007; Sharda, 2007) with two of the main elements replicating those selected by Kolb and MBTI (perception and
processing). As shown in Table 1, each main element consists of two sub-elements that represent the element a person may have, for
example a person may prefer to use visual or verbal materials (Graf et al., 2007).
The Perception element consists of two sub-elements: sensing and intuitive. Sensing students prefer to learn facts and concrete learning
materials, whilst intuitive students prefer abstract materials with general principles rather than concrete materials. The Input element
relates to the presentation of materials preferred by students. Visual students prefer to learn using pictures, diagrams, graphics, etc., whilst
verbal students prefer to listen or read the learning materials. Verbal students tend to be able to remember written or spoken text or a
sentence. For example, in an advertisement, some people remember the slogan, whilst others remember what the advertisement looks like.
In the Processing element, active students prefer to learn by doing and enjoy discussion with other people, whilst reflective students
prefer to reflect on the learning materials alone (learning by thinking it through). Active students can be categorised as extroverted and
reflective students can be categorised as introverted. In the Organisation of materials, sequential students learn in a step-by-step/linear
sequence, following a logical sequence, whilst global students absorb learning materials randomly and when they have learned enough,
they will understand the whole picture. Global students can also be categorised as holistic students and sequential students as serial
students whilst sensing e sequential students have a tendency to be convergent and intuitive and global students have a tendency to be
divergent students.
Amongst the theories, the Felder-Silverman learning style has been identified as having a greater number of benefits compared to other
learning style theories. The Felder-Silverman learning style model has been widely used in eLearning and GBL and its reliability and validity
have been tested (Cook, 2005; Kuljis & Liu, 2005; Litzinger, Lee, Wise, & Felder, 2005; Livesay, Dee, Nauman, & Hites, 2002; Van Zwanenberg,
Wilkinson, & Anderson, 2000; Zywno, 2003). When compared to other learning style models, the Felder-Silverman model represents el-
ements from most models, which indicates the generalisability of the model (Limongelli, Sciarrone, Temperini, & Vaste, 2009).
However, not all elements of learning style theory can be adopted in GBL. According to Becker (2007) one of the main characteristic of
games, regardless of the genre, is their interactivity, which requires players to actively interact with the game, and indicates that the
‘reflection’ element of the Felder-Silverman learning style may not be relevant to GBL. This statement is also supported by Boyle, Connolly,
and Hainey (2011) who indicated that games provided an active, experiential, situated and problem-based learning environment.
2.3. Adaptive GBL applications e empirical evidence
A systematic literature search was conducted on electronic databases including: Association for Computing Machinery (ACM), Cam-
bridge Journals Online, Institute of Electrical and Electronic Engineering (IEEE), Index to theses, IGI Global, Ingenta Connect, Science Direct,
Springer Link, Wiley Online Library, Extended Academic ASAP, Simulation and Gaming and Emerald. The following search terms were used:
(adaptive or adaptivity or personalisation or personalization or “learning style”) and (“serious games” OR “games-based learning”)
The search returned 1334 papers, however, after detailed analysis only 27 papers were identified as being relevant to adaptive GBL
applications with empirical evidence and these are summarised in Table 2 while only 2 papers are related to adaptive GBL based on learning
style.
Amongst the papers found, only two research papers contain empirical evidence related to learning styles: Lynch et al. (2008) and Hwang
et al. (2012). Lynch et al. (2008) developed Ootle-U to teach aspects of food safety. The game is adapted to the student's learning style, based
on Dunn and Dunn's theory: motivation/learning enjoyment (M); persistence towards task completion (P); sense of responsibility (R);
structure (S); alone versus peer (AP); auditory (A); visual (V); tactile (T); and kinesthetic (K). The authors suggested that M, P, R and S were
related to emotional preferences while A, V, T and K could be categorised as perceptual preferences. Their experiment involved 217 par-
ticipants to measure the learning effectiveness of the game. The effectiveness was measured by comparing the pre-test and post-test scores.
The experiment also assessed the impact of learning style on learning effectiveness and the correlation between the learning achievement
and the game. The results showed that the participants' knowledge in food safety was improved although the results indicated that the
improvement was not significant because the students had prior knowledge of the subject matter. With respect to learning style, the
experiment indicated that a greater number of participants preferred to mix their learning methods rather than consistently use a single
Table 1
Elements of Felder-Silverman's learning style.
Main Sub-elements A Sub-elements B

elements
Perception Sensing: prefer to use existing ways than exploring new ways, prefer to Intuitive: prefer to try new ways, prefer to learn abstract material,
learn concrete materials, careful with details not careful with details
Input Visual: prefer to learn materials presented in pictures, chart or diagrams Verbal: prefer to learn materials by text or audio, having difficulty
with visual style
Processing Active: learning by doing, social oriented Reflective: learning by thinking it through, impersonal oriented
Organisation Sequential: building up from specific knowledge until a more general Global: (learning from a general knowledge into a more specific
knowledge is attained knowledge
Table 2
Existing empirical evidence in adaptive GBL.
Authors Area Adaptivity

Felicia and Pitt (2007, 2008, 2009) Maths Learning style and personality traits (The Big-5 model).
Conati and Zhou (2002), Conati and Zhao (2004) Maths Cognitive theory of emotions (joy, distress, pride,
shame, admiration and reproach).
Peirce, Conlan, and Wade (2008) Physics of optics Number of interactions between the student and
certain elements of the game.
Lynch, Steen, Pritchard, Buzzell, and Pintauro (2008) Food safety Learning style based on Dunn and Dunn's theory.
Hwang, Sung, Hung, Huang, and Tsai (2012) Plantation Felder-Silverman (sequential e global).
Lee and Ko (2011) Logic programming Successful rate of the code execution.
Yongyuth, Prada, Nakasone, Kawtrakul, and Agriculture Changes occurring in the environment caused by
Prendinger (2010) the student's actions.
Demmel, Kohler, Krusche, and Schubert (2011) Languages Number of mistakes the student has made.
Smeddinck, Siegel, and Herrlich (2013) Exercise for Parkinson's Disease Patients Speed and Accuracy
Rodriguez-Cerezo, Sarasa-Cabezuelo, Gomez-Albarran, Introductory Computer Language Number of mistakes the student has made.
and Sierra, (2014)
Bernardini, Porayska-Pomsta, and Smith (2013) For young children with Autism Spectrum Cognitive and affective state
Conditions (ASC) practice
Sampayo-Vargas, Cope, He, and Byrne (2013) Spanish cognates The performance of the student
Kerfoot and Baker (2012) Urology Number of mistakes the student has made.
Hertzog, Poussin, Tangara, Kouriba, and Jamin (2014) Water Management/Irigation Changes occurring in the environment caused by
the player's actions.
Lester, Lobene, Mott, and Rowe (2014) Microbiology Paths/actions chosen by the player
Wilson, Dehaene, Dubois, and Fayol (2009); Siegler Numbers The performance of the student
and Ramani (2011)
Praet and Desoete (2014) Arithmetic The performance of the student
Kujala, Richardson, and Lyytinen (2010) Reading (Finnish) The performance of the student
Berns, Gonzalez-Pardo, and Camacho (2013) Language (German) The performance of the student
Tavassolian, Stanley, and Gutwin (2014) Time-balancing The performance of the student
Zhang et al. (2012) Exergame, Chinese culture, Olympics The performance of the student
Tsai, Lo, and Chen (2012) Language (Chinese) Facial expression
Tong and Chignell (2014) Cognitive Assessment The performance of the student
learning method. The authors suggested that perhaps the participants, who were considered as digital natives, interacted with various
different media in their everyday life.
Hwang et al. (2012) developed an adaptive GBL application based on learning style to teach about plantation. Their evaluation of the
game involved 46 participants and they investigated whether learning in a game that could match the student's learning style was better
than a game without any consideration of the student's learning style in terms of learning effectiveness and motivation. The experiment also
evaluated the easiness and usefulness of the game. A pre-test/post-test experimental design was used with an equal number of participants
in the control group (who played the game without any adaptivity) and the experimental group (who played the game with adaptivity based
on the learning style identified before the game). The learning style adopted in this research was the ‘sequential/global’ element of the
Felder-Silverman learning style, identified using their learning style questionnaire, while a test sheet was used in the pre-test and post-test
to measure the knowledge of the participant on the subject matter. The results showed that the adaptive version had a significantly higher
learning effectiveness.
From the empirical evidence generated from the adaptive GBL that has been discussed above, adaptivity has the potential for improving
learning effectiveness in GBL. Moreover, it has also shown:
The current trend in adaptive GBL research is the investigation of the effect of theories used as the basis for the adaptivity. Such theories
may be personality theories as researched by Felicia and Pitt (2009) or learning styles as used by Lynch et al. (2008) and Hwang et al.
(2012).
The majority of research in adaptive GBL uses a pre-test/post-test experimental design. This is mostly used when measuring learning
effectiveness as reflected in the differences between pre-test and post-test either within or between control and experimental groups.
The analysis methods adopted in the studies are mainly quantitative.
The adaptive systems mostly use log files or a database to collect information about interactions between the games and the students.
3. Learning-style-based adaptive GBL
For the purpose of this research, an adaptive GBL application based on learning styles was developed. The game was intended to teach
the basics of the database programming language SQL (Structured Query Language) while the learning style adopted in this game was the
Felder-Silverman learning style model, particularly the presentation elements (picture-text). The Felder-Silverman learning style model has
been widely used in eLearning and its reliability and validity have been tested. When compared to other learning style models, the Felder-
Silverman model represents elements from most models, which indicates the generalisability of the model. The selected genre of the game
developed for this research was role-playing games (RPG) using the NeverWinter Nights 2 engine.
In this study, there were three modes of the same game designed and developed:
A non-adaptive mode of the game. This mode treats all students the same and takes no account of the student's learning style.
An out-of-game adaptive mode. The characteristics of a student are identified by means of the Felder-Silverman learning style ques-
tionnaire completed in advance of playing the game and the gameplay is then customised according to the student's learning style.
An in-game adaptive mode. In this mode, the student's characteristics are identified during the gameplay. As it is possible for the
student to change learning style in the course of the game, the game has an adaptive system that can automatically customise the game
in real-time according to the student's current learning style. The student's current learning style is identified by analysing historical
interactions between the student and the game.
The difference between the modes concerns the nature of the adaptive approach adopted while the rest of the game elements such as
storyline, game environment, controls and game interface, are identical. The adaptive approach itself was implemented through the pre-
sentation of the learning materials presented by the conversation system of the game.
3.1. SQL
In this research, a module on Structured Query Language (SQL) was selected as the basis of research. SQL was developed as the standard
language to allow an individual to define and manipulate data stored in a relational database. SQL is used in many modern Database
Management Systems (DBMSs) available today and the language itself is descriptive and has a similar structure to English. Therefore, SQL
has been identified as one of the essential programming skills and it is normally taught as part of computing/IT courses.
SQL has two major components (Connolly & Begg, 2014):
Data Definition Language (DDL): related to the structural design of the database, such as: CREATE TABLE, ALTER TABLE and DROP TABLE.
Data Manipulation Language (DML): related to data manipulation such as retrieval, update, delete and insert.
DDL is normally used during the database design process while DML is used to manage the data flow from and into the database
including insert, update and delete data and more importantly the learner also can specify which data to be retrieved according to his/her
needs. The learning outcomes for the game to be developed in this research are as follows:
How to write an SQL command.

How to retrieve data from the database using the SELECT statement.
How to build SQL statements that:
- use the WHERE clause to retrieve rows that satisfy a simple condition;
- use the WHERE clause to retrieve rows that satisfy multiple conditions;
- sort query results using ORDER BY;
- use the aggregate functions of SQL;
- use of the DISTINCT keyword to eliminate duplicates rows from the result.
This will allow students to construct some basic forms of SQL retrieval syntax (SQL SELECT statements), such as:
SELECT * FROM user_tbl

SELECT * FROM user_tbl WHERE city ¼ ‘Glasgow’
SELECT firstname, lastname FROM user_tbl WHERE age > 27 AND city ¼ ‘Glasgow’
SELECT firstname, lastname FROM user_tbl ORDER BY age DESC
SELECT COUNT(*) FROM user_tbl
SELECT MIN (age) FROM user_tbl
SELECT MAX (age) FROM user_tbl
SELECT AVG (age) FROM user_tbl
SELECT DISTINCT (city) FROM user_tbl
In a database and SQL course, a case study is often used that requires learners to analyse a particular task and then create an SQL
statement according to the needs identified. In this research, this case study will be implemented through the game story. The game element
is what is expected to motivate students to learn SQL and also to give them the opportunity to test their knowledge through the game
missions. The process of completing the game involves a learner in appropriately applying the material delivered to create the correct SQL
query if they are to make progress through the game.
3.2. Gameplay
The game was developed to achieve the above learning outcomes in a 3D Role-Playing-Game (RPG) created through the Aurora toolset
from NeverWinter Nights 2. In this game, learners control an avatar that they can personalise at the beginning of the game. The avatar can be
controlled by point-and-click using a mouse and also by using keyboard shortcuts. The same method also controls the game's 3D camera.
3.3. Game story
When designing an educational game it is necessary to consider how to teach the learning materials without losing the ‘fun’ part of the
game so learners can remain motivated to learn the materials. One way of maintaining motivation is to blend the materials and the game
story in such a way that the learning materials are part of the challenge of the game (Marsh, Li, Klopfer, Chuang, Osterweil & Haas, 2011). It
was decided to base the game on the story of a criminal investigator, a role to be adopted by the learner, who undertakes quests for in-
formation to arrest criminal masterminds, using warrants the learner gains based on the evidence and information collected. During the
Fig. 1. Conversation sequence e first.
Fig. 2. Conversation sequence e second.
course of the game, the learner is also required to collect data and evidence by talking to Non-Player Characters (NPCs) before a warrant to
arrest the criminals can be issued. Figs. 1e3 show the sequence of conversation on the first mission.
The data and evidence collected are automatically stored in a fictional database while the conversation system allows the player to
choose the learning style, in this case the type of presentation, preferred. In the non-adaptive mode of the game, the game is passively
recording the type of presentation chosen by the player. In the in-game adaptive mode, the game's adaptive system automatically sets the
default type of presentation according to the type selected the most by the player. On each conversation, the player is also provided with an
option to change the type of presentation and such changes are recorded and analysed by the adaptive system for the next conversation.
Fig. 3. Conversation sequence e third.

The game is designed to require the learner to retrieve the required data and evidence from the fictional database by using appropriate
SELECT statements in order to get the warrant. The game structures learning by increasing the level of complexity of the SQL queries that the
learner must construct as the game progresses. When the player makes a mistake when constructing the statements, the player can retry it
and also will be given a chance to choose their preferred learning style by selecting their preferred presentation. As the process of real-life
crime investigation parallels to some extent the process of investigation reflected in the game story, this element may help the learner to
understand how the learning content is applied in real life.
The game contains three main missions and a number of side missions. For each mission, the learner learns different forms of SELECT
statements specified above. To help the learner complete the tasks, there are instructions specific to each task and the learner can choose
how these instructions are presented, either as text or picture/diagram, to reflect their learning style. By completing these tasks, the learner
can learn how to create and use SELECT statements by entering the appropriate statement into a textbox to solve a given task. The learner is
given six chances to create the SQL. Once the learner has successfully created SQL queries in response to the task requirements, the warrant
to arrest the key criminals will be issued, which will allow the learner to arrest these criminals. If the learner fails to create the correct SQL
after the six chances, the learner will still get the warrant so the learner is not blocked from progressing. When the key criminal on a mission
is arrested, the learner will receive a reward and positive feedback from NPCs, such as the chief inspector who usually gives the missions, and
then the game story will proceed to the next mission.
As for the detail of the story, at the beginning of the game the learner can create and personalise his or her own avatar. Once the avatar is
created, the learner will appear in the police headquarters as a new recruit. In the first mission, the learner is required to investigate the case
of a missing classified document, which later connects to further missions investigating cases of corruption, potential rebellion and an
assassination plot. For the purposes of gathering data and evidence, students will have to travel to particular areas of the game world to talk
to NPCs and sometimes they will encounter enemies. The game also contains fighting scenes that require a simple point-and-click using the
mouse. The storyline is designed in such a way that the learner can easily find out what to do by using the ‘Journal’, which provides details of
the missions including the relevant locations for the missions, and where to go by using a map that pinpoints important characters and
locations. Provision of the ‘Journal’ and the map are to help the learner focus on solving the missions, which are related to the learning
objectives, rather than be distracted by the mechanics of the game with related problems such as getting lost when trying to find the
relevant places or wasting time in finding NPCs. Figs. 4 and 5 provide some screenshots from the game.
4. Research questions and methodology
4.1. Research questions
Research Question: How do GBL and adaptive GBL compare to traditional (paper-based) learning?
The main purpose of this research question is to evaluate whether the use of a game with adaptivity based on a student's in-game
learning style can improve that student's learning compared to the use of one without any use of adaptivity, or a game with adaptivity
based on a student's learning style as identified from a learning style questionnaire and non-game/text-based learning. Thus, this exper-
iment comprises four independent groups:
A paper-based group that require the participants in the group to read the SQL materials from a textbook (control group);
A ‘non-adaptive game’ group that will play the game in non-adaptivity mode;
Fig. 4. Screenshot of an area e Central Plain.

Fig. 5. Screenshot of the map, inventory and biodata notebook.
An ‘out-of-game adaptive game’ group that will have their learning style identified by a learning-style questionnaire before playing the
game and each participant in the group will play the game that is customised to his or her previously identified learning style;
An ‘in-game adaptivity game’ group that will play the game in the in-game-adaptivity mode. This mode allows the game to auto-
matically identify and adapt to a student's learning style based on his/her interaction with the learning content.
The learning outcomes of each group were then analysed and compared.
4.2. Methodology
In this experiment, 120 university students with no knowledge of SQL voluntarily participated in the study. Only the participants who
scored 0 in the pre-test SQL test (Appendix 2) were selected in this study. The participants were 83 undergraduate students (69.2%) and 37
postgraduate students (30.8%) with 43 students from the University of the West of Scotland (35.8%), 46 students from Heriot-Watt Uni-
versity (38.3%), 19 students from Napier University (15.8%), 9 students from University of Edinburgh (7.5%), 2 students from Glasgow
University (1.7%) and 1 student from Strathclyde University (0.8%). The participants were also selected from various programmes. There
were 78 participants (65%) from computing programmes, 1 student from accounting (0.8%), 3 students from business (2.5%), 10 students
from engineering (8.3%), 1 student from finance (0.8%), 3 students from languages (2.5%), 1 student from law (0.8%), 7 students from life
science (5.8%), 9 students from management (7.5%), 2 students from mathematics (1.8%), 1 student from medicine (0.8%), 1 student from
nursing (0.8%), 1 student from psychology (0.8%) and 2 students from social science programmes (1.8%). For the purposes of analysis, the
participants were categorised into two groups, computing and non-computing students, because the number of participants in each of the
non-computing programmes were too low to be analysed individually.
The participants were recruited from the universities by visiting campuses, classes, libraries and student accommodation. Particularly at
UWS, student mailing lists and posters were used. The participants were recruited voluntarily without any incentive offered. When
approached, the participants were asked if they would like to participate in the experiment and the process of the experiment was
described. The participants were told the experiment was to investigate how the subject (SQL) is learnt and that this experiment would
involve tests to measure the learning effectiveness. There was no time limit for the experiment, however, the approximate duration of this
experiment was explained at this point. The treatment on each group described on Section 5.1 was also explained. Once the participants
agreed to voluntarily participate in the experiment, the participants were then randomly allocated to a group. For those who were allocated
to the game groups, the experiment was conducted by using the researcher's laptops while the paper-based group participants were given a
textbook. The textbook used was Database Systems: A Practical Approach to Design, Implementation and Management by Connolly and
Begg (2004) and the highlighted pages were the basic SELECT, conditional SELECT and aggregate functions (page 117e133) although the
participants had the freedom to read any sections they wished. The textbook contains a case study including SQL examples and solutions.
Besides presented examples and solutions relevant to the case study implemented in the game story, the game also presented the same SQL
theory as the textbook.
Before the experiment, the participants were asked to answer the Felder-Silverman learning style questionnaire (Appendix 1) to identify
their individual learning style. During the game, the interaction between the participants and the game were recorded automatically to
identify the participant's learning style during the game. At the end of the game, the participants were asked to answer an SQL post-test
(Appendix 3). The Felder-Silverman questionnaire, SQL pre-test and post-test were all paper-based. The SQL pre-test used different case
study than the post-test and the case study in game is different than the textbook. The game recorded interactions between the participants
and the learning materials and the records are to be used to analyse in more details on the learning style in GBL.
There were three experimental groups that represented each mode of the game and one control group in which the students would learn
SQL by using a textbook. Each participant was allocated randomly to one of the four groups. In this section, the groups will be referred to as:
the paper-based group, the non-adaptive game group, the out-of-game learning style (LS) group and the in-game adaptive group. In the
paper-based group, the participants were given a textbook about SQL while participants on the other groups learnt SQL through a game with
a mode of adaptivity described previously.
The participants were required to answer the SQL tests and those aspects of the game relating to the questionnaire. The SQL tests were
conducted twice: in both the pre-test and post-test. In the pre-test, the SQL test was used to identify the participants' knowledge of SQL.
Those who had no knowledge of SQL were selected for the experiment to ensure the participants had the same initial knowledge of SQL. By
doing so, the authors tried to reduce the possibilities of recruiting participants who may have learnt SQL before as they may be easily recall
what they have learned before considering the forms of SQL are similar and the materials in this experiment are normally taught together.
The SQL post-test was used to test the students’ knowledge of SQL after learning SQL through the game or after studying the prescribed
textbook. The SQL pre-test had similar questions to the post-test with 11 questions in total. The maximum points achievable for the SQL pre-
test and post-test was 16. The analysis of the learning effectiveness to be conducted was a non-parametric test according to the Levene
statistic ¼ 8.280 (p < 0.001), while the analysis for the completion time was also a non-parametric test according to the Levene
statistic ¼ 3.879 (p < 0.024). A Levene statistic with p < 0.05 means the homogeneity cannot be assumed. In this paper, the confidence value
used was 95% (p < 0.05).
5. Results
This section describes the results of the experiment starting from differences in learning effectiveness followed by the completion time
analysis between the experimental groups. Before discussing the results, it is noted that the tests were marked by the researcher and by an
independent marker, an SQL expert. A reliability of the marking of the tests was conducted using the Cronbach Alpha test (Kinnear & Gray,
2008). The test showed that the Cronbach Alpha value of the reliability in marking was 0.990, which means the post-test scores marked by
the researcher were consistent with the scores marked by the independent rater.
5.1. Learning effectiveness
The learning effectiveness between the paper-based group and the games group is compared to test the following hypothesis:
H0A: There is no difference in learning effectiveness between the paper-based group and the game groups.
H1A: There is a difference in learning effectiveness between the paper-based group and the game groups.
The learning effectiveness of the paper-based SQL learning for 30 university students had a mean of 2.6 (SD ¼ 3.8065). The mean score of
the learning effectiveness from the non-adaptivity game group, which consisted of 30 students, was 12.517 (SD ¼ 4.690). The learning
effectiveness of the out-of-game LS group, which consisted of 30 students, had a mean score of 12.35 (SD ¼ 4.065). The learning effec-
tiveness of the in-game adaptive group, which also consisted of 30 students, had a mean score of 14.042 (SD ¼ 2.354).
When compared to any mode of the game, a significant difference in learning effectiveness between the groups was found. By using
ManneWhitney U tests, the learning effectiveness for the non-adaptivity game group was found to be significantly higher when compared
to the paper-based learning group (Z ¼ 5.903, p < 0.001). The learning effectiveness of the paper-based learning group was also found to be
significantly lower to the out-of-game LS game group (Z ¼ 5.995, p < 0.001) and the in-game adaptive group (Z ¼ 6.495, p < 0.001) and so
H0A is rejected.
The analysis of the differences in learning effectiveness between the game groups has three hypotheses to investigate how differences
between the non-adaptivity game group and the other game groups and between the out-of game learning LS group and the in-game
adaptive group. The first hypothesis is:
H0B: There is no difference in learning effectiveness between the non-adaptivity game group and the out-of-game LS group.
H1B: There is a difference in learning effectiveness between the non-adaptivity game group and the out-of-game LS group.
A ManneWhitney U test showed there was no significant difference between the non-adaptivity group and the out-of-game LS group
(Z ¼ 0.872, p < 0.389) and so H0B is accepted.
The second hypothesis is:
H0C: There is no difference in learning effectiveness between the non-adaptivity game group and the in-game adaptive group.
H1C: There is a difference in learning effectiveness between the non-adaptivity game group and the in-game adaptive group.
A similar result was found when comparing the learning effectiveness of the non-adaptivity group to the game with the in-game
adaptivity (Z ¼ 0.378, p < 0.711) and so H0C is accepted.
The third hypothesis is:
H0D: There is no difference in learning effectiveness between the out-of game learning LS group and the in-game adaptive group.
H1D: There is a difference in learning effectiveness between the out-of game learning LS group and the in-game adaptive group.
Similar to the result of the first and second hypotheses, a ManneWhitney U test showed there was no significant difference between the
out-of game learning LS group and the in-game adaptive group (Z ¼ 1.546, p < 0.124) and so H0D is accepted.
Table 3
Comparison of learning effectiveness.
Groups Learning effectiveness between groups

Paper-based Group and Non-adaptive Group Z ¼ 5.903, p < 0.001 (Non-adaptive Group > Paper-based Group)
Paper-based Group and Out-of-game LS Group Z ¼ 5.995, p < 0.001 (Out-of-game LS Group > Paper-based Group)
Paper-based Group and In-game Adaptive Group Z ¼ 6.495, p < 0.001 (In-game Adaptive Group > Paper-based Group)
Non-adaptive Group and Out-of-game LS Group Z ¼ 0.872, p < 0.389
Non-adaptive Group and In-game Adaptive Group Z ¼ 0.378, p < 0.711
Out-of-game LS Group and In-game Adaptive Group Z ¼ 1.546, p < 0.124
All Groups c2 ¼ 58.153, p < 0.01
These tests showed that the learning effectiveness for the modes of the game were significantly higher than those for paper-based
learning. A KruskaleWallis test was then performed using to compare all these groups (paper-based, game without adaptivity, game with
out-of-game LS and game with in-game adaptivity) and was found to be significant beyond the 0.01 level (c2 ¼ 58.153, p < 0.01) indicating
there is a significant difference between the means of learning effectiveness between groups. A summary of the results is presented in Table 3.
5.2. Differences in learning effectiveness between learning styles
This section investigates the differences in the learning effectiveness for those students with different learning styles (picture/text) as
reflected by the SQL post-test. In the paper-based group, there were 8 participants (27%) with a text-based learning style and the mean for
learning effectiveness was 1.5 (SD ¼ 2.712), while 22 participants (73%) had a picture-based learning style and the mean for their learning
effectiveness was 3.0 (SD ¼ 4.114). In the non-adaptivity group, there were 13 participants (43%) with a text-based learning style and the
mean of the learning effectiveness was 13.481 (SD ¼ 3.209), while 17 participants (57%) had a picture-based learning style and the mean of
their learning effectiveness was 11.779 (SD ¼ 5.551). In the out-of-game LS group, there were 13 participants (43%) with a text-based
learning style and the mean for learning effectiveness was 10.788 (SD ¼ 4.377), while 17 participants (57%) had a picture-based learning
style and the mean of their learning effectiveness was 13.544 (SD ¼ 3.477). In the in-game adaptive group, there were 14 participants (47%)
with a text-based learning style and the mean for the learning effectiveness was 13.411 (SD ¼ 3.001), while 16 participants (53%) had a
picture-based learning style and the mean of their learning effectiveness was 14.594 (SD ¼ 1.486).
When comparing the learning effectiveness of participants with different learning styles in all groups, a ManneWhitney U test indicated
no difference in learning effectiveness between those participants who had a text-based learning style and those who had a picture-based
learning style (Z ¼ 0.011, p < 0.993). The same result was also found when comparing the learning effectiveness of participants with
different learning styles across the game groups (Z ¼ 1.477, p < 0.142). The result suggests the participants with text-based learning style
did not have a significantly higher learning effectiveness than the participants with picture-based learning style and vice versa.
5.3. Differences in learning effectiveness between programmes
This section investigates the differences in the learning effectiveness reflected by the SQL post-test for those on different programmes
(computing/non-computing). In the paper-based group, there were 4 participants (13%) from non-computing programmes and the mean for
learning effectiveness was 4.875 (SD ¼ 1.702), while there were 26 participants (87%) from computing programmes and the mean for their
learning effectiveness was 2.250 (SD ¼ 3.938). In the non-adaptivity group, there were 6 participants (20%) from non-computing pro-
grammes and the mean for learning effectiveness was 11.583 (SD ¼ 4.104), while there were 24 participants (80%) from computing pro-
grammes and the mean for their learning effectiveness was 12.750 (SD ¼ 4.877). In the out-of-game LS group, there were 19 participants
(63%) from non-computing programmes and the mean for learning effectiveness was 11.382 (SD ¼ 4.61), while there were 11 participants
(37%) from computing programmes and the mean for their learning effectiveness was 14.203 (SD ¼ 2.192). In the in-game adaptivity group,
there were 13 participants (43%) from non-computing programmes and the mean for learning effectiveness was 13.019 (SD ¼ 2.803), while
there were 17 participants (53%) from computing programmes and the mean for their learning effectiveness was 14.824 (SD ¼ 1.627).
When comparing the learning effectiveness of participants from different programmes in all groups, a ManneWhitney U test indicated
no difference in learning effectiveness between participants from non-computing programmes and participants from computing pro-
grammes (Z ¼ 0.274, p < 0.787). However, a ManneWhitney U test indicated the learning effectiveness of participants from computing
programmes across the game groups was significantly higher than the learning effectiveness for those on non-computing programmes
(Z ¼ 2.797, p < 0.006) potentially because participants from computing programme are more familiar with game environment which
allows them to effectively adapt to the learning process throughout the game.
5.4. Differences in learning effectiveness between education levels
This section investigates the differences in learning effectiveness, as reflected by the SQL post-test, for the different education levels
(undergraduate/postgraduate). In the paper-based group, there were 26 undergraduate participants (87%) and the mean for learning
effectiveness was 1.942 (SD ¼ 3.5618), while there were 4 postgraduate participants (13%) and the mean for their learning effectiveness was
6.875 (SD ¼ 2.4958). In the non-adaptivity group, there were 19 undergraduate participants (63%) and the mean of learning effectiveness
was 10.934 (SD ¼ 5.281) while there were 11 postgraduate participants (37%) and the mean for their learning effectiveness was 15.250
(SD ¼ 0.783). In the out-of-game LS group, there were 18 undergraduate participants (60%) and the mean for the learning effectiveness was
13.222 (SD ¼ 3.659), while there were 12 postgraduate participants (40%) and the mean for their learning effectiveness was 11.042
(SD ¼ 4.445). In the in-game adaptivity group, there were 20 undergraduate participants (67%) and the mean of the learning effectiveness
was 13.925 (SD ¼ 2.755) while there were 10 postgraduate participants (33%) and the mean of their learning effectiveness was 14.275
(SD ¼ 1.315).
When comparing the learning effectiveness of participants with different education levels in all groups, a ManneWhitney U test
indicated that the learning effectiveness of postgraduate participants was significantly higher than undergraduate participants (Z ¼ 2.138,
p < 0.033). However, a ManneWhitney U test indicated there was no significant difference in learning effectiveness between undergraduate
and postgraduate participants across the game groups (Z ¼ 0.068, p < 0.949). This finding might suggest that the learning from a textbook
required a more experienced learner than is required by a game.
5.5. Differences in learning effectiveness between genders
This section investigates the differences in learning effectiveness reflected by the SQL post-test according to gender (male/female). In the
paper-based group, there were 5 female participants (17%) and the mean for learning effectiveness was 4.2 (SD ¼ 4.0094), while there were
25 male participants (83%) and the mean for their learning effectiveness was 2.280 (SD ¼ 3.7669). In the non-adaptivity group, there were 7
female participants (23%) and the mean for learning effectiveness was 12.857 (SD ¼ 3.660) while there were 23 male participants (77%) and
the mean for their learning effectiveness was 12.413 (SD ¼ 5.029). In the out-of-game LS group, there were 8 female participants (27%) and
the mean for learning effectiveness was 11.094 (SD ¼ 4.698), while there were 22 male participants (73%) and the mean for their learning
effectiveness was 12.807 (SD ¼ 3.827). In the in-game adaptive group, there were 8 females participants (27%) and the mean for learning
effectiveness was 12.875 (SD ¼ 1.512), while there were 22 male participants (73%) and the mean for their learning effectiveness was 14.466
(SD ¼ 2.486).
When comparing the learning effectiveness of participants based on gender, a ManneWhitney U test indicated no difference in learning
effectiveness between male and female participants (Z ¼ 0.528, p < 0.600). However, a ManneWhitney U test indicated the learning
effectiveness of male participants across the game groups was significantly higher than the learning effectiveness for female participants
(Z ¼ 2.115, p < 0.035), suggesting that for this type of game (RPG) and/or subject matter may not be the most appropriate for female
learners.
5.6. Completion times
In this section, the completion time of the game groups are analysed. The completion time of paper-based learning group was not
included because the completion time of the group was not directly comparable to completion time of the game groups. The completion
time is the time required for the player to complete the game. The completion times were recorded automatically during the gameplay. This
analysis includes a comparison with the learning style and other characteristics associated with the player (gender, education level, pro-
grammes of study), as well as its correlation with the learning effectiveness. The mean completion time when learning through the game
without adaptivity was 94.67 min (SD ¼ 12.893), while the mean completion time for the out-of-game LS group was 90.67 min
(SD ¼ 14.284) and the mean completion time for the in-game adaptive group was 84.91 min (SD ¼ 8.174).
The analysis of the differences in completion times between the game groups has three hypotheses to investigate how differences
between the non-adaptivity game group and the other game groups and between the out-of game learning LS group and the in-game
adaptive group. The first hypothesis is:
H0E: There is no difference in completion times between the non-adaptivity game group and the out-of-game LS group.
H1E: There is a difference in completion times between the non-adaptivity game group and the out-of-game LS group.
When comparing the completion times between the non-adaptive group and the out-of-game LS group, a ManneWhitney U test showed
no significant difference in completion times between these two variants (Z ¼ 1.339, p < 0.184) and so H0E is accepted.
The second hypothesis is:
H0F: There is no difference in completion times between the non-adaptivity game group and the in-game adaptive group.
H1F: There is a difference in completion times between the non-adaptivity game group and the in-game adaptive group.
A ManneWhitney U test showed the completion times for the in-game adaptive group was shorter than for the non-adaptive group
(Z ¼ 2.814, p < 0.005) and so H0F is rejected.
The third hypothesis is:
H0G: There is no difference in completion times between the out-of game learning LS group and the in-game adaptive group.
H1G: There is a difference in completion times between the out-of game learning LS group and the in-game adaptive group.
Table 4
Comparison of completion times.
Groups Completion time between groups

Non-adaptive Group and Out-of-game LS Group Z ¼ 1.339, p < 0.184
Non-adaptive Group and In-game Adaptive Group Z ¼ 2.814, p < 0.005 (In-game Adaptive Group < Non-adaptive Group)
Out-of-game LS Group and In-game Adaptive Group Z ¼ 0.955, p < 0.345
All Game Groups c2 ¼ 7.056, p < 0.03
A ManneWhitney U test showed no significant difference in completion times between the out-of-game LS group and the in-game
adaptive group (Z ¼ 0.955, p < 0.345) and so H0G is accepted.
To conduct an overall analysis between the groups, the completion times for all the groups were compared and analysed using a
KruskaleWallis test. The test also showed that there was a significant difference in completion times between groups with the in-game
adaptive game requiring the shortest amount of time to complete, followed by the out-of-game LS game, and the game without adap-
tivity (c2 ¼ 7.056, p < 0.03). This finding might suggest that dynamically adapting the content is beneficial in supporting the student's
learning, allowing the student to learn more quickly. A summary of the results is presented in Table 4.
5.7. Correlation between completion times and learning effectiveness
The bivariate analysis using the Pearson correlation test (Kinnear & Gray, 2008) shows that the correlation between the completion times
and learning effectiveness in the non-adaptive group is weak (r(30) ¼ 0.349, p < 0.059). The correlation is also weak in the out-of-game LS
group (r(30) ¼ 0.349, p < 0.059). Similar to that shown for the previous groups, the correlation between learning effectiveness and the
completion time in the in-game adaptive group is weak (r(30) ¼ 0.182, p < 0.336). Overall, the results show completion time does not have
any effect on learning effectiveness. However, it is also shown the In-game Adaptive Group has a higher correlation between completion
time and learning effectiveness compared to other group. Perhaps by using a larger sample size, the significance in the correlation may be
confirmed.
5.8. Differences in completion times between programmes
In the non-adaptivity group, the mean of the completion times for the participants from non-computing programmes was 98.67
(SD ¼ 15.883), while from computing programmes the mean was 93.67 (SD ¼ 12.229). In the out-of-game LS group, the mean of the
completion times for the participants from non-computing programmes was 93.11 (SD ¼ 12.875), while from computing programmes
the mean was 86.45 (SD ¼ 16.201). In the in-game adaptive group, the mean of the completion times for the participants from non-
computing programmes was 84.62 (SD ¼ 7.985), while from computing programmes the mean for the completion times was 87.59
(SD ¼ 7.977).
To investigate the differences in completion times between the different programmes for each group, a ManneWhitney U test was used.
This test showed that there was no significant difference in completion times for participants from the different programmes in the non-
adaptive group (Z ¼ 0.104, p < 0.929), or in the out-of-game LS group (Z ¼ 1.509, p < 0.136) and in the in-game adaptive group
(Z ¼ 0.154, p < 0.893). The result suggests the participants’ from computing did not have a significantly faster completion time than the
non-computing participants and vice versa.
5.9. Differences in completion times between education levels
In the non-adaptivity group, the mean of the completion times for the undergraduate participants was 91.95 (SD ¼ 11.345), while for the
postgraduate participants it was 99.36 (SD ¼ 14.562). In the out-of-game LS group, the mean of the completion times for the undergraduate
participants was 89.61 (SD ¼ 13.682), while for the postgraduate participants it was 92.25 (SD ¼ 15.621). In the in-game adaptive group, the
mean of the completion times for the undergraduate participants was 84.30 (SD ¼ 8.163), while for the postgraduate participants it was
90.30 (SD ¼ 6.165). To investigate the difference in completion times on the basis of level of education for each group, a ManneWhitney U
test was used. This test showed no significant difference in completion time for participants of different education levels in the non-adaptive
game group (Z ¼ 1.142, p < 0.263), in the out-of-game LS group level (Z ¼ 0.382, p < 0.715) and in the in-game adaptive group
(Z ¼ 1.052, p < 0.311). The result suggests the postgraduate participants did not have a significantly faster completion time than the
undergraduate participants and vice versa.
5.10. Differences in completion times between learning styles
In the non-adaptivity group, the mean of the completion times for participants with a picture-based learning style was 94.06
(SD ¼ 14.289), while for those with a text-based learning style it was 95.46 (SD ¼ 11.326). In the out-of-game LS group, the mean of
the completion times for participants with a picture-based learning style was 91.12 (SD ¼ 13.402), while for participants with a text-
based learning style it was 90.08 (SD ¼ 15.903). In the in-game adaptive group, the mean of the completion time for participants
with a picture-based learning style was 88.19 (SD ¼ 8.534), while for those with a text-based learning style it was 84.14
(SD ¼ 6.982).
To investigate the difference in completion times between learning styles for each group, a ManneWhitney U test was used. This test
showed no significant difference in completion time for those participants with different learning styles in the non-adaptive game group
(Z ¼ 0.042, p < 0.975), in the out-of-game LS group (Z ¼ 0.231, p < 0.829), or in the in-game adaptive group (Z ¼ 1.172, p < 0.254). The
result suggests the participants with text-based learning style did not have a significantly faster completion time than the participants with
picture-based learning style and vice versa.
5.11. Differences in completion times between genders
In the non-adaptivity group, the mean of the completion time for female participants was 95.00 (SD ¼ 10.599), while for male partic-
ipants the mean was 94.57 (SD ¼ 13.727). In the out-of-game LS group, the mean for the completion time of female participants was 93.12
(SD ¼ 15.887), while for male participants it was 89.77 (SD ¼ 13.945). In the in-game adaptive group, the mean of the completion time of
female participants was 84.12 (SD ¼ 7.039), while for male participants it was 87.09 (SD ¼ 8.309).
To investigate the difference in completion times for each group according to gender, a ManneWhitney U test was used. This test showed
no significant difference in completion times for participants of different genders in the non-adaptive game group (Z ¼ 0.025, p < 0.991), in
the out-of-game LS group (Z ¼ 0.470, p < 0.654) or in the in-game adaptive group (Z ¼ 0.485, p < 0.647). The result suggests the male
participants did not have a significantly faster completion time than the female participants and vice versa.
6. Conclusions and future directions
Adapting the learning content according to the student's learning style has been found to have a positive impact on learning. To
investigate whether the same benefits apply to GBL, an experiment was conducted with research question as follows: How do GBL and
adaptive GBL compare to traditional (paper-based) learning? The experiment investigates the differences in learning effectiveness between
different adaptive modes of a game and more traditional (paper-based) learning and the difference in the completion times of the game
groups.
The first conclusion is GBL and both types of adaptive GBL applications can be used to teach SQL regardless of the learner's education
level, gender, programme of study or learning style as identified by the learning style questionnaire. Although the participants had no prior
knowledge of SQL, they were motivated to try to learn SQL when playing the games as they observed SQL to be part of the game and
understood that SQL was required to complete the game.
Second, the research found that all forms of GBL (without adaptivity and both types of adaptive GBL applications) produced higher
learning outcomes compared to paper-based learning of SQL. In general, this finding is in line with the empirical evidence and the sys-
tematic literature review by Hwang et al. (2012), Conati and Zhao (2004), Felicia and Pitt (2009), Hainey (2010) and Connolly et al. (2012)
and the empirical evidence produced by adaptive GBL identified in Section 2.3. It demonstrates the potential of GBL for use as a platform to
support the teaching SQL.
Third, although, the in-game adaptive group has the highest means of learning effectiveness compared to other game groups as
described in Section 5.1, the difference were not significant. However, the completion time of the in-game adaptive group is found to be
significantly lower than the other game groups which indicate that adaptivity has the potential to significantly shortened completion time.
Felicia and Pitt (2009) also found a reduction in completion times between their experimental adaptive GBL group and the paper-based
control group.
The results show the out-of game LS group has shorter completion times than the non-adaptive game group which may be because
there was only one type of learning material presented. However, it has a lower learning effectiveness than the non-adaptive game group
which indicates the player may have different preferences during the gameplay and it may be beneficial to allow the player to change their
preferences during the gameplay. These findings confirm the importance of adapting the game elements during the gameplay, in this case
the learning materials, based on the characteristics of the player. The in-game adaptivity has the advantage that it monitors and adapts the
game elements accordingly, which resulted in only relevant learning materials at the time are presented. By filtering the learning ma-
terials, it helps to shorten the player's completion time and reduce the process and cognitive load of the player when learning through the
game.
Fourth, when comparing the learning effectiveness of each group, the results show: 1) in general, there is no difference in learning
effectiveness between learning styles; 2) participants from computing programmes have significantly higher learning effectiveness
compared to participants from non-computing programmes in all game groups; 3) postgraduate participants have significantly higher
learning effectiveness compared to undergraduate participants in all groups; and 4) male participants have a significantly higher learning
effectiveness compared to female participants in all game groups. In terms of completion time, the results show: 1) participants in the in-
game Adaptive group require shorter time to complete the game followed by the out-of-game group and non-adaptive group; 2) there is no
correlation between completion time and learning effectiveness; 3) there is no significant difference in completion time between genders,
learning styles, programmes of study and education levels.
The results from this research have suggested that GBL, particularly adaptive GBL, can be used to teach a subject such as SQL. However, as
suggested by Hays (2005) and Hainey (2010), it is not possible to generalise from the findings produced here relating to teaching SQL as
learning content via GBL to other contents and other methods of delivery. Another limitation is that the findings may be specific to the
integration of these particular learning materials and the mechanics of the game. For example, the game developed for the purposes of this
research has a conversation system customised to accommodate the teaching of SQL so if this same game were to be used to teach a different
subject, the conversation system would need to be adapted to meet the needs of the new subject. Such modifications in the game's
specification could give rise to different learning outcomes. The result of the analysis involving sub-groups such as female/male, under-
graduate/postgraduate and computing/non-computing may require further investigation due to the imbalance between the number of
participants on each sub-group.
For future research, the adaptivity may be improved in future research particularly to address multiple learning styles in a game and to
create more complex adaptivity in various elements of the game and the results are expected to contribute to empirical evidence of the
beneficial effects of GBL and adaptive GBL in particular.
Acknowledgements
This work has been co-funded by the EU Lifelong Learning Programme under contract 519057-LLP-1-2011-1-UK-KA3-KA3NW
(Ed2.0Work e European Network for the integration of Web2.0 in education and work) and as part of the Games and Learning Alliance
(GaLA) Network of Excellence on ‘serious games’ funded by the European Union in FP7 e IST ICT, Technology Enhanced Learning (see http://
www.galanoe.eu).
Appendix 1. Felder-Silverman Learning Style Questionnaire.

Appendix 2. Pre-test SQL questions.

Appendix 3. Post-test SQL questions.

References
Becker, K. (2007). Digital game-based learning once removed: teaching teachers. British Journal of Educational Technology, 38(3), 478e488.
Bernardini, S., Porayska-Pomsta, K., & Smith, T. J. (2013). ECHOES: an intelligent serious game for fostering social communication in children with autism. Journal of Infor-
mation Sciences, 264, 41e60.
Berns, A., Gonzalez-Pardo, A., & Camacho, D. (2013). Game-like language learning in 3D Virtual environments. Computers and Education, 60, 210e220.
Bontcheva, K. (2002). Adaptivity, adaptability, and Reading behaviour: some results from the evaluation of a Dynamic Hypertext System. AH'2002, 69e78.
Boyle, E. A., Connolly, T. M., & Hainey, T. (2011). The role of psychology in understanding the impact of computer games. Entertainment Computing, 2(2), 69e74.
Budiharto, W., Rachmawati, R. N., Ricky, M. Y., & Chyntia, P. B. R. (2013). The psychological aspects and implementation of adaptive games for Mobile application. In In-
ternational Joint Conference on Awareness Science and Technology and Ubi-Media Computing (iCAST-UMEDIA). IEEE.
Charles, D., McNeill, M., McAlister, M., Black, M., Moore, A., Stringer, K., et al. (2005). In Player-centred game design: Player modelling and adaptive digital games. Proceedings of
DiGRA 2005 Conference: Changing Views e Worlds in Play (pp. 285e298).
Conati, C., & Zhao, X. (2004). Building and evaluating an Intelligent Pedagogical Agent to improve the effectivenesss of an education game. In International Conference on
Intelligent User Interfaces, IUI'04, Funchal, Madeira, Portugal (pp. 6e13).
Conati, C., & Zhou, X. (2002). Modelling students' emotions from cognitive appraisal in educational Games. In Proceedings of the 6th International Conference on Intelligent
Tutoring Systems. London, UK: Springer-Verlag, ACM.
Connolly, T. M., & Begg, C. E. (2004). Database systems: A practical approach to design, implementation and management (4th ed.). Addison Wesley Longman.
Connolly, T. M., & Begg, C. E. (2014). Database systems: A practical approach to design, implementation and management (6th ed.). Addison Wesley Longman.
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games.
Computers & Education, 59, 661e686.
Connolly, T. M., & Stansfield, M. H. (2006). Enhancing eLearning: using computer games to teach requirements collection and analysis. In Second Symposium of the WG HCI &
UE of the Austrian Computer Society, 23 November 2006, Vienna, Austria.
Cook, D. A. (2005). Reliability and validity of scores from the index of learning styles. Journal of Academic Medicine, 80(10), 97e101.
Demmel, R. B., Kohler, B., Krusche, S., & Schubert, L. (2011). The serious game: wemakewords. In Proceedings of the 10th SIGPLAN symposium on New ideas, new paradigms, and
reflections on programming and software (pp. 109e110). New York, USA: ACM.
Doukianou, S., Petridis, P., & Dunwell, I. (2014). A usability evaluation of game-based approaches assessing risk and delayed gratification. In Proceedings of the 6th International
Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES) (pp. 1e8). IEEE.
Felder, R. M., & Brent, R. (2005). Understanding student differences. Journal of Engineering Education, 94(1), 57e72.
Felicia, P., & Pitt, I. J. (2007). Evaluating the effect of personalities on the design of educational games. In Proceedings of the ECGBL Conference, Paisley, Scotland.
Felicia, P., & Pitt, I. J. (2008). Personalising educational games to students' learning styles. In Proceedings of the International Technology in Education and Development Con-
ference, Valencia, Spain.
Felicia, P., & Pitt, I. J. (2009). Profiling users in educational games. In T. M. Connolly, M. H. Stansfield, & E. Boyle (Eds.), Games-based learning Advancement for Multisensory
Human Computer Interfaces: Techniques and Effective Practices. Hershey: Idea-Group.
Gasimov, A., Tan, C. H., Phang, C. W., & Sutanto, J. (2010). Visiting mobile application development: what, how and where. In Proceedings of the International Conference on
Mobile Business and Global Mobility Roundtable (ICMBGMR) (pp. 74e81).
Graf, S., Viola, S. R., Leo, T., & Kinshuk. (2007). In-depth analysis of the Felder-Silverman learning style dimensions. Journal of Research on Technology in Education, 40(1),
79e93.
Hainey, T. (2010). Using games-based learning to teach requirements collection and analysis at tertiary education level. Doctoral dissertation. University of the West of
Scotland.
Hays, R. T. (2005). The effectiveness of instructional games: A literature review and discussion. Technical Report for Naval Air Warfare Center Training Systems Division. Retrieved
August 1, 2012, from http://www.dtic.mil/cgi-bin/GetTRDoc?AD¼ADA441935%26Location.
Hertzog, T., Poussin, J. C., Tangara, B., Kouriba, I., & Jamin, J. Y. (2014). A role playing game to address future water management issues in a large irrigated system: experience
from Mali. Agricultural Water Management, 137, 1e14.
Hwang, G. J., Sung, H. Y., Hung, C. M., Huang, I., & Tsai, C. C. (2012). Development of a personalized educational computer game based on students' learning styles. Special Issue
on Personalized Learning, 60(4), 623e638. Springer.
Ibanez, J., & Delgado-Mata, C. (2011). Adaptive two players videogames. Expert Systems with Applications, 38, 9157e9163.
Ismailovic, D., Haladjian, J., Kohler, B., Pagano, D., & Brugge, B. (2012). Adaptive serious game development. In 2nd International Workshop on Games and software engineering
(GAS) (pp. 23e26). IEEE.
Jackson, S. L., Krajcik, J., & Soloway, E. (1998). The design of learner-adaptable scaffolding in Interactive learning environments. In Proceedings of ACM CHI 1998 Conference on
human Factors in computing systems (pp. 197e200).
Jameson, A. (2003). Adaptive interfaces and agents. In J. A. Jacko, & A. Sears (Eds.), HumaneComputer Interface Handbook (pp. 305e330). Mahwah: Lawrence Erlbaum.
Karpinskyj, S., Zambetta, F., & Cavedon, L. (2014). Video game personalisation techniques: a comprehensive survey. Entertainment Computing, 5, 211e218.
Kerfoot, B. P., & Baker, H. (2012). An online spaced-education game to teach and assess residents: a multi-institutional prospective trial. Journal of the American College of
Surgeons, 214(3), 367e373.
Kinnear, P. R., & Gray, C. D. (2008). SPSS 16 made simple. New York: Psychology Press.
Kirriemuir, J., & McFarlane, A. (2004). Literature review in games and learning. Report 8. Bristol: NESTA, Futurelab.
Kolb, D. (1984). Experiential learning. Englewood Cliffs, New Jersey: Prentice-Hall Inc.
Kujala, J. V., Richardson, U., & Lyytinen, H. (2010). Estimation and visualization of confusability matrices from adaptive measurement data. Mathematical Psychology, 54,
196e207.
Kuljis, J., & Liu, F. (2005). A comparison of learning style theories on the suitability for elearning. In Proceedings of the 2005 ASTED International Conference on web Technologies,
applications, and Services (pp. 191e197).
Lee, M. J., & Ko, A. J. (2011). Personifying programming tool feedback improves Novice Programmers' learning. In Conference on International Computing Education Research
(ICER) (pp. 109e116). Providence, Rhode Island, USA: ACM.
Lester, J., Lobene, E., Mott, B., & Rowe, J. (2014). Serious games with gift: instructional strategies, game design, and natural language in the generalized intelligent framework
for tutoring. In R. Sottilare, A. Graesser, X. Hu, & B. Goldberg (Eds.), Instructional management: Vol. 2. Design Recommendations for adaptive Intelligent Tutoring systems (pp.
205e215). Orlando, Florida: U.S. Army Research Laboratory.
Limongelli, C., Sciarrone, F., Temperini, M., & Vaste, G. (2009). Adaptive learning with the LS-Plan system: a field evaluation. IEEE Transactions on Learning Technologies, 2(3),
203e215.
Litzinger, T. A., Lee, S. H., Wise, J. C., & Felder, R. M. (2005). A study of the reliability and validity of the Felder-Soloman Index of learning styles. In Proceeding of the 2005
American Society for Engineering Education Annual Conference and Exposition. American Society for Engineering Education.
Livesay, G. A., Dee, K. C., Nauman, E. A., & Hites, L. S., Jr. (2002). Engineering student learning styles: a statistical analysis using Felder's Index of learning styles. In 2002 ASEE
Conference and Exposition, Montreal, Quebec, June 2002.
Li, Y. N., Yao, C., Li, D. J., & Zhang, K. (2014). Adaptive difficulty scales for parkour games. Visual Languages and Computing, 25, 868e878.
Lynch, R. A., Steen, M. D., Pritchard, T. J., Buzzell, P. R., & Pintauro, S. J. (2008). Delivering food safety education to middle school students using a web-based, interactive,
multimedia, computer program. Journal of Food Science Education, 7, 35e42. Wiley.
Marsh, T., Li, Z. N., Klopfer, E., Change, X., Osterweil, S., & Haas, J. (2011). Fun and learning: blending design and development dimensions in serious games through narrative
and characters. In M. Ma, A. Oikonomou, & L. C. Jain (Eds.), Serious Games and Edutainment applications (pp. 273e288).
Melis, E., & Monthienvichienchai, R. (2004). They call it learning style but it's so much more. In Proceedings of World Conference on e-learning in Corporate, Government,
Healthcare, and Higher Education 2004, Chesapeake, VA (pp. 1383e1390).
Miller, L. M. (2005). Using learning styles to evaluate computer-based instruction. Computers in Human Behavior, 21, 287e306.
Ming-Chun, C., & Shyan-Ming, Y. (2013). The psychological aspects and implementation of adaptive games for mobile application. In International Joint Conference on
Awareness Science and Technology and Ubi-Media Computing (iCAST-UMEDIA) (pp. 163e169).
Mulwa, C., Lawless, S., Sharp, M., Arnedillo-Sanchez, I., & Wade, V. (2010). Adaptive educational hypermedia systems in technology enhanced learning: a literature review. In
Proceedings of the 2010 ACM Conference on Information Technology Education. ACM.
Oppermann, R., Rashev, R., & Kinshuk. (1997). ). Adaptability and adaptivity in learning systems. In A. Behrooz (Ed.), Knowledge Transfer (2nd ed.). (pp. 173e179). London:
pAce.
Peirce, N., Conlan, O., & Wade, V. (2008). Adaptive educational games: providing non-invasive personalised learning experiences. In Second IEEE International Conference on
Digital Games and Intelligent Toys Based Education (pp. 28e35). IEEE.
Praet, M., & Desoete, A. (2014). Enhancing young children's arithmetic skills through non-intensive, computerised kindergarten interventions: a randomised controlled study.
Teaching and Teacher Education, 39, 56e65.
Prensky, M. (2006). Don't bother me, mom, I'm learning!: How computer and video games are preparing your kids for 21st Century success and how you can help. St Paul: Paragon
House.
Price, L. (2004). Individual differences in learning: cognitive control, cognitive style, and learning style. Educational Psychology, 24, 681e698.
Rodriguez-Cerezo, D., Sarasa-Cabezuelo, A., Gomez-Albarran, M., & Sierra, J. L. (2014). Serious games in tertiary education: a case study concerning the comprehension of basic
concepts on computer language implementation courses. Computers in Human Behavior, 31, 558e570.
Sampayo-Vargas, S., Cope, C. J., He, Z., & Byrne, G. J. (2013). The effectiveness of adaptive difficulty adjustments on student motivation and learning in an educational
computer game. Computers and Education, 69, 452e462.
Sharda, N. (2007). Authoring educational multimedia content using learning styles and Story telling principles (pp. 93e102). Augsburg, Bavaria, Germany: EMME’07.
Siegler, R. S., & Ramani, G. B. (2011). Improving low-income childrens' number sense. Space, time and number in the brain (pp. 343e354).
Smeddinck, J., Siegel, S., & Herrlich, M. (2013). Adaptive difficulty in exergames for Parkinson's disease patients. In Proceedings of Graphics Interface Conference (pp. 141e148).
Smith, L. H., & Renzulli, J. S. (1984). Learning style preferences: a practical approach for classroom teachers. Theory into Practice, 23, 44e50.
Sweetser, P., & Wyeth, P. (2005). Gameflow: a model for evaluating player enjoyment in games. Computers and Entertainment, 3(3), 3.
Sweller, J. (1994). .Cognitive load theory, learning Difficulty and instructional design. Learning and Instructions, 4(4), 295e312.
Tavassolian, A., Stanley, K. G., & Gutwin, C. (2014). The effect of temporal adaptation granularity and game genre on the time-balancing abilities of adaptive time-varying
minigames. Entertainment Computing, 5, 43e54.
Tong, T., & Chignell, M. (2014). Developing a serious game for cognitive assessment: choosing settings and measuring performance. In Proceedings of the Second International
Symposium of Chinese CHI (pp. 70e79). ACM.
Tsai, T. W., Lo, H. Y., & Chen, K. S. (2012). An affective computing approach to develop the game-based adaptive learning material for the elementary students. In Proceedings of
the Joint International Conference on Human-Centered Computer Environments (pp. 8e13). ACM.
Van Zwanenberg, N., Wilkinson, L. J., & Anderson, A. (2000). Felder and Silverman's Index of learning styles and Honey and Mumford's Learning Styles Questionnaire: how do
they compare and do they predict academic performance? Journal of Educational Psychology, 20(3), 365e380.
Wilson, A. J., Dehaene, S., Dubois, O., & Fayol, M. (2009). Effects of an adaptive game intervention on accessing number sense in low-socioeconomic-status kindergarten
children. Mind Brain Education, 3(4), 224e234.
Yongyuth, P., Prada, R., Nakasone, A., Kawtrakul, A., & Prendinger, H. (2010). AgriVillage: 3D Multi-language internet game for fostering agriculture environmental awareness.
In Proceeding of the International Conference on Management of Emergent Digital Ecosystems (pp. 145e152). NY: ACM.
Zhang, M., Xu, M., Han, L., Liu, Y., Lv, P., & He, G. (2012). Virtual network Marathon with immersion, scientificalness, competitiveness, adaptability and learning. Computers and
Graphics, 36, 185e192.
Zywno, M. S. (2003). A contribution of validation of score meaning for Felder-Soloman's Index of learning styles. In Proceedings of the 2003 Annual ASEE Conference.
Washington, DC: ASEE.

E Games 6

Uploaded by

Copyright:

Available Formats

You might also like

E Games 6

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

E Games 6

Uploaded by

Copyright:

Available Formats

Computers & Education 86 (2015) 192e211

Contents lists available at ScienceDirect

Computers & Education

An application of adaptive games-based learning based on learning

2.1. Deﬁnition of adaptivity and adaptability

2.2. Learning styles

2.3. Adaptive GBL applications e empirical evidence

Main Sub-elements A Sub-elements B

Authors Area Adaptivity

3. Learning-style-based adaptive GBL

How to write an SQL command.

SELECT * FROM user_tbl

3.3. Game story

Fig. 1. Conversation sequence e ﬁrst.

Fig. 2. Conversation sequence e second.

Fig. 3. Conversation sequence e third.

4. Research questions and methodology

4.1. Research questions

Fig. 4. Screenshot of an area e Central Plain.

Fig. 5. Screenshot of the map, inventory and biodata notebook.

5.1. Learning effectiveness

Groups Learning effectiveness between groups

5.2. Differences in learning effectiveness between learning styles

5.3. Differences in learning effectiveness between programmes

5.4. Differences in learning effectiveness between education levels

5.5. Differences in learning effectiveness between genders

5.6. Completion times

Groups Completion time between groups

5.7. Correlation between completion times and learning effectiveness

5.8. Differences in completion times between programmes

5.9. Differences in completion times between education levels

5.10. Differences in completion times between learning styles

5.11. Differences in completion times between genders

6. Conclusions and future directions

Appendix 1. Felder-Silverman Learning Style Questionnaire.

Appendix 2. Pre-test SQL questions.

Appendix 3. Post-test SQL questions.

You might also like