Original Article

E-Learning and Digital Media

1–19
Enhancing statistical literacy ! The Author(s) 2021
Article reuse guidelines:
skills through real life sagepub.com/journals-permissions
DOI: 10.1177/2042753020987016
activities enriched with journals.sagepub.com/home/ldm

gamification elements:
An experimental study

€
Unal Çakıro
glu
Department of Computer Education and Instructional Technology, Fatih
Faculty of Education, Trabzon University, Trabzon, Turkey

Mustafa Güler
Department of Mathematics Education, Fatih Faculty of Education,
Trabzon University, Trabzon, Turkey

Abstract
This study attempts to determine whether gamification can be used as a pedagogical technique to
overcome the challenges in teaching statistics. A post-test quasi-experimental design was carried
out in gamified and non-gamified groups in order to reveal the effect of gamification elements in
cultivating students’ statistical literacy skills. Students in gamified group were also interviewed to
understand the function of gamification process. The results suggest that; although gamifying the
instructional process had a positive impact on developing students’ statistical literacy in medium
and high score students; surprisingly the influence of the gamification to the low- achieved scores
were not positive. The positive impact was discussed in accordance with the gradual structure of
statistical literacy and suggestions for successful gamification applications due to the context were
included.

Keywords
Gamification, progressive instructional processes, statistical literacy, Watson’s model

Corresponding author:
Mustafa Güler, Department of Mathematics Education, Fatih Faculty of Education, Trabzon University, Trabzon, Turkey.
Email: mustafaguler@trabzon.edu.tr
2 E-Learning and Digital Media 0(0)

Introduction
In an increasingly information-driven society, beyond a mathematical knowledge; statistical
literacy is considered as a necessary skill for all students in their daily life (Franklin et al.,
2005). In this circumstance, Wallman (1993) pointed out that statistical literacy training
should be started in young ages in elementary and primary grade students. NCTM (2000)
also puts emphasis on the skills of statistical literacy and suggests enriching mathematics
curriculum with statistics learning area at different school levels. Similarly in Turkey statis-
tics topics have taken place in the newly revised middle schools mathematics education
curriculum (Ministry of National Education, 2013).
A large body of the statistics literacy generally examined the development of students’
statistical knowledge (Callingham and Watson, 2004), revealing statistical literacy level
of the adults (DelMas, 2002; Schield, 2006) or defining how to use the statistical knowledge
in real world (Ben-Zvi and Garfield, 2004; Martinez-Dowson, 2010; Rumsey, 2002;
Schield, 2006; Watson and Kelly, 2008). Some studies also suggested some new ways for
enhancing statistical knowledge in young ages. Leavy and Hourigan (2015) argued that
appropriate implementations supported the improvement of statistical literacy by
introducing authentic statistical investigations in the lesson. For college students, Clair
and Chihara (2012) implemented a pedagogical strategy, team-based learning, and revealed
the effectiveness of the approach compared to the individual participation. The results of
the previous studies provide hints that students’ statistical literacy skills may be
improved in around specific processes (e.g. Clair and Chihara, 2012; Krishnan, 2015).
In this sense, various real world problems are suggested to figure out the statistical value
of the data, so that such kind of data may support statistical interpretation skills
(Woltman, 2017).
Although there is a widespread recognition of the necessity concerning teaching and
learning statistics, it is still viewed as a new area compared to other study fields in math-
ematics (Guler et al., 2016). Many efforts in the classroom level have been provided in years;
however, students still have many difficulties in understanding theoretical aspects, and
practicing the knowledge in their daily life (Cerrito, 1999; Ozmen and Guven, 2019;
Watson and Moritz, 2000). One reason for this is the gradual structure of the statistical
knowledge. Students receiving statistics courses are generally unable to pass these gradual
stages of operations with data, understanding the meaning of data and using data in real
contexts or daily experiences (Verhoeven, 2006). Finally, in line with the motivation, low
engagement is seen one of the reason of low achievement. In a recent study; applying
Merrill’s First Principles, researchers found that students’ conceptual understanding in
terms of literacy, reasoning, and thinking statistically were promoted; however specifically,
the understanding of statistical terminology did not develop to a satisfactory level (Tu and
Snyder, 2017). Overall, previous research has demonstrated that teachers should deal with
these challenges during teaching statistics a multidimensional manner.
In order to eliminate the challenges, this paper reports cultivating statistical literacy
where the main stages in learning statistics were taken as a basis and focuses to provide a
meaningful link among these stages via gamification elements. This is because; gamification
is suggested to provide motivation, encouragement and may regulate students’ behaviors in
the instructional processes (Buckley and Doyle, 2016). Thus, the study deals with develop-
ment of statistical knowledge and the features of gamification elements.
Çakıroglu and Güler 3

Theoretical framework
Understanding and reacting statistical messages are sometimes more complex than they
seem. In this case, Watson (1997) introduced one of the hierarchical and most referred
models. This model presents an accurate understanding of the statistical knowledge with
the cognitive hierarchy of statistical knowledge, by referring indicators of each stage
(Watson and Callingham, 2003). The model is briefly outlined in Figure 1.
Some researchers used this framework as a way to measure and assess statistical literacy
skills development and also suggested implementations for statistics courses in line with this
€
framework (e.g. Ozmen, 2015; Watson and Callingham, 2003). The core objectives in the
stages are as follows.
Tier 1- Basic understanding of statistical terminology: According to Watson (1997), the
skills at this stage contains core concepts and are directly related to specific topics in the
curriculum and the topics in this tier is delivered in a traditional way with students creating
and analyzing their own data sets. Considering Turkish middle school mathematics curric-
ula, the topics in this tier include percentage, median, mean, graphs, measures of spread
concepts.
Tier 2 - Embedding of language and concepts in a wider context: This stage involves
understanding statistics terminology when embedded in a social context. For instance,
when those who already learnt statistical concepts see news on television or newspaper
report including statistics, make reasoning or interpretation instead of making calculations.
In other words, to read, understand and interpret written reports, rather than just perform-
ing computations are at the forefront. In this stage when statistics is embedded in a context,
a higher level of mathematical thinking is required than the first tier.
Tier 3 - Questioning of claims: In this stage, students are expected to criticize the data
given in a context. For instance, when those who know the background of statistical terms,

Figure 1. Statistical literacy framework (adapted in Watson, 1997).

4 E-Learning and Digital Media 0(0)

read statistical data in the media, they intend to show or proof misleads insufficient or
inappropriate information instead of making calculations. In this phase, students are
hoped to be aware of the missing parts, inquire incorrect interpretations and to question
the conclusions.
According to Watson and Callingham (2003), although traditional textbook questions
could fulfil the requirements of lower level of objectives such as Tier 1, these questions were
insufficient to meet the targeted high-level skills such as providing motivating contexts to
challenge students’ critical thinking. In line of this reasoning, they recommend teachers to
investigate various contexts such as media reports to provide student engagement and moti-
vation. Within the scope of the study, this proposal was taken into consideration and
students were exposed to various contexts in the designed learning environment. Thus,
given the potential of gamification to enhance engagement and motivation, we enriched
the learning process through gamification elements.

Gamification
Although the question of what the elements are that keep people in this context is not old,
the idea of how to transfer these elements to the various settings for a given learning
objective has come to the fore in the last decade. Considering the history of gamification,
although it was defined only in early 2010s, it became a trend in the following years and
studies were carried out on different fields from marketing to education. The outstanding
popularity of gamification is originating in the belief in its potential to foster motivation,
behavioral changes, friendly competition and collaboration in different contexts, such as
engagement (Dichev and Dicheva, 2017). Gamification is somehow different from playing
games in the classroom but it is using game elements in non-game contexts (Deterding et al.,
2011; Zimmerling et al., 2019). As a versatile and multidisciplinary concept, gamification
covers a range of theoretical and experimental knowledge as well as various technological
platforms and domains through motivational components (Seaborn and Fels, 2015).
According to Huotari and Hamari (2017), the function of gamification is to invoke the
same psychological states and emotions as games do. Numerous studies has been conducted
on the use of various types of game mechanics and dynamics such as badges (Ortega-Arranz
et al., 2019), leaderboards (Çakıroglu et al., 2017) and real gift (Yaman and Güven, 2014),
among others. An adaptation of Bunchball, Inc’s (2010) summary of gamification elements
is illustrated in Figure 2.
According to Ofosu-Ampong (2020), there is still a need to response some questions
whether game design elements are suitable to all spheres of human activities and whether
their use in education is always desirable. He furthermore recommends addressing more
research questions to human–computer interaction researchers in gamification. This study
focuses on the effect of gamification on the development of statistical literacy skills of
middle school students in terms of the educational dimension. In this context, prior research
in using gamification elements in mathematics classrooms provided hints for us to design the
learning environment enriched with gamification elements.

Gamification in mathematics classrooms

While teaching conceptual or procedural knowledge, engaging in the tasks is a crucial
problem in mathematics classrooms. In this sense, Christy and Fox (2014) pointed out
Çakıroglu and Güler 5

Figure 2. Game mechanics and game dynamics (adapted from Bunchball, Inc, 2010).

that some traditional activities in mathematics classrooms are inherently uninteresting and
teachers are failing to engage students in the activities. Considering the idea that todays’
students are growing up with the games, gamification attracted the attention of educators
and they tended to adapt the principles of gamification as a part of the classrooms (Buckley
and Doyle, 2016).
Gamification is considered as a useful way to increase student motivation (van Roy and
Zaman, 2018) and engagement by providing students with clear, achievable goals (Landers
and Callan, 2011), by organizing enjoyable learning environments (Cohen, 2011; Landers
and Callan, 2011), encouraging competition and giving rewards (Hamari, 2013). Gnauk
et al. (2012) addressed that it stimulated student self-efficacy via feedback. Furthermore,
gamification provides inspiration for weaker students and enhances their willingness in the
activities (Suls et al., 2002). In educational contexts, gamification have been most commonly
implemented through digital points, badges, virtual goods, level, reputation, extra points, or
leaderboards (Deterding et al., 2011; Hamari, 2013; Hew et al., 2016; Zichermann and
Cunningham, 2011). For instance, leaderboards allow students to perform relative to
others; and this competitive atmosphere increases students’ endeavoring in the tasks
(Charsky, 2010). Since gamification elements are mainly inspired from the behavioral pat-
terns of the game activities in the real world, educators suggest applying gamification within
real life problems or using the gamification elements in the activities, which are validated in
real scenarios (Sim~ oes et al., 2013).
Studies in mathematics generally used the potential contributions of gamification in
engagement and motivational aspects. Focusing on the scoring in a primary school,
Kickmeier-Rust et al. (2014) implemented a simple app in Austrian classrooms found pos-
itive effects of an individualized and meaningful feedback about errors. In another study,
Goehle (2013) integrated two common video game systems, levels and achievements in the
online homework program and he noticed greater than a half of the participants engaged
with the program. However, some studies revealed the negative results in mathematics (e.g.
Attali and Arieli-Attali, 2015). For instance, the study conducted by Attali and Arieli-Attali
(2015) found no effect of point as a basic element of gamification on 6-8 grade middle school
6 E-Learning and Digital Media 0(0)

students’ academic performance. Hence, the empirical evidence regarding the effects of
gamification on teaching mathematics is still unclear; and a need exists for more research
in order to understand its effects and providing ways of implementing gamification.

Aim of the study

Although the prescription of statistics instruction is well defined by researchers (Gal, 2002;
Pfannkuch and Ben-Zvi, 2011; Watson, 1997) and reports such as GAISE (Franklin et al.,
2005), the practical difficulties are still enormous. For instance, Martinez-Dowson (2010)
highlighted the lack of providing connection between statistical knowledge and real world
context as one of the difficulties in statistics instruction. In order to overcome the challenges
in statistics instruction, an emphasis is required to overcome the challenges in passing
among the tiers. Also the inconsistent results in the studies about statistical literacy provide
an incentive to researchers to focus on designing more authentic learning environments.
Hypothesizing that gamification elements facilitate to learn complex mathematics topics,
it may also positively act in developing statistical literacy skills involving a hierarchical
stages construct. Therefore, the experiences of students were taken qualitatively and quan-
titatively together in order explain how do the gamified activities work on their learning. In
line with the overall purpose of the study the research question, “What is the role of
gamified instructional process in cultivating students’ statistical knowledge?” was guided
to the study.

Method
A quasi-experimental research was designed with a focus on achievement in statistics learn-
ing area. Two groups of 7th grade students received same topics and asked for to complete
same activities in the similar classrooms. In the experimental group, the intervention includ-
ed the tasks enriched with gamification elements, however the tasks in control group class-
room was not gamified. Our interest in gamification process is primarily on the tasks related
to the stages of statistics instructional process. Since it is difficult to determine the influence
of the gamification elements (badges, leaderboard and real gift) solely, the effects in the
experimental group are provided through the combinational effect of the gamification
elements.

Participants and research design

Two groups of students (with the average age range¼13–14) enrolled in a Turkish middle
school participated to the study. One class (n¼ 20, 10 male, 10 female) was assigned as
control group (CG), and the other (n¼21, 11 male, 10 female) was the experimental group
(EG). These two groups were two separate classes in a same public school, which were
created with the similar level of achievement and educational background and are taught
by the same mathematics teacher. The pre-test results applied to decide whether the groups
were equal showed that more than half of the students in both the control and experimental
groups left the majority of the questions blank since they did not experience the focused
mathematics objectives before. Therefore, instead considering the pre-test results, students’
previous year’s mathematics course grade (ranges from 0 to 100) was adopted as the indi-
cator of their current academic achievement as recommended in the literature (Bacon and
Bean, 2006; Mladenovic et al., 2016). Due to the test variable is normally distributed within
Çakıroglu and Güler 7

Table 1. Mathematics course grades of participants.

Groups n M SD t df F p

EG 20 55.65 21.85 .23 39 1,086 .820

CG 21 53.95 25.40

Table 2. Objectives of statistics learning area in 7th grade mathematics curriculum.

The main objective:
Formulating a research question, collecting and organizing data, interpretation and evaluation
Concepts:
Line chart, pie chart, median, mode
Sub-objectives:
Pupils should be taught to
1. Create a pie chart according to the data set and interpret the model.
 While creating a pie chart, utilize appropriate interactive software if required.
2. Generate a line chart for given data and interpret it.
 In addition, activities contain generating line charts of two data group.
3. Find the average, median and mode of a data set and interpret their meaning.
 Activities include tasks for understanding which of those are more useful for a given data set. Accordingly, utilize ICT
if it is needed.
4. Choose an appropriate way to illustrate the gathered data for research questions among pie chart,
frequency table, bar or line chart, and transition among these representation.
 Activities include tasks for illustrating the data allowing determining strong and weak aspects of different
representations.

groups, and other assumptions are appropriate, t-test was conducted. The results revealed
no significant differences between CG and EG found (see Table 1).
Since, the study focuses on statistics learning area of 7th grade Turkish mathematics
curriculum, and the learning objectives were summarized in Table 2.
Course content, which in ways to conduct a total of 12 hours during 5 weeks considering
objectives was enriched as, summarized in Table 3.
For each in-class activity, teacher firstly underlined the concept related to the relevant
topic and explained procedural knowledge for some calculations, and then enriched the
lesson using worksheets. For each of the tiers, some tasks were included in the worksheets.
For instance, for the forecast activity, students were asked to visit http://www.havaduru
mu15gunluk.net/ web site, and to note the day and night temperatures of their province in
related places in worksheet. Students were requested to response some questions from Tier 1
to Tier 3 relatively such as what they understand the given concept, how they calculate the
described concept using data they gathered from the internet and compare and making
interpretation with the data. Afterwards, similar implementation was carried out using a
ready-data set. For CG, students were asked to answer the questions in pc lab using direc-
tions given in worksheet. Both worksheets in EG and CG were collected. Different from
CG, in order to gamify the teaching process in EG, students’ worksheets were scored by
teacher using an answer key on the scale of 100.
8 E-Learning and Digital Media 0(0)

Table 3. Activities conducted during implementation.

Activity Content Duration

Forecast activity In computer laboratory, investigation of the 15-day weather 2 hours

forecast of province. Activities associated with mode,
median and average concepts considering the day and night
temperatures.
Family budget activity Before the lesson, students collect data about the income and 2 hours
monthly expense of their families, and during the lesson,
work on creating tables and creating pie charts about the
data they gathered. Solving similar problems.
Data detective Formulating a research question, collection of data to find out a 4 hours
solution for related problem, making an investigation about
how to analyze the data, data collection and its
interpretation.
Consumer price index Investigation the change in inflation using the data released by 2 hours
Turkish Statistical Institute distribution by the years, choos-
ing specific goods and expressing the change different
representatives
Election survey Discussion on the surveys before elections on newspapers, 2 hours
results of the last election, comparing different regions and
counties, choosing appropriate graphs to represent the data,
discussing data bias with reference to the differences
between regions in addition to surveys and real result

Gamification elements have been included in the process through two different channels.
The first was the digital online platform, where the students awarded by leaderboard and
budgets. The students were asked to visit the web address shared with them on the evening
of the activities implemented. Here, top five students in the previous week’s activities in the
EG group were announced on the leaderboard as a result of their performance in the pre-
vious week. Badges, which was used as another virtual gamification element, was given to
students who responded the questions of tier1 and tier2. Similar to leaderboard, the badges
were announced on the website and represented by silver and golden star so that the student
who collected three silver star was replaced with golden. Second channel was the real gift. At
the end of each activity multiple-steps awarded question, tier3, those who responded cor-
rectly each steps were awarded via real gift.
The elements of gamification integrated gamified instructional process on statistics topic
is illustrated in Table 4.

Instruments
In order to determine the influence of the gamification elements, students’ achievements
were revealed and their perspectives were gathered to explain the influences.
Student Achievement Scores: Final exam scores were used in order to define student
academic achievements. Since, Watson (1997) proposed a three-tiered hierarchy to explain
statistical literacy proficiency; the final examination questions (For sample questions, see
Appendix) covered this structure of statistics. In this study Watson’s model including three
Çakıroglu and Güler 9

Table 4. Gamification elements used in mathematics course.

Gamification dynamics Use of the dynamics

Leader board • For each activity, student performances were ranked. The top five
were placed on the leader board, and the leader board was then
printed A3 size and hung in the classroom.
Badges • For some questions on worksheet, students were given “good”
badges. For each three “good” one, the badges were exchanged via
“the best”.
Real gifts • Those who were fully correct responded award-winning question of
the week were awarded with small real gifts.

Table 5. Question dimensions of final exam items in terms of statistical literacy.

Item No Max points

Basic terminology (Tier 1) 1, 2, 10 30

Context (Tier 2) 3, 4, 7, 8 40
Questioning (Tier 3) 5, 6, 9 30
The reliability of the test is calculated with the Cronbach’s alpha coefficient as .81.

dimensions as understanding basic statistical terminology, understanding it in context, ques-

tioning claims made without proper statistical justification was considered to determine the
statistical literacy. The questions in the final exam are briefly described in Table 5.
Interviews: Interviews are used to explain students’ behaviors and perspectives by taking
the relationships searched as a result of their experiences (Ebenezer and Erickson, 1996).
Similarly, in this study, after final examination, interviews were administered to nine select-
ed students in line with their responses in order to explain how gamification elements
worked in their experiences of the activities. Interviewees were selected purposefully accord-
ing to their final exam scores (high, medium and low achiever students). The students were
asked questions including how their cognitive process took place during the intervention
with gamified activities.

Data analysis
In order to put forth the difference between two groups, t test was used for overall final
exam scores regarding the assumption of homogeneity of variance. On the other hand,
considering for each tier in the framework, we conducted Mann–Whitney U tests to com-
pare the means of two groups to reveal the differences between CG and EG in the context of
statistical literacy levels, as the normal assumption of t-test was not satisfied. In particular,
for the analysis comparing of each dimensions of statistical literacy (basic terminology,
context and questioning) between two groups, the .05 of significance level was used. The
p value represented a 2-tailed significance if the result was not in the expected direction.
Students’ performances regarding the overall scores of test were classified considering three
intervals: 0-33 for low, 34-66 for medium and 67-100 for high performances. Finally, inter-
views were transcribed and analysed descriptively. The responses of the students were coded
10 E-Learning and Digital Media 0(0)

according to three themes which were pre-identified in the context of interview questions: (1)
the contribution of gamification to facilitating learning, b) the affective contribution of
gamification c) the general evaluation of gamifying process. Some direct quotations of
students were provided to explore the interrelation between student achievements and gami-
fied learning setting.

Results
Academic achievements in gamified and non-gamified groups
The descriptive data show that the performance of students in (EG) (n¼21 and mean ¼ 66.5)
was higher when compared to (CG) (n ¼ 20 and mean ¼45). In other words, the students in
statistics classroom with gamification elements outperformed than non-gamified group. The
box-plot obtained from these data is shown in Figure 3.
Figure 3 presents a box-plot presentation of the distribution of statistical scores of EG
and CG. The test variable is normally distributed within both groups, therefore t-test was
conducted and the results showed significant difference between the EG and the CG in favor
of EG (see Table 6).

Performances in different stages of statistical knowledge

Considering students’ scores in the test, students in two groups were classified regarding to
their scores as low, medium and high. The distribution of the sub groups is shown in
Table 7.
The distributions of final scores suggest that there is a considerable difference between
two groups in favour of EG, which was arisen with the use of gamification elements.
Students’ performances regarding the stages of statistical literacy knowledge, both CG
and EG groups in terms of the tiers of statistical literacy and their test scores are outlined
in Figure 4. The data is used to interpret how the intervention affected to develop knowledge
in the tiers of statistical literacy.
Out of 30 points, although the tier 1 performance (mean scores in this stage) of both
groups were close to each other, EG showed higher performance. The mean scores differ-
ence between two groups was expanded in Tier 2, and statistically significant difference was
confronted for Tier 3 in favour of EG. The same gamification elements were used in three
stages by organizing them related to the tasks in the stages.

Understanding the effects of gamification elements in developing statistical knowledge

Interviews with 3 students from 3 sub-groups of students in EG provided some clues to
understand how the gamifying process affected to their academic achievements.
Accordingly, the qualitative data also guided researchers to make some inferences and
spanning between gamification elements and academic achievement. The questions in the
interviews were generally directed due to their work in the tasks. In this context, students
were asked whether the gamification elements have contributed their efforts in the tasks
both mentally or emotionally. Thus, their motivation and engagement were also asked.
Finally, the implementation is asked to them whether they find the process attractive.
The answers of the interviewees were coded in three main themes presented in Figure 5:
(1) contribution of gamification elements in facilitating learning (constructing new
Çakıroglu and Güler 11

Figure 3. Distribution of statistical literacy scores of EG and CG.

Table 6. Distribution of final scores among groups (N).

Low Medium High

CG 6 10 5
EG 3 8 9

Table 7. Results of independent t-test regarding student achievement scores (N).

Groups n M SD t df F p

EG 20 66.50 24.93 2.56 39 .382 .015

CG 21 45.00 28.68

knowledge), (2) contribution in affective domain (motivation, engagement, interest) and (3)
evaluation of overall the gamifying process.
For instance, all of the high and medium achievers stated that the gamifying learning
setting has contributed constructing new knowledge. They think that the process has
increased their motivation as well as engagement. For instance, H1 addressed that “. . .
When I saw my friends getting badges, something triggered me to get one of the badges.
However, the examples were different in the textbook; I sometimes tried to pose problems. I
made a list of nut crops regarding to the cities. I calculated their median, average etc.” M2
expressed that “. . . At first implementation, I asked myself why not I took place on
12 E-Learning and Digital Media 0(0)

26
24
22
20
18
16
Scores

14 EG
12
CG
10
8
6
4
2
0
Tier 1 Tier 2 Tier 3*
Statistical literacy skills

Figure 4. Final test scores in terms of statistical literacy of EG and CG (Mann–Whitney U-test results,
*p ¼ .005 < .05).

Figure 5. Summary of interviewees’ performance and the interview protocol.

leaderboard. Then I decided to study harder for the previous activities. I had definitely peak
motivation, I actually still have.” On the other side, majority of students in low group
expressed that the implementation did not affect their motivation nor contributed their
studies. As L1 identified, “The process had no contribution. I mean I was not whipped.
Çakıroglu and Güler 13

It was interesting, but I was not motivated. It was difficult to take place in the leaderboard
or to get a gift.”
All of the respondents at high and medium level were positive about the contribution of
implementation to their engagement. For example, M3 expressed that “I tried to engage
more in the lesson, because I had opportunity to get more badges. At the beginning, it was
so easy but then it became so hard to get one in the study period. I studied for a lot of
questions at home to be more active in the class.” Likewise, H2 asserted that “When you feel
you compete to collect more badges, take in to the leaderboard or most importantly for me
to take a gift; you necessarily should engage all the activities. In addition, it is not enough to
participate to activities during the lesson. Once we have collected the data (means data
detective activity), so I asked questions to the people on the street.” However, for 2 of 3
low achievers expressed that the implementation had no influence on their engagement.
While L3 specified that “I tried to win the teacher’s favour, I often ask to talk in the
class.”, In contrast, L2 stated, “I’m not sure it helped me to participate to the activities
more.” Similarly, the students addressed that the implementations attracted their interest.”
Interestingly, while medium and high level students prefer real gift or leaderboard as a
gamification element, badges was the main choice of the low achievers. In one noteworthy
statement from the interviews, L1 pointed out that “. . . It was difficult to win a gift or take
place top three on the leaderboard. However, it was easier to get a badge. I have found the
average of both and get a badge, I was so happy.”
To sum up, qualitative results indicate that the gamification elements used in the instruc-
tional process have contributed students’ efforts in the tasks as well as increasing their
motivation and engagement. Majority of the interviewees found the implementation inter-
esting and found the approach beneficial. On the other hand, while high and medium level
of the students had similar experiences, low ones gave vague answers for some questions.
Consequently, the overall perspective of the EG groups can be considered as satisfied.

Conclusions and discussion

In this study, middle school students received an instructional package about statistics in a
mathematics classroom enriched with some gamification elements. The results of the inter-
vention showed that students in gamified classroom performed better than the students in
non-gamified group. In this sense, previous studies reported some various results about the
influence of gamification on student achievement. For example, Tomaso (2014) conducted a
large-scale study with 20.000 participants from primary and secondary schools and noticed
a weak positive effect of student achievement and gamification implementation. Some other
studies concluded that the gamified activities positively and strongly affected o students’
academic performances (e.g. Çakıroglu et al., 2017; Domınguez et al., 2013).
In this study, gamification played a positive role on motivations of students. Similar
effects were also seen in a number of studies. For instance, Çakıroglu et al (2017) used a
combination of gamification elements in an ICT course and noticed that the gamified pro-
cess effected to students’ motivation and engagement that they indirectly had influences on
student achievement. In another study, Domınguez et al. (2013) found that gamifying an
instructional process positively affected student motivation and achievement, however par-
ticipation of the students did not change. In this study, the students in experimental group
showed slightly higher performances. In the intervention, it is thought that gamifying the
instructional process supported students to keep them in the direction of the learning
14 E-Learning and Digital Media 0(0)

objectives of the stages. In the context of teaching statistics, Tishkovskaya and Lancaster
(2010) suggested to follow a way from easy to difficult in order to build statistical literacy
skills. In this study, three tiers of Watson (1997) in developing statistical knowledge have
guided us in order to apply the instructional process. In other words, the structure of the
statistical knowledge provided us a well-organized way of gamifying. Hence, we gamified
the tasks in order to direct students to pass the stages. Thus, as a result, gamifying the
process supported students transiting among the stages of statistical literacy. In addition, the
students in EG expressed that the gamification elements used in classroom has developed
their engagement not only in the school, but also outside. This is to say some of them stated
that they also studied at home to get badges, take a place in leaderboard or win a gift.
The situation is considered as a dilemma in gamification whether it is a target itself or a
vehicle for achieving the tasks (Lee and Hammer, 2011). The results of this study
supported the idea that aiming to gain a gamification element in the study was a vehicle
for achievement as well.
On the other hand, NCTM (2000) suggests connections between mathematics subjects
and real-words situations in teaching statistical knowledge. In terms of passing the tiers, the
students’ perspectives reflected that passing from one to another was facilitated in experi-
mental group students. Thus, gamification elements in the experimental group is supported
students to proceed in the tiers of the statistical knowledge with keeping students motivated
and making a sense of game like environment.
In the study, the nature of the course allowed to integrate gamification elements easily in
to the instructional process. The experimental group students were asked to follow the
directions in the activities supported with worksheets. Especially badges were given in tier
1 and tier 2 in which knowing statistical concepts and terminology, and adapting and
implementing them in various contexts. Since Thompson and Iwata (2005) addressed to
take care of a large number of students getting badges, and the counter-effect of over
reinforce; we used a leaderboard in order to present the top performers of the previous
week selected in accordance with their total scores. Thus in the study, students’ perspectives
reflect that leaderboard allowed students to keep the external motivation fresh. Although
most of the low achieved students expressed that the gamification elements did not affect to
their motivation or engagement, they only found badges valuable because of getting it easy.
On the other hand, real gift was decided to be given for high-level cognitive questions,
requiring critical thinking, true interpretation or justification. During the activities related
to Tier1, students were asked to make basic descriptions and calculations such as median of
a given data set. The next tier they were requested making investigations such as collecting
data on the web and constituting a data set, then for instance, choosing appropriate way of
presenting data using the knowledge they already learned the previous tier. In the final tier,
different cases such as graphs including errors in or newspaper articles with data bias used to
understand how students provide justification the persuasiveness of the case. While badges
were frequently used for the first two tiers, real gift was given for tier3 due to its crucial role
of high-level cognitive questions, requiring critical thinking, true interpretation or justifica-
tion. Although, Authors in prior study found that using real gifts may not work for uni-
versity students; in this study for middle school students found them valuable for achieving
the tasks in the instructional process.
According to the GAISE report, the ultimate target of statistics instruction is developing
statistical literacy (Franklin et al., 2005). It is clearly addressed that for all stages of building
statistical literacy, gamified class descriptively performed higher. Despite there was no
Çakıroglu and Güler 15

statistically difference for the first two stages, in the third tier a statistical difference was
noticed between the final test results in favour of gamified group students. In conjunction
with the first two stages, the last stage defined as questioning of claims and connected to
critical thinking (Guler et al., 2016). In this regard, one of the contributions of the imple-
mentation is pushing students to make investigation, inquiry and furthermore, to use their
understanding in different contexts. Since the scores of the groups were both above the
average in two groups, one conclusion may be derived that gamification implementation
works more in higher cognitive level. In addition, one conclusion of this study is that while
the implementation accurately increased the motivation and engagement of medium and
high-level achievers, low performers remained distant. The results contribute to the instruc-
tional design processes in two folds. First, the study results support the idea that an appro-
priate gamifying process may provide positive learning performances during the learning
process (Bai et al., 2020; Buckley and Doyle, 2016; Hakulinen et al., 2015). On the other
hand, although a limited number of studies focus on the effect of gamification on academic
performance of low-achievers, a study conducted by Ding and Orey (2018) revealed that
students with low achievers tend not to engage a learning setting even enriched via gami-
fication elements. Similarly, the results obtained from this study showed that even if the
learning environment is gamified, the targeted success could not be achieved for the low. In
fact, the interview data show that students become insensitive to the gamification elements
after a while and joined the despair. At this point, perceptions to be unsuccessful prohibited
taking the advantage of the positive motivation provided by the gamification elements for
the low-achieved student.

Limitations and future implications

The study also has some certain limitations. First, this study was a small-scale study; a
further study is recommended with a larger group to provide different analysis techniques
for additional evidence. Second; since the study was purposefully limited to statistics learn-
ing area; and focused on the potential of gamification elements on the examples related with
real life implementations having progressive structure.
Consequently, a conclusion may be drawn that, since the target outcome is difficult to
achieve, or somehow boring for students, it may be a useful way inserting some gamification
elements in to the learning setting. In this sense, gamification process may be embedded
regarding the nature of the knowledge requiring for the objective. This may require some
experience for teachers or instructional designers about being aware of where students need
to be motivated or how they act in the tasks in the learning process. We believe our findings
provide insights on, and new possibilities for instructional design for statistics instruction.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/
or publication of this article.

Funding
The author(s) received no financial support for the research, authorship, and/or publication of this
article.
16 E-Learning and Digital Media 0(0)

ORCID iD
Mustafa Güler https://orcid.org/0000-0002-4082-7585

References
Attali Y and Arieli-Attali M (2015) Gamification in assessment: Do points affect test performance?
Computers & Education 83: 57–63
Bacon DR and Bean B (2006) GPA in research studies: An invaluable but neglected opportunity.
Journal of Marketing Education 28(1): 35–42.
Bai S, Hew KF and Huang B (2020) Is gamification “bullshit”? Evidence from a meta-analysis and
synthesis of qualitative data in educational contexts. Educational Research Review 30: 100322.
Ben-Zvi D and Garfield JB (2004) The Challenge of Developing Statistical Literacy, Reasoning and
Thinking. Dordrecht: Kluwer Academic Publishers.
Buckley P and Doyle E (2016) Gamification and student motivation. Interactive Learning
Environments 24(6): 1162–1175.
Bunchball, Inc. (2010) Gamification 101: An Introduction to the Use of Game Dynamics to Influence
Behavior [White paper]. Available at: www.bunchball.com/sites/default/files/downloads/gamifica
tion101.pdf (accessed 26 December 2020).
Callingham R and Watson JM (2004) Measuring statistical literacy. Journal of Applied Measurement
6(1): 19–47.
€ Başıbüyük B, Güler M, Atabay M and Memiş BY (2017) Gamifying an ICT course:
Çakıroglu U,
Influences on engagement and academic performance. Computers in human behavior 69: 98–107.
Cerrito PB (1999) Teaching statistical literacy. College Teaching 47: 9–13.
Charsky D (2010) From edutainment to serious games: A change in the use of game characteristics.
Games & Culture 5: 177–198.
Christy KR and Fox J (2014) Leaderboards in academic contexts: A test of stereotype threat and social
comparison explanations for women’s math performance. Computers & Education 78: 66–77.
Clair KS and Chihara L (2012) Team-based learning in a statistical literacy class. Journal of Statistics
Education 20(1): 1–20.
Cohen AM (2011) The gamification of education. Futurist 45: 16–17.
DelMas RC (2002) Statistical literacy, reasoning, and learning: A commentary. Journal of Statistics
Education 10(3): 1–14.
Deterding S, Dixon D, Khaled R, et al. (2011, September). From game design elements
to gamefulness: Defining gamification. In: Proceedings of the 15th International Academic
MindTrek Conference: Envisioning Future Media Environments (eds. A Lugmayr, H Franssila, C
Safran, et al.). New York: ACM Press, pp. 9–15.
Dichev C and Dicheva D (2017) Gamifying education: What is known, what is believed and what
remains uncertain: A critical review. International Journal of Educational Technology in Higher
Education 14(1): 9.
Ding L, Er E and Orey M (2018). An exploratory study of student engagement in gamified online
discussions. Computers & Education 120: 213–226.
Domınguez A, Saenz-de-Navarrete J, De-Marcos L, et al. (2013) Gamifying learning experiences:
Practical implications and outcomes. Computers & Education 63: 380–392.
Ebenezer JV and Erickson GL (1996) Chemistry students’ conceptions of solubility: A phenomenog-
raphy. Science Education 80(2): 181–201.
Franklin C, Kader G, Mewborn D, et al. (2005) Guidelines for Assessment and Instruction in Statistics
Education (GAISE) Report: A Pre-K-12 Curriculum Framework. Alexandria: American Statistical
Association.
Gal I (2002) Adults’ statistical literacy: Meanings, components, responsibilities. International
Statistical Review 70(1): 1–25.
Gnauk B, Dannecker L and Hahmann M (2012) Leveraging gamification in demand dispatch systems.
In: Proceedings of the 2012 Joint EDBT/ICDT Workshops, pp. 103–110.
Çakıroglu and Güler 17

Goehle G (2013) Gamification and web-based homework. Primus 23(3): 234–246.

Guler M, Gursoy K and Guven B (2016) Critical views of 8th grade students toward statistical data in
newspaper articles: Analysis in light of statistical literacy. Cogent Education 3(1): 1268773.
Hakulinen L, Auvinen T and Korhonen A (2015) The effect of achievement badges on students’
behavior: an empirical study in a university-level computer science course. International Journal
of Emerging Technologies in Learning 10(1): 18–29.
Hamari J (2013) Transforming homo economicus into homo ludens: A field experiment on gamifica-
tion in a utilitarian peer-to-peer trading service. Electronic Commerce Research and Applications 12:
236–245.
Hew KF, Huang B, Chu KWS, et al. (2016) Engaging Asian students through game mechanics:
Findings from two experiment studies. Computers & Education 93: 221–236.
Huotari K and Hamari J (2017) A definition for gamification: Anchoring gamification in the service
marketing literature. Electronic Markets 27(1): 21–31.
Kickmeier-Rust MD, Hillemann EC and Albert D (2014) Gamification and smart feedback:
Experiences with a primary school level math app. International Journal of Game-Based Learning
(IJGBL) 4(3): 35–46.
Krishnan S (2015) Fostering students’ statistical literacy through significant learning experience.
Journal of Research in Mathematics Education 4(3): 259–270.
Landers RN and Callan RC (2011) Casual social games as serious games: The psychology of gami-
fication in undergraduate education and employee training. In: Ma M, Oikonomou A and Jain LC
(eds) Serious Games and Edutainment Applications. London: Springer, pp. 399–423.
Leavy A and Hourigan M (2015) Motivating inquiry in statistics and probability in the primary
classroom. Teaching Statistics 37(2): 41–47.
Lee JJ and Hammer J (2011) Gamification in education: What, how, why bother? Academic Exchange
Quarterly 15(2): 146–151. Available at: www.uwstout.edu/soe/profdev/resources/upload/Lee-
Hammer-AEQ-2011.pdf
Martinez-Dowson R (2010) The effects of a course on statistical literacy upon students’ challenges to
statistical claims made in the media. Unpublished Doctoral Dissertation, Clemson University,
Clemson, USA.
Ministry of National Education (2013) Ortaokul Matematik Dersi 5, 6, 7 ve 8. Sınıflar O €gretim
Programı [Middle school mathematics curriculum for 5, 6, 7 and 8th grades]. Ankara: MoNE
Publishing.
Mladenovic M, Rosic M and Mladenovic S (2016) Comparing elementary students’ programming
success based on programming environment. International Journal of Modern Education and
Computer Science 8(8): 1–10.
National Council of Teachers of Mathematics (2000) Principles and Standards for School Mathematics.
Reston: National Council of Teachers of Mathematics.
Ofosu-Ampong, K. (2020). The Shift to Gamification in Education: A Review on Dominant Issues.
Journal of Educational Technology Systems 49(1) 113–137.
Ortega-Arranz A, Er E, Martınez-Mones A, et al. (2019) Understanding student behavior and per-
ceptions toward earning badges in a gamified MOOC. Universal Access in the Information Society
18(3): 533–549.
€
Ozmen ZM (2015) Examination of the statistical literacy levels of students from different undergrad-
uate programs. Unpublished Doctoral Dissertation, Karadeniz Technical University, Turkey.
Ozmen ZM and Guven B (2019) Evaluating students’ conceptual and procedural understanding of
sampling distributions. International Journal of Mathematical Education in Science and Technology
50(1): 25–45.
Pfannkuch M and Ben-Zvi D (2011) Developing teachers’ statistical thinking. In Carmen B, Gail B
and Chris R (eds) Teaching Statistics in School Mathematics-Challenges for Teaching and Teacher
Education. Dordrecht: Springer, pp. 323–333.
18 E-Learning and Digital Media 0(0)

Rumsey DJ (2002) Statistical literacy as a goal for introductory statistics courses. Journal of Statistics
Education 10(3): 6–13.
Schield M (2006) Statistical literacy survey analysis: Reading graphs and tables of rates andpercen-
tages. In A. Rossman & B. Chance (Eds.), Proceedings of the Seventh InternationalConference on
Teaching Statistics (ICOTS-7) [CD-ROM], Salvador, Brazil. Voorburg, The Netherlands:
International Statistical Institute.
Seaborn K and Fels DI (2015) Gamification in theory and action: A survey. International Journal of
Human Computer Studies 74: 14–31.
Sim~oes J, Redondo RD and Vilas AF (2013) A social gamification framework for a K-6 learning
platform. Computers in Human Behavior 29(2): 345–353.
Suls J, Martin R and Wheeler L (2002) Social comparison: Why, with whom, and with what effect?
Current Directions in Psychological Science 11(5): 159–163.
Thompson RH and Iwata BA (2005) A review of reinforcement control procedures. Journal of Applied
Behavior Analysis 38(2): 257–278.
Tishkovskaya S and Lancaster GA (2010, July). Teaching strategies to promote statistical literacy:
Review and implementation. In: Data and Context in Statistics Education: Towards an Evidence-
Based Society. Proceedings of the Eighth International Conference on Teaching Statistics. Voorburg:
International Statistical Institute. Available at: http://icots.info/8/cd/pdfs/contributed/ ICOTS8_
C193_TISHKOVSKAY.pdf (accessed 12 December 2020).
Tomaso P (2014) A quantitative assessment of the effect of games on learning. Doctoral Dissertation,
Colorado Technical University, USA.
Tu W and Snyder MM (2017) Developing conceptual understanding in a statistics course: Merrill’s
First Principles and real data at work. Educational Technology Research and Development 65(3):
579–595.
van Roy R and Zaman B (2018) Unravelling the ambivalent motivational power of gamification: A
basic psychological needs perspective. International Journal of Human-Computer Studies. Epub
ahead of print 26 April 2018. DOI: 10.1016/j.ijhcs.2018.04.009
Verhoeven P (2006) Statistics education in the Netherlands and Flanders: An outline of introductory
courses at Universities and Colleges. In: Proceedings of the Seventh International Conference on
Teaching Statistics (ICOTS-7) [CD-ROM] (eds. A Rossman and B Chance), Salvador, Brazil.
Voorburg, The Netherlands: International Statistical Institute.
Wallman KK (1993) Enhancing statistical literacy: Enriching our society. Journal of the American
Statistical Association 88(421): 1–8.
Watson J (1997) Assessing statistical literacy through the use of media surveys. In: Gal I and Garfield J
(eds), The Assessment Challenge in Statistics Education. Amsterdam: International Statistical
Institute/IOS Press, pp. 107–121.
Watson J and Callingham R (2003) Statistical literacy: A complex hierarchical construct. Statistics
Education Research Journal 2(2): 3–46.
Watson JM and Kelly BA (2008) Sample, random and variation: The vocabulary of statistical literacy.
International Journal of Science and Mathematics Education 6(4): 741–767.
Watson JM and Moritz JB (2000) Development of understanding of sampling for statistical literacy.
The Journal of Mathematical Behavior 19(1): 109–136.
Woltman M (2017) Promoting statistical thinking in schools with road injury data. Teaching Statistics
39(1): 26–29.
Yaman E and Güven N (2014) An important concept affecting students’ motivation levels: Rewards
and punishments. Journal of Human Sciences 11(1): 1163–1177.
Zichermann G and Cunningham C (2011) Gamification by design: Implementing game mechanics in web
and mobile apps. Sebastopol: O’Reilly Media.
Zimmerling E, H€ ollig CE, Sandner PG, et al. (2019) Exploring the influence of common game elements
on ideation output and motivation. Journal of Business Research 94: 302–312.
Çakıroglu and Güler 19

Author Biographies
€
Unal Çakıro
glu is currently a professor at the Department of Computer and Instructional
Technologies at Trabzon University. His academical qualifications were in Computer
Engineering (BSc), in Computer Engineering (MS) and Technology of Education (PhD).
He teaches instructional design, and IT related courses. His research interests include gami-
fication, online learning, intelligent tutoring systems, virtual reality in education.

Mustafa Güler is a PhD at Trabzon University, Fatih Faculty of Education. He graduated

from Mathematics Education department of Karadeniz Technical University in 2011 and
completed his master and doctoral program at the same university in 2014 and 2019 respec-
tively. His research areas are mathematics teacher education, mathematics learning and
teaching for elementary school students.

Appendix: Sample items

1. a) What does “average” mean? b) Give an example of an “average.”
3. The teacher asked his students how much pocket money (Turkish liras) they got from
their families the previous week and constructed the following figure according to students’
responses.

a) What does the graph tell us?

6. Your family decided to buy a new television and you narrowed your choice a LG or
Samsung. But people’s opinion is also valuable for you. Answer the following question.
b) A satisfaction survey of over a thousand people showed that Samsung had fewer
technical problems compared to LG. However, three of your Samsung user neighbors
told you that their televisions always break down. One of them got angry and grumbled:
“Yesterday I took it to the service again. Third time I took it to the service!” You want to
buy a durable product. Which one would you prefer? Why?