Evaluating a Game-Development Approach to
Teach Introductory Programming Concepts in
Secondary Education
Stamatios Papadakis
Department of Preschool Education, Faculty of Education, University of Crete
Crete, Greece
stpapadakis@gmail.com
Abstract— Owing to the development of multimedia and
computer technologies, many scholars have attempted to employ
digital learning content to improve students’ learning
motivation and learning achievement in CS courses. In this
pedagogical design, students interact and create their own
content related to curricular areas with several advantages, such
as motivation, fun, commitment, and enthusiasm, showing
improvements related to computational thinking and
computational practices. The purpose of the present study is to
evaluate the combined use of App Inventor and a game-
development approach in school lessons, as an example of
constructionist learning in an introductory programming course
for novices. This study was conducted in a public secondary
school in Greece. It used a quasi-experimental control group
design. The results of the study showed that the combined use of
App Inventor programming environment and a game
development approach has a positive effect on students’ basic
programming skills achievements and motivation in a CS
course.
Keywords— novice programmers; secondary education; app
inventor; game development approach.
I. INTRODUCTION
The demand for skilled computing professionals is
continuously increasing and consequently, the computing
discipline needs to attract quality students and educate
them to be capable and responsible professionals in the 21st
century (Petri and von Wangenheim, 2017). The ability to be
a creator rather than just a consumer of technology is
increasingly seen as an essential skill to participate fully in a
digital society Therefore, learning to program and
computational thinking is receiving growing interest in
education research.
II. THEORETICAL FRAMEWORK
Educational processes have been strengthened in recent
decades from scientific knowledge in pedagogy across
different models and methodological approaches in
educational research. The last few decades have witnessed a
great shift in education from teacher-centred to a learner-
centred methodology where students’ responsibility of the
process of their learning is maximised. Today’s educators are
leveraging technology tools that set challenges focused on
active learning in educational contexts (Sáez-López et al.,
2016). Within this learner-centred instruction, more attention
has been given to the strategy’s learners employ in the
learning process (Al Abri et al., 2017).
21st-century learners who grew up in a digital world
playing digital games also learn differently (Chang, 2010).
Educational games or game development approaches are
expected to contribute to deeper and more active learning,
through which students learn from their own experiences. The
use of games for educational purposes is a concept of
continuous academic interest. Digital games have been used
in classrooms since the 1970s with some of the most
successful educational titles being the Oregon Trail, the
Lemonade Stand, the Lure of the Labyrinth, the Orbit and the
Quest Atlantis (Stieler Hunt and Jones, 2017). Although,
most education computing courses are still taught in
traditional ways that may not be adequate to keep up with
modern concerns and may not support the necessary learning
(Petri and von Wangenheim, 2017), a number of surveys
indicate that the majority of students prefer programming
projects and assignments that one way or another involves
computer games (Kurkovsky, 2013).
The European Commission estimates that more than
725,000 high tech jobs will go unfilled in 2020. While digital
competency quickly becomes one of the most important
prerequisites for getting a job, too few students choose to
study computing to prepare them with the skills they will
need to be successful in the 21st century. A reason is that
although the learning of programming language is critical in
science and technology education, it might be difficult for
some students, especially novices. One possible reason
might be the fact that programming language, is too complex
cognitive activity and abstract for these students to
understand. There is extensive past literature in CS education
research dedicated to the various challenges faced by novice
programmers in their early encounters with algorithmic
constructs and the complex process of problem solving
involved in programming (Grover and Basu, 2017). For
instance, some of the basic computational concepts that
designers tend to use when they program are (Brennan and
Resnick 2012 as cited in Sáez-López et al., 2016):
 • Sequence
• Iteration (looping) Figure 2 Examples of code to display “Hello,
World!” on the screen with the use of two popular block-
• Conditional statements
based languages,
• Threads (parallel execution)
(a) Scratch
• Event handling
• User interface design
• Keyboard input.
The difficulty in understanding the abstract
programming concepts, as well as the traditional teaching
approach that is characterised by lack of interactivity,
reduces students’ motivation and additionally their interest
in learning programming (Piteira and Haddad, 2011 as cited
in Giannakoulas and Xinogalos, 2018, p.2). Most education
computing courses are still taught in traditional ways that
may not be adequate to keep up with modern concerns and (b) AIA
may not support the necessary learning (Petri and von
Wangenheim, 2017). According to Xinogalos and
Satratzemi the classic approach to teaching programming is
based on:
• a general-purpose programming language that is too
big and too idiosyncratic,
• a professional programming environment for that
programming language, and
• a set of problems from the area of number and symbol
processing (Xinogalos and Satratzemi, 2004, p.60) that are
hostile and alien to them.
III. RESEARCH
Figure 1 Examples of code to display “Hello, World!”
on the screen with the use of two popular text-based A. Research Questions
languages, In our research, we aim to test the following
(a) Java hypothesis: The creation of simple mobile games by students
using a suitable programming environment for novices can
improve the motivation and knowledge of first-year high
school students about programming. With this aim in mind,
this study addressed the following research questions:
1 Do the students who learn with the game development
approach show higher learning motivation for programming
than those who learn with the conventional learning
approach?
2 Do the students who learn with the game development
approach show higher learning achievement from those who
(b) C++ learn with the conventional learning approach?
B. Research Design
This study is a quasi-experimental research design,
where one group is exposed to a treatment and the results are
tested while a control group is not exposed to the treatment
and similarly tested in order to compare the effects of
treatment (Campbell and Stanley, 1963). The control group
used traditional teaching instruction and the experimental
group used a game-development teaching instruction with
the use of App Inventor. Figure 3 shows the experimental
procedure of this study. The intervention was applied in a
special section of the lesson entitled “Programming
environments for creating applications” (teaching duration
of 16 hours).
IV. EVALUATION
The researcher used an achievement test to measure the
first-grade students’ achievement on basic programming
concepts. The study of Kleinschmager and Hanenberg was
used as a basis for creating the questionnaire (Kleinschmager
and Hanenberg, 2011). The test in its final shape consisted
of twenty (20) multiple-choice items and covered aspects
related to computational concepts and computational
practices. The pre-test was given to ensure that the control
group was similar to the experimental group in their previous
basic programming skills.
The researchers used a questionnaire to measure the
first-grade students’ attitudes towards programming. The
questionnaire was developed on the basis of the Computer
Attitude Scale (CAS) (Loyd and Gressard, 1984). All aspects
were measured with the use of four-point Likert scales (Not
at all true to Very true).
The adaptation of both the questionnaires was made by
two experienced teachers who had taught the science course
for many years. This is a common approach for developing
achievement tests. All questionnaires were pilot tested prior
to the main study to establish reliability, validity, and
feasibility. They were distributed to 10 students who did not
participate in the study to minimise potential problems
related to the students’ understanding of the questionnaires,
the translation from English into Greek, the process of
conducting the investigation etc. No problems were
encountered during the pilot study. The Cronbach’s alpha
value of both the pre- test and the post-test questionnaire
were .82 and .84 respectively, showing good reliability in
internal consistency. The whole data was analysed in the
Statistical Package for the Social Sciences SPSS application
(ver. 23).
V. RESULTS
A. Equivalence between the groups
Prior to teaching intervention, statistical analyses of
differences between the experimental and the control group
were performed. The results after applying suitable statistical
tests showed that both groups did not differ significantly in
their characteristics, such as gender (X2(2) = .43, p>.05),
behavioural approach (F(1,28) = .62, p>.05) and knowledge
in programming (F(1,28) = .74, p>.05). In summary, the
results show that both groups are equivalent in term of
factors that are thought to underlie behavioural and
programming knowledge differences (Berdousis and
Kordaki, 2015; Chan et al., 2000).
B. Learning achievement
The result of the analysis on the two groups’ pre-test
scores indicated that there was no significant difference in
the prior knowledge of the basic programming concepts.
Thus, the analysis of covariance (ANCOVA) can be applied
to compare their post-test scores by excluding the impacts of
the pre-test scores and using the pre-questionnaire of
learning achievement as the covariant. A significance was
found for the learning achievement of the experimental
group and the control group after the teaching intervention
(F = 3.58, p <.05). The experimental group outperformed the
control group, indicating that the game-development
approach can effectively increase students’ knowledge of
basic programming concepts.
C. Learning motivation
Similarly, an analysis of covariance (ANCOVA) was
adopted by using the pre questionnaire of learning
motivation as the covariant. A significance was found for the
learning motivation of the experimental group and the
control group after the different learning activities (F = 5.57,
p<.05). The experimental group outperformed the control
group, indicating that the game-development approach can
effectively increase students’ learning motivation.
VI. DISCUSSION
The purpose of this study was to investigate the
differences between the effect of using traditional teaching
instruction and using a game-development approach on first-
grade students’ learning achievement and motivation in
programming. A pre-test was conducted to collect the data
before applying the treatment to prove the equivalents of the
control and experimental groups. The data were tested
statistically. The following hypotheses were tested at .05
significance levels
1 There is a significant difference in students’
achievements test between the control group who used
traditional teaching instruction and the experimental group
who used a game-development approach.
2 There is a significant difference in students’
motivation test between the control group who used
traditional teaching instruction and the experimental group
who used a game-development approach.
The results of the study were:
• A statistically significant difference has been found
between the mean scores of the experimental and control
group in their post-test scores regarding their achievement
on the basic programming concepts.
• A statistically significant difference has been found
between the mean scores of the experimental and control
group in their post-test scores, regarding their learning
motivation.
 VII. STUDY LIMITATIONS motivation and knowledge in basic programming concepts.
This study has various limitations:
• This was a pilot study with a small sample size. The ACKNOWLEDGMENT
study was limited to the first grade students at a public
The author would like to thank all the students who took
school in the second semester of the school year 2017–
part in this study.
2018. The effects with larger sample sizes should be further
investigated.
• This was a short-term study. More long-term REFERENCES
evaluations are required to investigate the influences on [1] Al Abri, A., Al Seyabi, F., Al Humaidi, S. and
student learning and motivation and examine their Hasan,
consistency with this study. A.H. (2017) ‘Grammar learning strategies in
Omani EFL classes: type and relation to student
• Results of this study are limited to the range of proficiency’, Journal of Studies in Education,
accuracy and suitability of the instruments used in this Vol.7, No. 2, pp.151–166.
study. [2] Allsopp, B.B. and Ejsing-Duun, S. (2016)
‘Programming concepts in playful programming
products’, 10th European Conference on Games
Based Learning: ECGBL 2016, p.1.
VIII. CONCLUSION
[3] Ariffin, M.M., Ahmad, W.F.W. and Sulaiman, S.
There is a growing consensus that computational (2017) ‘Culturally-enhanced computer game: the
thinking is a fundamental skill that everyone needs to impact on educational effectiveness’, The Social
Sciences, Vol. 12, No. 4, pp.694–701.
succeed in our complex and technological culture (Wing,
2006). The demand for skilled computing professionals is [4] Baytak, A., Land, S.M. and Smith, B.K. (2011)
continuously increasing and, consequently, the computing ‘Children as educational computer game designers:
an exploratory study’, Turkish Online Journal of
discipline needs to attract quality students and educate them Educational Technology, Vol. 10, No. 4, pp.84–92.
to be capable and responsible professionals in the 21st
[5] Berdousis, I. and Kordaki, M. (2015) ‘Gender
century. Computer programming although is an excellent differences and achievement in computer science:
way to develop computational thinking skills is considered a case study’, 6th World Conference on
as a complex cognitive activity. Students often give up Educational Sciences, pp.1161–1166.
Computer Science because they think it is confusing and [6] Brown, N. (2017) Pedagogy of Programming
difficult. There is extensive past literature in CS education Tools.Available online at: https://academicco
research dedicated to the various challenges faced by novice mputing.wordpress.com/author/twistedsquare/
programmers in their early encounters with algorithmic [7] Campbell, D.T. and Stanley, J.C. (1963)
constructs and the complex process of problem-solving ‘Experimental and quasi-experimental designs for
involved in programming (Grover and Basu, 2017). research’, Handbook of Research on Teaching,
Programming environments for novices such as Scratch, pp.171–246.
Alice, and recently MIT App Inventor have been [8] Chan, V., Stafford, K., Klawe, M. and Chen, G.
demonstrated to be effective in educational environments. (2000) ‘Gender differences in Vancouver
secondary students’, Women, Work and
Researchers have well recognised educational game- Computerization, Springer, Boston, MA, pp.58–
based learning approaches as being a highly potential 69.
approach to motivating learners of all ages. For example, [9] Chang, M. (2010) ‘Web-based multiplayer online
Papastergiou (2009) indicated that digital game-based role playing game (MORPG) for assessing
students’ Java programming knowledge and
learning could motivate students to learn in computer science skills’, Digital Game and Intelligent Toy
courses (Sung et al., 2017). This type of game creation would Enhanced Learning (DIGITEL), 2010 Third IEEE
not only enhance the playing experience but could also lead International Conference on, IEEE, pp.103–107.
to a depth and scope of understanding in the learning context [10] Chen, P. and Huang, R. (2017) ‘Design thinking
through different types of gaming tasks following the in App inventor game design and development: a
storyline (Sung et al., 2017). case study’, 2017 IEEE 17th International
Conference on Advanced Learning Technologies
Concluding, there are a lot of studies that demonstrate (ICALT), IEEE, pp.139–141.
the advantages of digital games in learning, not only for [11] Earp, J. (2015) ‘Game making for learning: a
transversal skills such as communication, collaboration, fine systematic review of the research literature’,
motor skill, etc. but also for specific skills in particular Proceedings of 8th International Conference of
knowledge domains (Moreno and Méndez, 2015). Thus, Education, Research and Innovation
(ICERI2015), pp.6426–6435.
further research into this area is required as it will provide a
valuable understanding of the potential of using a game- [12] Fincher, S., Cooper, S., Kölling, M. and Maloney,
J. (2010) ‘Comparing alice, greenfoot & scratch’,
based development approach as a tool for increasing student
 Proceedings of the 41st ACM Technical [24] Maloney, J., Resnick, M., Rusk, N., Silverman, B.
Symposium on Computer Science Education, and Eastmond, E. (2010) ‘The Scratch
ACM, pp.192–193. programming language and environment’,
Transaction on Computing Education, Vol. 10,
[13] Giannakoulas, A. and Xinogalos, S. (2018) ‘A No. 4, pp.16:1–16:15.
pilot study on the effectiveness and acceptance of
an educational game for teaching programming [25] Misfeldt, M. and Ejsing-Duun, S. (2015) ‘Learning
concepts to primary school students’, Education mathematics through programming: an
and Information Technologies, pp.1–24. instrumental approach to potentials and pitfalls’,
CERME 9-Ninth Congress of the European
[14] Grover, S. and Basu, S. (2017) ‘Measuring student Society for Research in Mathematics Education,
learning in introductory block-based pp.2524–2530.
programming: examining misconceptions of
loops, variables, and [26] Moreno, J. and Méndez, N.D. (2015) ‘Teaching
Boolean logic’, Proceedings of the 2017 ACM sciences in K-12 using 2D educational massive
SIGCSE Technical Symposium on Computer online games’, Anais temporários do LACLO
Science Education, ACM, pp.267–272. 2015, Vol. 10, No. 1, p.394.
[15] Guzdial, M. (2004) ‘Programming environments [27] Orfanakis, V. and Papadakis, S. (2014) ‘A new
for novices’, Computer Science Education programming environment for teaching
Research, pp.127–154. programming. A first acquaintance with
enchanting’, 2nd International Virtual Scientific
[16] Johnson, L., Becker, S.A., Cummins, M., Estrada, Conference - Scieconf 2014, EDIS - University of
V., Freeman, A. and Hall, C. (2016) NMC Zilina, Slovakia, pp.268–273.
horizon report: 2016 higher education edition, The
New Media Consortium, pp.1–50. [28] Orfanakis, V. and Papadakis, St. (2016) ‘Teaching
basic programming concepts to novice
[17] Kafai, Y.B. (2018) ‘Constructionist visions: hard programmers in secondary education using
fun with serious games’, International Journal of Twitter, Python, Ardruino and a coffee machine’,
Child-Computer Interaction, Vol. 18, pp.19–21. Hellenic Conference on Innovating STEM
[18] Kalogiannakis, M. and Papadakis, St. (2018a) ‘A Education (HISTEM), 16–18 December,
proposal for teaching ScratchJr programming University of Athens, Greece.
environment in preservice kindergarten teachers’, [29] Papadakis, S. (2018a) ‘Gender stereotypes in
in Finlayson, O., McLoughlin, E., Erduran, S. and Greek computer science school textbooks’,
Childs, P. (Eds): Electronic Proceedings of the International Journal of Teaching and Case
ESERA 2017 Conference. Research, Practice and Studies, Vol. 9, No. 1, pp.48–71. Papadakis, S.
Collaboration in Science Education, Part (2018b) ‘The use of computer games in classroom
15/Strand 15 (co-ed. Bodil Sundberg and Maria environment’, International Journal of Teaching
Kallery, Early Years Science Education), Dublin and Case Studies, Vol. 9, No. 1, pp.1–25.
City University, Dublin, Ireland, pp.2095–2105.
ISBN 978-1-873769-84-3 [30] Papadakis, S. (2019) ‘Evaluating the efficiency of
two programming environments in shaping
[19] Kalogiannakis, M. and Papadakis, St. (2018b) novices’ attitudes, perceptions, beliefs and
‘Pre-service kindergarten teachers acceptance of knowledge in programming: a comparison
“ScratchJr” as a tool for learning and teaching between Scratch and App Inventor’, International
computational thinking and Science education’, Journal of Teaching and Case Studies, Vol. 10,
Proceedings of the 12th Conference of the No. 1, pp.31–52.
European Science Education Research
Association (ESERA), Research, practice and [31] Papadakis, S. and Orfanakis, V. (2017) ‘The
collaboration in science education, Dublin, Ireland. combined use of lego mindstorms NXT and App
Inventor for teaching Novice programmers’, in
[20] Kelleher, C. and Pausch, R. (2005) ‘Lowering the Alimisis, D., Moro, M. and Menegatti, E. (Eds):
barriers to programming: a taxonomy of Educational Robotics in the Makers Era.
programming environments and languages for Edurobotics 2016. Advances in Intelligent
novice programmers’, ACM Computing Surveys, Systems and Computing, Vol. 560, Springer,
Vol. 37, No. 2, pp.83–137. Cham, pp.193–204.
[21] Kleinschmager, S. and Hanenberg, S. (2011) ‘How [32] Papadakis, S. and Orfanakis, V. (2018) ‘Comparing
to rate programming skills in programming novice programing environments for use in
experiments? A preliminary, exploratory, study secondary education: App Inventor for Android
based on university marks, pretests, and self- vs. Alice’, International Journal of Technology
estimation’, Proceedings of the 3rd ACM Enhanced Learning, Vol. 10, Nos. 1/2, pp.44–72.
SIGPLAN workshop on Evaluation and usability
of programming languages and tools, ACM, [33] Papadakis, S. and Kalogiannakis, M. (2017a)
pp.15–24. ‘Using gamification for supporting an
introductory programming course. The case of
[22] Kurkovsky, S. (2013) ‘Mobile game development: ClassCraft in a secondary education classroom’,
improving student engagement and motivation in Interactivity, Game Creation, Design, Learning,
introductory computing courses’, Computer and Innovation, Springer, Cham, pp.366–375.
Science Education, Vol. 23, No. 2, pp.138–157.
[34] Papadakis, St. and Kalogiannakis, M. (2017b)
[23] Loyd, B.H. and Gressard, C. (1984) ‘Reliability ‘Combining mobile technologies in environmental
and factorial validity of computer attitude scales’, education: a Greek case study’, International
Educational and Psychological Measurement, Journal of Mobile Learning and Organisation, Vol.
Vol. 44, No. 2, pp.501–505.
 11, No. 3, pp.256–277. pp.199–208.
[35] Papadakis, St., Kalogiannakis, M., Orfanakis, V. [47] Wilson, A. and Moffat, D.C. (2010) ‘Evaluating
and Zaranis, N. (2014) ‘Novice Programming Scratch to introduce younger schoolchildren to
Environments. Scratch & App Inventor: a first programming’, Proceedings of the 22nd Annual
comparison’, Proceedings of the 2014 Workshop Psychology of Programming Interest Group,
on Interaction Design in Educational Universidad Carlos III de Madrid, Leganés, Spain.
Environments (IDEE’14), ACM, New York, NY,
USA. [48] Wing, J.M. (2006) ‘Computational thinking’,
Communications of the ACM, Vol. 49, No. 3,
[36] Papadakis, S., Kalogiannakis, M., Orfanakis, V. pp.33–35. Xinogalos, S. and Satratzemi, M.
and Zaranis, N. (2017) ‘The appropriateness of (2004) ‘Introducing novices to programming: a
Scratch and App Inventor as educational review of teaching approaches and educational
environments for teaching introductory tools’, Proceedings of the 2nd International
programming in primary and Conference on Education and Information
secondary education’, International Journal of Systems, Technologies and Applications (EISTA
Web-Based Learning and Teaching Technologies, 2004), pp.60–65.
Vol. 12, No. 4, pp.58–77.
[37] Papastergiou, M. (2009) ‘Digital game-based
learning in high school computer science
education:
impact on educational effectiveness and student
motivation’, Computers & Education, Vol. 52, No.
1, pp.1–12.
[38] Petri, G. and von Wangenheim, C.G. (2017) ‘How
games for computing education are evaluated? A
systematic literature review’, Computers &
Education, Vol. 107, pp.68–90.
[39] Prensky, M. (2008) ‘Students as designers and
creators of educational computer games: who
else?’ British Journal of Educational Technology,
Vol. 39, No. 6, pp.1004–1019.
[40] Robins, A., Rountree, J. and Rountree, N. (2003)
‘Learning and teaching programming: a review
and discussion’, Computer Science Education,
Vol. 13, No. 2, pp.137–172.
[41] Sáez-López, J.M., Román-González, M. and
Vázquez-Cano, E. (2016) ‘Visual programming
languages integrated across the curriculum in
elementary school: a two year case study using
“Scratch” in five schools’, Computers &
Education, Vol. 97, pp.129–141.
[42] Stieler-Hunt, C.J. and Jones, C.M. (2017) ‘Feeling
alienated–teachers using immersive digital games
in classrooms’, Technology, Pedagogy and
Education, Vol. 26, No. 4, pp.457–470.
[43] Sung, H.Y., Hwang, G.J., Lin, C.J. and Hong,
T.W. (2017) ‘Experiencing the Analects of
Confucius: an experiential game-based learning
approach to promoting students’ motivation and
conception of learning’, Computers & Education,
Vol. 110, pp.143–153.
[44] Van Eck, R. (2006) ‘Digital game-based learning:
It’s not just the digital natives who are restless’,
EDUCAUSE Review, Vol. 41, No. 2, p.16.
[45] Weintrop, D. (2015) ‘Comparing text-based,
blocks-based, and hybrid blocks/text
programming tools’, Proceedings of the 11th
Annual International Conference on International
Computing Education Research, ACM, pp.283–
284.
[46] Weintrop, D. and Wilensky, U. (2015) ‘To block
or not to block, that is the question: students’
perceptions of blocks-based programming’,
Proceedings of the 14th International Conference
on Interaction Design and Children, ACM,