Vol 19 No 8 August 2020

IJLTER.
ORG
International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN: 1694-2493
e-ISSN: 1694-2116
Vol.19 No.8
International Journal of Learning, Teaching and Educational Research
(IJLTER)
Vol. 19, No. 8 (August 2020)
Print version: 1694-2493
Online version: 1694-2116
IJLTER
International Journal of Learning, Teaching and Educational Research (IJLTER)
Vol. 19, No. 8
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically those of translation, reprinting, re-use of illustrations,
broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.
Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Research is a peer-reviewed open-access journal which has been
established for the dissemination of state-of-the-art knowledge in the
fields of learning, teaching and educational research.
Aims and Objectives
The main objective of this journal is to provide a platform for educators,

teachers, trainers, academicians, scientists and researchers from over the
world to present the results of their research activities in the following
fields: innovative methodologies in learning, teaching and assessment;
multimedia in digital learning; e-learning; m-learning; e-education;
knowledge management; infrastructure support for online learning;
virtual learning environments; open education; ICT and education;
digital classrooms; blended learning; social networks and education; e-
tutoring: learning management systems; educational portals, classroom
management issues, educational case studies, etc.
Indexing and Abstracting

Research is indexed in Scopus since 2018. The Journal is also indexed in
Google Scholar and CNKI. All articles published in IJLTER are assigned
a unique DOI number.
Foreword
We are very happy to publish this issue of the International Journal of

Learning, Teaching and Educational Research.

Research is a peer-reviewed open-access journal committed to
publishing high-quality articles in the field of education. Submissions
may include full-length articles, case studies and innovative solutions to
problems faced by students, educators and directors of educational
organisations. To learn more about this journal, please visit the website
http://www.ijlter.org.
We are grateful to the editor-in-chief, members of the Editorial Board

and the reviewers for accepting only high quality articles in this issue.
We seize this opportunity to thank them for their great collaboration.
The Editorial Board is composed of renowned people from across the

world. Each paper is reviewed by at least two blind reviewers.
We will endeavour to ensure the reputation and quality of this journal

with this issue.
Editors of the August 2020 Issue

VOLUME 19 NUMBER 8 August 2020
Table of Contents
Training Professional Humanities’ Teachers: A Controversial Study about Generic Methods ................................... 1
Tamar Ketko
The Effects of Mobile Learning on Listening Comprehension Skills and Attitudes of Omani EFL Adult Learners ...
................................................................................................................................................................................................. 16
Abdullah Al-Shamsi, Abdo Mohamed Al-Mekhlafi, Saleh Al Busaidi and Maher Mohammad Hilal
What about Study Motivation? Students´ and Teachers’ Perspectives on What Affects Study Motivation ............ 40
Lena Boström and Göran Bostedt
The Dragon, the Knight and the Princess: Folklore in Early Childhood Disaster Education ..................................... 60
Maila D.H. Rahiem and Husni Rahim
Lecture-simulation-combined Education Improve Nursing Undergraduates' Knowledge and Attitude for

Palliative Care ....................................................................................................................................................................... 81
Yan Wang
Contextualising Computational Thinking: A Case Study in Remote Rural Sarawak Borneo .................................... 98
Nur Hasheena Anuar, Fitri Suraya Mohamad and Jacey-Lynn Minoi
Can Peer to Peer Interaction (PPI) be a Global Theme to Promote Engagement in Students of Diverse
Characteristics and Country Contexts? ............................................................................................................................ 117
Nazlee Siddiqui, Khasro Miah, Afreen Ahmad Hasnain and David Greenfield
Teacher Education Institutions in the Philippines towards Education 4.0 .................................................................. 137
Rivika Alda, Helen Boholano and Filomena Dayagbil
Influence of Demotivators on Acceptance of Technology: Challenges of Expatriate School Teachers while

Imparting Online Teaching ............................................................................................................................................... 155
Gokuladas V. K. and Baby Sam S. K.
Conceptual Framework of Evaluation Model on 4 C'S-Based Learning Supervision ............................................... 173

Eny Winaryati, Mardiana and Muhamad Taufik Hidayat
The Effect of Classroom Climate on Academic Motivation Mediated by Academic Self-Efficacy in a Higher
Education Institute in China .............................................................................................................................................. 194
Qiumei Wang, Kenny Cheah Soon Lee and Kazi Enamul Hoque
Metacognitive Writing Strategies Used by Omani Grade Twelve Students ............................................................... 214
Ibtisam Sultan Al Moqbali, Salma Al Humaidi, Abdo Al Mekhlafi and Maher Abu Hilal
Implementation of Online Learning during the Covid-19 Pandemic on Madura Island, Indonesia....................... 233
Priyono Tri Febrianto, Siti Mas'udah and Lutfi Apreliana Megasari
High School Students’ Difficulties in Making Mathematical Connections when Solving Problems ....................... 255
Jailani ., Heri Retnawati, Ezi Apino and Agus Santoso
Application of Rasch Model to Develop a Questionnaire for Evaluating the Quality of Teaching for Students’
Creativity Development ..................................................................................................................................................... 278
Thi Le Thuy Bui, Vyacheslav I. Kazarenkov and Van De Tran
The Challenges of South African Teachers in Teaching Euclidean Geometry ............................................................ 297
Simon A. Tachie
Health Professional Students’ Perceptions and Experiences of Remote Learning During the COVID-19 Pandemic
............................................................................................................................................................................................... 313
Fatmah Almoayad, Afrah Almuwais, Samiah F. Alqabbani and Nada Benajiba
Baseline Assessment in the Elementary Mathematics Classroom: Should it be Optional or Mandatory for
Teaching and Learning? ..................................................................................................................................................... 330
Mamsi Ethel Khuzwayo and Herbert Bhekumusa Khuzwayo
Authentic Videos in Teaching English to Engineering Students at Universities ........................................................ 350

Nataliia Saienko and Mariana Shevchenko
Quality Management of Educational Activities in the Training of Specialists in the Field of Health Care: the Case
of Ukrainian Medical HEIs ................................................................................................................................................ 371
Svitlana V. Gordiychuk, Liudmyla M. Kalinina, Irena E. Snikhovska and Olga V. Goray
Use of Augmented Reality to Improve Specific and Transversal Competencies in Students .................................. 393
Esteban Vázquez-Cano, Verónica Marín-Díaz, Wellington Remigio Villota Oyarvide and Eloy López-Meneses
How School Culture and Teacher’s Work Stress Impact on Teacher’s Job Satisfaction ............................................ 409
Susan Febriantina, Suparno Suparno, Marsofiyati Marsofiyati and Rusi Rusmiati Aliyyah
Investigating the Quality of University Education: A Focus on Supply Chain Management .................................. 424
Joash Mageto, Rose Luke and Gert Heyns
Exploring the Content Knowledge of Accounting Teachers in Rural Contexts: A Call for a Decoloniality
Approach ............................................................................................................................................................................. 447
Habasisa Vincent Molise
1

Vol. 19, No. 8, pp. 1-15, August 2020
https://doi.org/10.26803/ijlter.19.8.1
Training Professional Humanities’ Teachers:

A Controversial Study about Generic Methods
Tamar Ketko
Kibbutzim College of Education, Tel Aviv, Israel
https://orcid.org/0000-0001-7552-8545
Abstract. In the 21st century, generic methods and synergetic learning

have been widely embraced in the areas of pedagogical and professional
studies. It is crucial, especially in school-activity environments that
involve technology and digital knowledge. Those who are capable of
studying in teams and who promote ‘collective intelligence’ are likely to
become influential and inspiring students and teachers. By
understanding aligned visions from different viewpoints, students and
teachers can maximize their efforts and talents. The idea of collective
teacher efficacy (CTE) positively affects student outcomes and therefore
is an essential tool in teacher training and practices. We live in the
ongoing dynamics of integrated diverse thoughts, methods, disciplines,
and activities. To create a better ecology for qualitative existence,
numerous scholars and teachers, seek to devise necessary changes in
education and social initiatives. In a world split by regimes and values,
dealing with conflictual dilemmas is inevitable: preserving classical
methods on the one hand, and encouraging innovative attitudes on the
other. These contradictory approaches raise critical didactical questions
about training future teachers and educators without prejudicing their
fundamental essence. This article presents a three-years research of a
group of students, at the Kibbutzim College of Education, Tel Aviv, who
were trained to become professional teachers in the humanities, and their
pedagogical eco-systems. It discusses some dilemmas about progressive
school methods and focuses on some of the advantages and
disadvantages of the generical attitudes in their practical work, regarding
the gap between their first year of studying and the first year of teaching.
Keywords: Generic studies; Interdisciplinary; Pedagogical attitudes;

Professional training
1. Introduction
The differentiation, isolation, and preventing the blurring of identity, were always
the focal point of competition, tensions between tribes, peoples, cultures, and
governing mechanisms. This idea is also present in the theological sources and the
logic underlying division into categories. It is reasonable to assume that on these
foundations, seven fields of human wisdom have been consolidated, and
©2020 The author and IJLTER.ORG. All rights reserved.

2
separated from rituals and theology. These seven perceived components of a well-
balanced education system are divided into two groups. The first comprised
Latin, rhetorical language and logic, and the second, Mathematics, Geometry,
Astronomy, and Music. In the 12th century, after the first universities were
established in Western Europe, academic faculties also added Theology, Law,
Medicine, and Philosophy as fields of study. Continuing the tradition of the
guilds, which represented diverse skills in the Middle Ages, the importance of a
profession grew as did the clarification of professional skills. These had to be
studied in an orderly manner under the strict supervision of the “master,” to
guarantee the necessary level of execution and knowledge. This led to the
development of vocational schools and higher education institutions that taught
building, engineering, architecture, all types of technology, and the
accompanying practical fields of knowledge, which demanded accreditation and
a degree following advanced international standards (Doolitlle, 2015; Bergman,
2018).
Shifting chronologically to the Modern Age, it appears that the ability to manage
knowledge and its fabric of combinations is manifested not only in the degree of
aspiration to readjust it to present reality but also in the ability to respond to the
unexpected. The idea of focusing only on what is relevant to human existence and
the professional field questions the value of accumulated knowledge and the
acquisition of basic concepts and introductory infrastructures. Response to a
specific policy that each regime enforces, current events, and changing public
trends, dictates what study content will remain, and what will be deleted. Steps
of this kind necessarily demand innovative research methods and skills of follow-
up and control for measuring the educational yardstick and risk evaluation. This
is true because of the 21st century, which will most probably be characterized by
political, social, and cultural uncertainty, the impact of the media and social
networks on methods of choice, and the level of achievement and success in the
field (Goleman, 2006; Brophy, 2006).
We are now witnessing an ever-developing trend of research and workspace that

underscore the need for synergetic collaboration, which abandons professional
isolation and fortification within spheres of interest solely on vertical axes. This is
a horizontal perception that advocates spheres of knowledge relevant to
improved results, mainly in subjects that pertain to human life, such as medicine,
psychology, law, education, and teaching. Facing this contemporary global age
demands more brainstorming processes which include high numbers of
participants in the vein of the whole being greater than its parts (Plucker, Kennedy
& Dilley, 2019). The variety of processes and technical and scientific possibilities
creates opportunities for collaboration with people in faraway places, in
tangential spheres, most notably in the academic and educational fields.
The central discourse in this article examines different and contradictory aspects
of the process of assimilation of generic and synergetic methods in the educational
systems and the process of training teachers for the 21st century (Griffin & Care,
2014). The generic ecosystem demands the development of social and personality
skills, such as advanced skills in digital technology, language command, and

3
interaction with people who come from different cultures and fields. This also
mandates relinquishment of ego struggles, power positions, exclusivity, and the
control over copyrights in favor of a multiplicity of intelligence and partnership
in implementation and success (Plucker et al., 2019; Gamoran & An, 2016). It is
reasonable to assume that a reality that fosters teaching through generic training,
learning from afar, a decrease in the number of subjects and examinations, and a
shift toward thematic “tasting” in the school space, is and will be rapid. The
question is, will the result justify itself.
2. Personal success vs. group achievement: fostering generic learning

The synergetic concept has been taken from the language of organizational
management in diverse cycles: macro-level–global governmental forums, public
organizations, and academic institutions, and micro-level–municipal councils,
political movements, community centers, and schools. It is essential to understand
the uniqueness of the engendered perceptual change by incorporating the fields
of knowledge, and entrepreneurship skills, and their execution. Knowing that, we
can grasp the difference between the idea of one entrepreneur being innovative
and groundbreaking as he or she may be, and the entrepreneurship process is
undertaken by a team of several copywriters, each in his or her field. Recognize
the contribution of a successful plan, both, by empowering partners and
maximizing their skills is important in building mechanisms that guarantee the
best kind of assimilation. At the same time there may be risks involved in the
encounter of ideas and personal styles in every project, and doubtlessly in
education and learning. Before examining the new teaching methods, it is
necessary to clarify the foundations of the synergetic perception and what should,
or should not, be adopted to empower teachers and learning processes in schools
of the future generations.
Synergy is a joint activity or study that involves two or more participants who
come from different disciplines or professions. By collaborative work, they seek
to increase the value of their mission and enrich one another with ideas and
personal or guided knowledge. This process makes ‘the whole greater than the
sum of its parts (1+1=3) and it creates many thoughts and encourages diverse
discourse (Hattie, 2016). Moreover, it is a humanistic mechanism that explains
how team participation reinforces the ability to identify, understand, and solve
complex issues in almost every subject. Such sharing enables mediation and the
completion of each one’s lacunas separately, overcoming the weak points of each.
It is essential to underscore that the benefits of synergetic and generic
collaborative activity depend on the need, the ecosystems culture, the
participants’ abilities, and the risks facing those about to join. The importance of
sharing methods has also been expressed in encouraging continuous learning
from one another, seeing how others behave, think and operate, and viewing
things from a new perspective (Fullan, 2016). How does this affect the education
system and teacher training?
The generic and synergetic approach became an essential part of many

educational systems and teachers' training programs (Goleman, 2006; Rosiek &
Kinslow, 2016). It seems that the study content and choice of specialization and

4
professionalization streams have not been pursued obtaining a diploma, status,

or a threshold for promotion. Teaching has become a changing interactive space,
dependent on social, political, cultural, occupational, and interest-driven context.
Everything that takes place inside and between classrooms is a microcosm of what
goes on outside, with the relevant, pragmatic direction which has practical
implications on the community of learners (Carlson, 2017). The diversity of
intelligence, thinking teams, and synergetic work, leading multidisciplinary
initiatives and projects, has permeated the education system and teachers’
training. However, it is important to conduct an in-depth observation of the
dynamics of these frameworks, and the place of the individual within them. In the
final analysis, we are speaking of the life, coping, and success of each student
individually (Rothstein, 2017).
The ultimate methods for qualitative learning, which will preserve a high
standard of intellectual and professional curiosity, and address the needs of an
ever-advancing reality, shifts like a pendulum between the axis of time and
findings that change from time to time. Until twenty years ago, the results of direct
instruction, characterized by clear-cut definitions of the lesson objective, the
development of an individual relationship with learners, and skills for examining
the level of their theoretical and practical understanding, were lauded. Findings
show that personal and direct contact had the most significant effect on the level
of achievement and the student’s success in later stages of life (Hattie, 2015).
According to Hattie, a review of every learner, mainly those who were average or
below average, made the management of expectations and examination of the
complexity of the student’s character and abilities, imperative. He claimed that
explicit teaching transformed teachers into role models, rendering them self-
critical, and self-reflective vis-à-vis each student anew. In this way, in which
teachers could become “a teacher of him, or herself” mentors, they examine the
world through the eyes of their students, and sufficiently skilled to instill in them
these abilities (Nir, Ben David, Bogler, Dan & Zohar,2016; Schofer, 2019).
With the overusing of the traditional models of teaching means, lesson structure,
performance, and division into activity teams, the term “pedagogical innovation”
became frequently required. The idea of making more sources accessible does not
depend only on attractive digital and “less tiresome” appearances in contrast to
“old methods.” The beginning of the changing process of the academic
community considering the use of digital innovation was based mainly on an
empirical pilot study performed in the alternative, democratic, or “natural”
schools (Alammary, Sheard & Carbone 2014; Plucker, et al., 2019). At the same
time, social movements emerged, calling to bring education back to “human
nature,” eradicate the competition for grades and adjust achievement measures to
the individual pace of each learner (Goodman, Joshi, Nasim & Tyler, 2015). In
other words, assimilation of the technological means in the pedagogical and
academic systems was carried out slowly, coupled with professional and research
distrust. This was true despite the OECD findings of Paniagua & Istance (2018)
that showed how approaches that combine generic knowledge clusters with
digital innovation not only boost achievement but also help in cultivating values
of collaboration, mutual responsibility, social and emotional empathy, and

5
readiness for the 21st century. Alex Paniagua and David Istance, OECD
researchers, believe that this is closely linked to the degree of technological and
communicational exposure. Their findings show that all these changes
encouraged learning based on inquisitiveness, an in-depth study based on
experiential partnership, demonstrating an improvement in the level of
achievement (Paniagua & Istance, 2018: 77-84).
In this context, education researchers Calarco (2019) and Schofer (2019), who deal
with the development of schools affected by change over time, claim that thought
should be devoted to the tension created between what is desirable and real. On
the one hand, boundless openness concerning instilling skills for knowledge
management is encouraged, still, on the other hand, the school is turning into a
functional organization recruited to provide a precise response to a vital policy in
its existential environment. The neo-institutional theory that they explored tested
the innovative approaches on a dual reality test of the achievements of the
individual within the ordinary achievements of the team or group. The innovative
pedagogical approaches offer differential teaching and enhancement of the
motivation of students according to their abilities, together with online teaching
from afar outside the classroom framework. In this way, learners’ achievements
depend on them only, on their knowledge, literate and analytical skills, and ability
to concentrate. At the same time, circles of learning companions are encouraged,
corresponding to the group project method that demands shared and synergetic
responsibility for each study and research assignment. According to these
methods, the greater the number of knowledge spheres and research sources and
creativity, the greater the high-level and trailblazing achievements (Sahlberg,
2015, 2018). In such instances, commendation is accorded to all group members,
or the project, with no specific and special attention to one of them, only. The
objective of changes to the perception of education management systems and its
practical assimilation is to train students to become citizens and human beings.
They will be attuned to a future reality, not only on the product level. All forms
of thinking may change, as well as the value hierarchy and measures of evaluation
of success, decision-making, and the choice of career and specialization (Fullan,
2016; Calarco, 2019; Schofer, 2019).
Remaining for a lengthy period in a specific place to secure a higher position is no

longer relevant in a dynamic reality that shifts people from one place to another
in a short timeframe to meet the rapid pace of innovation. To this end, it is
imperative to invest in a different language of thinking and implementation skills,
such as creativity, critical thinking, problem-solving strategies, partnership
management, and practical synergy. Exploring innovative teaching methods
shows that generic skills are used and needed in progressive schools and are
necessary for free lessons and synergetic projects. This also relates to the PBL
(Project Based Learning) method that reasonably represents the perceptual
change in managing and instilling knowledge acquisition. Since coping with
content and the completion of assignments is usually carried out in teams, it is
essential to train moderators and teachers who will preside over learning and the
implementation of this collective work, people who will be able to orchestrate a

6
philharmonic work to guarantee the best well-suited product (Brophy, 2006;

Doolittle, 2015).
However, in a combined project, be it learning, research, or practical, it is

imperative to be attentive to the inner and interpersonal dynamics within each
team and group. This mandates an agreed-upon contract with well-defined
definitions and clear-cut conditions that relate to the strength of each participant,
the scope of his or her contribution, and ability to meet schedules despite the
individual pace, which is never identical. Also, it is crucial to relate to the capacity
to accept criticism and feedback in a democratic and empathic way at every stage,
mainly when it is mandatory to favor the success of the group or the assignment
over individual promotion. It is important to remember that not everyone is fit to
work in a team or a group (Goodman et al., 2015). This characteristic and the
willingness to relinquish ego and special status is not entirely natural or self-
evident. It is often essential to assess the nature of the assignment, and see whether
it promotes synergetic and energetic group work. Still, it obstructs a leap forward
by one of the team members who possesses a unique trait (Rosiek & Kinslow,
2016).
In such an event, they need distant space for action and a route of activity that is
separate from other group members so that they will not “interfere” their
performances and revelations in specific fields. This is true of teams of teachers,
members of the academic staff, or any other organization. We are witness to the
natural behavioral components of jealousy, competitiveness, the ability to take
genuine pride in the success of the other, and the ability to cope with human
differences. Learning methods and research, brainstorming, and generic and
synergetic endeavor may expose such human weaknesses and the gaps that may
arise may, even on the covert level, obscure the final results and the profit of the
collaborative process (Gamoran and An, 2016; Bergman, 2018; Virtanen & Tynjälä,
2019).
In the collective circles of the 21st century, students are the ones who play a central
role and not teachers, moderators, or principals. In these spaces, they are more the
facilitators of studying and research mentors who encourage critical thinking and
discover new concepts of creativity (Bauder & Rod, 2016), trying not to remove
the needs of students and team members, abilities, or personal style. Accepting
that this term is essential for any synergized process - is it possible not to demand
it, sometimes, in favor of the group’s interests and success? This issue boosted
renewed deliberation on thinking strategies and the implementation of training
professional teachers, particularly in middle schools and high schools. Through
several case studies that were examined in the Humanities Department at a Tel
Aviv Teacher Training College, several unexpected findings concerning the idea
of synergy in teaching and generic learning will be presented. It will be compared
with disciplinary training in separate streams according to “old” methods.

7
3. Going back to disciplinary learning: Methodology and study

characteristics
In research, it is customary to speak of three different models of teacher training.
The first is behavioristic and attributes importance to the “teacher’s technical
toolbox.” Regarding this approach, teaching is a profession measured by its
qualities and achievements, and it examines how a teacher meets the demands of
the subject and acts according to policy (Ashton, 1996; Ainscow, 2005). Here the
thematic and pedagogical content precedes any practical activity and therefore
comes before all experiential work. Training for the enrichment and the
development of creativity will be carried out only after proving theoretical
conversancy (Christianakis, 2010; Griffin & Care, 2014). The second model is
rationalistic, which relates to teachers not only as of the “executors” of policy and
an instrumental tool of the government, but as thinking, deliberating, and
autonomous human beings. This is despite the fact that although the spheres of
knowledge in which they are involved are defined by academic discipline and
prescribed didactic methods.
Nonetheless, a rational teacher is an intellectual who deals with the continuous

transformation of the pedagogical experience in a humanistic-liberal spirit,
according to valuable cultural assets. The intention is to sharpen the thinking of
teachers, and turn them into reflective and constructive human beings so that they
can translate theoretical courses into the practice of the cultural ecology in which
they live. The third model represents the teacher’s critical and reflective skills,
which place the learner on center stage. The idea is that the responsibility of
learning is passed on to the students, and knowledge acquisition, practice, and
meeting evaluation examinations. It thus cultivates the independence to reach
goals in ways that are suitable to their wishes. The core of this method is the
nurturing of dialogic pedagogy and emotional involvement with the student
during the learning process (Alammary, et al, 2014; Carlson, 2017).
Compared to previous methods, this method comes closer to liberal perceptions

and open, democratic, and enabling education. The approach creates an equal
process shared by teachers and students in the spirit of Freire (1997), according to
which teachers do not oblige students to accept their standpoints, nor do they use
their authority to impart their worldview. This type of teaching model envelops
the personality and characteristics of students which are honesty and authenticity,
motivation and responsibility, and the development of both introspective and
reflective awareness (Freire, 1997; Grollios, 2016). As mentioned earlier, these are
the three traditional models that serve as the foundation for developing learning
methods that fall in line with the changes in human ecology in the Western world.
Given the accelerated changes in the “real” world outside school, there was a need
to tighten and make more precise the rational, pragmatic, and practical connection
between content, values, and the shaping of the character of both students and
teachers, between their role as citizens, each in his or her area. The aim was to
leave the comfort zones of conceptual and ideological fixedness and cultivate
inquisitiveness toward learning and innovation, and digital skills instead of
rigidly preserving antiquated habits (Tamir, 2015; Doolittle, 2015; Ravitch, 2016).
Thus, while breaking down the barriers in the workplace which were between

8
classrooms, activity areas, and offices, concepts such as dialogue, openness,

collaboration, transparency, genericism, empathy, and interdisciplinary synergy,
began permeating pedagogical language. There was no doubt that the revolution
in the education system was apparent in the entire learning process (Carlson,
2017; Hannon & Peterson, 2017).
This research is based on a primarily qualitative study, and on a constructive

paradigm that makes it possible to examine different aspects in the Humanities
teachers' training department. Being the head of a department, eased to gather
data and follow a group of selected students following with the “field-based
theory,” a method that made it possible to gather information from individual
interviews of students during the process of their admission, training, and school
experience in both forms. Generic lessons on one hand and disciplinary lessons,
on the other. Let us consider a test case carried out at the Kibbutzim College of
Education, the largest academic institution for teacher training in Israel. The
dilemma concerns the lecturers and pedagogical counselors because of all that has
been stated so far, about the graduate interns. Firstly, there is a desire to preserve
a level of knowledge in the subject of specialization from all possible angles. Still,
then, there is a commitment to provide training that is suitable to the 21st century:
combined and online teaching, PBL skills, digital skills and “learning
experiences,” and Internet and cellular capabilities that replace conservative
frontal teaching. To investigate this conflict, individual interviews were
conducted in a chosen group of 48 first-year students.
4. Methods and results

As part of their studies, they had five practical hours a week in one of the high
schools in one of their subjects of specialization. They were supposed to study for
four years, and in their fourth year, begin a year of internship as specialized
teachers in high schools in two selected subjects in the humanities. It merits note
that this year students were trained according to the combined generic method,
i.e., one lecturer, an expert in one of the humanities. It was taught in pedagogical
seminars, with no specialized subject differentiation, with no differentiation
between the language of writers or men and women of religion, and that of the
historians. They expected to build the disciplinary lesson plans in history,
literature, or the Bible studies, under the curricula and demands of the schools in
which they worked. This approach stemmed from an interdisciplinary worldview
according to which it is essential that all students in teacher training experience
independent professional development and undertake responsibility for the
degree of success or failure of the class. They have to encourage individual
reflective skills to create “their own character” as teachers. This method makes it
imperative for future teachers to find supportive theoretical bodies of knowledge
on their own before they teach each chapter or topic in the classroom. It is crucial
when they unrelated the didactic and technological skills which they amassed in
generic training such as verbal analysis, content understanding, the
encouragement of dialogue and discussion, and discussion of test cases and their
relevance to reality.

9
At the end of their first year of practical work, arguments were raised about the
scope of professional readiness and their sense of security as teachers who will, in
the future, prepare students for the matriculation exams, or final papers in their
field of specialization. The main argument raised in numerous variations was
about their lack of scientific and disciplinary maturity beyond that which was
required according to the curriculum. For example, anyone planning to teach
literature in high school had to acquire broad intellectual, cultural, and artistic
education. They had to be conversant in numerous styles of writing, be exposed
to an enormous variety of writers, poets, and playwrights from different cultures
and periods, and receive pedagogical and scientific guidance in inter-textual and
provocative reading. The majority felt that their teaching was detached from an
in-depth foundation of knowledge, and this could not be achieved independently
in a way that would do justice to the profession.
Their answers show that most of them proved conversance in teaching and the
structural and digital changes in schools, in the PBL method, and in learning
outside the classroom. Also, it was clear to them that the skills of teaching and
learning had to fall in line with the synergetic and dynamic reality in which we
live, and therefore the role of the school in the life of the children was critical. This
state of affairs was manifested in the findings shown in Table 1: out of 48 subjects,
38 (approximately 80%) believe that generic learning diminishes comprehensive
theoretical and research expertise. Out of these 48 students, 43 (90%) believe that
combined generic training impairs the field of knowledge which they chose to
professionalize as specialized teachers. The majority argues that a significant
difference exists between training a specialized teacher and training a general
(homeroom) teacher. Teachers of the humanities, who are more ‘verbal,’ should
be given separate pedagogical guidance for each subject of their specialization.
Table 1: Students’ attitudes to generic training in the humanities, 2017

Specialized Generic Generic Sciences and The
teachers are training learning technology fit humanities
different from impairs the reduces generic demand
generic tutors specialized profound learning separate
of teams' teacher knowledge because they teaching due
projects and research are less verbal to multi-
verbal
contents
Yes 44 43 38 31 42
No 4 5 10 17 6

10
Yes
44 43 42
38
31
17
10
4 5 6
Specialized Generic training Generic learning Sciences and The humanities

teaches are impairs the reduces technology fit demand separate
different from specialized profound generic learning teaching due to
generic tutors of teacher knowledge and because they are multi-verbal
teams' projects research less verbal contents
Figure 1: Students’ attitudes to generic training in the humanities, 2017
Given the findings as mentioned above, a decision was taken to change the
process of the combined specialized pedagogical training of that group of
students. To this end, the generic method was eliminated in the department for
teaching the humanities in high schools. The academic staff undertook the task of
building separate didactic seminars for each subject of specialization, based on a
disciplinary division. For the three subjects, literature, history, and Bible studies,
six professional-pedagogical instructors were chosen for second and third-year
students; throughout the year they trained students in one field only. In this
framework, students were provided with theoretical and scientific bodies of
knowledge. They were exposed to research in their specific area and diverse
methods of teaching. The objective was to turn them into expert specialized
teachers who chose to specialize in this subject.
Over the two years of their training, the students continued to acquire experience
in different high schools, according to their academic planning and the demands
of their degree. When this period was concluded, the same 48 students were
interviewed again to discover the extent that this change contributed to their
success in the classrooms in comparison with the way, they felt in the generic
training framework. Findings left no room for doubt as shown in Table 2: over
90% reported a strong sense of security in teaching their specialized subject, and
their success in creating curiosity among their students. Over 80% claimed that
disciplinary guidance helped them decide which specialization they wanted to
choose in the future, and perhaps even continue toward attaining a master’s
degree. A similar percentage was found among those who claimed that the
generic system was professionally detrimental to their training, and to the subject
itself as a field of knowledge.

11
Table 2: Students’ attitudes to generic training in the humanities, 2020

There is a There is an Disciplinary Separate Separate
disadvantage advantage in training in professional professional
in generic generic teaching training training
learning as a learning as a improves the improves improves
basis for basis for students’ managerial the quality
disciplinary disciplinary achievements skills of teaching
learning learning
Yes 38 10 37 39 44
No 10 38 2 2 3
Not
0 0 9 7 1
sure
Yes No Not sure
44
38 38 37 39
10 10 9 7
2 2 1 3
0 0
There is a There is an Disciplinary Separate Separate

disadvantage in advantage in training in professional professional
generic learning generic learning teaching training training
as a basis for as a basis for improves the improves improves the
disciplinary disciplinary students’ managerial quality of
learning learning achievements skills teaching
Figure 2: Students’ attitudes to generic training in the humanities, 2020
These discoveries necessitate new thinking about pedagogical methods that train
teachers for future schools in the next generation. They also have implications on
preparing students to become citizens in the 21st century, with cognitive, mental,
and physical readiness to realize themselves and succeed in everything that life
has in store for them. Realizing this, it appears that it is essential to equip teachers
and students with as many methods and learning challenges as possible, without
discrediting one way in favor of another. The most obvious conclusion, voiced by
the majority of participants, relates to the importance of in-depth theoretical and
practical bodies of knowledge in their training as teachers who specialize in the
humanities. Reflective descriptions of what went on in their classrooms reveal that
students as well require more profound theoretical studies. This is counter to what
is usually believed concerning this generation - its lack of patience for in-depth
learning, reading, and writing.
The conclusions mentioned above, do not underestimate the value of generic,

synergetic learning and activity through projects and group assignments together
with means that are not solely theoretical. It is reasonable to assume that this is

12
only the beginning of a study that will expand to other areas of specialization and
other academic institutions. From all that has been stated in this article, it is crucial
to examine the balance, dosage, and assimilation of these methods in coordination
with the criteria of culture, location, social profile, time constraints, and
mandatory policies. However, the case study presented above, and the arguments
that have been raised from different and contradictory viewpoints, show that
these subjects demand caution and close professional reviewing, mainly in
teaching. It is imperative to focus on the overt and covert tension created between
the will of the individuals to promote their abilities, separate from their
commitment to invest efforts in promoting their group. Those who support PBL
and generic learning guarantee all students that they will express themselves and
contribute and will not be “devoured” by the group experience (Alammary et al.,
2014; Rosiek & Kinslow, 2016). Those who harbor doubts concerning this method
take care that the effort invested in learning means and methods, is greater than
the effort of having all students broaden and deepen their knowledge and
understanding, and encourage them to achieve self-fulfillment.
Another issue, that we should be aware of, is the collaborative and innovative
learning methods addressed to those who tend to be less prominent due to
personal and social inhibitions, or difficulties in expression. At the same time, one
should not ignore the fact that risk always exists that exceptional and gifted
students, who possess natural leadership skills, will do everything in their power
to curtail their natural characteristics. In their wisdom, they realize that this type
of learning and research method sanctifies partnership and the mutual
contribution of each one equally, and this, in turn, forces them to lower their
profile. From this derives the supreme importance of training future teachers and
enhancing their professional skills, so that they will possess the sensitivity and
education needed to detect these difficulties and know how to resolve them vis-
à-vis every student, both separately and as part of the group. These are the future
teachers who are supposed to become specialists through innovative approaches,
intending to be able to implement them in the schools where they will conduct
their practical work and later permanent work. Alongside the understanding that
to be a specialized teacher it is imperative to deeply study the area of
specialization, and continue to do so in the years to come, these teachers are aware
of the fact, that in the schools in which they will work – the staff thinks differently.
The study set-ups include more multidisciplinary projects, learning ‘outside the
classroom’ and online learning, and a free choice of classes and evaluation
methods.
It is obvious that the required hours for a degree in education diploma should
contain more practical work in digital pedagogy, with all the media means and
their incorporation into generic learning. However, this type of teaching
eliminates the uniqueness of the discipline and directs them to become “service
providers” and project managers in the classroom. It is reasonable to assume that
a student who wishes to specialize only in history and becomes a history teacher,
will find it harder to survive in his/her work at a school that changes its physical
and systemic structure (Gamoran & An, 2016). On the other hand, a teacher who
was granted accreditation to teach two or three subjects such as history and

13
literature through generic training will find it easier to adjust in this type of
innovative school. The interviews' answers over three years of training and
practical work, exposed more and more interesting facts regarding professional
and mental difficulties. Their training process included all the advanced
educational models, to introduce them to practical perception and reinforce their
steps along an axis built between the academic world and its demands for a
degree, and the school that adheres to the constraints of a policy determined each
time anew. Thus, the question arises again: why should they engage in
overqualified studies of their subject of specialization, as if they were medical
students? Why do they need academic degrees, cultivation of research skills, and
writing articles about professional and practical training, when in fact their status,
and presence as “specialized teachers” is diminishing in the classroom and
public? (Nir et al., 2016; Hannon & Peterson, 2017).
5. Conclusions
In modern classrooms, students participate in more active learning and are highly
motivated by working in project-teams and subject-groups. By developing
partnerships in understanding and analyzing failure, they are more likely to
retain knowledge. This pedagogical approach accords them the freedom to learn
in their way, and to solve common problems by carrying on open-minded debates
and brainstorming. Another important outcome of this article is that collaborative
learning redefines traditional student-teacher relationships, in and outside the
classroom (Paniagua & Istance, 2018; Sahlberg, 2018). It is imperative to examine
how demanding disciplinary learning (like in the past), more hours of reading
and practicing and fewer hours of recreation and screen games, devoid of group
background noise, and enhance the drive to explore and excel. This is essential to
enable maximal concentration in reviewing and enriching memory reservoirs and
the ability for greater and more complex analytical, cognitive, and mental
understanding.
The findings which are presented here, prove that the gap between theory and
practice, which derives from the complexity created by generic perceptions,
increases. These issues became critical for all graduators at the Israeli Colleges of
Education and are been discussed in teams of experts in the educational systems.
Although it is still early to arrive at a definite conclusion about the data of
continuous change in generic teaching, it performs in any pedagogical discourse
with more extensively in recent years. This is mainly due to the socio-cultural state
of affairs and because of geopolitical events that define the reality of the
background of all educational and disciplinary processes, and mostly, the face of
the next generation. The discussion around these issues needs more researches for
accomplishment to make better decisions in teachers' training, especially in the
Humanities studies.

14
6. References
Ainscow, M. (2005). Developing inclusive education systems: What are the levers for
change? Journal of Educational Change, 6(2), 109–124.
https://doi.org/10.1007/s10833-005-1298-4
Alammary, A., Sheard, J., & Carbone, A. (2014). Blended Learning in Higher Education:
Three Different Design Approaches. Australasian Journal of Educational
Technology, 30(4), 450 – 440. https://doi.org/10.14742/ajet.693
Ashton, P. T. (1996). Improving the Preparation of Teachers. Educational Researcher, 25(9),
21–35. https://doi.org/10.3102/0013189X025009021
Bauder, J., & Rod, C. (2016). Crossing the thresholds: Critical information literacy
pedagogy and the ACRL framework. College & Undergraduate Libraries, 23(3),
252–264. https://doi.org/10.1080/10691316.2015.1025323
Brophy, J. (2006). Observational Research on Generic Aspects of Classroom Teaching in
P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (pp.
755–780). New Jersey: Lawrence Erlbaum Associates Publishers.
https://doi.org/10.4324/9780203874790.ch33
Bergman, M. (2018). A Knowledge Representation Practionary - Guidelines Based on Charles
Sanders Peirce. New York: Springer International Publisher.
https://doi.org/10.1007/978-3-319-98092-8
Carlson, D. (2017). Teachers in Crisis: Urban School Problem and Teachers' Work Culture.
New York: Routledge.
Christianakis, M. (2010). Collaborative Research and Teacher Education. Issues in Teacher
Education, 19(2), 109-125.
Calarco, J. (2019). Social Class and Student-Teacher Interactions in T. Domina, B. G. Gibbs,
L. Nunn & A. Penner (Eds.). Education and Society – An Introduction to Key Issues
in the Sociology of Education, (pp. 96 – 110). California: University of California
Press. https://doi.org/10.2307/j.ctvpb3wn0.11
Doolittle, A. (2015). The best of Many Worlds: Methodological Pluralism in Political
Ecology. In L. B. Raymond (Ed.) The International Handbook of Political Ecology,
(pp. 515 – 529). Northampton: Edward Elgar Publications.
https://doi.org/10.4337/9780857936172.00047
Freire, P. (1997). Teachers as Cultural Workers – Letters to Those Who Dare Teach. New York:
Routledge, Westview Press. https://doi.org/10.4324/9780429496974
Fullan, M. (2016). The New Meaning of Educational Change. New York: Routledge and
Teachers College Press. https://doi.org/10.1007/1-4020-4451-8_12
Gamoran, A., & An, B. P. (2016). Effects of school segregation and school resources in a
changing policy context. Educational Evaluation and Policy Analysis, 38(1), 43–
64. https://doi.org/10.3102/0162373715585604
Goleman, D. (2006). Social intelligence: The new science of human relationships. New York:
Bantam Dell.
Goodman, A., Joshi, H., Nasim, B., & Tyler, C. (2015). Social and emotional skills in
childhood and their long-term effects on adult life. Washington: Institute of
Education.
Griffin, P., & Care E. (Eds.). (2014). Assessment and teaching of 21st century skills: Methods
and approach. New York: Springer.
Grollios, G. (2016). Paulo Freire. New York: Routledge.
Hannon, V., & Peterson, A. (2017). Thrive: Schools Reinvented for the real challenges we face.
London: Innovation Unit Press.
Hattie, J. (2015). What Doesn’t Work in Education: The Politics of Distraction. London:
Pearson.

15
Hattie, J. (2016). The applicability of Visible Learning to higher education. Scholarship of

Teaching and Learning in Psychology, 1(1),79–91.
https://doi.org/10.1037/stl0000021
Nir, A., Ben David, A., Bogler, R., Inbar, D., & Zohar, A. (2016). School autonomy and
21st-century skills in the Israeli educational system: Discrepancies between the
declarative and operational levels. International journal of educational management,
30(7), 1231-1246. https://doi.org/10.1108/IJEM-11-2015-0149
Paniagua, A., & Istance. D. (2018). Teachers and designers of learning environments: The
importance of innovative pedagogies. Paris: Center for Educational Research and
Innovation, OECD. https://doi.org/10.1787/9789264085374-en
Plucker, J., Clint K., & Dilley., A, (2019). What we know about collaboration" In E.
Eisenberg & O. Selivansky-Eden (Eds.). Adapting Israel's Education System for The
Challenges of the 21st Century, (pp. 31 – 42). Jerusalem: The Israel Democracy
Institute.
Ravitch, D. (2016). The Death and Life of the Great American School System: How Testing and
Choice Are Undermining Education. New York: Basic Books.
Rosiek, J., & Kinslow, K. (2016). Resegregation as Curriculum: The meaning of the new
segregation in U.S. public schools. New York: Routledge.
Rothstein, R. (2017). The color of law: A forgotten history of how our government segregated
America. New York: W. W. Norton.
Sahlberg, P. (2015). Finnish lessons. What can the world learn from educational change in
Finland? New York: Teachers College Press. https://doi.org/10.17323/1814-
9545-2014-4-260-268
Sahlberg, P. (2018). Finnish ED Leadership: Four Big, Inexpensive Ideas to Transform
Education. Carolina: Corwin.
Schofer, E. (2019). The Growth of Schooling in Global Perspective. Education and Society –
An Introduction to Key Issues in the Sociology of Education, 8–22.
https://doi.org/10.2307/j.ctvpb3wn0.5
Tamir, Y. (2015). Who's Afraid of Equality? Education and Society in Israel. Tel Aviv: Yedioth
Ahronoth and Chemed Books. [Hebrew]
Virtanen, A., & Tynjälä P. (2019). Factors Explaining the Learning of Generic Skills: A
Study of University Students’ Experiences" Teaching in Higher Education, 24 (7),
880-894. https://doi.org/10.1080/13562517.2018.1515195

16

Vol. 19, No. 8, pp. 16-39, August 2020
The Effects of Mobile Learning on Listening

Comprehension Skills and Attitudes of Omani
EFL Adult Learners
Abdullah Al-Shamsi, Abdo Mohamed Al-Mekhlafi*,

Saleh Al Busaidi and Maher Mohammad Hilal
Sultan Qaboos University, Muscat, Sultanate of Oman
https://orcid.org/0000-0002-5627-2609
https://orcid.org/0000-0003-2821-6199
https://orcid.org/0000-0002-9649-429X
https://orcid.org/0000-0002-7026-498X
Abstract. This study aimed to explore the effect of using mobile learning
on improving adult learners’ listening skills in Oman, to investigate their
attitudes, and to explore the factors that stand as barriers to its
implementation. The study is quasi-experimental consisted of two
groups, an experimental group (n=15) and a control group (n=16) from a
foundation program at a military educational institute. The research data
included the results of two sets of listening tests and learners’ responses
on an attitude questionnaire. The students in the experimental group
outperformed their counterparts in the control group as a result of the
mobile learning strategy. There was a statically significant improvement
in the experimental group students’ listening ability. Also, the
participants had positive attitudes towards using mobile learning in
improving their listening comprehension skills. The participants found
that mobile learning enhanced their motivation, increased their exposure,
expanded their vocabulary repertoire, and provided easy access to
“anytime” and “everywhere” learning. However, they emphasized some
challenges that were related to mobile software design, screen sizes of
mobile phones, network connections, and the appropriateness of the
listening content. Based on the findings, the study suggested some
educational implications and recommendations.
Keywords: Mobile learning; listening comprehension skill; attitudes;

language exposure; autonomous
1. Introduction
Mobile devices are the next generation of learning as they are extending into all
areas of human life (Kim, 2013). Mobile learning is providing us with
*Corresponding author: Abdo Mohamed Al-Mekhlafi; Email: raymoh123321@gmail.com
©2020 The authors and IJLTER.ORG. All rights reserved.

17
opportunities to change the existing learning methods and strategies and gives a
more flexible approach to manage learning experiences on the move (Kukulska-
Hulme & Traxler, 2005). Mobile learning technologies “help produce learning that
is personally customized, socially constructed, and which extends beyond the
classroom” (Holden & Sykes, 2011, p. 4). Several empirical studies assert that
mobile-learning is a useful and instructive tool for language learning, and it can
encourage students to be more autonomous and independent in their learning
process (Chen, 2016; Liu & He, 2015). Chen (2016) emphasized that “Mobile
learning apps provide multiple channels and modalities for adult learners to
practice language skills” (2016, p. 40).
Regarding language learning, researches in the literature have shown that

listening comprehension is crucial for second language acquisition (Feyten, 1991;
Richards, 2005). Listening ability is an essential factor that contributes
significantly to the second language learning process. Richards (2005) stated that
“The development of good listening skills is seen not only as something valuable
for its own sake but as something that supports the growth of other aspects of
language use, such as speaking and reading” (p. 85). However, many language
learners and teachers consider listening as the most difficult skill to be taught
(Aryana & Apsari, 2018).
In Oman, several studies assessed the students' listening comprehension skills.

They found out that Omani students have difficulty in listening comprehension
due to incompetent treatment of listening comprehension and insufficient
exposure to listening outside the classroom (Al-Busaidi, 1997). Also, the listening
materials and conventional teaching methods are of poor quality (Al-Belushi,
1999). Moreover, learners are unable to follow listening materials in a stressful
environment because of the cognitively demanding listening activities (Al-Issa,
2005). Likewise, Al-Handhali (2009) claimed that content issues, lack of exposure,
lack of encouragement, and teachers’ methodological decisions in classrooms all
contributed to listening comprehension difficulties. Therefore, the problem of this
current study lies in the weak performance of many Omani students’ in listening
comprehension and their insufficiency of exposure to the English language
outside classrooms.
Therefore, listening skills must be investigated further in Oman. New approaches

in teaching are needed to be adopted, and modern technologies are required to be
exploited and utilized. Al-Harrasi (2014) recommended that a less-stressful
environment is necessary for the classroom, and learners need more interactive
listening activities. Al-Belushi (1999) also recommended utilizing the latest
technologies that are made available for language learning. He urged teachers to
encourage students’ autonomy and independence in their learning process and to
give opportunities for individual students to listen to what interests them and to
listen in their own time and place. Thus, a shift towards integrating educational
technologies is required to give learners some opportunities to practise listening
comprehension skills outside the classroom independently. Therefore, to improve
the learners’ listening skills, it is recommended to increase time exposure to the
language by providing different listening materials for students to listen to in their

18
free time outside the classroom. Also, it’s essential to utilize advanced
technologies, online listening materials and to provide less stressful learning
environments (Al-Belushi, 1999; Al-Busaidi, 1997; Al-Handhali, 2009; Al-Issa,
2005).
Concerning utilizing advanced technologies, several studies have reported the

potential use of mobile-based technology in enhancing language learning and
accordingly have required EFL teachers to use mobile learning to perform
language learning activities (Al Aamri, 2011; Al Yafei & Osman, 2016; Beatty, 2010;
Chen, 2016; Kim, 2013). Crompton and Burke (2018) urged higher education
teachers to use mobile technologies to increase learning opportunities outside
classrooms. Mobile devices can allow language learners to access different
learning materials everywhere, flexibly, and at any time (Kim, 2013; Read &
Kukulska-Hulme, 2015). Also, they help to overcome many problems such as
anxiety of language learning, inadequate language practices, and deficiency of
language exposure (Rahimi & Soleymani, 2015). Furthermore, students can
develop self-regulation and self-assessment through mobile devices
(Gangaiamaran & Pasupathi, 2017; Liu & He, 2015; Zheng & Chen, 2018). Besides,
some mobile learning applications can provide opportunities for immediate
feedback and language analysis (Chen, Hsu & Doong, 2016). Mobile learning can
establish an educational electronic learning platform that offers motivating
educational experiences for instructors and students. It can also enhance the
learners’ self-regulated learning experiences and increase language exposure
outside the classroom.
Therefore, the importance of promoting listening comprehension skills and the

great opportunities that mobile-learning can positively offer has led to the need
to investigate this issue further in Oman. The primary purpose of this study, thus,
is to explore the impact of mobile-learning on improving listening comprehension
skills and explore the pedagogical attitudes of students towards the integration of
the mobile-learning in their classroom activities. The study addresses the
following research questions:
1. Are there any statistically significant differences in listening
performance between students who learn listening skills through
mobile devices and students who conventionally learn listening skills?
2. What are the students’ attitudes towards using mobile devices in
improving their listening comprehension skills?
3. What challenges do students face in using mobile devices?
2. Literature Review
The growing popularity of the term mobile learning among language learners and
the vitality of listening skill in language acquisition bring with them a shift in
focus that may impact the teaching and learning process. This dramatic shift
towards using m-learning in teaching English language listening skills can
provide access to listening materials from everywhere and at anytime. The
literature review provides a theoretical background of English listening
comprehension skills and information background about mobile learning and its
impacts on EFL contexts.

19
2.1 Listening Comprehension Skill in L2 Learning

Listening comprehension skill plays an active part in L2 learning (Brown, 2001;
Feyten, 1991; Richards, 2005; Vandergrift & Goh, 2012). Listening is an essential
skill in a way that exists in most of the activities we do throughout our lives (Al-
Belushi, 1999). A person cannot communicate or interact with others unless s/he
understands the spoken language (Rivers, 1966). Therefore, the rationale behind
teaching listening skills is to prepare English language learners for understanding
the actual speech in real-life communication contexts and for facilitating second
language acquisition. Besides, listening ability contributes significantly to the
predictability of foreign language acquisition process (Feyten, 1991) and creates a
channel by which the learner gains access to a great deal of comprehensible input
in the target language (Krashen, 2013; Rost, 2007). Furthermore, the development
of listening comprehension plays a significant role in developing other language
skills (Dunkel, 1986), expands the learners’ vocabulary repertoire and grammar
knowledge (Rost, 1994), and improves learners’ pronunciation of the target
language (Harmer, 2007). Listening plays an active part in the language learning
process, and language learners cannot maintain acquisition until a certain amount
of the listening input in the target language is intelligible.
2.1.1 Learners’ Problems in Listening Comprehension

Although listening plays a significant and constructive role in language learning,
language learners confront several difficulties and problems when practising
listening comprehension skills. Kim (2013) stressed that many English learners
find listening skill a challenging skill as it demands a complex process of
interpreting information from sound, especially when there are no visual aids.
Also, students need to comprehend and process both content knowledge (data)
and linguistic knowledge (language) simultaneously while they are doing the
listening. According to Namaziandost, Ahmadi and Keshmirshekan (2019),
limitations on learner’s listening ability are due to the listener’s limited
vocabulary, length of the discourse, inability to understand the speaker’s accent,
and the speaker’s speech rate. In his article, Goh (2000) notified that students tend
to forget what they hear quickly, are unable to recognize the meanings of words,
and tend to face difficulty to comprehend the intended purpose of the message
even though they had understood the literal meaning of the words. Thus, learners’
problems in listening comprehension are due to personal cognitive differences,
individual emotional statuses such as learners’ anxiety, and the context of the
spoken language (Vandergrift & Goh, 2012).
2.1.2 Contemporary Trends in Teaching Listening Comprehension

The difficulties that EFL learners face when listening to the target language and
the value of mastering listening skills for language acquisition emphasize the
importance of seeking new strategies and techniques to facilitate listening
comprehension skills. Teachers should help students improve their listening
comprehension proficiency by reducing students’ concern about listening and
providing a less worrying classroom environment (Al-Handhali, 2009). Also,
students should be encouraged to promote self-regulated learning to seek
listening opportunities outside the classroom (Gangaiamaran & Pasupathi, 2017;
Yabukoshi, 2018; Vandergrift & Goh, 2012). Moreover, there has been an

20
increasing call on employing more authentic materials in the learning process

(Vandergrift, 2007), and on providing more extensive listening exposure to the
target language outside the classroom (Lee & Cha, 2017).
Based on the above discussion, integration of mobile learning can enable students
to reduce their anxiety, increase their language exposure, enhance independent
learning, and develop some learning strategies so that learners are motivated to
seek more opportunities outside the classroom (Al Aamri, 2011; Al Yafei &
Osman, 2016; Chen, 2016; Kim, 2013; Kukulska-Hulme & Traxler, 2005; Liu & He,
2015; Rahimi & Soleymani, 2015). Vandergrift (2007) emphasized that “Broader
access to these new technologies will likely shift the focus from the classroom to
independent learning” (p. 206). Mobile devices can be utilized outside the
classroom to enhance the learning process and create a more self-regulated
learning environment (Zheng & Chen, 2018). Besides, mobile-based learning can
be a convenient tool in providing immediate feedback and assessment for
students so that they track their self-improvement in the language (Chen, Hsu &
Doong, 2016). Therefore, this study intends to examine the effect of using mobile
devices in improving listening comprehension skills.
To sum up, the previous studies in the literature show that listening
comprehension is a complex process which requires much listening exposure and
extensive practice outside the classroom. Therefore, teachers need to inspire
learners to become independent, to look for listening opportunities outside the
classroom through using English language media, to establish goals and means of
self-evaluation, and to keep a record of their performance. The use of mobile
learning can facilitate a shift from teacher-led education to student-led one, so that
students can listen to the language anytime and anywhere and be more self-
independent.
2.2 Mobile Learning (M-Learning)

Many scholars and practitioners have described mobile learning in different forms
(Grant, 2019). Brown (2005) defined M-learning as being a subset of E-learning that
is explicitly a form of web-based delivery of content and learning management;
moreover, it features with mobility, flexibility, and convenience when compared
to online learning. However, Peters (2007) went further in defining m-Learning as
being a model of flexible learning that is ‘just in time, just enough and just for me’.
Other scholars described mobile learning as a sort of portable education that uses
portable devices to access learning and knowledge on the move without the time
and location constraints (Kukulska-Hulme & Pettit, 2009; Traxler, 2009). El-
Hussein and Cronje (2010) attempted to provide a comprehensive definition of
mobile learning that is “any type of learning that takes place in learning
environments and spaces that take account of the mobility of technology, mobility
of learners, and mobility of learning” (p. 20).
2.2.1 Significance of Mobile Technology in EFL Context

M-learning recognizes learners’ diversity and individual differences to determine
the way of learning (Traxler, 2009). It has the potentials to provide authentic
listening materials such as songs and news in English for language learners

21
(Elfiona, Embryany & Pamela, 2019). Authentic materials can help the language
learners see how the language is related to the real word (Brown, 2001), enhances
their active involvement in the learning process (Hapsari & Ratri, 2014), and
fosters their self-confidence (Unver, 2017). Furthermore, mobile technology can
contextualize the learning activities, and it can combine between formal and
informal learning opportunities (Pulla, 2020). Thus, learning won’t only be limited
to the classroom setting but also will be extended to learning in real-life
communities.
Moreover, mobile technology gives students the flexibility and motivation to learn
at their own pace at a convenient time. It promotes autonomy and makes the
learning process more independent (Al-Hunaiyyan, Alhajri & Al-Sharhan, 2018;
Beatty, 2010; Kim, 2013; Read & Kukulska-Hulme, 2015; Traxler, 2009). It also
involves learners in determining their objectives, developing more useful learning
strategies, and determining the method and timing that they decide on (Raya &
Fernández, 2002). Besides, it can provide a variety of English expressions and
vocabulary (Kim, 2013). Hence, mobile learning can fit different learning styles,
directs learners to control their education, contextualize the learning experiences,
and frees learners from the formality of conventional education.
2.2.2 Challenges
Although mobile learning has proved to have significant merits in the learning
process, it cannot stand without some drawbacks. Rogers and Price (2009)
mentioned that overloaded information, distractions by mobile devices, and
difficulty in designing appropriate learning experiences that encourages
collaboration and interaction between learners are the three main challenges that
may occur when employing mobile technologies. Therefore, language teachers
must create mobile learning experiences which are not too bewildering or overly
complicated and make sure that learners are not working in isolation from their
counterparts. Likewise, Zhang (2019) found that a lack of internet access, a lack of
continuity of mobile data transfer, weak cellular signals in some areas can hinder
a real continuous learning experience on mobile devices. Likewise, Alrefaai (2019)
found that EFL learners face various challenges when they use mobile devices
such as technical problems, small screen sizes, distractions, the accuracy of the
information, health problems, and getting bored. In Oman, there is a shortage of
technological aids or a failure of some teachers to utilize them due to their lack of
knowledge or training on how to use educational technology (Al-Issa & Al-
Bulushi, 2012; Al-Musawi, 2007; Al-Senaidi, Lin & Poirot, 2009).
2.3 Emerging Mobile Technologies in the Omani context

Several studies in Oman examined the teachers’ and students’ attitudes towards
mobile-based learning. Al-Emran and Shaalan (2017) revealed that M-learning
could be adopted by all academics regardless of their age and qualifications.
However, instructors’ attitudes towards mobile technology are determined by
their beliefs about the effectiveness of mobile technology in education. The more
positive perception they have towards mobile technology, the more optimistic
they are towards the utilization of M-Learning. Likewise, Al Aamri (2011) found
that students like to use mobile devices while teachers do not want them to do so.

22
Teachers think that mobile devices could be a big distraction for both teachers and
students. Therefore, the researcher recommended fostering the use of mobile
technology in education and emphasizing the merits of mobile phone in the
classroom. In this respect, Al Yafei and Osman (2016) noted that m-learning could
be an effective medium for self-learning as it promotes autonomy and increases
learners’ motivation which helps in solving many motivational barriers that might
occur under fixed and even monotonous educational routines. Both learners and
language educators hold positive attitudes towards integrating mobile
technologies in the Omani context. However, there should be more studies
investigating the merits of mobile devices in the classroom to gain confidence in
using it.
In summation, the existing body of research shows that the use of mobile learning
in language learning, especially in listening comprehension, is an essential
contributor to second language learning. It increases the level of self-awareness
and ability of learners and decreases the level of anxiety. Moreover, the literature
provides conclusive empirical studies supporting the idea of using mobile
learning that helps language learners undertake the listening activities in a
scaffolded way and offers possibilities for interaction and collaboration. Thus, as
the assertion that M-learning can enhance listening comprehension skills for L2
learners is assumed and demonstrated empirically, supporting the use of M-
learning as a means for increasing listening ability can encourage the appearance
of this research in the future. Their unique features like portability, individuality,
and connectivity make mobile based-instruction an integrative, an interactive,
and innovative experience. The primary of the present study, therefore, is to
investigate the effects of M-learning on the development of L2 learners’ listening
ability.
3. Methods
This section discusses the research methodology and the procedures of designing
and applying the research instruments including the statistical analysis that were
adopted in analyzing and interpreting the results of the instruments, including a
description of the participants, the research design, data collection, and data
analysis.
3.1 Participants
The participants of the study were from the foundation program at a Military
Educational Institute (MEI), Oman. They were about 48 students enrolled in level
one. Two intact classes comprised the sample of the study, one as a control group
(n =16 students) and the other one as an experimental group (n=15 students). All
participants were full-time students registered for a 14-week course. They ranged
in age from 20 to 23. All participants had a similar educational background and
the same learning environment. The comprehension listening pre-test was
administered to both groups before the intervention to determine the equivalency
of the two groups in the English listening comprehension skill. The researcher
conducted an independent-samples t-test to compare the mean scores between
the two groups. Table 1 presents the means and standard deviations of the
students' scores on the listening pre-test before receiving the intervention.

23
Table 1. Independent Samples T-test Results for Pre-test before the intervention
Groups n Mean* SD df t-value p-value
Control group 16 16.19 4.215 29 -0.845 0.405
Experimental group 15 17.47 4.207
*Total score=25
The results showed that there was no significant difference in mean pre-test scores
between the control group (M = 16.19, SD = 4.215) and experimental group (M =
17.47, SD = 4.207) before the intervention, (t (29) = -0.845, p>.05). The p-value is
0.405 (p>.05), which advocates that there is no significant difference between the
two groups. These results indicate that the level of listening ability of the two
groups was equivalent at the start of the intervention.
3.2 Research Design

This study is a quasi-experimental research design in which the researcher used a
pre-test and a post-test to determine the effect of mobile learning on students’
listening comprehension skills. The research followed this sort of design as there
is no control of the random assignments of the subjects to the treatment group.
Fraenkel, Wallen and Hyun (2011) affirmed that a quasi-experimental design is an
experimental design in which the researcher cannot assign individual participants
to groups randomly. Based on this, the researcher selected the experimental group
and control group without randomization. The control group students received
the English listening materials lessons following a conventional way of teaching.
In contrast, the students in the experimental group worked with the same
listening materials through mobile devices using the mobile application (Google
Classroom). After the experiment, the researcher compared the performance of
both groups to gauge the effect of the mobile-based learning treatment on the
experimental group.
3.3 Research Instruments

To gather data, the researcher has administered a comprehension listening test
and an attitude questionnaire. A brief explanation of each comes below.
3.3.1 Listening Comprehension Test

The comprehension listening test was developed based on the objectives of the
English language program course and aims to gauge the effectiveness of using
mobile-based learning materials on the learners’ level of listening proficiency. The
test was administered twice: as a pre-test before the intervention to determine the
equivalence of the participants. Also, it was used as a post-test for both groups at
the end of the treatment to measure the effect of using mobile learning on
students’ listening ability.
A panel of the Head of English section, four EFL teachers who were teaching in
the foundation program, three evaluation experts from the Exam Cell in the
institute, and an external examiner specialist validated the content of the test.

24
There was an agreement among the panel about the suitability, clarity, and
relevancy of the test scoring.
The listening test was pilot-tested by the Exam Cell on a group of 73 level-one
students to establish its reliability. According to the results, the Cronbach alpha
showed that the listening test reached the right level of internal consistency at
about 0.87. Therefore, the researcher is confident that this test was reliable for data
collection.
3.3.2 The Questionnaire

The questionnaire survey was developed to assess the learners’ attitudes towards
using mobile learning for improving their listening skills. The researcher designed
the questionnaire based on the relevant literature and previous studies (Al Aamri,
2011; Al-Hunaiyyan et al., 2018; Al Yafei & Osman, 2016; Kim, 2013). The
questionnaire consisted of two main sections. In the first section, there were 20
statements scored on a five-point Likert scale (1= strongly disagree; 2= disagree;
3= neutral; 4= agree; and 5= strongly agree). In this section, the participants
indicate the level of their agreement and disagreement with the statements
regarding their attitudes towards using mobile devices in improving their English
listening skills. There are four main dimensions in this section: perceived
usefulness, motivation, self-management of learning, and intention to use.
In section two of the questionnaire, there were open-ended questions, including

what the participants liked most about using mobile devices in learning English
listening skills, the difficulties they faced, and other suggestions to improve the
implementation of m-learning. Appendix 1 shows the sections of the
questionnaire.
The initial version of the questionnaire was reviewed and checked by twelve
experts in the ELT and instructional technology field. The jury assessed the
validity of the survey in terms of its relevance, clarity, and suitability. Based on
their recommendations, the researcher made some modifications and changes
accordingly.
The questionnaire later was pilot-tested on a sample of 30 students to check its

reliability. Reliability analysis was calculated using Cronbach reliability
coefficient; the coefficient was (α = 0.893) to the statements of the questionnaire.
Thus, an alpha of 0.893 is an appropriate reliability coefficient as the statements
of the survey reached the right level of internal consistency.
3.4 Description of the Materials

The study used the materials of the course textbook. The listening materials in the
handbook are adapted from authentic sources to stimulate the learners’ interests
and engage them in classroom discussions. A wide variety of recoding contents—
including lectures, radio interviews, news reports, and informal conversations—
are utilized to provide opportunities for extensive and intensive listening
practices. The audio files are on a CD ROM that comes with the textbook.

25
Regarding the mobile-based listening materials, the researcher converted the

original content of the course textbook into an electronic version. They
emphasized the same course objectives and followed the same sequence of the
teaching lessons. The mobile-based materials were assessed and validated by the
course coordinator and the course teachers. They checked the validity of the
materials in terms of its relevancy, clarity, functionality, and presentation in the
mobile App.
3.5 Procedures and Implementation

This study aimed to investigate the students’ academic achievement in listening
skills and their attitudes towards using m-learning. Therefore, the researcher
divided participants of the research into two groups, a control group (used the
conventional method) and an experimental group (followed mobile-based
learning). The researcher firstly administered the listening pre-test for both
groups, which showed that there were no significant differences between the two
groups before the experiment.
The researcher conducted a tutorial for the experimental group to explain the plan
of the study and to practise on how to use the mobile application (Google
Classroom). Also, the researcher explained the instruments to the participants,
and consent forms were signed, too. The students in both groups were exposed to
the same listening materials, exercises and assignments for eight weeks. The
control group followed the usual teaching method of a paper and pencil, while
the experimental groups used the Google Classroom App.
In the last phase of the study, the post-test was administered to both groups to
determine the impact of the listening-oriented mobile learning materials on
students’ listening comprehension ability. Then, the students in the experimental
group completed the attitude questionnaire and reflected on the use of mobile
learning strategy.
3.6 Data Analysis

The researcher used the SPSS program (version 25) to analyze the listening
comprehension test scores and questionnaire data. Descriptive statistics,
including means and standard deviations, were computed for both instruments.
An independent sample t-test was conducted before and after the intervention to
compare the scores of both groups. The researcher also carried out a paired
sample t-test to see if the students in the experimental group made significant
improvements in listening proficiency after using the mobile App. Finally, to
investigate the students’ attitudes towards the mobile learning strategy in
learning English listening and the difficulties that they encountered, the
participants’ responses to the questionnaire were tabulated and interpreted.
4. Results
The study was based on a quasi-experimental design in which two groups are
involved with one group receiving the treatment. The results obtained from the
research instruments were analyzed and presented. Tables were used to present
and describe the data, and analysis and interpretations were followed.

26
4.1 The Effect of M-learning on Listening Comprehension Skill

To answer the first research question, the researcher administered a post-listening
test to both groups and used an independent sample t-test to compare the scores
of both groups. Table 2 presents the results of independent samples t-test of the
post-test after the intervention by groups.
Table 2. Results of Independent Samples T-test for Post-test after the intervention
Groups n Mean* SD t-value df p-value
Control group 16 17.13 3.74 -2.57 29 0.016
Experimental group 15 20.20 2.83
*Total score= 25
The results show a clear significant difference between the mean score of the
experimental group (M=20.20) and the control group (M=17.13). It resulted in a
statistically significant difference between the groups (t= -2.093, p<0.05) and in
favour of the experimental group. Thus, using m-learning was more effective than
the conventional method in improving the learners’ comprehension listening skill.
The eta squared (2 = 0.19) indicated a large effect size according to the guidelines
proposed by Cohen (1988) for interpreting this value: 0.01=small effect,
0.06=moderate effect, and 0.14=large effect. In other words, 19% of the variations
in the post-test scores were explained by mobile-based learning practices, which
means that mobile learning treatment was effective.
To further investigate the impact of m-learning on the experimental group, the

researcher also used a paired sample t-test. Table 3 summarizes the results of the
paired samples t-test in both tests for the experimental group.
Table 3. Results of paired Samples T-test

Groups n Test Mean* SD t-value Df p-value
Experimental group 15 Pre- 17.47 4.21 -3.54 14 0.003
15 Post- 20.20 2.83
*Total score=25
As shown in Table 3, the test results of the experimental group revealed a

significant improvement in the post-test (M=20.20, SD=2.83) over the pre-test
(M=17.47, SD=4.21). The results demonstrated that the mean scores were higher
for post-test after the intervention at a significant level (t(14)=- 3.54, p <0.05). The
results of the eta squared (2 = 0.47) also indicated a large effect size, according to
Cohen’s (1988) three levels for interpreting this value. In other words, 47% of the
variations in the post-test scores were explained by mobile-based learning
practices, which also means that mobile learning treatment positively affected the
learners’ listening ability.
4.2. The Attitude of the Participants

The data of the questionnaire were analyzed and addressed in four dimensions to
answer the second research question. The dimensions are as follows: perceived

27
usefulness, motivation, self-management of learning, and intention to use. The

respondents have shown different estimates of the statements of the
questionnaire. Table 4 presents the overall mean of the survey.
Table 4. The Dimensions of the Questionnaire

Dimension Mean SD
1. Dimension of Perceived Usefulness 4.32 0.42
2. Dimension of Motivation 4.15 0.37
3. Dimension of Self-Management of Learning 4.05 0.42
4. Dimension of Intention to Use 4.15 0.35
Overall Mean / Std. Deviation 4.17 0.39
As shown in Table 4, the participants generally tended to have a positive attitude

towards using mobile learning for teaching English listening skills (M = 4.17, SD
= 0.39). Thus, the results showed that the majority of participants had positive
attitudes towards emerging mobile learning in the learning process as a useful
tool for improving listening comprehension skills.
Each dimension of the questionnaire is further analyzed. Table 5 shows the

students perceptions of the usefulness of mobile learning.
Table 5. The dimension of Perceived Usefulness

Statement Mean SD
1. Mobile learning provided more extensive listening practice. 4.33 0.62
2. Listening practice through the mobile device improved my listening 4.47 0.52
ability.
4. I listen to audio materials using my mobile device more than once. 4.00 0.93
12. Listening practice through mobile devices helped me learn a variety 4.47 0.74
of English vocabulary.
The results showed that the students generally had a positive perception of the
usefulness of using mobile learning in learning the listening skill (M=4.32). The
participants in the experimental group think that mobile learning was useful in
improving their listening ability as mobile devices have successfully increased
their exposure to the target language and have expanded their vocabulary
repertoire.
Table 6 shows the students’ responses to the statements that tackled the
motivation dimension towards mobile learning.

28
Table 6. Dimension of Motivation

Statement Mean SD
5. Using mobile devices motivated me to practise the listening skill. 4.27 0.59
6. The mobile device reduced my anxiety in learning listening skill. 3.87 0.99
7. I enjoyed the exercises through my mobile device than the traditional 4.20 0.78
way.
17. I prefer mobile phone exercises to paper-based listening exercises. 4.13 0.74
18. I am satisfied with using the mobile device for practising listening 4.27 0.59
skills.
19. Mobile devices encourage self-studying outside classroom. 4.13 0.64
Overall, the results indicated that the students believed that mobile devices
motivated them to practise listening exercises better than the conventional
method of paper-based tasks (M=4.15). The participants in the experimental
group think that mobile learning has the potentials to encourage them to practise
listening skills outside the classroom.
Table 7 shows the students’ attitudes on mobile learning effectiveness towards

providing a flexible delivery of learning and directing the learners towards a more
independent self-management of learning.
Table 7. The dimension of Self-Management of Learning

Statement Mean SD
3. Mobile devices helped me to practise listening anytime and anywhere. 4.47 0.92
8. Mobile devices provided immediate feedback while listening. 4.13 0.74
9. Mobile devices assisted me in selecting listening tasks outside the 3.87 0.64
classroom.
10. Mobile devices helped me manage my listening activities outside the 3.80 0.78
classroom.
11. Mobile devices helped me evaluate my listening skills outside the 4.20 0.56
classroom.
13. I believe I can improve my listening skills alone through mobile 3.80 1.01
devices without the teacher’s help.
Generally, the students believed that mobile devices provide learning

dispositional characteristics like anytime and anywhere sort of learning, provision
of quick feedback and independency of teachers (M=4.05). Consequently, learners
can develop a more independent and self-directed style of learning. Therefore,
mobile learning has the predisposition to provide a self-management style of
learning.

29
However, the success towards shifting to a more self-management of learning

depends on the learners’ willingness and intention to seek their self-directed style
of learning outside the classroom. Therefore, examining the fourth dimension of
the plan to continue using mobile learning is crucial. Table 8 presents the students’
intention to continue using mobile learning to practise language learning further.
Table 8: The dimension of Intention to Use

Statement Mean SD
14. I would like to practise other English skills using mobile devices. 4.20 0.78
15. I encourage others to use mobile devices for English language 4.60 0.63
learning.
16. I would like to listen to authentic materials through my mobile device. 3.87 0.74
20. I’ll continue using mobile learning for learning English after the 3.93 0.96
course.
The results emphasized that the students had the willingness to engage with the
language learning process through mobile learning (M=4.15). The highest score
was on statement 15 (I encourage others to use mobile devices for English
language learning, M=4.60) followed by statement 14 (I would like to practice
other English skills using mobile devices, M= 4.20).
4.3 The Challenges of using Mobile Devices

The researcher used a thematic analysis of the open-ended questions following
coding methods to answer the third research question. There were four main
themes emerged from the analysis of the data using the coding method. These
themes are attributed to the following issues: mobile software-related issues,
mobile features-related issues, technical issues, and listening to content-related
problems.
Some students complained about some issues related to the features of mobile
software (Google Classroom). The design of mobile software did not allow the
learners to play the recordings and view the questions on the same page on their
mobile phones. Participant #3 said, “it was difficult to listen to the audio materials
and answer the questions at the same time”. Due to this issue, the participants
tend to forget what they heard quickly and faced difficulty to grasp the intended
meaning of the recordings. Participant #11 added, “When listening to the audio
materials, it was not possible to look at the question page at the same time, so we
had to close the listening page and open the questions page. For this, we often
forgot things or we were unable to answer directly”.
Other students complained about some mobile features-related issues. Most of the
complains related to the screen sizes of mobile phones. The participants said that
the screen sizes of mobile phones were small, which made it difficult for them to
read and answer the questions. Participants #6 wrote, “The words were tiny and
unclear due to the small screen of the phone”. Also, due to the small sizes, some
students faced difficulty in typing the answer on the screen. Participant #10

30
added, “The screen size of the phone was small, and therefore, there was difficulty
in reading some questions and answering them; the phone does not help much in
writing”.
Students also commented on technical issues and mentioned things including a

wireless network service and lack of internet access. Some students had some
difficulties accessing the Internet using the wireless network due to the lack of
internet coverage. Participant #7 wrote, “Internet in the college was slow.
Opening the audio file took a lot of time”. Alternatively, they had sometimes to
use their internet subscriptions to download the listening materials, which was
inconvenient for them.
Some students also mentioned some listening content-related issues like the audio
files were not very clear, and the speakers were very fast, which made it difficult
to understand the audio files. Participant #9 mentioned, “Sometimes the speaker
was not clear in pronouncing some words, and some recordings were high-
speed”.
Summing up, the findings of the study showed that there was a statistically
significant difference (p < 0.05) between the post-test mean scores of the
experimental group and the control group. Moreover, mobile learning is a novel
educational strategy that can bring effectiveness, incentives, and motivation to the
learning process; however, its implementation has some limitations and
challenges on software design, screen sizes of mobile phones, and networks
connectivity.
5. Discussion
The first research question asked, “Are there any statistically significant
differences in listening performance between students who learn listening skills
through mobile devices and students who conventionally learn listening skills?”
The findings to this question revealed that mobile-learning had a statistically
significant effect on the students’ listening comprehension skills. The learners in
the experimental group significantly outperformed the learners in the control
group in the post-listening test even though the two groups were equivalent in
the pre-listening test before the experiment. The findings of the study indicate the
usefulness of using mobile devices in enhancing English language listening
learning which lends support to several previous studies (Al Yafei & Osman, 2016;
Chen, 2016; Chen, Hsu & Doong, 2016; Lie & He, 2014; Rahimi & Soleymani, 2015;
Read & Kukulska-Hulme, 2015). They all provided support to the effectiveness of
mobile devices in enhancing the language teaching and learning process.
The improvement of the experimental group students in listening comprehension

skills might have been due to the potentials that mobile learning has provided.
The researcher noticed that the students in the experimental group were highly
interested in exploring learning the target language listening skills through their
mobile devices. The students translated their high degree of motivation towards
mobile learning into a higher level of engagement, exposure, and inclination to
explore more listening materials through their mobile devices. Read and

31
Kukulska-Hulme (2015) claimed that using a mobile app to motivate students’

practice on listening comprehension promotes intrinsic motivation for prolonged
exposure to the target language. Mobile learning has succeeded in creating a
relaxing learning atmosphere as learners could self-regulate their learning
process. The students in the experimental group were able to learn listening skills
anywhere and at any time rather than restricting themselves to a fixed place or a
limited time, as the case with the control group.
In contrast, the learners in the control group appear to have been less motivated
and less exposed to the target language outside the classroom, judging from their
failure to meet deadlines in submitting most of their homework. It seems that the
self-directing and portable attribute of the mobile learning strategy might
demonstrate to be an essential motivational factor in increasing exposure and in
enhancing autonomous learning over the non-portable conventional method.
Hence, sustaining motivation is required for language learners to keep on
listening to a considerable amount of listening materials and getting constant
exposure to the target language outside the classroom in a self-regulation manner
so that they gain significant improvements in their listening comprehension skills.
The second research question asked, “What are the students’ attitudes towards
using mobile devices in improving listening comprehension skills?” The findings
to this question suggested that the learners were influenced by the experiment
and showed a significant positive attitude towards mobile-based learning. They
believed that mobile learning was influential in improving their listening skills
and they frequently mentioned anytime and anywhere learning, extensive
practice, provision of a variety of vocabulary, motivation, and autonomy as the
most favourable characteristics of mobile learning.
Therefore, the findings of the study were in agreement with the results of most of
the previous studies as more exposure to the target language is provided outside
the classroom (Kim, 2013; Read & Kukulska-Hulme, 2015). Also, mobile learning
can reduce a lot of anxiety (Rahimi & Soleymani, 2015), enhance more self-
regulated learning (Gangaiamaran & Pasupathi, 2017; Liu & He, 2015; Zheng and
Chen, 2018; Yabukoshi, 2018). All of these studies claim that the use of mobile
learning technology makes the learning process enjoyable, valuable, and portable.
Furthermore, the findings of the present study demonstrated that mobile learning
is a critical motivator in increasing the learners’ incentive to further practise
listening activities outside the classroom, which are also supported by previous
studies (Al Yafei & Osman, 2016; Read & Kukulska-Hulme, 2015). Maria (2015)
explained that the use of technology could motivate young learners during their
EFL classes by creating a positive learning atmosphere as well as it can stimulate
teachers to be innovative and creative in their materials design and teaching
methods. The stimulating activities and the motivating use of mobile learning
enable students to practise the target language and become active explorers of the
English language; at the same time, they simultaneously improve their fluency
and proficiency level. This present study revealed that learners’ motivation could
be enhanced considerably through the use of mobile learning technology when

32
compared to the paper-based conventional teaching method. Substantially, the

ingenious features of mobile learning, including availability, adaptability,
individuality can generate multiple learning functions for our everyday life and
can increase learners’ motivation. The uses of mobile technologies have shifted
language learning from conventional-teaching methods focused on rote learning
to a more constructive, interactive, innovative, and portable learning experience.
Another appealing attribute of mobile technology is the ability to enhance

autonomous and self-regulated learning. Thus, the findings of this study are
consistent with the results of (Al-Yafei, 2016; Chen, Hsu & Doong, 2016; Lie & He,
2014; Zheng & Chen, 2018; Vandergrift & Goh, 2012). These studies demonstrated
that mobile learning could motivate learners to self-direct their learning process
outside the classroom and can maximize the exposure to the target language.
Beatty (2010) emphasized that the predisposition of autonomy and self-regulation
enable learners to be independent of teachers and can manage and control their
self-learning, which engages learners in developing metacognitive strategies for
listening comprehension skills. Maintaining a high degree of motivation
stimulates continuity of self-learning process. As a result, learners improve
optimization of language exposure and increase regulation of language learning
so that language listeners can achieve comprehension.
The third research question asked, “What challenges do students face in using
mobile devices?” The results of the study suggested that the majority of the
learners reported challenges that are related to limitations of the mobile software
design, screen sizes of mobile phones, networks connection, and the
appropriateness of the listening content. A review of literature on this regard
revealed that the main challenges of mobile learning are lack of network
connectivity and ownership (Zhang, 2019). Also, there are problems related to
difficulty in designing appropriate learning experiences, overloaded information,
and distractions by mobile devices (Rogers & Price, 2009). Al-Hunaiyyan et al.
(2018) also found that institutional challenges, design challenges, technical
challenges, evaluation challenges, and cultural and social challenges could hinder
utilizing mobile learning effectively. Likewise, Sophonhiranrak and Sakonnak
(2017) mentioned some of the flaws of mobile learning including the insufficient
size of mobile screens, network connections, learners’ knowledge and perceptions
towards m-learning, the appropriateness of the content provided, and the mobile
applications used. In this respect, it seems that literature corresponded with the
findings of this study as the challenges of mobile software design, mobile screen
sizes, technology infrastructure, and appropriateness of the content are the main
emerged themes when it comes to the real implementation of mobile learning.
The design of mobile software is related to the features of mobile applications such
as its functionality and ability to meet some specific requirements like flexibility
and usability (Ismail, 2016). The failure of a mobile app to meet the needs of
learners in mobile learning might contribute to some usability issues, and
accordingly hinders any advantages of the mobile learning strategy. Hence, for
learners to see the potentials of M-learning, it is vital to design and develop mobile
software that considers some desirable features such as convenience, usability,

33
reliability, performance and functionality. In this study, the participants

complained about the difficulty to navigate between the audio files page and the
questions page. In designing mobile applications, navigation is an important
design element that stands as a success factor for a mobile app because it enables
users to locate efficiently the information they need and to quickly get access to
any learning content (Garofalakis et al., 2007). Therefore, in the instance of mobile
learning, there is a need for mobile software that meets learners’ needs (Klimova,
2019), and gives special attention to functionality, usability, operability, and
attractiveness issues. Future researchers should ideally investigate the expected
quality characteristics of mobile software.
About mobile devices, the issue of mobile screen sizes can limit the sufficient
display of the images and information on mobile phones. Besides, when the screen
sizes are insufficiently small, they make typing on mobile phones very
inconvenient. Therefore, the content of information must be displayed sufficiently
on the mobile screen. Alhajri (2016) mentioned that “the organization of elements
and media on the mobile screen will undoubtedly influence the ease and quality
of learning, and has an important impact on learners’ cognitive load” (p. 2). Thus,
the sufficient display of information on the screen and a user’s interaction with
the learning content determines the success of mobile learning.
Besides, technology infrastructure such as connectivity and being able to get

connected to networks is very critical in mobile learning so that real-time and
ubiquitous learning is possible. Mobile learning is impossible without internet
access. The availability of technology and broadband infrastructure is essential if
teachers and students are to make full use of what mobile learning can offer.
Motiwalla (2007) mentioned that the promise of mobile devices in instant access
to knowledge anytime and anywhere has enormous benefits to learners, but will
be restricted until wireless data access becomes more efficient and widely
available. Hence, the availability of broadband and internet access for mobile
learning is an indispensable necessity.
Regarding the content challenge, some students complained that some audio files
were fast and incomprehensible. A similar problem was reported by Al-Busaidi
(1997) when indicated that speed delivery of the listening materials was one of the
major causes of listening difficulties for the Omani EFL students. Brown (2001)
also referred to this problem as a rate of delivery which is a factor that might block
the process of comprehending a spoken message. Therefore, designers of mobile-
based materials must consider the appropriateness of learning content so that the
benefit of mobile learning is attained. The listening materials must be
understandable, meaningful and purposeful to the target group of learners. For
the mobile learning method to arouse the learners’ desires to continue learning
and to improve their listening ability, the listening materials should address the
learners’ needs and be delivered in a non-distractive manner.

34
6. Conclusion
The current study emphasized the effectiveness of contextualizing m-learning
practices on improving listening skills and revealed the merits of the integration
of mobile phones in English language classes. Mobile learning can motivate
language learners to be self-regulated so that more practices of the language and
exposure take place. Also, the findings of the study showed that EFL learners have
a positive attitude towards the use of mobile phones in language learning.
However, the participants in the study stressed some challenges that were related
to mobile software design, screen sizes of mobile phones, network connections,
and the listening content. The proper development of the mobile learning strategy
requires pedagogically adjusted curriculum and approaches to fit the new
features of the learning-based mobile resources, and properly trained and
motivated teachers and learners.
7. The study limitations

The sample is limited to male students who were enrolled in English language
program courses in a military training institute in the academic year 2019-2020.
Also, the study is only tackling English listening comprehension skills; therefore,
the results of this study will not be applied to any other content area, but rather
confined to listening comprehension achievement. Besides, it was difficult for the
researcher to control the listening exposure of the control group outside the
classroom and limit that exposure to the normal listening classroom only as they
might use other mobile listening-oriented apps to improve their listening
comprehension ability.
8. Recommendations
The findings of this study imply the need for more exposure and practices in the
target language. Teachers should emphasize on extensive listening and encourage
students to listen for pleasure through adopting mobile technologies outside the
classroom. Furthermore, utilizing any mobile-based strategy in ELT curriculum
should be in line with a sufficient training of teachers on the method, design of
resources, and policies of evaluation, too. Therefore, the study recommended
more professional development training courses for all EFL teachers in the Omani
context aiming at familiarizing language teachers with mobile learning
applications. The study also suggested more investigations on the EFL teachers’
perceptions towards using mobile learning in their classroom activities in Omani
schools.
9. References
Al Aamri, K. S. (2011). The use of mobile phones in learning English language by Sultan
Qaboos University students: Practices, attitudes and challenges. Canadian Journal
on Scientific & Industrial Research, 2(3), 143-152.
Al-Belushi, O. (1999). The Internet and Omani students’ English language learning problems:
a critical study (Unpublished master’s thesis). University of Manchester, UK.
Al-Busaidi, S. (1997). An investigation into the causes of students' listening comprehension
problems in English at second secondary level in two schools in the Sultanate of Oman
(Unpublished master’s thesis) University of Exeter, Britain.

35
Al-Emran, M., & Shaalan, K. (2017). Academics’ awareness towards mobile learning in
Oman. International Journal of Computing and Digital Systems, 6(01), 45-50.
http://dx.doi.org/10.12785/IJCDS/060105
Alhajri, R. (2016). Prospects and challenges of mobile learning implementation: A case
study. Journal of Information Technology & Software Engineering, 6(5), 1-8.
http://doi.org/10.4172/2165-7866.1000189
Al-Handhali, H. K. (2009). Learners’ beliefs about listening in an Omani post-basic
school. Research perspectives on education in Oman, 137-148.
https://doi.org/10.1080/00094056.2014.872513
Al-Harrasi, K. T. S. (2014). Using “Total physical response” with young learners in
Oman. Childhood Education, 90(1), 36-42.
https://doi.org/10.1080/00094056.2014.872513
Al-Hunaiyyan, A., Alhajri, R. A., & Al-Sharhan, S. (2018). Perceptions and challenges of
mobile learning in Kuwait. Journal of King Saud University-Computer and
Information Sciences, 30(2), 279-289. https://doi.org/10.1016/j.jksuci.2016.12.001
Al-Issa, A. (2005). The role of English language culture in the Omani language education
system: An ideological perspective. Language, Culture and Curriculum, 18(3), 258-
270. https://doi.org/10.1080/07908310508668746
Al-Issa, A. S., & Al-Bulushi, A. H. (2012). English language teaching reform in Sultanate
of Oman: The case of theory and practice disparity. Educational Research for Policy
and Practice, 11(2), 141-176. https://doi.org/10.1007/s10671-011-9110-0
Al-Musawi, A. S. (2007). Current status of educational technologies at Omani higher
education institutions and their future prospective. Educational Technology
Research and Development, 55(4), 395-410.
https://doi.org/10.1007/s11423-007-9041-x
Alrefaai, I. (2019). Exploring EFL Graduate Students’ Attitudes toward, and Use of, Mobile
Phones in Language Learning. Arab World English Journal (AWEJ) Special Issue:
The Dynamics of EFL in Saudi Arabia, 70-84.
https://dx.doi.org/10.24093/awej/efl1.6
Al-Senaidi, S., Lin, L., & Poirot, J. (2009). Barriers to adopting technology for teaching
and learning in Oman. Computers & Education, 53(3), 575-590.
https://doi.org/10.1016/j.compedu.2009.03.015
Al Yafei, O., & Osman, M. E. (2016). Mobile Phone Apps: An Emerging E-Platform for
Vocabulary Learning and Retention. Journal of Applied Linguistics and Language
Research, 3(7), 286-308.
Retrieved from http://jallr.com/~jallrir/index.php/JALLR/article/view/481
Aryana, S., & Apsari, Y. (2018). Analysing Teacher’s Difficulties in Teaching
Listening. Eltin Journal, Journal of English Language Teaching in Indonesia, 6(2), 100-
106. https://doi.org/10.22460/eltin.v6i2.p100-106
Beatty, K. (2010). Teaching and researching computer-assisted language learning (C. N.
Candlin & D. R. Hall, Eds). Harlow, UK: Pearson Education Limited.
Brown, H. D. (2001). Teaching by principles: An interactive approach to language pedagogy.
White Plains, NY: Longman.
Brown, T. H. (2005). Towards a model for m-learning in Africa. International Journal on E-
learning, 4(3), 299-315.
Chen, X. (2016). Evaluating Language-learning Mobile Apps for Second-language
Learners. Journal of Educational Technology Development and Exchange, 9(2), 39-51.
https://doi.org/10.18785/jetde.0902.03
Chen, Y. L., Hsu, C. C., & Doong, J. L. (2016, March). Self-regulation Assessment of
Mobile Game-based Language Learning. In Society for Information Technology &
Teacher Education International Conference (pp. 882-886). Association for the
Advancement of Computing in Education (AACE).
Retrieved from https://www.learntechlib.org/primary/p/171788/

36
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ
Lawrence Erlbaum Associates, Publishers.
Crompton, H., & Burke, D. (2018). The use of mobile learning in higher education: A
systematic review. Computers & Education, 123, 53-64.
Dunkel, P. A. (1986). Developing listening fluency in L2: Theoretical principles and
pedagogical considerations. The Modern Language Journal, 70(2), 99-106.
https://doi.org/10.1111/j.1540-4781.1986.tb05250.x
Elfiona, E., Embryany, F., & Pamela, K. (2019). The Use of Mobile Application as the
Authentic Listening Materials for Tenth Graders. Proceeding IAIN
Batusangkar, 3(1), 111-116.
El-Hussein, M. O. M., & Cronje, J. C. (2010). Defining mobile learning in the higher
education landscape. Journal of Educational Technology & Society, 13(3), 12-21.
www.jstor.org/stable/jeductechsoci.13.3.12
Feyten, C. M. (1991). The power of listening ability: An overlooked dimension in language
acquisition. The modern language journal, 75(2), 173-180.
https://doi.org/10.2307/328825
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2011). How to design and evaluate research in
education. New York: McGraw-Hill Humanities/Social Sciences/Languages.
Gangaiamaran, R., & Pasupathi, M. (2017). Review on Use of Mobile Apps for Language
Learning. International Journal of Applied Engineering Research, 12(21), 11242-11251.
Garofalakis, J. D., Stefani, A., Stefanis, V., & Xenos, M. N. (2007, July). Quality Attributes
of Consumer-Based m-Commerce Systems. In ICE-B (pp. 130-136).
https://doi.org/10.5220/0002113101300136
Goh, C. C. (2000). A cognitive perspective on language learners' listening comprehension
problems. System, 28(1), 55-75. https://doi.org/10.1016/S0346-251X(99)00060-3
Grant, M. M. (2019). Difficulties in defining mobile learning: Analysis, design
characteristics, and implications. Educational Technology Research and
Development, 67(2), 361-388. https://doi.org/10.1007/s11423-018-09641-4
Harmer, J. (2007). How to Teach English (2nd ed). Harlow: Pearson Education Limited.
Hapsari, Y., & Ratri, D. P. (2014). Extensive listening: Let students experience learning by
optimizing the use of authentic materials, Jurnal Bahasa dan Sastra, 14(2), 251-260.
https://doi.org/10.17509/bs_jpbsp.v14i2.723
Holden, C. L., & Sykes, J. M. (2011). Leveraging mobile games for place-based language
learning. International Journal of Game-Based Learning (IJGBL), 1(2), 1-18.
https://doi.org/10.4018/ijgbl.2011040101
Ismail, M. I. B. (2016). A Review of the Challenges and Issues in Mobile Learning.
International Journal of Enhanced Research in Educational Development (IJERED), 4 (2).
Kim, H. S. (2013). Emerging mobile apps to improve English listening skills. Multimedia-
Assisted Language Learning, 16(2), 11-30.
https://doi.org/10.15702/mall.2013.16.2.11
Klimova, B. (2019). Impact of mobile learning on students’ achievement results. Education
Sciences, 9(2), 90. https://doi.org/10.3390/educsci9020090
Krashen, S. (2013). Second language acquisition: Theory, applications, and some
conjectures. Cambridge: Cambridge University Press.
Kukulska-Hulme, A., & Pettit, J. (2009). Practitioners as innovators: Emergent practice in
personal mobile teaching, learning, work and leisure. Mobile learning: Transforming
the delivery of education and training, 135-155.
Kukulska-Hulme, A., & Traxler, J. (Eds.). (2005). Mobile learning: A handbook for educators
and trainers. London, UK: Routledge.
Lee, Y. J., & Cha, K. W. (2017). Listening Logs for Extensive Listening in a Self-regulated
Environment. The Asia-Pacific Education Researcher, 26(5), 271-279.
https://doi.org/10.1007/s40299-017-0347-0

37
Liu, Q., & He, X. (2015). Using mobile apps to facilitate English learning for college students in
China (Bachelor’s thesis). University of Boras, School of business and IT,
Sweden.
Maria, D. (2015). Using Information and Communication Technologies to Motivate Young
Learners to Practice English as a Foreign Language in Cyprus. JALT CALL Journal,
11(3), 283-292. https://doi.org/10.29140/jaltcall.v11n3.198
Motiwalla, L. F. (2007). Mobile learning: A framework and evaluation. Computers &
education, 49(3), 581-596. https://doi.org/10.1016/j.compedu.2005.10.011
Namaziandost, E., Ahmadi, S., & Keshmirshekan, M. H. (2019). Listening comprehensions
problems and strategies used by intermediate EFL learners. Journal of English
Literature and Cultural Studies, 2(3), 28-41.
http://jelcsjournal.com/browse.php?a_code=A-10-53-5&slc_lang=en&sid=1
Peters, K. (2007). m-Learning: Positioning educators for a mobile, connected future. The
International Review of Research in Open and Distributed Learning, 8(2).
https://doi.org/10.19173/irrodl.v8i2.350
Pulla, S. (2020). Mobile learning and indigenous education in Canada: A synthesis of new
ways of learning. In Indigenous Studies: Breakthroughs in Research and Practice
(pp. 175-199). IGI Global. https://doi.org/10.4018/978-1-7998-0423-9.ch010
Rahimi, M., & Soleymani, E. (2015). The impact of mobile learning on listening anxiety
and listening comprehension. English Language Teaching, 8(10), 152-161.
http://dx.doi.org/10.5539/elt.v8n10p152
Raya, M. J., & Fernández, J. M. P. (2002) Learner Autonomy and New Technologies.
Educational Media International, 39(1), 61-68.
http://dx.doi.org/10.1080/09523980210131240
Read, T., & Kukulska-Hulme, A. (2015). The role of a mobile app for listening
comprehension training in distance learning to sustain student motivation. J.
UCS, 21(10), 1327-1338. http://dx.doi.org/10.3217/jucs-021-10-1327
Richards, J. C. (2005). Second thoughts on teaching listening. RELC Journal, 36(1), 85-92.
https://doi.org/10.1177/0033688205053484
Rivers, W. (1966). Listening comprehension. Modern Language Journal, 50(4), 196-204.
http://dx.doi.org/10.2307/322015
Rogers, Y., & Price, S. (2009). How Mobile Technologies Are Changing the Way. Mobile
technology for children: Designing for interaction and learning, 1-22.
https://doi.org/10.1016/b978-0-12-374900-0.00001-6
Rost, M. (1994). Introducing listening. London: Penguin.
Rost, M. (2007). Listening. In D. Nunan & R. Carter (Eds.), The Cambridge guide to teaching
English to speakers of other languages (pp. 7–13). Ernst Klett Sprachen.
Retrieved from http://studentsrepo.um.edu.my/id/eprint/8610
Sophonhiranrak, S., & Sakonnak, S. P. N. (2017, October). Limitations of Mobile Learning:
A Systematic Review. In E-Learn: World Conference on E-Learning in Corporate,
Government, Healthcare, and Higher Education (pp. 965-971). Association for the
Advancement of Computing in Education (AACE).
https://www.learntechlib.org/primary/p/181279/
Traxler, J. (2009). Current state of mobile learning. Mobile learning: Transforming the delivery
of education and training, 1, 9-24.
Unver, M. (2017). The use of authentic materials with low level learners of English.
European Journal of English Language Teaching, 2 (1), 75-92.
http://dx.doi.org/10.5281/zenodo.293186
Vandergrift, L. (2007). Recent developments in second and foreign language listening
comprehension research. Language teaching, 40(3), 191-210.
https://doi.org/10.1017/S0261444807004338
Vandergrift, L., & Goh, C. C. (2012). Teaching and learning second language listening:
Metacognition in action. Routledge.

38
Yabukoshi, T. (2018). Self-regulation and self-efficacy for the improvement of listening

proficiency outside the classroom. The Language Learning Journal, 1-14.
https://doi.org/10.1080/09571736.2018.1472626
Zhang, Y. (2019). Characteristics of Mobile Teaching and Learning. Handbook of Mobile
Teaching and Learning, 13–33. https://doi.org/10.1007/978-981-13-2766-7_5
Zheng, L., Li, X., & Chen, F. (2018). Effects of a mobile self-regulated learning approach on
students’ learning achievements and self-regulated learning skills. Innovations in
Education and Teaching International, 55(6), 616-624.
https://doi.org/10.1080/14703297.2016.1259080
Appendix 1
Students’ Attitudes towards Using Mobile devices in learning English

Listening Skills
First: In the following list, please indicate the level of your agreement and
disagreement with the statements regarding your attitudes towards using
mobile devices in English language learning.
Statements Strongly Disagree Neutral Agree Strongly
Disagree Agree
1. Mobile learning provided more
extensive listening practice.
2. Listening practice through the
mobile device improved my
listening ability.
3. Mobile devices helped me to
practise listening anytime and
anywhere.
4. I listen to audio materials using
my mobile device more than
once.
5. Using mobile devices
motivated me to practise the
listening skill.
6. The mobile device reduced my
anxiety in learning listening skill.
7. I enjoyed the exercises through
my mobile device than the
traditional way.
8. Mobile devices provided
immediate feedback while
listening.
9. Mobile devices assisted me in
selecting listening tasks outside
the classroom.
10. Mobile devices helped me
manage my listening activities
11. Mobile devices helped me
evaluate my listening skills

39
12. Listening practice through

mobile devices helped me learn a
variety of English vocabulary.
13. I believe I can improve my
listening skills alone through
mobile devices without the
teacher’s help.
14. I would like to practise other
English skills using mobile
devices.
15. I encourage others to use
mobile devices for English
language learning.
16. I would like to listen to
authentic materials through my
mobile device.
17. I prefer mobile phone
exercises to paper-based listening
exercises.
18. I am satisfied with using the
mobile device for practising
listening skills.
Second: Open Questions:

Please read and answer the following questions:
1. What did you like most about using mobile devices in learning English listening
skills?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
2. What difficulties did you face when using mobile devices for learning English
Listening skills?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
3. What do you suggest to improve the use of mobile learning in teaching English
listening skills?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
4. Do you have further comments you would like to add?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
>>>Thank you<<<

40

Vol. 19, No. 8, pp. 40-59, August 2020
What about Study Motivation?

Students´ and Teachers’ Perspectives on What
Affects Study Motivation
Lena Boström
Mid Sweden University, Sweden
https://orcid.org/0000-0001-9182-6403
Göran Bostedt
Mid Sweden University, Sweden
https://orcid.org/0000-0002-4398-5394
Abstract. One out of every four upper secondary school students in

Sweden interrupts their education, although the intention behind the new
Curriculum for Upper Secondary School (GY 11) was to increase
throughput of students with complete grades. Lack of study motivation
is the most important explanation for students dropping out. This article
analyzes study motivation from students and teachers’ perspectives. It is
based on interviews in three upper secondary school programs that were
analyzed with a qualitative approach and hybrid content analysis. Study
motivation is set in relation to motivational strategies, achievement, and
learning environment. The result showed similarities and differences in
perceptions. Both teachers and students pointed to the importance of
teachers, practical pedagogy, social relations, and the significance of
grades for study motivation. An important difference between
informants was that teachers put more emphasis on life skills and
adapted study groups, whereas students pointed to the physical learning
environment and teachers’ personalities as important. Conclusions in the
study point to complex interplay between internal and external
motivational factors and between situation, person, and learning
processes. This leads to validity of interactive and transactional
motivational perspectives. A broader and more in-depth study is needed
primarily to understand students’ perspectives.
Keywords: increasing motivation; decreasing motivation; students´ and

teachers’ perspectives; study motivation; upper secondary school
1. Introduction
One in every four students in Sweden drops out of upper secondary education.
However, the intention of the new upper secondary school reform, Curriculum
for Upper Secondary School (GY11), among other things, was to increase the

41
throughput of students with complete grades. Possible reasons for this trend are
that the new system imposes higher requirements for admission to the upper
secondary school’s national program, Swedish primary school students have
declining knowledge results, and primary school students are less prepared for
the increasing demands in secondary school (Skolverket, 2016). Lack of student
motivation is the most crucial explanation for the drop-out rate. More than half of
the upper secondary school students indicated in a national study (Sveriges
Elevkårer & Lärarnas Riksförbund, 2015) that they experienced low or non-
existent study motivation. The most important factors for increasing student
motivation are stated to be teachers’ subject competence, teacher–student
relationships, educational support, and access to student health. In the study,
teachers also pointed to students’ lack of prior knowledge from elementary school
and the need to learn better study techniques to pass upper secondary school.
Dropping out of upper secondary education has individual and national

consequences. The National Agency for Education has therefore implemented
activities that are close to business (Skolverket, 2016) to prevent drop-outs. The
National Agency for Education (Skolverket, 2019) has described the measures,
which include conferences for principals and process support for schools. Several
initiatives and key areas have been identified as necessary, such as early efforts to
develop a systemic approach and to maximize the local scope for action.
Motivation is a prerequisite for learning in school. However, scholars lack the
main actors’ perspectives on these efforts, namely the students and the teachers.
When students have low or non-existent study motivation or if students lack the
necessary prerequisites, schools must find ways to help them achieve the
knowledge goals. This study examines how students and teachers think about
students´ study motivation and how a school can match the students. According
to Skolverket (2019):
“Lack of motivation makes it difficult to take advantage of school education.
Conversely, those who do not qualify for teaching can suffer a lack of
motivation. Assuming a lack of motivation—and thus corresponding
compensation needs—motivation becomes something that needs to be added to
the learning processes. ” (p. 154).
Because the new upper secondary school we studied has not yet achieved the
ambitions that existed with upper secondary school reform regarding increased
throughput, and because causal analysis points to student motivation as an
important explanation, we studied upper secondary school students´ and
teachers’ ideas about student motivation. We gained in-depth knowledge of
students’ thinking about the issue of study motivation and compared their views
with the teachers, who are the other important group of actors in classrooms.
Another important reason for the interest in study motivation is that few national
studies in the field exist (Lundahl et al., 2015) and research on the interaction
between individual and learning environments in Swedish schools is limited
(Blomgren, 2016). Analytical models of learning (in this case, study motivation)
require an understanding of the interplay between individuals, educational
material, and the social context (Imsen, 2006). Another important incentive for the
research area is that there are few contemporary studies based on students as
informants about their study motivation (Giota, 2013, 2017). A recent study

42
(Hofverberg, 2020) points to several different perspectives on motivation that

need to be integrated in order to capture the complexity that students' driving
forces constitute and that research should be conducted close to practice in
collaboration with teachers.
The internal factors regarding study motivation are often highlighted in research,
but Håkansson and Sundberg (2012) warned seeing the issue of motivation as a
purely individual trait. They pointed out that motivation is contextual,
changeable, and arises in dynamic relationships between people. Furthermore,
factors such as family background class affiliation and grades (Skolverket, 2019),
socio-economic belonging, gender, and ethnic origin (Skolverket, 2018) are cited
as important for students’ study motivation. Several influencing factors can be
assumed to both support and cause study motivation deficiencies. However, the
responsibility for lack of motivation is often placed on the young people
themselves, especially from many municipal representatives (Lundahl et al.,
2015). Like Håkansson and Sundberg (2012), we believe that internal and external
factors need to be considered in a discussion about study motivation. Thus, to
analyze a lack of study motivation as a cause of low throughput in upper
secondary school, a perspective is required that not only focuses on individual
students but takes into account the entire school and classroom context (Imsen,
2006). This study contributes to the field by applying theories of educational
psychology in practice-related activities in school. This is justified based on
various research results (Giota, 2013, 2017; Hattie, 2009) that show motivational
factors influence students’ study results.
In this article, we analyze study motivation based on didactic aspects and learning
environments. Our interest in the issue of study motivation can be expressed as
an interest in analyzing both internal and external motivational factors. Responses
from students and their teachers from three academic programs in a municipal
upper secondary school constitute the empirical material. The school was located
in the central part of a large city. For 2015, the municipality reported it was eight
percentage points below the value for all municipalities in Sweden.
In view of the above problem, the purpose is to describe and analyze the upper
secondary school students’ study motivation or lack of motivation from the
students´ and teachers’ perspectives. The selected issues are as follows:
• What experiences do teachers and students have about what increases
students’ study motivation, and what is the cause of and explanation for
possible low study motivation?
• What differences and similarities exist in teachers and students’
perceptions?
• How is study motivation linked to various aspects of education according
to teachers and students?

43
The article initially describes the theoretical framework we used, namely

motivation theory, motivation strategies, and learning environment. Thereafter,
methodological approaches and the results are described. Finally, conclusions and
educational implications are presented in a discussion that returns the result to
the theoretical frameworks. The definitions that have been made are that the
article focuses on students´ and teachers’ perceptions of factors that are directly
linked to the school. Thus, how time outside the school affects students’ study
motivation is not addressed.
2. Theoretical Overview
Study motivation is a multifaceted concept. Therefore, we begin with a general
overview of motivation theory in the school context and then discuss motivation
in relation to three theoretical starting points: achievement, motivational
strategies, and learning environment. These assumptions are not mutually
exclusive but overlap in different respects.
2.1 Motivation Theory

Motivation is a research area found in various disciplines (Woolfolk & Karlberg,
2015). There are different perceptions of what influences student motivation. One
is that the interaction between teacher and student and access to student health
are the most important factors in raising student motivation (Sveriges Elevkårer
& Lärarnas Riksförbund, 2015). There are also different definitions and
understandings of the concept of study motivation. It is often described with the
dichotomous inner and outer motivation (Ryan & Deci, 2002). From within,
controlled motivation arises when an activity feels engaging and rewarding.
Internal motivation factors are about seeking and managing challenges based on
an endeavor to satisfy personal interests and use one’s abilities (Woolfolk &
Karlberg, 2015). In school, students with strong internal motivation show greater
endurance with the tasks, seek more understanding in terms of knowledge, and
try different strategies to achieve their goals (Giota, 2017).
Motivation is more often based on control when the result determines how
motivated the student is, or if there is a reward that attracts (e.g., credits, praise,
or useful skills in working life) or a “punishment” (e.g., missing student support,
delayed studies) that threatens them. External motivational factors imply an
endeavor to meet expectations or demands that come from or are perceived to
come from, outside the individual (Giota, 2017; Woolfolk & Karlberg, 2015).
Externally motivated students adopt more surface learning strategies, often
giving up when rewards and benefits are removed etc. (Giota, 2017). External
motivation can be divided into two subcategories of controlled or autonomous
motivation. Controlled external motivation is about being controlled by someone
else through reward, penalties, or fear of failure. Autonomous external motivation
implies in the context of a school that a student has taken on the values of the
school and makes an effort even if the activity does not give pleasure. Non-
existent motivation is the third aspect of motivation; that is, study motivation is
entirely lacking. Students give up, blame other factors than themselves, and do
not see the relationship between performance and results (Woolfolk & Karlberg,
2015).

44
Researchers have described the dichotomy of inner and outer motivation as clear
cut, whereas others believe that internal and external factors are interdependent;
that is, students internalize external causes (Vaanstenkiste et al., 2006) or internal
and external factors constitute endpoints on a continuum (Covington & Mueller,
2001). Therefore, because there is an interaction between internal and external
motivation, one can talk about motivation systems (Anderman & Anderman,
2009). The factors interact and, in many cases, depend on each other (Jerkeby,
2019). External motivation can be changed to the internal; they can exist
simultaneously and vary between different times and tasks. Determining when
students are driven by internal or external motivational factors is therefore tricky.
The most important difference is the student’s motives for their actions (i.e.,
whether it is internally or externally motivated; Jerkeby, 2019). For students who
find it challenging to find motivation in school, external motivation is a tool to get
started with the studies. Furthermore, motivation is not a fixed trait that one either
has or does not have. It changes, develops, and varies during the studies, and
there are many factors, both internal and external, that affect it. The interactive
theory of motivation makes it possible to capture and focus on the interaction
between a person and a situation (Stensmo, 2005).
2.2 Achievement and Motivation

The interactive motivation theory focuses on achievements and expectations
(Stensmo, 2005). It is about the desire of individuals to deal with a question and
their fear of failure and how this is affected by the expectations that exist in the
individual. This means that the interaction between the expectations of teachers,
parents, and others, as well as their ambitions, shape students and their
performance goals. Teachers’ significance for study motivation is well
documented in research; that is, how they convey expectations of students (Giota,
2013), organize classroom activities (Hattie, 2009), stimulate engagement and
effort, support individuals and groups, shape classroom climate (Hugo, 2011),
choose didactic work methods (Boström, 2013), teachers´ subject-specific
enthusiasm (Mahler, Großschedl & Harms, 2018) and lead learning (Stensmo,
2005). These are crucial interactive motivational factors along with the inner and
outer motivational factors. To analyze lack of study motivation as a cause of low
throughput in upper secondary school, a perspective is therefore required that not
only focuses the individual student but takes into account the entire school and
classroom situation (e.g., the relationship between student, teacher, and
educational materials (Imsen, 2006).
Another, but partly overlapping, theoretical point of departure is that motivation

is more about transaction than interaction (Perry et al., 2006). Motivation is then
understood not only as an individual trait but is about negotiation of meaning in
social interaction. Thus, motivation becomes an integrated process in a larger
whole, impossible to separate from learning, individual differences, and the
nature of tasks or social context. According to Perry et al. (2006), strong
relationships exist between motivation and (a) communicated expectations, (b)
clear feedback on results, (c) interactions between teachers and students and
among students and their peers, (d) positive climate, and (e) teachers´ leadership.
Blomgren’s (2016) summary regarding students’ perspectives on schoolwork, and
the importance it has for study motivation, is that study motivation is primarily

45
shaped by perceptions of success and failure, as well as perceived self-capacity.

This conclusion are similar to Perry et al. (2006).
2.3 Motivational Strategies

Strategies to increase student motivation can be understood and analyzed from
various perspectives (Jerkeby (2019). Motivation strategies interact with each
other in many different ways. If teachers are to contribute to students’ study
motivation, a “toolbox” of different motivational strategies is required
(Augustsson & Boström, 2016).
The following strategies are mentioned by various researchers in the field:

understanding and taking into account the complexity of events, students and
groups in the setting of teaching (Giota, 2013), the design of the tasks to enable
adaptation to individuals and groups (Boström, 2013), constructive evaluations,
taking into account time aspects and didactic diversity (Woolfolk & Karlberg,
2015), various teaching strategies and active work with metacognitive strategies
(Boström, 2013), differences in students' perceived best learning and teaching
strategies (Boström & Bostedt, 2020) and conscious leadership in the classroom
(Augustsson & Boström, 2016; Hattie, 2009). Paying attention to the emotions that
are brought about by success and failure and the teachers’ competence to handle
them at both group and individual level is also relevant in this context (Giota,
2013; Imsen, 2006). Teaching strategies that impair student motivation include
ineffective or no feedback (Giota, 2013), lack of connection, overly complicated
tasks (Hugo, 2011), slow pace, focus on being transparent and not learning, poor
planning, and punitive leadership. Other demotivating factors are unattractive
classrooms and negative mood in the class (Woolfolk & Karlberg, 2015).
Factors that motivate students are also complex. According to Farrington et al.
(2012), decisive factors that influence student learning outcomes are study-
oriented behavior, endurance in studies, academic mindset, constructive learning
strategies, and social ability. One’s self-esteem, experience ways of experience
events, and individual goals (Hugo, 2011; Wery & Thomson, 2013) are crucial to
study motivation, because allowing students to learn in their best individual way
is of decisive importance for the results. Furthermore, previous research points to
the importance of students’ perceptions of work tasks; that is, relevance, utility,
level of difficulty, working methods (Granström, 2012), feedback, and grouping
and group dynamics (Håkansson & Sundberg, 2012; Woolfolk & Karlberg, 2015;
Zimmerman, 2018). Also, students overall “experiences” of teachers’ didactic
competence (Hattie, 2009) and the importance of relationships (Aspelin, 2018) are
also considered to have a positive effect on study motivation.
Positive, neutral, or negative teaching strategies affect student motivation

accordingly. If teachers can match teaching strategies with students’ learning
strategies, then good conditions are created for student motivation and study
results. A concrete example is a student’s need for a clear teaching structure,
which is not always in line with teachers’ perceptions (Boström, 2013). Both upper
secondary school students and university students show a clear need for external
structure to perform better (Boström & Gidlund, 2016). Students are motivated if
they receive clear frameworks, instructions, deadlines, schedules, exemplary

46
examples and concretions, and regular feedback (Boström, 2013). This matching
pedagogy seems particularly essential for students in need of support or in a
classroom situation where behavioral problems occur (Gidlund & Boström, 2017).
Grönqvist and Vlachos (2008) found that different types of students are affected
in different ways, but that the match between student and teacher is crucial to
students’ study motivation. However, they emphasized, “Figuring out which
teachers are best suited in different situations is an open question” (p. 15).
2.4 Motivation and Learning Environments

The surrounding physical and social environment also affects students to varying
degrees (Ahlberg, 2001; Valsö & Malmgren, 2019). For many students, study
motivation is formed in the learning environment. According to Blomgren (2016),
this is evident in students’ descriptions of feelings and perceptions of success and
failure. Adapted learning environments with inclusive approaches are especially
crucial for students who have not previously succeeded in school (Gidlund &
Boström, 2017). Definitions of learning environment vary depending on scientific
starting points and disciplines and include different perspectives on learning.
Learning environments are described based on mental, social, and psychosocial
dimensions (Sveriges Kommuner och Landsting, 2017). Another definition of the
concept which goes further is including psychological, educational, cognitive,
socio-economic, physical, communicative, social, and organizational aspects
(Ahlberg, 2001).
If environments are to motivate students, then they should in some sense be good,
which in research has been described as working methods, attitudes, and the
physical layout of the classroom (Ahlberg, 2001). These factors can help create
good conditions for all students’ sense of participation in activities (Antonovsky,
2005). Insights into and knowledge of how good learning environments are
established are needed to understand student learning processes (Ahlberg, 2001).
The same goes for students, namely that they understand what constructive
learning environments are for them so that they can take responsibility for
learning (Boström, 2013; Jerkeby, 2019). Opportunities to meet students on their
own terms are about the knowledge and understanding of individual differences
and similarities regarding student learning. In a learning environment, mutual
interaction takes place where people affect and are influenced by the social and
physical environment (Björklid & Fischbein, 2011). Learning takes place between
people in a physical context and in a social context. An indispensable ingredient
in the learning process is the tools that teachers use, which can be either physical
or intellectual (Säljö, 2014). A good learning environment must therefore be
initiated, created, developed, and evaluated to best support a student’s study
motivation.
3. Empirical Starting Points

3.1. Upper Secondary School Programs
Three upper secondary school programs selected for empirical material collection
were the Social Sciences Program (SSP), the Health and Social care Program
(HSP), and the Individual Program (IP). The selection principles thus include both
academic and practical programs, as well as a representation of student groups

47
with various past successes or challenges in their learning processes. To address

our research questions, in spring 2017, we conducted six group interviews with
12 students, as well as three group interviews with 20 teachers from the three pro-
grams. The study is limited to factors that are directly linked to the school. How
time outside of school affects students' study motivation is not a primary part of
the study's interest. Furthermore, the study is limited to the four participating up-
per secondary school programs and group interviews with teachers and students.
The throughput figures for students at the upper secondary school in 2016 were
about 8% below the national average, whereas the municipality’s Child and Edu-
cation Board’s goals and resources plan (X Municipality, 2016) set high targets in
terms of increasing the proportion of students completing their upper secondary
studies in the course in 3 or 4 years. Thus, there was a clear political orientation
that affects the children and education administration and the work of the munic-
ipal upper secondary school. In the local business plans for 2015–2016, a crucial
area of development was found to increase student motivation. Such work was
perceived by those responsible for the programs as helping to raise the results in
the upper secondary school. The political ambitions of the upper secondary school
are also reflected in an operational priority from the administration.
3.2 Data Collection and Data Processing

The design of questions for the group interviews was adapted from Blomgren
(2016). The teachers interviewed consisted of those gathered at a work-place meet-
ing for the teachers’ college for the intended program or those who volunteered
to participate. The students were selected by teachers based on the criteria that the
group of students would include both boys and girls and students with varying
academic success. The interviews were conducted on-site at the school and were
recorded and transcribed. The interviews lasted between 40 min and 1.5 hr. The
transcribed interviews comprised approximately 250 A4 pages of text.
3.3 Method
We used a hybrid content analysis method (Fereday & Muir-Cochrane, 2006; Ris-
ing Holmström et al., 2015). We conducted group interviews as a data collection
method. All contributors were informed about the project’s aims and current eth-
ical research principles (Vetenskapsrådet, 2017). At the start of the interviews, all
informants were informed that participation was voluntary and that they could
cancel the interview at any time.
The interviews focused on organizational conditions, perceptions of interpersonal

processes, and individual characteristics. To achieve the purpose, we used a
hybrid content analysis that began with deductive analysis based on selected
theories and perspectives, and then moved on to an inductive analysis and finally
connected the theoretical starting points with the empirical material in the result.
A deductive (targeted) content analysis (Hsieh & Shannon, 2005) was initially
used to answer the research questions. The analysis was based on predetermined
themes when the interview material was analyzed (Mayring, 2000) and was
characterized by a more structured process compared with unconditional coding.
The deductive content analysis enables comparisons with results from previous

48
research and the results of discussions based on different selected theoretical

perspectives (Elo & Kyngäs, 2008). This article tests theories in the field of study
motivation that see motivation as a social and contextual interplay (Hugo, 2011;
Perry et al., 2006). The analysis of the interviews was based on four themes:
motivation, motivational strategies, learning environments, and more. Based on
the four themes, a categorization matrix was developed that was then
systematically used in the analysis of the interviews.
After an initial deductive analysis phase of the interview responses, the analysis
turned into an inductive approach (see Figure 1). With selected themes as breaks,
the categorization matrix was developed. Data were sorted via an inductive
process (i.e., the text “spoke freely” within each theme and generated categories).
After a close reading, the parts of the text that expressed identifiable ideas or
positions (units of meaning) were condensed by coding within each theme. The
empirical content was examined methodically, the texts were interpreted step-by-
step, and data were classified to distinguish patterns. The empirical material was
broken down into meaning-bearing units, which were condensed into shorter
sentences and then abstracted into codes, describing the content of the meaning-
units. Codes with similar content were combined into themes and organized into
categories. To make the analysis transparent, codes and categories were combined
in an analysis scheme. These categories were interpreted and presented in their
respective themes with some telling quotes, and finally, the two informant groups
were compared. The two researchers discussed the results of analysis thoroughly
until we reached consensus, a process that resulted in the further refinement of
categories and a final thematisation. To offer credible, generalisable results, we
have reported our methodological approach, means of categorisation and
analytical method herein.
Figure 1: The analysis process
The analysis process was not linear but had more of an iterative character, where
the process moved back and forth between the different phases. Through
reflective dialogues between the researchers, the data processing was carried out.
The participants were coded in the transcribed material with the numbers T1–T20
for teachers and S1–S12 for students to distinguish them more efficiently during
the processing of the data set.

49
4. Results and Analysis

The results are presented and analyzed based on the study’s purpose and its three
research questions. The presentation of the result is based on the developed theme
and categorization matrix. In the four themes (i.e., motivation, motivational
strategies, learning environments, and more), there are common and distinctive
categories within both the teacher and student groups, but also between them.
The categories that were condensed were teachers, students, structure, social
relations, and results.
4.1 Motivation
Regarding the upper secondary school students’ study motivation, several
influence categories were found. Teachers were the most important motivator,
according to the informants:
“I would say that perhaps the teacher’s most important task is to work with
and improve and develop students’ motivation. But I would also like to say
that it is the students’ most important task to become aware that their own
motivation is so incredibly crucial, and that motivation is something that can
change.” (T8).
Teachers are perceived as the single most important factor that affects students’
study motivation and through their leadership they can influence students’ inner
motivation is confirmed by other research (Hattie, 2009; Hugo, 2011; Håkansson
& Sundberg, 2012). The approaches that describe constructive teachers are clarity
and the ability to give constructive feedback and push students and to be
“interesting".
Students’ responsibility for study motivation was a prominent theme in the

empirical material. However, students and teachers’ images differed from each
other. The teachers placed more emphasis on the students’ inner motivation, with
descriptions that study motivation can be controlled from within and the
importance of students being responsible, wanting to learn, and seeing the
benefits of going to school. In the teachers’ answers, there was also a strong belief
that students work towards goals and sub-goals and know the purpose of their
studies.
The students, on the other hand, did not discuss to any great extent their inner
motivation. Two study-motivating aspects for them were to experience the
benefits of learning and participation in the planning of teaching. The interviews
did not provide unambiguous or comprehensive answers regarding the students´
strategies for building their inner motivation. However, the empirical material
showed that the teachers “views on how students should muster study motivation
differed from the students” (T9). Important questions include whether students
learned or understood the importance of mental strategies or whether teachers
understand students’ cognitive strategy.
Regarding the category of arrangements, students and teachers consistently

showed that a more practical and laboratory arrangement of lessons has a positive
effect on study motivation. This is in line with the results Boström and Bostedt
(2020) present in a new study on vocational classes´ study motivation.

50
Motivational teaching planning was described as “learning for working life” (T4).
The results are not entirely in line with international and national research on
learning strategies (Boström, 2013; Niemivirta, 2004). This research indicates that
at group level, there are differences in what are called perceptual preferences; that
is, learning-by-doing (Dewey, 1897) is an approach that may suit some students
well, others not. Boström (2013) showed, for example, that students in an upper
secondary school’s vocational program preferred teaching that is based on
learning-by-doing to a greater degree than students in academic programs.
Work-place learning was an example of how teaching methods in or about real -

life generate higher study motivation and higher student attendance compared
with regular school lessons. According to T2, having “courses that you can do close
to reality, it often becomes . . . easier for the students to become study-motivated.” If the
approach was not sufficiently well-planned or too monotonous, students’ study
motivation decreased. The design of assignments was also crucial for students’
study motivation (cf. Hugo, 2011; Håkansson & Sundberg, 2012). Another aspect
of the teaching structure concerned the upper secondary school common subjects,
which were not as popular with the students in the vocational programs as they
were in the academic programs. These subjects lowered the study motivation. The
time aspect was also emphasized by the students as an essential factor in
increasing or decreasing motivation: “Time is more important than methods . . . but
this is where you get a little time for certain things” (S5).
Teachers and students agreed on the importance of grades for study motivation,
namely that the presence of grades can both increase and decrease motivation: “If
I get a high grade on one task, I will be motivated for the other. Grades give motivation”
(S2); “If I get bad grades/. . ./ or if I am behind, then I cannot work at all. It will be a
vicious circle” (S4).
In summary, the results showed that study motivation could be seen as both a
controlled and autonomous external motivation-driven phenomenon (cf. Imsen,
2006; Woolfolk & Karlberg, 2015). In the teacher interviews, the perception
emerged that the students did not reflect sufficiently on what they had learned,
even though information was submitted so they would not fail. This is a strategy
that demonstrates control via external autonomous motivation (Wery &
Thomson, 2013). If students do not reflect on what they have learned, then it can
be seen as a rejection of the school’s mission to stimulate students’ metacognitive
competence.
4.2 Motivational Strategies

Motivational strategies were perceived as active behaviors or actions to create
motivation in the students, which can include students’ thoughts, feelings, and
actions, but also teachers’ actions or surrounding structures or cultures (Jerkeby,
2019). Similar themes that emerged about motivation can also be discerned
within this theme (i.e., teachers, students, structure, and results). Here, however,
social relations are also added as a category.
Teachers’ behaviors were of strategic importance, according to both teachers and

students, which is in line with current research (Håkansson & Sundberg, 2012;

51
Hattie, 2009; Hofvenberg, 2020; Jerkeby, 2019). One difference between teachers
and students was that they emphasized different teaching behaviors as important.
The teachers pointed to their knowledge competence, the importance of being up
to date on the subject, and the ability to enthuse the students and to see and build
on students’ strengths (cf. Mahler et al. 2018). The students emphasized teachers’
personal qualities in the treatment as important for study motivation (e.g., teachers
should be happy, understanding, and have the competence to provide support,
but should not stress the students). According to the teachers, students’
motivational behaviors were that they are responsible and curious. Behaviors that
reduce or remove motivation are, according to some of the teachers, different types
of defense mechanisms or “repressed dissatisfaction” (T12).
Students consistently mentioned that finding enjoyment in school work was a

motivational strategy: According to S8, “Study motivation means to do what is fun.
So it’s fun if you want to do it. If it’s boring, it’s not fun”. This was also commented
on by teachers: “Everything should be pleasurable. In general, I think students today are
pleasure-driven young people” (T20). Here we can discern generational differences
between teachers who prefer learning to be serious and young people who are
motivated by the teaching being pleasure-filled. This is a challenge to deal with
in everyday pedagogical practice.
According to the teachers and students, the motivation to study decreased or

increased depending on the structure of the courses (e.g., through good planning
and participation). Planning includes teachers’ lesson and course planning, joint
planning, and the students’ planning. The students emphasized the importance
of being able to choose a variety of working methods: “When I am motivated, there
is a variation in the teaching, with varying tasks and subject areas” (S6). Another
motivational strategy was “to get rid of the stamp of boredom” (T3).
Social relationships as motivational strategies recurred in the student and teacher

interviews. If teachers and students can build good relationships, then the study
motivation is affected in a positive sense. The pedagogical task is facilitated with
good relationships, for example, by giving feedback and making the right
demands. Similar to Hattie’s (2009) results, the interviews showed that
relationships between teacher and student were the most important study
motivating factor. According to T1, “One must build relationships, and the biggest
obstacle to reaching students who do not have motivation, it is the absence. Because if they
are not in place, then it is very tough to motivate them.” Results in the form of grades
as a motivation strategy were problematized by both teachers and students. The
overall picture was that grades could create study motivation, but that they could
further inhibit motivation if they were low. “The whip, it is to reach the good grades
then, because you should be able to apply to university or what-ever” (S10). The grades
as a structurally inhibiting and stress-related factor were also problematized by
the informants. The new grading system created excitement and stress for the
students, and the demands are very high. In this study, grades can be seen as an
area where motivation can arise as controlled external or autonomous external
motivation (Imsen, 2006; Woolfolk & Karlberg, 2015) and its effect on study
motivation can be both promoting or inhibiting.

52
4.3 Learning Environments

Descriptions of the learning environment within the teacher group focused on the
importance of adaptations for different groups of students at the individual level
and from the perspective of disabilities, in other words, a special educational
perspective (e.g. Ahlberg, 2001). This did not appear at all in the student group.
Some teachers emphasized that smaller groups of students were a motivating
factor. However, this view did not emerge in the interviews with the students. On
the other hand, well-being, security, and the class were described as essential
aspects in the learning environment by both informant groups. Well-being was
both about being comfortable with the teacher and the class, but also in the
physical sense, namely being comfortable on the premises. The effect of the
external environment on students’ study motivation was even described as
underestimated. Security was also emphasized in the interviews with the two
groups: “If you feel safe and comfortable and you enjoy being here, then homework and
assignments are easier” (S1). Both teachers and students emphasized the importance
of the class or group in the learning environment as an influencing factor for
students’ study motivation. The results above confirm previous research on good
learning environments (i.e., that social inclusion is important as a basis for safe
learning; Ahlberg, 2001).
4.4 Other
Within the theme, there were several distinctive perceptions between teachers and
students. The teachers emphasized that a consensus between school and parents
is crucial to creating a good basis for students’ study motivation. It was seen as
important that the teachers take the initiative for cooperation: “We have a reasonable
consensus with the parents. That we call home and tell now is going well. Trying to push
together. That it is not just that we have a discussion together, but we invite them” (T11).
From a student perspective, teachers considered it crucial that parents are not
“codependent” on their children’s negative school behaviors. Parents can under-
stand or even sanction students’ failures because they may have behaved in a sim-
ilar way when they were young. Regarding social relations, there were similar
views among teachers and students. The right peers were stated to be decisive for
study motivation in such a way that they could influence the study motivation
positively, and with the wrong peers, the influence became negative. However, it
seemed difficult to break away from a group of friends who do not want to study:
“If I hang out with some friends and they do not even want to work, I lose the motivation.
It is difficult to change friends. It’s up to me if I should follow them /. . . / I still have my
own responsibility” (S10).
A distinctive perception between students and teachers in the category of social

relations was the teachers ‘marking of “correct” parental support for students’
study motivation. The right parental support was described as a dialogue between
teachers and parents, where the parents do not sanction the students’ absences or
negative behaviors and where they influence their children’s positive views of the
school. The opposite was parents who more or less had given up or contributed
to a reduced study motivation for their children. The students expressed that pa-
rental support could vary. Some students had parents who supported them; oth-
ers did not have this support. A dilemma that the students described was that

53
supportive parents could also lead to perceived pressure for the students to con-
tinue to perform well. This could be perceived as a negative or problematic expec-
tation structure.
4.5 Students´ and Teachers’ Equal and Different Perceptions

In summary, it can be stated that the four themes of the analysis partly overlap.
The same applies to the five categories that were condensed based on the units of
meaning in the interviews. Apparent differences and similarities emerge in the
comparisons between students and teachers in terms of codes within each cate-
gory. This provides indications of the answers to the research questions about stu-
dents’ study motivation. Figures 2 and 3 below give an overview of the codes that
were condensed from the empirical data in each group.
There is a complex interplay between results and motivation regarding what

teachers and students put in the concept of study motivation. Study results affect
motivation and vice versa both in a positive and in a negative sense. Grades also
affect the study motivation in different directions.
Figure 2. Picture of condensed codes within each theme of the teacher interviews
The importance of teachers in promoting students’ academic motivation is

evident. Teacher leadership is also a crucial factor in study motivation. However,
there is a difference between the interview groups; teachers point more to the
importance of knowledge, whereas students emphasize more personal qualities
such as being understanding, happy, and giving support.
An substantial similarity between teachers and students regarding study

motivation is that lessons or subjects that have more practical (life-skill-oriented)
content are motivating and that the students have and see the benefit of the

54
knowledge. A major difference between teachers and students’ responses is that

the teachers emphasized “life skills” in learning more, such as strategies regarding
goals, objectives, and sub-objectives, whereas the students did not touch on these
strategies at all. A consistent view between teachers and students is the
importance of well-being and security in the learning environment and that the
class, groups, or peers should offer a motivating environment. The teachers
pointed out the importance of adaptations and smaller groups in the learning
environment. The students believed that the external learning environment, such
as rooms and benches, also plays an essential role in study motivation.
The significance of social relations is confirmed by Ahlberg’s (2001)

communicative relation-oriented theory, which focuses on the concept of learning
environment in a broader sense. It concerns cognitive, perceptual, socio-
emotional, and socio-cultural aspects, as well as communicative and linguistic
interactions. Regarding the surrounding environment, both teachers and students
pointed out the importance of the right peers as a motivating factor. Peer friends
can help increase or decrease study motivation. Distinctive within this theme was
that teachers pointed to the right parental support, whereas the students pointed
out that parental support can vary and have different effects.
Figure 3. Picture of condensed codes within each theme of the student interviews
5. Conclusions
In this final part the conclusions reflect the research questions, and implications
from the results of the study are described.
5.1 Research conclusions

To understand the phenomenon of study motivation, we require a synthesis of
theories (Blomgren, 2016; Wery & Thomson, 2013) and practical studies in
collaboration with teachers (Hofvenberg, 2020). Motivation can be analyzed from

55
an interactive (Stensmo, 2005) or a transactional perspective (Perry et.al 2006). The

perspective in this article is broader than a mere focus on individual
characteristics.
The first and the second research question about teachers´ and students´
experiences about factors increasing/decreasing students’ study motivation, and
differences and similarities between the populations, are answered in the study
from different aspects.
The teachers appear as a very important group of actors for the students’ study
motivation. Teachers play a crucial role for study motivation, i.e. particularly in
how they convey expectations to students and organize classroom activities This
conclusion is also found in Stenmos’ (2005) and Giota´s (2017) argumentation that
teachers function as motivators by stimulating commitment and effort,
strengthening teaching conditions, supporting individuals and groups, and
shaping the classroom climate. The teachers’ approach, choice of didactic working
methods, leadership etc. are an important interactive motivating factor. This is
also confirmed in Blomgrens study (2016) where motivation is also linked to
pedagogical approaches, learning environments, didactic issues, and the
importance of teachers, and links this with planning of teaching, learning, and
views on knowledge. Blomgren clarified that teachers’ didactic action competence
is crucial for a successful school operation (cf Augustsson & Boström, 2016). In
contrast to Mahler et al. (2018) and Blomgren (2016) we found no evidence that
teacher's subject-specific enthusiasm was crucial for study motivation. The
students in this study instead emphasized the teacher's personality as a
motivating factor, while the teachers emphasized teachers' subject competence as
crucial.
The study clearly demonstrates the need for a practically oriented and laborative
pedagogy (cf. Boström & Bostedt, 2020; Boström 2013) such as pedagogy built on
learning-by-doing (Dewey, 1897). The students especially emphasized that when
the pedagogy was adapted to their way of learning, it was motivating. Thus,
teachers' leadership in the didactic space is an important competence for teachers
to be able to motivate students (Augustsson & Boström, 2016). Also need for
variation in teaching, was also confirmed by the results.
The result show a number of learning strategies which support study motivation
at a collective level, but these do not emerge as clearly at the individual. The
teachers emphasized the importance of adaptations and smaller groups in the
learning environment, whereas the students believed that the external learning
environment also played an important role in the study motivation. According to
Blomgren (2016), it is particularly important that “… students’ ability is strengthened
by support that aims to get students to use effective learning strategies and make an effort”
(p. 243). Blomgren, however, did not clarify what effective learning strategies are
meant to include. In this study, a number of strategies have emerged, such as
students' needs for structure, the teacher's personality, participation in planning,
choices and appropriate learning methods.

56
In addition, the empirical results show that teachers and students have slightly
different views on students’ ability and insight into taking responsibility for their
own learning. The teachers believed in the students' own responsibility whereas
students prefer to highlight pleasurable learning. We believe that the discrepancy
between students ‘and teachers’ views should be clarified, problematized, and
used in a constructive way to further explore the issue of study motivation.
The third research question about how is study motivation linked to various
aspects of education according to teachers and students, is highlighted both in
the theoretical framework used and the results of the study: motivating or
demotivating factors, motivational strategies that support or inhibit students,
learning environments that support or hinder students' study motivation and
“other” factors. Good learning environments are thus important in school, as
well as in other work-places or learning situations (Björklid & Fischbein, 2011).
To best support students’ ability to take responsibility for their own learning,
knowledge is needed about the ways that good learning environments can be
established because people interact, influence, and are influenced by the social
and physical environment
5.2 Implications
A broader perspective, on study motivation as a composite phenomenon that
affects internal and external motivational factors and the relationships between
them, then becomes significant. One conclusion drawn from the empirical
material is that teachers need to encourage inner motivation, while at the same
time ensuring that external motivation promotes learning (Anderman &
Anderman, 2009; Wery & Thomson, 2013). It is preferable to seek out lack of study
motivation in such factors as environments, learning strategies, teaching
planning, individual ambitions, home-school interaction, didactic choices, and the
physical environment (Giota, 2017). Both internal and external motivational
factors thus need to be taken into account.
To reconnect with motivation theories, the empirical evidence in this study points
unequivocally to the validity of the interactive, as well as the transitive
perspective. It proves that well-being and security in the learning environment
are important and that the class/ group/peers constitute a motivational context
for the students. The conclusions in this article point to the validity of the
interactive motivational perspective, in which the student’s own choice and
responsibility for school work and learning (internal motivational factors) must
be linked to external motivational factors. There is an interaction between
situation and person (interaction), in which processes concerning negotiations of
meaning in the social interaction (transaction) take place. Motivation can thus be
seen as a process integrated into a larger whole, impossible to separate from
learning, individual differences, the nature of tasks, or societal context.
Important research questions for further studies are to a) broaden the study to
further study programs to find variations, b) conduct observational studies in the
classroom to examine teachers' and students' interaction, c) deepen the interviews

57
with students with, for example, case descriptions or d) conduct case studies at
different schools to examine school cultures.
6. References
Ahlberg, A. (2001). Lärande och delaktighet [Learning and participation]. Lund:
Studentlitteratur.
Anderman, E. M., & Anderman, L. H. (2009). Motivating children and adolescents in Schools.
Columbus, OH: Merrill/Prentice Hall.
Antonovsky, A. (2005). Hälsans mysterium [The mystery of health]. Stockholm: Natur och
Kultur.
Aspelin, J. (2018). Lärares relationskompetens [Teacher’s relationship skills]. Stockholm: Liber.
Augustsson, G., & Boström, L. (2016). Teachers’ leadership in the didactic room: A
systematic literature review of international research. Acta Didactica Norge -
tidsskrift for fagdidaktisk forsknings- og utviklingsarbeid i Norge, 10, 1-19.
https://doi.org/10.5617/adno.2883
Björklid, P., & Fischbein, P. (2011). Det pedagogiska samspelet. [The pedagogical interaction].
Lund: Studentlitteratur.
Blomgren, J. (2016). Den svårfångade motivationen: elever i en digital lärmiljö [The difficult-to-
capture motivation: Students in a digital learning environment]. (Diss) Göteborg:
Göteborgs Universitet.
Boström, L. (2013). Hur lär sig elever på sex olika yrkesprogram? En studie om skillnader
och likheter i lärstilar. [How do students learn in six different vocational
programs? A study of differences and similarities in learning styles]. Utbildning &
Lärande, 4(1), 48-65.
Boström, L., & Bostedt, G. (2020). Hur lär sig elever på två olika yrkesprogram? En studie
om skillnader och likheter i lär- och undervisningsstrategier av betydelse för
elevers studiemotivation. [How do students learn in two different vocational pro-
grams? A study about differences and similarities in learning and teaching strat-
egies of importance for study motivation]. Kognition og Pædagogik, 116, 100-115.
Boström, L., & Gidlund, U. (2016). Students’ need for structure—the forgotten learning
styles preference. In N. Preston (ed.), Assessment, performance and effectiveness (pp.
12-20). Nova Science Publishers.
Covington, M. V., & Mueller, K. J. (2001). Intrinsic versus extrinsic motivation: An
approach/avoidance reformulation. Education Psychology Review, 13, 157-176.
Dewey, J. (1897). My pedagogic creed. Retrieved from
http://dewey.pragmatism.org/creed.htm
Elo, S., & Kyngäs, H. (2008). The qualitative content analysis. Journal of Advanced Nursing
62(1), 107-115. https://doi.org/10.1111/j.1365-2648.2007.04569.x
Farrington, C., Roderick, E., Allensworth, E., Nagaoka, J., Keyes, T., Johnson, D., &
Beechum, N. (2012). Teaching adolescents to become learners. The role of noncognitive
factors in shaping school performance: A critical review. Chicago: University of
Chicago Consortium and Chicago School Research.
Fereday, J., & Muir-Cochrane, E. (2006). Demonstrating rigor using thematic analysis: A
hybrid approach of inductive and deductive coding and theme development.
International Journal of Qualitative Methods, 5(1), 80-92.
https://doi.org/10.1177/160940690600500107
Gidlund, U., & Boström, L. (2017). What is inclusive didactics? Teachers´ understanding
of inclusive didactics for students with EBD in Swedish mainstream schools.
International Education Studies, 10(5), 87-99.
https://doi.org/10.5539/ies.v10n5p87

58
Giotta, J. (2017). Den svårfångade motivationen, elevers välmående och skolprestationer

[The hard-captured motivation, student well-being and school achievement].
Lecture 2017-10-30, Sundsvall.
Giota, J. (2013). Individualisering i skolan – vilken, varför och hur? [Individualization at school -
which, why and how?] Vetenskapsrådets rapportserie, 3. Stockholm.
Granström, K. (2012). Tre aspekter på lärares ledarskap i klassrummet. In G. Berg, F.
Sundh, C. Wede (eds.), Lärare som ledare [Teacher as leader](pp. 27-48). Lund:
Studentlitteratur.
Grönqvist, E., & Vlachos, J. (2008). Hur lärares förmågor påverkar elevers studieresultat [How
teachers ‘abilities affect students’ learning outcomes]. IFAU. Rapport 2008:25
Håkansson, J., & Sundberg, D. (2012) Utmärkt undervisning. Framgångsfaktorer i svensk och
internationell belysning [Excellent teaching. Success factors in Swedish and international
lighting]. Stockholm: Natur och Kultur.
Hattie, J. A. C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to
achievement. New York: Routledge.
Hofvenberg, A. (2020). Motivation, students, and the classroom environment. Exploring the role
of Swedish students´ achievement goals in chemistry. (Diss) Umeå: Umeå Universitet.
Hsieh, H.-S., & Shannon, S. (2005). Three approaches to qualitative content analysis.
Qualitative Health Research, 15(9), 1277-1288.
https://doi.org/10.1177/1049732305276687
Hugo, M. (2011). Från motstånd till framgång - att motivera när ingen motivations finns [From
resistance to success - to motivate when there is no motivation]. Stockholm: Liber.
Imsen, G. (2006). Elevens värld. Introduktion till pedagogisk psykologi [The student’s world.
Introduction to educational psychology]. Lund: Studentlitteratur.
Jerkeby, S. (2019). Att redan vara motiverad – om hinder och möjligheter för lärande [To be
motivated already - about barriers and opportunities for learning]. Lund:
Studentlitteratur.
Lundahl, L., Lidström, L., Lindblad, M., Lovén, A., Olofsson, J., & Öst, J. (2015) Osäkra
övergångar. Resultatdialog 2015 (pp. 107-116). Stockholm: Vetenskapsrådet.
Mahler, D., Großschedl, J., & Harms, U. (2018): Does motivation matter? – The relationship
between teachers’ self-efficacy and enthusiasm and students’ performance. PLOS
ONE, 13(11). https://doi.org/10.1371/journal.pone.0207252
Mayring, P. (2000). Qualitative Content Analysis. Qualitative Social Research, 1(2).
Niemivirta, M. (2004). Skillnader mellan flickor och pojkar i inlärningsmotivation. Skola-
kön-inlärningsresultat [Differences between girls and boys in learning motivation.
School-gender-learning results]. Helsingfors: Utbildningsstyrelsen.
Perry, N. E., Turner, J. C., & Meyer, D. K. (2006). Classrooms as context for motivating
learning. In P.A. Alexander & P.H. Winne (ed.), Handbook of educational psychology
(pp. 1-64). Mahwah, N.J.: Erlbaum.
Rising Holmström, M., Häggström, M., & Kristiansen, L. (2015). Skolsköterskans
rolltransformering till den hälsofrämjande positionen [The role of the school
nurse in transforming the health-promoting position]. Nordic Journal of Nursing
Research, 25(4), 210-217. https://doi.org/10.1177/0107408315587860
Ryan, R. M., & Deci, E. L. (2002). An overview of self-determination theory. In E. L. Deci
& R. M. Ryan (ed.), Handbook of self-determination research (pp. 3-33). Rochester, NY:
University of Rochester Press.
Skolverket. (2016). Uppföljning av gymnasieskolan [Follow-up of upper secondary school].
Skolverket: Stockholm.
Skolverket. (2018). Från gymnasieskola till högskola [From upper secondary school to university].
Rapport 466. Stockholm: Skolverket.

59
Skolverket. (2019). Redovisning av Skolverkets uppdrag om att genomföra verksamhetsnära

insatser för att förebygga avhopp från gymnasieskolan [Report on the National Agency
for Education’s assignment to implement business-related initiatives to prevent dropping
out of upper secondary school]. Dnr 7.2.1-2016:32
Stensmo, C. (2005). Ledarskap i klassrummet [Leadership in the classroom]. Lund:
Studentlitteratur.
Sveriges Elevkårer & Lärarnas Riksförbund. (2015). Från avhopp till examen - en
undersökning bland gymnasieelever och lärare om faktorer som påverkar
genomströmningen i gymnasieskolan [From drop-out to graduation - a survey among
high school students and teachers about factors that affect the throughput in high school].
Stockholm.
Sveriges Kommuner och Landsting (SKL). (2017). Olika är normen. Att skapa inkluderande
lärmiljöer i skolan [Different is the norm. To create inclusive learning environments in
the school]. Sveriges kommuner och Landsting.
Säljö, R. (2014). Lärande i praktiken. Ett sociokulturellt perspektiv [Learning in practice. A socio-
cultural perspective]. Lund: Studentlitteratur
Vaanstenkiste, M., Lens, W., & Deci, E. L. (2006). Intrinsic versus extrinsic goal contents in
self-determination theory. Another look at the quality of academic motivation.
Educational Psychologist, 41, 19-31. https://doi.org/10.1207/s15326985ep4101_4
Valsö, M., & Malmgren, F. (2019). Fysisk lärmiljö: optimera för trygghet, arbetsro och lärande
[Physical learning environment: optimize for security, work peace and learning]. Lund:
Studentlitteratur.
Vetenskapsrådet. (2017). God forskningssed [Good research practice]. Vetenskapsrådet.
Wery, J., & Thomson, M. (2013). Motivational strategies to enhance effective learning in
teaching struggling students. British Journal of Learning Support. 28(3), 103-108.
https://doi.org/10.1111/1467-9604.12027
Woolfolk, A., & Karlberg, M. (2015) Pedagogisk psykologi [Individualization in a school con-
text]. Edinburgh: Pearson Education Limited.
X Municipality, (2016). Mål och resultatplan 2016 [Goal-and result plan 2016]. Barn- och
Utbildningsnämnden.
Zimmerman, F. (2018). Det tillåtande och det begränsande. En studie om pojkars syn på studier
och ungdomars normer kring maskulinitet [The permissive and the restrictive. A study
on boys ‘views on studies and adolescents’ norms about masculinity] (Dissertation).
Göteborg: Göteborgs universitet.

60

Vol. 19, No. 8, pp. 60-80, August 2020
The Dragon, the Knight and the Princess:

Folklore in Early Childhood Disaster Education
Maila D. H. Rahiem and Husni Rahim

UIN Syarif Hidayatullah, Jakarta, Indonesia
https://orcid.org/0000-0002-5618-2486
https://orcid.org/0000-0003-1797-5018
Abstract. This research investigated the use of folklore in early

childhood disaster education. A systematic analysis of the literature was
used to evaluate early childhood disaster education. The researchers
identified different concepts present in the literature; and
examined patterns, parallels, and regularities. The researchers
synthesized the principle of folklore use in early childhood disaster
education as part of this research. Folklores have a tremendous potential
to make early childhood disaster education effective if interpreted,
integrated, and demonstrated by science. This work provides a strong
foundation for further study into the same research issue by using
empirical data or research into how to make folklore an efficient tool for
early childhood disaster education.
Keywords: folklore; culture; disaster risk reduction; disaster prevention

and preparedness; early childhood education
1. Introduction
Children often tend to be the most affected and vulnerable to disasters (Kousky,
2016; Tanner, Lazcano, Lussier & Polack, 2009; Winser et al., 2004). This
vulnerability occurs both during and after a disaster. Young children are
physically vulnerable to unexpected and chronic disasters due, in part or in full,
to adult dependency (Peek, 2008). Disasters often result in the constant disruption
of children's lives, from families, schooling, accommodation, health care,
friendships and other core areas of their lives; they face the risk of parental
separation, relocation, trauma, illness, and death (Fothergill, 2017). Disasters
interfere with children's psychological health and social functioning (Norris et al.,
2002), cause children to suffer from depression, while a severe trauma could affect
them for many years to come in terms of their mental health and wellbeing (Bryant
et al., 2018). The psychological effects differ greatly across a vast range of
challenges, spanning a long time, and are linked to the nature of the event; most
people heal soon, while some progress to psychopathology, including post-
traumatic stress disorder (PTSD), severe depression, anxiety disorders, and even
addiction problems (Shultz, 2014). Disasters impair the long-term learning ability

61
of children (Gibbs et al., 2019). Children typically experience emotional and

behavioral issues, nightmares and trouble sleeping, and stress-inflicted anxiety
attacks (Somasundaram & van de Put, 2006).
Children need protection during a disaster, but children do have specific

capacities that can lead to efforts by the home and community to minimize the
disaster risks and impact (Fothergill, 2017). Initiatives to plan for disasters and
mitigate risks can be implemented by people to eliminate long-term social and
economic disturbances from the impact of these hazards. Education is regarded
as one of the best media to create a society that is ready for disasters. Using
education, we can raise awareness and understand the danger among children,
teach readiness, and demonstrate how to respond in times of disaster (Twigg,
2003). Children's disaster education initiatives will reinforce disaster
preparedness and resilience for both children and families (Torani, Majd, Maroufi,
Dowlati & Sheikhi, 2019).
Curriculum on Disaster Risk Reduction (DRR) can enhance pre-school quality,

engage children in DRR programs (Proulx & Aboud, 2019). It would also increase
children's understanding of disaster risks and preparedness in ways that might
reduce such disaster risks (Amri, Haynes, Bird & Ronan, 2018; Johnson, Ronan,
Johnston, & Peace, 2014; Proulx & Aboud, 2019). Disasters can happen when
children are at home or in school, when they are with their family or friends or
even alone. It is important to involve children in a disaster rescue plan and ensure
they know what to do when or if a dangerous situation occurs. Encouraging
children to think about the importance of precautionary action and preparedness
can narrow the gap between knowledge and action (Bosschaart, van der Schee,
Kuiper & Schoonenboom, 2016; Faber et al., 2014), and save many young lives.
They may not fully understand everything and only comprehend a few points, as
per their ability to cognize it. So, it is necessary to determine both what to teach
and how to teach children these skills and knowledge.
According to studies conducted in various countries, including Japan, there is a

direct link between education, increased risk perception, and students' risk
reduction measures (Torani et al., 2019). Disaster lessons are usually integrated
into curricula, including science, health, geography, literature and civic education,
and are organized according to the students' level. In Indonesia, it is integrated
into science, social studies, language, mathematics or even religious studies
(Rahiem, 2018). Thirty-nine states in the USA, using the Michigan Model for
Health, are integrating disaster preparedness curricula into their School Health
Program (Eisman et al., 2020). Natural disasters are addressed in geography
lessons in Australia and New Zealand (Duffy, 2014). In the United Kingdom,
disaster education is incorporated into textbooks used in science and geography
(Sharpe & Kelman, 2011). In Taiwan, a flood game was introduced to encourage
the active learning of players through exploration (Tsai, Wen, Chang & Kang,
2015). In early childhood settings, Iran's disaster education includes posters,
videos, role-plays, exercises, and games (Izadkhah & Hosseini, 2005); while in the
Philippines, teachers use animation videos, songs, and storybooks to teach about
disasters and preparedness for disasters (Ani, Daquio & Aquino, 2015).

62
Since the devastating earthquake and tsunami that affected several Asian
countries on December 26, 2004, studies have been conducted concerning the use
of local culture in disaster prevention programs. Donovan (2010) highlighted the
need to investigate the connection between cultural reactions and natural events,
otherwise known as 'geoculture.' Bankoff et al. (2013) argued that the root causes
of vulnerability lie within the past and progress of our society; therefore, culture
has a significant role in society – the relations between the world and our ultimate
survival. Jha and Jha (2011) emphasized the importance of indigenous knowledge
as a valuable tool that can promote the prevention, preparedness and response to
disasters in cost-effective, participatory and sustainable ways. Integrating shared
local values, common day-to-day experiences and local communal memories into
risk communication strategies and behavioral guidelines can be effective in
fostering citizen disaster preparedness (Appleby-Arnold, Brockdorff, Jakovljev &
Zdravković, 2018).
One form of culture that has been discussed as a potential for promoting disaster
prevention, preparedness, and responses is folklore. Some work has been
conducted that explores how indigenous people survived the destructive tragedy
as they learned about tsunamis through stories or songs that are part of their
culture. Indigenous people, including Thailand's Moken, Indonesia's
Simeulueans and many island groups on the Andaman and Nicobar Islands, were
more prepared as they heard from their predecessors about tsunamis.
Local knowledge of previous tragedies had been passed down from generation to
generation through word of mouth. As a result of this, when the ground began
trembling and the tide went out further than ever, the communities were aware
of the initial warning signs of earthquake and tsunami, and were able to react
accordingly before it was too late. Consequently, they had time to flee inland to
higher ground to effectively avoid the direct effects of the disaster (Mercer et al.,
2012). Culture has been a factor in the recovery of populations from disasters
where, in some cases, culture has served as a deterrent to successful DRR activities
(Kulatunga, 2010).
Previous scholars have discussed the importance of early disaster preparedness

education for children (Amri et al., 2018; Bosschaart et al., 2016; Faber et al., 2014;
Johnson et al., 2014; Proulx & Aboud, 2019); the role of culture in disseminating a
major program such as disaster preparedness and response (Appleby-Arnold et
al., 2018; Bankoff, 2013; Donovan, 2010; Jha & Jha, 2011; Kulatunga, 2010); and
how folklore, as one of the sources of local culture, has saved people from
disasters as they were more conscious of the hazards of the disasters and also
learned what to do when they occur (Mercer et al., 2012) . To the best knowledge
of the researchers, scholars have not looked at using folklore in early childhood
disaster education. As a result of this gap in existing knowledge, the researchers
investigated the idea of using folklore in disaster education in early childhood in
this study.
This study is significant in providing an overview of the possibility of

implementing a disaster prevention program in the kindergarten. This research is
also becoming a preliminary or background study for further exploring the
methods and strategies of using folklore in early childhood disaster education.

63
2. Background and Rationale

The researchers reviewed two sets of literature to establish the conceptual
framework within which the study was conducted, namely: early childhood
disaster education, and folklore in early childhood education.
2.1 Early childhood disaster education

Disaster education, which encompasses disaster risk management, prevention
and preparedness measures, is one method for minimizing negative disaster
impacts (Mulyasari, Takeuchi & Shaw, 2011). According to the 2005-2015 Hyogo
Framework for Action (United Nations, 2005), the goal of disaster education is to
develop a culture of protection and resilience at all levels and reduce the adverse
social and economic effects of hazards. Disaster awareness seeks to provide
information for individuals and communities to take steps to minimize their
vulnerability to disasters (Torani et al., 2019); to mitigate the risk concerning
hazardous incidents by helping children and families learn about both exposure
and building processes (Ronan, Alisic, Towers, Johnson & Johnston, 2015).
Education may increase children's perception of risk (Shreve et al., 2014). Disaster
education should be specifically discussed as a means of improving child
resilience and transmission of information to minimize the risk of disasters in their
homes (Torani et al., 2019). Historically, disaster awareness initiatives and the
media have provided adults with disaster risk information and ways to protect
their families, such as developing family emergency plans, securing home and
rental insurance, and storing food, water, and supplies (Mileti, 1995). Specially
designed disaster education for children is also required. Children have a unique
understanding of risks, and risk perceptions differ according to their age and also
gender (Haynes & Lassa, 2010; Tanner et al., 2009).
Starting education at an early age is suggested because people never forget what
they learn at an early age (Torani et al., 2019). According to the United Nations
Children's Fund (United Nations Children’s Emergency Fund, 2011), children's
disaster education services are meant to contribute to a dramatic shift in mentality
and attitude and a behavioral improvement towards a more effective disaster
reduction approach. If children are presented with the right information and
resources and are assisted by adults, they have a greater chance of success. This
notion is significant because almost one-third of the world's population is
children, and today's solutions can provide long-term impacts to today's children
and their societies.
School is the perfect place to prepare pupils for emergency preparedness. Schools
have both the physical resources and the personnel to respond to an emergency
quickly. They can assist the communities in their efforts to carry out risk analysis
and prevention of disasters. In the event of a disaster, schools are well-positioned
to play a wide variety of positions as emergency response and relief centers,
communication centers, supply depots and hubs of managing partner (Mutch,
2014). Citizens need to be prepared to face disasters and develop detailed
educational programs (Torani et al., 2019).

64
2.2 Folklore in early childhood education

Narratives have been commonly utilized in early years’ education as a useful tool
for the development of spoken language and literacy. Narratives support the
development of literacy, as they help children to learn ideas on how to use
language (Maureen, van der Meij & de Jong, 2018); and to encourage oral skills
(Nicolopoulou, Cortina, Ilgaz, Cates & de Sá, 2015). A narrative can also
encourage writing skills by inspiring children to create their own stories, change
stories they have read, and even write plays based on popular tales (Cassell, 2004;
Nicolopoulou et al., 2015)
Narratives are often considered advantageous due to the support they provide to
many other facets of children's growth, including cognitive, physical, emotional,
and spiritual. A narrative can power cognitive involvement, critical thinking, and
sequencing of stories (Agosto, 2016). It is a powerful tool for promoting self-
understanding (Lenox, 2000), fairness and diversity (Flewitt, 2017) and promoting
inclusive classrooms (Mardell & Kucirkova, 2017). Narratives are also used
frequently in kindergarten for Moral Education (Gunnestad & Thwala, 2011;
Rahim & Rahiem, 2013; Thambu, 2017; Thompson, 2011; Woodard, 2005).
Narratives depict social interactions, friendships and interpersonal relationships.
Children also learn from the experiences of the characters in narratives because
they offer an image of an individual's life and mind in the form of words (Rahiem,
Abdullah & Krauss, 2020).
Throughout this study, one type of narrative is explored in greater depth, folklore.
Dan Ben-Amos famously suggested a definition of folklore as artistic
communication in small groups in his seminal essay, "Toward a Definition of
Folklore in Context" (Ben-Amos, 1971). A folklore, in its traditional and
conventional form, is transmitted orally in most cases and serves as shared
tradition-based creations of a particular culture. The manifestations of modern
folklores include folk festivals, numerous ensembles of traditional songs and
dances, or folklore festivals held around the world (Harvilahti, 2004). Nowadays,
folklores are not just found in books, but also include interactive folklore stories
or folk-based movies. Ben-Amos (2014) re-conceptualized the meaning of folklore
as a specific kind of communication that is distinct in terms of language, image,
sound, motion and performance.
Folklore is one of the most popular children's narratives used for language
programs in pre-school, and has the power to influence children, teach behavior
or good practice and is very similar to the lives of individuals. Researchers
observed in many cultures, folklore is associated with stories of dragons, warriors,
and princesses. For example, in Indonesia, Balinese, Javanese and Kalimantanese
folklores depict an earthquake that is associated with an angered dragon. Another
story involves a princess who resided in Mount Rinjani, Lombok, which, when
she was disturbed by human attitudes, would lead to a volcanic eruption that
destroys nature. A story from Siau, North Sulawesi, involved the main character,
Sakeha, who was a brave warrior after a tsunami had occurred and helped
everyone that had been affected to recover. Folklore is important in the study of
repeatable practices that people deem conventional, connotative, and significant.
Folklore often includes ideas that go into creating a culture at different levels, from

65
person to nation (Bronner, 2019). Individually and socially, humans are

continually in contact with different cultural powers and have a psychological
need for tradition, and the reshaping of common practices (Bronner, 2011).
Cultural psychologists theorize that, through interaction with people, material

and symbolic artifacts, children grow into cognitively capable adults within their
cultural groups (Tomasello, 2009). Folk tales constitute an essential part of the
everyday life and traditions of many societies. These are used to reflect and
illustrate particular social environments from which children learn social
behavior patterns as interactive communicative events. Culture stimulates the
mind, while folktales are cultural memories (Olajide, 2010). On the other hand,
literacy seeks to make the human mind more functional so that it can most
rewardingly solve life's problems. Agbenyega et al. (2017) examined the role of
African folklore, specifically Ghanaian folk stories, in the development of child
cognition and social life concept formation. They found that traditional oral
storytelling can be an important part of early childhood education to improve
critical thinking about social life for children.
By using folk stories to construct a dramatization of an imaginary scenario that is

culturally prized, the children are on stage acting in the drama, and the audiences
are watching the drama. They can see and hear the message within the story
despite not having witnessed it themselves directly. This method of delivery helps
children switch from a lower mental function to a higher mental, as they form
conceptions of values that are central to social life function (Fleer, 2013). It shows
that early childhood educators can use folk tale dramas and video with a
stimulated recall to help children participate in imaginative recall to give them
new insights. What about using folklore to involve children in the imaginary
recollection of disaster incidents, giving them new insights into what to do if an
emergency occurs so that they can respond with the awareness of the incident and
the best way to defend themselves and others? The researchers used this thinking
as one of two starting concepts to further examine and develop. Recognizing the
importance of early childhood disaster education and that folklore is a valuable
medium for early childhood education, the researchers focused on examining the
use of folklore for early childhood disaster education.
As a result of this framework, the concepts that were formed, which the
researchers proposed to discuss conceptually, included:

66
Folklore in
early
childhood
education
Early
childhood
disaster
education
What about using folklore in early

childhood disaster education?
Figure 1: Conceptual Framework
3. Methodology
What about using folklore in early childhood disaster education? This is the
emerging question for this conceptual paper. The researchers explored this
question and built a hypothetical concept via literature research and critical
thinking.
A conceptual paper is a powerful means of theory building (Jaakkola, 2020).

Conceptual review papers can theoretically enrich the field by reviewing extant
knowledge, noting concerns and inconsistencies, identifying important gaps in
knowledge, key insights, and proposing agendas for future research (Hulland,
2020). The result of this process is a theoretical contribution that refines, re-
conceptualizes, or even replaces existing ways of viewing a phenomenon.
This conceptual paper is the base study for the researchers’ larger empirical study
on folklore for early childhood disaster education. Conceptual papers bridge
existing theories in interesting ways, link work across disciplines, provide multi-
level insights, and broaden the scope of our thinking; another important
consideration is the need to create rational and detailed arguments about these
connections, rather than only testing them (Gilson & Goldberg, 2015).
In this conceptual paper, arguments are not derived in the conventional sense
from empirical data, but involve assimilating and integrating information in the
form of concepts and hypotheses that have been formed beforehand. Researchers
examined previous empirical studies on the use of folklore for disaster education,
then built concepts and theories based on the results of these previous studies.
The discussion and analysis of this conceptual paper used one of four templates
for conceptual papers offered by Jaakkola (2020). The four templates are Theory
Synthesis, Theory Adaptation, Typology and Model. Due to the respective aims,
approach for using theories and contribution potential, this paper employs the
Theory Synthesis as the template.

67
A theory synthesis paper offers conceptual convergence through several

hypotheses or sources of literature. These papers provide a new or improved view
of a concept or phenomenon through a unique linking of previously unconnected
or incompatible parts. According to MacInnis (2011), summarizing helps
researchers by encapsulating, digesting, and reducing what is known to a more
concise and manageable area of study. The integration helps to enable researchers
to see a concept or phenomenon in a new way by transforming previous findings
and theory into a novel higher-order perspective that links phenomena which
may have been previously considered distinct (MacInnis, 2011). Such papers may
also explore the conceptual underpinnings of an emerging theory or explain
conflicting research findings by providing a more parsimonious explanation that
pulls disparate elements into a more coherent whole.
To explore the use of folklore for early childhood disaster education, the
researchers started by examining concepts of disaster education in early
childhood education and folklore for early childhood education. These two
concepts became the focus for further investigation. Then, the researchers
investigated the focal phenomenon that is not adequately addressed in the
existing research. The focal phenomenon was the use of folklore stories for early
childhood education. The researchers identified differing conceptualizations of
that phenomenon by analyzing literature that researchers gathered from Google
Scholar using the keywords folklore disaster and folktale disaster. The studies
observed did not specifically discuss the level of early childhood. The researchers
examined the patterns, resemblances and regularities in the observed premises,
looked for an explanation from the theoretical framework, and then finally
concluded the study.
As a result of this framework, the following concepts were formed, which the
researchers proposed to discuss conceptually:
Figure 2. Methodology

68
4. Findings and Discussions

Folklore has acquired considerable popularity among community-based DRR
practitioners as one source of local knowledge for disaster mitigation (Mercer et
al., 2012). Throughout this study, the researchers gathered and identified concepts
from previous studies on the use of folklore for disaster prevention, preparedness,
response, and disaster education programs. These gathered articles are not unique
to early childhood education, while there also did not appear to be any specific
research into the use of folklore in education in early childhood crises. This
resulted in an expansion of the scope of existing research. The researchers
searched for published Google Scholar studies using the keywords: folklore
disaster education, folklore disaster mitigation, folklore volcano mitigation,
folklore tsunami mitigation, folklore earthquake mitigation, folklore climate
change, folklore environment, and folklore indigenous knowledge disaster.
Fifteen articles were found that discussed natural disasters and folklore. Of the
fifteen documents, only two addressed education, and even these were not
comprehensive. The articles were reviewed before the relevant concepts were
described inductively. Three key concepts emerged from the mentioned literature,
namely: 1) the disaster risk management that has six subgroup concepts; 2)
disaster recovery and resilience that consists of two subgroup concepts; 3)
indigenous knowledge and science that consists of two subgroup concepts.
Disaster risk Disaster recovery and Indigenous knowledge

management resilience and Science
Disaster risk Resilient Dialogue of

reduction communities indigenous
knowledge and
science
Conserving the
Disaster recovery
environment Folklore is
superstition
Mitigation and
evacuation
Early warning
system
Disaster
awareness
Disaster
Prevention
Figure 3. Emerging Concepts
4.1. Disaster Risk Management

There are seven articles on folklore that discuss disaster management—three
articles from Indonesia, two India, one from the USA and another from Japan.
Gadeng, Maryani and Rohmat (2018) explored the indigenous knowledge Smong
(folklore) in the community of Simeulue. Qualitative methods were used,

69
including data gathering using in-depth questioning, participatory evaluation,

analysis of the documentary and literature. The study revealed that the
indigenous knowledge of Smong is a tsunami early warning system introduced by
the Simeulue community in the forms of manafi-nafi (folklore), mananga-nanga (a
cradle song often used to calm a baby to sleep) and nandong (humming). During
the 2004 Tsunami, Smong helped save uncountable lives on Simeleu Island. The
death toll on the island of Simeleu was just seven people, while on the island of
Aceh, which was close by, the death toll reached 170,000.
Sukmawan and Setyowati (2017) examined Indonesian folklore stories that

include environmental messages, and explores the understanding of high school
English teachers in using Indonesian folklore to teach language. Content analysis
and interviews were used to assess the applicability and suitability of folklore for
foreign language lessons. They collected 17 different Indonesian folklores with
messages about the environment, either implicitly or explicitly. The messages are:
tragedy occurs due to the ignorance and violence of humans against nature, and
it is the man himself who can keep the disaster from occurring. Folklore help the
teacher prepare students to read, speak and write foreign languages; develop
students' knowledge of environmental protection; improve cultural literacy;
foster positive behavior, and practice critical thinking.
Troll et al. (2015) investigated the local climate at the Merapi volcano and
combined this knowledge with recent developments in volcanology and volcanic
control sciences. Researchers incorporated Merapi's historical, cultural, and
scientific analysis in investigating Merapi folklore's geological background and
obtained an enhanced underlying of Merapi's long-term behavior. Researchers
also studied the folklore's significance in strengthening the local communities'
resilience and responsiveness. Troll et al (2015) interpreted how the ancient people
used the Merapi – Kraton – South Sea axis in local folklore to explain and
rationalize the dynamic interplay of geological processes. Though now enveloped
in mystery, these oral traditions may be considered an ancient disaster mitigation
resource, which makes them possibly useful in helping to promote successful
dialogue with a range of target parties and community groups along the slopes of
the volcano.
Jha and Jha (2011) looked at the traditional knowledge of the Lepcha tribal
community of Sikkim Himalayas on disaster management. Techniques for data
collection include general observations over a 14-year period (1996-2010) and
structured and unstructured interviews with indigenous Lepcha people. They
assumed that the Lepcha people of Sikkim developed intricate philosophies further
to understand the existence of disasters and their causes. These beliefs appear
folkloristic as depicted in their stories and are based on sound principles that help
guide actions and behaviors that can actively communicate and limit the impacts
of disasters. Such philosophies also help people build close-knit social networks
that make them more resilient when a tragedy affects them.
Pareek and Trivedi (2011) examined how people in the tribal societies in Rajasthan
view and handle natural hazards and adverse weather conditions, including their
methods for early warning and dealing with future events, as well as their views

70
of their short- and long-term ecosystem impacts. Investigation approaches include

analysis of the literature as a qualitative study, accompanied by informal
conversations, open interviews, focus group interactions, workshops, cultural
mappings and extensive interviews. The study showed that indigenous
knowledge systems have allowed the different cultures in the region to live in
harmony with their ecosystem for generations and that their traditional
knowledge systems are valuable resources for conserving the environment and
handling natural disasters.
Becker, Johnston, Lazrus, Crawford, and Nelson (2008) described the role of
traditional knowledge and how it can assist in managing emergencies. They
explored a specific case study in which a traditional Native American tale was
paired with modern hazard prevention techniques to produce a tsunami hazard
awareness film. They claimed that traditional knowledge can be successfully used
to conduct disaster education and improve alert response. The video, entitled
"Run to Higher Ground!" is an example, and indigenous communities and the
public at large (both in the United States and internationally) have readily taken
this as an instructional resource.
4.2. Disaster Recovery and Resilience

Four further articles discuss how folklore is useful for disaster recovery and
resilience.
King, Goff, and Skipper (2007) assessed the range of Māori Environmental
Knowledge (MEK) relating to natural hazards and discusses this specific
information for management and mitigation in New Zealand. Researchers
discussed myths, poems, place names and legends related to natural disasters
through a thorough examination of documents in literature. Māori narratives and
songs convey their vulnerabilities and resilience. Oral traditions, place names and
environmental indices include important sources of expertise and awareness that
are ignored. The study discovered despite the difficulties of using MEK, this
environmental experience has some contributions to promoting the preparedness
and prevention of contemporary natural hazards in New Zealand.
Cashman and Cronin (2008) investigated ways in which language, imagery, and
metaphor explain volcanic events and may connect diverse cultures (both present
and past) in their attempt to understand volcanic catastrophes. New Zealand
Māori's have introduced volcanoes into the tribal and individual lineages. The
study revealed Local tradition may provide a powerful community education
resource and an important way of helping individuals and societies heal
psychosocially from volcanic disasters.
Parsizadeh, Ibrion, Mokhtari, Lein, and Nadim (2015) examined how the
socioeconomic environment and community values affected the understanding of
earthquake risk and readiness of populations in Bam, Iran, before the major 2003
earthquake catastrophe. Data were obtained during two weeks in September 2013,
nearly ten years after the Bam 2003 earthquake, using in-depth interviews with 30
earthquake survivors in the Bamand Baravat districts. They observed that Poems,
folktales, oral traditions, and legends in Iran have considerable ability to alert the

71
disasters and recall past seismic incidents and to help create and establish an
"earthquake culture" in Iran.
Birkalan’s study (2001) describes the accounts of experiences at Story House —

Masal Evi, Turkey, where researcher and teams used traditional folktales as coping
resources to help children overcome the hardships of this massive tragedy. The
storytelling events included about 35 children aged 3-15 years. The study
suggested that fairytales are useful instruments for children to learn, socialize,
and interact with their outside world, particularly in a post-earthquake crisis.
Using folktales as narratives in their own right required making those activities
intended for children. This also meant providing the children with an "imaginary"
space to express the inexpressible and say the unthinkable.
4.3. Indigenous Knowledge and Science

Four articles were also found that referred to traditional knowledge and science
for disaster mitigation.
King and Goff (2010) explored oral Māori traditions and how science explains the
incidents portrayed in Māori tales. The study explained the crucial aspect that
blends MEK and contemporary science to create new theories on massive
environmental disruption along the coasts of Australia and New Zealand. The
initiative would require not only recognition of other ways of knowing but also
open dialogue with Māoris, and respect them to share their own stories.
Dube and Munsaka (2018) looked at the importance of indigenous knowledge to

Zimbabwe's disaster risk management practices. Interviews and observations
were used to collect data from 40 research participants from the district of
Tsholotsho in Matabeleland, northern Zimbabwe. They noticed that local
communities use the indigenous knowledge available to incorporate systemic
steps as part of disaster risk management strategies. Nevertheless, this indigenous
knowledge is often shunned by practitioners who say that indigenous knowledge
lacks evidence, contextualized to particular populations and that knowledge
cannot be confirmed scientifically.
Ludwin et al. (2007) examined local myths and folklore related to earthquakes,
landslides, and tsunamis in Cascadian oral traditions (part of North America’s
northern Pacific coast) and Japanese written traditions, especially in the Edo area
(now Tokyo). The study revealed the visual vocabulary of folklore and mythology
could be a valuable addition to modern geological evidence to create a reliable
historical record of volcanic development.
Orihara, Kamogawa, Noda & Nagao (2019) investigated the Japanese folklore that
says rare sightings of deep-sea fish are a reference to the earthquake. If this legend
is proved real, the presence of deep-sea fish may be valuable knowledge for
minimizing disasters. The researchers created a directory of newspaper accounts,
scholarly papers, and the naval museum. They analyzed the details on the
presence of deep-sea fish for the prevention of disasters. The study discovered the
spatiotemporal link between the actions of deep-sea fish and the earthquakes was
barely noticed from this study. Therefore, this Japanese myth is believed to be a
superstition due to the illusory connection between the two incidents.

72
4.4. Theory Synthesizing

These fifteen papers, explained above, contain three critical points:
1. Folklore is one type of indigenous or traditional knowledge that has been
used since ancient times to warn about an impending catastrophe and to
educate about mitigation.
2. Folklore is often believed to be the source of healing after a catastrophe
has occurred.
3. Indigenous information such as that in a folklore needs to be combined
and clarified by science to provide the provision.
Based on these findings, the questions remain concerning using folklore in early
childhood disaster education. The researchers analyzed the trends, differences
and inconsistencies of the reviewed literature to synthesize theory on using
folklore in early childhood disaster education.
Traditional or indigenous or local knowledge is a system of experiential

information gained through constant environmental observation and contact.
Many communities still retain this kind of knowledge, and this knowledge can
make a major contribution to emergency management for natural hazards.
Traditional knowledge can help understand the nature of local hazards, suggest
appropriate mechanisms for risk reduction and response, and even provide
recovery options based on past experiences (Becker et al., 2008).
Local communities have continued to rely extensively on their systems of

traditional knowledge in observing environmental and managing natural
disasters. These communities have collectively created a large body of
information on mitigation and prevention of disasters, early warning,
preparedness and response, and recovery from disasters. This knowledge is
acquired by observation and research and is also based on cumulative wisdom
handed down from generation to generation (Grenier, 1998). Human reactions to
hazards are affected by social, socioeconomic and cultural influences;
societies evolve their unique approach. Consequently, a complete and unbiased
vulnerability and risk assessment must take into account the cultural and social
variables involved (Cashman & Cronin, 2008; Donovan, Suryanto & Utami, 2012;
Harris & Ripepe, 2007)
Traditional knowledge of disasters can assist in emergency mitigation by

identifying unique characteristics of documented local hazards and, in some
situations, by providing information on disaster response processes (Becker et al.,
2008). Traditional wisdom in the context of oral histories, folklore, mythology, and
songs could be a cross-cultural appealing and a powerful way to convey
contemporary messages about the danger and possible disasters.
Hiwasaki, Luna, Syamsidik, and Shaw (2014) assured that local and indigenous
knowledge are key to enhancing communities' resilience to hazards and climate
change impacts. Nevertheless, scientists, practitioners, and policy-makers have
yet to develop a structure of it. They believe that such knowledge needs to be
integrated and clarified with science and technology before it can be used in
disaster risk reduction policies, education, actions and adaptation to climate

73
change. Integrating traditional and indigenous knowledge with science is an

important mechanism that enables practitioners and scientists to carry out
research and practices to improve communities' resilience. The integration also
helps decision-makers to bring measures that enable these policies into practice.
Such programs encourage the use of local and indigenous knowledge and enable
communities to use their knowledge supplemented with outside knowledge to
make educated choices on the implementation of their plans for mitigation and
disaster risk reduction.
Mercer et al. (2012) suggest that local and "Western" knowledge should be
incorporated and used in conjunction with each other to derive the best aspects of
all understandings, in order to produce the most successful outcomes. They
further propose that cautious community consultation and engagement is the best
way to build solutions that accommodate this combination and thereby boost
resilience. Local and indigenous knowledge that helps societies develop their
resilience, which cannot be clarified or combined with science, is classified
separately. Communities may continue to practice that knowledge, free from
criticism from experts, politicians, and professionals (Hiwasaki et al., 2014).
Folklore has the immense potential to make a successful contribution to early

childhood disaster education. Besides entertaining, folklore is a strong
socialization and communication device that can mold and enhance traditions,
convictions, values and attitudes (Cocks, 2013). However, traditional and
indigenous knowledge has to be combined and clarified with science before it can
be used in disaster risk management and climate change policies, education, and
initiatives. There should be empirical evidence in the stories for using folklore in
early childhood disaster education; children should not blindly believe anything
that has not been proven to be right. Children need to know if a story that is being
told might not be true or is unproven. Teachers and parents should be
knowledgeable about the tale and the facts behind it, and therefore the folklore is
indeed insightful and could teach children better to prevent, prepare, respond and
recover after a disaster has occurred.
Below is an illustration of how the researchers underwent this systematic process

until a conclusion was reached.

74
Culture has a significant role to promote disaster prevention, preparedness and response (Bankoff et
al., 2004; Donovan, 2010; Kutalunga, 2010; Jha & Jha, 2011; Appleby-Arnold et al., 2018)
Folklore is
important in the Education is one of the best media to create a society that is prepared for
study of stylized, disasters (Twigg, 2003; Torani et al, 2019).
ritualized, and
sometimes
structured Children can benefit from DRR programs, in
repeatable improving knowledge and understanding of
activities disaster risks and preparedness (Johnson et al.,
(Bronner, 2016) 2014; Amri et al., 2018; Torani et al., 2019 ).
Folklore may Children are the worst affected and most vulnerable to
help improve School is the
perfect location disasters (Wisner, Blaikie, Cannon, & Davis, 2004; Peek,
children's critical
thinking about to educate 2008; Tanner et al, 2009; Kousky, 2016)
social life children for
(Agbenyega et al, disaster
Children show considerable
2017); and preparedness Children are creative social beings
and often overlooked
enhanced (Mutch, 2013. and active agents and have played
capabilities, skills and
memory to help an important role in the
strengths that can lead to preparation and recovery of their
children indulge efforts to minimize disaster families and communities
in imaginative risks and impacts (Fothergill, (Fothergill, 2017).
recall to give 2017)
them new
insights (Fleer,
2013)
GATHERED & REVIEWED ARTICLES
Folklore disaster education Folklore disaster mitigation
Folklore
Folklore Folklore Folklore
Folklore Folklore indigenous
volcano tsunami earthquake
climate change environment knowledge &
mitigation mitigation mitigation
Disaster
Analyze the patterns, parallels, and regularities of the fifteen collected literature
Folklore has been used for Folklore needs to be

Folklore has been used for
disaster prevention, communicated with, integrated
recovery and building resiliency
preparedness, and reponse to, and clarified by science
Synthesize Theory/Answering Question
What about using folklore in early childhood disaster education?
Folklore has an enormous potential to make early childhood disaster education

successful if it is also conveyed, incorporated and explained by science
Figure 4. Process of analysis and theory synthesizing

75
5. Conclusion
Traditional local knowledge, like folklore, is found to be a strong communication
tool to help educate and mitigate disasters. In post-disaster circumstances, many
people also use folklore to help the community rebuild and become more resilient.
However, scholars believe that if folklore is not conveyed, combined and
explained with evidence, it functions less efficiently. This paper looks at the
probability of using folklore in early childhood disaster education. Researchers
found that folklore is potentially a promising platform for educating young
children about disaster prevention, preparedness, response and recovery.
However, science should also help to convey, incorporate and clarify the texts and
contexts of the folklore. This work used systematic analysis methods to examine
the existing information. There are some drawbacks in the study design; from the
papers studied, many do not address the whole matter comprehensively; others
are often based solely on literature reviews, while others are empirical studies
containing just a few samples. The number of papers found is also very limited.
More detailed and well-structured studies are needed for the continuation of this
work. This research serves as a strong base for future investigation into how to
make folklore a successful resource for use in early childhood disaster education.
Acknowledgment
We thank the Center of Research and Publication (PUSLITPEN-LP2M), UIN
Syarif Hidayatullah Jakarta, Indonesia and the Ministry of Religious Affairs,
Republic of Indonesia for the financial support provided to conduct this research.
We would especially like to thank Adam Batten for his review and informative
feedback throughout the production of this writing.
6. References
Agbenyega, J. S., Tamakloe, D. E., & Klibthong, S. (2017). Folklore epistemology: how does
traditional folklore contribute to children’s thinking and concept development?
International Journal of Early Years Education, 25(2), 112–126.
https://doi.org/10.1080/09669760.2017.1287062
Agosto, D. E. (2016). Why Storytelling Matters: Unveiling the Literacy Benefits of
Storytelling. Children and Libraries, 14(2), 21. https://doi.org/10.5860/cal.14n2.21
Amri, A., Haynes, K., Bird, D. K., & Ronan, K. (2018). Bridging the divide between studies
on disaster risk reduction education and child-centred disaster risk reduction: a
critical review. Children’s Geographies, 16(3), 239–251.
https://doi.org/10.1080/14733285.2017.1358448
Ani, P. A. B., Daquio, C. R. O., & Aquino, A. P. (2015). Republic Act 10121: An Approach
in Strengthening Disaster Risk Reduction and Management in the Philippines.
Retrieved August 3, 2020, from https://ap.fftc.org.tw/article/838
Appleby-Arnold, S., Brockdorff, N., Jakovljev, I., & Zdravković, S. (2018). Applying
cultural values to encourage disaster preparedness: Lessons from a low-hazard
country. International Journal of Disaster Risk Reduction, 31, 37–44.
https://doi.org/10.1016/j.ijdrr.2018.04.015
Bankoff, G. (2013). Mapping Vulnerability. Routledge.
https://doi.org/10.4324/9781849771924
Becker, J., Johnston, D., Lazrus, H., Crawford, G., & Nelson, D. (2008). Use of traditional
knowledge in emergency management for tsunami hazard. Disaster Prevention and
Management: An International Journal, 17(4), 488–502.
https://doi.org/10.1108/09653560810901737

76
Ben-Amos, D. (1971). Toward a Definition of Folklore in Context. The Journal of American

Folklore, 84(331), 3. https://doi.org/10.2307/539729
Ben-Amos, D. (2014). A Definition of Folklore: A Personal Narrative. Estudis de Literatura
Oral Popular / Studies in Oral Folk Literature, (3), 9.
https://doi.org/10.17345/elop20149-28
Bosschaart, A., van der Schee, J., Kuiper, W., & Schoonenboom, J. (2016). Evaluating a
flood-risk education program in the Netherlands. Studies in Educational Evaluation,
50, 53–61. https://doi.org/10.1016/j.stueduc.2016.07.002
Bronner, S. J. (2011). Explaining Traditions: Folk Behavior in Modern Culture. Lexington:
University Press of Kentucky.
Bronner, S. J. (2019). Toward a definition of folklore in practice. In The Practice of Folklore
(pp. 64–82). University Press of Mississippi. https://doi.org/10.2307/j.ctvkwnnvd.6
Bryant, R. A., Gibbs, L., Gallagher, H. C., Pattison, P., Lusher, D., MacDougall, C., …
Forbes, D. (2018). Longitudinal study of changing psychological outcomes following
the Victorian Black Saturday bushfires. Australian & New Zealand Journal of Psychiatry,
52(6), 542–551. https://doi.org/10.1177/0004867417714337
Cashman, K. V., & Cronin, S. J. (2008). Welcoming a monster to the world: Myths, oral
tradition, and modern societal response to volcanic disasters. Journal of Volcanology
and Geothermal Research, 176(3), 407–418.
https://doi.org/10.1016/j.jvolgeores.2008.01.040
Cassell, J. (2004). Towards a model of technology and literacy development: Story
listening systems. Journal of Applied Developmental Psychology, 25(1), 75–105.
https://doi.org/10.1016/j.appdev.2003.11.003
Cocks, D. (2013). Ecohumanism and Other Stories. In Global Overshoot (pp. 349–375). New
York, NY: Springer New York. https://doi.org/10.1007/978-1-4614-6265-1_6
Donovan, K. (2010). Doing social volcanology: exploring volcanic culture in Indonesia.
Area, 42(1), 117–126. https://doi.org/10.1111/j.1475-4762.2009.00899.x
Donovan, K., Suryanto, A., & Utami, P. (2012). Mapping cultural vulnerability in volcanic
regions: The practical application of social volcanology at Mt Merapi, Indonesia.
Environmental Hazards, 11(4), 303–323.
https://doi.org/10.1080/17477891.2012.689252
Dube, E., & Munsaka, E. (2018). The contribution of indigenous knowledge to disaster risk
reduction activities in Zimbabwe: A big call to practitioners. Jàmbá: Journal of Disaster
Risk Studies, 10(1). https://doi.org/10.4102/jamba.v10i1.493
Duffy, N. (2014). Opportunities for disaster resilience learning in the Australian
curriculum. Australian Journal of Emergency Management, 29(1), 12–16.
https://doi.org/10.1007/s11069-016-2435-5
Eisman, A. B., Kilbourne, A. M., Ngo, Q., Fridline, J., Zimmerman, M. A., Greene, D., &
Cunningham, R. M. (2020). Implementing a State‐Adopted High School Health
Curriculum: A Case Study. Journal of School Health, 90(6), 447–456.
https://doi.org/10.1111/josh.12892
Faber, M. H., Giuliani, L., Revez, A., Jayasena, S., Sparf, J., & Mendez, J. M. (2014).
Interdisciplinary Approach to Disaster Resilience Education and Research. Procedia
Economics and Finance, 18, 601–609. https://doi.org/10.1016/S2212-5671(14)00981-2
Fleer, M. (2013). Affective Imagination in Science Education: Determining the Emotional
Nature of Scientific and Technological Learning of Young Children. Research in
Science Education, 43(5), 2085–2106. https://doi.org/10.1007/s11165-012-9344-8
Flewitt, R. (2017). Equity and diversity through story: A multimoda perspective. In T.
Cremin, R. Flewitt, B. Mardell, & J. Swan (Eds.), Storytelling in early childhood:
enriching language, literacy and classroom culture (pp. 150–167). Abingdon, Oxon:
Routledge.

77
Fothergill, A. (2017). Children, Youth, and Disaster. In Oxford Research Encyclopedia of

Natural Hazard Science. Oxford University Press.
https://doi.org/10.1093/acrefore/9780199389407.013.23
Gadeng, A. N., Maryani, E., & Rohmat, D. (2018). The Value of Local Wisdom Smong in
Tsunami Disaster Mitigation in Simeulue Regency, Aceh Province. IOP Conference
Series: Earth and Environmental Science, 145, 012041. https://doi.org/10.1088/1755-
1315/145/1/012041
Gibbs, L., Nursey, J., Cook, J., Ireton, G., Alkemade, N., Roberts, M., … Forbes, D. (2019).
Delayed Disaster Impacts on Academic Performance of Primary School Children.
Child Development, 90(4), 1402–1412. https://doi.org/10.1111/cdev.13200
Gilson, L. L., & Goldberg, C. B. (2015). Editors’ Comment. Group & Organization
Management, 40(2), 127–130. https://doi.org/10.1177/1059601115576425
Gulay, H. (2010). An earthquake education program with parent participation for
preschool children. Educational Research and Reviews, 5(10), 624–630.
Gunnestad, A., & Thwala, S. (2011). Resilience and religion in children and youth in
Southern Africa. International Journal of Children’s Spirituality, 16(2), 169–185.
https://doi.org/10.1080/1364436X.2011.580726
Harris, A. J. L., & Ripepe, M. (2007). Regional earthquake as a trigger for enhanced
volcanic activity: Evidence from MODIS thermal data. Geophysical Research Letters,
34(2), L02304. https://doi.org/10.1029/2006GL028251
Harvilahti, L. (2004). Folklore and Oral Tradition. Oral Tradition, 18(2), 200–202.
https://doi.org/10.1353/ort.2004.0065
Haynes, K., & Lassa, J. (2010). Child centred disaster risk reduction and climate change
adaptation: roles of gender and culture in Indonesia.
Hiwasaki, L., Luna, E., Syamsidik, & Shaw, R. (2014). Process for integrating local and
indigenous knowledge with science for hydro-meteorological disaster risk reduction
and climate change adaptation in coastal and small island communities. International
Journal of Disaster Risk Reduction, 10, 15–27.
Hulland, J. (2020). Conceptual review papers: revisiting existing research to develop and
refine theory. AMS Review, 10(1–2), 27–35. https://doi.org/10.1007/s13162-020-
00168-7
Izadkhah, Y. O., & Hosseini, M. (2005). Towards resilient communities in developing
countries through education of children for disaster preparedness. International
Journal of Emergency Management, 2(3), 138.
https://doi.org/10.1504/IJEM.2005.007355
Jaakkola, E. (2020). Designing conceptual articles: four approaches. AMS Review, 10(1–2),
18–26. https://doi.org/10.1007/s13162-020-00161-0
Jha, V., & Jha, A. (2011). Traditional knowledge on disaster management: A preliminary
study of the Lepcha community of Sikkim, India. Indian Journal of Traditional
Knowledge, 10(1), 173–182.
Johnson, V. A., Ronan, K. R., Johnston, D. M., & Peace, R. (2014). Evaluations of disaster
education programs for children: A methodological review. International Journal of
Disaster Risk Reduction, 9, 107–123. https://doi.org/10.1016/j.ijdrr.2014.04.001
King, D. N., & Goff, J. R. (2010). Benefitting from differences in knowledge, practice and
belief: Māori oral traditions and natural hazards science. Natural Hazards and Earth
System Sciences, 10(9), 1927–1940. https://doi.org/10.5194/nhess-10-1927-2010
King, D. N., Goff, J., & Skipper, A. (2007). Māori environmental knowledge and natural
hazards in Aotearoa‐New Zealand. Journal of the Royal Society of New Zealand, 37(2),
59–73. https://doi.org/10.1080/03014220709510536

78
Kousky, C. (2016). Impacts of natural disasters on children. Future of Children.

https://doi.org/10.1353/foc.2016.0004
Kulatunga, U. (2010). Impact of culture towards disaster risk reduction. International
Journal of Strategic Property Management, 14(4), 304–313.
https://doi.org/10.3846/ijspm.2010.23
Lenox, M. F. (2000). Storytelling for Young Children in a Multicultural World. Early
Childhood Education Journal, 28, 97–103.
https://doi.org/https://doi.org/10.1023/A:1009599320835
Louise, G. (1998). Working with Indigenous Knowledge: A Guide for Researchers | IDRC -
International Development Research Centre. Retrieved from
https://www.idrc.ca/en/book/working-indigenous-knowledge-guide-
researchers
Ludwin, R. S., Smits, G. J., Carver, D., James, K., Jonientz-Trisler, C., McMillan, A. D., …
Wray, J. (2007). Folklore and earthquakes: Native American oral traditions from
Cascadia compared with written traditions from Japan. Geological Society, London,
Special Publications, 273(1), 67–94. https://doi.org/10.1144/GSL.SP.2007.273.01.07
MacInnis, D. J. (2011). A Framework for Conceptual Contributions in Marketing. Journal
of Marketing, 75(4), 136–154. https://doi.org/10.1509/jmkg.75.4.136
Mardell, B., & Kucirkova. (2017). Promoting democratic classroom communities throung
storytelling and story acting. In T. Cremin, R. Flewitt, B. Mardell, & J. Swan (Eds.),
Storytelling in early childhood: enriching language, literacy and classroom culture (pp. 169–
185). Abingdon, Oxon: Routledge.
Maureen, I. Y., van der Meij, H., & de Jong, T. (2018). Supporting Literacy and Digital
Literacy Development in Early Childhood Education Using Storytelling Activities.
International Journal of Early Childhood, 50(3), 371–389.
https://doi.org/10.1007/s13158-018-0230-z
Mercer, J., Gaillard, J. C., Crowley, K., Shannon, R., Alexander, B., Day, S., & Becker, J.
(2012). Culture and disaster risk reduction: Lessons and opportunities. Environmental
Hazards, 11(2), 74–95. https://doi.org/10.1080/17477891.2011.609876
Mileti, D. (1995). Public hazards communication and education: the state of the art. 1–12.
Mulyasari, F., Takeuchi, Y., & Shaw, R. (2011). Chapter 7 Implementation Tools for Disaster
Education. https://doi.org/10.1108/S2040-7262(2011)0000007013
Mutch, C. (2014). The role of schools in disaster preparedness, response and recovery:
what can we learn from the literature? Pastoral Care in Education, 32(1), 5–22.
https://doi.org/10.1080/02643944.2014.880123
Nicolopoulou, A., Cortina, K. S., Ilgaz, H., Cates, C. B., & de Sá, A. B. (2015). Using a
narrative- and play-based activity to promote low-income preschoolers’ oral
language, emergent literacy, and social competence. Early Childhood Research
Quarterly, 31, 147–162. https://doi.org/10.1016/j.ecresq.2015.01.006
Norris, F. H., Friedman, M. J., Watson, P. J., Byrne, C. M., Diaz, E., & Kaniasty, K. (2002).
60,000 Disaster Victims Speak: Part I. An Empirical Review of the Empirical
Literature, 1981–2001. Psychiatry: Interpersonal and Biological Processes, 65(3), 207–239.
https://doi.org/10.1521/psyc.65.3.207.20173
Olajide, S. B. (2010). Folklore and Culture as Literacy Resources for National
Emancipation. International Education Studies, 3(2), 200–205.
https://doi.org/10.5539/ies.v3n2p200
Pareek, A., & Trivedi, P. C. (2011). Cultural values and indigenous knowledge of climate
change and disaster prediction in Rajasthan, India. Indian Journal of Traditional
Knowledge, 10(1), 183–189.
Parsizadeh, F., Ibrion, M., Mokhtari, M., Lein, H., & Nadim, F. (2015). Bam 2003
earthquake disaster: On the earthquake risk perception, resilience and earthquake

79
culture – Cultural beliefs and cultural landscape of Qanats, gardens of Khorma trees
and Argh-e Bam. International Journal of Disaster Risk Reduction, 14, 457–469.
Peek, L. (2008). Children and Disasters: Understanding Vulnerability, Developing
Capacities, and Promoting Resilience-An Introduction. In Children, Youth and
Environments (Vol. 18). Retrieved from http://www.colorado.edu/journals/cye.
Proulx, K., & Aboud, F. (2019). Disaster risk reduction in early childhood education:
Effects on preschool quality and child outcomes. International Journal of Educational
Development, 66, 1–7. https://doi.org/10.1016/j.ijedudev.2019.01.007
Rahiem, M. D. H., Abdullah, N. S. M., & Krauss, S. E. (2020). Moral Education through
Dramatized Storytelling: Insights and Observations from Indonesia Kindergarten
Teachers. International Journal of Learning, Teaching and Educational Research, 19(3),
475–490. https://doi.org/10.26803/ijlter.19.3.26
Rahiem, M. D. (2018). Faith and Disaster Resilience: What can Islamic Education Teach
Children to Help Prepare Them for A Disaster? TARBIYA: Journal of Education in
Muslim Society, 5(2), 178–192. https://doi.org/10.15408/tjems.v5i2.9964
Rahim, H., & Rahiem, M. D. H. (2013). The Use of Stories as Moral Education for Young
Children. International Journal of Social Science and Humanity, 2(6), 454–458.
https://doi.org/10.7763/IJSSH.2012.V2.145
Ronan, K. R., Alisic, E., Towers, B., Johnson, V. A., & Johnston, D. M. (2015). Disaster
Preparedness for Children and Families: a Critical Review. Current Psychiatry
Reports, 17(7), 58. https://doi.org/10.1007/s11920-015-0589-6
Sharpe, J., & Kelman, I. (2011). Improving the disaster-related component of secondary
school geography education in England. International Research in Geographical and
Environmental Education, 20(4), 327–343.
https://doi.org/10.1080/10382046.2011.619810
Shreve, C., Fordham, M., Anson, S., Watson, H., Hagen, K., Kush, W., … Karanci, N.
(2014). Report on risk Perception and Preparedness. Report on risk Perception and
Preparedness.
http://www.academia.edu/download/36213263/Deliverable_D1.1_FINAL.pdf
Shultz, J. M. (2014). Perspectives on disaster public health and disaster behavioral health
integration. Disaster Health, 2(2), 69–74. https://doi.org/10.4161/dish.24861
Somasundaram, D. J., & van de Put, W. A. C. M. (2006). Management of trauma in special
populations after a disaster. The Journal of Clinical Psychiatry, 67 Suppl 2, 64–73.
Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/16602818
Sukmawan, S., & Setyowati, L. (2017). Environmental Messages as Found in Indonesian
Folklore and Its Relation to Foreign Language Classroom. Arab World English Journal,
8(1), 298–308. https://doi.org/10.24093/awej/vol8no1.21
Tanner, T., Lazcano, J., Lussier, K., & Polack, E. (2009). Children, climate change and disasters:
An annotated bibliography. Retrieved from
http://mobile.opendocs.ids.ac.uk/opendocs/handle/123456789/2373
Thambu, N. (2017). Storytelling and Story Reading: A Catalyst for Inculcate Moral Values
and Ethics among Preschoolers. International Journal of Academic Research in Business
and Social Sciences, 7(6). https://doi.org/10.6007/IJARBSS/v7-i6/3143
Thompson, M. (2011). Developing moral values in children: Observations from a
preschool. IFE PsychologIA, 19(2), 394–411.
https://doi.org/10.4314/ifep.v19i2.69584
Tomasello, M. (2009). The question of chimpanzee culture, plus postscript. In K. N. Laland
& G. Galef (Eds.), The question of animal culture (pp. 198–221). Harvard University
Press.

80
Torani, S., Majd, P., Maroufi, S., Dowlati, M., & Sheikhi, R. (2019). The importance of
education on disasters and emergencies: A review article. Journal of Education and
Health Promotion, 8(1), 85. https://doi.org/10.4103/jehp.jehp_262_18
Troll, V. R., Deegan, F. M., Jolis, E. M., Budd, D. A., Dahren, B., & Schwarzkopf, L. M.
(2015). Ancient oral tradition describes volcano–earthquake interaction at merapi
volcano, indonesia. Geografiska Annaler: Series A, Physical Geography, 97(1), 137–166.
https://doi.org/10.1111/geoa.12099
Tsai, M.-H., Wen, M.-C., Chang, Y.-L., & Kang, S.-C. (2015). Game-based education for
disaster prevention. AI & SOCIETY, 30(4), 463–475. https://doi.org/10.1007/s00146-
014-0562-7
Twigg, J. (2003). The Human Factor in Early Warnings: Risk Perception and Appropriate
Communications. In Early Warning Systems for Natural Disaster Reduction (pp. 19–26).
Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-
55903-7_4
United Nations. (2005). International Strategy for Disaster Reduction Hyogo Framework
for Action 2005-2015: Building the Resilience of Nations. World Conference on Disaster
Reduction (A/CONF.206/6). https://doi.org/10.1017/CBO9781107415324.004
United Nations Children’s Emergency Fund. (2011). 2011 Humanitarian action for
children: Building resilience. In Unicef. Retrieved from
http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Humanitarian+
Action+for+Children+building+resilience#0
Winser, B., Blaikie, P., Cannon, T., Davis, I., Torres, R., Azócar, G., … Shirley, W. L. (2004).
At Risk: Natural Hazards, People’s Vulnerability and Disasters - Piers Blaikie, Terry
Cannon, Ian Davis, Ben Wisner - Google Books. Geoforum.
https://doi.org/10.1016/J.GEOFORUM.2014.12.013
Woodard, J. (2005). Head, Heart and Hands: Waldorf Education. Journal of Curriculum and
Pedagogy, 2(2), 84–85. https://doi.org/10.1080/15505170.2005.10411551

81

Vol. 19, No. 8, pp. 81-97, August 2020
Lecture-simulation-combined Education
Improve Nursing Undergraduates' Knowledge
and Attitude for Palliative Care
Yan Wang
Macao Polytechnic Institute, Macao, China
https://orcid.org/0000-0002-4609-3033
Abstract. Continuous improvement for the teaching method is security

for promoting the quality of palliative care. The didactic lecture and
simulation-based learning were always used independently, which may
negatively affect the results of palliative care education. This research
aimed to explore the effectiveness of lecture–simulation-combined
education in improving nursing undergraduates' knowledge and
attitude of palliative care. A quasi-experimental single-group pre/post-
test design was adopted to evaluate the variables with a sample of
voluntary fifty-two nursing undergraduates before and after an 18-hour
lecture-simulation-combined palliative care course. The course was
composed of two hours didactic lecture, ten hours of lecture-simulation-
combined learning, and six hours of simulation practise. Two
questionnaires, named Palliative Care Quiz for Nursing and Frommelt
Attitude towards Care of the Dying, were used to evaluate students'
knowledge and attitude of palliative care before and after the course. By
paired sample t-test, the results showed increased scores in students'
knowledge (p<0.05) and attitude (p<0.05) after the course. Lecture-
simulation-combined education is an effective strategy to improve
nursing students' knowledge and attitude of palliative care. Lecture-
simulation-combined education could either be good at equipping
students with theoretical knowledge, but also be capable of helping
students to construct a positive attitude on palliative care.
Keywords: Palliative care; Lecture-simulation-combined; Attitude;

Knowledge; Nursing undergraduates
1. Introduction
According to the latest population projections, Macao will enter the "hyper-aged
society" in 2031 with the elderly's proportion reaching 22.4% (Statistics and
Census Service of Government of Macao Special Administrative Region, 2014).
With increasing ageing, the requirement for palliative care in Macao is
demanding. Palliative care, as an essential part of continuum nursing care, is to
secure dignity and comfort at the last stage of people's life. The American

82
Association of Colleges of Nursing (AACN) identifies palliative care as one of

the core competencies of qualified nurses (American Association of Colleges of
Nursing, 2019). Because nurses accompany with the dying patients more than
other medical staff, they become the centre and connection of the team (Smith,
Macieira & Bumbach, 2018). Every nursing student needs to master seventeen
palliative care competencies before graduation, which include symptom control,
comfort supply, communication, team-work, and self-coping. Providing
palliative care is not only challenging but also anxiety-provoking because of
incurable diseases and impending death. Still, Macao's nursing curriculum has
been slow to integrate palliative care as a compulsory course. Nursing
undergraduates were not educated enough to cope with the situation of dying.
The under-prepared nurses feel anxious and stressed when carrying out the
palliative care, resulting in a poor quality of nursing and compromising patient
satisfaction (Rodrigues, 2015; DʼAntonio, 2017; Cant & Cooper, 2017).
1.1 The knowledge of palliative care

Knowledge is defined as "knowing something with the familiarity that acquired
through experiences such as understanding of a science or technique" (English
Oxford Living Dictionaries, 2019). Palliative care, defined by the World Health
Organization, is "an approach that improves the quality of life of patients and
their families facing the problem associated with life-threatening illness" (World
Health Organization, 2002). Palliative care knowledge includes basic concepts,
ethical principles, and theories on terminal symptoms management (mechanism,
assessment, medications, and supplementary treatment). Nurses' insufficiency
knowledge is one of the main barriers to providing excellent palliative care, and
nurses' knowledge deficit might originate from inadequate education during
their undergraduate learning (Jaykumar, Karthikbabu, Karvannan, Kumar,
Prem, Sisodia & Syed, 2012). Lacking a particular course, the knowledge of
palliative care just "threaded" through the nursing curriculum; therefore,
students' palliative care knowledge was not enough.
1.2 The attitude of palliative care

Attitude is defined as "a settled way of thinking or feeling about something"
(English Oxford Living Dictionaries, 2019). Attitudes are formed by evaluating
favourable or unfavourable effects on specific entities. As experiences increases,
attitudes change accordingly. The attitude of palliative care includes how the
nurse regards palliative care and how about the position of nursing in palliative
care (Frommelt, 2003). Feeling useless and unable to help dying patients and
their families might accumulate into negative attitudes towards palliative care
(Robinson & Epps, 2017). Exposure to the processes of patients' dying might
remind nurses of their mortality, which may also result in a negative attitude
towards palliative care. Passive attitudes will harm the quality of care and
increase the burnout of nurses.
1.3 Medical simulation education

Medical educators defined simulation as "an event or situation made to resemble
clinical practise as closely as possible" (Grossman, 2013). In other words, medical
simulation means that educators and learners conduct learning and teaching
activities in a simulated clinic scenario. The reported benefits of simulation

83
including 1) improving critical thinking and clinical reasoning skills; 2)

improving the knowledge acquisition; 3) improving putting the theory into
practice; 4) improving the communication skills; 5) improving the problem-
solving skills and nursing techniques; and 6) providing supportive and
nonthreatening learning conditions (Gillan, 2014; Carman, Sloane, Molloy, Flint
& Phillips, 2016; Tamaki, 2019). With limited palliative care practice units and
ethical considerations, students did not have enough chance to practise
palliative care, so the medical educators used simulation more widely in recent
years.
1.4 Lecture-simulation-combine education for palliative care

Initially, palliative care education predominantly carried out by theoretical
lectures which may be complained by both students and educators for lacking in
practising. For example, Berndtsson used a didactic five-week palliative care
course for the year three nursing students (Berndtsson, Margareta & Rejnö,
2019). Their study included 12 lectures: physical and psychological changes in
the dying process, palliative diagnoses, terminal symptoms management, ethical
issues, caring for families, and nurses' advocating roles in palliative care.
Nevertheless, only with didactic lecture, students may find difficulties to use the
knowledge. It has been concluded the didactic lecture was not good at training
communication which was the most mentioned difficulty by researchers.
On the other side, among the increasing numbers of palliative care simulation
researches, most of them used simulation without enough theoretical
preparation. For example, Dame and Hoebeke (2016) ran a 15-minute simulation
scenario, followed by a discussing and sharing debriefing to teach palliative
care. Researchers conducted two simulation scenarios to train students'
competency of palliative care (Valen, Holm, Jensen & Grov, 2019). Students
should not only be encouraged to be kind and patient but also they should be
equipped enough with knowledge before they enter palliative situations.
Carman et al. (2016) suggested that the simulation should combine theoretical
knowledge learning with practical application to provide palliative care
education. Therefore, in this study, researchers designed educational
interventions as a combination of theoretical lectures with simulation-based
learning. This research aimed to investigate whether the lecture-simulation-
combined course could improve nursing undergraduates' palliative care
knowledge and attitudes.
2. Research design and method

2.1 Design
A quasi-experimental single-group pre/post-test design was adopted to
investigate the effectiveness of the lecture-simulation-combined palliative course
in improving nursing undergraduates' knowledge and attitudes.
2.2 Hypothesis
The hypothesis generated in this research was that the nursing undergraduate'
would have increased knowledge and attitude of palliative care after the 18-hour
lecture-simulation-combined course.

84
2.3 Participants and setting

Nursing students of Year 2 and Year 3 from a four-year undergraduate nursing
program in a Macao institute were invited to enrol in this research. The research
ruled out students of Year 1 and because they were still learning fundamental
nursing theories and are not ready for palliative care. Year 4 students have too
many clinical practising hours to coordinate with the schedule of this research.
Fifty-two nursing students enrolled in and finished the 18-hour palliative care
course. There were 15 (28.8%) males and 37 (71.2%) females. Twenty-three
(44.2%) students were from Year 2 while twenty-nine (55.8%) were students
from Year 3.
2.4 Interventions
The 18-hour palliative care course included a 2-hour theoretical lecture, a10-hour
lecture-simulation-combined terminal symptoms learning, and 6-hour
simulation scenarios practising. Table 1 shows the course design. The course
lasted for around one month, two times per week. To ensure every participant's
active involvement, researchers divided fifty-two students into five groups, and
there were about ten students in each group. The research ran the 18-hour-
course five times, and each time had the same teaching contents and educators.
Table 1. The framework of lecture-simulation-combined course in palliative care

Content Time
• Introduction for palliative care 2 hours
• Terminal symptom management for pain 2 hours
• Terminal symptom management for unconsciousness and respiratory 2 hours
and cardiovascular problems
• Terminal symptom management for digestive problems 2 hours
• Terminal symptom management for malignant wounds 2 hours
• Psychological, spiritual and social supporting in palliative care 2 hours
• Hospice scenario simulation: expectable death 3 hours
• Hospice scenario simulation: un-expectable death 3 hours
Total 18 hours
The first two-hour lecture introduced fundamental theories and communication

principles of palliative care. In the next ten hours, we focused on the mechanism,
assessment, medications, complementary therapies, and nursing interventions of
terminal symptoms. The involved terminal symptoms were pain, dyspnoea,
fatal arrhythmia, fatigue, unconsciousness, anorexia, nausea and vomiting,
thrush, hiccup, ascites, intestinal obstruction, and malignant ulcer/fistulae. Each
symptom was taught firstly by didactic lectures and then followed by students'
practising on manikins to care symptoms while communicating with patients.
Terminal symptoms were demonstrated mainly by the high-fidelity manikin.
Educators also used pictures and wounds apparatus to show symptoms which
the manikin could not mimic. While students' practising, the tutor used the vocal
system of the manikin to mimic the patient's voice, allowing the communication
between the role-players. Tutors merged the communication skills, cultural and
spiritual assessment, and interdisciplinary cooperation into 10-hour learning.
Table 2 showed the example of educational design in lecture-simulation-
combined learning.

85
Table 2. An example of lecture-simulation-combined learning for pain

Content Time
Lecture: pain management for terminal patients 60 min
Simulation setting
• Background: 54-year-old, male, Buddhist, pancreatic cancer, stage IV,
complaining "pain all over my body".
• High-fidelity manikin: moaning, sweating, heart rate: 146/min, blood
pressure: 169/92 mmHg.
• Pictures: an old man with painful facial expressions and body
postures; thrush.
• Educator: complained pain through the vocal system of the manikin.
• Pain management strategy: Oramorph, oral solution, 10 mg, Bid.
Students' practising in simulation 60 min
• Identify various pain origins by communication with the patient:
thrush because of chemotherapy, abdominal pain because of
pancreatic cancer, pain in the bone because of metastasis, muscular
pain because of long-term bedridden.
• Assess and document the pain through the Visual Analogue Scale, the
patient's facial expression, body posture, and fluctuating vital signs.
• Evaluate and explain the current pain management strategy.
• Suggest revising for the pain management strategy: medication
adjustment and supplementary therapy.
• Practise oral hygiene.
• Health educate: the usage of Xylocaine mouthwash before the dinner
to control the pain of thrush.
• Comfort the patient continuously according to the patient's
background.
The last six hours were composed of a 3-hour-scenario for the expectable death
and a 3-hour-scenario for the un-expectable death. Educators grouped all
learned symptoms in scenarios. At the beginning of the class, the educator
introduced the case. Ten students were divided voluntarily into the Patient
team, Nurse team, or Family team. Forty-five minutes were used for group
discussion. Based on the framework drafted by the tutor, students in the Patient
team and Family team detailed the possible interactions between the patient,
nurses, and families. Students in the Nurse team did not know the details of the
scenario, and they just made the nursing care plan according to the background
of the patient. Every team picked one student to participate in the scenario. The
scenario lasted 60 minutes. The educator controlled the manikin, and three
students from each team role-play the patient (by manikin's vocal system), the
family member, and the nurse respectively. The rest students were observers.
The scenarios were followed by a 60-minute debriefing guided by the tutor and
attended by all students. We invited the role-played nurse, patient, and family
member to share their experiences of the scenario, asking the observers to
discuss their ideas about death. Table 3 showed the example of scenario design
for un-expectable death.

86
Table 3. An example of un-expectable death scenario

Simulation setting
• Background: 67-year-old, male, acute myocardial infarction with IV degree of heart
function
• High-fidelity manikin (symptoms show chronologically)
o Moaning, sweating, old inferior wall myocardial infarction (ECG), bowel
sound hyperaction
o Severe dyspnoea, vomiting
o R-ON-T phenomena, then ventricular fibrillation (ECG)
o Unconsciousness
Students' role playing
• The patient
o Refuse to use the bedpan and insist on going to the toilet for defecation.
• The nurse
o Persuade and help the patient defecate by using the bedpan.
o Ensure oxygen supply.
o Monitor the electrocardiogram on the screen continuously and closely.
o Identify the fatal arrhythmia and inform the doctor instantly.
o Prepare instruments and medicines for resuscitation.
o Inform the family member in time.
o Communicate with the family member.
• The family member
o Panic.
o Cannot accept the deterioration of the patient.
2.5 Instruments
The Palliative Care Quiz for Nursing (PCQN) was used to examine palliative
care knowledge. The scale was composed of 20 items. The dimension of
philosophy and principles has four items, the dimension of pain and symptom
management has 13 items, and the dimension of psychosocial and spiritual has
three items. Students choose "true", "false" or "do not know" for each item, and
the overall score is gained by calculating the number of correct responses. The
overall scores range from 0 to 20, with higher scores representing higher levels
of knowledge. The validity of PCQN was confirmed during its development
with an acceptable alpha of 0.78. The correlation coefficient in test-retest
reliability was 0.56 (p>0.05) (Ross, McDonald & McGuinness, 1996).
The Frommelt Attitude towards Care of the Dying (FATCOD) scale was adopted
to evaluate students' attitudes towards end-of-life caring. It is composed of 30
items, rating on a five-point Likert-type scale. Item 1, 2, 4, 12, 16, 18, 20, 21, 22,
23, 24, 25, 27, and 30 are all positively statements ranging from 1 for strongly
disagree to 5 for strongly agree. All others are negative, scoring from 1 for
strongly agree to 5 for strongly disagree. The final score ranges from 30 to 150,
with a higher score representing a more positive attitude. The items were
classified into the patient-centred and family-centred dimensions. The reported
internal consistency of the FATCOD was alpha=0.89 (Frommelt, 2003).

87
2.6 Data collection

Participants were required to fill the demographic collecting questionnaires, the
PCQN and the FATCOD before the course. They finished the post-test of PCQN
and FATCOD immediately after the closure of the course.
2.7 Statistical methods

The data were analysed by SPSS version 26.0. The Chi-square test and a paired
sample t-test were adopted to evaluate scores' differences between before and
after the 18-hour course.
2.8 Ethical considerations

The research obtained informed consent from all participants. Research
informants have been informed of their voluntary participation, and they could
terminate their participation at any time without penalty. The personal data of
all participants were kept confidential by coding numbers. Institutional review
board approval was obtained (Grant number RP/ESS-02/2018).
3. Results
3.1 The knowledge of palliative care
The pre-mean score of PCQN was 8.98 (SD 2.46, range: 4-13). The post-mean
score of PCQN was 12.98 (SD 2.47, range: 8-18). Twenty-two participants (42.3%)
scored more than ten before the education, while forty-seven (90.4%) scored
more than ten after the education. Table 4 shows the sub-total scores before and
after the workshop for each dimension. Students' palliative care knowledge was
improved after the course in the dimensions of philosophy and principles
(p<0.05), pain and symptoms management (p <0.05), and psychosocial and
spiritual care (p<0.05). The score details of PCQN were shown in Table 5.
Table 4. Differences of the PCQN between pre-test and post-test (n=52)

Dimension Pre Post
Mean SD Range Mean SD Range t p
Philosophy and 1.62 0.89 0-3 2.38 0.89 0-4 -6.492 .000*
principles
Pain and 6.23 1.58 3-10 8.75 1.40 5-11 -14.073 .000*
symptom
management
Psychosocial and 1.13 0.77 0-3 1.85 0.72 0-3 -5.892 .000*
spiritual
Total score 8.98 2.46 4-13 12.98 2.47 8-18 -17.535 .000*
*: p<0.05

88
Table 5. The response of the PCQN (n=52)

Item Pre correct number (%) Post correct number (%) χ2 p
Philosophy and principles
Q1 25 (48.1) 37 (71.2) 5.751 .016*
Q9 17 (32.7) 28 (53.8) 4.740 .029*
Q12 23 (44.2) 28 (53.8) 0.962 .327
Q17 20 (38.5) 31 (59.6) 4.656 .031*
Pain and symptom management
Q2 29 (55.8) 48 (92.3) 18.059 .000*
Q3 13 (25.0) 19 (36.5) 1.625 .202
Q4 35 (67.3) 49 (94.2) 12.133 .000*
Q6 19 (36.5) 32 (61.5) 6.502 .011*
Q7 12 (23.1) 40 (76.9) 30.154 .000*
Q8 39 (75.0) 46 (88.5) 3.155 .076
Q10 16 (30.8) 40 (76.9) 22.286 .000*
Q13 1 (1.9) 13 (25.0) 11.886 .001*
Q14 33 (63.5) 40 (76.9) 2.252 .133
Q15 46 (88.5) 42 (80.8) 1.182 .227
Q16 11 (21.2) 17 (32.7) 1.759 .185
Q18 48 (92.3) 44 (84.6) 0.848 .357
Q20 27 (51.9) 25 (48.1) 0.154 .695
Psychosocial and spiritual
Q5 4 (7.7) 14 (26.9) 5.442 .020*
Q11 34 (65.4) 48 (92.3) 9.743 .001*
Q19 24 (46.2) 34 (65.4) 3.898 .048*
*: p<0.05
3.2 The attitude of palliative care

Before the course, students earned a mean score of 108.92 (SD 7.40) in the total
score of FATCOD, 70.37 (SD 5.92) in the "patient-centred" dimension, and 38.56
(SD 2.61) in the "family-centred" dimension. After the course, students earned a
mean score of 119.37 (SD 8.57) in total score, 78.89 (SD 6.84) in the "patient-
centred" dimension, and 40.48 (SD 2.30) in the "family-centred" dimension. The
differences between pre score and post score in FATCOD showed statistical
significance (p<0.05). Table 6 showed the score of FATCOD.
Table 6. FATCOD Score of pre/post-course

Items Pre Post
Mean SD Mean SD t p
Patient-centred
70.37 5.92 78.89 6.84 -18.457 .000*
dimension
Q1 4.40 0.66 4.94 0.73 -5.156 .000*
Q2 3.56 0.78 4.44 0.73 -7.458 .000*
Q3 2.90 0.87 3.52 0.75 -4.657 .000*
Q5 3.87 0.72 4.06 0.73 -2.018 .049*
Q6 3.92 0.74 4.39 0.63 -5.196 .000*
Q7 3.14 0.74 3.44 0.80 -2.675 .010*
Q8 2.50 0.64 3.10 0.77 -5.080 .000*
Q9 3.12 0.83 3.58 0.70 -4.964 .000*
Q10 2.75 0.88 3.67 0.76 -8.724 .000*
Q11 3.27 0.89 3.77 0.83 -5.369 .000*
Q13 3.75 0.71 3.89 0.73 -1.188 .240

89
Q14 3.56 0.83 3.75 0.62 -3.120 .003*

Q15 3.85 0.83 3.98 0.92 -0.961 .341
Q19 3.90 1.02 4.37 0.84 -3.150 .003*
Q21 4.10 0.50 4.33 0.51 -3.546 .001*
Q23 4.52 0.61 4.56 0.57 -0.814 .420
Q25 3.10 1.00 4.00 0.63 -7.138 .000*
Q26 2.50 0.83 2.87 0.77 -4.428 .000*
Q27 3.50 0.70 3.87 0.69 -4.696 .000*
Q30 4.17 0.38 4.39 0.53 -2.844 .006*
Families-centred .000*
38.56 2.61 40.48 2.30 -7.937
dimension
Q4 4.50 0.51 4.73 0.49 -3.546 .001*
Q12 3.94 0.61 4.19 0.53 -3.244 .002*
Q16 4.04 0.44 4.31 0.47 -3.964 .000*
Q17 1.58 0.67 1.39 0.49 2.018 .049*
Q18 4.50 0.54 4.65 0.48 -1.935 .059
Q20 4.08 0.62 4.25 0.48 -2.901 .005*
Q22 3.96 0.44 4.33 0.55 -5.019 .000*
Q24 4.10 0.63 4.35 0.68 -2.360 .022*
Q28 4.46 0.61 4.77 0.58 -4.761 .000*
Q29 3.40 0.87 3.52 0.78 -2.579 .013*
Total score 108.92 7.40 119.37 8.57 -24.126 .000*
*: p<0.05
4. Discussion
4.1 Course design
Palliative care education was conducted through a variety of teaching methods,
including traditional lectures, clinical case discussions, hospice care facilities
visiting, and simulation. Although educators used the simulation increasingly,
the theoretical lecture rarely combined with it. One or two hours of simulation,
maybe not enough to teach enough knowledge and construct positive attitudes.
The lecture was good at teaching knowledge, while the simulation did well in
the application of knowledge and practising communications. The two teaching
methods have equal value, and neither should not be neglected in palliative care
education. This study designed the course with a suitable length of learning
hours and reasonable learning process, aiming to make students benefit from the
two methods.
The objective of palliative care is not to cure but to relieve and comfort, and the
medical and nursing principles should be changed accordingly. If we kept
pushing our nurses to save the lives of human beings, enormous pressure and
un-avoided frustration would cause passive attitudes of nurses. We set the
acceptance of death as the first objective of our course. We took some time to
explain the philosophy of death by lecture and guided deep reflection after the
simulation scenarios. The principles of therapeutic communication in a dying
situation were taught firstly in the lecture. Then we set the dying situations
which included the dying patient mimicked by the educator and the families
mimicked by the students. Participants could practise communication without
being afraid of making mistakes. Role-playing family members allowed students
to profoundly understand the families' dilemma, which may result in more

90
tolerance. We constructed our course based on handling terminal symptoms.

Although persons died of different diseases, different diseases share some
common terminal symptoms. We would take more aggressive and sophisticated
methods to make patients avoid suffering. For example, we taught particular
pain assessment scales for severely ill patients, emphasised that addiction
should not be the primary concern, and introduced the complementary
techniques and spiritual console for the pains. Only when participants found
themselves useful in helping the dying patients and their families, the positive
attitude could be constructed. In recent five years, most studies made the
simulation as the only intervention to carry out palliative care education. For
example, Tamaki (2019) carried out an 80-min simulation in end-of-life
education. There was an innovative study with a 6-week online virtual activity
through Second Life®, which was composed of 3-hour in pre-simulation
learning, 1-hour in a virtual simulation, and 3-hour in guided refection (Sanborn,
Cole, Kennedy & Saewert, 2019). Only one research combining lecture with
simulation was identified. Myers (2018) gave participants a 90-min lecture,
followed by a 45-min simulation in the experimental group while a 45-min case
study in the control group. This research has also proved the lecture-simulation-
combined method an effective method for teaching palliative care by students'
positive changes in knowledge and attitude.
4.2 Knowledge of palliative care

The low mean score of PCQN with a mean score of 8.98 (SD 2.46) before the
course showed inadequate palliative care knowledge in Macao nursing
undergraduates. Previous studies shared similar results and proved nursing
students' insufficient knowledge. Al Qadire's (2014) research investigated 220
Jordan nursing students' palliative care knowledge. The results showed
insufficient knowledge (mean=8 out of 20; SD 3.1). The same result was found in
India's research (Chari, Gupta, Choudhary & Sukare, 2016). Four hundred and
nine Saudi Arabia nursing students got a mean score of 5.23 out of 20 (SD 3.24)
(Aboshaiqah, 2020). Dimoula surveyed 529 Greek nursing students, gaining a
mean score of 8.2 (SD 2.8) (Dimoula, Kotronoulas, Katsaragakis, Christou,
Sgourou & Patiraki, 2019).
This course got a statistically significant improvement in students' palliative

knowledge by an elevated mean score of 3.0 points. All dimensions in the PCQN
showed positive changes. A similar result was found in Tamaki's (2019) study.
They used a knowledge questionnaire, which was quoted from nation nursing
examination in Japan, to prove the simulation intervention could improve
nursing undergraduates' knowledge of palliative care. Researchers used an end-
of-life simulation elevated students' PCQN scores by 2.34 points (Kirkpatrick,
Melin-Johansson & Bergh, 2017). In Myers' (2018) research, the group of lecture
with simulation gained 1.23 points elevation in the mean score of PCQN.
There were four items in the dimension of philosophy and principles. Q1

showed the correction of the misunderstanding that palliative care was only for
the person who was very near death. We emphasised the principle of palliative
care in the first 2-hour lecture that palliative care should begin at the very
beginning of diagnosing of a terminal disease. Before the course, most students

91
thought nurses should stand neutrally and kept emotional detachment to avoid
burnout (Q9). In the debriefing, the role-played nurse shared his experience that
it was impossible for a nurse to "detach" the terminal patient. His idea that not
detaching but coping with the emotional connection was the only way to avoid
nurses' burnout gained agreement from other students. Thirty-two students
originally agreed on the judgment (Q17) palliative care will cause the nurses'
inevitable burnout. After the course, the number decreased to twenty-one. We
could induce that the confidence in coping palliative care increased in some
students after the course. There were thirteen items in the dimension of pain and
symptom management. The correct ratios were increased in six items while
remained unchanged in seven items. Pain is the most serious and long-lasting
symptom affecting the quality of life of the most dying patients. Pain
management is at the core of the terminal symptom control. We taught students
the rules of medication should be adjusted in palliative care, with more focus on
pain-relieving rather than drug addiction (Q7, Q13). The pharmacy was taught
in detail (Q2, Q8, 14), and some supplementary treatment, such as music therapy
and aromatherapy, were introduced by the updated evidence (Q4). Although
the results were not ideal, reciting the knowledge encouraged by a closed-book
exam and continuous practising may be a practical way to enforce the learned
knowledge. There were three items in the psychosocial and spiritual dimension
in which all items gained positive changing. For instance, most participants
believed families accompany until the patient's death was crucial (Q5) firstly. In
the course, we emphasised that families should be included in psychosocial
caring, and nurses should try to help families avoiding severe suffering and
tortures, which might last for the rest of their life. After the course, added
students realised avoiding severe attacks for the patient's family was the nurse's
duty. In summary, extensive and profound learning through the lecture-
simulation-combined could obtain positive changes in students' knowledge.
4.3 Attitudes of palliative care

The FATCOD has been world-widely used in earlier studies for nursing
undergraduates and nurses. The reported mean score was 96.96 (SD 8.30) in
Palestine (Abu-el-Noor & Abu-El-Noor, 2016), 123 (SD 10.1) in Sweden (Henoch,
Melin-Johansson & Bergh, 2017), 93.83 (SD 5.96) in Indonesia (Muhamad,
Baskoro & Dicky, 2018), 95.81 (SD 8.03) in India (Paul, Renu & Thampi, 2019),
and 111.9 (SD 10.2) in Greece (Dimoula et al., 2019). Compared with other
countries, Macao's score was in the middle position with a pre-test score of
108.92 (SD 7.40). Our study showed a statistically significant improvement in
palliative care attitude after the 18-hour course by 10.45 points. This result is in
line with Berndtsson's finding that the mean score was increased of 9.6 points
after a dedicated five-week course which covers 12 lectures (Berndtsson et al.,
2019). Also, using theoretical education in five weeks, Henoch improved nursing
students' attitudes by 6 points (Henoch et al., 2017).
There were twenty items in the patient-centred dimension, and seventeen items
have been statistically significantly improved. For example, most students
reported difficulties in communication. Educators used every chance in
simulation learning or practising, allowing students to practise communication.
Especially, in the debriefing session, educators set the scenario, letting the

92
patient ask the nurse "Am I dying?" Then, we discussed how to answer that
question properly in the debriefing after the scenario. After the course, students
gained higher scores in talking about the impending death with patients (Q3,
Q11) and finding it more acceptable to construct an intimate relationship with
the dying person (Q9, Q14). Students were found to be braver to take part in
palliative care (Q5), no longer running away from the dying situation (Q15, Q26)
after the course. It also has been noticed students become more acceptable about
death, not regarding death as the worst things that could happen to a person
(Q2), feeling less upset with patients' giving up hope of getting better (Q8).
Before the course, students might be hard to accept the words "death welcomed
by the dying (Q10)". After the course, more students chose a neutral attitude to
this statement. In the simulation, we set a scenario requiring the nursing to
discuss with patients' families, whether the honest answer should be given in
response to the patient's asking. The role-played nurse chose to support the
ideas of being honest with the patient in the scenario. The student explained and
shared her thoughts and behaviours in the debriefing, which aroused intense
discussion. Accordingly, we found a positive change in Q27. We found there
was no statistical difference in Q23, which indicated our students advocating
flexible visiting time even before the course. Also, the scores of Q13 and Q15
were not improved by the course, showing students were still struggling on the
edge of negative/positive. This result recommended that more extensive and
profound educational intervention might be needed to construct a more stably
positive attitude.
When it comes to the family-centred dimension, Q18 was the only item that was
not changed statistically, because students got a high mean score before the
course. We found before the course nursing students have already admitted the
importance that we should not let the death of the patient harm the remaining
life of the families. Our course corrected the misunderstanding that at the very
last stage of the patient dying process, nurses need to withdraw and let families
stay with the patient solely (Q17). After the course, students become more
understanding of the idea that caring for families should be carried throughout
the whole palliative period (Q7). Students have always been alerted when facing
the patient' families under the pressure of behaving wrongly. After the course,
students become more acceptable about the families' anxiety and would not
regard the families as an interference to the treatments (Q29). Furthermore, we
detected more cooperation that students were willing to invite families to attend
the physical care and psychological connection with the dying person.
5. Conclusion
This research showed an insightful picture of the Macao nursing
undergraduates' knowledge and attitude in palliative care. Their knowledge and
attitude of palliative care were not sufficient for the challenges in hospice
situations. A compulsory palliative care course should be considered taking into
the nursing curriculum.
The course design focused on terminal symptom management, merged with the
principles of palliative care and communication skills. Relieving terminal

93
symptoms and supplying comfort may improve the quality of life for the patient
who was at the last stage of life, also may make their dignity possible. To be
helpful, instead of being helpless, nursing students gained positive attitudes.
Mainly, we conducted two educational methods: the didactic lecture and

simulation practise. The lecture could teach knowledge comprehensively,
profoundly and systematically. Students could apply the learned knowledge for
the simulated patient right after the lecture. This connection between lecture and
simulation decrease the gap between theoretic learning and application. Also,
the simulation permitted students to practise communication with terminal
patients, and role-playing allowed them to understand the sufferings of patients
and their families. Debriefing played a vital role in the teaching process, which
could guide students in sharing their feelings and reflect the life and death of
human beings.
6. Recommendation
Our study supported that palliative care education should be combined into the
nursing bachelor curriculum to ensure undergraduates' competency in palliative
care. The eighteen hours was, in fact, not enough for including all terminal
symptoms. More hours were recommended for the future’s course. Although the
simulation has been widely used in nursing education, the theory preparation
should not be neglected. We strongly recommended lecture-simulation-
combined method for future palliative care education. With positive changes in
both knowledge and attitude, nursing students are expected to have better
coping, which will benefit their future professional and personal career.
7. Limitation
A small number of convenience samples from a single institution limited the
generalizability of the research results. Only using self-reported data may cause
response bias. Objective methods, such as objective structured clinical
examination, were needed to assess students' palliative care behaviour which
belongs to the psychomotor domain. Follow-up retest also should be carried to
find the stability of the effectiveness of palliative care education.
8. Acknowledgement
Sincere thanks to Macao Polytechnic Institute (No: RP/ESS-02/2018) for
sponsoring the research.
I sincerely appreciate Dr Haobin YUAN, who helped to train students in the
course, and to all students who participated in this study.
9. Reference
Aboshaiqah, E. A. (2020). Predictors of palliative care knowledge among nursing
students in Saudi Arabia: A cross-sectional study. Journal of Nursing Research,
28(1), E60. https://doi.org/10.1097/jnr.0000000000000301
Abu-El-Noor, N. I., & Abu-El-Noor, M. K. (2016). The attitude of Palestinian nursing
students toward caring for dying patients: A call for change in health education
policy. J Holist Nurs, 34(2),193-199. https://doi.org/10.1177/0898010115596492

94
Al Qadire, M. (2014). Nurses' knowledge about palliative care: A cross-sectional survey.

Journal of Hospice & Palliative Nursing, 16(1), 23-30.
http://doi.org/10.1097/NJH.0000000000000017
American Association of Colleges of Nursing. (2019). Competencies and recommendations
for education undergraduate nursing students preparing nurses to care for the seriously
ill and their families. Retrieved from https://www.mypcnow.org/wp-
content/uploads/2019/03/New-Nursing-Palliative-Care-Competencies.pdf
Berndtsson, K. E., Margareta, K. G., & Rejnö, C. U. Åsa. (2019). Nursing students'
attitudes toward care of dying patients: A pre- and post-palliative course study.
Heliyon, 5(10), E02578. https://doi.org/10.1016/j.heliyon.2019.e02578
Cant, R., & Cooper, S. (2017). The value of simulation-based learning in pre-licensure
nurse education: A state-of-the-art review and meta-analysis. Nurse Education in
Practice, 27, 45-62. http://doi.org/10.1016/j.nepr.2017.08.012
Carman, J. M., Sloane, R., Molloy, M., Flint, E., & Phillips, B. (2016). Implementation of a
learning bundle to promote end-of-Life education for prelicensure nursing
students. Journal of Hospice & Palliative Nursing, 18(4), 356-363.
http://doi.org/10.1097/NJH.0000000000000258
Chari, S., Gupta, M., Choudhary, M., & Sukare, L. (2016).Knowledge and attitude of
nursing students towards palliative care: The role of focused training. Int Jour of
Biomed Res. 7(9), 629-632. https://doi.org/10.7439/ijbr.v7i9.3576
Dame, L., & Hoebeke, R. (2016). Effects of a simulation exercise on nursing students'
end-of-life care attitudes. The Journal of Nursing Education, 55(12), 701-705.
http://dx.doi.org/10.3928/01484834-20161114-07
DʼAntonio, J. (2017). End-of-Life nursing care and education: End-of-life nursing
education: Past and present. Journal of Christian Nursing, 34(1), 34-38.
http://doi.org/10.1097/CNJ.0000000000000338
Dimoula, M., Kotronoulas, G., Katsaragakis, S., Christou, M., Sgourou, S., & Patiraki, E.
(2019). Undergraduate nursing students' knowledge about palliative care and
attitudes towards end-of-life care: A three-cohort, cross-sectional survey. Nurse
Education Today, 74, 7-14. https://doi.org/10.1016/j.nedt.2018.11.025
English Oxford Living Dictionaries. (2019). Retrieved from
https://en.oxforddictionaries.com/definition/knowledge
Frommelt, H. K. (2003). Attitudes toward the care of the terminally ill: An educational
intervention. Am J Hosp Palliat Care, 20(1),13-22.
https://doi.org/10.1177/104990910302000108
Gillan, P. J. (2014). End of life care simulation: A review of the literature. Nurse Education
Today, 34(5), 766-774. http://doi.org/10.1016/j.nedt.2013.10.005
Grossman, S. (2013). Development of the palliative care of dying critically ill patients
algorithm: Implications for critical care nurses. Journal of Hospice & Palliative
Nursing, 15(6), 355-359. http://doi.org/10.1097/NJH.0b013e3182987b1c
Henoch, I., Melin-Johansson, C., & Bergh, I. (2017). Undergraduate nursing students'
attitudes and preparedness toward caring for dying persons-a longitudinal
study. Nurse Education in Practice, 26, 12-20.
http://doi.org/10.1016/j.nepr.2017.06.007
Kirkpatrick, A., Cantrell, M., & Smeltzer, S. (2019). Relationships among nursing student
palliative care knowledge, experience, self-awareness, and performance: An end-
of-life simulation study. Nurse Education Today, 73, 23-30.
https://doi.org/10.1016/j.nedt.2018.11.003
Jaykumar, S., Karthikbabu, S., Karvannan, H., Kumar, S., Prem, V., Sisodia, V., & Syed,
N. (2012). Study of nurses' knowledge about palliative care: A quantitative cross-

95
sectional survey. Indian Journal of Palliative Care, 18(2), 122-127.

http://doi.org/10.4103/0973-1075.100832
Muhamad, Z., Baskoro, S., & Dicky, K. E. (2018). Nursing students' attitudes towards
caring for dying patients. Nurse Media: Journal of Nursing, 8(1), 25-34.
https://doi.org/10.14710/nmjn.v8i1.17270
Myers, A. M. (2018). The effects of simulation and case study methods of instruction on
undergraduate nursing students' knowledge and attitudes regarding palliative care
(Order No. 10680835). ProQuest Dissertations & Theses Global. (2016870703).
Retrieved from
https://search.proquest.com/docview/2016870703?accountid=12207
Paul, S. S., Renu, G., & Thampi, P. T. (2019). Creating a positive attitude toward dying
patients among nursing students: Is the current curriculum adequate? Indian
journal of palliative care, 25(1), 142-146.
https://doi.org/10.4103/IJPC.IJPC_148_18
Robinson, E., & Epps, F. (2017). Impact of a palliative care elective course on nursing
students' knowledge and attitudes toward end-of-life care. Nurse Educator, 42(3),
155-158. https://doi.org/10.1097/nne.0000000000000342
Rodrigues, S. G. (2015). Translation and validation of the coping scale with death: A
study with nurses. Revista De Enfermagem Referência, (7),113-122.
http://dx.doi.org/10.12707/RIV14084
Ross, M. M., McDonald, B., & McGuinness, J. (1996). The palliative care quiz for nursing
(PCQN): The development of an instrument to measure nurses. Journal of
Advanced Nursing, 23, 126-137. http://doi.org/10.1111/j.1365-
2648.1996.tb03106.x
Sanborn, H., Cole, J., Kennedy, T., & Saewert, K. (2019). Practicing interprofessional
communication competencies with health profession learners in a palliative care
virtual simulation: A curricular short report. Journal of Interprofessional Education
& Practice, 15, 48-54. https://doi.org/10.1016/j.xjep.2019.01.010
Smith, M. B., Macieira, T. G. R., & Bumbach, M. D. (2018). The use of simulation to teach
nursing students and clinicians' palliative care and end-of-life communication: A
systematic review. Am J Hosp Palliat Care, 35(8), 1140-1154.
https://doi.org/10.1177/1049909118761386
Statistics and Census Service of Government of Macao Special Administrative Region.
(2014). Trends and challenges of the ageing population. Retrieved from
https://www.dsec.gov.mo/getAttachment/f5ecdbb4-ad4e-47fb-9937-
98850909b844/pdf1.aspx?disposition=attachment
Tamaki, T. I. (2019). The effectiveness of end-of-life care simulation in undergraduate
nursing education: A randomised controlled trial. Nurse Education Today, 76, 1-7.
https://doi.org/10.1016/j.nedt.2019.01.005
Valen, K., Holm, A., Jensen, K., & Grov, E. (2019). Nursing students' perception on
transferring experiences in palliative care simulation to practice. Nurse Education
Today, 77, 53-58. https://doi.org/10.1016/j.nedt.2019.03.007
World Health Organization. (2002). WHO Definition of Palliative Care. Retrieved from
https://www.who.int/cancer/palliative/definition/en/

96
Appendix 1
Palliative Care Quiz for Nursing

Q1: Palliative care is only appropriate in situations where there is evidence of a downward
irreversible deterioration. (F)
Q2: Morphine is the standard used to compare the analgesic effect of other opioids. (T)
Q3: The extent of the disease determines the method of pain treatment. (F)
Q4: Adjuvant therapies are important in managing pain. (T)
Q5: It is crucial for family members to remain at the bedside until death occurs. (F)
Q6: During the last days of life, drowsiness associated with electrolyte imbalance may
decrease the need for sedation. (T)
Q7: Drug addiction is a major problem when morphine is used on a long-term basis for the
management of pain. (F)
Q8: Individuals who are taking opioids should also follow a bowel regime (laxative
treatment) (T)
Q9: The provision of palliative care requires emotional detachment. (F)
Q10: During the terminal stages of an illness, drugs that can cause respiratory depression are
appropriate for the treatment of severe dyspnea. (T)
Q11: Men generally reconcile their grief more quickly than women. (F)
Q12: The philosophy of palliative care is compatible with that of aggressive treatment. (T)
Q13: The use of placebos is appropriate in the treatment of some types of pain. (F)
Q14: In high doses, codeine causes more nausea and vomiting than morphine. (T)
Q15: Suffering and physical pain are synonymous. (F)
Q16: Pethidine is not an effective analgesic for the control of chronic pain. (T)
Q17: The accumulation of losses makes burnout inevitable for those who work in palliative
care. (F)
Q18: Manifestations of chronic pain are different from those of acute pain. (T)
Q19: The loss of a distant relationship is easier to resolve than the loss of one that is close or
intimate. (F)
Q20: Pain threshold is lowered by fatigue or anxiety. (T)

97
Appendix 2
Frommelt Attitude towards Care of the Dying

Q1. Giving care to the dying person is a worthwhile experience.
Q2. Death is not the worst thing that can happen to a person.
Q3. I would be uncomfortable talking about impending death with the dying person.
Q4. Caring for the patient's family should continue throughout the period of grief and
bereavement.
Q5. I would not want to care for a dying person.
Q6. The non‑family caregivers should not be the one to talk about death with the dying
person.
Q7. The length of time required giving care to a dying person would frustrate me.
Q8. I would be upset when the dying person I was caring for gave up hope of getting
better.
Q9. It is difficult to form a close relationship with the dying person.
Q10. There are times when the dying person welcomes death.
Q11. When a patient asks, "Am I dying?" I think it is best to change the subject to
something cheerful.
Q12. The family should be involved in the physical care of the dying person.
Q13. I would hope the person I'm caring for dies when I am not present.
Q14. I am afraid to become friends with a dying person.
Q15. I would feel like running away when the person actually died.
Q16. Families need emotional support to accept the behavior changes of the dying person.
Q17. As a patient nears death, the nonfamily caregiver should withdraw from his/her
involvement with the patient.
Q18. Families should be concerned about helping their dying member make the best of
his/her remaining life.
Q19. The dying person should not be allowed to make decisions about his/her physical
care.
Q20. Families should maintain as normal an environment as possible for their dying
member.
Q21. It is beneficial for the dying person to verbalise his/her feelings.
Q22. Care should extend to the family of the dying person.
Q23. Caregivers should permit dying persons to have flexible visiting schedules.
Q24. The dying person and his/her family should be the in‑charge decision‑makers.
Q25. Addiction to pain‑relieving medication should not be a concern when dealing with a
dying person.
Q26. I would be uncomfortable if I entered the room of a terminally ill person and found
him/her crying.
Q27. Dying persons should be given honest answers about their condition.
Q28. Educating families about death and dying is not a non‑family caregiver
responsibility.
Q29. Family members who stay close to a dying person often interfere with the
professional's job with the patient.
Q30. It is possible for non‑family caregivers to help patients prepare for death.

98

Vol. 19, No. 8, pp. 98-116, August 2020
Contextualising Computational Thinking:

A Case Study in Remote Rural Sarawak Borneo
Nur Hasheena Anuar, Fitri Suraya Mohamad and Jacey-Lynn Minoi

Universiti Malaysia Sarawak, Kota Samarahan, Sarawak, Malaysia
https://orcid.org/0000-0002-3677-8514
https://orcid.org/0000-0003-4460-8061
https://orcid.org/0000-0002-9464-2631
Abstract. The paper describes an exploratory case study on novice

indigenous children’s learning characteristics as they learn
Computational thinking (CT) competencies, such as abstraction,
decomposition, and algorithmic thinking. It employs a quasi-
experimental research design with pre-test and post-test instruments.
Twenty-two children of an underprivileged Penan community living in a
remote village in Sarawak Borneo participated. Through the study, they
learned Computational thinking skills using localised instructional
strategies, with Scratch™ as their tool to programme. The study used
observational field notes, comprehension checks, and participants’
learning products as primary data sources. Findings showed that
indigenous children’s learning characteristics were primarily ‘learning-
by-making’, collaborative, highly motivated, playful, curious, and
imaginative while they attempted to learn Computational thinking. The
intervention (treatment) group performed marginally better than the
control group in the pre-test and were substantially better in the post-test
performance. Findings illustrate a direction in which novice indigenous
children could learn and be informed about Computational thinking
practices and skills through a mix of game-based learning, collaborative
learning, problem-based learning, and project-based learning. Findings
also revealed how participants appeared to have gained self-confidence,
illustrated creativity on task and were self-critical throughout their
participation in the study.
Keywords: Computational thinking; Remote classroom; Indigenous

children; Penan community; Case study
1. Introduction
Developing computational thinking skills from a young age has become crucial in
today’s setting as it is seen to fulfil the demand of social and global economic
growth (Wing, 2010; Grover, 2018; Haseski, Ilic & Tugtekin, 2018). At the macro
level, ensuring an increase of uptake among young students to pursue STEM
education, and later careers, has become a critical agenda in Malaysia’s Education

99
system. ‘STEM For All’ initiative is a national strategy by the Malaysian Ministry
of Education (MOE) to enhance the participation in STEM and STEAM (Science,
Technology, Reading, Arts and Mathematics)” (Mustafa, 2019). The initiative was
prompted by the decrease of STEM students to a mere 44% in 2019. Another
STEM movement in Malaysia is the mydigitalmaker initiative, spearheaded by
Malaysian Digital Economy Corporation (MDEC). MDEC leads Malaysia’s digital
economy. The programmes and activities are designed in collaboration with
private and public academia. They have collectively impacted 1.2 million students
(Malaysia Digital Economy Corporation, 2019). Under mydigitalmaker initiative,
“Hour of Code” programme has been organised countrywide to cultivate
programming interest and skills among young students.
In line with current demands for 21st Century Learning, MOE has revised its
STEM initiative and Digital Economy, and both standard curricula of primary and
secondary schools, as reported in the Malaysian Education Blueprint 2013-2025
(Ministry of Education, 2013). Beginning 2017, Primary One students have begun
learning basic computer skills such as placing text and images into Word
processors, making slides for the presentation and searching information via an
Internet browser (Kementerian Pendidikan Malaysia [KPM], 2014; Ling, Saibin,
Abidin & Aziz, 2017). Despite the aspirations of the newly revised curriculum,
Malaysian students have not advanced to application-level due to lack of
technology access and exposure. They are still regarded as digital technology
consumers.
The term ‘computational thinking’ became popular when Papert first introduced
LOGO programming to young children for training their procedural thought
(Grover & Pea, 2013). Since then, educators have often begun teaching and
implementing CT in K-12 by introducing computational skills and computational
engagement (Kafai & Burke, 2017; Weintrop et al., 2016). Moschella (2019) defined
computational thinking as mental abilities to perform problem-solving,
abstraction, using algorithms, restructuring processes, reformulating problems,
and implementing solutions. Computational thinking skills are now considered
as a fundamental skill, much like reading, writing, and counting while addressing
real-world issues (Wing, 2006).
A fundamental concern is that educators are unmotivated to adopt computational

thinking strategies in the classroom. Many educators generally are not inspired
due to time limitations to using machine technology, coupled with the lack of
pedagogical skills (Selby, 2014). Despite these challenges, the practices of
computational thinking should be carried out with specific methods and tools as
early as possible (Moschella, 2019). Similar context to this study, My Elephant
Friend, was used as a game tool for education in programming and computational
thinking to educate young novices in India (Unnikrishnan et al., 2016).
Some of the popular tools used by teachers in schools to teach computational

thinking and programming skills, include CS Unplugged, Code.org curricula, and
Scratch (Israel, Pearson, Tapia, Wherfel & Reese, 2015). They mainly involve
interactive and computerised exercises that facilitate the comprehension of

100
children’s computing concepts. However, there is a dearth in the literature about

how teachers at remote rural locations dealt with the teaching of computational
thinking skills. To date, there is no research on how the teaching of computational
thinking is localised to a context, especially for remote rural primary schools.
Hsu, Chang and Hung (2018) suggested that project-based learning, problem-
based learning, collaborative learning, and game-based learning as the most
effective teaching techniques used in early school years to facilitate computational
thinking. They further explained how project-based learning approach enables
learners to learn CT skills in positive attitudes. As a project-based approach is a
group-based activity, it may motivate the learners to work with their peers in
achieving the goals. Through the problem-based approach, students start by
recalling prior knowledge related to a situational problem and apply CT skills into
it. The main advantage of these learning strategies is that it enhances learners’
understanding of new knowledge while building a positive attitude towards
learning and becoming self-directed learners (Cantillon, Hutchinson, & Wood,
2003; Tseng, Chang, Lou & Chen, 2013). Inexperienced learners may face difficulty
understanding concepts in the absence of problem-solving and metacognitive
techniques (Blumenfeld et al., 1991).
It was also suggested that learning CT skills through game-based approach

increased students’ motivation and the ‘flow’ of a learning experience
(Kazimoglu, Kiernan, Bakon & MacKinnon, 2012). In the present study, an
unplugged game was developed by adopting a game-based approach as it was
intended to help students to understand algorithms concept in a playful manner.
With Scratch™, novice learners are expected to encounter less difficulty as it
supports low-level programming and high-ceiling learning environment.
Learners could create and be creative without being evaluated by the programme
or receive syntax error messages from Scratch™.
The focus of this paper is to present a study on learning characteristics of a group

of young novice underprivileged indigenous students, as they learn
computational thinking skills for the first time through a deliberately localised
instructional strategy. We targeted three CT skills: abstraction, decomposition,
and algorithmic thinking, as they have the closest affinity to what primary school
students could use in game-based activities and programming tasks, based on
their environmental setting and knowledge readiness. The CT capabilities in this
study refer to cognitive skills rather than practical skills. The study incorporates
an adaptive model for young novice children to learn CT concepts and practices
through a redesigned CT activity. The data collected are meant to improve the
way children are taught Computational Thinking in a remote learning
environment.
Taking a cue from Hsu et al.’s findings, the study also adapted the problem-based
approach, in which drawing and colouring tasks were localised to a situational
problem. Kafai and Burke (2014) stated that learning how to programme and skills
to interact with others are crucial steps in broadening computational
participation. However, computational participation may be challenging for

101
students who come from low socio-economic backgrounds and reside in distant
schools due to their restricted Internet accessibility and technology usage
(Mohamad, Yeo, Abd Aziz & Rethinasamy, 2010). Thus, the process of acquiring
CT skills for students with various abilities, interest, and different learning paces,
especially novices and underprivileged, maybe highly challenging. Right from the
beginning, we were aware of the challenges to conduct the study with the target
group. Participants worked in a team and helped each other when working on
assigned Scratch™ projects. The learning tasks designed for the study seemed to
have enabled them to build their confidence in dealing with digital technology.
2. Computational Thinking in the Classroom

Computational thinking has been associated with learning of computer science.
For decades, researchers and educators had teamed up to build a computational
thinking model and define its definition, engage, and assess children’s capability
to think computationally. Wing (2006, 2008, 2010) described CT as fundamental
mental activity and skills to both human and machine in solving problems
including logical thinking, analytical thinking, mathematical thinking,
engineering thinking, scientific thinking, and algorithmic thinking to find a
solution. Until today, there is no conclusive definition of CT for all as educators
have various learning demands and objectives to achieve.
In middle- and high- schools of Western countries, CT is taught through a flexible

approach in higher education (National Research Council, 2011). For instance, the
CS Principles course (http://www.csprinciples.org/) attempts to incorporate the
concept of computing and practices of computational thinking such as the
development of digital artefact for creative expression targeted on high school
students (Grover et al., 2013). Through the course, students were reported to be
able to master computational skills and solve problems, communicate new ideas,
and collaborate with others. However, at the point of writing, there is no known
model which has been developed for teachers in remote rural schools to teach
primary school students about Computational thinking skills.
According to Papert and Harel (1991), Constructionism is a ‘learning-by-making’

approach. It is a suitable approach to learning computational thinking skills.
Robles et al. (2017) and Hsu et al. (2018) found that teaching CT skills through
constructionism approach can implement with or without computers. For
instance, CS Unplugged (https://csunplugged.org/) is a module that is used to
teach young learners about computer science topics without using computers. The
topics consist of CT elements implement through intriguing games and paper-
based materials. The advantage of Constructionism and unplugged approaches
are cost-effective and suitable for all age (Durkin et al., 2018). For example,
unplugged cybersecurity modules and activities developed by Durkin and his
colleagues (2018) to promote engagement and understanding in K-12 educators
of computer science issues. The result is promising as K-12 teachers are highly
engaged with their students and allow rapid implementation into their classroom.
Asad, Tibi and Raiyn (2016) admit that learning programming is not easy for
children, as it involves the use of computational and algorithmic thinking. Ben-

102
Ari (as cited in Asad et al., 2016) suggested that Scratch supports the development
of CT skills through low-level programming with a highly supportive learning
environment. Scratch is a visual programming tool that helps learners to retain
information better when dealing with abstract concepts and ideas. It is designed
to support a wide range of learners, including novice, low digital literate, and
adult users (Lin, 2015), also, highly engaging. Learners will not encounter syntax
error when using Scratch for building codes blocks. A language selection function
is included in Scratch, which enables students to choose their language
preferences. Hence, researchers suggest that Scratch is the most suitable tool for
novice learners to learn computational thinking skills.
Moschella (2019) discussed the teaching of three programming elements (i.e.,

abstraction, decomposition and use of algorithms), and their correlation with
young learners and methods adopted for computational thinking for primary
school children. She used a systematic literature review to extract standard tools
commonly used by educators to teach algorithmic thinking, abstraction, and
decomposition at the primary school level. She also found that and Scratch™ and
game-based learning were widely used to teach algorithmic thinking.
Game-based learning (GBL) is a part of the wider gamification approach that use
game mechanics or techniques to learn concepts in a serious but playful manner
to engage learners (Paniagua & Istance, 2018). Kazimoglu et al. (2012) found that
learning CT skills through GBL approach increased students’ motivation and the
‘flow’ of a learning experience. It helps students to understand concepts quickly
and discover how algorithms work. Scratch™ is widely used by young and adult
learners to create creative computational artefacts such as Pinball and Shooter -
games that have a similar approach to GBL.
Problem-based learning (PBL) is a vital process for students to acquire knowledge

and skills (Baharom & Palaniandy, 2013). They had claimed that students started
to learn from problems that occur in their everyday life, which creates prior
knowledge. Slowly, it builds the foundation to obtain new ideas and confidence
in tackling existing problems. Hsu et al. (2018) found that project-based learning
is the most relevant strategy to introduce computational thinking skills at schools.
Project-based learning is a constructivist approach whereby students learn how
to engage and solve problems with a scientific design within a group learning
environment (Tseng et al., 2013). Students who participated in project-based
learning will tend towards positive in learning. Using a project-based learning
approach, students can advance their development of CT skills and develop a
better understanding of the skills. Hsu et al. (2018) supported problem-based
learning is a widely used method in primary school to teach and develop CT skills
of different subjects. It may involve students to develop the program design as
their project-based assignment.
Collaborative learning has also been found to be useful in CT activities as students

would discuss similar problems, communicate with common strategies and plan
for solutions. According to Perković, Settle, Huang and Jones (2010), there are four
dimensions of collaborative learning which are circumstances, interactions

103
between group members, learning systems, and outcomes of collaborative

learning. It allows students to work together, to exchange ideas, and interacting
throughout the learning process. It also helps them to develop teamwork and
interpersonal skills when participating in computational activities.
While most studies reviewed were conducted in developed nations, it is evident

that the challenges in teaching Computational thinking skills in classrooms as
diverse as Malaysia would be different. In the context of this study, teachers in a
remote school may find teaching computational thinking skills to indigenous
children exigent. While the push from the national curriculum toward a more
progressively global trend in digitally integrated ways of learning, teachers in
remote schools seem to affix to their rural learning environments. To date, there
has not been any known research done to understand how indigenous children at
Sarawak learn computational thinking skills such as algorithms, abstraction, and
decomposition, especially in their social context of learning. Localising a relatively
foreign knowledge such as computational thinking would impose a unique
undertaking, as the rural students most likely do not grow up with digital tools
nor simulated environments which would encompass the need to learn
abstraction, decomposition and use of algorithms.
Yadav, Hong and Stephenson (2016) claimed that offering computer science
subject in a rural school would serve a massive challenge as school administrators
and the school board would not see it as a priority subject. In other words,
computer science topics would not be considered in a school examination
subjects’ list. The rationale for excluding computer science subjects in remote
schools may be due to the lack of resources to teach the subject (such as expertise
from IT background, digital equipment, and Internet connectivity). It was
suggested that such subjects could still be offered through the use of unplugged
(non-computer) materials and practical computer tools such as Scratch™.
The present study was designed to introduce localised instructional strategies to

teach computational thinking skills such as algorithmic thinking, abstraction, and
decomposition for a group of young indigenous Penan children in remote rural
Sarawak. Findings from the study would not be generalised to other indigenous
communities in the country as they may embrace dissimilar social, emotional,
mental, and spiritual values.
3. Methodology
The study employs a case study approach to observe, measure and analyse
behavioural and learning impact among 22 young indigenous students who live
in a remote rural location in Sarawak Borneo. Field observations were carried out
to document interactions, behaviours and competencies of the school students
and their teachers. A set of localised instructional strategies were then carefully
selected and developed to introduce Computational thinking concepts to the
participants. They were divided into two groups; one group received an
instructional intervention, where an unplugged coding game was designed to
facilitate the comprehension of Computational thinking concepts. Both groups

104
were introduced Scratch as a tool to produce learning tasks which encapsulate an

understanding of how computational thinking works.
The location of the study is a remote rural village of just over 600 indigenous
Penan, a once nomadic hunter-gatherer tribe which has settled in the area in the
past twenty years (Zaman & Falak, 2019). In 2010, it was estimated about 16,281
of Penan population had settled in Sarawak (Zaman et al., 2016). For Penan
students, they prefer to speak in Penan language as their first language in the
school, instead of the national language, Bahasa Melayu. It is common for rural
communities such as these to use their language for communication due to the
strong cultural influence and attachment in the community. From their formal
education, they gradually learn to use Bahasa Melayu as the primary
communicative language with anyone outside of their tribe (Law, 2018).
The school was built on a wooden structure, and it houses six classrooms and two
administrative rooms. All students come from the same village, and they all live
at the school’s hostel. They eat, sleep and study together, and undeniably their
camaraderie with each other was formidable. During the time of the study, there
are 56 students attended its Primary School. After they complete Primary Six, they
would leave the community to live in a boarding school about one hour away by
boat. There is no gravel road access to the village; the community rely on a river
as their primary transportation route and for providing basic water needs. The
electrical supply was provided by diesel-fuelled single-user generators and a
small hydro-electric unit which was donated by the Government of Japan.
Before the study commenced, we sought permission from the Research Ethics
Committee of University Malaysia Sarawak, the Sarawak State Education
Department, and the Educational Research and Planning Department of the
Ministry of Education in Putrajaya. We have addressed all four main elements of
human and ethics research – respect for persons, respect for personal autonomy,
beneficence and justice, throughout the conduct of the research. On top of the
seeking ethics approval and permission from official channels, we also sought
permission from the community elders using an established cultural protocol
which was developed by Phoa (2009) (Minoi et al., 2018).
The study was conducted over three separate visits to the school and the
community. In the three visits which took place over 12 months, we observed and
interacted with the community members and leaders, and with the schoolteachers
and principal.
Twenty-two children from the school volunteered to participate in our study.

They attended Primary Four (15) and Primary Five (7) classes, aged 10 and 11,
eleven boys and eleven girls. None had any experience in programming and no
background knowledge in computational thinking skills. We describe them as
active, obedient, shy, and possess a high level of curiosity. Through a subject
taught at school, called Information and Communication technology, they have
learned about the critical functions of a computer, and they were familiar with
word processing and digital multimedia. Unfortunately, they have minimal

105
access to computers because of the lack of technology tools at the school and their
own homes. The students mostly preferred to play outdoor sports such as football
and badminton, instead of sedentary games.
Our initial interactions and analysis enabled the design of the localised
instructional strategies, which we eventually used in the introductory sessions on
Computational thinking (Minoi et al., 2018).
Two key activities were designed and developed for the study. One was called
“unplugged coding game” and the second one was focused on introducing
Scratch™ as a tool for programming.
Participants are divided into two groups - the control and treatment groups. The
control group has ten students comprised of four boys and six girls, while the
treatment group had 12 students, seven boys and five girls. For the setting (see
Table 1), both groups shared one classroom space during Lesson 1, 2, 3, and 7 as
they needed to collaborate in the activities. During Lesson 4, 5 and 6, both groups
are separated into two classroom spaces. The treatment group received an
instructional intervention in a classroom while a facilitator in another classroom
monitored the control group.
The treatment group studied a more challenging CT concept and activities such
as ‘variables’ and ‘creating a quiz using Scratch’ with an instructor’s guidance and
facilitation. Using the ‘variable’ concept, they were required to apply it in the
Scratch programme. Both groups were asked to construct a quiz program. The
control group studied at their own pace, with booklet provided and without much
guidance. The control group are encouraged to explore Scratch and learn how to
create any game they desired. A facilitator was available throughout the treatment
phase to assist them when needed.
Table 1: Setting for the study experiment
We designed and deployed a set of localised instructional strategies to introduce

computational thinking skills such as algorithmic thinking, problem
decomposition, and abstraction (see Table 2).

106
Table 2: Computational Thinking activities and instructional strategies
We conducted the lessons on Computational thinking with both groups, and we

observed their interactions, behaviours and comprehension while on task.
Comprehension was measured through a pre-test and a post-test instrument,
which assessed their level of knowledge about Computational thinking. The
instrument consists of questions on algorithmic skill, abstraction, and problem
decomposition. There is a total of five questions in the instrument; four items
focused on knowledge about Scratch™ and Programming, and another item was
written to look at algorithmic thinking (see Appendix). Participants were
provided thirty minutes to undertake each test. The items were rated using scales
ranging from 1 to 4 (from “beginner – showing little understanding of the
concept”, to “advanced - fluent understanding of the concept”.
Photos and videos taken during the lessons were analysed using content analysis.
Narrative analysis was used to examine the interactions which took place during
the lessons.
4. Results and Findings

Twenty-one responses are collected from both treatment and control groups. One
missing data is detected when one participant from the control group was absent
during the post-test (see Fig 1). The horizontal axis indicates an ascending order
of participants from 1 to 10 while the vertical axis shows the percentage of CT
score gained by each sample.

107
Co nt ro l G ro u p
Pre test Post test
50
40
30
20
10
0
1 2 3 4 5 6 7 8 9 10
Figure 1: CT results of the Control group
The CT pre-test results show that the treatment group performed better than the
control group, with a difference of 3.5% (see Table 3). It shows that participants in
the treatment group performed slightly better as compared to the control group.
Table 3: The mean score and differences of scores for both groups
After the post-test, the treatment group shows an improvement of 1.7% while the
control group shows a decrease of 6.2%. It indicates that whole-class instruction
and open-ended activities were helpful for these participants to learn CT skills,
even though the results are not conclusive. However, other factors, such as
personal language competencies and problem-solving ability, might have also
influenced the results. Based on Fig 2, the results of the treatment group show
only six participants had improved their post-test performance while others either
showed a decrease or their scores remained unchanged.

108
T REAT ME NT G RO UP
Pre test Post test
50
40
30
20
10
0
1 2 3 4 5 6 7 8 9 10 11 12
Figure 2: CT results of the Treatment group
Overall, participants demonstrated high motivation during all of the

computational thinking activities. In the unplugged game-based coding activity,
22 participants were divided into five groups and these groups are further divided
into subgroups. Each group formed two more subgroups of two to three people,
as they were instructed to compete with the other subgroups during the game-
based activity.
The unplugged coding game produced a mixed reaction among the participants.
Although they are repeatedly reminded of the game mechanics, most still failed
to comprehend them. Some of the female participants found it challenging to
comprehend the mechanics of the game, and they eventually gave up playing.
Based on our observations, the game might not have resembled anything they
have played before, and they might have lost interest in it because the game did
not present a worthy challenge to them. The phenomenon was evident among
those who did not notably score high in the pre-test and post-test. We assumed
that their performance was slowed by the unplugged coding game, which had
required participants to use abstract thinking skills to identify steps and directions
to the desired goal (see Fig 3). In our observation, it was a challenge for them to
think abstractly and systematically. We conclude that these participants might
have required extended support and repetitive scaffolding to guide them to
internalise the “foreign” algorithmic concepts.
Figure 3: Participants attempting to play an unplugged algorithmic thinking game

109
In the “Getting started with Scratch” activity, the Scratch™ programme was
introduced to the participants to get them to familiarise with the blocks and utilise
the tool. The Malay language is used as a medium to teach them Scratch. Notes
are given to guide them getting started with Scratch in case they missed the steps
demonstrated by the instructor. It was observed that during the sessions, the
participants communicated in their mother tongue, the Penan language. The code-
switching between Penan and Malay as they interacted with the facilitators and
among themselves seemed to have supported their understanding of the assigned
tasks.
In the problem-solving session, participants were assigned to draw a map based

on a scenario given and label the vital information such as location, places
(schools), object (boat), and time of departure and arrival (see Fig 4). This lesson
aims to practice abstraction skill, decomposition skill, and algorithmic thinking.
Before drawing the map, they were briefed with a familiar situation of a travelling
problem. The facilitator used a similar example and demonstrated an illustration
for them to visualise the intended situation. The details of information were added
to the map as essential elements for mark-up and problem-solving.
Figure 4: A travel map was drawn by using Abstraction skills
During the tinkering session, the participants began to use Scratch™ to complete
the tasks given. Most of the groups managed to complete the tasks. Some of the
participants have had to seek further guidance and support, as they were
uncertain about the steps to add or change the sprites in Scratch™.
To measure participant’s understanding of using Scratch™ as a tool for

programming, questions such as “Do you know what the sprite was doing in the
program?” and “Could you describe the flow of the blocks you generated just
now?” were asked to the participants. A participant gave only a partial answer as
he could not understand the flow of the Scratch™ programme. Our observations
revealed that some participants had needed more time and support to fully
understand how Scratch™ works. Some had trouble understanding the value of
basic steps when arranging code blocks in the programme. However, throughout
all of the sessions, they displayed a high level of curiosity and determination. They
intentionally made an effort to try parameters to test and play the sprites within
Scratch, a sign of comprehension progress.

110
At times, in between the sessions, we observed that some of the female

participants showed anxiety and displacement. The male participants generally
showed a higher interest in learning programming. When they were able to use
the code blocks successfully to create new games, they displayed enthusiasm and
excitement. The male participants overall presented a more serious commitment
to the assigned tasks. When they found a problem, they would take the initiative
to send someone from their team to ask for clarification from the facilitator.
In another instance, one group successfully created a new sprite using a painting
tool in Scratch. It was not taught in any of the sessions. While creating the sprite’s
design, one female participant from the group faced issues navigating with the
mouse. Her limited navigational skills hampered her effort to complete the design
of the new sprite for her team. It was interesting to observe the group interaction
and support transpired throughout the predicament faced by the leader in their
team, much of which contributed to the success of the group as a whole.
Before any of the session started, the participants were divided into two different
classes. Twelve participants are grouped as the treatment group in a classroom
while the rest in the control group in another classroom. We only fully observed
the treatment group while the control group was left on their own to manage the
assigned tasks.
In the first session, the twelve participants in the treatment group were divided
into four groups. A facilitator started to introduce the concept of variables to them
and demonstrating on the whiteboard. All participants paid close attention
throughout the explanation. Daily examples local to the participants were used to
elaborate on the concepts introduced in the session. They learned to make
connections between variables made up of their own names, favourite subjects in
school and the daily items they used at home. In the following session, the
participants learned how to create a quiz using Scratch™. Using the variable
concept they had just learned, they learned a numeric example and experimented
on it to create a conditional statement (IF-THEN).
IF number > 1,
THEN True,
ELSE False.
Using examples which have already been discussed, the participants learned to
apply the information into a quiz in Scratch™. Then, the participants construct
their arithmetic equation and write it on paper. All participants were able to
complete the tasks without difficulties.
In these initial sessions, both treatment and control groups were placed in the
same classroom. Before the session began, the participants were given options to
choose what type of project they want to create. Majority of them chose to create
Scratch game, and only one group chose to create a Math quiz. During the project
creation phase, the participants used decomposition skills to arrange the blocks
and designed the structures of their team’s programmes. They selected each block

111
carefully and connected it to the appropriate block to create a smooth flow of the
intended action. Most of the groups who had opted to create a Scratch™ game
were observed to have preferred to read a provided guidebook which contained
all information about creating games in Scratch™.
The Math quiz group, surprisingly, worked very well with each other and were
on task. They were able to complete the Math quiz programme collaboratively
and had asked for minimal assistance throughout the creation process. They
invited the facilitators to try out their Math quiz as soon as they were done. The
group showed excitement, relief and pride when their programme tested well. At
this point, the group members began to explore Scratch™ even further. They
started experimenting with creating new sprites on their own. We observed that
they were able to cope with the task by referring to the guidebook provided.
Participants then used decomposition skills during the project development to

arrange the code blocks and design the program. Each block should be selected
carefully and attached to the corresponding block to ensure a smooth output.
They began to familiarise themselves with the use of a mouse and create a Scratch
game without the help of the teacher and facilitator.
Figure 5: A participant created a game in Scratch using Decomposition skills
Fig 6 shows a new Fish sprite drawn by the participants without being instructed.
Their unsupervised attempts demonstrated their ability and keenness to work
independently and use available materials such as Scratch booklet to create new
sprites.
New sprite
is drawn by
one of the
participants.
Figure 6: A fish game built by one of the groups

112
In summary, these participants able to adopt computation thinking skills such as

abstraction, problem decomposition, and algorithmic thinking. After a few
sessions, the participants able to familiarise with the use of Scratch and apply CT
skills. Although each group created little during the task, they showed positive
engagement during the activities such as asking questions, helping others and
play with their peers. In contrast, these activities were not able to determine their
level of CT skills because the instructor and facilitators guided all activities.
During collaborative learning, the participants worked on an art-based activity,

exploration with Scratch™, and creating a Scratch™ project. We saw how most of
the participants enjoyed drawing and colouring activity, notably much more than
the programming tasks. Brennan and Resnick (2012) described that not only
pointing, clicking, browsing, and chatting as skills to develop a computational
thinker. A computational thinker can do more than a consumer, whereby they use
computation skills for computational expressions (Brennan and Resnick, 2012).
Although the students had never experienced working with numbers before they
participated in the study, they were able to cope with the computational thinking
tasks. Observations captured in the study were synonymous with Yadav et al. ’s
(2014) suggestion on young children being introduced early to computational
thinking skills to solve computational problems. Scratch™ is a powerful tool for
children who are inexperienced to practice CT and think like a designer.
Throughout their use of computational thinking skills, these children were also
becoming more interested in using technology for problem-solving.
Figure 7: A participant drew a new sprite using the Editor tool in Scratch
As observed, many children enjoyed participating in art-based activity (see Fig 7),
which is comfortable and freer to do what they want. The study suggests that
children can be creative and innovative if they are encouraged to explore the topic
through art and active learning. Furthermore, they are more likely to recall their
learning experience through meaningful experience, which linked to creative
problem-solving and discovery learning (Winner et al., 1888). The same authors
also found that art education could enhance students’ learning motivation,
reducing the number of students in absenteeism, and promoting greater
participation. At the end of the activity, these children can perceive technology as
a powerful medium to express creative ideas rather than solely following the
teacher’s instruction. When doing project-based learning, they appeared to have

113
asked more questions as they were eager to know about which function blocks
were required to make the sprites to move.
5. Conclusion
Based on CT performance during the study, it showed that these novice
indigenous children were able to learn and use the “foreign” concepts of
abstraction, decomposition, and algorithmic thinking. They exhibited learning
characteristics such as ‘learning-by-making’, collaborative, highly motivated,
playfulness, curiosity, and creativity. Although the CT post-test results showed
that understanding of computational concepts was poor, confidence and
engagement levels have visibly increased during the construction of their
programming projects. We observed their need for supportive peers, guided
scaffolding to learn the contents, and a positive learning environment. Language
proficiency also played a role in enabling the learning of “foreign” concepts such
as computational thinking in a remote setting as such. The codeswitching from
Penan to Malay in verbal interactions, and to English as they read prompts from
Scratch, might have affected the way the participants understood computational
thinking skills.
The study has illustrated the effectiveness of mixing game-based learning,

collaborative learning, problem-based learning, and project-based learning to
present a new “foreign” concept to young novice indigenous children in engaging
and understanding considered CT skills. These techniques, when used with
localised examples which exist in the participants’ home environment, were
deemed culturally adaptable to the learning of computational thinking skills.
Preliminary results from the control group’s post-test suggest that an unguided
instructional approach is less effective than guided instructional strategies. Future
research is needed to pursue a deeper understanding of the development of CT
skills among young novice learners such as these Penan school children.
Computational thinking skills may not be an obvious schema in their living
environment and teaching such concepts would require a deeper understanding
of how teachers could make connections between indigenous knowledge and
curriculum-prescribed learning contents.
6. Acknowledgement
The research project was funded by Newton AHRC-MoHE (UK SEA-NUOF JPTS
(BPKI)2000/04/07/05(33). The authors appreciate the support of the Institute
Social Informatics and Technological Innovations (ISITI) and Dr John Phua from
Universiti Malaysia Sarawak (UNIMAS), the Ministry of Education, Sarawak
State Education Department, and guidance from all teachers at SK Long Lamei
throughout the study.
7. References
Asad, K., Tibi, M., & Raiyn, J. (2016). Primary School Students’ Attitudes toward Learning
Programming through Visual Interactive Environments. World Journal of
Education, 6(5), 20–26. https://doi.org/10.5430/wje.v6n5p20
Baharom, S., & Palaniandy, B. (2013). Problem-Based Learning: A Process for the
Acquisition of Learning and Generic Skills. The 4th International Research

114
Symposium on Problem-Based Learning (IRSPBL), 47–55. Retrieved from

http://tree.utm.my/wp-content/uploads/2013/11/Problem-Based-Learning-
A-Process-for-the-Acquisition-of-Learning-and-Generic-Skills.pdf
Berger, D. N. (2019). The Indigenous World 2019. Retrieved from
https://www.iwgia.org/en/documents-and-publications/documents/4-the-
indigenous-world-2019/file.html
Blumenfeld, P. C., Soloway, E., Marx, R. W., Krajcik, J. S., Guzdial, M., & Palinscar, A.
(1991). Motivating project-based learning: Sustaining the doing, supporting the
learning. Educational Psychologist, 26(3–4), 369–398.
Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the
development of computational thinking. AERA, 1–25. Retrieved from
https://web.media.mit.edu/~kbrennan/files/Brennan_Resnick_AERA2012_CT
.pdf
Cantillon, P., Hutchinson, L., & Wood, D. (2003). ABC of learning and teaching in medicine:
Learning and teaching in the clinical environment. London, England: BMJ Publishing
Group Ltd. Retrieved from
http://edc.tbzmed.ac.ir/uploads/39/CMS/user/file/56/scholarship/ABC-
LTM.pdf
Creswell, J. W. (2012). Educational research planning conducting and evaluating quantitative
and qualitative research (4th ed.). Boston, MA: Pearson.
Creswell, J. W., & Piano Clark, V. L. (2007). Designing and conducting mixed methods
research. Australian and New Zealand Journal of Public Health, 31(4), 388–388.
https://doi.org/10.1111/j.1753-6405.2007.00096.x
Durkin, S. S., Fees, R. E., da Rosa, J. A., Murray, M. M., & Moran, A. L. (2018). Unplugged
cybersecurity: An approach for bringing computer science into the classroom.
International Journal of Computer Science Education in Schools, 2(1).
https://doi.org/10.21585/ijcses.v2i1.21
Grover, S. (2018). The 5th ‘C’ of 21st Century Skills ? Try Computational Thinking (Not Coding).
Retrieved from https://www.edsurge.com/news/2018-02-25-the-5th-c-of-21st-
century-skills-try-computational-thinking-not-coding
Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the
field. Educational Researcher, 42(1), 38–43.
https://doi.org/10.3102/0013189X12463051
Haseski, H. I., Ilic, U., & Tugtekin, U. (2018). Defining a new 21st Century Skill-
Computational Thinking: Concepts and trends. International Education Studies,
11(4), 29. https://doi.org/10.5539/ies.v11n4p29
Hsu, T. C., Chang, S. C., & Hung, Y. T. (2018). How to learn and how to teach
computational thinking: Suggestions based on a review of the literature.
Computers and Education, 126(July), 296–310.
Israel, M., Pearson, J. N., Tapia, T., Wherfel, Q. M., & Reese, G. (2015). Supporting all
learners in school-wide computational thinking: A cross-case qualitative analysis.
Computers and Education, 82(March), 263–279.
Kafai, Y. B., & Burke, Q. (2014). Connected gaming: Towards integrating instructionist and
constructionist approaches in K-12 serious gaming. Proceedings of International
Conference of the Learning Sciences, ICLS, 1(January), 86–93. Retrieved from
https://www.researchgate.net/publication/287602204_Connected_gaming_To
wards_integrating_instructionist_and_constructionist_approaches_in_K-
12_serious_gaming
Kafai Y.B., Burke Q. (2017). Computational participation: Teaching kids to create and

115
connect through code. In Rich P., Hodges C. (eds), Emerging Research, Practice, and
Policy on Computational Thinking. Educational Communications and Technology: Issues
and Innovations (ECTII). Springer, Cham. https://doi.org/10.1007/978-3-319-
52691-1_24
Kazimoglu, C., Kiernan, M., Bacon, L., & MacKinnon, L. (2012). Learning programming at
the computational thinking level via digital game-play. Procedia Computer Science,
9(December), 522–531. https://doi.org/10.1016/j.procs.2012.04.056
Kementerian Pendidikan Malaysia [KPM]. (2014). RPT TAHUN 1 KSSR RPT Dunia Sains
Dan Teknologi (SK) Tahun 1. Retrieved from
http://www.sistemguruonline.my/2014/10/rpt-tahun-1-kssr-rpt-dunia-sains-
dan.html
Law, S. (2018) Vitaliy of Mother Tongue Language Among Penan and Chinese
Communities in Sarawak. Universiti Malaysia Sarawak. Unpublished thesis.
Lin, V. J. (2015). Computational thinking and technology toys (Honors Thesis Collection,
Wellesley College). Retrieved from
https://repository.wellesley.edu/thesiscollection/307
Ling, U. L., Saibin, T. C., Labidin, J., & Aziz, N. A. (2017). Preliminary investigation:
Teachers’ perception of computational thinking concepts. Journal of
Telecommunication, Electronic and Computer Engineering, 9(2), 23–29. Retrieved from
https://www.researchgate.net/publication/324886429_Preliminary_Investigati
on_Teachers’_Perception_on_Computational_Thinking_Concepts
Malaysia Digital Economy Corporation [MDEC]. (2019). Who we are. Retrieved from
https://mdec.my/about-mdec/who-we-are/
Ministry of Education [MOE]. (2013). Malaysia Education Blueprint 2013-2025 (Preschool
to Post-Secondary Education). Kementerian Pendidikan Malaysia. Retrieved from
https://www.moe.gov.my/menumedia/media-cetak/penerbitan/dasar/1207-
malaysia-education-blueprint-2013-2025/file
Mohamad, F. S., Yeo, A. W., Abd Aziz, N., & Soubakeavathi, R. (2010). Borneo children in
an international digital playground: Intercultural issues and idiosyncrasies. In
Proceedings of the 3rd ACM International Conference on Intercultural Collaboration
(ICIC ’10). Association for Computing Machinery, New York, USA.
https://doi.org/10.1145/1841853.1841869
Moschella, M. (2019). Observable computational thinking skills in primary school
children: How and when teachers can discern abstraction, decomposition and use
of algorithms. INTED2019 Proceedings, 1(March), 6259–6267.
https://doi.org/10.21125/inted.2019.1523
Mustafa, Z. (2019, March 20). STEM policies set for an evolution. New Straits Times.
Retrieved from https://www.nst.com.my/education/2019/03/471176/stem-
policies-set-evolution
Paniagua, A., & Istance, D. (2018). Teachers as designers of learning environments: The
importance of innovative pedagogies [e-book]. Educational Research and Innovation,
OECD Publishing, Paris. https://doi.org/10.1787/9789264085374-en
Papert, S., & Harel, I. (1991). Situating Constructionism. Constructionism, 1–17. Retrieved
from http://web.media.mit.edu/~calla/web_comunidad/Reading-
En/situating_constructionism.pdf
Perković, L., Settle, A., Hwang, S., & Jones, J. (2010). A framework for computational
thinking across the curriculum. Proceedings of the Fifteenth Annual Conference on
Innovation and Technology in Computer Science Education - ITiCSE ’10.
https://doi.org/10.1145/1822090.1822126
Robles, G., Barone, D., Brackmann, C. P., Román-González, M., Moreno-León, J., & Casali,
A. (2017). Development of computational thinking skills through unplugged

116
activities in primary school. In Proceedings of the 12th Workshop on Primary and

Secondary Computing Education (WiPSCE ’17), ACM, November, 65–72.
https://doi.org/10.1145/3137065.3137069
Selby, C. C. (2014). How can the teaching of programming be used to enhance computational
thinking skills? (Doctoral dissertation, University of Southampton, England,
United Kingdom). https://doi.org/10.1016/j.jsv.2010.04.020
Tseng, K. H., Chang, C. C., Lou, S. J., & Chen, W. P. (2013). Attitudes towards science,
technology, engineering and mathematics (STEM) in a project-based learning
(PjBL) environment. International Journal of Technology and Design Education, 23(1),
87–102. https://doi.org/10.1007/s10798-011-9160-x
Unnikrishnan, R., Amrita, N., Muir, A., & Rao, B. (2016). Of elephants and nested loops:
How to introduce computing to youth in rural India. In Proceedings of IDC 2016 -
The 15th International Conference on Interaction Design and Children, 137–146.
https://doi.org/10.1145/2930674.2930678
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016).
Defining computational thinking for mathematics and science classrooms. Journal
of Science Education and Technology, 25(1), 127–147.
https://doi.org/10.1007/s10956-015-9581-5
Wing, J. M. (2006). Computational thinking. Communications of the ACM.
https://doi.org/10.1145/1118178.1118215
Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical
Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.
https://doi.org/10.1098/rsta.2008.0118
Wing, J. M. (2010). Computational thinking: What and why?. The link - The Magazine of the
Carnegie Mellon University School of Computer Science. Retrieved from
https://www.cs.cmu.edu/~CompThink/resources/TheLinkWing.pdf
Winner, E., Goldstein, T. R., & Vincent-Lancrin-Stéphan. (1888). Art for art’s sake?:
Overview. OECD. Retrieved from
http://www.oecd.org/education/ceri/ART%20FOR%20ART%E2%80%99S%20
SAKE%20OVERVIEW_EN_R3.pdf
Xie, Q. (2016). Research design and methods. In English Language Training in the
Workplace. Technical and Vocational Education and Training: Issues, Concerns and
prospects, 22, 99–116. Springer, Cham. https://doi.org/10.1007/978-3-319-30157-
0_5
Yadav, A., Hong, H., & Stephenson, C. (2016). Computational thinking for all: Pedagogical
approaches to embedding 21st-century problem-solving in K-12 classrooms.
TechTrends, 60(6), 565–568. https://doi.org/10.1007/s11528-016-0087-7
Yadav, A., Mayfield, C., Zhou, N., Hambrusch, S., & Korb, J. T. (2014). Computational
thinking in elementary and secondary teacher education. ACM Transactions on
Computing Education, 14(1). https://doi.org/10.1145/2576872
Zaman, T., & Falak, H. (2019). Framing indigenous knowledge in the digital context:
Technologies, methods and tools. International Journal of End-User Computing and
Development. https://doi.org/10.4018/ijeucd.2018070103
Zaman, T., Kulathuramaiyer, N., & Yeo, A. W. (2016). eToro: Appropriating ICTs for the
management of Penans’ indigenous botanical knowledge. In L. E. Dyson, L. E., S.
Grant, & M. Hendriks (eds.), Indigenous people and mobile technologies: Routledge
studies in new media and cyberculture (1st ed., pp. 253-264). New York: Routledge.
https://doi.org/10.4324/97813157593

Appendix 1
Computational Thinking Questions

Soalan Pemikiran Komputational
(1)
Nama: ____________________________
Jantina: Lelaki / Perempuan
Kelas Darjah: __________
2018
A1
Computational thinking pre-test questions
1. The plane needs to find its way to the circle. Write the steps on how to move to the
destination. You may use instructions such as go forward, turn right, and turn left.
Pesawat perlu mencari laluannya ke bulatan. Tulis langkah-langkah bagaimana untuk bergerak ke
destinasi. Anda boleh menggunakan set arahan ke hadapan, belok kanan, dan belok kiri.
Jawapan
A2
2. The sprite is going forward when you press the ‘0’ button. But it cannot move
backwards. Could you help to solve this problem? Write your solution at any space
below.
Sprit akan bergerak ke hadapan apabila anda menekan butang '0'. Tetapi ia tidak dapat bergerak
ke belakang. Bolehkah anda membantu menyelesaikannya? Tulis penyelesaian anda di mana-mana
ruang di bawah.
Jawapan:
Bergerak ke hadapan Bergerak ke belakang
A)
B)
A3
3. Your class teacher has assigned you a task which is to create a program. There are two
characters, John and Esa in the program. The condition is John will say “Apa khabar?” to
Esa and Esa will say the same thing to John. But they say it at the same time. How to fix
that?
Guru kelas anda telah memberikan anda satu tugas iaitu menghasilkan satu program. Terdapat
dua watak, John dan Esa dalam program ini.Syaratnya ialah John akan berkata "Apa khabar?"
kepada Esa dan Esa akan mengatakan perkara yang sama kepada John. Tetapi John dan Esa
mengatakannya pada masa yang sama. Bagaimana untuk membetulkannya?
Jawapan
A4
4. A cat has nine lives. If it collides with a squirrel, one life is lost. If all nine lives are lost,
the game is over. Fill in the blanks in this script to make these rules work.
Seekor kucing mempunyai sembilan nyawa. Jika ia bertembung dengan tupai, satu nyawa akan
hilang. Sekiranya semua nyawa hilang, permainan ini akan ditamatkan. Isi ruang kosong dalam
skrip ini untuk membuat peraturan ini berfungsi.
Jawapan
A)
B)
C)
D)
A5
5. Your class teacher has assigned you to create a calculator machine by programming. The
problem is “What number to add with 71 so that you get 203?”
Guru kelas anda telah menetapkan anda untuk membuat matematik kalkulator dengan
pengaturcaraan. Masalahnya ialah "Nombor apakah yang perlu ditambah dengan 71 untuk
mendapatkan 203?"
Definition (Istilah):
• Ask (bertanya)
• Wait (tunggu)
Jawapan
A)
B)
C)
D)
A6
Appendix 2
Computational Thinking Questions

Soalan Pemikiran Komputational
(2)
Nama: ____________________________
Jantina: Lelaki / Perempuan
Kelas Darjah: __________
2018
A7
Computational thinking post-test questions
1. The plane needs to find its way to the circle. Write the steps on how to move to the
destination. You may use instructions such as go forward, turn right, and turn left.
Pesawat perlu mencari laluannya ke bulatan. Tulis langkah-langkah bagaimana untuk bergerak ke
destinasi. Anda boleh menggunakan set arahan ke hadapan, belok kanan, dan belok kiri.
Jawapan
A8
2. The sprite is going forward when you press the ‘0’ button. But it cannot move
backwards. Could you help to solve this problem? Write your solution at any space
below.
Sprit akan bergerak ke hadapan apabila anda menekan butang '0'. Tetapi ia tidak dapat bergerak
ke belakang. Bolehkah anda membantu menyelesaikannya? Tulis penyelesaian anda di mana-mana
ruang di bawah.
Jawapan:
Bergerak ke hadapan Bergerak ke belakang
A)
B)
A9
3. Your class teacher has assigned you a task which is to create a program. There are two
characters, John and Esa in the program. The condition is John will say “Apa khabar?” to
Esa and Esa will say the same thing to John. But they say it at the same time. How to fix
that?
Guru kelas anda telah memberikan anda satu tugas iaitu menghasilkan satu program. Terdapat
dua watak, John dan Esa dalam program ini.Syaratnya ialah John akan berkata "Apa khabar?"
kepada Esa dan Esa akan mengatakan perkara yang sama kepada John. Tetapi John dan Esa
mengatakannya pada masa yang sama. Bagaimana untuk membetulkannya?
Jawapan
A10
4. A cat has eleven lives. If it collides with a squirrel, one life is lost. If all eleven lives are
lost, the game is over. Fill in the blanks in this script to make these rules work.
Seekor kucing mempunyai sebelas nyawa. Jika ia bertembung dengan tupai, satu nyawa akan
hilang. Sekiranya semua nyawa hilang, permainan ini akan ditamatkan. Isi ruang kosong dalam
skrip ini untuk membuat peraturan ini berfungsi.
Jawapan
A)
B)
C)
D)
A11
5. Your class teacher has assigned you to create a calculator machine by programming. The
problem is “What number to add with 71 so that you get 203?”
Guru kelas anda telah menetapkan anda untuk membuat matematik kalkulator dengan
pengaturcaraan. Masalahnya ialah "Nombor apakah yang perlu ditambah dengan 71 untuk
mendapatkan 203?"
Definition (Istilah):
• Ask (bertanya)
• Wait (tunggu)
Jawapan
A)
B)
C)
D)
A12
117

Vol. 19, No. 8, pp. 117-136, August 2020
Can Peer to Peer Interaction (PPI) be a Global

Theme to Promote Engagement in Students of
Diverse Characteristics and Country Contexts?
Nazlee Siddiqui*
University of Tasmania, Australia
https://orcid.org/0000-0002-1841-3095
Khasro Miah
North South University, Bangladesh
https://orcid.org/0000-0002-6499-3545
Afreen Ahmad Hasnain

Southeast University, Bangladesh
https://orcid.org/0000-0002-4474-6942
David Greenfield
University of Tasmania, Australia
https://orcid.org/0000-0002-0927-6025
Abstract. Investigating postgraduate students’ experience of peer-to-

peer interaction (PPI) to promote engagement, across diverse student
characteristics and country contexts, is rare, but a task necessary to
improve outcomes for increasingly diverse students in higher education.
This study implemented a questionnaire survey in an Asian developing
(i.e. Bangladesh; n=65) and a Western developed country (i.e., Australia;
n=28) to address two research questions: first, is student experience of
PPI to promote engagement consistent across developing and developed
country contexts? Second, do characteristics of students influence their
experience of PPI? In both contexts, PPI facilitated students’ discussion
of readings from different viewpoints, cognition to apply classroom
learning to work and teamwork and practical problem-solving skills. In
the developed country, students’ age negatively correlated to
engagement with readings (r=-.644) and cognition of applying
classroom learning to work (r=-.649). In the developing country, age did
not impact on the experience of PPI, whereas a lack of adequate
technology had a negative impact. Working students in the developed
country, unlike that of the developing country, were critical of relying
on peers, reflecting the influence of individualism cultural orientation.
The study implies PPIs can be a global theme to promote student
engagement if developed in alignment with the pedagogy of social
constructivism and academic and cognitive student engagement themes.

118
Furthermore, academics should design PPIs in partnership with

students, accommodating the PPIs to the characteristics of relevant
student cohorts and contexts. Future studies of a greater sample size will
facilitate the agenda for effective PPIs for all students.
Keywords: Peer-to-peer interaction; Diversity of students; Pupils

engagement; Student engagement; Developed country
1. Introduction
Investigation of student engagement is a decades’ old journey, but at the same
time, it is an ongoing agenda in education systems (Holmes, 2018). Scholars
indicate a global trend of increasing student diversity in higher education
(Andrew, 2019; Perez, Robbins, Harris, & Montgomery, 2020; Sanger, 2020;
Tanaka, 2019; Universities Australia, 2019). How student diversity shapes the
themes of student engagement is an evolving process. Peer to peer interaction
(PPI) is recognised as a common theme of student engagement (Kahu & Nelson,
2018; Zhoc, Webster, Li, & Chung, 2018). Understanding of the PPI is also
evolving, being an important aspect of ongoing investigations of student
engagement (Siddiqui, Miah, & Ahmad, 2019).
PPI is a learning and teaching practice that is frequently used in higher

education(Beaumont, Mannion, & Shen, 2012; Power & Vaughan, 2010; Stigmar,
2016). It involves students working collaboratively with a common purpose in
real-time, in one to one or small teams, either in the face to face or online mode.
PPI has to negotiate with diversity factors at a contextual level such as the
different technology platforms, teacher’s capability, and level of resource
availability in the learning environment (Jawhar & Subahi, 2020; Kahu & Nelson,
2018). Moreover, PPI practices rely on students’ personal capability to socially
interact with peers who are likely to be of different profiles, for example, by
lifestyle (working versus non-working) or culture (local versus international) or
age (matured versus young).
It has become necessary to critically investigate the interrelation between PPI

and enhanced student engagement, across students from different countries, to
inform the macro contextual factors involved in this interrelationship. Moreover,
investigations are required to identify PPI learning and teaching approaches that
can be accommodated to benefit all students, at a micro-level, regardless of
students’ personal capability and characteristics. The premise of the
investigations lies in PPI being a learning activity that is influenced by
individual students and the surrounding social world (Broer, Bai, & Fonseca,
2019; Lave & Wenger, 1991). Academics who are aware of how PPI is
experienced by students of divergent backgrounds at the macro, as well as micro
levels, will be better equipped to facilitating an inclusive learning environment
(Lawrie et al., 2017). Moreover, academics with such awareness can promote a
generalised learner-centered approach for collaborative learning (Hoidn, 2016).
Authors find that studies of comparative assessment of PPI between students of

different contexts are rare. An exception is a study of Lam et al. (2016) that

119
reported an insignificant influence of PPI in school students’ engagement across

twelve European, the United States of America (USA), and Southeast Asian
countries. It is possible that the situation in higher education is different from
what is reported at schools. Previous literature evidenced the beneficial
influence of PPI in higher education across different countries, but also pointed
out that more than 60% of studies in the review belonged to the western
education context (Stigmar, 2016). The study of PPI in Asian developing country
contexts is underrepresented (Siddiqui et al., 2019) and a comparative
assessment of PPI between contrasting education contexts (for example,
developing versus developed country) is exceptional.
1.1 Research Questions

This study investigates students’ experience of PPI in promoting engagement,
with an application of the same PPI assessment across students of two divergent
country contexts. The two research questions addressed in this study are:
Question 1: Is student experience of PPI to promote engagement consistent
across developing and developed country contexts?
Question 2: Do characteristics of students influence their experience of PPI?
1.2 PPI and engagement of students of different characteristics and contexts

In the early literature of student engagement, for example in the work of Astin
(1984); the phenomenon was described as students’ overall involvement in the
learning experience to yield desirable academic achievements. Further research
has clarified student engagement as a multi-dimensional phenomenon;
consisting of emotional, behavioural, cognitive and sociocultural themes
(Bowden, Tickle, & Naumann, 2019; Zhoc et al., 2018).
According to the contemporary framework of student engagement by Zhoc et al

(2018), PPI builds student engagement through social interaction with peers.
This theme includes collaborative academic learning, as well as, beyond the
classroom interactions between student peers. The theme of PPI functions side
by side to the other four themes of student engagement, which are: academic
engagement, covering online and offline student behaviours to achieve academic
learning; cognitive engagement, consisting of students’ psychological
investment to master knowledge; social engagement with teachers,
encompassing supportive interactions between teacher and student in the
academic environment; and affective engagement, encompassing students’
emotional bonding with the learning institution (Zhoc et al., 2018).
The literature provides consistent evidence of PPI’s role in student engagement

in higher education. Globally, in developed countries such as the USA
(National Survey of Student Engagement, 2019), United Kingdom (Neves, 2019),
Australia (Coates, 2010; Quality indicators for learning and teaching, 2020);
learning with peers is a common theme in the national survey of student
engagement in higher education. A foundational literature to understand
students’ experience of PPI is the pedagogy of social constructivism, which
proposes students achieve deeper learning with constructive peer interactions
(Powell & Kalina, 2009). Academics need to spend substantial time and

120
resources to design PPIs with features of social constructivism such as critical

thinking, collaborative tasks and the task being meaningful to students (Powell
& Kalina, 2009; Siddiqui, Miah, & Ahmad, 2019; Van Bergen & Parsell, 2019).
Students experiencing effective PPIs were reported to have greater success with
teamwork and enhancement of skills such as problem-solving, communication,
learning autonomy, work related cognition and ethical thinking (Siddiqui et al.,
2019; Stigmar, 2016; University of the Free State, 2019).
However, current literature, as summarised above, provides a generalised

appreciation of the interrelation between PPI and enhanced student engagement.
It does not provide insights into how students’ age, gender, race, religion,
cultural orientation, sexual orientation, working and non-working and
socioeconomic country contexts (developing versus developed) (Andrew, 2019;
Sanger, 2020) influence the interrelation between PPI and student engagement.
Kahu and Nelson’s conceptual framework of student engagement supports the
notion that students’ experience of PPI is subjective to their cultural background
and the learning institute’s resources (Kahu & Nelson, 2018). Previous research
also reported students’ gender (Lam et al., 2016), working versus non-working
status (Creed, French, & Hood, 2015), cultural dimensions such as individualism
versus collectivism (Morera & Galván , 2019) and developed versus developing
country contexts (Ming Ming & Chow, 2011) are important considerations to
understand students’ learning process. Furthermore, learning is a product of
engagement between the person participating in education and the surrounding
social world (Broer, Bai, & Fonseca, 2019; Lave & Wenger, 1991). Hence, the
experience of an academic assessment of PPI is expected to be influenced by
students’ various characteristics and country contexts.
2.Method
2.1 Research context and Participants
The sample students for this study were selected from two different
postgraduate management units located in an Asian developing and Western
developed country (United Nations, 2014) respectively, Bangladesh and
Australia.
The developing country education setting is a private university, with subject
ranking for management, in the range of 401-450 by Quacquarelli Symonds(QS)
World university ranking (QS World University Rankings, 2020). The developed
country educational setting is a public university, with an overall QS ranking of
291(QS World University Rankings, 2020). Two of the authors are academics in
the chosen universities in the two countries. Hence the universities and classes
were chosen purposively, following the criteria of knowledge of the class’s
course curriculum, and access to designing of the PPI assessment. Fulfillment of
the selection criteria was essential to effectively develop the PPI assessment and
investigate the research phenomenon.
Sample students from the university in developing and developed countries

were studying in face to face and blended learning mode, respectively. In terms
of cultural orientation, the developing country is of the collectivism dimension,
with a score of 20 (Hofstede Insights, 2020). This score reflects students in this

121
country belonged to a society that upholds strong relationships and commitment

among group members. On the other hand, the developed country is high on the
individualism dimension, with a score of 90 (Hofstede Insights, 2020). So,
students in this country belonged to a self-reliant society where a reward is
based on individual merit. Upon comparing two cases in contrasting contexts,
the study applied the extreme-comparative method (Shelly, Ooi, & Brown, 2019)
to draw learning on PPI’s functioning with student engagement.
The sample cohort in the developing country comprised of sixty-five students,
representing 80% of the class. This cohort largely represented students aged
below 30 at 60% and females at 58%. The developed country sample consisted of
twenty-eight students, representing 38% of the class. Out of this cohort, aged 30
years and older and females were 61% and 75% respectively. Around 35% and
96% of students were working and studying simultaneously, in the developing
and developed countries, respectively. There were no international students in
any of the sample cohorts. Students from the developing country represented
the Indian ethnicity at 100% and the religion of Islam at 97%. Students from the
developed country largely represented the British ethnicity at 75% and the
religion of Christianity at 89%. The extracted sample from each class provided
sufficient information to answer the research questions regarding a student’s
experience of PPI. Hence the sample size was justified by information-
orientation, being adequate to investigate the targeted phenomenon in this
mixed methods education research (Ahrens & Zascerinska, 2014; Onwuegbuzie,
& Collins, 2007).
2.2 Research design and procedure
This research collected quantitative and qualitative perspective of students’
experience of a PPI assessment, applying a cross-sectional survey, within the
convergent parallel mixed methods research design (Creswell, 2014). In both the
study contexts, the survey took place after students experienced the PPI
assessment. The assessment was part of the study in the semester. The study
was first conducted at the university in the developed country, in 2017, and later
in 2018, in the university in the developing country.
The PPI assessment was designed with the concept of social constructivism
(Powell & Kalina, 2009; Vygotsky, 1962). Accordingly, the assessment
incorporated collaborative tasks, critical thinking, and content that was relevant
to students’ topics of learning. The PPI was a real-time learning activity
(online/audio discussion in real-time), where 3-5 students collaborated as a
team and made two submissions. One, a submission of 350-400 words answer to
a question that students wrote as a team. As explained in an earlier publication
of the researchers (reference is withheld to facilitate blind peer review), to
answer the question, the team members needed to be oriented with the unit
readings. In this submission, each team explained how specific learning from the
readings could be applied to manage real-life workplace scenarios. The PPI
assessment covered 5% of the total mark of the unit, with the same mark being
applied to the whole team. The lecturers/unit coordinators arranged the class
into teams of all working students or a combination of working and non-
working students. This arrangement was considered beneficial for students’
cognitive engagement towards the application of classroom learning.

122
The ethics protocol of the study in the developed country was approved by the
human research ethics committee at the Australian University (reference number
H0015793). In the absence of a human research ethics committee in the
university in Bangladesh, the study followed the guideline of the approved
protocol from the Australian University. The survey was voluntary and
confidential for all participants, which was important, as researchers were also
unit coordinator/lecturer in the investigated units. All enrolled students were
invited for the survey, through an announcement in the online learning platform
of the unit, for the university in Australia, and the university website, for the
university in Bangladesh. In these announcements, a link was provided for
students to participate in the survey via survey monkey. Further information on
the survey procedure in the developing country, is available in an earlier
publication (Siddiqui et al., 2019).
2.3 Survey Instrument
The survey instrument was purposely built for this study. A sample of the
survey questions is attached in Appendix 1. The survey included a section of
nominal scales to capture a student’s age, gender, religion, ethnicity, and status
of simultaneous work and study. Another section of Likert scales of 1 (Strongly
Disagree) to 5 (Strongly Agree) to collect a students’ experience of an association
between their experience of PPI and student engagement. Two single-item
variables in this section were: one, PPI enhances student engagement with
reading materials. Two, PPI enhances engagement with thinking of applying
classroom learning to work. There was also a section that collected students’
opinions in free-text comments (qualitative), regarding their answer on the
Likert questions and suggestions to improve the PPI assessment.
2.4 Data Analysis
The quantitative and qualitative data analyses were done simultaneously by two
separate researchers in this study. As shown in the result section, the research
questions were answered, placing the quantitative and qualitative results side by
side, with equal importance to each.
Quantitative data were analysed in three main ways. In all analyses, a
respondent was excluded if any answer was missing in any question. First, a
profile analysis was conducted, using percentage break-down, on the factors of
age, gender, religion, ethnicity, and the status of simultaneous work and study.
Second, analysis of variance (ANOVA) was completed, at a 95% confidence level,
to compare students’ experience of PPI in promoting student engagement
between the developing and developed countries. Variables subjected to
ANOVA were PPI enhances engagement with reading materials and PPI
enhances engagement with thinking of applying classroom learning to work
(Table 3). The homogeneity assumption was adhered to as the Levens test
reported statistically insignificant variance by median on both the variables.
Moreover, the data distribution was not of extreme nature as the absolute value
of the kurtosis index for both the variables were below 7 (Byrne, 2016). Lastly,
Pearson’s correlation, at 99% confidence level, was conducted within developing
and developing country, between the variables of age, gender, ethnicity, religion
and PPI enhances engagement with reading materials and PPI enhances
engagement with thinking of applying classroom learning to work. The

123
correlation analysis assessed the association between students’ characteristics

and their experience of PPI. The association between the variables was assessed
as low, moderate, and high if the value of coefficient correlation (r) were below
0.3, between 0.3 to below 0.5 and above 0.5 respectively (Cohen, 1988).
The qualitative data (free-text comments) in the survey were analysed, taking
guidance from the steps of thematic analysis, coined by Braun and Clarke(2006).
The first step was to become familiar with the data, through a reading of each
respondent’s comments and making notes of initial thoughts. In the second step,
all comments were collated in an excel file, in two worksheets, one for the
developing country and the other for the developed country data set. These
datasets were then reorganised into a set of interesting statements or quotations,
according to the research questions. This step ended with labelling a set of
quotations into codes such as “different viewpoints of readings” and “readings
made interesting”. A sample of the coding process is provided in Table 1.
Table 1: Coding process
Code Sample quotations from respondents
Readings “When we discuss with each other, it is interesting as lots of things are
made there which we can connect with the reading materials (Respondent #3
interesting from developing country).”
“Peers different point of views made the readings interesting (Respondent

#30 from developing country).”
“Through online collaboration we discussed so many issues related to our

reading materials. I never found the reading materials so interesting before
(Respondent #45 from developing country)”.
“Although I was only assigned a small component to report on, I read up

on my other team members topics so I could fully engage with them
during the discussion. I liked the readings too (Respondent #27 from
developed country)”.
“It was great working with other classmates. Not only does it draw
attention more closely to the reading materials, but it also was great to
draw on the readings to provide support and be supported by others in the
class. (Respondent #28 from developed country)”.
The third step involved identifying initial themes by merging different codes
that reflected similar notions. For example, the codes of “different viewpoint of
readings” and “readings made interesting” generated the theme of “Discussion
of readings from different viewpoints” (Table 2). In the fourth step, the initial
themes were reviewed further against all comments to decide the frequency of
each theme in the individual data set of the developing and developed countries.
In the last two steps, the themes were finalised and named, upon reviewing
these for commonalities and divergence between the developing and developed
country data sets. If the review detected themes that were different between the
two countries, then those themes were reported as “distinct themes”. The theme
of “challenge with peers in the developed country” (Table 4) is an example of a
distinct theme.

124
3. Results
In this section, the quantitative and qualitative results are presented according to
the research questions. The first two sections (3.1 and 3.2) have answered the
first research question, “is student experience of PPI to promote engagement
consistent across developing and developed country contexts?”. The third
section (3.3) has answered the second research question, “do characteristics of
students influence their experience of PPI?”.
3.1 Similarities in the experience of PPI
There were similarities in students’ experience of PPI in the two divergent
contexts, as reported in the quantitative part of the survey. A substantial share of
students, that is, 97% and 50% from developing and developed countries,
respectively, found PPI promotes student engagement.
Students’ opinions, provided in the qualitative section of the survey, generated

six common themes as presented in Table 2. In this table, the readers will also
find the frequency of the themes and sample quotations.
Table 2: Common themes in students’ experience of PPI in the divergent countries

Common themes Sample Quotations
(Frequency)
Theme 1: Discuss readings “Peers different point of views made the readings interesting
from different viewpoints (Respondent #30 from developing country).”
(89% in the developing and
“I'm a clinician, so my level of experience in management is
98% in developed country) limited. I was lucky enough that the other members of our
team had experience in this field and could discuss the
readings from their views. I felt really supported in this
format (Respondent #14 from developed country).”
Theme 2: Sharing reading “I believe group study or peer interaction helps us to quickly
work with peers grasp the readings, as we could share the readings and teach
each other what we read online (Respondent #40 from
developing country).”
89% in developed country)
“I find this subject incredibly difficult, so sharing the load
with team members and being able to split the content …and
teach each other what we have read about was great
(Respondent #6 from developed country)
Theme 3: Workplace “It helped me to increase skills for my work by learning how
relevant skill of Shared to manage a group to be efficient and complete the work
learning/teamwork timely. I can apply the same approach with which I
maintained coordination in this peer interaction
(Respondent #22 from developing country)”
73% in the developed
country) “My other team member has a very different background to
myself and works in a lesser clinical area, so it was good to
hear examples form their workplace also. (Respondent #3
from developed country.”
Theme 4: Workplace “There are many issues for which there are no solutions in
relevant skill of practical the book. Peer interaction helped to talk about these issues
problem- solving (88% in the and find practical solutions just like what I do at work.
developing and 62% in (Respondent # 65 from developing country) “.

125
developed country) “At work, I always needed to but was afraid of looking into
financial statements. But I found the concept of
'productivity' and 'efficiency’ in the financial statements
interesting, as explained by the team, though a shared
learning process. I plan to explore them for work issues in
the future (Respondent #8 from developed country).
Theme 5: Ease of online “Particularly, in last minute preparation, there is nothing to
communication with peers beat the advantages of online collaboration (Respondent #40
from developing country).”
28% in the developed “The online collaboration gave me the confidence to speak up
country) as I am always nervous to do that (Respondent #9 from
developed country).”
Theme 6: PPI had neutral “I believe the peer interaction was too short and had neutral
impact impact on my readings (Respondent #50 from developing
country).”
12% in developed country) “The timeline was a little tough. We needed to have more
time to learn from the team and apply the learning to work
(Respondent #2 from developed country).”
The two most frequent themes were the same across the students in developing
and developed countries. Which were: theme 1, PPI helped to discuss readings
from different viewpoints and theme 2, PPI facilitated sharing of reading work
with team members. Students shared their experience of how the PPI was a
supportive mechanism to understand the readings from different viewpoints of
team members and making the readings interesting (comments from respondent
#30 and #14 in Table 2). Moreover, across the two contexts, most students
agreed that PPI helped to share the load of readings with team members
(comments from respondent #40 and #6 in Table 2).
In both countries, as shown in themes 3 and 4 in Table 2, PPI came across as a
platform to attain workplace relevant skills of shared learning, teamwork, and
practical problem-solving. This happened, as during the PPI, students enjoyed
learning about the diverse work environment of peers and engaged in joint
coordination of the PPI assessment (comments from respondent # 22 and # 3 in
Table 2). Students also felt an association between PPI and practical problem-
solving, since the team discussion allowed unpacking of complex problems in a
language that they are familiar with (comments from respondent # 65 and #8 in
Table 2).
Additionally, as per the theme 5, few students, in both the contexts, resonated
with the notion that PPI can make it easy to communicate with peers through
online medium. These students found online interaction to be speedy (real-time)
and lesser intimidating (comments from respondent # 40 and # 9 in Table 2).
Lastly, as per the theme 6, few students commented that PPI was not a
significant influencer in their learning. This was the least frequent theme in both
the contexts and were experienced by students who felt a common hurdle with
the PPI assessment was the inadequacy of time (comments from respondents #
50 and #2 in Table 2).

126
3.2 Differences in the experience of PPI

There was a disparity in the level of endorsement for PPI between students in
developing and developed countries, as reported in the quantitative part of the
survey. As shown in Table 3, comparatively, developing country students
provided significantly stronger endorsement on the two variables of PPI
enhances engagement with reading materials (F-value=12.99*) and thinking of
application of classroom learning to work (F-value= 15.83*).
Table 3: Students’ experience of PPI between developing and developed country

Students in Students in Mean square F-value
Developing Developed between
Country Country groups
n (%) 62 (75) 21 (25)
Missing value 3 7
Variable: PPI enhances engagement with reading materials
Mean (SD) 4.4 (.68) 3.6 (1.2) 9.25 12.99*

Variable: PPI enhances engagement with thinking of applying classroom learning to
work
Mean (SD) 4.5 (.69) 3.7 (.85) 8.53 15.83*
* The mean square difference is significant at P <.05 level (2 tailed)
Table 4 presents themes that pointed to differences in the experience of PPI in

the two countries.
Table 4: Distinct themes in the experience of PPI in the divergent countries

Distinct Themes Sample Quotations
(Frequency)
Theme 1: Improving “Skype, Facebook services are often not working here. We faced this
PPI with better problem and completed the task with group messaging and audio
technology in the recording on phone (Respondent #46 from developing country).”
developing country
(8%) “We talked over WhatsApp/Viber. It was audio conversation, as the
internet could not support video conversation. I wanted the PPI to be
a video conversation, to make it more interesting with capturing of
the facial expressions (Respondent #52 from developing country). “
Theme 2: Challenge “My reading of the provided materials was not instigated by the team
with peers in the project, with most read before the completion of the project. The group
developed country project provided a platform to discuss certain readings, though I
(18%) found discussion on reading materials to be more engaging with
workplace managers than with fellow students. (Respondent #4 from
developed country).”
Members were difficult to engage early on. While I am very quick to

reply to emails and correspondence back and forth, it was more
difficult with other members who were not so engaged. (Respondent
#8 from developed country).

127
In the developing country, few students commented on PPI experience being

compromised due to technology issues. This notion generated Theme 1, which is,
improving PPI with better technology. This theme, as reported in Table 4,
indicated students had difficulty in accessing Facebook/Skype and resorted to
phone recording for the PPI (comment from respondent #46). A student also felt
the need for video technology to make the PPI experience more interesting
(comment from respondent #52).
In the analysis of qualitative comments, few students in the developed country
were found to be more critical of the benefits of PPI. Theme 2, challenges with
peers, which is reported in Table 4, was drawn from this notion. According to
this theme, students felt that discussion of readings with work colleagues was
more engaging than the same with peers (comment from respondent #4). Some
students also indicated that PPI can be ineffective if all team members do not
equally engage with PPI (comment from respondent #8).
3.3. Influence of student characteristics on students’ experience of PPI

Table 5 presents the quantitative survey results regarding the influence of
student characteristics in the experience of PPI.
Table 5: Influence of student characteristics in the experience of PPI
Variables Students in Students in

Developing Country Developed Country (n= 21;
(n= 62; missing value =3) missing value =7)
1 2 3 1 2 3
1. Age 1 .093 .07 1 -.644** -.649**
2. PPI enhances .093 1 .522** -.644** 1 .920**
engagement
with reading
materials
3. PPI enhances .07 .522** 1 -.649** .920** 1
engagement
with thinking
of applying
classroom
learning to
work
** Correlation is significant at P <.01 level (2 tailed)
The data revealed no influence of gender, ethnicity, and religion on students’

experience of PPI across the developing and developed countries. However, as
shown in Table 5, in the developed country, age had a significant and high
negative influence on students’ experience of PPI. As per which, more aged
students felt PPI reduces engagement with readings (r=-.644) and thinking of
applying classroom learning to work (r=-.649). Furthermore, a strong positive
correlation was noted between PPI induced engagement with readings and
thinking of applying classroom learning to work. This relationship between the
two variables applied to students in developing (r =.522), as well as, developed

128
(r=.920) country. This point is noteworthy, as a cumulative influence of the

negative experience of PPI is expected for the aged students in the developed
country.
Analysis of qualitative comments in the survey generated two themes regarding
how student characteristics influenced the experience of PPI in both countries.
Table 6 presents these themes for students of each country, along with the
frequency and sample quotations.
Table 6: Themes for the influence of student characteristics on the experience of PPI
Themes Sample Quotations
Theme 1: “Since I am working in corporate environment, it definitely helped me
Characteristic of to think how I should implement these theories in my workplace
working versus Non- (Respondent #33 from developing country)”.
working influenced
students’ experience “I was hesitant with group discussions, as I am not working and can’t
of PPI (26% in provide practical examples. (Respondent #53 from developing
developing and 18% context).”
in developed
country) “Difficult to engage team members due to ranging work
patterns. Would find it easier to review material on my own and
submit a discussion post (Respondent #19 from developed
country).”
“It is difficult to process discussion with peers with openness. It could

be good but that is not how we have worked all these years. At an
organisational level, there is not much of taking on different views of
how things could be done (Respondent #24 from developed
country).”
Theme 2: “My team members despite working full time and also doing masters
Characteristic of full time, managed to match time with others and collaboratively
Individualism versus completed the work. It was not an easy task, but we did it because we
collectivism were committed to team’s success in this assessment (Respondent #63
influenced students’ from developing country).”
experience of PPI
(40% in developing “I liked the PPI because it gave a better chance to know strengths and
and 57% in develop weaknesses of team members. This kind of online collaboration will
country) help to networking and allow different walks of people to work as a
team (Respondent #44 from developing country).
“Everyone comes to the Masters with their own reason and goals. You
are not necessarily working with people that are like minded and this is
always challenging when you are being assessed on group work
(Respondent # 20 from developed country). “
“Same grade for all when the contribution was not equal by all
members… My suggestion: submission of document with detailed
outline of individual group members work; which would include a sign
off from all group members. This would be helpful for markers to mark
the assignment fairly and equally (Respondent #13 from developed
country). “

129
According to theme 1 in Table 6, the characteristic of working versus non-

working had varied influences on the experience of PPI in the two countries. In
the developing country, working students had a positive experience with PPI.
This happened, as familiarity with the work environment helped them to
imagine how to apply the learning from PPI to work (comment from respondent
#33). Some non-working students were also hesitant with PPIs, for not being
able to contribute to the discussion due to lack of work experience (comment
from respondent #53).
In contrast, in the developed country, few working students struggled to
synchronise meeting time with other working peers (comment from respondent
#19). Moreover, they were hesitant to receive peer’s opinion with openness,
claiming that such openness is at odds with their work practices (comment from
respondent #24).
Theme 2 in Table 6, notes the cultural characteristic of Individualism versus
collectivism influenced students’ experience of PPI. Students in the developing
context exhibited a collectivism cultural dimension that positively influenced
their experience of PPI. They commented on successful experience with PPI, due
to all members’ commitment to the success of the team (comment from
respondent #63). Few other students perceived PPI to be a good mechanism to
know the capability of team members (comment from respondent #44).
Moreover, they imagined PPI to facilitate networking and teamwork with
people of different professions and contexts.
On the other hand, individualism characteristic was evidenced in students in the
developed country. In line with the individualism characteristic, they were
uncomfortable to rely on team members and the PPI assessment being assessed
on the basis of group work (comment from respondent #20). Moreover, some
students wanted to document all team member’s contribution, to enhance
fairness in marking of the PPI (comment from respondent # 13)
4. Discussions
This study investigated the experience of PPI between students in an Asian
developing country versus a Western developed country. Current literature on
PPI and student engagement are skewed towards the western setting (Stigmar,
2016) and equally lacking of a comparative view of contrasting contexts. This is a
study that compared students’ experience of PPI in promoting student
engagement, between developing versus developed country contexts, as well as,
student’s age, gender, ethnicity, religion, cultural orientation, and working
versus non-working status.
The first core finding in this study is a substantial portion of students, across the
developing and developed country, had a positive experience of PPI in
promoting student engagement. Common mechanisms behind the positive
experience of PPI were the sharing of tasks and open discussion with peers. It
was important that peers had the capability to offer different viewpoints or come
from a variety of study or work backgrounds such as clinical versus
management (Swain, 2013). This insight is aligned with the previous literature
that reported how students benefit from active and collaborative tasks,

130
pedagogy of social constructivism and social engagement with peers (Powell &
Kalina, 2009; Van Bergen & Parsell, 2019; Zhoc et al., 2018). Across the countries,
PPI facilitated greater involvement with and understanding of academic
readings. This is evidence of PPI’s linkage to the theme of students’ academic
engagement (Zhoc et al., 2018). Furthermore, the study evidenced PPI to help
students in attaining workplace relevant skills (for example, teamwork and
practical problem-solving) and enhancing cognition to apply classroom learning
to work scenarios. This insight is in line with earlier reported benefits of PPI
(Stigmar, 2016) and endorses PPI’s interaction with the theme of students’
cognitive engagement (Zhoc et al., 2018).
The above finding implies PPI can be used as a global theme to promote student
engagement across developing and developed countries. It is seen that PPI has
the universal capability to enhance engagement, regardless of students’ context
or individual characteristics. However, a required precondition is that academics
will need to design the PPIs within the pedagogy of social constructivism.
Moreover, PPIs will need to be designed in alignment with the other themes in
the student engagement framework, for example, academic and cognitive
engagement, to ensure greater benefit for students (Zhoc et al., 2018). The
above finding also contrasted the reporting of insignificant influence of PPI on
school students’ affective, behavioural and cognitive engagement (Lam et al.,
2016). The contrasting evidence supports PPI can differently function between
the school and higher education contexts. Future study of how the role of PPI in
student engagement is distinguished in higher education, against the school
studies, can assist in developing more context suitable PPI practices.
The second core finding in this study is student’s experience of PPI to promote
engagement is influenced by the technology in the context. It was seen that lack
of adequate technology in the developing country, negatively impacted students’
experience of PPI. Unlike the previous literature (Holzweiss, Joyner, Fuller,
Henderson, & Young, 2014; MacNeill, Telner, Sparaggis-Agaliotis, & Hanna,
2014), this study did not find technical difficulties compromised students’
experience of PPI in the developed country. This could reflect advancement in
technology-enhanced education in the developed country i.e. Australia (Horvath
et al., 2019). In comparison, the technology condition (for example, availability of
a university’s own online learning platform or access to social media such as
Facebook) is much inferior in higher education in the developing country
(Siddiqui et al., 2019). However, most students in the developing country
benefited from the tested PPI assessment despite the technical difficulties. This
infers academics in the developing country should not refrain from PPI
assessments on account of inferior technology. They should design PPI practices
carefully, either face to face or online, mitigating possible influence of inferior
technology to students’ cognitive, behavioural, and affective engagement
(Bodily, Leary, & West, 2019).
The other core finding is that the interrelation between students’ experience of
PPI and enhanced engagement is influenced by students’ cultural orientation,
working versus non-working status, and age. This finding is similar to certain
previous literature (Creed et al., 2015; Morera & Galván, 2019) that reported
these specific student characteristics to be important considerations in students’

131
learning. The study further clarified that the nature of the influence of a specific
student characteristic, varied between developing versus developed countries.
For instance, in the developing country, students’ age did not influence the
experience of PPI in promoting engagement. Whereas, for more aged students in
the developed country, PPI reduced their academic as well as cognitive
engagement. Hence, academics need to recognise students’ age as an important
characteristic of influence when designing PPI. Academics in the developed
country may need to give greater guidance throughout the PPI experience if the
aged students are involved.
Working students behaved differently in the two countries. This cohort of
students in the developing country showed collectivism, in line with the
country’s cultural dimension (Hofstede Insights, 2020), and reflected a stronger
commitment for the success of the PPI assessment. In contrast, working students
in the developed country portrayed individualistic cultural orientation and were
doubtful of the team’s commitment to the PPI assessment. Therefore, PPI’s role
to promote student engagement is shaped by the combined functioning of
various characteristics, for instance, work status and cultural orientation of
students. An overall implication of the study findings is academics should
design PPIs in partnership with students to ensure enhanced student
engagement.
The proposed partnership will help to accommodate PPI learning and teaching
approaches to diverse characteristics of the relevant student cohort and context.
For example, a partnership between academics and students can explore ways to
empower mature students in a developed country or international students from
Asian backgrounds towards inclusive and more effective PPI experience. PPIs
could be designed with a combination of face to face and online interactions; if
these students are finding it harder to open up to peers in an online environment.
Academics in the developed country may provide more consultation hours to
teams with working students, exchanging opinions on PPI design and associated
benefits with greater clarity. The rationale of partnership, as authors have
proposed here, resonates with recognising students as partners in learning and
teaching (Higgins, Dennis, Stoddard, Maier, & Howitt, 2019) and will facilitate
accommodating PPI exercises to the need of diverse student cohorts.
5. Conclusion
This study evidenced similarities and differences in students’ experience of PPI
in promoting engagement in an Asian developing and a Western developed
country. Notwithstanding, PPI was endorsed as a global theme that can promote
student engagement across the divergent contexts. It was noted that PPIs will
have the universal capacity to enhance student engagement if designed with
alignment to academic and cognitive engagement themes. Moreover, the
pedagogy of social constructivism should guide the development and practice of
PPI. Factors such as available technology in the country context and students’
age, cultural orientation and working status influenced the role of PPI in
enhancing student engagement. It was also seen that student characteristics of
age and cultural orientation (i.e., individualism versus collectivism) had
influenced the PPI experience differently between students in developing and

132
developed countries. Hence, the study confirms the characteristics of students

influence their experience of PPI. The implication is academics and students
should partner in PPI teaching and learning practices, facilitating the
accommodation of the PPIs to the various need of diverse student cohorts.
Amidst the growing diversity of students in higher education, such partnerships
will promote PPIs that are inclusive and effective for all students.
6. Limitations
There were a few limitations in this study. The sample profile of students in both
the developed and developing country was female dominant. While the female
dominance is reflective of the student population in higher education in the
developed country(Universities Australia, 2019), that is not the case for the
developing country(World Bank, 2019). Furthermore, the study has drawn
sample from only one university and discipline from each country and did not
mirror the respective population by characteristics such as age, work status,
ethnicity, and religion. Hence, the findings of the study are not generalisable to
the respective higher education population. The sample size of the study was
also small for a quantitative check of the causal relationship between diversity
factors and students’ experience of PPI. The study did not follow an
experimental design, which raised restriction to apply in-depth statistical
comparisons of the experience of PPI between the different cohorts of students.
Nevertheless, the study captured students’ empirical experience of PPI with
mixed methods research design, highlighting critical considerations for the
effectiveness of PPI with a diverse cohort of students. Future studies with a
greater number of participants are needed to pursue the agenda for effective
PPIs for all students.
7. References
Ahrens, A., & Zascerinska, J. (2014). A framework for selecting sample size in
educational research on E-Business application. 2014 11th International
Conference on e-Business (ICE-B), 39-46.
Andrew, P. (2019). The Quest for Diversity in Higher Education. Pepperdine Policy Review,
11(4). Retrieved from
https://digitalcommons.pepperdine.edu/ppr/vol11/iss1/4
Astin, A. (1984). Student Involvement: A Development Theory for Higher Education.
Journal of College Student Development, 40, 518-529.
Beaumont, T., Mannion, A., & Shen, B. (2012). From the Campus to the Cloud: The
Online Peer Assisted Learning Scheme. Journal of Peer Learning, 5(6), 20-31.
Bodily, R., Leary, H., & West, R. E. (2019). Research trends in instructional design and
technology journals. British Journal of Educational Technology, 50(1), 64-79.
doi:10.1111/bjet.12712
Bowden, J. L.-H., Tickle, L., & Naumann, K. (2019). The four pillars of tertiary student
engagement and success: a holistic measurement approach. Studies in Higher
Education, 1-18. doi:10.1080/03075079.2019.1672647
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative
Research in Psychology, 3, 77-101. doi:10.1191/1478088706qp063oa
Broer, M., Bai, Y., & Fonseca, F. (2019). A Review of the Literature on Socioeconomic
Status and Educational Achievement. In M. Broer, Y. Bai, & F. Fonseca (Eds.),

133
Socioeconomic Inequality and Educational Outcomes: Evidence from Twenty Years of

TIMSS (pp. 7-17). Cham: Springer International Publishing.
Byrne, B. M. (2016). Structural Equation Modeling with AMOS: Basic Concepts, Applications,
and Programming (3rd ed.). New York: Routledge.
Coates, H. (2010). Development of the Australasian survey of student engagement
(AUSSE). Higher Education, 60(1), 1-17. doi:10.1007/s10734-009-9281-2
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. New York, NY:
Routledge Academic
Creed, P. A., French, J., & Hood, M. (2015). Working while studying at university: The
relationship between work benefits and demands and engagement and well-
being. Journal of Vocational Behavior, 86(Supplement C), 48-57.
doi:https://doi.org/10.1016/j.jvb.2014.11.002
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches (4th ed. ed.). Thousand Oaks, CA: Sage.
Higgins, D., Dennis, A., Stoddard, A., Maier, A. G., & Howitt, S. (2019). 'Power to
empower': conceptions of teaching and learning in a pedagogical co-design
partnership. Higher Education Research and Development, 38(6).
Hofstede Insights. (2020). Country Comparison. Retrieved from https://www.hofstede-
insights.com /country-comparison/
Hoidn, S. (2016). Student-Centered Learning Environments in Higher Education Classrooms.
New York, United States: Palgrave Macmillan.
Holmes, N. (2018). Engaging with assessment: Increasing student engagement through
continuous assessment. Active Learning in Higher Education, 19(1), 23-34.
doi:10.1177/1469787417723230
Holzweiss, P. C., Joyner, S. A., Fuller, M. B., Henderson, S., & Young, R. (2014). Online
graduate students’ perceptions of best learning experiences. Distance Education,
35(3), 311-323. doi:10.1080/01587919.2015.955262
Horvath, D., Stirling, E., Bevacqua, J., Coldrey, M., Buultjens, P., Buultjens, M., & Larsen,
A. (2019). Plan, prepare and connect : How investing in understanding and
tracking the evolving needs of online students informs the development of
targeted programs for transition and success. Journal of University Teaching and
Learning Practice, 16(1).
Jawhar, S., & Subahi, A. (2020). The Impact of Specialty, Sex, Qualification, and
Experience on Teachers’ Assessment Literacy at Saudi Higher Education.
International Journal of Learning, Teaching and Educational Research, 19, 200-216.
doi:10.26803/ijlter.19.5.12
Kahu, E. R., & Nelson, K. (2018). Student engagement in the educational interface:
understanding the mechanisms of student success. Higher Education Research &
Development, 37(1), 58-71. doi:10.1080/07294360.2017.1344197
Lam, S.-f., Jimerson, S., Shin, H., Cefai, C., Veiga, F. H., Hatzichristou, C., . . . Zollneritsch,
J. (2016). Cultural universality and specificity of student engagement in school:
The results of an international study from 12 countries. British Journal of
Educational Psychology, 86(1), 137-153. doi:10.1111/bjep.12079
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation .
Cambridge:Cambridge University Press. doi:10.1017/CBO9780511815355
Lawrie, G., Marquis, E., Fuller, E., Newman, T., Qiu, M., Nomikoudis, M., . . . van Dam,
L. (2017). Moving Towards Inclusive Learning and Teaching: A Synthesis of
Recent Literature. Teaching & Learning Inquiry, 5(1), 1-13.
doi:10.20343/teachlearninqu.5.1.3
MacNeill, H., Telner, D., Sparaggis-Agaliotis, A., & Hanna, E. (2014). All for One and
One for All: Understanding Health Professionals’ Experience in Individual

134
Versus Collaborative Online Learning. Journal of Continuing Education in the

Health Professions, 34(2), 102-111. doi:10.1002/chp.21226
Ming Ming, C., & Chow, B. W. Y. (2011). Classroom Discipline Across Forty-One
Countries: School, Economic, and Cultural Differences. Journal of Cross-Cultural
Psychology, 42(3), 516-533. doi:10.1177/0022022110381115
Morera, I. & Galván, C. (2019). Hofstede’s Cultural Dimensions In The Educational
Context. VIII International conference on intercultural education and International
conference on transcultural health: The value of education and health for a
global,transcultural world. The European Proceedings of Social & Behavioural
Sciences Retrived from
https://www.europeanproceedings.com/files/data/article/85/5276/article_85
_5276_pdf_100.pdf
National Survey of Student Engagement. (2019). NSSE 2019 overview. Retrieved from
https://safesupportivelearning.ed.gov/resources/national-survey-student-
engagement-2019-overview
Neves, J. (2019). UK Engagement Survey 2019. Retrieved from https://www.advance-
he.ac.uk /knowledge-hub /uk-engagement-survey-2019
Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A typology of mixed methods sampling
designs in social science research. The Qualitative Report, 12(2), 281.
Perez, R. J., Robbins, C., Harris, L., & Montgomery, C. (2020). Exploring Graduate
Students' Socialization to Equity, Diversity, and Inclusion. Journal of Diversity in
Higher Education, 13, 133-145. Retrieved from
https://doi.org/10.1037/dhe0000115
Powell, K. C., & Kalina, C. J. (2009). Cognitive and Social Constructivism: Developing
Tools for an Effective Classroom. Education, 130(2), 241-250. Retrieved from
/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=
EJ871658&site=eds-livehttp://www.projectinnovation.biz/education_2006.html
Power, M., & Vaughan, N. (2010). Redesigning Online Learning for International
Graduate Seminar Delivery. The Journal of Distance Education / Revue de l'ducation
Distance, 24(2), 19-38.
QS World University Rankings. (2020). QS Top Universities. Retrieved from
https://www.topuniversities.com/universities/north-south-university#923555
Quality indicators for learning and teaching. (2020). 2019 Student Experience Survey.
Retrieved from https://www.qilt.edu.au/docs/default-source/ses/ses-
2019/2019-ses-national-report.pdf?sfvrsn=6486ec3c_10
Sanger, C. S. (2020). Diversity, Inclusion, and Context in Asian Higher Education. In C. S.
Sanger & N. W. Gleason (Eds.), Diversity and Inclusion in Global Higher Education:
Lessons from Across Asia (pp. 1-28). Singapore: Springer Singapore.
Shelly, B., Ooi, C.-S., & Brown, N. (2019). Playful learning? An extreme comparison of
the Children’s University in Malaysia and in Australia. Journal of Apllied Learning
and Teaching, 2(1), 16-23. Retrieved from
https://www.semanticscholar.org/paper/ Playful-learning-An-extreme-
comparison-of-the-in-in-Shelley Ooi/ec66ffefbe4779f504a749f84b210e430f4b0d4f
Siddiqui, N., Miah, K., & Ahmad, A. (2019). Peer to Peer Synchronous Interaction and
Student Engagement: A Perspective of Postgraduate Management Students in a
Developing Country. American Journal of Educational Research, 7(7), 491-498.
Retrieved from http://pubs.sciepub.com/
Stigmar, M. (2016). Peer-to-peer Teaching in Higher Education: A Critical Literature
Review. Mentoring & Tutoring: Partnership in Learning, 24, 1-13.
doi:10.1080/13611267.2016.1178963

135
Swain, G. (2013). Diversity Education Goals in Higher Education: A Policy Discourse Analysis.
(Doctor of Philosophy (PhD)). University of Maine, Electronic Theses and
Dissertations. Retrieved from
https://digitalcommons.library.umaine.edu/etd/1957
Tanaka, M. (2019). The international diversity of student engagement. In M. Tanaka (Ed.),
Student Engagement and Quality Assurance in Higher Education,1, (. London:
Rutledge.
United Nations. (2014). World economic situation prospects. Retrieved from
https://www.un.org
/en/development/desa/policy/wesp/wesp_current/2014wesp_country_classi
fication.pdf
Universities Australia. (2019). Higher education: facts and figures. Retrieved from
https://www.universitiesaustralia.edu.au /wp-
content/uploads/2019/08/190716-Facts-and-Figures-2019-Final-v2.pdf
University of the Free State. (2019). Creating pathwy for student success. Retrieved from
https://www.ufs.ac.za/sasse/sasse-home
Van Bergen, P., & Parsell, M. (2019). Comparing radical, social and psychological
constructivism in Australian higher education: a psycho-philosophical
perspective. The Australian Educational Researcher, 46(1), 41-58.
doi:10.1007/s13384-018-0285-8
Vygotsky, L. S. (1962). Thought and language. Cambridge: M.I.T. Press, Massachusetts
Institute of Technology.
World Bank. (2019). Bangladesh Tertiary Education Sector Review: Skills and Innovation for
Growth. Retrieved from Washington DC 20433
http://documents.worldbank.org/curated/en/303961553747212653/pdf/Bangl
adesh-Tertiary-Education-Sector-Review-Skills-and-Innovation-for-Growth.pdf
Zhoc, K., Webster, B., Li, J., & Chung, T. (2018). Higher Education Student Engagement
Scale (HESES): Development and Psychometric Evidence. Research in Higher
Education. doi:10.1007/s11162-018-9510-6

136
Appendix I. Survey Instrument (sample questions)
i) The assessment that I experienced with my peers (other students in the course),
as part of the study in this postgraduate course, enhanced my engagement with
the reading materials?
Please choose any ONE of the options below and clarify the chosen option
1. Strongly Disagree 2. Disagree 3. Neutral 4. Agree 5. Strongly Agree
Why: ………………………………………………………………………………………
……………………………………………………………………………………………
……………………………………………………………………………………………
ii) The assessment that I experienced with my peers (other students in the
course), as part of the study in this postgraduate course, made me think of ways
to apply learning from this course to workplace scenarios?
Please choose any ONE of the options below and clarify the chosen option
1. Strongly Disagree 2. Disagree 3. Neutral 4. Agree 5. Strongly Agree
Why: ………………………………………………………………………………………
……………………………………………………………………………………………
……………………………………………………………………………………………
iii) I would like to recommend the following changes so that the assessment
I experienced with my peers could be more useful to enhance my overall
learning experience in this course:
…………………………………………………………………………………………
……………………………………………………………………………………….
……………………………………………………………………………………………
iv) Please answer the following questions
• What is your gender:

• What is your age
• What is your religion:
• What is your ethnicity:
• Education completed (Please choose only one option): a) Bachelor from
Private University in Bangladesh b) Bachelor from a public university in
Bangladesh c) Others
• Are you studying (Please choose only one option): a) Part-time b) Full-
time c) Studying and working simultaneously d) Others………………….

137

Vol. 19, No. 8, pp. 137-154, August 2020
Teacher Education Institutions in the Philippines

towards Education 4.0
Rivika Alda, Helen Boholano and Filomena Dayagbil

Cebu Normal University, Cebu City, Cebu Philippines
https://orcid.org/0000-0002-1546-9042
https://orcid.org/0000-0003-4365-3370
https://orcid.org/0000-0002-3373-9931
Abstract. Education must reflect the world it is training students for.

Education 4.0 ensures that teaching-learning experiences will take
advantage of the limitless opportunities created by advanced technology.
This study determines the readiness of teacher education institutions in
the Philippines for Education 4.0 in the areas of faculty, teaching and
learning, infrastructure, and research as perceived by the administrators
and faculty members of teacher education institutions in the Philippines.
This study made use of survey questionnaire indicating the four
parameters mentioned, and data were analysed using quantitative
method. The findings of this study suggest that administrators and
faculty members perceived that they are ready in terms of their skills in
selecting and integrating digital resources for teaching and learning as
they are also given capacity buildings through seminars and conferences
related to technological literacy. However, they are neither skillful in
using the learning management system and other online class modality;
nor in the utilization of augmented reality, robotics, and digital enablers
like 3D printing. This finding may be attributed to the unavailability and
inaccessibility of digital infrastructure and virtual laboratories in most
teacher education institutions. The respondents also disagreed that the
teacher education institutions are ready in terms of research programs
and initiatives for Education 4.0. Thus, there is a need for teacher
education institutions to rethink infrastructure planning, redesign
research initiatives, and strengthen teacher-training capabilities to be
Education 4.0 ready.
Keywords: Industry 4.0; Education 4.0; Teacher education institutions;

Teacher readiness; Teaching and learning
1. Introduction
The fourth Industrial Revolution “blurs the lines among the physical, digital, and
biological worlds” (Shahroom & Hussin, 2018, p. 316). It transforms the world
through its impressive progress in creating technological breakthroughs and

138
limitless possibilities. These technological advancements have dominated the

lives and lifestyles of people.
Schwab (2016), purports that these great shifts brought about by the Fourth
Industrial Revolution (4th IR) in human history offer either considerable promise
or potential peril. The Global statistics in Whelsh (2018) reveal that in the fourth
industrial revolution, 65% of the students in grade school will later on work in
jobs that are yet to be invented. At least 49% of the existing posts can be replaced
by machines as most activities are already automated. Machines will also
outperform at least 80% of the skills people are trained in the last 50 years. While
others may see that the future of the internet provides balance and equal access to
almost everything, others warn that it may also be used for control and
exploitation. “In the new manufacturing regime enabled by 4IR technologies,
sometimes called the Internet of Things (IoT), nearly anything can be designed
on a computer and then printed on 3D printers that create objects in countless
materials or even biological tissues” (Penprase, 2018, p. 217).
Industry 4.0 has posed a significant challenge to the educational landscape in the
Philippines and worldwide. According to World Economic Forum Asian
Development Bank (2017, p.7) that 4IR “is building on the digital technologies of
the 3rd IR, and powered by a wide range of breakthroughs in the digital realm
(such as artificial intelligence), physical realm (new materials), as well as the
biological realm (bio-engineering)”. With the shift in the world of learning
brought about by the 4th IR, Education 4.0 - a model of education for the future
emerges, affording educators potentially far-reaching challenges (Fisk, 2017). It is
an era of creating innovative knowledge that becomes obsolete the next day, and
new knowledge emerges (Puncreobutr, 2016). Pangandaman et al. (2019) purport
that Industry 4.0 necessitates that the educational system supports a globalized
learning environment and experiences that are "automatized, networked,
virtualized, and flexible." Schools should prepare students for the future of work.
Students should be capable and ethical in the utilization of these new
technological tools. They should also be critical, creative, self-directed, reflective,
and decent citizens and leaders. This scenario calls for higher learning institutions
to keep abreast of these latest trends and come up with new and responsive ways
to enhance teaching and learning.
Data from Future of Jobs Reports, World Economic Forum (2016) in Haron (2018,
p. 12) highlighted the top essential skills for 2020. These are “complex problem
solving, critical thinking, creativity, management, coordinating with others,
emotional intelligence, judgment and decision-making, service orientation,
negotiation, and cognitive flexibility”. With this, schools should also be able to
connect education and employment, improve forecasts by establishing linkages to
industry and labor-market trends, and re-examine education and labor policy for
them to stay driven and relevant to the everchanging market realities (Lodder,
2016 in Haron, 2018). In the same context, Xing and Marwala (2017) had laid down
the framework for Teaching in IR 4.0, which will be the focus of Education 4.0.
They are the following: "teaching (wearables assisted teaching-learning-and-
training, embrace massive open online courses, cultivating innovative talent,

139
generalize blended learning); research (open innovation, evolutionary &

revolutionary innovations, new technological advancement, driven research and
development, shorten innovation cycles); and service (University-as-a-Platform
(UaaP), Education-as-a-Service (EaaS), Internationally-linked Programs)."
Moreover, Fisk (2017) also presented the following trends related to Education
4.0. First, learning is ubiquitous; thus e-learning tools and flipped classroom
approach provides greater opportunities for remote, and self-directed learning.
Second, there is a spiraling of contents tailored for each student. This allows for
mastery of the competencies and facilitates learning from simple to complex.
Third, students choose the delivery mode that they want; thus, flexible learning
approach may be adopted. Fourth, students are exposed to project-based tasks to
develop their collaborative and problem-solving skills which are beneficial in
their future careers. Fifth, students are given more opportunities for participatory
learning through field experiences such as practicums, mentoring, and
cooperative projects. Sixth, students are given opportunities to apply their
theoretical knowledge to practice and use their reasoning skills to make sound
interpretations and judgments. Seventh, teachers use authentic assessments in
evaluating learners' performance. Eighth, students will be involved in the
conceptualization and enhancement of the curriculum. Lastly, students are
encouraged to become responsible for their learning. These nine trends highlight
the shift of the limelight of learning responsibilities from the teacher to the
students.
Furthermore, Dunwill (2016) purports that the advancement of technologies

provides limitless possibilities in education. Thus, it will keep on altering the
teaching practices and the learning process in general. Other educational
institutions from different countries around the world have also started to
embrace these changes. In Malaysia, Hussin (2018) pointed out that tertiary
education in the country supports a flexible seating arrangement that is more
supportive of both individual and collaborative workspaces. Alternative
assessments are used to accommodate multiple learning styles, and Massive Open
Online Courses (MOOCs) and other online learning platforms are already
utilized. Similarly, data from U.S. Government (2017) showed that the country
had been leading foundational researches in artificial intelligence (AI) cascading
it down to research initiatives on strategic computing, big data, and brain
research. Likewise, Canada supports Learning in the Digital Age, highlighting
Digital and Media Literacy, which gives students the adaptive capabilities to be
an active part in the global digital society (Brown-Martin, 2017). Penprase (2018,
p. 211) cited that the “the proliferation of new educational institutions and new
curricula after the first two industrial revolutions enabled the technical and
managerial capacity to implement the massive expansion of the economy and
manufacturing that arose in the twentieth century”. The most familiar
exponential technology is the exponential increase in computer power and
decreasing cost in storage, which obeys a geometric relation commonly known
as Moore’s Law.

140
Besides, the following initiatives and movements of various countries have also
been recorded to address Education 4.0. The ASEAN Work Plan on Education
2016-2020 incorporated pertinent priorities, such as the utilization of ICT through
the expansion of ASEAN Cyber University (ACU). Meanwhile, Singapore has also
been very consistent with their remarkable results in the Program for
International Student Assessment (PISA). The country has put premium in its
education system evident in its integration into its overarching national strategy
(Brown-Martin, 2017). On the other hand, Myanmar has undertaken ways to
restructure their higher education institutions through these four phases-
“foundation building, interdisciplinary institutions, mainstreaming 21st-century
skills and multidisciplinary collaborations” (Mustafa, 2018, p. 31). In Malaysia,
Higher Education Framework 4.0 (MyHE 4.0) was established for the universities
to change their curriculum and delivery to ensure that universities will produce
well-rounded graduates who can easily adjust and fill in jobs that are yet to exist.
Education 1.0 was teaching through lectures and memorization, Education 2.0
introduced the usage of technology and the internet, Education 3.0 was on
knowledge production; while an innovative-production is needed for Education
4.0 (Siltharm, 2017). With the demands of Education 4.0, are higher education
institutions in the Philippines ready? The study of Pangandaman et al. (2019)
described the current state of Philippine higher education in addressing
Education 4.0. They have concluded that educational facilities, technological
advancements, and research are important features of a tertiary institution in
becoming an education 4.0 ready.
The study of Halili (2019) on teaching and learning in the fourth industrial
revolution underpinned the following four components to meet the challenges of
Education 4.0. First, redesigning of the learning spaces utilizing collaborative
tables and smartboard. Second, employing varied education processes such as
heutagogy, peeragogy, and cybergogy. Third, integrating an interdisciplinary and
a flexible curriculum wherein schools can employ innovations without being
constricted to traditional curriculum practices. Lastly, employing technological
advancements such as robotics, artificial intelligence, virtual and augmented
reality, cloud computing, big data, 3D printing, and so forth to aid teaching and
learning. Wallner and Wagner (2016) also purports the crucial drivers of
complexity in teaching and research undertakings in Education 4.0. These are
interculturalism and multiculturalism; mobile devices and media platforms; self-
paced and individualized study plans; a variety of available resources and
technologies; innovation and production of new knowledge; and, creation of
"enabling spaces" for students.
It is the intent of this study to determine the readiness of teacher education

institutions in the Philippines for Education 4.0 in the areas of faculty, teaching
and learning, infrastructure, and research as perceived by the administrators and
faculty members of teacher education institutions in the Philippines. The findings
of the study could be used as a basis for policy formulation or strategic actions to
enable teacher education institutions to meet the requirements of Education 4.0.

141
2. Methodology
This study employed descriptive quantitative research design utilizing survey
questionnaire as the main data gathering tool. The questionnaire (see Appendix
1) is composed of four parameters under study: faculty, teaching and learning,
infrastructure, and research. The Likert scale is used as this provides an effective
way for the researchers to draw out relevant responses from the participants. Each
parameter has eight indicators wherein participants will choose from a multiple-
choice option - strongly agree, agree, disagree, strongly disagree. Before the
conduct of the study, the researchers tried out the questionnaire to the faculty
members and administrators of a teacher education institution to establish
reliability.
The participants of this study are four hundred fifty (450) faculty members and
administrators from different teacher education institutions in the country. The
administrators, in the context of this study, are those teachers who also hold
administrative positions in their school like the vice-presidents, deans, and
department chairs.
Ethical considerations highlighted in this study include (a) informed consent, (b)
beneficence, (c) confidentiality and anonymity, and (d) respect for privacy, and
the right to withdraw from the study anytime. Before the participants answered
the survey questionnaire, the researchers gave the rationale of such research
undertaking. Informed consent was sought and the participants may or may not
answer the questionnaire given. There were also informal interviews conducted
to help the researches make sense of the responses in the questionnaire. The
participants will not be linked to the data. Their identify and affiliation will also
be kept confidential.
After the data were collected and summed up, results were tallied, collated, and
tabulated for analysis and interpretation.
3. Results and Discussion

The study determined the readiness of teacher education institutions in the
Philippines as perceived by administrators and faculty members in the areas of
faculty readiness, teaching and learning practices, infrastructure requirements,
and research programs and initiatives.
Table 1: Faculty Readiness

Weighted
As a faculty, I am ready for Education 4.0 because I … Description
Mean
1. am technologically literate 2.89 Agree
2. am skillful in the use of learning management
system, google classroom, and online class modality 2.4 Disagree
3. accept the vital role of technology in the 21st century Strongly
and beyond 3.51 Agree
4. integrate digital technology in teaching and
learning 2.6 Agree
5. attend seminars and conferences to enhance my Strongly
technological and pedagogical skills 3.51 Agree

142
6. experiment new approaches using technology 2.63 Agree

7. lead in the development of innovative materials 2.54 Agree
8. collaborate and build partnerships from other Agree
institutions 2.81
TOTALITY 2.86 Agree
Educators play a significant role in any decision making and policy reform as they
are the navigators and the direct implementers of these curricular changes. They
need to be at the forefront of these innovations with their commitment to ensuring
an all-out effort for quality education. With Education 4.0, are the faculty members
in teacher education institutions ready to be Teacher 4.0?
Table 1 shows the faculty members' perceived readiness for Education 4.0 as
mostly determined by their skills in selecting, developing, designing, and
integrating relevant digital resources for teaching and learning. In totality, with a
mean average of 2.86, the faculty members agreed that they are ready for
Education 4.0. They agreed in most statements that characterized Teacher 4.0 like
being technologically literate, integrating digital tools in their classroom practices,
experiment and lead in the development of innovative tools, and establish
partnerships from other institutions. The result implies that faculty members in
teacher education institutions do not only acknowledge the fact that they need to
take advantage of the digital tools available but also to take the lead in adopting
and conceptualizing new methods, a new concept of teaching, to keep abreast
with the fast-paced technological advancements. Moreover, teacher education
institutions should be consistent in their role in fostering innovation and
increasing higher-order skills among their students for improved quality of life. It
is for this reason that continuous training is afforded to faculty members. The
result shows that the participants strongly agreed that capacity buildings through
seminars and conferences related to technological literacy have helped them
enhance their digital skills. Most universities give importance to capacity-building
programs according to the individual development plan of their faculty members
and their strategic directions. A skilled teacher is a vital resource in any
educational institution. Given this, a teacher's professional training and
development is a necessary component that supports innovative and valuable
teaching.
However, despite the participants' claim that they are technologically literate,
most of them, although they are good at manipulating digital tools, have little
exposure to learning management systems and other online class modality.
Learning Management Systems or LMS provides both teachers and students
informational content and educational resources online. LMS is a way to ease the
work of educators by giving them a chance to take advantage of numerous e-
Learning opportunities and platforms which are flexible. Options include google
classroom, Moodle, Schoology, neo-LMS, etc. which offers teachers an easier and
faster way of tracking student records and in presenting course materials and
documents. Teachers can use these tools for online, blended, or flipped
classrooms. The result suggests that faculty members in teacher education have
not extensively used online or blended learning in their classes. Most universities
in the country still give premium to face to face interactions and not so much on

143
blended learning, online and, or distance learning. Most of them have not offered
MOOCs as well.
Massive open online courses (MOOCs) have been gaining popularity in today's
educational institutions, where physical, cultural, and socioeconomic boundaries
no longer stop students from continuing and furthering their studies. Xing and
Marwala (2017), in their framework for Teaching in IR 4.0, highlighted blended
learning and massive open online courses. In the Philippines, the University of
the Philippines Open University (UPOU) has started offering MOOCs since 2013.
In some universities, face-to-face courses are incorporated with some online
components, like in flipped classrooms, in which traditional in-class activities are
complemented, rather than replaced, with online undertakings. However,
MOOCs are getting more popular as this resolves student concern on accessibility
and time. These are self-instructional and self-paced courses enabling students to
have control over when and where to learn using the devices of their choice.
In view of the learning opportunities and pedagogical advantages of online

learning, school administrators need to give importance to teacher training on
developing online materials, navigating in various online platforms, and looking
for innovative ways of using such platforms to improve learning outcomes.
Table 2. Teaching and Learning Practices

Weighted
My classes are Education 4.0 because I… Description
Mean
1. use the following strategies in the delivery of
instruction:
a. individualized teaching 2.70 Agree
b. gamification and simulation 2.72 Agree
c. problem and inquiry-based teaching and learning 2.67 Agree
d. augmented and virtual reality 2.00 Disagree
2. boost students’ creativity through digital enablers like
1.93 Disagree
3D printing and robotics
3. use technology-based assessment tools (ex. Kahoot,
2.79 Agree
Quizlet, etc.)
4. develop 21st-century skills (ex. problem solving,
2.96 Agree
collaboration, reflection, critical thinking, communication)
5. teach digital citizenship (technology ethics, social,
ethical and legal responsibilities in the utilization of 2.77 Agree
technological tools and resources, etc.)
6. use individualized modular instructional materials 2.69 Agree
7. expose students to more participatory learning through
3.08 Agree
field experiences
8. accommodate multiple learning styles through flexible
3.08 Agree
assignments
TOTALITY 2.67 Agree
The respondents claim that they are technologically literate and are capable of
navigating through different digital tools as reflected in their teaching-learning
practices. Table 2 reveals that the faculty members in teacher education
institutions agreed that their teaching and learning practices are Education 4.0.

144
They claimed that they use various teaching strategies like individualized
teaching, gamification and simulation, and problem and inquiry-based teaching
and learning. Moreover, they also utilize personalized modular instructional
materials, expose students to hands-on learning through field experiences, and
provide flexible assignments accommodating students' different learning styles.
However, it was also revealed that they have not fully utilized the use of digital
enablers like 3D printing, robotics, and augmented and virtual reality. Teaching
and the whole learning process in the context of Education 4.0 not only challenges
universities to teach enduring understandings and relevant technologies but also
to employ new ways to transfer learning among students quickly and simply. The
whole community of learning needs to keep abreast of the current trends and
employ teaching models to respond effectively in this new learning environment
(Prieto, et al, 2015). With these rapid and spontaneous economic and social
changes, teacher education institutions should see that they are preparing their
graduates for a future that has never been this uncertain. What jobs will still be
available years from now? What skills will still be relevant to them? What
knowledge is considered the most essential considering the limitless possibilities
and limitless access to data and information? What problems will arise years from
now, and will they be ready to deal with these problems? With this so many
uncertainties thus, teachers should not only be able to put up with the needs of
the digital communities but also to see to it to teach students critical life skills
(creativity, critical thinking, problem-solving, adaptability, etc). Technological
change may have the effect of undermining rather than empowering workers and
their children (Berger & Engzell, 2020) especially in the teaching and learning
practices. They added that the interplay between automation on one hand and the
distribution of bargaining power and access to educational opportunities on the
other should be studied.
Moreover, as reflected in the results, educational institutions need to invest in new

technology like 3D printing technologies and augmented reality. These help
students develop the 4 C's of 21st-century learning: creativity, critical thinking,
communication, and collaboration. To add, augmented Reality (AR) and Virtual
Reality (VR) significantly “shifts the location and timing of education and
training” (Lee, 2012, p. 1). In virtual reality, one is brought to a virtual space or in
an artificial environment, while augmented reality (AR) allows students to view
and interact with computer-generated images in the real world. This brings
teaching and learning to a different level and shows a lot of promise. Moreover,
3D printers provide real-world application and knowledge and should also be
standard tools in higher education - and not just in STEM subjects. These
immersive tools promote students' problem-solving and creativity skills as they
become actively involved in the conception, design, and execution of their
projects.
However, the choice and use of any learning innovation depend on the
availability of these technologies and infrastructure. Thus, universities should
also make these tools and infrastructure available and accessible for both teachers
and students. Moreover, the availability of these tools is complemented by

145
teachers' skills in using them. Some inhibitors and reasons for teachers not using
these tools may include their lack of experience in ICT and the absence of support
and mentoring given to them. Teacher capability programs should consist of
training them with the skills in designing, creating, and applying these assistive
and immersive tools to address the new and emerging paradigms of learning.
Corollary to this, universities need to maximize the power of these immersive
technologies not only for effective teaching and learning but also for relevant and
responsive research and innovation.
Table 3. Infrastructure Requirements

The institution where I’m currently employed is ready for Weighted Description
Education 4.0 because it… Mean
1. has an office that takes charge of the ICT needs of the
2.93 Agree
university/college
2. has a stable internet connection which is accessible both
2.39 Disagree
for teachers and students
3. has available teaching and learning spaces that provide
2.49 Disagree
greater opportunity for collaboration
4. has digital infrastructure which provides open access to
1.85 Disagree
the repository of information and scientific data.
5. has technology that provides access to blogs, vlogs,
wikis, google classroom, Edmodo, and other related sites
2.47 Disagree
and reference databases (e.g., Google Scholar, Ebsco Host,
LISA, etc.)
6. has a computer laboratory with internet connection in
which students can watch educational videos and 2.51 Agree
collaborate with other students and other universities
7. has remote and virtual laboratories 2.17 Disagree
2.40 Disagree
TOTALITY
The availability and accessibility of high-quality and updated infrastructure

among teacher education institutions is a crucial foundation of conducive learning
environments. It facilitates instructional quality, thus improve student outcomes.
Educational institutions can respond to Industry 4.0 by harnessing the potential
of digital infrastructure, open educational resources (OER), networked education,
and lifelong learning.
Table 3 shows that the faculty members and administrators perceived that teacher
education institutions are not ready in terms of infrastructure requirements.
Although there is an office that takes care of the ICT needs of their universities,
and the availability of computer laboratories, they still have to establish reliable
internet connectivity and the accessibility of remote and digital infrastructure.
Moreover, there is also a concern about the availability of virtual laboratories and
state-of-the-art libraries and teaching-learning spaces that allow for flexibility to
be Education 4.0 ready.
Most universities have already embraced several changes in the delivery of

instruction, which includes the use of digital tools and digital applications. The
most common cry among educators is the slow internet connection or limited to

146
no access from those who live in the provinces. This situation can be considered
an educational setback and may significantly impact teachers' performance in
delivering instruction as well as student's academic success and career
opportunities. Also, virtual laboratories help students be exposed to and perform
on real live online trials. Obtaining practical skills through experiments provides
students a more profound sense of the concepts taught to them. Virtual laboratory
activities can save money, time, and effort but still safe and interactive, thus
allowing for more student involvement. Schools also need to envision the future
of academic libraries through the availability of state-of-the-art libraries. The
scenario of so many piled hard-bound books may no longer be the trend, but a
wide array of e-books that are all a click away.
Acatech (2014), as cited in Morrar et al. (2017, p. 14) purported that “Industry 4.0
as the Internet of Things (IoT), data and services that will change future
production, logistics, and work processes”. This means that everything is
connected to the internet be it device, processes, people, and even animals
through embedded systems, and wireless connection. Most of the universities and
colleges do not have a reliable internet connection and a stable learning
management system (LMS) for online learning and distance education. Faculty
members are willing to innovate, but they do not have the digital infrastructure,
which provides unlimited access to the repository of information and scientific
data. There are also limited resources for further commercialization of research
outputs and innovations.
Education 4.0 is highly related to innovation. According to Morrar et al. (2017, p.

15), "industry 4.0 represents a shift toward an innovation-based economy with
knowledge, data, and the IoT as central concepts". Most colleges and universities
in the Philippines do not have remote and virtual laboratories essential in the
fourth industrial revolution. Some universities have virtual laboratories for
electronics, manufacturing, and other areas but these are not enough for
innovations.
Likewise, new technologies are emerging faster, being adopted more quickly and
delivering greater impact than ever before. Fourth Industrial Revolution
technologies will create new ways for citizens to connect, to trade with each other,
and to access services that are currently not available (World Economic Forum
Asian Development Bank, 2017). Many students and teachers have hopeful
expectations for the future of the Fourth Industrial Revolution especially in the
delivery of instruction and the outcomes of education. If people focus only on
leveraging the information that they have and do not pay sufficient attention to
building basic infrastructure, then it will be a failure to make the Fourth
Industrial Revolution a reality (Yoon, 2017).
An institution of higher learning must have a reliable internet connection, which

is accessible both for teachers and students. "Though broadband users can get
high-quality video feedback, 56kbps-modem users cannot get good video quality.
Multicast video is another issue for Internet/Intranet video broadcasting.
Multicast is a one-to-many connection in which multiple clients receive the same

147
stream from a server”, Sheng, et.al. (2000, p. 53). Educators are excited about
robotics, IoT, digitalization, automatization, and teleconferencing. They are also
willing to help their institution attain its objectives in Education 4.0. Some of the
respondents said that they are willing to innovate for their learners if there is
considerable support for training and infrastructure budget to uplift their ICT
resources. This statement implies that educators are proactive in Industry 4.0.
Table 4. Research Programs and Initiatives

The institution where I’m currently employed is Weighted Description
Education 4.0 ready because… Mean
1. research undertakings are geared towards
2.11 Disagree
innovations
2. there is an allocated budget for research activities 2.55 Agree
3. research outputs are used for policy
2.66 Agree
recommendations
4. some research outputs generate new models for
2.01 Disagree
teaching and assessment
5. there is multidisciplinary collaboration in the
3.03 Agree
conduct of research
6. there is a local partnership in research projects 2.15 Disagree
7. international partnership is evident in the conduct of
1.96 Disagree
research activities
8. there is a program for research mentoring and
2.97 Agree
coaching
TOTALITY 2.43 Disagree
The country expects higher education institutions to lead in creating

breakthroughs in education through research. Research production,
dissemination, and utilization is even a critical predictor in university rankings.
Moreover, faculty research production is also an essential element for their
success as this is one basis for hiring or promotion and institutional reputation.
Giving premium to research and innovation is also a reliable indicator that teacher
education institutions respond to the needs of Industry 4.0 as they continuously
explore and enable new possibilities where humans and technology are aligned.
However, Table 4 reveals that the respondents disagreed on most of the

statements, which means that the teacher education institutions are not ready for
research programs and initiatives for Education 4.0. Although the respondents
agreed that there is research mentoring and coaching, and some research outputs
are used for policy recommendations, most of them disagreed that research
undertakings are geared towards innovations and research outputs that generate
new models for teaching and assessment.
Education resources at this time are more focused on pedagogical and content
knowledge and very limited to technological advancement and research
undertakings and innovations. There is a need for educators to collaborate with
both local and international researchers and take advantage of the limitless
possibilities of communication channels. Institutions should also make
international collaboration part of a university's culture. For some time, these
partnerships have contributed endlessly to academic and scientific progress. A

148
social network, according to Newman (2001), is a “collection of people, each of

whom is acquainted with some subset of the others”. These people from different
places or countries are glued together, for example as researchers, because they
have the same expertise or interest. The goal is to expand the network creating
more collaboration and communication. As the total number of connections and
research collaborators increases, the best practices will be shared and utilized.
These collaborations afford universities research opportunities and cultural
awareness. “The workplace of the future will not necessarily be a fixed
geographical location but may be geographically distributed and functionally
divided” (Johannessen, 2018, p. 27). Thus, it is now easier to collaborate with
researchers in the world. They will also pave the way for benchmarking best
practices in teaching, and curriculum development among institutions.
Johannessen (2018) mentions that the fourth industrial revolution will shape the
workplaces of the future, which will be entirely different from the workplaces
created by the first, second, and third industrial revolutions especially in
terms of research and innovations.
Hoeschl, Bueno and Hoeschl (2017) specified that ethics is very important in
conducting research. In their study, they identified and evaluated very important
issues and concerns that need to be considered to improve the connections
between technology and society. There are also many things to consider especially
in the maintenance of the equipment. Meckawey (2019) analyzed the maintenance
concept that will be applied in maintenance process due to the 4th industrial
revolution tools, which should also be implemented.
Bland et al. (2005) concluded that, "when individual faculty's research

productivity is the goal, nothing substitutes for these four factors: recruiting
faculty with a passion for research; providing them with formal mentoring
programs; facilitating their networks; and providing time for them to do
research”. To answer the call of Education 4.0, high levels of research productivity
as well as developing and maintaining an influential culture of research should
be evident. Thus, institutions desiring to create a culture of research must give
high regard in the acquisition of relevant resources for faculty development.
4. Conclusion
This study tried to find out if teacher education institutions in the Philippines are
ready for Education 4.0. The researchers determined the “readiness for Education
4.0” through the different identified areas based on an extensive review of related
literature and studies. Various studies pointed out these four areas should be
emphasized by teacher education institutions in responding to Education 4.0:
faculty, teaching and learning, infrastructure, and research. Since there is now a
significant shift in the future of learning, where the use of digital tools and digital
platforms is no longer an option but a way of life, it becomes a necessity for
educational institutions to prepare, embrace, and act on these changes. The
symbiotic relationship between education institutions and society, in general, has
transformed into a whole new level, and this calls for a dynamic transformation
of both. Moreover, this study confirms the importance of technological and

149
pedagogical innovations, which can be achieved only if the Industry 4.0 is

recognized simply in the form of technical, infrastructure, pedagogical, and
research innovation. Education 4.0 sheds light on the bright side of its potential
such as technological innovations, IT infrastructure, teaching and learning
(pedagogical), and research engagement.
With the transition to Industry 4.0, school administrators should emphasize the
urgency to redesign the education system. They should also work closely with
people from the industry sectors in reskilling and upscaling their graduates.
Further, policymakers should think of the global impact of Industry 4.0 on the
current education system and provide for appropriate human resource training
and development, support for infrastructure acquisition, and enough budget for
research and innovation.
Moreover, with the sudden shift of most universities in the Philippines (because
of the pandemic), from face-to-face and blended teaching, to online synchronous
and online asynchronous teaching, this provides opportunities for future
researches that would further elaborate the findings of this study. This study
could be extended and expounded in terms of how higher education institutions
respond now to the immediate need of upgrading their IT infrastructure and
furthering research initiatives. Further work is also suggested to examine the
practices of higher education institutions in responding to the call of Education
4.0 especially during this time of pandemic.
5. References
Berger, T., & Engzell, P. (2020). Intergenerational Mobility in the Fourth Industrial
Revolution. https://doi.org/10.31235/osf.io/zcax3
Bland, C. J., Center, B. A., Finstad, D. A., Risbey, K. R., & Staples, J. G. (2015). A theoretical,
practical, predictive model of faculty and department research productivity.
Academic Medicine, 80(3), 225‐237. https://doi.org/10.1097/00001888-200503000-
00006
Brown-Martin, G. (2017). Education and the fourth industrial revolution. Retrieved from
https://medium.com/regenerative-global/education-and-the-fourth-industrial-
revolution-cd6bcd7256a3
Dadios, E. P., Culaba, A. B., Albert, J. R. G., Paqueo, V. B., Orbeta, A. C., Serafica, R. B.,
Bandala, A. A., & Bairan, J. C. A. C. (2018). Preparing the Philippines for the Fourth
Industrial Revolution: A Scoping Study. Philippines Institute for Development
Studies. Discussion Paper Series No. 2018-11.
Dunwill, E. (2016). 4 changes that will shape the classroom of the future: Making education
fully technological. Retrieved from https://elearningindustry.com/4-changes-
will-shape-classroom-of-the-future-making-education-fully-technological
Fisk, P. (2017). Education 4.0 … the future of learning will be dramatically different, in
school and throughout life. Retrieved from
https://www.thegeniusworks.com/2017/01/future-education-young-
everyone-taught-together/
Halili, S. (2019). Technological advancements in education 4.0. The Online Journal of
Distance Education and e-Learning, 7(1), 63-69. Retrieved from
http://www.tojdel.net/journals/tojdel/articles/v07i01/v07i01-08.pdf
Haron, H. (2018). Education in the era of IR 4.0. Keynote Speech at the 2018 International
Conference on Information Management and Technology (ICIMTech 2018), 3 – 5

150
September 2018. Jakarta, Indonesia. Retrieved from

http://umpir.ump.edu.my/id/eprint/22486/1/Education%20in%20the%20Era
%20of%204.0.pdf
Hoeschl, M. B., Bueno, T. C., & Hoeschl, H. C. (2017, November). Fourth industrial
revolution and the future of engineering: could robots replace human jobs? How
ethical recommendations can help engineers rule on artificial intelligence. In 2017
7th World Engineering Education Forum (WEEF) (pp. 21-26). IEEE.
https://doi.org/10.1109/weef.2017.8466973
Hussin, A. A. (2018). Education 4.0 made simple: Ideas for teaching. International Journal
of Education and Literacy Studies, 6(3), 92-98.
https://doi.org/10.7575/aiac.ijels.v.6n.3p.92
Johannessen, J. A. (2018). Automation, innovation and economic crisis: Surviving the fourth
industrial revolution. Routledge. https://doi.org/10.4324/9781351039864
Lee, K. (2012). Augmented reality in education and training. TechTrends, 56(2),13-21.
https://doi.org/10.1007/s11528-012-0559-3
Meckawey, M. (2019, February). The Maintenance’s Strategy & the 4 th Industrial
Revolution. In 2019 International Conference on Fourth Industrial Revolution
(ICFIR) (pp. 1-8). IEEE. https://doi.org/10.1109/icfir.2019.8894795
Morrar, R., Arman, H., & Mousa, S. (2017). The fourth industrial revolution (Industry 4.0):
A social innovation perspective. Technology Innovation Management Review, 7(11),
12-20.
Mustafa, Z. (2018, May 16). Asean experience for IR 4.0. New Straits Times, pp. 1–6.
Retrieved from https://www.nst.com.my/education/2018/05/369959/asean-
experience-ir-40
Newman, M. E. (2001). The structure of scientific collaboration networks. Proceedings of the
national academy of sciences, 98(2), 404-409.
Overseas Research Report (2015). Current status on science and technology in ASEAN
countries. Center for Research and Development Strategy, Japan Science and
Technology Agency. Retrieved fromhttps://www.jst.go.jp/crds/pdf/en/CRDS-
FY2014-
OR- 02_EN.pdf
Pangandaman, H. K., Ali, N. D., Lambayong, J. H. C., & Ergas, M. L. G. (2019). Philippine
higher education vis-à-vis education 4.0: A scoping review. International Journal of
Advanced Research and Publications, 3(3), 65-69. Retrieved from
http://www.ijarp.org/published-research-papers/mar2019/Philippine-Higher-
Education-Vis-vis-Education-40-A-Scoping-Review.pdf
Penprase, B. E. (2018). The Fourth Industrial Revolution and Higher Education. In N.
Gleason (ed.), Higher Education in the Era of the Fourth Industrial Revolution. Palgrave
Macmillan Singapore. https://doi.org/10.1007/978-981-13-0194-0_9
Puncreobutr, V. (2016). Education 4.0: New challenge of learning. St. Theresa Journal of
Humanities and Social Sciences, 2(2). Retrieved from
http://www.stic.ac.th/ojs/index.php/sjhs/article/view/Position%20Paper3
Shahroom, A. A., & Hussin, N. (2018). Industrial revolution 4.0 and education.
International Journal of Academic Research in Business and Social Sciences, 8(9), 314–
319.
Sheng, W. U., Choo-Min, L. I. M., & Khiang-Wee, L. I. M. (2000). An integrated Internet-
based control laboratory. IFAC Proceedings Volumes, 33(31), 49-53.
Sikdar, M., & Balwaria,)R. (2013). Role of teachers amidst educational reform – passive
bystanders or active participants? International Journal of Scientific and Research
Publications, 3(10). Retrieved from http://www.ijsrp.org/research-paper-
1013/ijsrp-p2224.pdf

151
Schwab, K. (2016). The Fourth Industrial Revolution. World Economic Forum

Leapfrogging to education 4.0: Student at the core (2017). FICCI Higher Education Report.
Retrieved from
https://www.parthenon.ey.com/po/en/perspectives/leapfrogging-to-
education-4-0-student-at-the-core
TESDA (2016). Technological changes is coming: The fourth industrial revolution. Labor
market intelligence report.
U.S. Government. (2017). Preparing for the future of artificial intelligence. Ai & Society,
32(2), 285–287. https://doi.org/10.1007/s00146-016-0685-0
Wallner, T., & Wagner, G. (2016). Academic education 4.0. In the Proceedings of the END
International Conference on Education and New Developments, 12-14 June.
Ljubljana, Slovenia. Retrieved from
https://www.researchgate.net/publication/304115292_ACADEMIC_EDUCATI
ON_40
Welsh, B, (2018, April 18). Education 4.0 — How we will learn in the fourth industrial
revolution. Retrieved from https://medium.com/@brianna_91610/education-4-0-
how-we-will-learn-in-the-fourth-industrial-revolution-e17206b73016
World Economic Forum Asian Development Bank (ADB). (2017). ASEAN 4.0: what does
the Fourth Industrial Revolution mean for regional economic integration? World
Economic Forum, Geneva, Switzerland. http://dx.doi.org/10.22617/TCS179126-
2
Xing, B., & Marwala, T. (2017). Implications of the fourth industrial age on higher
education. Computers and Society, Cornell University.
http://arxiv.org/abs/1703.09643
Yoon, D. (2017). What we need to prepare for the fourth industrial revolution. Healthcare
informatics research, 23(2), 75-76. https://doi.org/10.4258/hir.2017.23.2.75

152
Appendix 1
Teacher Education Institutions in the Philippines Towards Education 4.0
The purpose of this survey is to determine the readiness of Teacher Education Institutions in the
Philippines with regard to Education 4.0. This will only take approximately 5-10 minutes to
complete. Your answers will be greatly appreciated. Thank you very much.
Name of Institution (optional) _____________________________________________

Category: _____SUC ___ LUC
____ Private ( Sectarian) ___ Private (Non-Sectarian)
Accreditation level: _______________________Accrediting body: ___________________
CHED award: _____COE ____ COD ____ Not Applicable
Position in the University/College:_____________________________________________
Part I. Directions: Please check the appropriate column that best applies to you and to
your Teacher Education Institution or College.
A. Faculty Readiness Strongly Agree Disagree Strongly
As a faculty in the institution where I’m Agree Disagree
employed, I…
1. am technologically literate
2. am skillful in the use of learning
management system, google
classroom, and online class modality
3. accept the important role of
technology in the 21st century and
beyond
4. integrate digital technology in
teaching and learning
5. attend seminars and conferences to
enhance my technological and
pedagogical skills
6. experiment new approaches using
technology
7. lead in the development of
innovative materials
8. collaborate and build partnerships
from other institutions
Others:
D. Teaching and Learning Practices

My classes are Education 4.0 because I …
1. use the following strategies in the delivery
of instruction:
a. individualized teaching
b. gamification and simulation
c. problem and inquiry-based
teaching and learning
d. augmented and virtual reality

153
2. boost students’ creativity through

digital enablers like 3D printing and
robotics
3. use technology-based assessment tools
(ex. Kahoot, Quizlet,etc)
4. develop the 21st century skills (ex.
problem solving, collaboration,
reflection, critical thinking,
communication)
5. teach digital citizenship (technology
ethics, social, ethical and legal
responsibilities in the use of technology
tools and resources, etc)
6. use individualized modular
instructional materials
7. expose students to more hands-on
learning through field experience such
as internships, mentoring projects and
collaborative projects
8. accommodate multiple learning styles
through flexible assignments.
Others:
E. Infrastructure Requirements
The institution where I’m currently
employed is ready for Education 4.0 because
it..
1. has an office that takes charge of the
ICT needs of the university/college
2. has strong internet connection which
is accessible both for teachers and
students
3. has available teaching spaces with
flexible layouts, equipped with
technology that enables students and
staff to connect to screens wirelessly for
collaboration
4. has digital infrastructure which
provides open access to scientific data
and knowledge, further
commercialization of research,
innovation, products and services.
5. has technology that provides access to
blogs, vlogs, wikis, google classroom,
Edmodo, and other related sites and
reference databases (e.g. Google Scholar,
EbscoHost, LISA, etc.)

154
6. has computer laboratory with internet

connection in which students can watch
educational videos and collaborate with
other students and other universities
7. has remote and virtual laboratories
Others:
F. Research Programs and Initiatives

The institution where I’m currently
employed is Education 4.0 ready because.. …
1. research undertakings are geared
towards innovations
2. there is an allocated budget for
research activities
3. research outputs are used for policy
recommendations
4. some research outputs generate new
models for teaching and assessment
5. there is multidisciplinary collaboration
in the conduct of research
6. there is local partnership in research
projects
7. international collaboration is evident
in the conduct of research activities
8. there is a program for research
mentoring and coaching
Others:

155

Vol. 19, No. 8, pp. 155-172, August 2020
Influence of Demotivators on Acceptance of

Technology: Challenges of Expatriate School
Teachers while Imparting Online Teaching
Gokuladas V. K. and Baby Sam S. K.

Indian Schools Oman, Muscat, Sultanate of Oman
http://orcid.org/0000-0002-5177-4351
http://orcid.org/0000-0002-7984-9167
Abstract. In the wake of the Covid-19 outbreak, academicians are

resorting to technology-enabled remote learning to impart education.
The main objective of this study is to identify those factors that could
potentially demotivate educators at primary and secondary levels of
education during the remote teaching process. This study will also look
at the impact of these demotivators on the perception of educators
regarding the technology-acceptance of E-learning and attitude towards
E-learning. Data collected from 1174 school educators with respect to
various challenges in E-learning and their acceptance of technology as
an alternative mode of teaching have been analyzed through correlation
and regression analysis. This study identified major Extrinsic and
Intrinsic Demotivators that affect the performance of school educators
while imparting education through remote teaching. The results
showed that Perceived Usefulness and Perceived Ease of Use played a
mediating role in the relationship between Extrinsic & Intrinsic
Demotivators and the Attitude towards E-learning. The outcome of this
study is of greater relevance to the School Management Committees and
the School Administration to appropriately strategize their plans to
implement E-learning as an alternative mode of education in schools.
Keywords: Acceptance of Technology; Demotivators; E-learning; School

Educators
1. Introduction
Technology has been of great assistance to many industries including education
wherein considerable importance is being provided to students’ centric
education encompassing the way they aspire to learn. It has been playing a
critical role in 21st-century education by assisting academicians to create, store,
and disseminate explicit knowledge in the most efficient and accessible way
(Oye, Salleh & Noorminshah, 2011). Since most of the educational institutions
are resorting to remote teaching through flip sessions owing to the Covid-19
outbreak, it is the time to address concerns made by researchers like Ehlers and

156
Hilera (2012) that more empirical pieces of evidence are essential to understand
the potential factors that could ensure the success of these remote sessions.
At this juncture, it is of greater research interest to identify those factors that

could positively influence the successful implementation of e-learning or remote
teaching process. In order to effectively roll out remote teaching programs,
teachers need to be convinced and acquainted with the technology and its
worthiness in imparting education in the best possible manner. In a study
conducted by Banas and York (2017) among pre-service teachers, it was found
that teachers’ motivation such as intrinsic and extrinsic goal orientation
developed through authentic learning process was one of the predictors of their
intention to integrate technology for their instructional methods. For successful
implementation of remote teaching, it is imperative that interactivity with
students is maintained at the highest level. There are a number of other
instructional methods such as the ‘Trialogical’ learning approach that have been
found highly effective in imparting active and student-centered learning even in
distant learning (Sansone, Cesareni, Ligorio, Bortolotti & Buglass, 2019).
Therefore, educators also need to be conversant with technologies such as blogs,
discussion forums, virtual meetings, etc., so as to ensure that they successfully
deliver the sessions through remote teaching (Hadad, 2007). These studies point
to the fact that remote teaching would be highly effective when it is properly
designed and utilized.
At the same time, it is a matter of great concern for educators whether they
would be successfully utilizing the E-learning technology to perform their
functions. One of the reasons for this apprehension could be the feeling among
most of the educators about their capability to utilize the technology most of
them were taking technological assistance in teaching for the first time in their
professional life. Ward, Peters & Shelley (2010) observed that while imparting
E-learning, faculty had concerns about the technical features of the Learning
Management System or other educational related technologies. Teaching online
could also influence image beliefs of faculty whereby they experienced the
feeling of inferiority while conducting online courses as compared to the
traditional classroom (Allen & Seaman, 2015). Wingo, Ivankova and Moss (2017)
revealed faculty were concerned that their perceptions such as the effectiveness
of online classes towards students’ success, the requirement of technical
supports, expected extra workload, managing online classes, etc., could be
detrimental to their job performance. These apprehensions would not only
affect their performance but also result in low motivation and job satisfaction
among educators. Therefore, identification of those factors that influence
educators in their pursuit of imparting online education would help school
administrators to adopt appropriate strategies for successful implementation of
remote teaching. The main objective of this study is to identify those factors that
could potentially demotivate educators at primary and secondary levels of
education during the remote teaching process. This study will also look at the
impact of these demotivators on the perception of educators regarding the
technology-acceptance of E-learning and attitude towards E-learning.

157
2. Review of related literature

2.1 Theoretical Framework
The Motivation-Hygiene Theory proposed by Herzberg depicts two factors i.e.,
the factors that result in motivation or job satisfaction and the factors that could
result in job dissatisfaction (Herzberg, Mausner & Snyderman, 2017). This
theory proposes that these factors motivate individuals to perform better and
attain job satisfaction would be different from those factors that demotivate
them leading to job dissatisfaction. On the other hand, there is Expectancy
Theory proposed by Vroom (1964) which postulates that people act consciously
and rationally in their job so as to ensure that they get the right rewards for the
same. In other words, individuals are generally driven not only by inherent
motivations but also by stimulus from external sources that help them to achieve
their professional objectives. In view of the above these theories, it could be
reasonably assumed that while the employees work towards the goal with full
efforts and commitment, there could be certain factors that could motivate and
demotivate them during the course. It is the ability of an individual to overcome
those demotivators through the help of those motivating factors that will
ultimately decide the success of the employees in their profession.
2.2 Demotivators
Similar to the concept of motivators that help individuals to perform better,
there could be demotivators that hinder such development. However, there are
fewer studies that researched extensively on demotivators as compared to that
of the studies on motivators. This is quite evident in the education sector as
fewer studies are carried out on demotivators related to remote teaching. These
demotivators could potentially affect the dissatisfaction of teachers with respect
to their profession leading to weak learning outcomes (Hettiarachchi, 2013).
Johnson (2000) observed that the flexibility of teaching conditions and the
corresponding curriculum that matches the teaching needs could prevent the
demotivation of educators. Low-quality teaching resources also contribute to
the demotivation of teachers which in turn resulted in low performance
(Menyhárt, 2008). Barın, Turgay and Sahin (2018) also identified that extrinsic
factors such as teaching facilities, school administration, and the curriculum
could adversely influence the motivation of educators. It was also identified that
demotivators could result in a lack of self-motivation. Han and Yin (2016)
identified that the relationships with and qualities of colleagues played an
important role in motivating or demotivating teachers at the workplace.
2.2.1 Intrinsic and Extrinsic demotivators

Another dimension of the concept of motivation is whether it is intrinsically or
extrinsically motivating or demotivating. Ryan and Deci (2000) argued that
motivation has its own role in the field of education because motivation in this
field is related to various actions and intentions. They also observed that while
intrinsic motivation reflected the natural human propensity to learn and
integrate, extrinsic motivation was roused as a result of external control with
considerable variation. Intrinsic motivation such as enjoyment, excitement, and
attachment towards a job would positively influence the effectiveness of the job.
Also, any factor would be considered as intrinsic when the external supports are
not required for the learner to continue the activity. Gange and Deci (2005)

158
identified that intrinsic reasons were more dominant than extrinsic reasons as
far as the participation of the individuals in activities was concerned. The
motivational factors for teaching fraternity could either help the educator to
perform exceptionally well. Even though the E-learning system currently being
followed by various educators to impart education during these trying times has
the advantage of connecting people from any part of the world (Kim, 2006), the
readiness of the educator was the crucial fact for the success of remote teaching
(Teddy So & Swatman, 2010).
Seaman (2009) observed that faculty performed well in online teaching when
they were trained, supported, and mentored appropriately. Sansone, Cesareni,
Ligorio, Bortolotti and Buglass (2019) identified that training programs
conducted through specific roles by the participants resulted in creating positive
perceptions of collaborative learning as well as the use of technology in
classrooms. Faculties were also found positive about using technology to impart
education when they saw it as an avenue for professional development
(McQuiggan, 2012; Panda & Mishra, 2007). On the other hand, it was observed
in the study conducted by Barton and Haydn (2006) that just providing adequate
information might not be ideal to develop the competencies of teachers in using
Information and Communication Technology facilities as observed by the
trainee teachers. Erkaya and Rocha (2013) suggested that teachers remain
unaffected by the negative external factors when they are intrinsically
motivated. Some of the motivators for online teaching faculty are flexible work
schedules (Shea, 2007) and the flexibility of online instruction (Green, Alejandro
& Brown, 2009).
On the other hand, factors such as include inadequate technical support,

increased workload, not having enough time to prepare, concern about course
quality, feeling of not-being technically competent, and concern over the quality
of students’ work could demotivate educators in online teaching (Cook, Ley,
Crawford & Warner, 2009). Glanz (2014) identified excessive time commitment
in online teaching was one of the top demotivators for teaching fraternity. In the
online mode of teaching, educators were apprehensive about the possibility of
continuous communication with students in order to track the activities of the
students (Glanz, 2014). However, intrinsic factors appeared to be more
influential than extrinsic factors on educators. Increased workload has been
identified as one of the top demotivators for educators in remote teaching (Shea,
2019; Lloyd, Byrne & McCoy 2012; Alsofyani, Aris, Eynon & Majid, 2012). The
extra-time that educators used for the preparation of their online session had
adversely affected some of them (Chapman, 2011).
It was suggested by Ertmer and Ottenbreit-Leftwich (2012) that when teachers

change their approach from technology integration to technology-enabled
learning through implementing powerful instructional methods, they would be
in a better position to engage the students in a more meaningful and relevant
learning environment. At this juncture, the concept of learning motivation
proposed by Gardner and Lambert (1972) appeared to be relevant even though it
was related to language acquisition. According to them, there are two types of
learning motivation i.e., instrumental motivation and integrative motivation.

159
While instrumental motivation denoted learning the language as an instrument

to achieve career goals, integrative motivation reflected the passion to learn a
language to interact with that culture effectively. Similarly, the need of the hour
is the integrative approach on the part of teachers to assimilate the technological
advancement to the teaching process rather than the instrumental approach just
to equip them to deal with the challenges of remote teaching.
Bolliger and Wasilik (2009) observed that the satisfaction of online teaching
faculty was influenced by institutional issues such as higher workload and
increased time commitment. Educators feel reluctant to online teaching mainly
because of the fear of change, workload issues, quality of learning outcomes, and
concerns about technological assistance (Bacow, Bowen, Guthrie, Long & Lack,
2012; Betts & Heaston, 2014). Training educators in E-learning facilities did
enhance their satisfaction (Lackey, 2011). Lee (2001) observed that educators
required strong institutional support in various forms to run the E-learning
effectively since technical issues were found to be adversely affecting the
satisfaction of teachers (Bolliger & Wasilik, 2009). Allen and Seaman (2015)
observed that concern about the quality of teaching through online could also
adversely influence educators.
2.3 Perceived Ease of Use and Perceived Usefulness

Technology Acceptance Model (TAM) proposed by Venkatesh and Davis (2000)
stipulated that Perceived Ease of Use and Perceived Usefulness are predictors of
the acceptance behavior of people towards technology. TAM has been proved to
be successful in identifying the acceptance of technology by knowledge workers
including educators (Lee, Kozar & Larsen, 2003). King and He (2006) identified
through a meta-analysis that TAM was a powerful predictive model that could
be applied to various contexts. BurtonJones and Hubona (2005) asserted that the
end-users of technology would develop cognitive responses based on their
perceptions of usefulness and ease of use of technology. These responses would
in turn result in the development of a corresponding attitude towards
acceptance of technology and further behavioral intentions. It was also observed
that PU is a significant predictor of attitude towards the acceptance of
technology and further behavioral intention (Wong et al., 2013).
2.4 Attitude towards Technology

Schechter (2000) asserted that the comfort and proficiency of teachers with the
technology would positively influence the use of technology and further usage
in the classroom. The attitude of teachers does influence the infusion of
technology into the classroom environment (Demetriadis et al., 2003). There are
few variables that contributed to the attitude of teachers towards technology
such as experience in teaching, gender, and subjects handled (Jimoyiannis &
Komis, 2007). ChanLin (2005) identified that teachers who possessed creativity
were more likely to absorb the technology for imparting education and were
also influenced by factors such as learning environment and curriculum-related
issues. It has been identified that the attitude of teachers towards technology
acceptance would also be influenced by their familiarity with the usage of
technology (Teo, Lee &Chai, 2008).

160
3. Conceptual Model and Hypotheses
The research conceptual model proposed in this study is as given in Figure 1.
Figure 1 – Research Model
We defined demotivators as those factors that could potentially influence the

teachers negatively during the process of handling sessions through remote
learning. We also categorized these demotivators into two categories i.e.,
extrinsic demotivators and intrinsic demotivators. We classified factors such as
‘technical interruptions during E-learning sessions’, ‘low bandwidth of internet
connectivity’, ‘long-preparatory works before each session’, ‘low-control over
students during E-learning sessions’, and ‘lack of technical know-how’ as the
main extrinsic demotivators. Factors such as ‘the feeling of low-connectivity
with students’, ‘perceived lack of physical presence’, ‘feeling of not being able to
provide hands-on experience’, ‘perceived difficulty in carrying out assessments
online’, and ‘perceived difficulty in getting feedback’ are categorized as intrinsic
demotivators. While Perceived Usefulness denoted as the usefulness of the E-
learning process for imparting teaching, Perceived Ease-of-use reflected the
easiness of handling E-learning sessions. Past research has identified that
perceived ease-of-use and perceived usefulness had significant positive
predictability in developing attitudes towards usage of technology which is
categorized as the attitude towards E-learning.
Ghanghesh (2013) pointed out the intrinsic factors could also include a good
relationship with people and job responsibility. It was also identified that
intrinsic motivation exercised more influence than extrinsic motivation on the
job performance of teachers. Such a positive influence of intrinsic motivation
could in turn result in a better workplace attitude (Thatcher, Liu & Stepina,
2002). Boekaerts (2001) observed that an individual’s behavior is influenced by
two types of motivation i.e., intrinsic and extrinsic motivation. Teo, Wong and
Chai (2009) observed that behavioral intention towards computers by teachers
was significantly influenced by the Perceived Usefulness (PU) and Perceived
Ease of Use (PEOU). In view of the above, the following hypotheses have been
proposed in this study.

161
H1: Extrinsic and intrinsic demotivators will have significant negative

relationships with the perceived usefulness of the E-learning process among
school educators.

relationships with perceived ease-of-use of the E-learning process among school
educators.

relationships with the attitude towards the E-learning process among school
educators.
Davis (1989) proposed the Technology Acceptance Model which postulates that
the attitude of an individual towards technology is influenced by the Perceived
Ease-Of-Use (PEOU) and Perceived Usefulness (PU) of the technology. Ma,
Andersson and Streith (2005) explained that the intentions to use technology by
educators were significantly influenced by their perceived usefulness of the
technology. Therefore, we propose the following hypotheses also in this study.
H4: Perceived ease-of-use of the E-learning process will have a significant

positive relationship with the attitude towards the E-learning process among
school educators.
H5: Perceived usefulness of the E-learning process will have a significant

positive relationship with the attitude towards the E-learning process among
school educators.
4. Methodology
4.1 Sample
As one of the consequences of the Covid-19 outbreak, academicians across the
globe have been prompted to switch over to the online mode of teaching for a
prolonged period, probably for the first time in their professional life. Even
though educators in the field of higher education might have had the experience
in remote teaching occasionally, most of the school educators must have been
experiencing this phenomenon for the first time. Therefore, we approached 2400
school educators of Indian origin who are remotely teaching expatriate Indian
students in the Sultanate of Oman with a self-administered questionnaire. We
resorted to a simple random sampling method as this was the only approach of
data collection possible during this pandemic period. We received a total of
1174 correct responses after having eliminated duplicates through IP address.
4.2 Demographic descriptive

Out of 1174 respondents, 946 are females and 228 are males which form 80% and
20% respectively. In this study, we categorized the respondents into four
categories on the basis of their age i.e., within the age bracket of ‘20 to 30’, ‘31 to
40’, ‘41 to 50’, and ‘above 50 years’. While 6% of the respondents belonged to the
age bracket of 20 to 30, 459 of them belonged to the age bracket of 31 to 40. The
biggest number of respondents belonged to the category of 41 to 50 (40.9%)
whereas the category of above 50 years of age consisted of only 14% of the
respondents. 599 respondents (51%) are imparting education in rurally located

162
schools whereas 575 (49%) of them are belonging to schools located in urban
areas of the country.
As far as the teaching level of the respondents is concerned, KG level

represented 10%, Primary level with 32%, Middle level with 26%, Secondary
level with 14%, and Senior secondary with 18% respectively. This representation
shows that schools are adequately fulfilling the remote learning requirements of
all sections of students. With respect to the educational qualifications of the
respondents, 68% of them possess post-graduate degrees and 28% of them are
graduates. The details of the demographics of the respondents are summarized
in Table 1.
Table 1. Demographic profile

Measure Items Frequency Percent
Gender Female 946 80.6
Male 228 19.4
Age_Group 20 to 30 70 6.0
31 to 40 459 39.1
41 to 50 480 40.9
Above 50 165 14.1
Location Rural 599 51.0
Urban 575 49.0
Teaching level KG 115 9.8
Primary 374 31.9
Middle 307 26.1
Secondary 165 14.1
Senior Secondary 213 18.1
Educational Trained Teacher 35 3.0

Qualifications Graduate 333 28.4
Post Graduate 798 68.0
Doctorate 8 .7
4.3 Measurement Tools

The items (Davis, 1989, Table 6, p. 329) for measuring Perceived Usefulness (PU)
and Perceived Ease-Of-Use (PEOU) (revised 10 items each from the earlier
original 14 items scale) were adopted as part of the questionnaire. These items
were reported to have construct validity through multitrait-multimethod and
factor analysis (Davis, 1989). In order to measure the Attitude towards E-
learning, another 5 items on a seven-point scale developed by Ajzen and
Fishbein (1980) that measured five different types of attitudes were adopted in
this study. Based on the review of the literature, we also categorized the number
of extrinsic and intrinsic demotivators and scored them as per the number of
such demotivators experienced by the educators (0 for ‘no difficulties’, 1 for one
difficulty experienced, and so on till 5 as we have categorized maximum of five

163
such demotivators as extrinsic and intrinsic factors. All these items have been
displayed in Appendix A.
5. Results
5.1 Descriptive Statistics
Descriptive analysis of variables under this study has revealed that the
respondents have scored above average level in respect of Perceived Usefulness
and Perceived Ease of Use (3.1 and 3.26 respectively). In the case of Attitude
towards E-learning also the mean score is well above the average marking. On
the other hand, the respondents experienced fewer difficulties as far as both
extrinsic and intrinsic motivators are concerned. However, it has been observed
that the educators experienced more of intrinsic challenges than that of the
extrinsic factors. In other words, these are indicators that it is more of
psychological difficulties that bothered the educators than technical difficulties
in running the E-learning process. The details of descriptive statistics are as
given in Table 2.
Table 2. Descriptive Statistics (Means and S.D)

n = 1174 Mean S.D
Extrinsic_Demo 1.59 .962
Intrinsic_Demo 1.84 1.131
PU 3.10 .826
PEUS 3.26 .616
ATTITUDE 5.29 1.125
5.2 Reliability
Cronbach’s Alpha Reliability analysis has been carried out in respect of all 25
items (10 items each for PU & PEOU, and 5 items for Attitude towards E-
learning) that confirmed the coefficient of .91 for these items. According to
Cortina (1993), this range of co-efficient is an excellent level of acceptance.
5.3 Analysis
As far as the extrinsic demotivators are concerned, 53% of the respondents felt
that Technical Interruption during the session was the primary challenge for
them. This is followed by Low bandwidth issues (40%), long technical
preparation time (33%), less control over students during remote teaching (30%),
and operational discomforts of being not-so-tech-savvy (12%) as major extrinsic
challenges faced by the teachers in online-education. The details are as given in
Figure 2.

164
700
573
600
500 430
400 361
331
300
200 133
100
0
Technical Low Bandwidth Technical Low control over Discomfort being
Interruptions Preparation time students not a tech-savvy
Figure 2 - Major extrinsic challenges of teachers in online teaching
At the same time, there are also some intrinsic challenges being faced by these
teachers in online education. These are explained in Figure 3. A whopping 75%
of the teachers felt that low-connectivity with the students through remote
teaching was their major challenge in remote teaching. This is followed by
challenges in teaching specific subjects that require more of the physical
presence of the teacher (40%), lack of hands-on teaching (35%), difficulty in
assessments (30%), and the lack of feedback from students (16%). This result
showed that lack of sufficient feedback from students through remote teaching
aggravated the concerns of educators which otherwise would have been
minimal in a traditional classroom.
900 819
800
700
600
500 435
379
400 322
300
169
200
100
0
Less connectivity Subject necessity Lack of hands-on Not suitable for Lack of feedback
with students for physical learning assessements
presence
Figure 3 - Major intrinsic challenges of teachers in online teaching
Correlation analysis was performed on the research variables and the output of
the same is as provided in Table 3. All the research variables have been found to
be significantly related to each other. A statistically significant a moderate-to-
strong positive relationship was observed among the three research variables of
technology acceptance i.e., perceived ease of use of E-learning, perceived
usefulness of E-learning in imparting education, and attitude towards E-learning
(r = .60, p < .0001). This indicates that with the increase in the perceived ease of

165
use and perceived usefulness of E-learning, it is very likely that the positive
attitude towards E-learning will also be developed. Yet another dimension in
this correlation analysis is the significant moderate positive relationship between
intrinsic demotivators and extrinsic demotivators (r = .59, p < .0001). This shows
that any incremental changes in the number of intrinsic demotivators are likely
to result in a corresponding increase in extrinsic demotivators. A probable cause
for this positive relationship could be the result of the frustration of educators
having encountered challenges of varying degrees while negotiating them. As
far as the relationship between demotivators and the research variables are
concerned, it has been observed that the demotivators exhibited a significant
negative relationship with research variables (r ranging from -.24 to -.35, p <
.0001). However, it is also important to note that extrinsic demotivators
exhibited a significant negative relationship with perceived ease of use than
intrinsic demotivators whereas intrinsic demotivators showed a significant
negative relationship with perceived usefulness than extrinsic demotivators.
This shows that the lesser the extrinsic demotivators, the higher will be
perceived ease of use. Similarly, the lesser the number of intrinsic demotivators,
the higher will be the perceived usefulness of E-learning. Demotivators were
also observed to be having a weak negative significant relationship with the
attitude towards E-learning.
Table 3. Correlation Analysis

Extrinsic_Demo Intrinsic_Demo PU PEUS
Intrinsic_Demo .588**
PU -.311** -.334**
PEUS -.350** -.277** .604**
Attitude -.268** -.239** .605** .601**
**. Correlation is significant at the 0.01 level (2-tailed).
5.4 Discussion
In order to get further insight into these relationships, regression analyses were
performed to test the hypotheses proposed in this study. First, we conducted a
multiple regression model with Attitude towards E-learning as a dependent
variable and the rest of the research variables as independent variables. The
results of the regression indicated the predictors explained 45.2% of the variance
(R2 =.45, F(4, 1174)= 242.998, p<.0000). It was found that both Perceived
Usefulness and Perceived Ease of Use significantly predicted Attitude towards
E-learning (β= .51 and β= .67 respectively, p<.000). This is placed in Table 4.
Thus, hypotheses H4 & H5 which postulated that Perceived Usefulness and
Perceived Ease of Use respectively will have a positive relationship with
Attitude towards E-learning have been supported in this study.
Subsequently, regression models that had Perceived Usefulness and Perceived
Ease of Use as dependent variables and Demotivators as independent variables
have also been developed and placed in Table 5 & 6 respectively. With regard to
the model with Perceived Usefulness as the dependent variable, the regression
model reported F(2, 1174)= 88.623, p<.0000) and model with Perceived Ease of

166
Use as the dependent variable, the regression model reported F(2, 1174)= 87.900,
p<.0000). It was found that both Extrinsic Demotivators (β= -.15, p<.000) and
Intrinsic Demotivators (β= -.17, p<.000) significantly predicted Perceived
Usefulness. Similar findings were also observed with respect to Perceived Ease
of Use wherein both Extrinsic Demotivators (β= -.18, p<.000) and Intrinsic
Demotivators (β= -.06, p<.000) significantly predicted Perceived Ease of Use.
These findings not only supported hypotheses H1 & H2 which proposed that
Extrinsic and Intrinsic demotivators will have a significant negative relationship
with the Perceived Usefulness and Perceived Ease of Use of E-learning but also
revealed the mediating role of Perceived Usefulness and Perceived Ease of Use
in controlling the effect of Demotivators on the dependent variable Attitude
towards E-learning of school educators. However, H3 which postulated that
Extrinsic and intrinsic demotivators will have a negative relationship with the
attitude towards the E-learning process among school educators has not been
supported in this study.
Table 4. Regression Analysis – Attitude towards E-learning as a dependent variable

Unstandardized Standardized
Model B Std. Error Beta T Sig.
1 (Constant) 1.558 .167 9.310 .000
PU .514 .038 .377 13.576 .000
PEUS .668 .051 .365 13.156 .000
Ex_Demo -.028 .032 -.024 -.871 .384
In_Demo .002 .027 .002 .067 .946
a. Dependent Variable: ATTITUDE
Table 5. Regression Analysis – Perceived Usefulness of E-learning as a dependent

variable
Model B Std. Error Beta T Sig.
1 (Constant) 3.650 .047 77.412 .000
Ex_Demo -.151 .029 -.176 -5.222 .000
In_Demo -.168 .025 -.230 -6.837 .000
a. Dependent Variable: PU
Table 6. Regression Analysis – Perceived Ease of Use of E-learning as a dependent

variable
Model B Std. Error Beta t Sig.
1 (Constant) 3.666 .035 104.301 .000
Ex_Demo -.183 .022 -.286 -8.498 .000
In_Demo -.059 .018 -.109 -3.237 .001
a. Dependent Variable: PEUS
In view of the above findings, the predictive model of hypothesized

relationships among variables is as given in Figure 4.

167
Figure 4 – Relationship among variables
6. Conclusion
A model to understand the influence of Extrinsic and Intrinsic demotivators on
the Technology Acceptance variables has been presented and validated in this
study. There are several implications of this empirical output which are
explained in succeeding paragraphs. First, demotivators play an influential role
in the morale of educators. While challenges such as technical interruptions and
low bandwidth could be tackled through organizational support, various
training programs could result in better acceptance of technology as identified
by Sansone et al., (2019). Acquainting with the technology through such training
programs could mitigate such feelings of not getting connected with students
more realistically. In this study, we identified that the lesser the number of
demotivators, be it extrinsic or intrinsic, the greater will be the perceived
usefulness and the perceived ease of use of technology by educators for
conducting online teaching. While the majority of the extrinsic demotivators
could be tacked through administrative actions by the school management,
adequate counseling and training programs could help the teachers to overcome
the intrinsic challenges of imparting education through remote teaching.
Second, though we identified that demotivators had a negative correlation with
all other research variables, these demotivators did not have any direct influence
on the Attitude towards the E-learning of educators. However, mediation by
Perceived Usefulness and Perceived Ease of Use on the relationship between
demotivators and Attitude towards E-learning emphasized the importance of
TAM in the educational sector when there is a paradigm shift in the teaching
process. Therefore, blended learning, integration of face-to-face teaching &
online approaches (Graham, 2006), and adopting innovative learning
approaches such as the ‘Trialogical’ learning approach (Sansone et al., 2019)
would be effective and flexible processes through which transformative changes
could happen in the field of education (Hancock & Wong, 2012). The findings of
this study would pave the way for educational institutions to chalk out different
strategies to minimize the demotivators so as to ensure that academicians take

168
the education to the next level of technology-assisted teaching and learning.

Various Extrinsic and Intrinsic demotivators identified in this study would be of
greater relevance to the educational institutions which venture into E-learning as
an alternative mode of interacting with students.
7. Limitation and Future Scope of this Study

This study focused only on teachers of primary and secondary education. A
comparative study involving faculty in higher education institutions will
provide further insights into various differing challenges being faced by these
educators while implementing remote teaching. Another limitation of this study
is that the participants of this study were the expatriate teachers of Indian origin.
A detailed study involving native teachers is likely to provide various challenges
of E-learning in the context of culture.
8. References
Ajzen, I., & Fishbein, M. (1980) Understanding attitudes and predicting social behavior,
Englewood Cliffs, NJ: Prentice-Hall. Retrieved from
https://ci.nii.ac.jp/naid/10011527857/
Allen, I. E., & Seaman, J. (2015). Grade Level: Tracking Online Education in the United States.
Babson Survey Research Group. Babson College, 231 Forest Street, Babson Park,
MA 02457. Retrieved from https://files.eric.ed.gov/fulltext/ED572778.pdf
Alsofyani, M. M., Aris, B. B., Eynon, R., & Majid, N. A. (2012). A preliminary evaluation
of short blended online training workshop for TPACK development using the
technology acceptance model. Turkish Online Journal of Educational Technology-
TOJET, 11(3), 20-32.
Bacow, L., Bowen, W., Guthrie, K., Lack, K., & Long, M. (2015). Barriers to Adoption of
Online Learning Systems in U.S. Higher Education. doi:10.18665/sr.22432
Banas, J. R., & York, C. S. (2017). Pre-service teachers’ Motivation to use technology and the
impact of Authentic learning exercises. In L. Tomei (Ed.), Exploring the New Era of
technology-infused Education (pp.121–140). Hershey, PA: IGI Global.
doi:10.4018/978-1-5225-1709-2.ch008
Barin, M., Turgay, H. A. N., & Şahin, S. A. R. I. (2018). Factors affecting teacher
motivation and demotivational-reasoning in a second service area school in
Turkey: A single-case research. Dil ve Dilbilimi Çalışmaları Dergisi, 14(4), 226-242.
Retrieved from https://dergipark.org.tr/en/download/article-file/650778
Barton, R., & Haydn, T. (2006). Trainee teachers’ views on what helps them to use
information and communication technology effectively in their subject teaching.
Journal of Computer Assisted Learning, 22, 257–272. doi:10.1111/j.1365-
2729.2006.00175.x
Boekaerts, M. (2001). Motivation, Learning, and Instruction. International Encyclopedia of
the Social & Behavioral Sciences, 10112–10117. doi:10.1016/b0-08-043076-7/02432-
3.
Bolliger, D. U., & Wasilik, O. (2009). Factors influencing faculty satisfaction with online
teaching and learning in higher education. Distance Education, 30(1), 103–116.
doi:10.1080/01587910902845949.
Burton-Jones, A., & Hubona, G. S. (2005). Individual differences and usage behavior.
ACM SIGMIS Database, 36(2), 58–77. doi:10.1145/1066149.1066155
ChanLin, L. J. (2005). Development of a Questionnaire for Determining the Factors in
Technology Integration among Teachers. Journal of Instructional Psychology, 32(4),
287-292. Retrieved from https://www.learntechlib.org/p/98822/

169
Chapman, T. K. (2011). A critical race theory analysis of past and present institutional
processes and policies in teacher education. In A. F. Ball & C. A. Tyson (Eds.),
Studying diversity in teacher education, 237-256. Rowman & Littlefield Publishers
Inc.
Cook, R. G., Ley, K., Crawford, C., & Warner, A. (2009). Motivators and Inhibitors for
University Faculty in Distance and e-learning. British Journal of Educational
Technology, 40(1), 149–163. doi:10.1111/j.1467-8535.2008.00845.x
Cortina, J. M. (1993). Interaction, Nonlinearity, and Multicollinearity: implications for
Multiple Regression. Journal of Management, 19(4), 915–922.
doi:10.1177/014920639301900411
Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of
Information Technology. MIS Quarterly, 13(3), 319. doi:10.2307/249008
Demetriadis, S., Barbas, A., Molohides, A., Palaigeorgiou, G., Psillos, D., Vlahavas, I., &
Pombortsis, A. (2003). “Cultures in negotiation”: teachers’ acceptance/resistance
attitudes considering the infusion of technology into schools. Computers &
Education, 41(1), 19–37. doi:10.1016/s0360-1315(03)00012-5
Ehlers, U.-D., & Hilera, J. R. (2011). Special Issue on quality in e-learning. Journal of
Computer Assisted Learning, 28(1), 1–3. doi:10.1111/j.1365-2729.2011.00448.x
Erkaya, O. R. (2012). Factors that motivate Turkish EFL teachers. International Journal of
Research Studies in Language Learning, 2(1). doi:10.5861/ijrsll.2012.135
Ertmer, P. A., & Ottenbreit-Leftwich, A. (2012). Removing obstacles to the pedagogical
changes required by Jonassen’s vision of authentic technology-enabled learning,
Computer & Education, 1-8. doi:10.1016/j.compedu.2012.10.008.
Gagné, M., & Deci, E. L. (2005). Self-determination theory and work motivation. Journal
of Organizational Behavior, 26(4), 331–362. doi:10.1002/job.322
Gardner, R. C., & Lambert, W. E. (1972). Attitudes and Motivation in Second-Language
Learning. Rowley, Mass.: Newbury House.
Ghenghesh, P. (2013). Job Satisfaction and Motivation - What Makes Teachers Tick?
British Journal of Education, Society & Behavioural Science, 3(4), 456–466.
doi:10.9734/bjesbs/2013/5156
Glanz, J. (2014). Action research: An educational leader's guide to school improvement.
Rowman & Littlefield.
Graham, C. R. (2006). Blended learning systems: Definition, current trends, and future
directions. In Handbook of blended learning: Global Perspectives, local designs. San
Francisco, CA: Pfeiffer. Retrieved from
http://www.click4it.org/images/a/a8/Graham.pdf
Green, T., Alejandro, J., & Brown, A. H. (2009). The Retention of Experienced Faculty in
Online Distance Education Programs: Understanding Factors that Impact their
Involvement. The International Review of Research in Open and Distributed Learning,
10(3). doi:10.19173/irrodl.v10i3.683
Hadad, W. (2007). ICT-in-education toolkit reference handbook. InfoDev. Retrieved from
http://www.infodev.org/en/Publication.301.html (accessed on 29 Dec 2019)
Hancock, S., & Wong, T. (2012). Blended Learning. Retrieved from
http://sites.wiki.ubc.ca/etec510/Blended_Learning#cite_note-3 (accessed on 10
Sep 2019)
Han, J., & Yin, H. (2016). Teacher motivation: Definition, research development, and
implications for teachers. Cogent Education, 3(1).
doi:10.1080/2331186x.2016.1217819
Herzberg, F., Mausner, B., & Snyderman, B. B. (2017). Perspective. The Motivation to
Work, 120–139. doi:10.4324/9781315124827-15

170
Hettiarachchi, S. (2013). English Language Teacher Motivation in Sri Lankan Public

Schools. Journal of Language Teaching and Research, 4(1). doi:10.4304/jltr.4.1.1-11
Jimoyiannis, A., & Komis, V. (2007). Examining teachers’ beliefs about ICT in education:
implications of a teacher preparation program. Teacher Development, 11(2), 149–
173. doi:10.1080/13664530701414779
Johnson, D. T. (2000). Teaching mathematics to gifted students in a mixed-ability classroom.
Reston, VA: ERIC Clearinghouse on Disabilities and Gifted Education. Retrieved
from http://jukebox.esc13.net/estar34/estar34_d1_gifted_article.pdf
Kim, S. (2006). Public service motivation and organizational citizenship behavior in
Korea. International Journal of Manpower, 27(8), 722–740.
doi:10.1108/01437720610713521
King, W. R., & He, J. (2006). A meta-analysis of the technology acceptance model.
Information & Management, 43(6), 740–755. doi:10.1016/j.im.2006.05.003
Lackey, K. (2011). Faculty development: An analysis of current and effective training
strategies for preparing faculty to teach online. Online Journal of Distance Learning
Administration, 14(4), 8- 17. Retrieved from https://ols.syr.edu/wp-
content/uploads/2012/04/Faculty-Development-An-Analysis-of-Current-and-
Effective-Training-Strategies-for-Preparing-Faculty-to-Teach-Online.pdf
Lee, J. (2001). Instructional support for distance education and faculty motivation,
commitment, satisfaction. British Journal of Educational Technology, 32(2), 153–160.
doi:10.1111/1467-8535.00186
Lee, Y., Kozar, K. A., & Larsen, K. R. T. (2003). The Technology Acceptance Model: Past,
Present, and Future. Communications of the Association for Information Systems, 12.
doi:10.17705/1cais.01250
Lloyd, S. A., Byrne, M. M., & McCoy, T. S. (2012). Faculty-perceived barriers to online
education. Journal of online learning and teaching, 8(1). Available at
https://jolt.merlot.org/vol8no1/abstracts.htm
Ma, W. W., Andersson, R., & Streith, K.-O. (2005). Examining user acceptance of
computer technology: an empirical study of student teachers. Journal of Computer
Assisted Learning, 21(6), 387–395. doi:10.1111/j.1365-2729.2005.00145.x
Menyhárt A. (2008). Teachers or lecturers? The motivational profile of university
teachers of English. Working Papers in Language Pedagogy, 2, 119‐137. Retrieved
from http://langped.elte.hu/WoPaLParticles/W2Menyhart.pdf
McQuiggan, C. A. (2012). Faculty Development for Online Teaching as a Catalyst for
Change. Online Learning, 16(2). doi:10.24059/olj.v16i2.258
Oye.N.D, Salleh, M., & Noorminshah. (2011). Knowledge Sharing in the Workplace:
Motivators and Demotivators. International Journal of Managing Information
Technology, 3(4), 71–84. doi:10.5121/ijmit.2011.3406
Panda, S., & Mishra, S. (2007). E‐Learning in a Mega Open University: Faculty attitude,
barriers, and motivators. Educational Media International, 44(4), 323–338.
doi:10.1080/09523980701680854
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of
intrinsic motivation, social development, and well-being. American Psychologist,
55(1), 68–78. doi:10.1037/0003-066x.55.1.68
Sansone N., Cesareni, D., Bortolotti, I., & Buglass, S. (2019). Teaching technology-
mediated collaborative learning for trainee teachers. Technology, Pedagogy and
Education, 28(3), 381-394. doi:10.1080/1475939X.2019.1623070.
Sansone, N., Cesareni, D., Ligorio, M. B., Bortolotti, I., & Buglass, S.L., (2019). Developing
knowledge work skills in a university course, Research Papers in Education, 35(1),
23-42.
doi:10.1080/02671522.2019.1677754

171
Schechter, E. L. (2000). Factors relating to classroom implementation of computer technology in

elementary schools (pp. 1-128). St. John's University (New York), School of
Education and Human Services. Retrieved from
https://www.learntechlib.org/p/123270/
Seaman, J. (2009). Online Learning as a Strategic Asset. Volume II: The Paradox of
Faculty Voices--Views and Experiences with Online Learning. Results of a
National Faculty Survey, Part of the Online Education Benchmarking Study
Conducted by the APLU-Sloan National Commission on Online
Learning. Association of Public and Land-Grant Universities. Retrieved from
https://files.eric.ed.gov/fulltext/ED517311.pdf
Shea, P. (2019). Bridges and Barriers to Teaching Online College Courses: A Study of
Experienced Online Faculty in Thirty- Six Colleges. Online Learning, 11(2).
doi:10.24059/olj.v11i2.1728
Teddy So, K. K., & Swatman, P. (2010). The Diminishing Influence of Age and Gender on
the e-Learning Readiness of Teachers in Hong Kong. Lecture Notes in Computer
Science, 477–488. doi:10.1007/978-3-642-14657-2_43
Teo, T., Lee, C. B., & Chai, C. S. (2008). Understanding pre-service teachers’ computer
attitudes: Applying and extending the Technology Acceptance Model (TAM).
Journal of Computer Assisted Learning, 24(2), 128-143. Doi:10.1111/j.1365-
2729.2007.00247.x
Teo, T., Lee, C. B., Chai, C. S., & Wong, S. L. (2009). Assessing the intention to use
technology among pre-service teachers in Singapore and Malaysia: A
multigroup invariance analysis of the Technology Acceptance Model (TAM).
Computers & Education, 53(3), 1000–1009. doi:10.1016/j.compedu.2009.05.017
Thatcher, J. B., Liu, Y., & Stepina, L. P. (2002). The role of the work itself. Proceedings of the
2002 ACM SIGCPR Conference on Computer Personnel Research - SIGCPR ’02.
doi:10.1145/512360.512365
Venkatesh, V., & Davis, F. D. (2000). A Theoretical Extension of the Technology
Acceptance Model: Four Longitudinal Field Studies. Management Science, 46(2),
186–204. doi:10.1287/mnsc.46.2.186.11926
Vroom, V. H. (1964). Work and motivation. New York: Wiley. Retrieved from
https://psycnet.apa.org/record/1964-35027-000
Ward, M. E., Peters, G., & Shelley, K. (2010). Student and faculty perceptions of the
quality of online learning experiences. The International Review of Research in Open
and Distributed Learning, 11(3), 57. doi:10.19173/irrodl.v11i3.867
Wingo, N. P., Ivankova, N. V., & Moss, J. A. (2017). Faculty Perceptions about Teaching
Online: Exploring the Literature Using the Technology Acceptance Model as an
Organizing Framework. Online Learning, 21(1). doi:10.24059/olj.v21i1.761
Wong, K. T., Osman, R., Goh, P. S. C., & Rahmat, M. K., (2013). Understanding Student
Teachers' Behavioural Intention to Use Technology: Technology Acceptance
Model (TAM) Validation and Testing. Online Submission, 6(1), 89-104. Retrieved
from https://files.eric.ed.gov/fulltext/ED539841.pdf

172
Appendix A – List of constructs used in the study

Revised 10 item scale for perceived usefulness (Davis, 1989, Table 6, p. 329)
Item
No. Candidate item for psychometric measures for perceived usefulness
1 Using e-learning platform improves the quality of my teaching
2 Using an e-learning platform gives me greater control over my work
3 The E-learning platform enables me to accomplish tasks more quickly
4 The E-learning platform supports critical aspects of my work
5 E-learning platform increases my productivity
6 E-learning platform improves my job performance
The E-learning platform allows me to accomplish more work than would
7
otherwise possible
8 The E-learning platform enhances my effectiveness on the work
9 E-learning platform makes it easier to do my work
10 Overall, I find the E-learning system useful in my work
Revised 10 item scale for perceived ease-of-use (Davis, 1989, Table 6, p. 329)
Item
No. Candidate item for psychometric measures for perceived usefulness
1 I find it burdensome to use the E-learning platform
2 Learning to operate an E-learning system is easy for me
3 Interacting with the E-learning system is often frustrating
4 I find it easy to get the E-learning system to do what I want it to do.
5 The E-learning system is rigid and inflexible to interact with.
It is easy for me to remember how to perform my tasks using an E-learning
6
system
7 Interacting with the E-learning system requires a lot of my mental efforts.
8 My interaction with the E-learning system is clear and understandable
9 I find it takes a lot of efforts to become skillful at using an E-learning system
10 Overall, I find the E-learning system easy to use.
5 item scale developed for measuring Attitude towards technology (Ajzen and Fishbein,
1980)
Item
All things considered, my using of E-learning is my job is :-
No.
1 Good …………………………………………………………….Bad
2 Wise …………………………………………………………….Foolish
3 Favorable……………………………………………………….Unfavorable
4 Beneficial ………………………………………………………..Harmful
5 Positive …………………………………………………………..Negative

173

Vol. 19, No. 8, pp. 173-193, August 2020
Conceptual Framework of Evaluation Model on 4

C'S-Based Learning Supervision
Eny Winaryati
Universitas Muhammadiyah Semarang, Semarang, Indonesia
https://orcid.org/0000-0002-4698-002X
Mardiana
Universitas Muhammadiyah Lampung, Lampung, Indonesia
https://orcid.org/0000-0002-2227-1341
Muhamad Taufik Hidayat

Universitas Muhammadiyah Surakarta, Surakarta, Indonesia
https://orcid.org/0000-0002-7787-254X
Abstract. The 21st-century learning requires the implementation of 4

character skills, including (1) Critical Thinking and Problem Solving, (2)
Creativity and Innovation, (3) Collaboration, and (4) Communication (4Cs)
(MESp 4C's). This framework was used to evaluate the achievement of
supervision on 4Cs learning by teachers. The research method used was
Grounded theory, which focuses on building a conceptual framework
through the inductive analysis of various data, phenomena, information,
and theories, using several systematic procedures. MESp 4C's was built in
5 stages. The first is based on the relevance of the learning stages
(planning, learning, and assessment processes) with the learning
supervision stages (pre-observation, observation, and post-observation)
and the evaluation stages of the Stake model (antecedent, transaction,
outcomes). Second, it is necessary to modify the Stake evaluation model
with other evaluation models through the Organizational Elements
Model (OEM). The OEM is a stage for evaluators to determine the
appropriate evaluation model to use. Proper modification of the
evaluation model was obtained, namely The Discrepancy Evaluation Models
(DEM). The Interim Product stage was more appropriate to use than the
outcome so that the evaluation model stages became Antecedent,
Transaction, and Interim Product (ATIp). Third, every step of the
supervision implementation of 4C's will always be evaluated. Fourth,
there were feedback activities (self, peer, superior) to produce continuous
learning evaluation-supervision activities, based on the 360o feedback
theory. Five, MESp 4C's was built as an evaluation model with a modified
six-cell Stake model. MESp 4C's was equipped with the data on the
suitability of descriptions between instances and observations,
assessment data between observations and standards, data on
contingencies and congruence, and feedback activities. This study's

174
recommendation is the need for the design and testing of MESp 4 in

several schools.
Keywords: Conceptualization framework; Evaluation model; Learning

supervision; 4C’s-based
1. Introduction
The quality of education begins with the quality of learning, which will impact the
quality of graduates. The teacher's role is very strategic because everyday teachers
meet with their students to transfer knowledge and skills. Teaching is a complex,
multidimensional, and dynamic endeavor, highly time-dependent, and social and
cultural. Measuring teacher quality, performance, or teaching effectiveness is
much more important than measuring teacher qualifications (Martínez, Schweig,
& Goldschmidt, 2016). The rate of teacher learning will increase when it is
strengthened by the principal's quality of supervision (OECD, 2005; Sabandi,
2013).
Strong quality of supervision results in complete supervision information and

better learning techniques. Conversely, weak supervision, such as incomplete,
inappropriate, and inaccurate supervision, will impact the low quality of learning
(Zhou, 2018). The research results above were corroborated by Daud et al. (2018),
stating that the implementation of high learning supervision can improve teacher
teaching attitudes and competencies. Appropriate efforts need to be made to
strengthen the implementation of supervision and supervisor support in learning
to produce superior teachers with useful teaching competencies in the classrooms.
In the 21st century, the world of education is required to have the readiness to face
the increasingly complex challenges of life. The 21st-century learning skills
encourage broader knowledge, attitudes, and skills to succeed in school, at work,
and in life more broadly (Wolters, 2010). Students should have the critical
thinking, problem-solving, communication, and collaboration (Griffin, McGaw &
Care, 2012). Partnership for 21 Century Skills identifies 21st-century skills, including
(1) Communication, (2) Collaboration, (3) Critical Thinking and Problem Solving, and
(4) Creativity and Innovation (4 C's) (Partnership, 2015).
The success of supervision includes the supervision of learning, which is

determined by feedback with formative assessment, provision of social and
emotional support, good interpersonal relationships, positive supervisory
alliances, clarity of measurable aspects of assessment, teacher involvement in
evaluation, and sufficient time availability by giving adequate reflective space.
The above conditions create a positive relationship in the evaluation process,
which will impact adequate supervision (Kilminster et al., 2007; Bambling & King,
2014; Youngstrom & Gentile, 2018; OECD, 2009a, 2009b, 2009c; Bahri, 2014).
Principals are required to have skills in supervising teachers who impact
improving teacher’s practice evaluation (Reinhorn, Johnson & Simon, 2017).
1.1 Literature Review

The studies related to evaluating learning practices were carried out, including the
assessment implementation based on 360-degree feedback theory. Master (2014)

175
conducted a formative evaluation that impacted administrators' feedback to

teachers regarding various aspects of teaching that influenced future decisions
and students’ emotions. The above is confirmed by Martínez, Schweig &
Goldschmidt (2016) that the aim of the teacher evaluation system's design is to
predict student test scores optimally. It is an effort to improve teaching and
teacher performance in the workplace with a significant focus on student
achievement. Marsh, Bush, Strunk, Lincove and Huguet (2017) reinforce the above
research evaluating teacher learning success by administrators, and their social
relationships place a reciprocal interaction between the principal and teachers.
Administrators make observations based on the rubrics compiled with the
teachers regarding teacher performance and opportunities for improving
learning. The information generated will impact the adjustment possibility of
repair facilities. Involvement in teacher learning evaluation process by
administrators provides direction on how policies will influence implementation
(Master, 2014).
There is still a chance to develop an evaluation model that can be used to assess
the achievement of supervision conducted by school principals with an
assessment based on the 360-degree feedback theory. Given the learning demands
that accommodate the 4 C's, supervision is expected to assess the implementation
of 4C's by teachers to their students.
1.2 Theoretical and Practical gaps

There were some findings related to the administration of learning supervision in
the field, such as the principal who did not give the comprehensive explanation
about teaching strategy, indirect feedback or suggestion, ineffective, no schedule,
and no follow up action based academic (Winaryati & Mufnaety, 2012; Yunus,
Lestari & Raharjo, 2016). The study results indicate that the unsupervised
valuation of the learning process produces only typical values (Büchler, Brattoli &
Ommer, 2018). Most teachers reported that the assessment and feedback they
received beneficial and fair for the development of the profession (OECD, 2009b;
Youngstrom & Gentile, 2018).
Figure 1: The correlation between supervision quality and learning quality

176
Based on the explanation above, supervision is needed that promotes practical,

efficient, impactful, and sustainable, reciprocal activities. This indicates the need
for an evaluation tool used to assess whether the supervision carried out by the
principal has been effective or not. The evaluation results are recommendations,
which are followed up by supervision activities to improve learning (Winaryati &
Mufnaety, 2012). Continuous evaluation is needed in assessing the learning
supervision activities that occur through an evaluation model framework.
Supervisors' role is very urgent, especially in the globalization era, which is full of
dynamic changes in innovation and fast-moving transformation. A supervisor is
required to be able to follow the above developments. There are four character
skills (4 C's) of 21st century learning that students, teachers, and principals must
possess. (Partnership, 2015; Wolters, 2010). Griffin et al. (2012) emphasize that the
4 C's are an urgent topic in the education system agenda. The Conference Board
survey (Scott, 2015) found that professionalism, good work ethics, oral and written
communication, teamwork, collaboration, critical thinking, and problem-solving
skills are essential skills. To face success in today's world, students must possess
critical thinking, problem-solving, communication, and collaboration (Partnership,
2009). The above needs require supervision activities that can encourage the
improvement of 4 C's skills, and evaluation is carried out to obtain information on
the extent to which the implementation of the 4 C's-based supervision is carried
out.
Partnership (2009) states that monitoring and assessing teaching followed up with
improvements as a result of feedback activities for sustainable programs is very
important. The OECD research results (2009b); Youngstrom and Gentile (2018)
revealed that most teachers reported that the assessment and feedback they
received was useful and fair for their professional development. The results of this
study indicate the need for feedback assessment from the principal to the teachers.
Cormack et al. (2018) reinforce the need for a 360-degree evaluation model to
provide comprehensive student evaluations and essential information for many
related groups.
The 360-degree feedback is a system or process (teachers) receiving assessments

from people who work around them (superiors, peers (other teachers)),
subordinates (students), and colleagues (other staff) (Tee & Ahmed, 2014). The
tabulation results of the 360-degree feedback help teachers identify strengths and
weaknesses and motivate them to do better. Feedback recipients gain insight into
how others see themselves and have the opportunity to adjust. It develops the
skills, such as listening, planning, setting goals, providing the ability to work
together in teams, character, and effective leadership (Kanaslan & Iyem, 2016;
Cheng & Wu, 2020). The essence of the 360-degree feedback combines multiple
evaluations using the input from various sources. The sources include coworkers,
subordinates, customers, one’s self, and supervisors. Tee and Ahmed (2014);
Hosain (2016) calls it through multi-source feedback, multi-rater feedback, multi-
level feedback, upward assessment, and peer review.

177
The research results of the 360-degree feedback above encourage the need for a
conceptual framework that analyzes the evaluation of 4Cs-based learning
supervision. It describes a comprehensive evaluation model's components and
describes the main aspects that must be considered for designing a learning
supervision evaluation model. The planned Model is framed in the context of the
goal of overcoming the problem of supervision as well as the implementation of
the continuous evaluation (Winaryati & Mufnaety, 2012).
1.3 Theoretical Model

The evaluation model on learning supervision was performed to provide
convenience for the user to use and positively impact the improvement of
learning. The evaluation model was chosen to put forward an evaluation method
approach consisting of formative and summative evaluation methods. Formative
evaluation provides opportunities for improvement (Shute, 2008). Summative
evaluation ensures that the required standards have been met and as a source of
documentation that is indispensable for teachers' accountability for their
professionalism and results in quality teaching practice (Ola, 2013; Tang & Chow,
2007). The Model presented is expected to describe the elements and relationships
within the proposed conceptual framework and impact future policy practices
(Attwel, 2006; Stufflebeam, 1969; Madaus et al., 1983).
Figure 2. Correlation between evaluation-learning supervision-learning
1.4 Research Purpose

The evaluation model's output can evaluate, predict teacher performance, and
recommend actions needed and that must be taken (Ola, 2013). Evaluation
contributes to creating a teaching profession that is rich in knowledge and skills.
An essential aspect is how the evaluation framework can complement each other,
avoid duplication, and be consistent with objectives.
The research objective was to produce a conceptual evaluation model on the 4 C's-
based learning supervision implemented. Also, the study results obtained the data
that the feedback had an impact on the quality of teacher learning and was
strengthened by the stages of academic supervision (Kemendikbud, 2017). This
becomes the basis for the learning evaluation-supervision model built to
accommodate the needs.

178
The studies on supervision were found a lot, but the 4 Cs implementation's

supervision does not exist. The evaluation results' recommendations will have an
impact on improving classroom learning by the teachers and the mastery
evaluation of the principal as a supervisor regarding the implementation of the 4
C's. This article will direct the conceptual framework of developing an appropriate
evaluation model based on learning supervision stages based on the 4 C's.
1.5 Research Questions

This study seeks to answer: how are the stages of building a conceptual framework
for an evaluation model based on 4 Cs-based supervision, based on theories,
definitions, facts, phenomena, benefits, objectives, concepts, variables, etc. related
to MEPs4Cs.
2. Method
This study used the Grounded Theory (GT) method. This qualitative research
method focuses on creating a conceptual framework by building an inductive
analysis of various data, phenomena, information, and theories using several
systematic procedures. The aim is to develop a theory.
The reasons for choosing the method are based on various references to the results
of previous studies. Chun, Birks, and Francis (2019) state that the GT research
method is a qualitative research method that uses several systematic procedures
to develop theory. Glaser and Holton (2004: p. 43) convey that GT is a set of
integrated conceptual hypotheses generated systematically to produce inductive
theories on substantive areas. Charmaz K (2009) defines GT as a method of
conducting qualitative research that focuses on creating a conceptual or theoretical
framework by building an inductive analysis of data (page 187). The following is
a description of the GT method's application chart and how the conceptual
framework is buil
Figure 3. The Flow Conceptual Framework Construction

179
How to Build a Concept Framework

The conceptual outline (Figure 3) was used to construct the MEPs 4Cs. The
research design began with selecting a theoretical research paradigm that can
inform and guide the research process (Searcy & Mentzer, 2003). This process
connected the chosen research paradigm with the empirical world, the existing
domain to be developed, and the current reality phenomena. From this
perspective, the research design was made in a schema. The conceptual
framework serves as a "map" or "steering wheel" that will guide the realization of
the objectives of the discussion (Masías, 2005).
The conceptual framework is a visual representation of the theory with the

phenomena being studied. Concept maps consist of two things: concepts and the
relationships between them (Maxwell, 1996). The conceptual framework is the
researcher's understanding of how certain variables in his study are connected.
The paper was extracted from peer-reviewed and open-source journals reinforced
by the research data conducted by researchers. The definitions from various
sources were linked together so that a conceptual framework was designed
according to the rules. The conceptual framework was built on the following
keywords:
1. The theory is a collection of variables, problems to solve, definitions, and
related propositions, providing a systematic view of phenomena by
specifying the relationships between various variables to explain existing
phenomena. The theory has the components of concepts, facts, phenomena,
definitions, propositions, and variables (Mullerl & Urbach, 2017; Jaccard &
Jacoby, 2009; Gioia & Pitre, 1990; Nunnally & Bernstein, 1994; Suddaby,
2010).
2. A theory must explain "what a construct is, how and why it is related, and
to whom it applies, when and where it applies, and how it works (Whetten,
1989; Bacharach, 1989, p. 496; Wheeler, 2019).
3. The theory explains a phenomenon that has been repeatedly tested and
found to be consistent over a long period (Wheeler, 2019).
4. A construct is a kind of concept which is to present a categorization or
classification of objects or events into one symbol (Waller, Yonce, Grove,
Faust & Lenzenweger, 2013). The constructs can be networked with one
another (Cronbach & Meehl, 1955). In the conceptual framework, the
concepts/constructs/variables can be added considered to be relevant, and
then the relationship between them is explored or tested (Khoso, 2019;
Wheeler, 2019). Construction is a fundamental concept that includes theory
in which the constructs are inter-related by propositions, and a theory is
made if all these elements are united (Gregor, 2006). A concept can also be
an idea (Weeler, 2019).
5. Between theories, they are characterized by a unique network of constructs
and relationships, and between similar theories, they are related based on
their boundaries. The system of constructs, propositions, and the resulting
assumptions is also referred to as nomological networks, which is a kind of
fingerprint theory (Gregor, 2006).

180
6. A concept is a hypothetical construct consisting of several variables, and it

can also be networked with other constructs (Cronbach & Meehl, 1955).
7. This research contains real, actual, and empirical domains to include critical
realism using an abductive approach. The basic strategy was to interpret and
recontextualize a phenomenon with a conceptual framework or several
thought concepts, and it becomes a new conceptual framework. The
abductive approach is defined as developing a preliminary working
hypothesis prior to inductive data analysis (Peirce, 1998).
3. Result
Conceptual Framework for Building an Evaluation Model
The Model's conceptual framework was built based on the analysis of several
theories, the facts obtained, the phenomena that occur, and then it was constructed
using an inductive approach. There are several variables for which data must be
obtained. In-depth analysis was carried out during the research regarding the
implementation of supervision. The phenomena were: the principal did not tell
the whole of the teaching strategy, was not quick to provide feedback /
suggestions, and implemented academic supervision was still ineffective,
unscheduled and there was no follow-up. In addition, several field data were
obtained that reinforced the above phenomena. The field facts obtained a
description that it is necessary to evaluate the implementation of supervision.
A complete understanding related to several theories about learning, supervision,

and evaluation was required. Also, the understanding and implementation were
needed regarding planning readiness, learning and assessment processes and
feedback. The definition of teaching, availability of the syllabus, lesson plans,
learning objectives, indicators, methods and media used, and assessments must
have been prepared.
The skills characters (4Cs) of 21st century learning are the demands that must be
understood and implemented. The teachers transferred the above skills to the
students so that the students had 4Cs competence. It was expected that the
students will have readiness to face the needs, problems and challenges in the
future. The supervisors need to supervise the extent to which the teachers in their
learning have implemented 4Cs. An evaluation was carried out to obtain data on
the extent to which 4C-based supervision was carried out.
There was a complete understanding related to the meaning, definition, goals,

needs and expectations of learning supervision. It was about what the supervisor
should do when the teachers prepared, how the learning process, assessment, and
reflective feedback were carried out. Besides, it was the basis for how 360 degree
feedback was implemented, what strategies were involved, and who were
involved in the feedback. The questions of what, why and how were the basis for
the researchers to develop the instruments for supervision and evaluation.
There was an understanding and relevance between supervision and evaluation,

i.e., how the evaluation model's construction was constructed to answer what,
why, and how the evaluation activity was carried out. Some of the data, theories,

181
initial concepts, and empirical evidence were the variables networked to form a
construction and to become the expected conceptual framework for the Model.
The selection of the evaluation model used was based on its definition, objectives,
functions, benefits, and ways of implementing it. There are several evaluation
models, each of which has advantages and disadvantages. It was the basis for the
need for a combination of evaluation models. The evaluation model determined
had accommodated its relevance to learning, 4 C's and learning supervision. The
conceptual framework of this evaluation model was structured based on the steps
with the flow based on a systematic sequence.
Purpose
Implementation Phases
Definition Planning Phenomena

Phases
Preparation Assessment Facts
21st century Phenomena
learning 4 C’s Learning Supervision
Benefit
Purpose
4 C’s-based
Evaluation Definition
Learning
Model Supervision
How to How to
implement 360 0
implement
Feedback
Model 4 C’s-based
Evaluation Learning Who
Feedback
Purpose Supervision involved
The Use Benefit Self

The Model of the Sustainable 4 Evaluation
C’s-based Learning Supervision-
Evaluation
Figure 4. Concept Framework of Building Evaluation Model
4. Discussion
4.1 Building 4 C's-Based Learning Supervision Evaluation Content
The preparation stages (MESp 4C's) are as follows; First, strengthening teacher
competence to improve the teaching and learning process in schools is very
important. Selvi (2010) states that the contribution of all teaching competencies
simultaneously or together has a significant influence on improving the learning

182
process's performance quality. This means that learning process is a target that
must be put forward. Therefore, the supervision of learning program must be
carried out. Daresh (1989) and Glickman, et al (2007) argue that academic
supervision is a series of activities to help teachers develop their ability to manage
learning process to achieve learning goals. This indicates that academic
supervision is inseparable from evaluating teacher performance in managing
learning (Sergiovanni, 1987) and improving learning quality (Kemendikbud, 2017;
Prabowo & Yoga, 2016). Learning supervision is carried out at the planning,
implementation, and learning assessment stages, as well as post-learning
feedback. Supervision is a humane mentoring activity through democratic
relationships, openness and friendship (Bafadal, 1992; Maralih, 2014).
Third, the academic supervision stage consists of three stages: pre-observation

(observation/ meeting before learning), observation (observation of learning) and
post-observation (feedback meeting) (Depdikbud, 2017). Pre-observation contains
the preparation and planning of learning, while the observation stage is the
implementation of the planning that has been prepared. Feedback can be used to
increase its effectiveness in the classroom, as well as formative assessments (Hattie
& Timperley, 2007; Shute, 2008; Darto, 2014). The implementation of academic
supervision by the principal is carried out in three stages: the initial survey before
conducting academic supervision, class visits to find out the course of learning,
and reviewing the results of class visits and providing the right solutions in
overcoming problems faced by teachers (Ajasan, 2016).
Fourth, the hope is that through the implementation of this learning supervision
it can be carried out sustainably. This reflective feedback process is carried out
through learning supervision. Weak supervision will have an impact on the
lacking quality of teachers and the quality of learning is not optimal (Zhou, 2018).
This suggests that based on the studies related to 360-degree feedback, there can
be an assessment of the people who work around teachers from superiors, peers
(other teachers), subordinates (students), and colleagues (other staff).
Fifth, learning in the 21st century requires the possession of 4 skills consisting of:
(1) Communication, (2) Collaboration, (3) Critical Thinking and Problem Solving,
and (4) Creativity and Innovation. The four 21st century skills above are often
termed 4C's skills. Students, teachers and school principals must own these 4 Cs.
Then, the supervision carried out by the principal can assess the achievement of
the 4 Cs in learning, and evaluation is carried out to assess the extent to which 4C-
based supervision is implemented.
Sixth, based on the substance of the pre-observation and observation stages above,
it indicates: (a) Conformity between planning and implementation. (b)
Supervision emphasizing the gap between the standard formulations that have
been determined and the reality that occurs (results of observations). (c) Each of
them has a phasing process (including input, process and output), until a product
is produced. The products from the pre-observation stage will contribute /
influence the observation stage and the post-observation.

183
4.2 Building MESp 4C's framework

Based on the learning supervision stages above, the Stake Model (Countenance
Evaluation Model) is the right choice. Stake divides the evaluation object into 3
categories: Antecedents, Transactions, Outcomes. Antecedents are the sources /
models / inputs that exist in a system to be developed, such as energy, finance,
student characteristics, and goals to be achieved, conditions that exist before
instruction that may be related to results. Transaction includes the activity plan
and the process of its implementation in the field including the sequence of
activities, time scheduling, teacher-student interaction forms, how to assess
learning outcomes, etc. Engagement in a dynamic sequence or meeting is a process
of instruction. Outcomes are the results achieved by students, teacher's reaction to
a system, the side effects of the system concerned, and the impact of instructional
experiences, (Popham, 1993: 5-15; Stake, 1977: 372-390; Fernandes, 1984: 8 -10;
Wood, 2001: 18-27; Owston, 2008).
Description Matrix Judgment Matrix

Intention Observations Standards Judgement
C C
Congruence
o o ANTECEDENT
R n n
a ti ti
Congruence TRANSACTION
ti n n
o g g
n e e
a n n
l c Congruence c
OUTCOMES
i i
e e
s s
Figure 4. Implementation of Stake Model (Stake’s, 2000:351)
There are two approaches to the Stake's Model: contingencies and congruence.
Contingencies are logical relationships between the 3 dimensions above
(antecedent, transaction, outcomes). Congruence is a conformity between what is
expected (criteria) and what happens / the results in plans, processes and results
(Popham, 1993: 5-15; Stake, 2000: 350-351). Outcome depends on the transaction
and on antecedent conditions. Through simultaneous analysis of several aspects,
evaluation will find significant interactions for improvement. The figure of
evaluation flow is shown below.

184
Intended Antecedents Congruence Observed Antecedents
Logical Contingency
Logical Contingency
Intended Transactions Congruence

Observed Transactions
Logical Contingency Logical Contingency
Intended Interim Product Congruence Observed Interim Product
Source: Stake, 1977: 372-390

Figure 5. Data Description Process of contingencies and congruence
Based on the relevance of the learning stages that the teachers must carry out, there
is conformity with the learning supervision stages, and there is conformity with
the Stake's evaluation model. The adjustment flow is as follows:
Learning Supervision Stakes Model
Planning Pre- Observation Antecedent
Learning Process Observation Transaction
Assessment
Post Observation Outcomes
Figure 6. Adjustment Flow of Learning Phases-Supervision-Stakes

Evaluation Model
The results of the research above convey that the success of learning supervision
includes constructive feedback and adequate reflective space. The results of this
feedback provide the room for continuous evaluation. Related to this, MESp 4C's
needs feedback so that cyclic activities will be carried out. The idea of the Model
above is in line with what was conveyed by Tang and Chow (2007). The results of
their research convey that feedback communication from the results of learning
observations is a discussion of the results of mentoring and peer coaching. The
feedback content is produced by an assessment oriented towards improving
learning in the context of summative evaluation to produce deeper insights from
the supervision of teaching practices.

185
The explanation strengthens the explanation above that a temporary product is

produced to go to the next stage in the pre-observation and observation stages.
Post-observation is a temporary product as a result of feedback for improvement
at the next pre-observation stage. In this sense, it means the production of
temporary product. Besides, the outcome is a long-term result. Therefore, it is
necessary to modify the outcome stage with other evaluation models. Supervision
is to assess the existing standards and the performance carried out, and then an
assessment of the gaps that occur is an option. The evaluation model based on the
gap theory is The Discrepancy Evaluation Model (DEM).
Roger Kaufman developed organizational Elements Model (OEM) as a tool that

can be used to identify different elements in a system. A system is "a set of
interrelated components that work together to achieve a common goal", (Porter,
2005a). OEM provides a systemic framework, designs and implements an effective
way to achieve the desired result. Kaufman (1988) divides OEM into five elements
that interact with each other: Inputs, processes, products, outputs, and outcomes.
Products and outputs are the results that occur in an organization; outcome is the
result outside an organization (Chyung, 2005). Input is raw material; process is
how to do it; products are temporary results in process; and output is the
organization's achievement. As a product delivered to society; outcome is an effect
for society (Porter, 2005). Kaufman (2006: 6-16) affirms that OEM is a stage for
evaluators to determine the right evaluation model.
Model Process Outcome

Input Product Output
s
Formative Formative Summative -------------------------
Summative ------
CIPP Context Process Product -------------------------
Input ------
CSE Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
Stake Antecedent Transaction Outcome -------------------------
------
Tyler Goal 1 Attainment ------------------------------------------
Identification -----
Provus Installation Implementa Fulfil ----------Cost----------
(DEM) Tion Objectives? --
Benefit
(Depending upon objectives)
Sciven’s -----------------------------------------------------------------------------
Goal Free ----
Stake Resp -----------------------------------------------------------------------------
---
Source: Kaufman and Thomas (1980: 137).
Figure 6. Relationship of OEM with Several Evaluations.
OEM is a stage for evaluators to determine the appropriate evaluation model

(Kaufman, 2006: 10-16). Based on this understanding, it is necessary to carry out a
mapping to place an appropriate evaluation stage. Based on the OEM stage

186
description, the appropriate DEM stage is the interim product (temporary

product). The interim product describes the relationship between the program
process and the temporary product. Mapping via OEM is compatible with the
evaluation similarity that underlies the gap theory. The evaluation model with a
substance based on the gap theory (between standards and observations, and
between intention and implementation) of the Stake's Model is The Discrepancy
Evaluation Model (DEM). These models emphasize the gap theory as a tool for
making judgments based on standards and performance (Provus, 1969: 9,18).
Stake's difference is that there are additional congruence activities (conformity
between instances and observations) and contingencies (logical relationships in
Antecedents, Transactions, and Outcomes). DEM is in the process of repairing,
recycling, or stopping the program.
The Discrepancy Evaluation Model (DEM), designed by Malcolm Provus in 1969,

is an effective way to evaluate academic programs. DEM is called program gap
evaluation. The program gap is a condition between what is expected in the plan
and generated in program implementation. Gap evaluation is intended to
determine the level of conformity between the program's standards and the actual
appearance of the program. Standards are the criteria that have been developed
and established with significant results. Provus (1969) defines evaluation as a tool
to make a judgment on the advantages and disadvantages of an object based on
standards and performance. This model is also considered a constructive
approach and oriented towards systems analysis (Provus, 1969: 10-14; Steinmetz,
2000: 135).
DEM offers a systematic pragmatic approach to a variety of evaluation needs.

DEM can be utilized to structure important information gathering both for
information and making decisions. The main emphasis of the DEM is on self-
evaluation and systematic improvement of a program.
DEM divides the evaluation stage into five stages: Program Design, Program
Operation, Program Interim Products, Program Terminal Products, and Program
Cost. Provus argue that all programs have a life cycle. Because a program consists
of development steps, many evaluation activities mean integrating integration in
each of its components.
S
C D 2 3 C D 4 C D 5
1 C D
P P P P
A A A A
(Source: Provus 1969: 13)
Figure 7. Flowchart of DEM Evaluation Process Phases

187
Based on the stages of the combination of the Stake's Model with DEM, the steps
of the Antecedents, Transaction, Interim Product, abbreviated ATIp model is
produced. In the ATIp evaluation model, there are several definitions related to
standard, observation, and intensity. Standard is a benchmark/measure that must
be met, and that is expected by the stakesholder / government and has been set.
In this article, the standard formulation is based on the academic supervision
guidebook (2017), four characters of 21st-century learning skills; Permendiknas RI,
(No. 21,22,23,24, 2016). Intense is what the teacher means. Observation is what the
observer feels. The stages of the ATIp evaluation process are described as follows:
Antecedent Observation
Intention
Transaction
Observation Standard
Interim
Product
Congruence Judgment
Contingencies
Information Learning
Improvement
Figure 9. Evaluation Process Phases through ATIp
There is some additional information from the figure: 1) judgment is the

gap/conformity between standards and observations; 2) congruencies are the
gaps/matches between intention and observation; 3) contingencies are the
gaps/relationships in antecedents, transactions, and interim products. The
assessment results of the activities of judgment, congruencies, and contingencies
produce information used to improve learning.
Based on the explanation above and clarified with the figure, an evaluation
model's conceptual framework on 4 C-based learning supervision is produced.
The issue of post-learning supervision feedback is adopted through continuous
evaluation-supervision activities. The MESp 4C conceptual framework results in
the relationship of 6 cells of the Stake model modified and compatible with the
learning supervision stages. The six MESp 4C cells are described as follows:

188
INTENTION STANDARD
OBSERVATION
Antecedents Antecedents Judgment Antecedents
Transaction Transaction Transaction
Interim Interim Interim

Products Products Products
Contingencies Congruence
PRE OBSERVATION OBSERVATION POST OBSERVATION
Figure 8. Relationship of 6 Cells of MESp 4C’s and Learning Supervision Phases
5. Conclusion
The quality of teacher learning will have an impact on the quality of graduates;
the quality of supervision will affect the quality of learning, and evaluation is
carried out to assess the extent to which the supervision process is carried out. The
learning in the 21st-century era is required to create the learning with 4 (four)
character skills including Critical Thinking and Problem Solving, Creativity and
Innovation, and Collaboration and Communication (4 C's). The evaluation carried
out is expected to provide information on the implementation of 4 C-based
supervision.
The studies on supervision were found a lot, but the ones related to the 4 Cs
implementation supervision did not exist. The evaluation results'
recommendation will have an impact on improving classroom learning by the
teachers and the evaluation of the principal's mastery as a supervisor regarding
the implementation of 4 C's. This article will lead to the conceptual framework of
developing an appropriate evaluation model based on the stages of learning
supervision based on 4 C's.
MESp 4C's is built based on five stages. The first is based on the relevance of the
learning stages (planning, learning, and assessment processes) with the learning
supervision stages (pre-observation, observation, and post-observation), as well
as the evaluation stages of the Stake model (antecedent, transaction, outcomes).
Second, it is necessary to modify the Stake evaluation model with other evaluation
models through the Organizational Elements Model (OEM). OEM is a stage for

189
evaluators to determine the appropriate evaluation model to use. Proper

modification of the evaluation model was obtained, namely The Discrepancy
Evaluation Models (DEM). The Interim Product stage was more appropriate to use
than the outcome one so that the evaluation model stage became Antecedent,
Transaction, and Interim Product (ATIp). Third, every step of 4C's implementation
supervision will always be evaluated. Fourth, there were feedback activities (self,
peer, superior) to produce continuous evaluation-supervision learning activities,
based on the 360o feedback theory. Fifth, it made MESp 4C's like an evaluation
model with a modified six cells of the Stack model. MESp 4C's was equipped with
the data on the suitability of the descriptions between instances and observations,
the assessment data between observations and standards, contingencies and
congruence data, and feedback activity data.
Recommendations and Research Limitations

The research recommendation is the need to compile the MESp 4C's design and to
conduct trials in several schools. This study's limitation is that the data on the
results of the mod,el effectiveness test cannot be obtained quickly because it
requires an extended trial period and involves many schools.
6. References
Ajasan, N. (2016). Efektivitas pelaksanaan supervisi akademik oleh kepala sekolah dalam
meningkatkan kinerja guru di SMK negeri 1 Meulaboh [The effectiveness of the
implementation of academic supervision by the principal in improving teacher
performance at State Vocational High School 1 Meulaboh]. Jurnal Administrasi
Pendidikan Program Pascasarjana Unsyiah 4(3). Retrieved from
http://jurnal.unsyiah.ac.id/JAP/article/view/4795
Attwel, G. (2006). Evaluating e-learning a guide to the evaluation of e-learning No Title. Evaluate
Europe Handbook Series, 2, 5–46. Retrieved from http://pontydysgu.org/wp-
content/uploads/2007/11/eva_europe_vol2_prefinal.pdf
Bacharach, S. B. (1989). Organizational theories: some criteria for evaluation. Academy of
Management Review (14:4), pp. 496-515. Retrieved from
https://gent.uab.cat/diego_prior/sites/gent.uab.cat.diego_prior/files/Bacharac
h_AMR_1989_14_4.pdf
Bafadal, I. (1992). Supervisi pengajaran: teori dan aplikasinya dalam membina profesional guru
[Teaching supervision: theory and its application in fostering teacher professionals].
Jakarta: Bumi Aksara. Retrieved from
https://opac.perpusnas.go.id/DetailOpac.aspx?id=333958
Bahri, S. (2014). Supervisi akademik dalam peningkatan profesionalisme guru. Visipena
Jurnal, 1(1), 100–112. doi:10.46244/visipena.v5i1.236
Bambling, M., & King, R. (2014). Supervisor social skill and supervision
outcome. Counseling and Psychotherapy Research, 14(4), 256–262.
doi:10.1080/14733145.2013.835849
Buchler, U., Brattoli, B., & Ommer, B. (2018). Improving spatiotemporal self-supervision
by deep reinforcement learning. In Lecture Notes in Computer Science (including
subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics) (Vol. 11219 LNCS, pp. 797–814). Springer Verlag. Retrieved from
doi:10.1007/978-3-030-01267-0_47
Charmaz, K. (2009). Shifting the grounds: Constructivist grounded theory methods.
In Developing grounded theory: The second generation (pp. 127–154). New York.
Retrieved from https://www.mendeley.com/catalogue/5837981d-1495-351d-
9c13-f5621133a056/

190
Cheng, T. F., & Wu, H. C. (2020). A follow-up study on vocational high school principals’
opinions about 360 degree evaluation feedback and their leadership effectiveness
and behavior change. Asia Pacific Education Review, 21(1), 65–81.
doi:10.1007/s12564-019-09608-x
Chun Tie, Y., Birks, M., & Francis, K. (2019). Grounded theory research: A design framework for
novice researchers. SAGE Open Medicine, 7. doi:10.1177/2050312118822927
Chyung, S. Y. (2008). Foundations of instructional and performance technology. Canada: HRD
Press, Inc. Retrieved from
https://downloads.hrdpressonline.com/files/3420F080326114240.pdf
Cormack, C. L., Jensen, E., Durham, C. O., Smith, G., & Dumas, B. (2018). The 360-degree
evaluation model: A method for assessing competency in graduate nursing students. A
pilot research study. Nurse Education Today, 64, 132–137.
doi:10.1016/j.nedt.2018.01.027
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests.
Psychological Bulletin, 52(4), 281-302. doi: 10.1037/h0040957
Daresh, J. C. (1989). Supervision as a proactive process. White Plains, NY: Longman. Retrieved
from https://www.amazon.com/Supervision-Proactive-Process-John-
Daresh/dp/B003MAMGHA
Darto. (2014). Supervisi akademik terprogram dalam peningkatan kompetensi guru pada
pelaksanaan pembelajaran [Programmed academic supervision in improving
teacher competence in the implementation of learning]. Jurnal Penelitian Tindakan
Sekolah Dan Kepengawasan, 1(1). Retrieved from
file:///C:/Users/ASUS/Downloads/75-147-1-SM.pdf
Daud, Y., Dali, P. D., Khalid, R., & Fauzee, M. S. O. (2018). Teaching and learning
supervision, teachers’ attitude towards classroom supervision and students’
participation. International Journal of Instruction, 11(4), 513–526.
doi:10.12973/iji.2018.11432a
Fernandes, H. J. X. (1984). Evaluation educational of program. Jakarta: National Education
planning evaluation and curriculum development. Retrieved from
https://www.researchgate.net/publication/338445328_Evaluation_of_Curriculu
m_Implementation_2013_Batubara_District
Gioia, D. A., & Pitre, E. (1990). Multiparadigm perspectives on theory building. Academy
of Management Review (15:4), pp. 584-602. doi:10.2307/258683
Glickman, C. D. (1981). Developmental supervision: alternative practices for helping teacher
improve instruction. Virginia, Alexandria: ASCD. Retrieved from
https://eric.ed.gov/?id=ED208487
Gregor, S. (2006). The Nature of theory in information systems, MIS Quarterly (30:3), pp.
611-642. Retrieved from
https://misq.org/skin/frontend/default/misq/pdf/TheoryReview/Gregor.pdf
Griffin, P., McGaw, B., & Care, E. (2012). Assessment and teaching of 21st century
skills. (Vol. 9789400723245, pp. 1–345). Springer Netherlands. doi:10.1007/978-94-
007-2324-5
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research.
Vol. 77, No. 1, pp. 81-112 DOI: 10.3102/003465430298487
Hosain, Md. S. (2016). 360 Degree feedback as a technique of performance appraisal: Does
it really work? Asian Business Review, 6(1), 21. doi:10.18034/abr.v6i1.779
Jaccard, J., & Jacoby, J. (2009). Theory construction and model building skills, (1. ed.). New
York, NY, USA: The Guilford Press. Retrieved from
http://eprints.qums.ac.ir/788/1/Theory%20Construction%20and%20Model%20
%20Building%20Skills.pdf
Kanaslan, E. K., & Iyem, C. (2016). Is 360 degree feedback appraisal an effective way of
performance evaluation?. International Journal of Academic Research in Business and
Social Sciences, 6(5). doi:10.6007/ijarbss/v6-i5/2124

191
Kaufman, R. (1988). Preparing useful performance indicators. Training & Development

Journal, 42(9), 80–84. Retrieved from
http://libproxy.boisestate.edu/login?url=http://search.ebscohost.com/login.as
px?direct=true&db=tfh&AN=9126902&site=ehost-live
Kaufman, R. (2006). Defining and delivering measurable value: a mega thinking and
planning primer. Performance Improvement Quarterly, 18(3) pp. 6-16. Florida State
University. doi:10.1111/j.1937-8327.2005.tb00337.
Kemendikbud. (2017). Panduan supervisi akademik. Jakarta: Kemendikbud. Retrieved from
http://repositori.kemdikbud.go.id/9042/1/KSP_M10.pdf
Khoso, P. A. (2019). What are the differences between conceptual framework and the
theoretical framework?. Universiti Kebangsaan Malaysia. School of Social,
Development & Environmental. SOSDES. Retrieved from
https://www.researchgate.net/post/What_are_the_differences_between_conce
ptual_framework_and_theoretical_framework/2
Kilminster, S., Cottrell, D., Grant, J., & Jolly, B. (2007). AMEE Guide No. 27: Effective
educational and clinical supervision. Medical Teacher, 29(1), 2-
19. doi:10.1080/01421590701210907
Madaus, G. F., Scriven, M. S., & Stufflebeam. (1983). The Cipp model for program
evaluation. In Evaluation Models (pp. 117–141). Springer Netherlands.
doi:10.1007/978-94-009-6675-8_7
Maralih. (2014). Peranan supervisi dalam peningkatan kualitas pendidikan [The role of
supervision in improving the quality of education]. Jurnal Qathruna, 1(1), 179–192.
Retrieved from
http://jurnal.uinbanten.ac.id/index.php/qathruna/article/view/251
Marsh, J. A., Bush-Mecenas, S., Strunk, K. O., Lincove, J. A., & Huguet, A.
(2017). Evaluating teachers in the big easy: How organizational context shapes
policy responses in New Orleans. Educational Evaluation and Policy Analysis, 39(4),
539–570. doi:10.3102/0162373717698221
Martínez, J. F., Schweig, J., & Goldschmidt, P. (2016). Approaches for combining multiple
measures of teacher performance. Educational Evaluation and Policy Analysis, 38(4),
738–756. doi:10.3102/0162373716666166
Masías, R. (2005). Prólogo a la llamada investigación cualitativa en ciencias sociales
Colombia Internacional, núm. 62, julio-diciembre, pp. 120-131 Universidad de Los
Andes Bogotá, D.C., Colombia. Retrieved from
http://www.scielo.org.co/pdf/rci/n62/n62a08.pdf
Master, B. (2014). Staffing for success. Educational Evaluation and Policy Analysis, 36(2), 207–
227. doi:10.3102/0162373713506552
Maxwell, J. A. (1996). Qualitative research design, an interactive a halroach. California: Sage
Publications, Inc. Retrieved from
https://www.scirp.org/(S(351jmbntvnsjt1aadkposzje))/reference/ReferencesPa
pers.aspx?ReferenceID=1424076
Mullerl, B., & Urbach, N. (2017). Understanding the why, what, and how of theories in is
research. Research Center. Finance & Information management: University of
Groningen. Retrieved from
https://pdfs.semanticscholar.org/780c/0d7e229e0ae37f213185c639d621ac4d9620
.pdf
Nunnally & Bernstein, I. H. (1994). Psychometric theory (3 nd ed). New York: McGraw Hill.
Retrieved from
https://journals.sagepub.com/doi/abs/10.1177/014662169501900308
OECD (2005). Teacher matter: Attracting, developing, and retaining effective teacher. Paris:
OECD. Retrieved from https://www.oecd.org/education/school/34990905.pdf
OECD (2009a). OECD Review on Evaluation and Assessment for Improving School Outcomes:
Design and Implementation Plan for the Review. Paris: OECD [OLIS Document

192
EDU/EDPC(2009)3/REV1]. Retrieved from

http://www.oecd.org/education/school/44568070.pdf
OECD (2009b). Creating effective teaching and learning environments: first results from TALIS.
Paris: OECD. Retrieved from
OECD (2009c). A Conceptual framework and examples of country practices. Evaluation,
(December). Retrieved from
Ola, A. F. (2013). A framework of an improved model for evaluation of instructors’
performance in higher institutions of learning. Journal of Research & Method in
Education (IOSRJRME), 3(2), 64–69. doi:10.9790/7388-0326469
Owston, R. (2008). Models and methods for evaluation. Toronto, Canada: York University.
Partnership. (2009). P21 framework definitions. Retrieved from
https://files.eric.ed.gov/fulltext/ED519462.pdf
Partnership. (2015). P21 Partnership for 21st century learning. Partnership for 21st Century
Learning, 9. Retrieved from
http://www.p21.org/documents/P21_Framework_Definitions.pdf
Peirce, C. S. (1998). On the logic of drawing history from ancient documents. The Essential
Peirce (Vol. 2), C. S. Peirce (ed.). Bloomington, IL, USA: Indiana University Press,
pp. 75-11. Retrieved from
https://altexploit.files.wordpress.com/2017/11/charles-s-peirce-nathan-houser-
christian-j-w-kloesel-peirce-edition-project-peirce-edition-project-the-essential-
peirce_-selected-philosophical-writings-volume-2_-1893-1913-india.pdf
Popham, W. J. (1993). Educational evaluation. Boston: Allyn and Bacon. Retrieved from
https://www.worldcat.org/title/educational-evaluation/oclc/25675424
Prabowo, S., & Yoga, D. S. (2016). Supervisi kunjungan kelas sebagai upaya membina
professional guru SLTP/SLTA [Supervision of class visits as an effort to foster
professional junior / senior high school teachers]. Jurnal Sosial Humaniora, 9(1), 96.
doi:10.12962/j24433527.v9i1.1281
Provus, M. (1969). The discrepancy evaluation models. An approach to local program
improvement and development. Pittsburgh: Public School.
Reinhorn, S. K., Johnson, S. M., & Simon, N. S. (2017). Investing in development: Six high-
performing, high-poverty schools implement the Massachusetts teacher
evaluation policy. Educational Evaluation and Policy Analysis, 39(3), 383–406.
doi:10.3102/0162373717690605
Sabandi, A. (2013). Supervisi pendidikan untuk pengembangan profesionalitas guru
berkelanjutan [Educational supervision for continuous teacher professional
development]. Pedagogi: Jurnal Ilmu Pendidikan, 8(2), 1–9.
Scott, C. L. (2015). The futures of learning 3: what kind of pedagogies for the 21st century?
Education Research and Foresight, UNESCO Digital Library, 1–21. Retrieved from
https://unesdoc.unesco.org/ark:/48223/pf0000243126
Searcy, D., & Mentzer, J. T. (2003). A Framework for conducting and Evaluating research.
Journal of Accounting Literature, 22, 130-167. Retrieved from
https://www.mendeley.com/catalogue/5198beaf-abec-366f-994f-01c03c9f02c5/
Selvi, K. (2010). Teachers’ competencies. Cultura International Journal of Philosophy of Culture
and Axiology, 7(1), 167-175. doi:10.5840/cultura20107133
Sergiovanni, T. J., & Starratt, R. J. (1998). Supervision: A redefinition (6th ed.). Boston:
McGraw-Hill.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–
189. doi:10.3102/0034654307313795
Stake, RE. (1977). The countenance of educational evaluation. In A. A. Ballack & H. M.
Kliebard (Eds.), Curriculum and evaluation (pp. 372-390). Berkeley, CA: McCutchan.
doi:10.4135/9781412984157.n13

193
Stake, R. E. (2000). Program evaluation, particularly responsive evaluation. Evaluation models

viewpoints on educational and human services evaluation (2nd Ed. Pp. 340-362).
New York: Kluwer Academic Publisher. Retrieved from
https://link.springer.com/chapter/10.1007/0-306-47559-6_18
Steinmetz, A. (2000). The discrepancy evaluation model. In D. L. Stufflebeam, G. F. Madaus
& T. Kellaghan (Eds.), Evaluation models: Viewpoints on educational and human
services evaluation (2nd ed., pp. 127-143). Hingham, MA: Kluwer Academic
Publishers. doi:10.1007/0-306-47559-6_7
Stufflebeam, D. L. (1969). Evaluation as enlightenment for decision-making. In H. B.
Walcott (Ed.), Improving educational assessment and aninventory of measures of affective
behavior (pp. 41-73). Washington, DC: Association for Supervision and Curriculum
Development and National Education Association
Suddaby, R. (2010). Construct clarity in theories of management and organization.
Academy of Management Review, 35(3), 346–357. Retrieved from https://sci-
hub.tw/10.5465/AMR.2010.51141319
Tang, S. Y. F., & Chow, A. W. K. (2007). Communicating feedback in teaching practice
supervision in a learning-oriented field experience assessment framework.
Teaching and Teacher Education, 23(7), 1066–1085. doi:10.1016/j.tate.2006.07.013
Tee, D. D., & Ahmed, P. K. (2014). 360 degree feedback: An integrative framework for
learning and assessment. Teaching in Higher Education, 19(6), 579–591.
doi:10.1080/13562517.2014.901961
Waller, N. G., Yonce, L. J., Grove, W. M., Faustb, D., & Lenzenweger, M. F. (2013). A Paul
Meehl reader: Essays on the practice of scientific psychology. A Paul Meehl Reader: Essays
on the Practice of Scientific Psychology (pp. 1–578). Taylor and Francis.
doi:10.4324/9780203759554
Wheeler, K. (2019). What is the difference between a theory and a concept?. Faculty sponsor for
Alpha Chi Honor Society, Tennessee Delta Chapter. Retrieved from
https://www.quora.com/What-is-the-difference-between-a-theory-and-a-
concept
Whetten, D. A. (1989). What Constitutes a Theoretical Contribution?. Academy of
Management Review, 14(4), 490-495. doi:10.5465/amr.1989.4308371
Winaryati, E., & Mufnaety. (2012). Kompetensi kepala sekolah sebagai supervisor pada
pembelajaran IPA SMP di kota Semarang [The competence of the principal as a
supervisor in junior high school science learning in the city of Semarang]. Prosiding
Universitas Muhammadiyah Semarang. Retrieved from
http://jurnal.unimus.ac.id/index.php/psn...
Wolters, C. A. (2010). Self-regulated learning and the 21 st century competencies. USA
University of Houston.
Wood, B. B. (2001). Stake's countenance model: evaluating an environmental education
professional development course. The Journal of Environmental Education, 32(2) 18-
27. doi:10.1080/00958960109599134
Youngstrom, M. J., & Gentile, P. A. (2018). Supervision. In B. Schell & G. Gillen (Eds.),
Willard & Spackman's Occupational Therapy (13 Ed., Chapter: 72). Wolters Kluwer
Health/Lippincott Williams & Wilkins
Yunus, G. A., Lestari, W., & Raharjo, T. J. (2016). Pengembangan Model Supervisi
Akademik Berbasis Evaluasi Diri Bagi Guru SMA [Development of Self-
Evaluation Based Academic Supervision Model for High School
Teachers]. Educational Management, 5(1), 12–22.
Zhou, Z. H. (2018). A brief introduction to weakly supervised learning. National Science
Review. Oxford University Press. doi:10.1093/nsr/nwx106

194

Vol. 19, No. 8, pp. 194-213, August 2020
The Effect of Classroom Climate on Academic

Motivation Mediated by Academic Self-Efficacy
in a Higher Education Institute in China
Qiumei Wang
University of Malaya, Kuala Lumpur, Malaysia
Youjiang Medical University for Nationalities, Baise, China
https://orcid.org/0000-0001-9698-059X
Kenny Cheah Soon Lee

https://orcid.org/0000-0001-9881-582X
Kazi Enamul Hoque

https://orcid.org/0000-0002-8152-9250
Abstract. Learning theories and empirical findings have widely

recognized academic motivation as the driving force behind student
academic performance. However, recent findings have indicated the need
to pay attention to drive academic motivation among undergraduate
students in China. Therefore, this present study examines the direct and
indirect effects of classroom climate on student academic motivation in
higher education. A total of 119 undergraduate students from a medical
college in China was approached to respond to a survey questionnaire.
The data analysis result shows that classroom climate significantly and
positively affects student academic motivation and that the relationship
is significantly mediated by academic self-efficacy. Particularly, the
relationship established in this study additionally indicates that a
planned and cohesive classroom climate can effectively improve student
academic motivation. Indicatively, a planned and fair classroom climate
effectually builds students' self-efficacy in their learning. This study
highlights the indispensable and pivotal role that a positive classroom
climate plays in motivating students to be more engaged and learn
effectively. Also, the findings of this study provide guidance and
references for stakeholders, school administrators, and lecturers in
building and sustaining a positive and healthy classroom climate for
better learning.
Keywords: Classroom Climate; Academic Motivation; Academic Self-

Efficacy; Academic Performance

195
1. Introduction
Learning theories like Self-determination Theory (Deci & Ryan, 1985) and Self-
worth Theory (Covington, 1992), as well as empirical researches, have clearly
posited the crucial role that academic motivation plays in facilitating students’
academic performance (Bailey & Phillips, 2016; Cleary & Kitsantas, 2017; Froiland
& Worrell, 2016; Ladd & Sorensen, 2017). Effective learning process relies upon
the triggering of academic motivation (Deci & Ryan, 2000). Students who find
interest, value, and enjoyment in the learning process, indicate more engagement
and persistence in learning activities (Pelletier, Fortier, Vallerand & Brière, 2001;
Ratelle, Guay, Vallerand, Larose & Senécal, 2007). Consequently, strongly
motivated students exert more effort to complete tasks (Wolters, 2004), and
display higher levels of academic achievement (Mega, Ronconi & De Beni, 2014;
Supervia & Bordas, 2018). Evidently, the role of motivation in learning is heavily
documented; however, the complexity and breadth of the notion of motivation
requires an introspective lens using classroom climate and self-efficacy.
It is quite disturbing that researches have revealed a low level of academic

motivation among undergraduate students (Gao & Gao, 2015; Liu, 2013; Qiu,
2013), especially among science students in China (Gou, 2016; Zhao, 2015). In
addition, only 45 percent of undergraduate students conveyed that they only
focused on their studies in professional course classes. When it came to the
selective course classes, the percentage declined to 18 percent (Wang & Lin, 2018).
This could be explained by the disconnection between the instructional goals
employed in undergraduate courses and the classroom climate, which could have
an influence on the academic and developmental motivation of learners
(Lerdpornkulrat, Koul & Poondej, 2018). Instructional strategies are not sufficient
to drive motivation (Wang & Lin, 2018) since motivation is individual-driven.
Therefore, personal factors like self-efficacy could be a potential factor in
improving students’ motivation through the classroom environment.
Among various school factors, preceding studies tend to investigate the well-
known effects of teachers on students’ academic motivation (Maulana, Helms-
Lorenz, Irnidayanti & van de Grift, 2016; Saggaf, Nasriyah, Salam & Wirawan,
2018); however, limited information is offered to explain the effects of classroom
climate on student academic motivation and how the effect happens in higher
education setting in China. In a systematic review of literature on the effects of
classroom climate, Wang, Degol, Amemiya, Parr and Guo (2020) established the
abundance of studies relating a positive learning environment with psychological
well-being of students; however, it was noted that a great deal of these studies
were centered on young learners as they cited the need to further examine this
relationship in the higher education setting particularly in Asian countries. In
response to this, an analysis of the relationship between classroom climate,
academic self-efficacy, and students’ motivation could establish the need to build
and sustain a positive learning environment in higher education institutions in
China.
Recent research conducted by Huang and Bai (2017) has found the positive
influence of classroom climate on undergraduates’ academic motivation and the

196
mediating effect that self-efficacy exerts in their relationship. However, Huang

and Bai (2017) did not take into account other factors regarding classroom climate
except for classmate relation and teacher support, nor did they test the reliability
and validity of the research instrument, which makes the findings questionable.
Thus, this study presents a specific and validated illustration of the relationship
among classroom climate, student academic motivation, and self-efficacy in the
context of undergraduate medical students in China. On the strength of the above
mentioned theoretical and empirical research approaches, three research
questions are formulated for the present study:
1. How are perceived classroom climate and academic self-efficacy related to
student academic motivation?
2. Which dimension of perceived classroom climate significantly predicts
student academic motivation and academic self-efficacy?
3. Does academic self-efficacy have a mediating effect on the relationship
between classroom climate and student academic motivation?
Self-determination Theory
This study takes Self-determination Theory (SDT) as an approach to understand
student motivation in the classroom context. SDT presents a remarkable
framework based on several motivation theories to understand the reciprocity of
sociocultural conditions and individual personality (Deci & Ryan, 2008). Besides
the conceptualization of motivation from external to internal, SDT also
emphasizes the concept of basic psychological needs. SDT theorists believe that
three internal psychological needs in human nature need to be met through
interaction with the environment: autonomy or the desire to establish inner
coherence and to feel self-directed; competence or the needs to feel confident in
one's interactions with the social environment through exercising and expressing
one's abilities; relatedness or the desire to feel integral and connected with others
(Deci & Ryan, 1985; Ryan, Rigby & Przybylski, 2006). Accordingly, the
environment that satisfies or fulfils students’ basic needs is predicted to support
their self-motivation, engagement, and well-being. In organizations, the
significance of motivation is determined by the extent it influences professional
satisfaction, task performance, and holistic wellness (Deci, Olafsen & Ryan, 2017).
That is to say, the extent to which students’ psychological needs are met by the
perceived classroom climate is expected to be associated with their motivation to
learn.

197
Figure 1: The Hierarchical Model for Motivation (source: Vallerand, 1997)
In order to differentiate short-term motivation from long-term motivation, SDT

maps the Hierarchical Model for Motivation to categorize motivation into three
levels: global, contextual, and situational (Guay, Mageau & Vallerand, 2003;
Vallerand, 1997; Vallerand, 2000) as shown in Figure 1. The global level is highly
abstract since the motivational orientations of an individual are due to personal
identity. The contextual level is less abstract as it describes the perceived
relationship of an individual to other concrete materials or organizations like
schools. The situational level, which is the focus of this study, is more specific
compared to global and contextual levels because it depicts an individual’s
motivational response to a specific activity. In the educational setting, students’
general feelings towards school and learning affect their approaches to the
learning environment and tasks. Conversely, classroom learning activities can
affect the students’ approaches to learning in general (Stolk, Jacobs, Girard &
Pudvan, 2018).
Academic Motivation
Conceptualized in the context of SDT, academic motivation is the driving force
that empowers students to learn or participate in the learning process (Ryan &
Deci, 2002). Academic motivation is usually divided into intrinsic motivation and
extrinsic motivation (Ryan & Deci, 2000). Intrinsic motivation has been
conceptualized as the internal driving force of an individual to participate in a
particular activity for his curiosity, interest, or fulfilment (Deci, Cascio & Krusell,
1975). It is linked with other components like exploration, learning goals, and
intrinsic intellectuality (Vallerand, 1992). In contrast, extrinsic motivation is an
inducement for a person to participate in a particular activity so as to gain rewards
or to avoid punishment (Dev, 1997).

198
Academic motivation is recognized as one of the most effective driving forces of

student learning (Tucker, Zayco & Herman, 2002) as well as the only factor that
guarantees students’ academic achievement (Griffin, MacKewn, Moser &
VanVuren, 2012). Among the numerous factors contributing to students’
performance and academic achievement, academic motivation is considered as
one of the most significant elements (Tucker, Zayco & Herman, 2002).
Subsequently, it helps to enhance students’ engagement (Pavlou, 2006), learning
autonomy, and academic performance (Rana, Mahmood & Reid, 2015) in teaching
and learning processes. More importantly, intrinsic academic motivation also has
a positive significance on the reduction of stress and depression of undergraduate
students, which could bring about a positive classroom atmosphere free from
interpersonal conflicts (Huang, Lv & Wu, 2016).
Classroom Climate
Classroom climate is conceptualized as the intellectual, physical, emotional, and
social environment in which students learn (Ambrose, Bridges, DiPietro, Lovett
& Norman, 2010). It includes the physical environment of the classroom, the
interaction between teacher and students, and teacher’s behavioral and academic
expectations of the students (O’Brennan, Bradshaw &Furlong, 2014). Wang et al.
(2020) characterize a positive classroom setting as a safe, harmonious, and open
environment, which could be influenced by the dynamics of the participants and
the prevalent learning culture.
Empirical evidence suggests that a self-governed and healthy environment in the

classroom positively contributed to students learning outcomes (Barksdale, 2017;
Reyes, Brackett, Rivers, White & Salovey, 2012), and academic motivation
(Anderson, Hamilton & Hattie, 2004). A classroom with a teacher’s coaching and
inspiration will lead to better coaching and encouragement among students, as
well as motivation enhancement and prolonged student engagement
(Seritanondh, 2013). Students in a teacher-supported and teacher-involved
classroom indicate a higher level of motivation as students gain more enjoyment
in the learning process and have more potential to achieve academic success;
whereas, students in a classroom of teacher-control and peer-competition are
more likely to face challenges like learning-weariness and anti-school feelings (Fry
& Coe, 1980). According to previous studies, a classroom climate of involvement,
autonomy, and meaningfulness advances students’ motivational goal orientation
(Ciani, Middleton, Summer & Sheldon, 2010; Lerdpornkulrat, Koul & Poondej,
2018). A classroom with good interpersonal relationship promotes student
intrinsic motivation (Levy-Tossman, Kaplan & Assor, 2007; Nelson & DeBacker,
2008). Conversely, a negative classroom climate with poor peer relationships
deters the growth of students’ autonomous motivation in mathematics (Reindl,
Berner, Scheunpflug, Zeinz & Dresel, 2015).
Additionally, in an agreeable and pleasant classroom climate, students feel safer,

have more confidence in learning, and are not afraid of making mistakes (Morin,
Marsh, Nagengast & Scalas, 2014). A classroom climate of order significantly leads
to the increase of reading motivation of efficacy, compliance, challenge, and
aesthetics, and the whole process contributes to greater student academic

199
achievement (Mucherah, Finch, Smith & Ambrose-Stahl, 2014). Specifically, a

teacher-centered classroom climate facilitates undergraduates’ self-efficacy in
mathematics learning (Peters, 2013).
Academic Self-efficacy
Academic self-efficacy is conceptualized as one’s confidence in his or her ability
to complete learning tasks or achieve educational goals (Bandura, 2007). It is
found to be multi-faceted and could impact how individuals feel, think, and carry
out instructive assignments (Sharma & Nasa, 2014). Self-efficacy is developed on
four bases of information, which are vicarious experiences, enactive mastery,
physical and emotional states (Schunk & Pajares, 2002). As such, academic self-
efficacy could be pertinent in student educational performances as it comes along
with corresponding behaviors and motivations that empower or debilitate
compelling execution.
Scholars have posited that positive teaching and learning outcomes rely on
affective and psychological factors like self-efficacy (Bandura, 1977). Students,
who believe that they have adequate capacity to complete tasks, tend to take
specific actions to achieve their learning goals, which will bring about positive
academic outcomes and success (Bandura, 1977). Additionally, Veresova (2016)
argued that student academic self-efficacy was resolutely associated with their
academic motivation and contributed to their learning performance as well as
academic achievement. Other empirical researches also confirmed the positive
effects of student self-efficacy on their academic motivation (Chi & Xin, 2006;
Taheri-Kharameh, Sharififard, Asayesh, Sepahvandi & Hoseini, 2018). These
findings are reinforced by recent experimental research demonstrating that the
training of self-efficacy-building strategies significantly contributes to the
improvement of students’ academic self-efficacy and motivation. In addition, the
results also confirm the causal-effect relationship between academic self-efficacy
and student motivation (Cave, Evans, Dewey & Hartshorn, 2018).
3. Method
Research Design
A quantitative design and survey strategy were used to meet the research
objectives. Data were drawn from 119 undergraduate students from a medical
college located in Guangxi, China. Among the sample students, there were 25
males (21%) and 91 females (76.5%). The average age of the participants was 21
(SD = 1.4). The online questionnaire was distributed to undergraduate medical
students via QQ-email in different sessions. In PLS-SEM, the needed sample size
could be calculated by means of power analyses according to the part of the model
obtaining the largest number of predictors (Hair, Hult, Ringle & Sarstedt, 2017).
Hair, et al. (2017) further suggested researchers use Programs such as G*Power to
do power analysis specific to model set-ups when utilizing PLS-SEM. G*power is
a free online software used to do power analysis and sample size calculation. The
analysis result of G*Power shows that the required sample size of this study is 92
(effect size=.15, alpha= .05, beta=.80, No. of predictor=5), indicating that 119
samples are adequate for data analysis and the sample size does not have an effect
on the result.

200
Instruments
The current study used a questionnaire adopted from several scales. These scales
consisted of a total of 43 items with a numerical rating scale ranging from 1 (not
agree at all) to 9 (highest agreement). As a first step, the instrument was sent to
three experts to validate the content. Then, a panel of bilingual experts was invited
to validate the translation via a double back translation technique (Zikmund,
Babin, Carr & Griffin, 2010).
Classroom Climate
Classroom climate was assessed with Classroom Climate Questionnaire (CCQ).
The CCQ is a 26-item questionnaire developed by Hu (2010) to assess
undergraduate students’ perception of classroom climate in a college course.
Classroom climate questionnaire includes five dimensions: a) cohesive, b)
supportive, c) participative, d) planned, and e) fair. Cohesive dimension mainly
reflects the extent to which students learn from each other, help each other, and
support each other in the classroom. Supportive dimension focuses on the degree
by which teachers are friendly, trustful, interested, and helpful to students in the
classroom. Participative dimension reflects the degree of students’ concentration,
interest, participation, and enjoyment in classroom discussion and learning.
Planned dimension refers to the extent by which students complete classroom
activities, goals, and tasks. Fair dimension refers to the degree by which students
are treated fairly by teachers. Reliabilities of the five dimensions are .904
(cohesive, n=6 item), .921 (supportive, n=6 item), .911 (participative, n=4 item),
.876 (planned, n=5 item), and .929 (fair, n=5 item). The results of confirmative
factor analysis reveal that the indicator loading of each item is higher than .70, and
AVE value of each dimension is in the range of .67 -.80. In addition, the HTMT
value of each dimension is smaller than .85.
Academic Self-efficacy
Academic self-efficacy was measured by a sub-scale titled Self-Efficacy for
Learning and Performance of Motivated Strategies for Learning Questionnaire
(MSLQ) developed by Pintrich, Smith, Garcia, and McKeachie (1991). The sub-
scale was utilized to measure the level of students’ perceived ability and
confidence in a course. MSLQ was designed on the basis of a general cognitive
perspectives on learning motivation and strategies and a theoretical framework
proposed by McKeachie, Pintrich, Lin, and Smith (1986). This instrument has been
widely used for many years and has been tested as reliable and valid in
contemporary researches (Basila, 2016; Bibi, 2017; Rush, 2013; Taylor, 2012,
Vaculíková, 2016). The reliability test indicates a high reliability of this scale. The
reliability of the self-efficacy is .936 (n=8 item). The result of confirmative factor
analysis reveals that the indicator loading of each item is higher than .70, AVE
value is .71, HTMT value is smaller than .85.
Student Academic Motivation

Student academic motivation was measured by a sub-scale (i.e. goal orientation)
of MSLQ. Goal orientation regards students’ goals in a subject or course as a
whole, i.e. academic motivation. Intrinsic goal orientation assessed students’ self-
perceived degrees to participate in learning tasks out of reasons like interest,
curiosity, and challenge. Extrinsic goal orientation regards students’ self-

201
perceived degrees to participate in learning tasks out of reasons, for example

grades, rewards, competing with others, or evaluation by others. Therefore, this
study adopts intrinsic goal orientation to evaluate intrinsic motivation and
extrinsic goal orientation to assess extrinsic motivation. Reliabilities of the two
dimensions are .842 (intrinsic, n=4 item), and .866 (extrinsic, n=4 item). The result
of confirmative factor analysis shows that the indicator loading of each item was
higher than .70, and AVE value of each dimension is .68 and .72 respectively.
Additionally, the HTMT value of each dimension is smaller than .85.
4. Results
Table 1 summarizes the descriptive statistics of classroom climate (CC), self-
efficacy (SE), academic motivation (AM), and the sub-constructs of classroom
climate and academic motivation. The means of all constructs were higher than
6.34, indicating that students showed a generally high level of perceived academic
self-efficacy, academic motivation, and classroom climate.
Table 1: Mean, SD, Skewness and Kurtosis of Constructs

Construct Mean SD Skewness Kurtosis
Cohesive 7.49 .96 -.54 -.08
Supportive 6.42 1.29 -.54 .47
Participative 6.64 1.29 -.61 .74
Planned 7.31 .96 -.37 -.18
Fair 7.23 1.18 -.62 .13
Intrinsic 7.34 .99 -.64 .18
Extrinsic 7.52 1.08 -.66 .18
SE 7.04 1.09 -.39 -.21
AM 7.43 .94 -.67 .17
CC 7.02 .96 -.34 -.48
With regard to the first research question, Pearson correlation was employed to
analyze the relationship between perceived classroom climate, academic self-
efficacy, and academic motivation using SPSS 24. Inter-correlations among the ten
constructs are shown in Table 2. The result revealed strong correlation between
classroom climate and academic self-efficacy (r=.75, p < .001), classroom climate
and academic motivation (r = .66, p < .001), and students’ self-efficacy and their
academic motivation (r=.75, p < .001). Unexpectedly, all the constructs were also
strongly associated.
Table 2: Inter-correlations among Constructs

Construct 1 2 3 4 5 6 7 8 9
1. CC 1
2. AM .66*** 1
3.SE .75*** .75*** 1
4.Cohesive .79*** .63*** .60*** 1
5.Supportive .86*** .41*** .53*** .57*** 1
6.Participative .89*** .48*** .61*** .62*** .81*** 1
7.Planned .83*** .72*** .75*** .65*** .57*** .63*** 1
8.Fair .83*** .60*** .69*** .59*** .61*** .60*** .73*** 1

202
9.Intrinsic .64*** .90*** .74*** .61*** .40*** .47*** .70*** .59*** 1

10. Extrinsic .55*** .92*** .62*** .53*** .35*** .39*** .62*** .49*** .65***
Note. *** Correlation is significant at the .001 level (2-tailed)
In order to examine which dimension of perceived classroom climate significantly

predicts student academic motivation and academic self-efficacy, stepwise
multiple regression technique was employed to explore the causal effect among
the constructs. Table 3 illustrates the output of multiple regression analysis of the
dimension of classroom climate on academic motivation and self-efficacy.
Table 3: Multiple Regression of Classroom Climate on Academic Motivation and Self-

efficacy
Dimension of
b β t Sig. R² %
Classroom Climate
Planned-> AM .532 .546 6.526 .000 .518 51.8
Cohesive-> AM .265 .271 3.244 .002 .557 3.9
Planned-> SE .602 .531 6.044 .000 .567 56.7
Fair-> SE .281 .304 3.462 .001 .603 3.8
The above result specified that among the five dimensions of classroom climate,
two dimensions significantly correlate and contribute (55.7%) to student academic
motivation. The dominant planned dimension (β=.546, p=.000) was found to
contribute 51.8% of the variance to student academic motivation, and the cohesive
dimension (β=.271, p=.002) contribute 3.9% of the variance to academic
motivation. Accordingly, when the planned dimension and cohesive dimension
of the classroom climate increase by one unit of standard deviation, student
academic motivation will increase by .546 and .271 unit of standard deviation,
respectively. Hence, the planned dimension and cohesive dimension have a
strong and modest effect on student academic motivation (Muijs, 2011),
respectively.
In regard to the significant predictors of classroom climate to student academic

self-efficacy, the dimension of planned (β=.531, p=.000) and fair (β=.304, p=.001)
were concluded in the prediction model. The planned dimension was found to
contribute 56.7% of the variance to academic self-efficacy (R²=.567), and the
combination of planned and fair dimensions accounted for 60.3% of the student
self-efficacy (R²=.603). Based on the interpretation showed in Table 4, the two
models demonstrated a large effect size on academic self-efficacy as a whole.
Additionally, the dimension of planned and fair had a strong and modest effect
on self-efficacy, respectively.
Subsequently, software SmartPLS 3 was applied to examine the indirect influence

that academic self-efficacy exerted on the relationship between classroom climate
and student academic motivation. Table 4 covers the relationship between
variables in the model, and Figure 2 presents a structural illustration of the
relationships. In Figure 2, the inner model depicts the path coefficients (β) and T-
values, and the outer model shows the T-values.

203
Table 4: Summary of the Direct and Indirect Effect in the Model

Path β t Sig.
CC -> AM .242 2.454 .014
CC -> SE .762 16.409 .000
SE -> AM .576 6.356 .000
CC -> SE -> AM 5.688 .000
Figure 2: The Structural Model Assessment
According to the output shown in Figure 2 and table 4, the direct effect of
classroom climate on student academic motivation is estimated at β =.242, and the
indirect effect which is the multiplication of both indirect paths (.762 × .576) equals
to .439, leaving the standardized total effect (indirect + direct) at .681 (Hair et al,
2010). The direct path of classroom climate and academic motivation is supported;
meanwhile, the two indirect paths (classroom climate -- academic self-efficacy --
academic motivation) are also confirmed (see Table 4). The result reveals that
academic self-efficacy partially mediates the relationship between classroom
climate and student academic motivation since both direct and indirect effects are
established.
5. Discussion
Results obtained from data analysis have a number of implications; however, they
should be interpreted with SDT and previous related findings.
First, the classroom climate significantly and positively affects student academic
motivation. The findings of this study affirm the research results of Huang and
Bai (2017) and Lerdpornkulrat (2018), which indicated that a healthy and positive
classroom climate was pivotal for the improvement of student academic
motivation. Specifically, a cohesive classroom climate, in which students learn
and support each other, significantly improves academic motivation. This is

204
echoed by the findings of Anderson et al. (2004) noting that when secondary
students felt a higher level of friendship in a classroom, they were more motivated
to learn. More importantly, a planned classroom, in which students complete
classroom tasks in high quality, greatly contributes to the improvement of student
academic motivation (R²=51.8%).
Nevertheless, this study also found that a supportive classroom climate was
closely correlated with (p<.001) but insignificantly influences student academic
motivation. This is aligned with the previous findings (Fry & Coe, 1980) that
teacher support was indispensable for the enhancement of student academic
motivation; however, this is inconsistent with the findings of Huang and Bai
(2017) stating that teacher support significantly and positively influenced student
academic motivation. A probable reason for this inconsistency could have been
the participant characteristics. As adult learners, undergraduates are less affected
by teachers compared with primary and secondary students. According to the
latest research, medical students indicated a higher level of autonomy in learning
(Su, Ye, Li, Wei, Du & Lu, 2014). They have heavy learning tasks and devote a
long time in learning so as to meet their higher academic requirements, making
them live in a comparatively strong and competitive learning atmosphere. This
atmosphere consequences further facilitates the development of their autonomy
learning.
Second, the influence of classroom climate on student academic motivation is

mediated by academic self-efficacy. The result is consistent with SDT and the
findings reported in the research conducted in mainland China context (Huang &
Bai, 2017). Students, who believe that their psychological needs are met in the
social environment or who study in a positive and healthy classroom climate,
indicate a higher level of confidence in their ability and academic motivation.
Third, a planned and fair classroom contributed to the improvement of self-

efficacy. This is echoed by the research findings of Mucherah et al. (2014), which
shows that a well-planned classroom climate helps to increase students’ reading
motivation. When the students complete classroom learning tasks and are treated
equally, it helps them to build their self-confidence to achieve their learning goals
and then improves their motivation to pursue personal development.
Fourth, the descriptive result recommends that the respondents of this study are
highly motivated to learn. This is contradictory to the findings of Gao & Gao
(2015), Liu (2013), and Qiu (2013) which indicate the insufficiency of academic
motivation among college students. A conceivable explanation for this might be
the better academic background of medical students compared with other
disciplined students. In China, high school students need to get a higher score in
the national college entrance examination so as to get the chance to be medical
students.
Finally, the planned dimension of classroom climate has a very robust impact on
student motivation (β=.546) and self-efficacy (β=.531), indicating that among the
three psychological needs, competence is the most important for undergraduate

205
medical students in China. Overall, this finding is supported by the assumptions

of SDT.
6. Conclusion
After analyzing the data collected from 119 medical students from a higher
education institute, the results concluded the significant influence that classroom
climate exerted on student academic motivation and the mediating effect of
academic self-efficacy in the relationship. This study also indicated that among
the five dimensions of classroom climate, a planned and fair classroom climate
significantly improved student self-efficacy and a cohesive and planned
classroom climate helps to motivate students to learn. This stresses the essence of
a healthy and well-organized classroom climate in student learning. For
policymakers and stakeholders, more related educational policies regarding
classroom climate should be made, and they can also set classroom climate
evaluation as a criterion to assess education practitioners. Standards regarding the
management of positive classroom climates are heavily valued in America
(Schonert-Reichl et al, 2017); whereas, related standards and policies in China
cannot be found in the present literature. For school administrators, they need to
safeguard teachers’ actions within the classroom, which are consistent with
research-based factors advised by this study and previous literature to promote
student academic motivation and performance. Also, school administrators can
launch programs that can effectively facilitate the classroom climate to improve
student academic motivation. For lecturers, they can obtain the message that the
promotion of a positive classroom climate is highly crucial and beneficial for
students’ academic performance. They can also build a classroom environment
with a harmonious student relationship, treat the students in an equivalent way,
and offer them more prospects to complete learning tasks. Despite the
highlighted significant contributions of this study, further research could give a
generalizable perspective by covering a larger sample size and population from
different colleges and universities. In addition, exploratory studies could further
explain the ways classroom environment are designed to further motivate
students and improve their self-efficacy. Nevertheless, the findings of this study
accentuate the notion that more meaning should be embedded in the design of
learning tasks to meet students’ needs of competence and to build their confidence
and motivation.
7. References
Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K. (2010). How
learning works: Seven research-based principles for smart teaching. CA: John Wiley &
Sons.
Anderson, A., Hamilton, R. J., & Hattie, J. (2004). Classroom climate and motivated
behaviour in secondary schools. Learning Environments Research, 7(3), 211-225.
doi:10.1007/s10984-004-3292-9
Bailey, T. H., & Phillips, L. J. (2016). The influence of motivation and adaptation on
students’ subjective well-being, meaning in life and academic
performance. Higher education research & development, 35(2), 201-216.
doi:10.1080/07294360.2015.1087474

206
Bandura, A. (2007). Much ado over a faulty conception of perceived self-efficacy grounded
in faulty experimentation. Journal of Social and Clinical Psychology, 26, 641–658.
doi:10.1521/jscp.2007.26.6.641
Barksdale, C. J. (2017). Examining the Relationship Between Classroom Climate and Student
Achievement of Middle School Students (Ed.D. Thesis). University of Houston-Clear
Lake, Ann Arbor.
Basila, C. L. (2016). Academic performance in college online courses: The role of self-regulated
learning, motivation and academic self-efficacy (Ph.D. Thesis). State University of New
York at Albany, Ann Arbor.
Bibi, A. (2017). Factors affecting Differential Equation Problem Solving Ability of Pre-university
Level Students In a Selected Province in Pakistan (Unpublished Doctroral
Dissertation). Kuala Lumpur, Malaysia: University of Malaya.
Cave, P. N., Evans, N. W., Dewey, D. P., & Hartshorn, K. J. (2018). Motivational
partnerships: increasing ESL student self-efficacy. Elt Journal, 72(1), 83-96.
doi:10.1093/elt/ccx027
Chi, L. P., & Xin, Z. Q. (2004). The Measurement of Learning Motivation and its
Relationship with Self-Efficacy among College Students. Psychological Development
and Education, 21(2), 64-70.
Ciani, K. D., Middleton, M. J., Summers, J. J., & Sheldon, K. M. (2010). Buffering against
performance classroom goal structures: The importance of autonomy support and
classroom community. Contemporary educational psychology, 35(1), 88-99.
doi:10.1016/j.cedpsych.2009.11.001
Cleary, T. J., & Kitsantas, A. (2017). Motivation and self-regulated learning influences on
middle school mathematics achievement. School Psychology Review, 46(1), 88-107.
doi:10.17105/spr46-1.88-107
Covington, M. V. (1992). Making the grade: A self-worth perspective on motivation and school
reform. Cambridge University Press.
Deci, E. L., Cascio, W. F., & Krusell, J. (1975). Cognitive evaluation theory and some
comments on the Calder and Staw critique. Journal of personality and social
psychology, 31(1), 81-85. doi:10.1037/h0076168
Deci, E. L., & Ryan, R. M. (1985). The general causality orientations scale: Self-
determination in personality. Journal of research in personality, 19(2), 109-134.
doi:10.1016/0092-6566(85)90023-6
Deci, E. L., & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs
and the self-determination of behavior. Psychological Inquiry, 11(4), 227-268.
doi:10.1207/s15327965pli1104_01
Deci, E. L., Olafsen, A. H., & Ryan, R. M. (2017). Self-determination theory in work
organizations: The state of a science. Annual Review of Organizational Psychology
and Organizational Behavior, 4, 19-43.
Dev, P. C. (1997). Intrinsic motivation and academic achievement: What does their
relationship imply for the classroom teacher?. Remedial and special education, 18(1),
12-19. doi:10.1177/074193259701800104
Froiland, J. M., & Worrell, F. C. (2016). Intrinsic motivation, learning goals, engagement,
and achievement in a diverse high school. Psychology in the Schools, 53(3), 321-336.
doi:10.1002/pits.21901
Fry, P., & Coe, K. (1980). Interaction among dimensions of academic motivation and
classroom social climate: A study of the perceptions of junior high and high school
pupils. British Journal of Educational Psychology, 50(1), 33-42. doi:10.1111/j.2044-
8279.1980.tb00795.x

207
Gao, X. L., & Gao, L., (2015) The characteristics and grade differences in learning
motivation, learning strategies, learning investment among undergraduate
students. China Journal of Health Psychology, 23 (2), 274-277.
Gold, A. H., Malhotra, A., & Segars, A. H. (2001). Knowledge management: an
organizational capabilities perspective. Journal of Management Information Systems,
18(1), 185–214. doi:10.1080/07421222.2001.11045669
Gou, Y. K. (2016). The Relationship between College Students’ Social Support, Achievement
Motivation, and Mobile Phone Dependence (Master’s thesis). Shandong Normal
University, Jinan.
Griffin, R., MacKewn, A., Moser, E., & VanVuren, K. (2012). Do learning and study skills
affect academic performance? An empirical investigation. Contemporary Issues in
Education Research, 5(2), 1-7. doi:10.19030/cier.v5i2.6928
Guay, F., Mageau, G. A., & Vallerand, R. J. (2003). On the Hierarchical Structure of Self-
Determined Motivation: A Test of Top-Down, Bottom-Up, Reciprocal, and
Horizontal Effects. Personality and Social Psychology Bulletin, 29(8), 992–1004.
doi:10.1177/0146167203253297
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010). Multivariate
Data Analysis: Pearson education. Upper Saddle River, New Jersey: Pearson
Prentice Hall.
Hair Jr, J. F., Sarstedt, M., Hopkins, L., & G. Kuppelwieser, V. (2014). Partial least squares
structural equation modeling (PLS-SEM) An emerging tool in business research.
European Business Review, 26(2), 106-121. doi:10.1108/ebr-10-2013-0128
Hair Jr, J. F., Sarstedt, M., Ringle, C. M., & Gudergan, S. P. (2017). Advanced issues in partial
least squares structural equation modeling. Sage Publications.
Hair Jr, J. F., Hult, G. T. M., Ringle, C., & Sarstedt, M. (2017). A primer on partial least squares
structural equation modeling (PLS-SEM) (Second ed.). Los Angeles: Sage
publications.
Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant
validity in variance-based structural equation modeling. Journal of the Academy of
Marketing Science, 43(1), 115-135. doi:10.1007/s11747-014-0403-8
Hu, Z. X. (2010). Research on the Assessment Model of Class Climate in higher Education
Institutes. Heilongjiang Researches on Higher Education. 28(2), 46-49.
Huang, C., & Bai, Q.-h. (2017). The Relationship Between Classroom Climate, Self-Efficacy,
and Learning Motivation. DEStech Transactions on Social Science, Education and
Human Science, 308-312. doi:10.12783/dtssehs/ermm2017/14731
Huang, Y., Lv, W., & Wu, J. (2016). Relationship between intrinsic motivation and
undergraduate students’ depression and stress: the moderating effect of
interpersonal conflict. Psychological reports, 119(2), 527-538.
Khalaila, R. (2015). The relationship between academic self-concept, intrinsic motivation,
test anxiety, and academic achievement among nursing students: Mediating and
moderating effects. Nurse Education Today, 35(3), 432-438.
doi:10.1016/j.nedt.2014.11.001
Kline, R. B. (2011). Principles and practice of structural equation modeling. New York:
Guilford Press.
Ladd, H. F., & Sorensen, L. C. (2017). Returns to teacher experience: Student achievement
and motivation in middle school. Education Finance and Policy, 12(2), 241-279.
doi:10.1162/edfp_a_00194
Lerdpornkulrat, T., Koul, R., & Poondej, C. (2018). Relationship between perceptions of
classroom climate and institutional goal structures and student motivation,
engagement and intention to persist in college. Journal of Further and Higher
Education, 42(1), 102-115. doi:10.1080/0309877x.2016.1206855

208
Levy-Tossman, I., Kaplan, A., & Assor, A. (2007). Academic goal orientations, multiple
goal profiles, and friendship intimacy among early adolescents. Contemporary
Educational Psychology, 32(2), 231-252. doi:10.1016/j.cedpsych.2006.06.001
Liu, W. (2013). The current situation of learning motivation and learning strategies among
independent college students (Master’s thesis). Liaoning Normal University,
Liaoning.
Maulana, R., Helms-Lorenz, M., Irnidayanti, Y., & van de Grift, W. (2016). Autonomous
motivation in the Indonesian classroom: Relationship with teacher support
through the lens of self-determination theory. The Asia-Pacific Education
Researcher, 25(3), 441-451. doi:10.1007/s40299-016-0282-5
Mega, C., Ronconi, L., & De Beni, R. (2014). What makes a good student? How emotions,
self-regulated learning, and motivation contribute to academic
achievement. Journal of educational psychology, 106(1), 121. doi:10.1037/a0033546
McKeachie, W. J., Pintrich, P. R., Lin, Y. G., & Smith, D. (1986). Teaching and learning in the
college classroom: A review of the research literature. Ann Arbor, MI: National Center
for Research to Improve Postsecondary Teaching and Learning, The University of
Michigan.
Morin, A. J. S., Marsh, H. W., Nagengast, B., & Scalas, L. F. (2014). Doubly Latent
Multilevel Analyses of Classroom Climate: An Illustration. Journal of Experimental
Education, 82(2), 143-167. doi:10.1080/00220973.2013.769412
Mucherah, W., Finch, H., Smith, V., & Ambrose-Stahl, D. (2014). Exploring the relationship
between classroom climate, reading motivation, and achievement: A look into 7th
grade classrooms. International Journal of Learning, Teaching and Educational
Research, 8(1), 93-110.
Muijs, D. (2011). Doing Quantitative Research in Education with SPSS (Second ed.). Thousand
Oaks, CA: SAGE Publications Inc.
Nelson, R. M., & DeBacker, T. K. (2008). Achievement motivation in adolescents: The role
of peer climate and best friends. The journal of experimental education, 76(2), 170-
189. doi:10.3200/jexe.76.2.170-190
O'Brennan, L. M., Bradshaw, C. P., & Furlong, M. J. (2014). Influence of Classroom and
School Climate on Teacher Perceptions of Student Problem Behavior. School
Mental Health, 6(2), 125-136. doi:10.1007/s12310-014-9118-8
Pavlou, V. (2006). Pre‐adolescents' perceptions of competence, motivation and
engagement in art activities. International Journal of Art & Design Education, 25(2),
194-204. doi:10.1111/j.1476-8070.2006.00484.x
Pelletier, L. G., Fortier, M. S., Vallerand, R. J., & Brière, N. M. (2001). Associations Among
Perceived Autonomy Support, Forms of Self-Regulation, and Persistence: A
Prospective Study. Motivation and Emotion, 25(4), 279-306.
Pintrich, P. R., Smith, D. A., García, T., & McKeachie, W. J. (1991). A manual for the use of
the motivational strategies for learning questionnaire (MSLQ). Ann Arbor, MI:
University of Michigan, National Center for Research to Improve Postsecondary Teaching
and Learning.
Qiu. K. R. (2013). An Empirical Study of College Students' Learning Motivation——Based on
the Survey of 6 Universities (Master’s thesis). Shantou University, Shantou.
Rana, R. A., Mahmood, N., & Reid, N. (2015). Motivation and science performance:
influence on student learning in science. Science Institute (Lahore), 27(2), 1445-1452.
Ratelle, C. F., Guay, F., Vallerand, R. J., Larose, S., & Senécal, C. (2007). Autonomous,
controlled, and amotivated types of academic motivation: A person-oriented
analysis. Journal of Educational Psychology, 99(4), 734–746. doi:10.1037/0022-
0663.99.4.734

209
Reindl, M., Berner, V. D., Scheunpflug, A., Zeinz, H., & Dresel, M. (2015). Effect of negative
peer climate on the development of autonomous motivation in mathematics.
Learning and Individual Differences, 38, 68-75. doi:10.1016/j.lindif.2015.01.017
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions
and new directions. Contemporary educational psychology, 25(1), 54-67.
doi:10.1006/ceps.1999.1020
Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006). The motivational pull of video games:
A self-determination theory approach. Motivation and emotion, 30(4), 344-360.
doi:10.1007/s11031-006-9051-8
Rush, K. (2013). A Quantitative Evaluation of Gender, Nationality, and Generational/Age
Influence on Academic Motivation. (Ph.D.). Northcentral University, Ann Arbor.
Saggaf, M. S., Nasriyah, N., Salam, R., & Wirawan, H. (2018). The Influence of Teacher's
Pedagogic Competence on Learning Motivation of Student of Office Administration
Expertise Package. Paper presented at the 8th International Conference of Asian
Association of Indigenous and Cultural Psychology (ICAAIP 2017). doi:
10.2991/icaaip-17.2018.24
Sarstedt, M., Ringle, C. M., Smith, D., Reams, R., & Hair Jr, J. F. (2014). Partial least squares
structural equation modeling (PLS-SEM): A useful tool for family business
researchers. Journal of Family Business Strategy, 5(1), 105-115.
doi:10.1016/j.jfbs.2014.01.002
Schonert-Reichl, K. A., Kitil, M. J., & Hanson-Peterson, J. (2017). To reach the students,
teach the teachers: A national scan of teacher preparation and social and
emotional learning. A report prepared for the CASEL. Collaborative for Academic,
Social, and Emotional Learning (CASEL). Vancouver, B.C.: University of British
Columbia.
Schunk, D. H., & Pajares, F. (2002). The development of academic self-efficacy.
In Development of achievement motivation (pp. 15-31). Academic Press.
Seritanondh, S. (2013). Teacher leadership styles and student psychological characteristics
affecting the study methods of foundation English courses in higher education: A
case study of education and humanity/liberal arts students in Thailand. The
Journal of Behavioral Science, 8(1), 17-36.
Sharma, H. L., & Nasa, G. (2014). Academic self-efficacy: A reliable predictor of
educational performances. British Journal of Education, 2(3), 57-64.
Stolk, J. D., Jacobs, J., Girard, C., & Pudvan, L. (2018). Learners’ Needs Satisfaction,
Classroom Climate, and Situational Motivations: Evaluating Self-Determination
Theory in an Engineering Context. 2018 IEEE Frontiers in Education Conference
(FIE), 1-5. doi:10.1109/fie.2018.8658880
Su. Z. P., Ye, P., Li, J. P., Wei, T. Du, Z. M., & Lu, Z. J. (2014). Analysis on the Differences
of Autonomous Learning Ability of Medical Undergraduates. CHINA MEDICAL
EDUCATION TECHNOLOGY, 28(4): 444-451.
Supervia, P. U., & Bordas, C. S. (2018). School Motivation, Emotional Intelligence and
Academic Performance in Students of Secondary Education. Actualidades En
Psicologia, 32(125), 95-112.
Taheri-Kharameh, Z., Sharififard, F., Asayesh, H., Sepahvandi, M., & Hoseini, M. H.
(2018). Relationship between Academic Self-efficacy and Motivation among
Medical Science Students. Journal of Clinical and Diagnostic Research, 12(7), 7-10.
doi: 10.7860/JCDR/2018/29482.11770
Taylor, R. (2012). Review of the motivated strategies for learning questionnaire (MSLQ) using
reliability generalization techniques to assess scale reliability (Doctoral dissertation).
Auburn University, Auburn.

210
Tucker, C. M., Zayco, R. A., & Herman, K. C. (2002). Teacher and child variables as
predictors of academic engagement among low-income African American
children. Psychology in the Schools, 39(4), 477-488. doi:10.1002/pits.10038
Vaculíková, J. (2016). The Third Round of the Czech Validation of the Motivated Strategies
for Learning Questionnaire (MSLQ). International Education Studies, 9(7), 35-46.
Vallerand, R. J. (1997). Toward a hierarchical model of intrinsic and extrinsic motivation.
In Advances in experimental social psychology (Vol. 29, pp. 271-360). Academic Press.
Vallerand, R. J. (2000). Deci and Ryan's self-determination theory: A view from the
hierarchical model of intrinsic and extrinsic motivation. Psychological Inquiry,
11(4), 312–318. URL: http://www.jstor.org/stable/1449629 .
Verešová, M., & Foglová, L. (2016). Academic self–Efficacy, heteronomous and autonomous
evaluation of academic achievement of adolescents. Paper presented at the 7th
International Conference on Education and Educational Psychology (ICEEPSY
2016). URL: http://dx.doi.org/10.15405/epsbs.2016.11.91
Wang, M. T., Degol, J. L., Amemiya, J., Parr, A., & Guo, J. (2020). Classroom climate and
children’s academic and psychological wellbeing: A systematic review and meta-
analysis. Developmental Review, 57, 100912.
Wang, Q., & Lin, J. (2018). An Empirical Analysis of the Status Quo of Learning Situation
and Motivation among College Students--Taking college Students in Shanxi
University Communities as an Example. University Education, 9(12):143-146.
Zikmund, W. G., Babin, B. J., Carr, J. C., & Griffin, M. (2010). Business Research Methods (8th
ed.). Mason, OH: South-Western Ceng.
Zhao, H. J. (2015) The relationship between learning burnout and learning motivation: the
mediating role of attribution (Master’s thesis). Southwest University, Chongqing.

211
Appendix 1
Questionnaire for students

Dear students,
This survey is conducted to better understand the relationship between
classroom climate, student academic self-efficacy and learning motivation
so as to provide reference for the improvement of teaching and learning
quality.
The questionnaire only needs to be answered according to your actual
situation, there is no right or wrong answer. This survey does not require
you to provide your names, and your answers are kept confidential. We
sincerely hope to get your cooperation and support. Thank you.
Part I: Demographic Information

Please tick or fill in your relevant information in the description below
Gender： A. male B. female
School Year: A. 1 B. 2 C. 3 D. 4
Age: ________
Part II:
Please circle one of the options that you think is most suitable in the
following items in accordance with your actual situation. For example:
not agree at all 1 2 3 4 5 6 7 8 9 highest agreement
classroom climate
1=not agree at all
Item
9=highest agreement
1 I am very friendly to my classmates. 1 2 3 4 5 6 7 8 9
Many of the students in this course are
2 1 2 3 4 5 6 7 8 9
my friend
I am very happy to work with my
3 1 2 3 4 5 6 7 8 9
classmates on this course.
I will help my classmates in this course
4 1 2 3 4 5 6 7 8 9
if they have trouble.
5 The classmates in this course like me. 1 2 3 4 5 6 7 8 9
In this course, I got help from other
6 1 2 3 4 5 6 7 8 9
students.
7 The lecturer is concerned about me. 1 2 3 4 5 6 7 8 9
The lecturer took the trouble to help
8 1 2 3 4 5 6 7 8 9
me.
9 The lecturer cares about my feeling. 1 2 3 4 5 6 7 8 9
10 The lecturer is happy to talk to me. 1 2 3 4 5 6 7 8 9
The lecturer is interested in the learning
11 1 2 3 4 5 6 7 8 9
problems I have encountered.
The lecturer walked around the
12 1 2 3 4 5 6 7 8 9
classroom and talked to us.

212
I put forward my own point of view in

13 1 2 3 4 5 6 7 8 9
the discussion.
My insights and opinions are used in
14 1 2 3 4 5 6 7 8 9
class discussion.
Classmates and I discuss ways to solve
15 1 2 3 4 5 6 7 8 9
problems.
Everyone asked me to explain the
16 1 2 3 4 5 6 7 8 9
solution to the problem.
17 I follow the lesson plan to learn 1 2 3 4 5 6 7 8 9
I understand the learning objectives of
18 1 2 3 4 5 6 7 8 9
this course.
I understand the goal I have to work on
19 1 2 3 4 5 6 7 8 9
in this course.
20 I always concentrate on my class. 1 2 3 4 5 6 7 8 9
I try my best to understand the teaching
21 1 2 3 4 5 6 7 8 9
content.
The lecturer gave me as much attention
22 1 2 3 4 5 6 7 8 9
as other students.
The lecturer gave me as much help as
23 1 2 3 4 5 6 7 8 9
other students.
I have the opportunity to speak in class
24 1 2 3 4 5 6 7 8 9
as much as other students.
The lecture treats me as good as other
25 1 2 3 4 5 6 7 8 9
students.
I got the opportunity to participate in
26 the discussion in class, as much as 1 2 3 4 5 6 7 8 9
other students.
Self-efficacy
1=not agree at all
Item
9=highest agreement
1 I believe I will receive an excellent grade 1 2 3 4 5 6 7 8 9

in this class.
I'm certain I can understand the most
2 difficult material presented in the 1 2 3 4 5 6 7 8 9
readings for this course
I'm confident I can learn the basic
3 1 2 3 4 5 6 7 8 9
concepts taught in this course.
I'm confident I can understand the most
4 complex material presented by the 1 2 3 4 5 6 7 8 9
instructor in this course.
I'm confident I can do an excellent job on
5 the assignments and tests in this 1 2 3 4 5 6 7 8 9
course.

213
6 I expect to do well in this class 1 2 3 4 5 6 7 8 9

I'm certain I can master the skills being
7 1 2 3 4 5 6 7 8 9
taught in this class.
Considering the difficulty of this course,
8 the teacher, and my skills, I think I will 1 2 3 4 5 6 7 8 9
do well in
Academic motivation
1=not agree at all
Item
9=highest agreement
In a class like this, I prefer course
1 material that really challenges me so I 1 2 3 4 5 6 7 8 9
can learn new things.
In a class like this, I prefer course
2 material that arouses my curiosity, even 1 2 3 4 5 6 7 8 9
if it is difficult to learn.
The most satisfying thing for me in this
3 course is trying to understand the 1 2 3 4 5 6 7 8 9
content as thoroughly as possible
When I have the opportunity in this

class, I choose course assignments that
4 1 2 3 4 5 6 7 8 9
I can learn from even if they don't
guarantee a good grade
Getting a good grade in this class is the
5 1 2 3 4 5 6 7 8 9
most satisfying thing for me right now.
The most important thing for me right
now is improving my overall grade point
6 1 2 3 4 5 6 7 8 9
average, so my main concern in this
class is getting a good grade.
If I can, I want to get better grades in this
7 1 2 3 4 5 6 7 8 9
class than most of the other students
I want to do well in this class because it
8 is important to show my ability to my 1 2 3 4 5 6 7 8 9
family, friends, employer, or others

214

Vol. 19, No. 8, pp. 214-232, August 2020
Metacognitive Writing Strategies Used by

Omani Grade Twelve Students
Ibtisam Sultan Al Moqbali
Ministry of Education, Oman
https://orcid.org/0000-0002-3669-170X
Salma Al Humaidi*, Abdo Al Mekhlafi and Maher Abu Hilal

Sultan Qaboos University, Al Khoudh, Muscat 123, Oman
https://orcid.org/0000-0001-8579-7749
https://orcid.org/0000-0003-2821-6199
https://orcid.org/0000-0002-7026-498X
Abstract. The current study investigated the extent to which

metacognitive writing strategies are used by Omani EFL (English as a
Foreign Language) grade twelve students. It also explored if there is a
significant relationship between grade 12 students' use of metacognitive
writing strategies and their language performance. Besides, it examined
gender differences in the use of these strategies. The sample of the study
was 263 Omani EFL grade twelve students from Batinah North
Governorate in the academic year 2018/2019. Metacognitive Strategies of
Writing Survey (MSWS) and semi-structured interviews were used to
collect data. The results of the study revealed that students use the three
types of metacognitive writing strategies; planning, monitoring and
evaluating at high frequencies. However, the findings revealed no
significant relationship between students' use of the three types of
metacognitive strategies and their language performance. Moreover,
students differ in terms of gender in using metacognitive writing
strategies. Female students tend to use more metacognitive strategies in
writing compared to male students. In light of these findings,
implications for practice and future studies have been recommended.
Keywords: Metacognitive strategies; Writing; Language performance;

Oman
1. Introduction
Over the past decades, writing has received more attention from researchers due
to the growing recognition of language acquisition as well as professional
demands. Researchers explain that possessing good writing skills has a positive
impact on learners’ academic achievement (Abedin, Hossein, Naseri, and
Taghizadeh, 2013; Binandeh, Rahmani and Raoofi, 2017; Briody, Shang, and Wei,

215
2012) as it helps learners “to remember, observe, think, and communicate.” (as
cited by Barras in Briody et al., 2012, p.155). Moreover, getting better job
perspectives and promotions in the world of globalization where English is
considered a lingua franca, put more emphasis on writing skill as a fundamental
requirement (Binandeh et al., 2017, Briody et al., 2012). Yet, writing in a second
language is not an easy task. Unlike speaking, writing is a complex and
challenging process in which writers try to transmit their knowledge, thoughts
and ideas on papers to be read and understood by a reader who is not present.
Accordingly, it requires a proper choice, link, and sequence of sentences to ensure
that the text “can be interpreted on its own” (Byrne, 1979, p.4, 5). Therefore,
researchers in the area of second language learning have been trying to find
techniques to help and support learners in writing, especially struggling writers.
Indeed, there has been a popular agreement and recognition of the significant role
of writing strategies in enhancing language performance (Diaz, 2013; Nasihan &
Cahyono, 2017; Okasha & Hamdi, 2014; Binandeh et al., 2017). Compared to L1 in
which writers can automatically access grammatical and lexical collections while
writing (Binandeh et al., 2017), it is more challenging in the foreign language that
compels writers to consciously use strategy to access the knowledge they have
(Okasha & Hamdi, 2014).
Writing strategies refer to particular processes, methods, or techniques deployed

by learners to enhance and improve their writing (Baker & Boonkit, 2004; Bai, Hu
and Gu, 2014). According to Riazi and Wenden (as cited in Xiao, 2016), there are
four types of writing strategies: rhetorical, metacognitive, cognitive, and social/
affective strategies. Rhetorical strategies help writers to organize and present their
ideas, whereas metacognitive strategies include monitoring and evaluating the
writing process. Cognitive strategies are concerned with implementing the actual
writing, and social/ affective strategies focus on interaction with others,
motivation, emotions, and attitudes in writing (Xiao, 2016). Among these
strategies, metacognitive is considered the most important as they assist learners
to direct, guide, and control their learning as well as foster the way to be active,
aware, independent, autonomous, and life-long learners (Boghian, 2016;
Bouirane, 2015; Haukas, 2018; Oz, 2005).
This study attempts to investigate the metacognitive writing strategies used by

grade 12 students in Oman through answering the following research questions:
1. To what extent do grade twelve Omani students use the different types of
metacognitive writing strategies?
2. Is there any significant relationship between the types of metacognitive
writing strategies students use and their English language performance?
3. Are there any significant differences between grade twelve males and
females in the type of metacognitive writing strategies they use?
Actually, several important gaps have been identified with regard to studies on
metacognitive writing strategies. First, there is little research on school students’
metacognitive writing strategies as the majority of the work targeted learners in
the higher education sector. Second, investigating metacognitive writing
strategies in the Omani context is absent and there is a scarce descriptive study

216
exploring metacognitive strategies. Hence, because of these gaps, and to further

contribute to this field of study, the results of the present research will hopefully
assist teachers to be aware of the metacognitive writing strategies students use
and could lead to a varied inclusion of these strategies in the future syllabus.
Moreover, the study could provide suggestions for enhancing writing classes in
light of the findings of this study.
Byrne (1979) defines writing as "a sequence of sentences arranged in a particular
order and linked together in certain ways." (p.1). Hacker, Keener, and Kircher
(2009) take this definition one-step further by laying stress on the writing process.
They note, "writing is the production of thought for oneself or others under the
direction of one's goal-directed metacognitive monitoring and control, and the
translation of that thought into an external symbolic representation." (Hacker et
al., 2009, p.154). Similarly, Flower and Hayes (1981) state that writing is a process
of decisions and choices. However, it was not until the 1980s that researchers
started to consider writing as a process rather than a mere product (Onozawa,
2010); transforming from focusing on accuracy to exploring writers’ progress
through the writing task (Xiao, 2016). Hence, several models have emerged to
describe and understand the complexity of the writing process.
2.1 The Cognitive Process Model

The Cognitive Process Model was introduced to fill the gaps in the Stage Model
of Writing that puts great attention on the written product neglecting how the
writers approach the writing task (Flower & Hayes, 1981). The advocates of this
model were Linda Flower and John Hayes. According to them, the main unit in
the writing process is the mental process, which learners focus on to generate their
final written product (Flower & Hayes, 1981; Onozawa, 2010). The Cognitive
Process Model encompasses three major units presenting writing elements: the
task environment, writers' long- term memory, and writing process. First, the task
environment includes the rhetorical problem, and the text produced so far.
Rhetorical problem refers to the assignment from which the learners can identify
the topic, audience, and their role. Solving the problem needs higher-order
thinking skills and strategies, which indicate the role of metacognitive strategies
in this process. Besides, the text that is already written influences the writer's
choice of either to continue to write or to stop to modify what has been written.
The second phase in the cognitive process is the writer's long-term memory that
includes his knowledge about the topic, the audience, the plans, and the problem
that contributes to the flow of the writing process. The last phase is the writing
process. This stage includes three processes: planning, translating, and reviewing.
In the planning process, the writers make decisions and choices about the
knowledge that will be used in their composition. It involves three sub-processes:
generating ideas, organizing, and goal setting. Translating refers to transforming
ideas into "visible language" (Flower & Hayes, 1981, p. 373). Finally, reviewing
includes two sub-processes, which are evaluating and revising.
The aforementioned processes are recursive, which is a key feature of the

cognitive process model (Hacker et al., 2009; Victori, 1995). In this spiral and

217
cyclical process, the writer plays the role of a monitor as he needs to check his
composition throughout the process and thinks how to modify it to reach the
intended meaning. For instance, in the translating process, the writer might
monitor what he has already written which might lead to a "new cycle of planning
and transforming." (Flower and Hayes, 1981, p.374).
Moreover, Hacker et al. (2009) explain that monitoring might include reading, re-
reading, reflecting, and reviewing. They argue that these strategies are monitoring
strategies of thoughts, which presents writing as "applied metacognitive" (p. 161).
Despite paying less attention to grammar and structure and giving little
significance to the final product, the cognitive process model helps in generating
and increasing students' intrinsic motivation as learners feel free to manage their
own writing without being stuck to the traditional controlled writing (Onozawa,
2010).
2.2 The Knowledge Telling and the Knowledge Transforming Models

The Knowledge telling process and knowledge transforming process were
influential within the process model. Bereiter and Scardamalia introduced the
model in 1992 to address the problem of how to translate thoughts into written
form (Victori, 1995). They claim that "not all writers are able to transform those
mental ideas into some kind of coherent written form" (Victori, 1995, p. 21). More
specifically, the two models explain the differences between mature (expert) and
immature (novice) writers in the way they transform their knowledge and ideas
into the writing process.
2.2.1 The Knowledge Telling Model

It is a model of immature writers. In this model, the writer focuses his attention
on the topic. He tries to find some cues to activate his passive knowledge stored
in his memory and then starts writing directly. Once the writer starts composing,
the produced units of the text act like another cue or identifier for getting another
stored knowledge (Bereiter and Scardamalia, 2013). Accordingly, the writer
generates a text without applying the problem-solving strategies put forward by
Flower’s and Hayes’s model.
2.2.2 Knowledge Transforming Model

This is a model of mature writers. It is also considered a process within which the
former model is embedded (Bereiter and Scardamalia, 2013). In this model, the
text is not transcribed immediately. Instead, it is preceded by problem-solving
aspects; e.g. problem analysis, planning, setting goals, and paying attention to the
readers (Victori, 1995). Through this process, writers go through continuing
revision and rethinking which leads to new sub-goals for the text (Bereiter and
Scardamalia, 2013). Mature writers who use this model are expected to use
metacognitive strategies during their composition.
Interestingly, Camps (as cited in Victori, 1995) argues that most of the students at
school follow the knowledge-telling model, which explains the obstacles students
face in writing. Bereiter and Scardamalia (as cited in Victori, 1995) recommended
that to improve students' composition, they should go beyond the knowledge
telling model; use more knowledge transforming which requires applying higher-

218
level thinking strategies. In other words, students need to be aware of and use a
variety of metacognitive writing strategies while completing writing tasks.
2.3 Writing Strategies

Due to the transformation in the learning process that views learners as active
thinkers, language-learning strategies have been considered as a significant
cognitive factor involved in writing acquisition (Victori, 1995 & Zhan, 2016).
Writing strategies refer to particular methods, processes, or techniques deployed
by learners to enhance and improve their writing (Baker & Boonkit, 2004; Bai et
al, 2014). Victori (1995) noted that there is no agreement on these processes as well
as the labels used for them. This has led to the emergence of a variety of writing
strategy taxonomies; e.g. Arndt, 1987; Riazi, 1997; Sasaki, 2000; Victori, 1990; and
Wenden, 1991 (as cited in Mu, 2005). Yet, one of the contributions that are worth
mentioning is Mu's taxonomy (2005) as it synthesizes the aforementioned
taxonomies. Mu classifies writing strategies into five types: rhetorical,
metacognitive, cognitive, communicative, and social/ affective.
2.4 Metacognitive Writing Strategies

Metacognitive writing strategies can be defined as strategies that “are used to
monitor the writing process consciously and evaluate the effectiveness of writing
actions.” (Xiao, 2016, p.20). Metacognitive strategies play a vital role in language
learning. Metacognitive writing strategies such as paying attention and
overviewing help learners to stay focused and maintain concentration while
writing (Oxford, 1990, p. 136). In addition, these strategies assist learners to plan
their learning efficiently and effectively. Moreover, metacognitive writing
strategies promote language acquisition, as the learner is aware of his learning
(Rahimi & Katal, 2012). Moreover, learners’ ability to adapt their learning to new
contexts and tasks will increase as they are exposed to more strategies (Chick,
2014). More significantly, metacognitive writing strategies contribute to more
autonomous and independent learning (Diaz, 2013; Mu, 2005). Hence,
metacognitive writing strategies “can lead to more profound learning and
improved performance, especially among learners who are struggling.”
(Anderson, 2002, p.2). Metacognitive writing strategies are categorized into three
types: planning, monitoring and evaluating.
2.4.1 Planning
Planning involves thinking about and focusing on the goals to be accomplished
and how to achieve them (Anderson, 2002). Self-questioning strategies are used
to identify various components such as purpose, audience, and strategies to be
used ( Diaz, 2013; Xiao, 2016). Flower and Hayes (1981) listed three types of
planning strategies: generating ideas, setting goals, and organizing. Generating
ideas involve forming new information, using old ideas and information from
long-term memory, making connections, and thinking about examples. Setting
goals include content goals (e.g. text structure and audience), and process goal
(how to proceed). Finally, organizing refers to filtering and choosing the most
useful content generated and putting them in a writing plan (Xiao, 2016).

219
2.4.2 Monitoring
Monitoring, also identified as self- monitoring, is "a conscious control and
regulation of the writing process." (Xiao, 2016). It involves assessing, self-
questioning, verifying, controlling, directing, and arranging the writing
performance while composing (Anderson, 2002; Diaz, 2013; Xiao,2016).
Monitoring helps learners to evaluate the effectiveness of writing strategies and
to decide whether ideas need to be changed or added.
2.4.3 Evaluation/ self- evaluating

Evaluation is defined as checking-back to decide about the effectiveness and the
quality of one’s composition with respect to the intended goals (Anderson, 2002;
Nemouchi, 2017, Xiao, 2016). It also encompasses reactions that should be taken
based on the evaluation of the composition.
2.5 Studies on Metacognitive Writing Strategies

Razi (2012) investigated the metacognitive writing strategies of 250 Cypriot
University students. The results demonstrated that less than half of the
participants were aware of metacognitive writing strategies; indicating "lack of
training in terms of strategy use" (Razi, 2012, p.10). However, a strong positive
correlation was found between students’ grades and metacognitive writing
strategies; successful students used more metacognitive strategies than medium
and weak students did. For further clarification, the researcher interviewed six
teachers who reported metacognitive strategies were confined by time, which
might contribute negatively to students’ awareness of these strategies. Similar to
Razi (2012), Briody et al. (2012) conducted a study to explore the relationship
between English writing ability level and EFL learners’ use of metacognitive
strategies. Majoring in applied English, 152 students at a university in Taiwan,
completed a questionnaire designed by the researchers along with a composition
test to classify their writing levels. The results revealed that high-level writers use
more metacognitive strategies than low-level writers do, particularly in the
planning and reviewing stages. Furthermore, by employing a multiple regression
analysis, it was found that the planning stage is an important factor to predict the
level of students’ writing. Accordingly, the researchers suggested that instructors
train students on how to plan before writing; e.g. generating complete ideas and
making an outline before writing. Zhan (2016), working with 93 Chinese college
students, found that among four categories of metacognitive strategies, selective
attention was the most frequently used while self- evaluating was the least. The
researcher also found that successful writers used metacognitive strategies more
than unsuccessful writers, which shows the role of these strategies in improving
students’ writing. In the same vein, Azizi et al. (2017) examined the correlation
between the use of metacognitive writing strategies and writing performance. The
study was based on a questionnaire and a writing test completed by 30 Iranian
female EFL learners and it showed a strong and positive correlation between
metacognitive strategies and students' writing performance. In addition, unlike
Razi’s finding, the evaluating stage showed the strongest correlation to predict
writing performance.
In contrast to previous studies, a study by Abdollahzadeh (2010), on 230

undergraduate students in Iran, showed no significant differences in the

220
frequency of writing strategy use between high- and low-level writers. He

attributed this to the role and effect of instruction they receive in their first year at
university; e.g. taking notes, writing memos, and summarizing. Such findings
are important because they suggest that teaching students to use strategies
through explicit instruction and feedback has a significant impact on their writing
performance. Moreover, the findings revealed no significant differences between
males and females in using writing strategies; they both use metacognitive
strategies at a similar frequency. Similar to the study by Abdullahzadeh (2010),
Mutar and Nimehchisalem (2017) explored the frequency of using writing
strategies among 132 Iraqi high school students using Petric and Czarl’s Writing
Strategy Questionnaire. The results showed no significant differences between
high and low proficient students in using these strategies. However, the study
revealed that female students used more writing strategies than male students
did.
3. Methodology
3.1 Research design
This study aims to investigate metacognitive writing strategies used by Omani
EFL grade twelve students. To achieve this, the study adopted a quantitative
descriptive design as the researchers “express the relationship between variables
using effect statistics, such as correlations, relative frequencies, or differences
between means” (Hopkins, 2000, p.1). A questionnaire was used as the main
instrument to collect quantitative data. Besides, for further investigation, a semi-
structured interview was conducted with students to highlight the metacognitive
strategies they utilize when writing.
3.2 Sampling
The population of the current study was grade 12 students in the Omani
governmental schools in Batinah North Governorate for the academic year 2018/
2019. The governorate had 7927 grade 12 students (4080 males and 3847 females).
Convenience sampling was used to select the sample of the study by which two
male, and two female post- basic schools were chosen, and two classes were
selected from each school. Convenience sampling, also known as accidental
sampling, is a type of nonrandom sampling techniques where a group of the
population is chosen due to certain criteria; e.g. being easy to access, are willing
to participate, or are available at a particular time (Etikan; Musa and AlKassim,
2016). The sample size comprised 263 students including 140 females (53.2%) and
123 males (46.8%). Their grades in the first semester's final test ranged from 14 to
70 with a mean of 41.14 and a standard deviation of 14.99. The rationale for
targeting students in grade 12 is that grade 12 is the exit level from which students
join higher education institutions, where being autonomous, independent and
life-long learners – (the skills promoted by metacognitive strategies) is needed.
3.3 Instrumentation
3.3.1 The Metacognitive Strategies of Writing Survey (MSWS)
The main instrument of the current study was the Metacognitive Strategies of
Writing Survey (MSWS) which was developed by the researchers after a review of
the literature with special reference to Xiao (2016)’s Survey of Metacognitive
Writing Strategies. The survey consisted of two parts (Appendix A). The first part

221
aimed to collect general biographical information about the participants. This

information included gender, school’s name, and student’s number in the name
list which was used to record his/ her mark in the first semester English final test.
In the second part, students were asked to respond to a questionnaire of 27 items,
divided into three categories presenting the types of metacognitive writing
strategies, as follows: twelve planning items, nine monitoring items, and six self-
evaluating items. Students were asked to indicate to what extent they used the
given strategies and their responses are obtained on a 5-point Likert scale ranging
from 1(never) to 5 (always).
Content validity was verified for the questionnaire, which originally consisted of
37 items, by a number of experts who evaluated the relevance and clarity of each
item. Based on their feedback, some modifications were made to the questionnaire
items. The order of some items was also modified and some items were reworded
to make them clearer. In addition, the questionnaire was translated into Arabic
and validated by professional translators to avoid any misunderstanding caused
by students’ proficiency level. The translators have good experience in translation
and linguistics as they have been working at Sultan Qaboos University for more
than 15 years. Furthermore, to verify the reliability of the instrument, the
questionnaire was piloted on 32 students including 17 females and 15 males, and
internal consistency reliability was found to be reliable at .926, as measured by
Cronbach’s alpha.
3.3.2 The Semi-Structured Interview

To investigate deeper, the questionnaire was supported by semi-structured
interviews with four female and four male students. Among them, there were four
high- proficiency and four low- proficiency students who were selected based on
their grades in the Final English Language Test (semester one), their English
teachers' evaluation, and their willingness to participate. The interview focused
on eliciting the strategies employed by those students throughout the writing
process and it was audio recorded. The researchers recommended the teachers to
hold the interview in Arabic, so that the students from different levels are able to
understand the questions clearly and respond accurately. The interview questions
were validated by the same panel that validated the MSWS. They were also
piloted by volunteering teachers to ensure clarity of the items. Two males and two
females were selected from the piloting sample for the interviews. Two of them
were high- achievers (one male and one female) and the other two were low
achievers. In response to the validation panel’s and the interviewers’ feedback,
some items were reworded to make them clearer for the interviewees. Similar to
the MSWS, the questions were translated into Arabic and reviewed by the same
translators who recommended some minor changes in the Arabic statements to
avoid any ambiguity.
3.3.3 Students’ Semester Grades

According to Nambiar (2009), language performance is one of the factors that
affect students' use of learning strategies. Hence, addressing this factor is of great
importance to understand students' use of metacognitive learning strategies in
writing. In the current study, students' semester scores of the final English
Language Test (semester one) were recorded and used as a measure of their

222
English language performance. End of Semester tests in Oman evaluate students'

overall performance as their scores reflect their skills in reading, listening, and
writing. Each student was required to write his/ her list number in the first section
of the questionnaire which was used later to record their scores in the final test.
3.3.4 Analysis procedures

The Metacognitive Strategies of Writing Survey (MSWS) was administered to the
participants and the data collected was then analyzed through the Statistical
Package of Social Sciences (SPSS). Descriptive statistics, Pearson product-moment
correlation coefficients, and t-test were computed to answer the three research
questions, respectively.
After questionnaire data were collected, the interviews were conducted and
audio-recorded in Arabic by the volunteering teachers. Then, the interviews were
transcribed by the researchers. The most frequent patterns of responses were
highlighted. These patterns as well as other responses were discussed with the
students. The strategy types (planning, monitoring, and evaluation) served as
themes for content analysis.
4. Results
4.1 Omani EFL Grade Twelve Students’ Use of Metacognitive Writing
Strategies
To answer the first research question, 'To what extent do grade twelve students
in Oman use different types of metacognitive writing strategies?', the
Metacognitive Strategies of Writing Survey (MSWS) was administered to the
participants, and the data collected was then analyzed through Statistical Package
of Social Sciences (SPSS). Descriptive statistics; namely means and standard
deviations, of the entire metacognitive writing strategies, mean scores for the
three types of metacognitive writing strategies, and individual strategies were
calculated for the whole sample. According to Oxford's (1990) classification of the
mean scores of the strategy use, there are three main levels to describe students'
strategy use: high (ranging from 3.5 to 5), medium (ranging from 2.5 to 3.4), and
low (ranging from 1.0 to 2.4). Table 1 displays the means and standard deviations
of the overall metacognitive writing strategies and the three categories of these
strategies. As shown in the table, the participants reported a high level of
metacognitive writing strategy use as the overall mean of the metacognitive
writing strategies use was 3.82. Similarly, all of the categories had mean scores
above 3.5, indicating that all the participants utilize the three categories of
metacognitive strategies at high frequencies in their writing.
Table 1: Means and Standard Deviations for Grade 12 Students’ Use of Metacognitive
Strategies in Writing (N= 263)
Category M SD
Planning 3.83 .69
Monitoring 3.82 .74
Evaluation 3.81 .89
Overall Use 3.82 .68

223
Among the different categories of metacognitive writing strategies, planning

strategies (M= 3.83, SD= .69) were reported as the most frequently used strategies,
while evaluation strategies were reported as the least frequently used category
(M= 3.81, SD= .89). This might be attributed to the frequent training of these
strategies by teachers and the various planning tasks students come across in their
textbooks (Engage with English). These tasks include different techniques to plan
and organize writing texts such as graphic organizers, mind maps, diagrams, and
tables. Yet, the differences between the mean scores of the three categories are not
very high (3.83, 3.82, 3.81).
4.2 The Relationship between the Type of Metacognitive Writing Strategies

Students Use and their English Language Performance.
To answer the second research question, "Is there any significant relationship
between the types of metacognitive writing strategies students use and their
English language performance?", Pearson product-moment correlation
coefficients were computed to explore the relationships between students'
performance (measured by students' grades in English Language final exam of
semester one, school year 2018/2019) and the three types of metacognitive writing
strategies; planning, monitoring, and evaluation. Table 2 presents means and
standard deviations of students' total use of planning, monitoring and evaluation
strategies, as well as their grades in the English final exam of semester one along
with the bivariate correlations among the variables.
Table 2: Means and Standard Deviations of Students’ Total Use of Planning,

Monitoring and Evaluation Strategies, and Students’ Grades along with the Bivariate
Correlations among the Variables
Variable N M SD 1 2 3 4
1. Planning 247 45.92 8.31 - .73 .68 -.05
2. Monitoring 254 30.55 5.93 - .77 .01
3. Evaluation 257 22.85 5.36 - -.03
4.Grades 245 41.14 14.99 -
**p < .01.
The following guidelines, suggested by Cohen (as cited in Pallant, 2011), were
used to interpret the coefficient values.
Table 3: Cohen’s Guidelines of interpreting Correlation Coefficients

Correlation Coefficient (r) Strength
.10 to .29 Small
.30 to .49 Medium
.50 to 1.0 large
Results showed that there was a very weak negative relationship between the use
of planning strategies and students’ grades. Similarly, there was a very weak
negative relationship between the use of evaluation strategies and students’
grades. In addition, the analysis displayed a very weak positive relationship
between the use of monitoring strategies and students’ grades. However, the

224
relationship between the use of the three strategies and students’ grades were not
significant (p > .01).
4.3 The Differences in Students’ Use of Metacognitive Writing Strategies

with Respect to Gender
To answer the third research question, ‘Are there any significant differences
between grade twelve males and females in the type of metacognitive writing
strategies they use?’, an independent-samples t-test was run to examine gender
differences on the students’ use of the three types of metacognitive writing
strategies; planning, monitoring, and evaluation. Table 4 presents means,
standard deviations, and p-value of the three strategies use for males and females.
Table 4: Means, Standard Deviations, and p-value of the Three Strategies Use for
Males and Females
Strategies Gender N M SD p-value
Planning Males 117 3.60 .71 .000
Females 130 4.03 .61
Monitoring Males 120 3.63 .79 .000
Females 134 3.99 .66
Evaluation Males 121 3.5 .94 .000
Females 136 4.08 .75
Results showed that there were significant differences between males and females
in the use of planning strategies (t(245) = -5.21, p = .000), monitoring strategies
(t(252) = -3.95, p = .000), and evaluation strategies (t(228.68) = -5.45, p = .000). The
average use of each category of metacognitive writing strategies of females was
higher than the average use of males; female students use more metacognitive
writing strategies than their male counterparts do. Oxford (1993) indicated that
females are generally more active strategy users than males (as cited in Al Abri,
2017). Statistics also revealed that gender accounted for 9.98%, 5.83%, and 10.43
% of the variance in the use of planning, monitoring, and evaluation strategies,
respectively. Accordingly, the magnitude of the differences in the means between
males and females was moderate (Cohen, 1998).
4.4 Interview Findings

For a deeper understanding, semi-structured interviews were conducted with
eight students- four females and four males. Among them, there were four high
achievers and four low achievers. The strategy types (planning, monitoring, and
evaluation) served as themes for content analysis.
1.Planning
The interview data was in alignment with the questionnaire results that all
participants (100%) referred to planning strategies as the most used strategy in
their writing. 75% spend more time planning for their essays and they pay more
attention to generating and organizing ideas.
• "I start to read the statements in the exam question to know the type of writing
and the number of words I have to write. Secondly, I start to write points or
ideas to help me (I write them in pencil). Then I start to put them in pen.”

225
On the other hand, 25% of students spend less time in the planning and, indeed,
there is no clear indication in their responses about making an outline before
starting to write their essays. In fact, they mostly think about grammar, structures,
and words they need to use in the task.
• “I think about the basics of writing that essay. Then I write the essay and the
conclusion.”
• "First, ideas, and then I think about grammar and vocabulary. “
Regarding the audience, only one interviewed student considered readers in the
planning stage, which confirms the quantitative results where students reported
items concerning the audience as the least used planning strategy.
• “I make sure the ideas are clear to the reader.”
2.Monitoring
While writing, 87.5% of the students believe that it is very crucial to check their
writing. For instance, they make sure that their vocabulary is right, their work is
clear to the audience and they make sure that they remain focused on the topic.
• “Yes, this is one of the most important things to do. So, your writing is clear to
the reader.”
However, twenty-five percent of the respondents indicated that their main

concern while writing is the local errors (grammar, vocabulary, spelling).
• “I check vocabulary and grammar.”
• “I count the words to check if I reach the required words, number and I check
the layout of my essay.”
3. Evaluation
All the students highlighted the significance of self- evaluation as they indicated
that they use a variety of techniques to evaluate their writing. For example, re-
reading their essays several times, reading their essays loudly to check if they
make sense, and using programs and applications to help them in their
evaluation. They explained that these techniques are of paramount importance to
check the clarity of the essays to the reader and to check that ideas are sufficient
to clarify the topic:
• “Yes, I do. I read them aloud to make sure they make sense. If not, I do it again.”
• “I read it again two or three times to organize it and make the ideas clear to the
reader. I also check grammar and vocabulary.”
Nevertheless, similar to monitoring strategies, fifty percent of the students stated

that their focus is the local errors; namely grammar, vocabulary, and spelling:
• “Yes, I check punctuation and verbs.”
• “I check my essay and words spelling.”
Furthermore, all the students reported that the strategies mentioned in the
questionnaire they completed (MSWS) are very useful to improve their writing:
• “They are very useful strategies. I am glad they come up with strategies.”
• “They are wonderful. They help me to minimize the mistakes in writing”.

226
Moreover, students were asked to describe the difficulties they encounter when
they write essays. Fifty percent of the students refer to planning and generating
ideas as the biggest challenge they encounter when writing.
• “I have problems with the subject itself. If the subject is new, I need more time to
generate ideas.”
And a quarter of them i.e. 25% of them pointed out that they have an issue with
handwriting which impacts the clarity of their compositions.
• “Yes, my handwriting is very bad.”
5. Discussion
This study revealed that Omani EFL grade12 students use metacognitive writing
strategies at a high level of frequency. It was also found that they utilize the three
types of metacognitive writing strategies: planning, monitoring, and evaluation,
at different frequency levels. Planning strategies were reported as the most
frequently used strategies, whereas evaluation strategies were the least used
strategies among all the participants. These findings were confirmed by the
results of the semi-structured interview, which showed that all the interviewee
students begin writing tasks by planning and organizing their ideas. These results
corroborate the findings of Razi (2012), Bai et al. (2014), and Zhan (2016), whose
participants reported using planning strategies at a high frequency compared to
other metacognitive strategies. They explained that students perceive planning as
a vital element of good writing, which involves spending time thinking about
ideas, words, phrases, and sentences they use in their compositions.
With regard to students’ use of the writing metacognitive strategies and their
language performance, the results of the current study showed there was no
significant relationship between the types of metacognitive writing strategies
students use and their English language performance. This confirms the findings
of Abdollahzadeh (2010) who found no significant differences between high and
low proficient students in using writing strategies. This finding also supports
what Mutar and Nimehchisalem (2017) reported on the differences between high-
and low- proficiency level students. They concluded that the two groups use
writing strategies in a very similar way. However, the findings of this study are
not in line with those of Azizi, Estahbanati, and Nemati (2017), Bouirane (2015),
and Razi (2012) who found that there is a strong positive correlation between
students metacognitive writing strategy use and their grades; increasing the
strategies corresponds to increasing students' scores or proficiency level. The
findings of the present study also do not echo the studies by Zhan (2016) and Liu
(2015) which revealed significant differences between successful and unsuccessful
students i.e. high proficient students use more writing strategies than students
with low proficiency levels. The reason for this lack of consistency could be the
definition of students' language performance. While the current study used
students' grades in an End semester final English test to measure students'
performance, other studies identify students' performance based on their scores
in writing tests administered to students before completing the targeted
questionnaire. The current study also found that there were significant differences
between males and females in the use of metacognitive writing strategies.

227
Similarly, Bouirane (2015) explored the difference in using metacognitive

strategies in relation to gender and found that female students use more
metacognitive strategies than male students do. Likewise, Mutar and
Nimehchisalem (2017) and Liu (2015) studied the writing strategies used by high
school students in Iraq and China, respectively, and revealed that there is a
significant difference between female and male students’ strategy use. Female
students employed better and more writing strategies compared to male students.
This result, however, does not seem to be conclusive. Surat, Rahman, Mahmod,
and Kummin (2014) investigated the use of metacognitive strategies among
university students in Malaysia and did not find any difference in the use of
metacognitive strategies based on gender. The findings match to some extent
those found by Abdollahzadeh (2010) who conducted a study to investigate
writing strategies used by Iranian university students and concluded that male
and female students use writing strategies with approximately the same
frequency.
6. Conclusion and Recommendations

This study investigated the metacognitive writing strategies used by Omani EFL
grade twelve students. The researcher administered a questionnaire and used a
semi-structured interview to determine the extent to which students use the three
types of metacognitive writing strategies: planning, monitoring, and evaluation.
The current study also explored the relationship between students' English
language performance and the types of metacognitive writing strategies they use.
It also looked at differences between male and female students in using the three
types of target strategies.
The results of the study revealed that grade twelve students use the three types of
metacognitive writing strategies at high frequency. The findings also indicated
that there is no relationship between students' language performance and the use
of the three types of strategies. Students with high-performance levels and low-
performance levels use these strategies similarly. The results of the present study
also disclosed that there are significant differences between males and females in
the use of all the types of metacognitive writing strategies. Females tend to use
more strategies than their male counterparts do.
Although the present study revealed that students use the three types of
metacognitive writing strategies at high frequencies, they reported using the
individual strategies at different rates. Accordingly, there is a need to raise
students’ awareness of various metacognitive strategies they can access to control
and evaluate their writing. To achieve this, teachers should equip students with
sufficient practice of these strategies explicitly through a systematic writing
strategy instruction and implicitly through tasks, activities, and materials in
writing classes targeting these strategies. Students also need to be taught how to
monitor their writing, so they can identify the problems and shortcomings of their
compositions which in turn helps them to take responsibility for their learning.
This can be achieved by providing an ongoing practice to evaluate, modify, and
correct their work, as well as by providing students with constructive feedback,
by their teachers, which "enhance learning, promote the acquisition of skills, and

228
drives professional growth and development." (Omer & Abdulrahim, 2017, p. 45).
Moreover, the results of the present study showed that students place emphasis
on local errors, which is probably a result of adopting the product-oriented
approach in writing. Consequently, teachers should consider developing
students’ monitoring skills to tackle global errors as well as local ones by adopting
the process and genre approaches to writing.
Furthermore, professional development and in-service programs should consider

the importance of metacognitive writing strategies and conduct workshops to
raise teachers' awareness of the importance of these strategies. Teachers also
should be equipped with techniques that assist them in implementing these
strategies in writing classes.
Curriculum designers are also expected to consider these strategies when

designing writing units. These units should include tasks and activities that
enhance students’ metacognitive writing strategies. For instance, providing
students with guidelines to check their performance in a form of a checklist can
be a very effective technique by which students can highlight the aspects they
need to evaluate in their writing. Moreover, the study revealed students' growing
interest in using different programs and applications to improve their writing skill
which should direct curriculum developers' attention for integrating technology
in writing classes to enhance students' writing, as well as raising their motivation.
One of these promising applications is using Blogs to write diaries, which is very
inspiring and encouraging to enhance students’ writing performance.
8. References
Abdollahzadeh, E. (2010). Undergraduate Iranian EFL learners’ use of writing
strategies. Writing & Pedagogy, 2(1), 65-90. https://doi.org/10.1558/wap.v2i1.65
Abedin, M., Hosseini, M., Naseri, E., & Taghizadeh, M. (2013). In the importance of EFL
learners' writing skill: Is there any relation between writing skill and content score
of English essay test. International Letters of Social and Humanistic Sciences, 6, 1-12.
https://doi.org/10.18052/www.scipress.com/ilshs.6.1
Al Abri, A., Al Seyabi, F., Al Humaidi, S., & Hasan, A. H. (2017). Grammar learning
strategies in Omani EFL classes: Type and relation to student proficiency. Journal
of Studies in Education, 7(2), 151-166.
Anderson, N. J. (2002). The Role of Metacognition in Second Language Teaching and
Learning. ERIC Digest.
Azizi, M., Estahbanati, N., & Nemati, A. (2017). Meta-Cognitive awareness of writing
strategy use among Iranian EFL learners and its impact on their writing
performance. International Journal of English Language & Translation Studies, 5(1),
42-51.
Baker, W., & Boonkit, K. (2004). Learning strategies in reading and writing: EAP
contexts. RELC Journal, 35(3), 299-328.
https://doi.org/10.1177/0033688205052143
Bai, R., Hu, G., & Gu, P. Y. (2014). The relationship between use of writing strategies and
English proficiency in Singapore primary schools. The Asia-Pacific Education
Researcher, 23(3), 355-365. https://doi.org/10.1007/s40299-013-0110-0
Bereiter, C., & Scardamalia, M. (2013). The psychology of written composition. New York, NY:
Routledge.

229
Binandeh, M., Rahmani, S., & Raoofi, S. (2017). An investigation into writing strategies
and writing proficiency of university students. Journal of Language Teaching and
Research, 8(1), 191-198. https://doi.org/10.17507/jltr.0801.24
Boghian, I. (2016). Metacognitive learning strategies in teaching English as a foreign
language. Journal of Innovation in Psychology, Education and Didactics, 20(1), 53-62.
Bouirane, A. (2015). Metacognitive language learning strategies use, gender, and learning
achievement: A correlation study. International Journal of English Language and
Translation Studies, 3(2), 119-132.
Briody, P., Shang, H., & Wei, Z. (2012). The Relationship between English writing ability
levels and EFL learners’ metacognitive behaviour in the writing
process. International Journal of Academic Research in Progressive Education and
Development, 1(4), 154-180.
Byrne, D. (1979). Teaching writing skills. England: Longman.
Chick, N. (2014). Thinking about one is thinking. Retrieved from
https://cft.vanderbilt.edu/guides-sub-pages/metacognition/.
Cohen, A. (1998). Strategies in learning and using a second language. London: Longman.
Diaz, G. (2013). Metacognitive writing strategies in academic writing: the path to enhance
autonomy and to become expert writers (Master’s thesis). National University of
Cordoba. Retrieved from http://hdl.handle.net/11086/1130
Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and
purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1-4.
https://doi.org/10.11648/j.ajtas.20160501.11.
Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition
and Communication, 32(4), 365-387. https://doi.org/10.2307/356600
Hacker, D. J., Keener, M. C., & Kircher, J. C. (2009). Writing is applied metacognition. New
York, NY: Routledge.
Haukås, Å. (2018). Metacognition in language learning and teaching. New York, NY:
Routledge.
Hopkins, W. G. (2000). Quantitative research design. Sportscience, 4(1), 1-8.
Liu, G. (2015). Investigating the English writing strategies used by Chinese senior high
school students. Theory and Practice in Language Studies, 5(4), 844-850.
https://doi.org/10.17507/tpls.0504.21
Mu, C. (2005). A taxonomy of ESL writing strategies. Retrieved from
https://eprints.qut.edu.au/64/1/64.pdf
Mutar, Q. M., & Nimehchisalem, V. (2017). The effect of gender and proficiency level on
writing strategy use among Iraqi high school students. Arab World English
Journal, 8(2). https://doi.org/10.24093/awej/vol8no2.12
Nambiar, R. (2009). Learning strategy research—where are we now. Reading Matrix: An
International Online Journal, 9(2) ,132-149.
Nasihah, M., & Cahyono, B. (2017). Language learning strategies, motivation, and writing
achievement of Indonesian EFL students. Arab World English Journal, 8(1), 250-263.
https://doi.org/10.24093/awej/vol8no1.18
Nemouchi, O. (2017). Metacognitive strategies for EFL writing. Revue Sciences Humaines,
(48), 159-174. https://doi.org/10.34174/0079-000-048-080
Okasha, M. A., & Hamdi, S. A. (2014). Using strategic writing techniques for promoting
EFL writing skills and attitudes. Journal of Language Teaching and Research, 5(3),
674.
Omer A., & Abdularhim M. (2017). The criteria of constructive feedback: The feedback
that accounts. J Health Spec, 5(1), 45-48. https://doi.org/10.4103/2468-
6360.198798

230
Onozawa, C. (2010). A study of the process writing approach: A suggestion for an eclectic
writing approach. Research Note, 153-163.
Oxford, R. (1990). Language learning strategies. Boston, Massachusetts: Heinle & Heinle
Publishers.
Oz, H. (2005). Metacognition in foreign/second language learning and teaching. Hacettepe
University Journal of Education, 29, 147-156.
Pallant, J. (2011). SPSS survival manual (4th ed). Cross Nest, NSW: Allen & Unwin.
Rahimi, M., & Katal, M. (2012). Metacognitive strategies awareness and success in learning
English as a foreign language: an overview. Procedia-Social and Behavioral
Sciences, 31, 73-81. https://doi.org/10.1016/j.sbspro.2011.12.019
Razı, Ö. (2012). An Investigation into the metacognitive writing strategies of Turkish
Cypriot University students. Retrieved from
http://web.aou.edu.lb/files/lebanon/An%20Investigation%20into%20the%20
Metacognitive.pdf
Surat, S., Rahman, S., Mahamod, Z., & Kummin, S. (2014). The Use of Metacognitive
Knowledge in Essay Writing among High School Students. International Education
Studies, 7(13), 212-218. https://doi.org/10.5539/ies.v7n13p212
Victori, M. (1995). EFL writing knowledge and strategies: An integrative study (Order No.
9804043). Available from ProQuest Dissertations & Theses Global. (304279487).
Retrieved from
https://ezproxysrv.squ.edu.om:2110/docview/304279487?accountid=27575
Xiao, Y. (2016). An exploratory investigation into the metacognitive knowledge and metacognitive
strategies of university EFL writers in China (Doctoral dissertation). New York
University.
Zhan, R. (2016). An investigation into the metacognitive strategy use in ESL writing: A
case study. DEStech Transactions on Social Science, Education and Human Science.
https://doi.org/10.12783/dtssehs/hsc2016/3485

231
Appendix A
Metacognitive Strategies of Writing Survey (MSWS)
Strategies I use in writing How frequent I use the strategies
1. Planning Never Seldom Sometimes Usually Always

1. I pay attention to the genre of
the writing task (e.g., a letter,
descriptive, a narrative).
2. I pay attention to the purpose
of the writing task (e.g., to
express opinion, to thank
someone, to complain about a
service).
3. I consider the reader of the
writing task (e.g., the teacher,
supervisor, classmates).
4. I pay attention to the language
of the writing task (e.g., the
wording and grammar).
5. I set up my goals based on the
requirement of a writing task. 
6. I make a plan for achieving my
goals. 
7. I generate ideas to include in
my writing. 
8. I make a plan for organizing
the ideas of my essay. 
9. I first decide what the thesis
statement of the essay is.
10. I think about the topic
sentence of each paragraph.
11. I think about examples to
support the ideas in my essay.
12. I recall the model essays
related to the writing topic. 
2. Monitoring Never Seldom Sometimes Usually Always
13. I check to see if my essay
meets the requirements of the
writing task.
14. I check if I consider the reader
in my essay.
15. I check to see if the content of
my essay is relevant to the topic.
16. I check to see if the
organization of my essay is
logical. 
17. I check to see if the language
of my essay is clear. 

232
18. I check to see if my word

usage is accurate. 
19. I check to see if my grammar
is correct.
20. I adjust my writing plan.
21. I try to identify my problems
during the process of writing.
３. Evaluation Never Seldom Sometimes Usually Always
22. I review the content of my
essay to see if any addition or
deletion is needed. 
23. I mainly focus on reviewing
the accuracy of my grammar.
24. I review my use of the words
to see if they are correct.
 25. I check the organization of
my essay to see if it is clear. 
26. I review my essay holistically
to see if it achieves the goals of
writing. 
27. I think the English writing
strategies (planning, monitoring,
evaluating) mentioned in this
survey are very helpful.

233

Vol. 19, No. 8, pp. 233-254, August 2020
Implementation of Online Learning during the

Covid-19 Pandemic on Madura Island, Indonesia
Priyono Tri Febrianto*

Universitas Trunojoyo, Bangkalan, East Java Province, Indonesia
https://orcid.org/0000-0002-0753-1037
Siti Mas’udah and Lutfi Apreliana Megasari

Universitas Airlangga, Surabaya, East Java Province, Indonesia
https://orcid.org/0000-0001-8715-4448
https://orcid.org/0000-0001-5689-7839
Abstract. This study aimed to determine the online learning process and
the associated obstacles experienced by students. With the background of
the ongoing Covid-19 pandemic outbreak, this study sought to uncover
what social constructions the students engage in related to the new policy
for online learning that has recently been enforced by the Indonesian
government. A quantitative study was conducted with a descriptive
research approach. There were 274 college student respondents from
Madura, Indonesia. This study found that not all of the students prefer
online learning, inclusive of expressing their disapproval that online
learning is effective. Social, economic and cultural factors are important
indicators that online learning has not been able to be effectively carried
out in a number of regions in Indonesia. Rural communities are not ready
to welcome the latest learning methods and they are still comfortable
using conventional methods. Online learning on Madura Island is not only
constrained by technical problems and facilities but also by the human
resources that need to be encouraged to accept the latest learning models.
Students consider this learning model to not be beneficial due to the
presence of several obstacles, including the geographical area. This is
because rural areas make it difficult to access the internet. They also have
to buy an internet quota. Internet access in some applications requires a
large quota so this becomes problem for some students, especially for
those who come from lower-middle income families. Therefore the
availability of supporting facilities and infrastructure as well as the
facilitated internet access among rural communities, financial assistance
and the socialisation of the importance of online learning is necessary.
Keywords: Online learning; Students; COVID-19; Conventional learning;

Obstacles
*
Corresponding author: Priyono Tri Febrianto; Email: priyono.febrianto@trunojoyo.ac.id

234
1. Introduction
Learning media and systems are starting to shift from conventional learning to
the latest learning process by utilising technological assistance. Schools and
teachers strive to incorporate digital learning media in order to create a more
effective and applicable learning environment. In the 20th century, the digital
transformation has spread to almost all spaces and fields, including education.
Known as e-learning, online learning media creates community spaces among
the students that are not merely educational but also entertainment. This is
because they explore the student’s digital literacy (Tan, 2013). Through the
benefits of social media and various platforms, students and teachers have the
opportunity to optimise the learning process. Previous studies have suggested
that e-learning is focused on the students' understanding of collaborative work
such as how to build meaning, how to negotiate together and how to understand
each other (support) in a learning process that is conducted at a distance
(Friedman & Friedman, 2020).
E-learning is also used to maximise the learning process even though it is carried
out without face-to-face meetings. This type of learning is an alternative to
accommodating traditional forms of learning. E-learning is considered to be
quite effective because it helps the teachers to build virtual classrooms in
accordance with the conditions of learning in the classroom (Putranti, 2013).
Typically, teachers in developed countries optimise their use of the current
digital platforms in order to keep up with the platforms used by the students
and to create comfortable learning spaces. Learning media focuses on how
shared spaces can be used to communicate ideas and produce specific learning
content so as to improve the skills of both the students and teachers (Hansch et
al., 2015).
A study conducted by Donelly (2006) stated that e-learning is a medium where

face-to-face learning interactions are used to create a virtual discussion area.
Through e-learning, teachers and students can remove the place and time
barriers that can be a learning constraint. Learning can also be carried out by
utilising social media such as Facebook, Youtube, Twitter, Instagram, Blogger
and Quora by and for the students. E-learning also offers networking ease and
knowledge sharing opportunities but it has negative effect in terms of
potentially wasting time (Salmon et al., 2015; Hollis & Was, 2016; Brownson,
2014).
Moreover, the positive aspects of online learning allow students with limited
time available and geographical distance to get an education. Meanwhile, the
negative aspect is the difference in competition because electronic
communication is not natural when creating learning spaces (Kock, Verville, &
Garza, 2017). Through e-learning, the teachers are not required to meet with the
students in order to be able to create a learning space. However, by using one of
the platforms available, learning space discussions can occur easily. In addition,
e-learning offers flexible hours as the teachers can send assignments at any time
and from anywhere. If effective teaching and learning practices can be

235
performed automatically, this can support the process of improving the learning
practices (Agostinho et al., 2011).
A study conducted by Herayanti, Fuaddunnazmi and Habibi (2015) determined

that the benefits of e-learning including overcoming the limitations of the face-
to-face lesson frequency between the students and teachers through the features
of online platforms. Learning spaces that are usually in one room can now be
replaced by online learning. It is also beneficial for the teachers if they give
additional lessons. Previous studies have suggested that e-learning makes it
easier for the students to share resources, record the learning material, and feel
comfort when engaging in the discussion. However, the student’s ability to
understand the lessons varies because e-learning is not as natural as face-to-face
learning (Dabbagh & Kitasantas, 2012). E-learning is not applicable in all areas.
In remote areas, internet signal constraints and a lack of access and tools for
online learning make it difficult. Moreover, e-learning in several places is not
necessarily considered able to replace face-to-face learning.
In e-learning, teachers are not only engaging in a discussion about the study
materials resulting in the students doing assignments as in the various levels of
education including elementary through to high school. Learning also requires
comprehensive content. Some of the content in online learning in the form of
text, images, video and audio requires the students to use communication and
information technology independently in a structured and valid learning
syllabus (Prastiyo, Djohar & Purnawan, 2018; Hartsell & Yuen, 2006; Kuang-
Chih & Kriegman, 2018). Some subjects need content to support the learning that
takes place. Images, videos and audio are needed to support their learning. This
can be carried out through online learning.
On the other hand, social media like Youtube can be an alternative learning
media. This video-based form of social media brings in great benefits when it is
part of the learning process. A study conducted by Kruse and Veblen (2012)
found that the utilisation of Youtube for digital learning brings in opportunities
for educators to increase the intensity of uploading videos to Youtube. This
supports a participatory culture in terms of making circulatory videos, images
and allowing for user expressions. This indicates that social media is not merely
used as a means of expression. It is also used as part of an effective learning
process. Previous studies have found that schools and universities use Youtube
as a relevant learning medium because it is not only an academic learning
medium. It is also an information and entertainment search tool (Moghavvemi,
2018; Chau, 2010; Lin & Polaniecki, 2009).
Even though e-learning is the latest learning system in use, not everyone accepts
it well. The benefits of online learning face participation injustice because some
teachers doubt the use of online media as a learning tool (Nacu et al., 2014).
Teachers who are of the “baby boomer” generation are less able to use online
media in the learning process. On the other hand, some social media also has
shortcomings when it is used as learning media. Previous studies have
mentioned that some educational institutions use social media such as Facebook

236
to send the students work and promote it. However, there are limitations in
terms of Facebook's ability to make albums and transmitting the student’s
assessment scores (Whittaker, Howarth & Lymn, 2014; Madge et al., 2009;
Selwyn, 2009).
Even so, teachers, academics and lecturers still try to use digital platforms as a
learning tool. A study conducted by Greenhow and Lewin (2015) pointed out
that social media - as a form of online learning media - has the potential to
bridge the formal and informal aspects through a participatory digital culture
where young people easily adopt the role of consumers. In this case, social
media is used in such a way as to support learning. For example, a teacher can
use applications like Zoom or Instagram Live video to broadcast a discussion. In
addition, the use of e-learning has penetrated smartphones that have eventually
become a form of flexible learning media because they are integrated with the
internet and all relevant applications (Martono & Nurhayati, 2014; Arista &
Kuswanto, 2018; Irwansyah et al., 2017).
Thoms and Eryilmaz's study (2014) mentioned that students experience higher
interactions when using online learning media including sharing it with their
social communities and a high level of satisfaction. This makes digital learning
more interesting than face-to-face learning. This confirms that online learning is
proven to have a positive influence on the students. By joining social media,
students not only attend lessons but they also interact with fellow users,
including their school friends. Previous studies have found that digital learning
media offers an easier approach with a focus on content delivery and business as
well as the ability to challenge the traditional learning models used in the classes
(Yuan, Powell & Olivier, 2014; Liu, 2005).
On the other hand, digital learning does bring in significant benefits. Digital
learning through videos is very effective because of the aspects of flexibility
which can help with the depth and retention of knowledge, in addition to
motivating an interest in learning. It can be used to illustrate the relevance of
concepts (Thomson & Bridgstock, 2014). Not only on Youtube, but various other
social media currently also offer video-based and non-video views. In fact, both
the students and teachers can choose from various social media platforms. On
the other hand, digital media constitutes the acceptance that there is a high
prevalence of technology use among students including learning applications,
Google, podcasts and e-books (Gutmann et al., 2015; Bolliger, Supanakorn, &
Boggs, 2010; Annad, 2008).
A study conducted by Megan (2015) found that various universities are trying to
develop the digital skills of prospective graduates through learning media in
order to increase their resources. This is where social media can be used to
enhance the positive online reputation and increase the opportunities for post-
tertiary employment. This implies that the students are prepared for both
expertise based on their knowledge and mastering of technology and
information for the implementation of the aforementioned knowledge. Digital
learning can facilitate the students in being creative through the use and creation

237
of interesting content, both informal and extracurricular, while providing

competitive advantages. However, this can cause disparities (Brown,
Czerniewicz, & Noakes, 2015; Peppler & Kafai, 2007; Greenhow & Robelia,
2009).
A study conducted by Kirkwood and Price (2013) explained that e-learning

provides a highly structured context that engages the students successfully and
supports the achievement of their skills, problem solving and teamwork. By
using various platforms including social media, teachers can provide case
studies within the learning material. Furthermore, digital media also provides
space to work as a team.
Although online learning is predicted to be a very effective learning system, it

still has weaknesses. The weaknesses include the correlation between course
structure and dialogue, therefore increasing the structure alone reduces the
dialogue. Conversely, increasing the dialogue reduces the structure present in
distance education (Horzum, 2015; Bound, 2010; Power, 2008). Therefore not all
of the learning materials can be applied through distance learning.
This study has raised the issue of online learning among the students in Madura,
East Java. It used the background of the Covid-19 pandemic outbreak because
the learning process in Indonesia has recently shifted to digital learning. Digital
learning in Indonesia still faces obstacles because of the limited internet access
and supporting infrastructure. This study is important because not all regions in
several countries around the world have adequate enough access to the internet
to apply online learning.
The implementation of online learning is supported by several theories. Driver
and Bell (in Susan & Tony, 1995) explained the theory of constructivism.
Constructivism theory is a view of constructivism in which the students have a
goal, can be involved in their learning and are able to carry out or construct
knowledge more individually. Learning activities in the classroom not only play
a role in increasing the students' knowledge but they also involve setting the
situation in the classroom. In addition, a curriculum can contain learning tools,
materials and resources.
The essence of constructivism theory is that the students must discover and
transform complex information by themselves, check for new information using
old rules and revise it if the rules are no longer appropriate (Trianto, 2007). This
is fully contained in online learning where students have the freedom to obtain
information and organise their own learning. Students are no longer assisted by
the teacher face-to-face when solving problems but they are encouraged to
explore and solve problems or problems are assigned independently.
Sagala (2008) also explains constructivism theory. The essence is that the
students must discover and transform the information from the teacher

238
independently. The teacher's job is to facilitate the students in obtaining and

providing the information that will become relevant knowledge.
Learning according to constructivism theory is a process of forming knowledge.

This formation is done by the students themselves. All of the learning processes
are directed to the students as the learning subjects so then students can be
independent when developing their knowledge, drafting concepts, actively
thinking, and giving meaning to something that is learned. The teacher acts as a
designer and creates an environment that makes learning possible.
Meanwhile, the behaviourist learning theory explains that in learning, there are
changes in behaviour (Farooq & Javid, 2012). Online learning also provides a
stimulus for the students to be actively involved in the learning process even
though it is through virtual media. Changes also occur when conventional (face-
to-face) learning switches to online learning, which causes major changes in the
learning styles and systems.
In the learning theory of connectivism, this theory supports the existence of a

new perspective on how learning can take place in a virtual space. This is in
accordance with the concept of online learning where the virtual space is a place
for learning activities where all involved are digitally connected to each other.
Rusman (2011) stated in his theory that the characteristics of online learning
include interactivity, accessibility, and enrichment. Even though the students
and teachers do not meet face-to-face, they can carry out the learning process
interactively by maximising the use of technology. Learning can run through
various applications such as the use of video or audio and message-based
programs. Access to learning that is easily accessible can ultimately enrich the
students' knowledge and the nature of education can be realised properly.
Meanwhile, the use of instructional media is an important aspect in the running

of education. The theory of Heinich et al. (1982) states that teachers need to
consider the selection of appropriate learning media for students in order to
create successful learning activities including objectives, content, motivation,
technicality, ownership, and instructions for use. Therefore the teachers really
have to understand what media is right for the students because not all digital
media can be accepted and run smoothly to go on to become a learning medium.
In addition, online learning is a learning system that is more open. It also fulfils
the emotional needs of the students. Kaler (2012), in their learning needs theory,
revealed that the learning needs of students and the online learning
environment are the same. Students often feel bored in conventional learning
because there is nothing new offered in face-to-face meetings. Through online
learning, students encounter challenges and they get freedom and independence
from learning online.

239
3. Research Methods
A quantitative study was conducted using a descriptive research approach. The
study was conducted using structured interviews with 274 college students from
Bangkalan, Sampang, Pamekasan and Sumenep. The study took on the social
setting of the Covid-19 pandemic because the learning system in Indonesia has
been directed towards online learning, even though this type of learning process
is considered by some parties to be not ready in several places in Indonesia.
The data collection was carried out by distributing questionnaires through a

Google form that was sent to the students in remote areas on Madura. In
addition to the survey data obtained from the Google form, this study also
triangulated the data in order to reveal the reality of the study in more depth.
Data triangulation was conducting using in-depth interviews with the students,
lecturers and parents of students who were the respondents in this study. Some
of the in-depth interviews were conducted face-to-face with the informants
while adhering to the health protocols by implementing social distancing and
wearing masks. Some were carried out over the telephone because of the Covid-
19 pandemic in which some people only wanted to be interviewed by telephone.
The data obtained was then processed and categorised based on the topics
studied. After that, the data was analysed, interpreted, dialogued and discussed
using the theories and previous studies relevant to the research topic. The
conclusions and recommendations are based on the results of this study.

4.1. Student’s Responses to Online Learning during the Covid-19 Pandemic
Since the Covid-19 pandemic has exploded in various countries, this outbreak
has disrupted many sectors including that of education. In Indonesia, the
ongoing pandemic has a major impact and influence on the education sector.
Schools and universities are temporarily closed to prevent widespread
transmission. The government has then decided to replace the conventional
learning system with learning from home using online media.
This study chose college students living in Madura, East Java as the respondents.
The responses varied, ranging from liking to disliking online learning. While
there is an ease in terms of the information technology that can be used to
optimise learning, it turns out that online learning in some of the regions in
Indonesia requires more serious handling to create a better learning climate. In
fact, several Indonesians are not accustomed to using online media or digital
media, including in the setting of education. The content presented on social
media has not been able to be absorbed or even used to optimise education. So
far, education in some of the regions of Indonesia is still conventional where the
teachers explain things in front of the class. This condition strongly correlates
with the gaps and inequalities found in the facilities in the country. Learning in
Jakarta (capital of Indonesia) may be better and run well given all of the
available facilities but learning in the outer and remote areas such as Papua,

240
Maluku and the tip of Nusa Tenggara cannot enjoy such effective facilities and
learning.
With the Covid-19 outbreak, people have been required to become able to follow
the system set up by the government. In addition to learning through online
media, students are also given access to learning through national television
channels. However, this option is intended for elementary school students while
the other levels depend on learning through the internet. Therefore society has
no choice other than to follow the lessons on digital media.
The responses given by the participants regarding online learning varied. There
were respondents who considered e-learning to be more effective and efficient
because it has no time and place constraints. Table 1 shows several of the
reasons why students like online learning. During the pandemic, they really
took advantage and felt that the use of online media was very beneficial. This is
also in accordance with Kaler's (2012) theory that students can lessen their
boredom due to conventional learning. They are challenged by online learning.
Table l: Reasons for Preferring Online Learning (N=274)

Reasons for Preferring Online Learning Frequency Percentage
More effective 40 14.60%
More efficient 69 25.18%
Easier to conduct 162 59.13%
Easier to understand the learning material 3 1.09%
Total 274 100.00%
Source: Primary data
Although online learning is the best choice and it is very beneficial for some
students, there were respondents who claimed that e-learning is less enjoyable.
One reason why they do not prefer e-learning is because they cannot afford to
access the internet. It should be noted that not all Indonesians enjoy internet
facilities. As said by Rin (age 20, a college student):
“Apart from being a pandemic reason, online learning also seems to be
easier to put into practice. We have more study time and we can study
freely at any time. If in class I only have to pay attention to the lecturers'
explanations, with online learning, I have a lot of time reading books”.
In this study, the college student respondents came from various regions in
Madura Island. For some of the respondents from rural and remote areas, using
the internet facilities is still difficult. Moreover, most of the students choose to go
back to their hometowns and leave their rented house in the campus
environment. While they can access Wi-Fi and a fast internet network in the
campus environment and rented houses, it is different when they return to their
homes that are often located in rural areas. Apart from having to buy an internet
quota, internet access in some of the applications requires a large quota.
Therefore these factors inhibit the students from accessing e-learning, especially
those who come from lower-middle income families.

241
Table 2 shows that e-learning in Indonesia still faces some obstacles. The
responses given by the respondents varied in terms of their dislike of online
learning. Apart from being limited to internet access, they also feel that online
learning is not as easy as conventional learning. Due to the habits of
conventional learning, many people consider the lessons given through e-
learning to be less optimal.
Table 2: Reasons for Not Preferring Online Learning (N= 274)

Reasons for Not Preferring Online Learning Frequency Percentage
Too high of a cost to access the internet 37 13.50%
Cannot understand the lessons through e- 88 32.12%
learning because it is limited
Uncomfortable because not face to face 49 17.88%
Burdens students with assignments 100 36.50%
Total 274 100.00%
Students are accustomed to conventional learning that involves listening to their

lecturers speaking in front of the class and writing on the board. This condition
is not found during e-learning. Although applications such as Zoom offer share
screen features, it cannot make the students feel as comfortable as studying in
the conventional way does. In addition, there were also respondents who stated
that online learning cannot take place optimally because it is different from face-
to-face meetings. This is influenced by the habits of a society that still relies on
face-to-face communication.
Some of the previous studies conducted in developed countries that have

adopted online learning first, such as in the United States, initially experienced a
number of doubts regarding cost, convenience, flexibility, and a lack of
confidence in the abilities of the students. But at the end in 2011, education in
America was innovated after considering the aspects of the delivery model,
educational curriculum, the role of the faculty, student recruitment, and student
support (Clerkin & Simon, 2014). Meanwhile, research by Moore, Dickson-Deane
and Galyen (2011) found that online learning systems and conventional learning
in Europe and America have been run well.
Research by Crick et al. (2020) noted that the online learning switch that
occurred during the Covid-19 pandemic suggests a much more positive attitude.
Learning has more leverage and it is done more efficiently. However, this study
also noted a number of concerns that could shift the role of the educational
institutions, leading to a changing workload and job fragility.
Ahmad, Sosa and Musfy (2020) stated that online learning is indeed beneficial.
But, on the subject of architecture and design, online learning faces major
obstacles. The design of the architectural education institutions used in the
United Arab Emirates during the Covid-19 pandemic experienced difficulties in
terms of the teaching design without contacting the students. In addition, they

242
feel that online learning is something that is forced on students. In contrast to

Western societies that are accustomed to distance learning, the present study
shows that some people on Madura Island still consider that distance learning
makes it difficult to improve the quality of education. As said by Mar (age 21, a
college student):
“Online learning imposes a variety of assignments on us. Study time is
also not the same as on campus, so to make up for that shortcoming, our
lecturers give more assignments than usual”.
Distance learning is also interpreted as "self-study" by some of the students and

instructors. For some of the lecturers and teachers, giving additional
assignments is their way of optimising learning. This is considered to be a
substitute material for the learning that takes place face to face. However, many
people judge that this kind of learning is placing a burden on students. Online
learning is judged not to provide convenience for students. Instead, they are
burdened by the variety of assignments given to them by their lecturers.
On the other hand, the new methods applied during the outbreak were
responded to by various respondents. Not all of the students like online learning
that relies on certain applications. Some of them still like the conventional
learning methods. As shown in Table 3, there are several methods that are
preferred by the students on Madura Island. A study conducted by Kobayashi
(2017) found that not all students understood and liked all of the different types
of online learning application available. Some of the students do not prefer rich
learning media. They prefer lean media in the form of online slide presentations
recorded using internet-based audio.
Based on constructivism theory, online learning provides opportunities for the

students to explore more knowledge independently. This can be seen from the
students in Madura who stated that online learning provides freedom in
learning and the students find it easy to access learning. They stated that their
learning is efficient because it is not obstructed by place and time, which means
that the students could more freely participate in learning.
Table 3: Preferred Method for Implementing Online Learning (N-274)

Preferred method for implementing online Frequency Percentage
learning
PowerPoint 97 35.57%
Module 45 16.33%
Video 122 44.55%
Assignment/Quiz 10 3.55%
Total 274 100.00%
Table 3 shows that most of the respondents prefer video-based learning

materials. Learning through a video is considered to be an effective and fun
method. Apart from displaying images, video is a tool that can convey
information because it contains both audio and visual elements. Youtube is one

243
of the social media platforms that can be used to optimise online learning. As
stated by Feb (age 38, a lecturer):
“Learning on our university uses Zoom as a medium. Even though sometimes I
feel that there are a lot of challenges, this is quite helpful so that education can
continue during a pandemic”.
On the other hand, Powerpoint is still the learning method favoured by the
respondents. PowerPoint offers an effective and efficient learning system
because the lecturers can summarise the teaching materials used which can then
be shared with students. However, PowerPoint can make the students when it
comes lazy to finding other material or reading original sources.
A study conducted by Abrami et al (2011) stated that there are three types of
interactions in online learning, namely student-student interactions, student-
teacher interactions and student-content interactions. Student-content
interactions include reading informational texts, using study guides, watching
learning videos, interacting with multimedia, participating in simulations and
doing assignments. In an online learning system, the teachers do not monitor as
closely as in conventional learning in schools. However, the teachers are
required to try to provide educational and fun teaching so then the online
learning feels like learning directly from the teacher.
While e-learning abroad is easily accessible, it is different for Indonesians who

live in remote areas who find it difficult. Not all learning in Indonesia can be
conducted through face-to-face teaching and video conferencing. This is because
there are many obstacles in terms of the supporting facilities such as the
availability of personal computers and the internet networks. This makes online
learning not so applicable in several regions in Indonesia, including in Madura.
Previous studies have explained that there is no significant difference between

online learning and conventional learning (face-to-face). Online interaction
fosters closeness and a better use of time to interact than face-to-face interactions
(Fortune, Spielman & Pangelinan, 2011; Schoenfeld-Tacher, McConnell, &
Graham, 2001; Rovai & Jordan, 2004). However, this is clearly different from the
results of this study. The respondents stated that conventional learning (face-to-
face) was better than online learning. This is inseparable from the various
problems that we have mentioned above.
The use of digital media as an online learning tool can replace conventional
learning. The online learning in this study is in accordance with Rusman's
theory that online learning also implements interactivity, accessibility, and
enrichment. Although most learning is done through Zoom videos, learning on
Madura Island is still trying to be maximised. This is also in accordance with
Heinich's theory that lecturers and educational institutions really need to
consider what media is appropriate to use amidst the various obstacles.

244
Table 4: Learning System Preferred by the Students (N=274)

Learning System Preferred by the Frequency Percentage
Students
E-learning 4 1.46%
Conventional learning 270 98.54%
Total 274 100.00%
Table 4 shows that most of the students prefer conventional learning rather than
e-learning. This strengthens the assumption that certain regions in Indonesia,
including Madura, are not ready for online-based learning. Conventional
learning is still an optimal system because it has been used for years. Although
the students are young people who are familiar with the latest gadgets and
information technology, in reality, they have not been able to accept online
learning well.
When examined further, the respondents in this study had important responses
such as an inability to take advantage of the available opportunities properly.
The education system around the world is slowly changing, with all of the
changes being increasingly influential for various countries, including Indonesia.
Online learning has been used in various countries around the world. Even one
of the universities in Indonesia, the Indonesian Open University, has been using
this method for a long time. However, this learning system faces many obstacles
to the goal of being adopted by many students living in rural or remote areas.
A study conducted by Moran, Seaman and Tinti-Kane (2011) stated that digital
learning media creates a business space for technology development companies
that can offer effectiveness and efficiency through the technology used for
learning to form a collaboration. Unfortunately, this has not been well-utilised
by society. Various social media platforms and other applications are still
limited in their use for communication.
On the other hand, the use of online media, which is expected to be an

alternative to conventional learning, has not been able to be interpreted as a
means of supporting the learning process. In online learning, students are asked
to concentrate on the tasks given by the teacher. However, the use of the internet
bothers them because they are involved in social media and other applications
that make them want to play rather than learn (Hollis & Was, 2016). This
happens in all students from low levels of education through to tertiary
education. On one occasion, the students forgot to study and chose to open other
applications such as online games and social media.
Previous studies have suggested that digital learning media also uses social
media applications such as Facebook, Twitter, Tumblr and Snapchat, which
were originally social networks but are now fully utilised. This is because many
people use the internet (Selwyn & Stirling, 2015; Veletsianos, 2015; Junco,
Heiberger & Loken, 2010). This indicates that the synergy between the students
and teachers can optimise the learning process. It should be noted that learning

245
media is a form of social media application that has the user’s characteristics.
Facebook, for example, is used by baby boomers while Twitter is used by people
of all ages. This indicates that teachers and students can collaborate when
utilising social media so as to create a familiar learning space.
These things are different in Indonesia. Several universities in Indonesia provide

a special platform to accommodate the students and lecturers so then they can
meet in one place (the platform). The students and lecturers are required to use
platforms that have been provided by the policy makers at the university. On
another occasion, they used the Zoom application to discuss the lesson. This is
different from other countries that use the latest social media as a discussion
room. Some of the lecturers in Indonesia including Madura do not have social
media, so this becomes an obstacle when the learning system is changed as
suddenly as it happened during the Covid-19 pandemic.
Furthermore, the students' dislike of learning online is because the students are
not familiar with online learning. Through online learning, new behaviours are
formed. This is in accordance with behaviourism learning theory where learning
creates behaviour change. In the end, students do not depend on the lecturers'
explanations and they actively seek knowledge and develop insights through
the digital media. On the other hand, this also relates to the theory of
connectivist learning where digital learning also requires connectivity to achieve
the learning goals.
Table 5: Emerging Impacts Related to Online Learning (N=274)

Emerging Impacts Related to Online Learning Frequency Percentage
Online learning is less effective 194 70.80%
Overall learning experience is distracted 80 29.20%
Total 274 100.00%
Table 5 shows that the respondents admitted that online learning has weak
points for them. Online learning is considered to be less effective because the
students are accustomed to using conventional media. In addition, online
learning is still considered not to be the best solution during the Covid-19
pandemic. Those who feel the convenience of face-to-face learning cannot
completely turn to online learning suddenly. As stated by Nur (age 49, a
lecturer):
“Online learning brings challenges for lecturers. Usually we can discuss in class
for a long time and in an effective atmosphere. But online learning feels different
because the discussion doesn't work as naturally as conventional learning”.
Although online media facilitates student learning, the online presence still
encounters obstacles because the process is not like a face-to-face meeting, and
therefore it makes use of online discussion facilities, wikis, and a blog system
(Yang et al., 2016). Some lecturers use the public discussion rooms available on
several platforms. Lecturers who master the various social media platforms
easily adapt and follow a changing learning system. Those who are tech savvy

246
use social media like Youtube and Twitter as learning media. A study conducted
by Batt and Cummins (2016) concluded that the social media platforms that can
be used as digital learning media are Youtube and Facebook. This is because
they are easy to use and familiar to almost everyone. In addition, education
practitioners adopt social media to form an online community, especially on
Twitter where online learning is easily digested because of the tweet/retweet
facilities that are able to reach many people up to the point of trending (Shen,
Kuo, & Minh Ly, 2017).
The previous studies have explained that online learning media is easily
accessible because almost every student or parents has a smartphone or laptop.
They use networking and social media as an online learning method (Hamilton
et al., 2016; Melhuish & Falloon, 2010; Parker et al., 2011). The present study
shows that some of the remote areas in Indonesia including Madura face many
obstacles to implementing the online learning policies as confirmed by the
government. In this case, this government needs to make a pretty hard effort to
catch up with the learning system. The Covid-19 outbreak has also become a
form of evaluation for the education system in Indonesia.
Davies and Graff's study (2005) found that a greater level of online interaction
does not lead to better performance when it comes to achieving final grades. The
students who fail at learning tend to interact less and are less able to utilise the
digital learning media. This kind of interaction shows that online media cannot
yet be positioned as an optimal learning method, particularly as there are those
who are passive to the learning method itself. Social media such as Reddit can
also be used as an online learning media in which there are questions and
answers, debates, corrections and presenting information to other users
(Haythornthwaite et al., 2018).
4.2. Problems Experienced by the Students during Online Learning

Online learning is more flexible despite the limited supporting facilities. The
lecturers and students are not bound by time and place when it comes to
discussing the lessons. Although they do not meet using livestreaming, the
lecturers can share explanatory videos or study materials in the form of text
through e-mails or other applications. Online learning makes it easy for the
teachers to send material anywhere at any time, and the students can also follow
the learning process anywhere and at any time while connected to the internet
(Wardoyo, 2016; Bourne, Harris, &Mayadas, 2005). This learning can be done in
Madura but it is indeed necessary to make even better efforts to implement this
policy.
A study conducted by Marks, Sibley and Arbaugh (2005) stated that various
online media that can be used as learning tools such as streaming media,
PowerPoint and hyperlinks. They also found out the factors influencing the
quality of online learning such as the quality of the distance learning (due to
work, family flexibility and geography) and the student’s personal
characteristics (experience and gender). In addition to PowerPoint, another
media used as a learning tool is Prezi, one of the applications that is similar to

247
PowerPoint that qualifies as a learning medium because Prezi's validity is

secure, practical, and effective (Hartini et al., 2017; Akgün, Babur, & Albayrak,
2016; Rodhi & Wasis, 2014). The convenience offered by online learning actually
has a big impact on both the students and teachers who are truly able to make
the most of it, particularly due to the vast choice of streaming applications that
are currently spreading across various platforms. Wikan Sakarinto, I Made Andi
Arsana and Agus Affianto are some of the teachers from Indonesia who have
used social media a lot, so that they are close to their students. Wikan Sakarinto,
for example, has a Youtube channel that can be accessed by his students relating
to the study materials.
Furthermore, there are also problems with online learning among the families of
the students. At the time of the Covid-19 outbreak, the government had not yet
blocked inter-city mobility. The students chose to return to their hometowns
rather than to stay in their rental houses. In the end, they had to learn from
home. However, this becomes a problem for some students as the atmosphere at
home is not the same as where they live in the rental house near campus.
Staying at home means that they do more homework and help their parents at
work, so they sometimes neglect their tasks. In this case, the family has an
influence on the learning process of the students.
A study conducted by Yeboah and Smith (2016) explained that satisfaction and
the use of social media have no relationship with the participants’ academic
performance. However, there is an indication that flexibility and the
convenience of time, self-confidence, a lack of support, independent learning
skills and language/linguistic differences can affect the way that students learn.
In the case of the learning system in Indonesia, the community still considers
social media to be a playground. This is influenced by the old assumptions that
learning should involve books and that books are a source of knowledge.
Meanwhile, education has experienced a shift in values and the methods of
learning used from the point when e-books were first introduced. However,
people are still comfortable using conventional media compared to electronic
devices as a learning support.
Therefore the new learning system is not accompanied by a new perspective. At

a time when the development of education in various countries is advancing
rapidly, rural and remote communities are still faced with technical problems.
This subsequently causes a problem in that online learning still cannot be
utilised as an alternative to conventional learning.
In addition, society in some of the rural and remote regions of Madura Island
still cannot understand that learning can take place anywhere and at any time.
They still cannot "let go" of the teacher figure. For traditional societies, the
teacher becomes a perfect example when they appear in front of the class,
wearing a uniform, explaining the lesson, and interacting with the students face
to face. This condition is different from the other countries that present teachers
in a digital form. Even though Indonesia has developed an online-based
academic course platform, this application has not yet attracted a large number

248
of students. As shown in Table 6, the students feel uncomfortable using online

media to access learning material for various reasons.
Table 6: Reasons for online learning not benefiting the students (N=274)
Reasons for online Gender Frequency Percentage
learning not Female Male
benefiting the
F % F %
students
The lecturer cannot 40 14,60% 30 10,95% 70 25.48%
explain things directly
in the class physically
Students are 53 19,35% 21 7,66% 74 26.83%
burdened with
assignments
Need to buy an 27 9,85% 38 13,87% 65 23.90%
internet quota
Signal is difficult to 23 8,39% 42 15,33% 65 23.79%
access in certain areas
Total 274 100 %
Source: Primary Data
Table 6 shows that for some of the respondents, conventional learning has not
been able to be replaced by other methods. Online learning is considered to be
something that is not yet effective and efficient, so they feel that online learning
does not benefit them at all. The students are not familiar with the use of online
learning media and they also feel that online learning is increasingly
overwhelming the students with many tasks. Meanwhile, the male students
revealed that the signal is difficult to access in remote areas. Rahma (age 21,
female student) said the following:
“There are many assignments given by lecturers. Even though during
the pandemic I am at home and have to help my parent do household
chores because I am girl”.
Indonesian society adheres to a patriarchal culture where there is a sexual

division of labour. As revealed by Rahma, she felt burdened by the domestic
chores that were imposed on her because she was a woman. Therefore she felt
that the large number of lecture assignments from the lecturers increased her
life burden. This is in contrast to men who are generally free from domestic
work. As expressed by the parents of Andi (age 20, student) and Dony (age 20,
student), the male students complained more about the internet signal and
quota fees that had to be bought.
“But the problem is the internet signal. The signal is sometimes not smooth,
because our house is in the remote area”. (Dew, age 55, parent of Andi).
“My child buys internet quota more often. Online learning looks simple, but it
costs a lot for the internet”. (Sur, 57, parent of Dony).
A study conducted by Stiller and Köster (2016) found that the burden
experienced by students when using online media as a learning tool includes
multidimensional learning task models, differences in technological

249
understanding, and a lack of knowledge of how to use online media. Since there
is a learning policy at home, it makes the teachers unable to provide direct
material and control their students, so the learning process is left up to the
parents and the teacher will make an evaluation afterwards. This is not
appropriate in some communities in Indonesia, especially among the lower-
middle class or upper class where the parents are very busy at work. They
therefore do not pay attention to their children's learning hours.
Previous studies have explained that one of the supportive factors of online
learning is the use of infographics. These are visualisation tools used to increase
retention and understanding in terms of student attraction. This is because
infographics are a common visual medium used for presenting the learning
material (Elena-Gallagher et al., 2017; Matrix & Hodson, 2014; Akhamd et al.,
2017). Unfortunately, not all teachers and lecturers can present infographics, so
their online learning seems stiff and monotonous.
The Covid-19 outbreak has had an important impact on education in Indonesia.

Previous conventional learning has turned into online learning in order to
prevent the spread of the virus. The policy applied in all schools and universities
in Indonesia still faces a number of obstacles. Although almost all regions in
Indonesia are covered by internet services, the community still faces obstacles
when accessing online education.
Not all college students like online learning because of the ineffective methods
involved. Social, economic and cultural factors are important indicators that
online learning has not been able to be applied in a number of regions in
Indonesia including Madura Island. In addition, the community is not ready to
welcome the latest learning methods and it is still comfortable using the old
methods.
5. Conclusions and Implications

This study concludes that the online learning policies in some regions of
Indonesia are still encountering a number of obstacles. Not only are these
technical obstacles such as the availability of learning facilities but also the social
and cultural conditions of Indonesian society that have not been able to follow
this new system quickly. It takes time and consistent practice. This policy needs
to be supported by the provision of facilities both technical and in the form of
human resources in order to make the education system in Indonesia even
better.
The research implications are good starting points to use to conduct similar
studies determining the appropriate learning delivery modality in such trying
times. Online learning is a necessity especially in difficult times like the current
Covid-19 pandemic. Efforts are needed to maximise the progress of learning.
Educational institutions should pay attention to the fact that online learning
requires internet access in order for the student to be connected. Meanwhile,
there are many students in Indonesia who are constrained by the cost of doing
this, including students on Madura Island. In addition, not all areas on Madura

250
Island can receive an internet signal well. There are remote areas that experience
a limited internet signal. This is what hampers online learning.
Because they are not used to it, students find it difficult to study online. In this
case, the lecturers and students can innovate to create a learning atmosphere
that is not boring and effective in terms of the knowledge transfer method used.
Changes in the behaviour of both lecturers and students is needed so then the
educational goals can be optimally realised. In addition, educational institutions
also need to make their distance learning innovations more effective and
efficient.
This study produce recommendations regarding online learning policies. The

Ministry of Education and Culture of Indonesia needs to pay attention when
dealing with the problems of access to education in rural and remote areas. The
central and regional governments should provide adequate facilities and access
to education, especially in rural areas inclusive of financial assistance, especially
for the poor who have difficulty accessing the internet.
6. References
Abrami, P. C., Bernard, R. M., Bures, E. M., Borokhovski, E., & Tamim, R. M. (2011).
Interaction in distance education and online learning: using evidence and theory
to improve practice. Journal of Computing in Higher Education, 23(2-3), 82–103.
Agostinho, S., Bennett, S., Lockyer, L., & Harper, B. (2011). The future of learning design.
Learning, Media and Technology, 36(2), 97–99.
Ahmad, L., Sosa, M., & Musfy, K. (2020). Interior Design Teaching Methodology During
the Global COVID-19 Pandemic. Interiority, 3(2), 163-184. doi:10.7454/in.v3i2.100
Akgün, O. E., Babur, A., & Albayrak, E. (2016). Effects of lectures with Powerpoint or
Prezi presentations on cognitive load, recall, and conceptual learning.
International Online Journal of Educational Sciences, 8(3), 1-11.
Annad, D. (2008). Learning efficacy and cost-effectiveness of print versus e-book
instructional material in an introductory financial accounting course. Journal of
Interactive Online Learning, 7(2), 152-164.
Arista, F. S., & Kuswanto, H. (2018). Virtual physics laboratory application based on the
android smartphone to improve learning independence and conceptual
understanding. International Journal of Instruction, 11(1), 1-16.
Batt, A. M., & Cummins, N. M. (2016). E-learning on the road: online learning and social
media for continuous professional competency. Irish Journal of Paramedicine, 1(1),
1-9.
Bolliger, D. U., Supanakorn, S., & Boggs, C. (2010). Impact of podcasting on student
motivation in the online learning environment. Computers & Education, 55(2),
714–722.
Bound, H. (2010). Developing quality online dialogue: Dialogical inquiry. International
Journal of Teaching and Learning in Higher Education, 22(2), 107-119.
Bourne, J., Harris, D., & Mayadas, F. (2005). Online engineering education: Learning
anywhere, anytime. Journal of Engineering Education, 94(1), 131–146.
Brown, C., Czerniewicz, L., & Noakes, T. (2015). Online content creation: looking at
students’ social media practices through a Connected Learning lens. Learning,
Media and Technology, 41(1), 140–159.
Brownson, S. M. (2014). Embedding social media tools in online learning courses. San
Francisco, USA: The Clute Institute International Business & Education
Conferences.

251
Chau, C. (2010). YouTube as a participatory culture. New Directions for Youth

Development, 2010(128), 65–74.
Clerkin, K., & Simon, Y. (2014). College for America: Student-Centered, Competency-
Based Education. Change: The Magazine of Higher Learning, 46(6), 6–13.
doi:10.1080/00091383.2014.969141
Crick, T., Knight, C., Watermeyer, R., & Goodall, J. (2020). The Impact of COVID-19 and
“Emergency Remote Teaching” on the UK Computer Science Education
Community. UKICER '20: United Kingdom & Ireland Computing Education Research
Conference, 31-37. doi:10.1145/3416465.3416472
Dabbagh, N., & Kitsantas, A. (2012). Personal Learning environments, social media, and
self-regulated learning: A natural formula for connecting formal and informal
learning. The Internet and Higher Education, 15(1), 3–8.
Davies, J., & Graff, M. (2005). Performance in e-learning: online participation and student
grades. British Journal of Educational Technology, 36(4), 657–663.
Elena-Gallagher, S., O’Dulain, M., O’Mahony, N., Kehoe, C., McCarthy, F., & Morgan, G.
(2017). Instructor-provided summary infographics to support online learning.
Educational Media International, 54(2), 129–147.
Farooq, M. U., & Javid, C. Z. (2012). Attitude of students towards E-learning: A study of
English language learners at Taif University English Language Centre. NUML
Journal of Critical Inquiry, 10(2).
Fortune, M. F., Spielman, M., & Pangelinan, D. T. (2011). Students’ perceptions of online
or face-to-face learning and social media in hospitality, recreation and tourism.
MERLOT Journal of Online Learning and Teaching, 7(1), 1-16.
Friedman, L. W., Friedman, H. H. (2020). Using social media technologies to enhance
online learning. Journal of Educators Online, 10(1), 1-22.
Greenhow, C., & Lewin, C. (2015). Social media and education: reconceptualizing the
boundaries of formal and informal learning. Learning, Media and Technology,
41(1), 6–30.
Greenhow, C., & Robelia, B. (2009). Informal learning and identity formation in online
social networks. Learning, Media and Technology, 34(2), 119–140.
Gutmann, J., Kühbeck, F., Berberat, P. O., Fischer, M. R., Engelhardt, S., & Sarikas, A.
(2015). Use of learning media by undergraduate medical students in
pharmacology: A prospective cohort study. PLOS ONE, 10(4), e0122624.
doi:10.1371/journal.pone.0122624
Hamilton, L. A., Franks, A., Heidel, R., McDonough, S. K. L., & Suda, K. J. (2016).
Assessing the value of online learning and social media in pharmacy education.
American Journal of Pharmaceutical Education, 80(6), 1-6.
Hansch, A., Hillers, L., McConachie, K., Newman, C., Schildhauer, T., & Schmidt, P.
(2015). Video and online learning: Critical reflections and findings from the field.
SSRN Electronic Journal. doi:10.2139/ssrn.2577882
Hartini, S., Misbah, Dewantara, D., Oktovian, R.A., & Aisyah, N. (2017). Developing
learning media using online Prezi into materials about optical equipments.
Jurnal Pendidikan IPA Indonesia, 6(2), 313-317.
Hartsell, T., & Yuen, S. C. Y. (2006). Video streaming in online learning. AACE Journal,
14(1), 31-43.
Haythornthwaite, C., Kumar, P., Gruzd, A., Gilbert, S., Esteve del Valle, M., & Paulin, D.
(2018). Learning in the wild: coding for learning and practice on Reddit.
Heinich, R., Molenda, M., Russell, J. D., & Smaldino, S. E. (1982). Instructional Media and
Technologies for Learning (7th Edition). New Jersey: Prentice – Hall Inc.
Herayanti, L., Fuaddunnazmi, M., & Habibi. (2015). Pengembangan media pembelajaran
berbasis Moodle pada mata kuliah fisika dasar [Development of Moodle-based

252
learning media in basic physics courses]. Jurnal Pendidikan Fisika dan Teknologi,
1(3), 205-209.
Hollis, R. B., & Was, C. A. (2016). Mind wandering, control failures, and social media
distractions in online learning. Learning and Instruction, 42, 104–112.
Horzum, M. B. (2015). Interaction, structure, social presence, and satisfaction in online
learning. Eurasia Journal of Mathematics, Science & Technology Education, 11(3), 505-
512.
Irwansyah, F. S., Yusuf, Y. M., Farida, I., & Ramdhani, M. A. (2017). Augmented Reality
(AR) technology on the android operating system in chemistry learning. The 2nd
Annual Applied Science and Engineering Conference (AASEC 2017), 24 August 2017
(Vol. 288). Bandung, Indonesia. doi:10.1088/1757-899X/288/1/012068
Junco, R., Heiberger, G., & Loken, E. (2010). The effect of Twitter on college student
engagement and grades. Journal of Computer Assisted Learning, 27(2), 119–132.
Kaler, C. B. (2012). A Model of Successful Adaptation to Online Learning for College-
Bound Native American HIgh School Students. Multicultural Education &
Technology Journal, 6(2), 60-76. doi:10.1108/17504971211236245
Kirkwood, A., & Price, L. (2013). Technology-enhanced learning and teaching in higher
education: what is “enhanced” and how do we know? A critical literature
review. Learning, Media and Technology, 39(1), 6–36.
Kobayashi, M. (2017). Students’ media preferences in online learning. Turkish Online
Journal of Distance Education, 18(3), 4-15.
Kock, N., Verville, J., & Garza, V. (2007). Media naturalness and online learning:
Findings supporting both the significant- and no-significant-difference
perspectives. Decision Sciences Journal of Innovative Education, 5(2), 333–355.
Kruse, N. B., & Veblen, K. K. (2012). Music teaching and learning online: Considering
YouTube instructional videos. Journal of Music, Technology and Education, 5(1),
77–87.
Kuang-Chih L., & Kriegman, D. (n.d.). Online learning of probabilistic appearance
manifolds for video-based recognition and tracking. 2005 IEEE Computer Society
Conference on Computer Vision and Pattern Recognition (CVPR’05), 20-25 June 2005.
San Diego, USA. doi:10.1109/cvpr.2005.260
Lin, C. C., & Polaniecki, S. (2009). From media consumption to media production:
Applications of YouTube™ in an eighth-grade video documentary project.
Journal of Visual Literacy, 28(1), 92–107.
Liu, Y. (2005). Effects of online instruction vs. traditional instruction on students’
learning. International Journal of Instructional Technology and Distance Learning,
2(3), 57-64.
Madge, C., Meek, J., Wellens, J., & Hooley, T. (2009). Facebook, social integration and
informal learning at university: “It is more for socialising and talking to friends
about work than for actually doing work.” Learning, Media and Technology, 34(2),
141–155.
Marks, R. B., Sibley, S. D., & Arbaugh, J. B. (2005). A structural equation model of
predictors for effective online learning. Journal of Management Education, 29(4),
531–563.
Martono, K. T., & Nurhayati, O. D. (2014). Implementation of android based mobile
learning application as a flexible learning media. International Journal of Computer
Science Issues, 11(3), 168-174.
Matrix, S., & Hodson, J. (2014). Teaching with infographics: practising new digital
competencies and visual literacies. Journal of Pedagogic Development, 3(2), 17-27.
Megan, P. (2015). ’Create a better online you’: Designing online learning resources to
develop undergraduate social media skills. In Field, R & Nelson, K (Eds.)
Proceedings of the 2015 Students Transitions Achievement Retention and Success

253
(STARS) Conference, (pp. 1-5). The Office for Learning and Teaching (OLT),
Australia.
Moghavvemi, S., Sulaiman, A., Jaafar, N. I., & Kasem, N. (2018). Social media as a
complementary learning tool for teaching and learning: The case of youtube. The
International Journal of Management Education, 16(1), 37–42.
Moore, J. L., Dickson-Deane, C., & Galyen, K. (2011). e-Learning, online learning, and
distance learning environments: Are they the same? The Internet and Higher
Education, 14(2), 129–135. doi:10.1016/j.iheduc.2010.10.001
Moran, M., Seaman, J., & Tinti-Kane, H. (2011). Teaching, learning, and sharing: How
today’s higher education faculty use social media. Babson Survey Research Group.
Nacu, D. C., Martin, C. K., Pinkard, N., & Gray, T. (2014). Analyzing educators’ online
interactions: a framework of online learning support roles. Learning, Media and
Technology, 41(2), 283–305.
Peppler, K. A., & Kafai, Y. B. (2007). From SuperGoo to scratch: Exploring creative digital
media production in informal learning. Learning, Media and Technology, 32(2),
149–166.
Power, M. (2008). The emergence of a blended online learning environment. MERLOT
Journal of Online Learning and Teaching, 4(4), 503-514.
Prastiyo, W., Djohar, A., & Purnawan. (2018). Development of Youtube integrated
Google Classroom based E-learning media for the light-weight vehicle
engineering vocational high school. Jurnal Pendidikan Vokasi, 8(1), 53-66.
Putranti, N. (2013). Cara membuat media pembelajaran online menggunakan Edmodo
[How to make online learning media using Edmodo]. Jurnal Pendidikan
Informatika dan Sains, 2(2), 139-147.
Rodhi, M. Y., & Wasis. (2014). Pengembangan media pembelajara berbasis Prezi untuk
meningkatkan keterampilan berpikir kritis pada materi kalor [Development of
Prezi-based learning media to improve critical thinking skills on heat material].
Jurnal Inovasi Pendidikan Fisika (JIPF), 3(2), 137-142.
Rovai, A. P., & Jordan, H. (2004). Blended learning and sense of community: a
comparative analysis with traditional and fully online graduate courses. The
International Review of Research in Open and Distributed Learning, 5(2).
https://doi.org/10.19173/irrodl.v5i2.192.
Rusman. (2011). Pembelajaran Berbasis Teknologi Informasi dan Komunikasi, Mengembangkan
Profesionalitas Guru [Information and Communication Technology Based Learning,
Developing Teacher Professionalism]. Jakarta: PT. Rajagrafindo Persada.
Sagala, S. (2008). Konsep dan Makna Pembelajaran [Learning Concepts and Meanings].
Bandung: Alfabeta.
Salmon, G., Ross, B., Pechenkina, E., & Chase, A.M. (2015). The space for social media in
structured online learning. Research in Learning Technology, 23, 1-14.
Selwyn, N. (2009). Faceworking: Exploring students’ education‐related use of Facebook.
Selwyn, N., & Stirling, E. (2015). Social media and education … now the dust has settled.
Schoenfeld-Tacher, R., McConnell, S., & Graham, M. (2001). Journal of Science Education
and Technology, 10(3), 257–265.
Shen, C., Kuo, C. J., & Minh Ly, P. T. (2017). Analysis of social media influencers and
trends on online and mobile learning. The International Review of Research in Open
and Distributed Learning, 18(1). doi:10.19173/irrodl.v18i1.2640
Stiller, K. D., & Köster, A. (2016). Learner attrition in an advanced vocational online
training: The role of computer attitude, computer anxiety, and online learning
experience. European Journal of Open, Distance and E-Learning, 19(2), 1–14.
Susan, M., & Tony. (1995). Learning to Teach in The Secondary School. London: Routledge.

254
Tan, E. (2013). Informal learning on YouTube: Exploring digital literacy in independent

online learning. Learning, Media and Technology, 38(4), 463–477.
Thoms, B., & Eryilmaz, E. (2014). How media choice affects learner interactions in
distance learning classes. Computers & Education, 75, 112–126.
Thomson, A., & Bridgstock, R. (2014). ‘Teachers flipping out’ beyond the online lecture:
Maximising the educational potential of video. Journal of Learning Design, 7(3),
67-78.
Trianto. (2007). Model-Model Pembelajaran Inovatif Berorientasi Kontstruktivistik. Konsep,
Landasan Teoritis-Praktis dan Implementasinya [Constructivist Oriented Innovative
Learning Models. Concept, Theoretical-Practical Basis and Its Implementation].
Jakarta: Prestasi Pustaka.
Veletsianos, G. (2011). Higher education scholars’ participation and practices on Twitter.
Journal of Computer Assisted Learning, 28(4), 336–349.
Wardoyo, C. (2016). Developing learning media based on e-learning on accounting
subject for senior high school students. Dinamika Pendidikan, 11(2), 84-93.
Whittaker, A. L., Howarth, G. S., & Lymn, K. A. (2014). Evaluation of Facebook to create
an online learning community in an undergraduate animal science class.
Educational Media International, 51(2), 135–145.
Yang, J. C., Quadir, B., Chen, N. S., & Miao, Q. (2016). Effects of online presence on
learning performance in a blog-based online course. The Internet and Higher
Education, 30, 11–20.
Yeboah, A. K., & Smith, P. (2016). Relationships between minority students online
learning experiences and academic performance. Online Learning, 20(4), 1-26.
Yuan, L., Powell, S., & Olivier, B. (2014). Beyond MOOCs: Sustainable online learning in
institutions. USA: Centre for Educational Technology, Interoperability and
Standards.

255

Vol. 19, No. 8, pp. 255-277, August 2020
High School Students’ Difficulties in Making

Mathematical Connections when Solving
Problems
Jailani, Heri Retnawati and Ezi Apino

Universitas Negeri Yogyakarta (Yogyakarta State University),
Yogyakarta, Indonesia
https://orcid.org/0000-0001-5552-255X
https://orcid.org/0000-0002-1792-5873
https://orcid.org/0000-0001-9711-2807
Agus Santoso
Universitas Terbuka (Open University)
South Tangerang, Banten, Indonesia
https://orcid.org/0000-0003-3696-2132
Abstract. The primary purpose of studying mathematics is that students

can solve problems, both mathematical and real-life problems. In this
way, mathematical connections play an essential role in enabling students
to solve mathematical problems. Students’ difficulties in mathematical
connections can cause difficulties in solving problems. This study aims to
describe the mathematical connections difficulties experienced by
students when solving mathematical problems. This study is qualitative
with a phenomenological approach. Data were collected by using
mathematical connection tests and interviews after the test. The
participants of this study were 31 high school students from five schools
in Yogyakarta Special Region and Central Java Province, Indonesia. Data
analysis began with analyzing all students’ answers in solving
mathematical problems and categorizing the types of difficulties
experienced by students. Thematic analysis of the interview data was
conducted to reveal the causes of difficulties experienced by students
when making mathematical connections. The findings showed that most
students experienced difficulties in making mathematical connections,
such as in different representations, part-whole relationships,
connections between mathematical concepts, and interrelationships
between mathematical procedures. Several causes of such difficulties and
further actions were also discussed in this study.
Keywords: mathematical connections; mathematical problem-solving;

students’ difficulties; mathematics learning; phenomenological study

256
1. Introduction
Mathematics is a compulsory subject learned from primary to higher education.
This subject plays an essential role in the advancement and development of
science and technology, and also contributes directly to human survival. Besides,
mathematics is not just arithmetic, but it can also be used to practice a variety of
thinking skills, such as critical thinking (Appelbaum, 2000; Lince, 2016; Suh &
Seshaiyer, 2013), creative thinking (Leikin & Pitta-Pantazi, 2013; Lince, 2016),
logical thinking (Hodge, 2003; Lince, 2016), and higher-order thinking skills
(Apino & Retnawati, 2017, 2019). The importance of mathematics encourages
many countries to keep creating innovations in strategies and approaches to
learning mathematics to make mathematics more understandable and applicable
in real-life situations.
One of the keys to learning mathematics is problem-solving (NCTM, 2000; Palmér

& Van Bommel, 2020; Van Zanten & Van den Heuvel-Panhuizen, 2020). Problem-
solving is in line with the spirit of mathematics as a means to develop thinking
skills. In the context of mathematics education, this problem solving is used to
introduce and familiarize students with how to understand a phenomenon
related to mathematical concepts and things associated with the application of
mathematics in everyday life (Osman et al., 2018). By using problem-solving, the
students are then expected to be able to plan and find solutions to various
problems systematically and logically (Albay, 2019). This ability is fundamentally
important as it can help the students face increasingly complex challenges in life.
Many kinds of literature categorize problem-solving as one of the competencies

that must be possessed for success in the 21st century besides critical thinking,
creativity, collaboration, and communication (Albay, 2019; English & Gainsburg,
2016; Partnership for 21st Century Learning, 2019). Today, problem-solving is no
longer seen as a written skill, but from a broader perspective, it evolves into an
essential skill used to compete in the world of work and even to answer the
challenges of this era. In order to reach this skill, other abilities are needed. As
formulated by NCTM (2000), in mathematics learning standards, another ability
such as mathematical connection must be practiced by students in addition to
problem-solving. NCTM (2000) highlights that the mathematical connection is a
tool for problem-solving. Mathematical connections help students recognize and
use relationships between mathematical ideas and use them in different contexts
(Dolores-Flores, Rivera-López & García-García, 2019). Having strong
mathematical connections will also enhance mathematical understanding (García-
García & Dolores-Flores, 2018; Kenedi, Helsa, Ariani, Zainil & Hendri 2019; Silver,
Mesa, Morris, Star & Benken, 2009) and student achievement (Kartikasari &
Widjajanti, 2017; Ndiung & Nendi, 2018). Hence, making mathematical
connections is necessary for students to be successful in mathematics education.
Mathematical connections are generally associated with three things, namely

connections related to the application of mathematics to real-life contexts (Blum,
Galbraith, Henn & Niss, 2007; Monroe & Mikovch, 1994; Mwakapenda, 2008;
Özgen 2016), mathematical connections with other disciplines (Blum et al., 2007;
Mwakapenda, 2008; Özgen 2016), and connections between mathematical ideas

257
or concepts themselves (Blum et al., 2007; Eli, Mohr-Schroeder & Lee, 2013;
Monroe & Mikovch, 1994; Mwakapenda, 2008). The connection between
mathematical ideas or concepts by Businskas (2008) is referred to as
interconnections in mathematics, while García-García and Dolores-Flores (2018)
mention them as intra-mathematical connections. The ability to make connections
between mathematical ideas or concepts themselves (interconnections or intra-
mathematical connections) is crucial for understanding mathematical concepts
(Anthony & Walshaw, 2009; Berry & Nyman, 2003; García-García & Dolores-
Flores, 2020; Mhlolo, 2012) and for their application to other scientific disciplines
(Mhlolo, 2012; Mhlolo, Venkat & Schäfer, 2012). Thus, interconnection in
mathematics becomes an interesting topic to discuss. Businskas (2008) suggests
that most literature only focuses on examining the connection between
mathematics and real-world situations without exploring how interconnection in
mathematics itself. Besides, García-García and Dolores-Flores (2020) reveal that
there is still little research focused on investigating the mathematical connection
process when students solve mathematics problems. Hence in this study,
mathematical connections are focused on interconnection in mathematics itself
when students solve mathematical problems.
Businskas (2008) suggests a framework for thinking about mathematical

connections in practice. The framework of thinking can be operationally used as
an indicator of mathematical interconnections consisting of five categories. The
categories include (1) different representation as a form of mathematical
connections; (2) part-whole relationships; (3) connections where A implies B; (4)
connections showing that A is a procedure for doing B; and (5) instructional
oriented connection that shows how certain concepts are pre-requisites for
understanding related concepts (Businskas, 2008). García-García and Dolores-
Flores develop a framework similar to Businskas and have implemented it in their
research (e.g., García-García & Dolores-Flores, 2018; 2020). That framework
includes several types of mathematical connections indicators, including the
following: different representations, procedural features, reversibility, and
meaning as a connection. Through these indicators, mathematical connections can
be identified more quickly and the extent to which mathematical connections can
be easily measured.
Some other studies also report that the students’ mathematical connection abilities
at various levels need serious attention. Lapp, Nyman, and Berry (2010) report
that undergraduate students encountered difficulties in making connections
between various concepts, such as the connection between eigenvalues and
eigenvectors in algebra learning. Dolores-Flores et al. (2019) explore the
mathematical connections of pre-university students when solving tasks
involving the rates of change. The result showed that the students only made
mathematical connections of procedural types. In contrast, the students scarcely
made other types of mathematical connections, such as the common features and
the generalization. Siregar and Surya (2017) analyze the junior high school
students’ abilities in mathematical connections using tests of mathematical
connections, and the result showed that their abilities were still low. Similarly,
Kenedi et al. (2019) investigate the elementary school students’ mathematical
connection ability in solving mathematics problems, and the result showed that

258
their abilities were also low. The results of these studies can be used as a basis for
improving the quality of mathematics learning that is oriented towards enhancing
mathematical connections.
In practice, several studies report some obstacles in teaching mathematical

connections. Dolores-Flores et al. (2019) report that a lack of conceptual
understanding and difficulties in manipulating algebra were the main obstacles
in making mathematical connections. Arjudin, Sutawidjaja, Irawan, and Sa’dijah
(2016) also report that students’ difficulties in making mathematical connections
were caused by errors in connecting with conceptual knowledge and procedural
knowledge. Lack of familiarity with the mathematical connection problems also
becomes another obstacle for students in developing their mathematical
connection ability. In this way, Agustini, Suryadi, and Jupri (2017) suggest
familiarizing students with mathematical connection problems by using open-
ended problems. These findings indicate that actions are needed to improve the
quality of mathematics learning, which is oriented towards strengthening
mathematical connection skills.
The starting point for improvement can be made by identifying students’

difficulties in solving mathematics problems (Hadi, Retnawati, Munadi, Apino &
Wulandari, 2018; Rafi & Retnawati, 2018; Wijaya, van den Heuvel-Panhuizen,
Doorman & Robitzsch, 2014). In this context, we view that investigating student
difficulties when making mathematical connections in solving mathematical
problems is urgent. Dolores-Flores et al. (2019) point out that mathematical
connections occur when students carry out specific tasks and, therefore, we can
identify its processes from the writings or arguments produced by the students.
Besides, García-García and Dolores-Flores (2018) argue that students have an
important role in reflection for process and learning improvement related to
mathematical skill connections. Thus, the findings from the results of identifying
what students do in solving mathematics problems that require mathematical
connection skills can be used as a reflection to improve mathematics learning.
Even though there have been many studies related to mathematical connection,
there are still a few studies that are focused on investigating the difficulties
experienced by students in making mathematical connections when solving
mathematics problems. The study of García-García and Dolores-Flores (2018) is
more focused on exploring the types of mathematical connections made by high
school students in performing Calculus tasks. Furthermore, the study of García-
García and Dolores-Flores (2020) also explores the types of mathematical
connections in solving Calculus application problems but involves pre-university.
Payton (2019) conducts a study that focuses on interventions that aim to develop
mathematical connections. Zengin (2019) examines the use of GeoGebra software
for the development of mathematical connections skills. Other studies are
conducted to explore mathematical connections involving various
representations (e.g., García-García & Dolores-Flores, 2018; Mhlolo, 2012; Mhlolo
et al., 2012; Moon, Brenner, Jacob & Okamoto, 2013). Based on this review, we are
motivated to explore students’ difficulties in making mathematical connections,

259
especially when students interconnect mathematical concepts or procedures

when solving mathematics problems.
Based on the urgency and the review that has been put forward, this study aims
to describe the students’ difficulties in making mathematical connections when
solving mathematical problems. We hope that the findings of this study can
provide a framework for educators to improve best practices in mathematics
learning, especially related to the development of students’ mathematical
connection skills.
2. Methods
2.1. Type of Study
This study was classified as phenomenology, a type of qualitative research
methods. The primary aim of phenomenological research was to reduce the
experiences of people with a particular phenomenon to find descriptions of the
universal essence (Creswell, 2013). The phenomenon in this study was about the
students’ difficulties in making mathematical connections, especially in solving
mathematical problems. One common phenomenon which occurred when
students solved mathematical problems was that students already knew a
concept, but they had difficulty in connecting the concept with other
mathematical concepts. This phenomenon often harms their understanding of
mathematical concepts and their academic achievement in learning mathematics.
2.2. Participants
The participants of this study were 31 eleventh grade high school students from
five schools in Yogyakarta Special Region and Central Java Province, Indonesia.
We selected the schools as representatives with high, medium, and low academic
achievement. Six students came from school with high academic achievement, 21
students came from school with medium academic achievement, and 12 students
came from school with low academic achievement. When we conducted the
study, the students involved in this study were between 15 and 17 years old.
2.3. Data Collection

We collected data through tests and interviews. The test consisted of
mathematical problems (items) containing concepts that had been taught at
schools. It comprised of three items, and each of them required a mathematical
connection ability to solve it. We took these items from the college entrance exam
questions in Indonesia that had been released and re-validated. The validity of the
test instrument was proven through content validation by asking experts to agree
on the item’s suitability with the indicators. The experts were two high school
mathematics teachers and two mathematics education lecturers. The indicators of
mathematical connection abilities used in this study included different
representations (DR), part-whole relationships (PWR), connections where A
implies B (IM), and connections showing that A is a procedure for doing B (PD).
We used item 1 to measure the students’ mathematical connection ability to solve
the problem of a two-variable linear equation (see Figure 2). We used item 2 to
measure the students’ mathematical connection ability to solve the problem of a
trigonometric equation (see Figure 3). We used item 3 to measure the students’

260
mathematical connection ability to solve the problem of a circle (see Figure 4).
Before taking the test, we informed students that the test results would not affect
their academic achievement in school, but we asked them to take the test
seriously. The test was not administered in the regular classroom but taken vacant
or available time as agreed by the teacher, students, and researchers in each
school. We gave students 30 minutes to complete the test.
We conducted interviews with the students right after they finished the test. The
interviews aimed to know the students’ perceptions of the problems being tested.
In this case, we asked the students to show which difficult parts of the problem-
solving process they found. The interviews were also focused on finding obstacles
or causes of the difficulties encountered by the students in solving mathematical
connection problems. Before the study, we informed the students that the tests
and interviews conducted during the data collection would not affect their
academic achievement at school. We conducted interviews right after all students
have completed the mathematical connection test in each school. Students have
interviewed alternately one by one for about 15 minutes. The answers of each
student during the interview were noted directly by the researchers.
2.4. Data Analysis

Data analysis was preceded by analyzing the answer sheet of each student.
Scoring was done by referring to the scoring guidelines prepared by researchers.
The scoring guidelines contained solutions and steps for solving (procedures) that
students must take when solving problems. In this study, these procedures
represent indicators of mathematical connections (see Figure 2 for an example).
We gave a score of 1 for each correct procedure performed by students for each
mathematical connection indicator, while for the incorrect procedure, we gave a
score of 0.
In Problem 1, there were two procedures of DR, one procedure of IM, and two
procedures of PD (see Figure 2). Students were categorized as “High Mastery” if
they performed all the procedures correctly for each indicator. If students only
performed one correct procedure on the DR and PD indicators, then students were
categorized as “Average Mastery”. In contrast, if all procedures performed by
students did not meet the three mathematical connection indicators, then they
were categorized as “Low Mastery”.
In Problem 2, there were six procedures of DR, one procedure of IM, and two
procedures of PD (see Figure 3). In Problem 2, if students performed at least four
correct procedures on the DR indicator, students were categorized as “High
Mastery”. If students only performed one to three correct procedures, then they
were categorized as “Average Mastery”, meanwhile if all the procedures
performed by students were incorrect, then students were categorized as “Low
Mastery”. For the IM indicator in Problem 2, the categorization of students was
the same as Problem 1.
For Problem 3, there was one procedure of DR, two procedures of PWR, one
procedure of IM, and three procedures of PD (see Figure 4). For DR and IM

261
indicators, students were categorized as “High Mastery” if they could perform

the procedure correctly. In contrast, if the procedure was incorrect, then students
were categorized as “Low Mastery”. For the PWR indicator, students were
categorized as “High Mastery” if they performed two procedures correctly,
“Average Mastery” if they performed only one procedure correctly, and “Low
Mastery” if they performed all procedures incorrectly. As for the PD indicator,
students were categorized as “High Mastery” if they performed three procedures
correctly, “Average Mastery” if they only performed one to two procedures
correctly, and “Low Mastery” if all the procedures were incorrect.
The data obtained were then tabulated based on the students’ ability categories
(High Mastery, Average Mastery, and Low Mastery) and mathematical
connection indicators (DR, PWR, IM, and PD) for each problem. From this
tabulation, we obtained the number and percentage of students for each category
of ability and mathematical connection indicators for each problem. In this study,
we considered students in the Average Mastery and Low Mastery categories as
participants who experienced mathematical connection difficulties. Then some of
their answers were selected to map the errors made by students for each
mathematical connection indicator for each problem. We presented the results of
the mapping narratively.
The results of the interviews were qualitatively analyzed using an analytical

technique developed by Bogdan and Biklen (2007) by analyzing the relationships
between themes from the qualitative data obtained from the interviews. The
transcripts of the interview data were then coded independently by two
researchers. If there were differences in the coding results, the two researchers
together reviewed the data then agreed to determine the correct coding. If there
were no differences in coding results, all researchers analyzed the relationship
between the coding so that we found the sub-themes. The relationship between
the sub-themes was then analyzed so that we found the main themes. We
presented the thematic results of the analysis in the tables.
This study involved all researchers during the data analysis processes to ensure
the validity and reliability in interpreting the qualitative data. All researchers had
experienced in the field of qualitative research, especially related to educational
issues. Then we discussed the results of the data analysis in a Focus Group
Discussion (FGD) involving all researchers and two experts. The two experts were
lecturers outside the researchers’ institution and had experienced in qualitative
research related to educational issues. The FGD discussed research findings and
its suitability and consistency with the data obtained from participants.
3. Results
3.1. Description of difficulties in students’ mathematical connection ability
In this study, we see the students’ mathematical connection ability through four
primary indicators, namely different representation (DR), part-whole
relationships (PWR), connections where A implies B (IM), and connections
showing that A is a procedure for doing B (PD). We categorized the ability of the
students to master each of the mathematical connection indicators into three levels

262
of mastery, including the following: “High Mastery,” “Average Mastery,” and

“Low Mastery”. We use the “Average Mastery” and “Low Mastery” levels as a
basis for identifying students who experience mathematical connection
difficulties. In other words, when the mastery of each indicator only reaches the
level of “Average Mastery” and “Low Mastery,” the students were then
categorized as experiencing mathematical connection difficulties. We present the
percentage of students who experience mathematical connections difficulties and
the level of student mastery of each indicator in each problem in Table 1.
Table 1. Percentage of students experiencing mathematical connection difficulties

High Average Low
Difficulties 1)
Indicators of Mathematical Mastery Mastery Mastery
Connections
n % n % n % n %
Problem 1:
Different Representation (DR) 16 42.11 7 18.42 15 39.47 22 57.89
Part-Whole Relationships (PWR) - 2) - - - - - - -

Connections where A Implies B
10 26.32 - - 28 73.68 28 73.68
(IM)
Connections that show that A is
17 44.74 6 15.79 15 39.47 21 55.26
a procedure for doing B (PD)
Problem 2:
Different Representation (DR) 18 47.37 10 26.32 10 26.32 20 52.63
Part-Whole Relationships (PWR) - 2) - - - - - - -

18 47.37 - - 20 52.63 20 52.63
(IM)
10 26.32 9 23.68 19 50.00 28 73.68
Problem 3:
Different Representation (DR) 5 13.16 - - 33 86.84 33 86.84
Part-Whole Relationships (PWR) 1 2.63 15 39.47 22 57.89 37 97.37

7 18.42 - - 31 81.58 31 81.58
(IM)
2 5.26 13 34.21 23 60.53 36 94.74
Notes:
1) Difficulties in mathematical connection are the total number of students who have
average and low mastery in each indicator.

2) The indicator is not tested in the item.
n is the number of students.
Table 1 shows that the percentage of students who have high mastery in each
indicator tested for each question was under 50%. In general, this number shows
that most students still experienced difficulty for each indicator of mathematical

263
connection ability. For Problem 1, Connections, where A Implies the students

perceived B (IM) as the most difficult indicator, where 73.68% of the students
experienced difficulty in this indicator. For Problem 2, however, the most difficult
indicator perceived by the students was connection showing that A is a procedure
for doing B (PD), where 73.68% of students experienced difficulty in this indicator.
For Problem 4, when compared to Problem 1 and Problem 2, the percentage of
student difficulties for each indicator of mathematical connection ability was
much higher. Of the four mathematical connection indicators tested in Problem 4,
more than 80% of the students had difficulty, and even 97.37% of the students had
difficulty in the part-whole relationships (PWR) indicator. For the PWR indicator,
in particular, Problem 1 and Problem 2 have not accommodated this indicator,
and therefore, we could not compare the students’ difficulty for this indicator in
Problem 3. The fact that the PWR indicator reached the highest percentage
showed that the students experienced great difficulty in constructing answers
using this indicator. Overall, we concluded that the students’ mathematical
connection difficulties occur in all indicators where PWR is the most difficult
indicator, and PD is the second most difficult indicator for the students. The
comparison between the students who had mastered and those who faced
difficulties in making mathematical connections can be seen in Figure 1.
Figure 1. The comparison between the students who had mastered and those who
faced difficulties of a mathematical connection
Figure 1 shows that the gap between students who had mastered the indicators
and those who had not mastered the indicators was too big, especially in Problem
3. From Figure 1, we can see that the number of students who have not mastered
the indicators is more prominent than those who have mastered the indicators of
mathematical connection in each mathematics problem. Meanwhile, the gap for
DR, PWR, PD indicators in all problems (Problem 1, Problem 2, and Problem 3)
looked different. Thus, we conclude that for each mathematical connection
indicator in each given problem, the number of students who experienced
difficulty is higher than those who did not experience it.

264
Qualitative data were also collected to confirm the findings. Based on the results
of qualitative data reduction from students’ interviews, we present some
difficulties faced by the students in solving mathematical connection problems in
Table 2.
Table 2. Identification of students’ difficulties in making mathematical connections

when solving problems
Identification of Student Difficulty Conclusion
The students were unfamiliar with mathematical The students were not well
connection problems. trained to solve mathema–
The students had difficulty in reading mathematical tical connection problems
patterns. because of a lack of fami–
The students had difficulty in connecting mathematical liarity.
concepts.
The students had difficulty in mathematic manipulation.
The students did not understand several terms in the The students lacked con–
problems. ceptual understanding.
The students had difficulty in understanding the
problems completely.
The students found it hard to identify important points
from the problems.
The students forgot mathematical formulas.
The problems included too long words. The students were not in–
The problems required repeated reading. terested in word problems.
Based on Table 2, we can understand that there are three main causes of students’
difficulties in solving mathematical connection problems. The first is a lack of
familiarity with mathematical connection problems, as seen in the following
excerpts of student interviews.
“... I’ve never worked on a problem about fractions like in Problem 1.” (Student 3)
“... I’ve never met a problem about a circle with another circle. A problem with a
circle usually asks for tangents only. That’s not about a circle intersecting another
circle. The problem is difficult and unfamiliar to me.” (Student 1)
The student interviews showed that so far, the students were only used to
working on routine problems. Thus, they faced difficulty when doing non-routine
problems, such as mathematical connection. Besides, they had difficulty in
reading mathematical patterns, as expressed by two students in the following
interview excerpts.
“... if we can find the pattern, the trigonometric identity is actually easy, but it’s
difficult for me to find the pattern because I don’t have the good sense to do that.”
(Student 12)
“... I have difficulty seeing the pattern because of its fraction, and in my opinion,
solving linear equations in this problem is very difficult.” (Student 10)
The difficulty of linking concepts and manipulating mathematical operations

caused the students to fail in their attempt to determine the next procedure for
solving the problem, as illustrated in the following interview excerpts.

265
“... the problem is not too confusing, but I just don’t really understand the
trigonometric identity and quadratic equation, then I find it hard to associate them.”
(Student 5)
“... I stopped at A and B because not find the comparison to calculate the value of A
and B.” (Student 12)
Lack of conceptual understanding is one of the causes of the students’ difficulty

in solving mathematical connection problems, as stated by the students in the
following excerpts.
“... I’ve difficulty in this problem; I don’t understand the meaning of ‘radius’ and
also the problem in general. So, I have no idea what the problem means.” (Student
2)
“... I’m having a hard time understanding the problem; applying the trigonometric
identity is quite difficult for me.” (Student 10)
“... I don’t understand about calculating the area of a circle, and I find it hard to
separate the parts of the circle.” (Student 5)
The student interviews proved that the students were still weak in the basic
concepts of mathematics. In this case, they did not know the term ‘radius’ and
how to calculate the area of a circle. Their low comprehension of such
fundamental concepts caused them to have difficulty in understanding the
purpose of the problem. Therefore, they failed to plan the correct procedure for
solving the problem.
One of the common challenges faced by the students in problem-solving activities

was that they were lazy to read word problems. The following excerpts illustrate
this.
“... the word problem consisted of long sentences that demanded a lot of
concentration on understanding the points. Well, I can understand the points. So,
I separated the points first and then calculated the area.” (Student 9)
“... what was asked in the problem was actually quite simple, but the description
was not straightforward.” (Student 11)
It appeared that the students did not enjoy reading the context or rather long
information. This condition led to difficulty in understanding the problem as a
whole and also planning mathematical procedures used to solve the problem. As
a result, the students’ progress in problem-solving related to mathematical
connections would be hindered. To illustrate the students’ difficulties in making
mathematical connections in solving mathematical problems, we will present
examples of cases of student difficulties for each indicator of mathematical
connection in the next section.
Table 1 shows that the percentage of students who had difficulty in mathematical
connections for each indicator was higher than those who had no difficulty.
Although the gap between these two groups varied for the three types of
problems given, the percentage of students who had difficulty always dominated

266
those who had no difficulty. In the next section, we will describe some cases
regarding the student difficulty in mathematical connections for each indicator.
3.2. Different Representation (DR) as a form of a mathematical connection

Of the three problems given to the students, the average percentage of students
who experienced difficulty for different representations (DR) indicator was
65.79%. This percentage clearly shows that most students still experienced
difficulty in applying this indicator. We present an example of student difficulty
in this indicator in Figure 2.
Figure 2. Example of student answer with difficulty in Different Representation (DR)
The example of student answer in Figure 2 shows that this student failed to solve
the problem correctly. The failure started when the student made a mistake in
designing a mathematical model appropriate to the context of the problem. In step
1
1, the student defined two new variables, namely 𝑝 and 𝑞, where 𝑝 = 𝐴−2𝐵 dan
1
𝑞 = 𝐴+2𝐵. It can be observed that these two variables could not be linked to the
two initial equations found in Problem 1. Also, in step 2, it shows that the equation
model made was out of sync with the results obtained in stage 1. Based on this
finding, it is clear that the student’s problem occurred when he was not able to
𝐴𝐵
represent the form 𝐴2 −4𝐵2 Into another form, and therefore incorrectly took
another procedure to solve Problem 1.

267
Figure 3. Another example of student answer with difficulty in Different

Representation (DR)
In Figure 3, we present another example of student difficulty in the different

representations (DR) indicator. In Figure 3 [a], the error in representing a different
form of trigonometric equation occurred in step 4 (inside the box). Likewise, the
example of the student’s answer in Figure 3 [b], in the red box, it is clear that
cos 2𝑥
students incorrectly changed the form sin 2𝑥 into tan 𝑥. Meanwhile, in Figure 3
[c], the error began in step 2, which the student manipulated the form of
2 sin 𝑥 . cos 2𝑥 and cos 𝑥 . sin 2𝑥 by using the formula for multiplying sine and
cosine, and we can see the error in the red box. Examples of these errors provide
empirical evidence that the students still encountered difficulties in the different
representations (DR) indicator for solving mathematical connection problems.
3.3. Part-Whole Relationships (PWR)

In this study, the PWR indicator for solving mathematical connections problems
was only found in Problem 3. The percentage of students who did not master this
indicator was the highest among all of the indicators, at 97.37%. This finding
strongly indicated that in the context of the given problem, the students’ ability to
do part-whole relationships was still low. We present examples of student

268
answers that indicate that they have difficulties with the PWR indicator in Figure
4.
Figure 4. Example of the student answer with difficulty in Part-Whole Relationship

(PWR) indicator
Figure 4 [a] shows that in step 1, the student actually understood the meaning of
Problem 3. However, it appeared that the students had difficulty calculating the
segment area of a large circle. Based on the student’s answer in Figure 4 [a], the
student calculated the area of a large circle segment = the area of a large circle –
the area of 1/2 small circle. What the student did in this step indicated that he was
not able to find the basic concepts used to calculate the area of segment (such as
the rules of cosine, the area of the section, and the area of a triangle). In other
words, the student encountered difficulty in finding part-whole relationships to

269
find the segment range of the large circle. Next, Figure 4 [b] showed the other
example of a student’s answer in solving Problem 3. In Figure 4 [b] can be seen
that the student found it hard to analyze the parts (see step 1) or the basic concepts
used to find the whole solution to Problem 3 (see step 2 and step 3).
3.4. Connections where A implies B (IM)

IM indicator in mathematical connections is operationally visible when students
can show a relationship that a mathematical concept results in another concept. In
the three problems given to the students, each procedure for solving these
problems contains an IM indicator. Based on the test of the three problems having
the indicator, there were only 30.70% of the students who mastered the indicator.
In contrast, the remaining 69.30% of the students did not master the indicator. We
present the examples of student answers with difficulties in the connections
where A implies B (IM) indicator in Figure 5.
Figure 5. Example of student answer with difficulty in Connections Where A Implies

B (IM) indicator
From algebra theory, the student’s work in Figure 5 [a] was correct, but it was
ineffective and required more time to find the final answer. If we compare the
student’s response in Figure 5 [a] with the solution to Problem 1 (see Figure 2),
the difference is evident. The student’s work in Figure 5 [a] clearly showed that
he/she was not able to simplify the equations given. This finding is because the
𝐴𝐵 𝐴 𝐵
student was not able to manipulate the form from 2 2 into × . As a
𝐴 −4𝐵 𝐴−2𝐵 𝐴+2𝐵
result, the student failed to find another more straightforward concept (for
example, doing mathematical modeling), leading to the discovery of the concept
of a two-variable linear equation system. If the student succeeds in finding a more
straightforward form of the two-variable linear equation system given, the
procedure for solving Problem 1 will be more comfortable.

270
Meanwhile, as seen in Figure 5 [b], the student made a mistake in step 6. In this
step, the student assumed/defined tan 𝑥 = 𝑦, so the trigonometric equation in
step 5 could be changed into a quadratic equation 𝑦 2 + 5𝑦 − 6 = 0. By dividing it
into factors (factorizing), the solutions from the quadratic equation were 𝑦 = −6
dan 𝑦 = 1 (in Figure 6 [b], the student wrote the solution as 𝑥1 = −6 and 𝑥2 = 1).
In this case, the solution of the quadratic equation found should be changed into
the solution for the trigonometric equation. Because the student previously
defined tan 𝑥 = 𝑦, the solution of the quadratic equation is converted into
tan 𝑥 = −6 and tan 𝑥 = 1. Due to some errors in performing the procedure, the
final solution for determining the value of tan(𝑥1 + 𝑥2 ) was also incorrect. From
this case, it is clear that the students still had difficulty in connecting a
mathematical concept that results in other concepts.
3.5. Connections showing that A is a Procedure for Doing B (PD)

Another difficulty experienced by the students in mathematical connections is
related to the indicator of connections showing that A is a procedure for doing B
(PD). If the students were able to show that a mathematical procedure is applied
to obtain another procedure, we consider them competent at this PD indicator.
Each of the three problems tested contained the PD indicator, and we obtained
the average percentage of the students who experienced difficulties for this
indicator was 74.56%. We present the examples of student answers with difficulty
in PD indicator in Figure 6.
Figure 6. Example of student answer with difficulty in connections showing that A is

a Procedure for Doing B (PD) indicator
In Figure 6 [a], the student was able to find a solution to the trigonometric
equation (see step 4 and step 5) resulted from the previous process, even though
they did not assume first that tan 𝑥 = 𝑥. However, some errors still occurred in
step 7 and step 8, when the student associated that the value of 𝑥 is the angle (see
step 7). The angles were then substituted with the equation tan(𝑥1 + 𝑥2 ) (see step
8), even though the intended final solution from Problem 3 was different. This
finding clearly showed that the student was not able to connect between the

271
procedures performed in steps 4, 5, and 6 with the equation tan(𝑥1 + 𝑥2 ) (step 8).
Likewise, the student’s answer in Figure 6 [b], when the student calculated the
area of a section (see step 3, inside of the box), they wrote that the section area of
90°
the large circle was 360° × 𝑎𝑟𝑒𝑎 𝑜𝑓 𝑎 𝑙𝑎𝑟𝑔𝑒 𝑐𝑖𝑟𝑐𝑙𝑒. However, the students did not
show what procedure was applied to find a 90° angle. According to the illustration
drawn by the student (see step 1), a 90° angle was determined based on the
assumption that the triangle formed was a right triangle, not associated with
cosine rules (see the solution of Problem 3 in Figure 4). This finding showed that
the students were not able to show what procedure should be previously applied
to calculate the section area of the circle.
5. Discussion
The results of this study generally indicate that most students have not mastered
the mathematical connection skills well. For each mathematical connection
indicator, most students have mastered the different representation indicator
(DR) (47.37%) and the connection where A implies B (IM) (47.37%) in Problem 2.
Whereas in other indicators, the percentage of students the master is still low.
Many students have not mastered the mathematical connection indicators for
each given mathematical problem, indicating clearly that most students still have
difficulty making mathematical connections in solving mathematics problems.
Students who have been able to make good mathematical connections show that
they have a good understanding of using various mathematical concepts and
procedures and their relationships to solve problems. This finding is consistent
with the opinion of García-García and Dolores-Flores (2020) and Payton (2019).
Conversely, students who have difficulties tend not to master the basic concepts
and mathematical procedures needed to solve problems, as stated by students
during interviews. Besides, the unfamiliarity of students in solving mathematics
problems also causes them to experience difficulties. This unfamiliarity also
impacts students’ low interest in word problems, where students admit to being
lazy to read problems with too long words, as stated by students during
interviews.
The results of this study support the findings of several previous studies, which
show that the students still experience difficulties in mathematical connection
(Dolores-Flores et al., 2019; Kenedi et al., 2019; Lapp et al., 2010; Siregar & Surya,
2017). Although previous studies involved participants at different levels, from
elementary school to undergraduate levels, the obstacles faced by each participant
were almost the same in general. Those obstacles consist of difficulty making
representations in other forms, doing part-whole relationships, making
implications, and showing the interrelationships between mathematical
procedures. These mathematical connection difficulties also occurred in this
study.
Of the four mathematical connection indicators studied in this study, most

students had difficulty making the part-whole relationship (PWR). This finding is
consistent with the findings of Mhlolo (2012) and García-García and Dolores-
Flores (2020). When doing PWR in solving mathematics problems, students
should see the connection between general concepts through specific parts in the

272
form of particular examples (Mhlolo, 2012). In this case, we understand that

students are considered capable of making PWR if they apply specific concepts or
procedures properly as part of the general concept to be completed. However, in
this study, most students failed to do this. To overcome it, according to Mhlolo
(2012), students need to be accustomed to doing and expressing generalizations
both through deductive and inductive reasoning patterns. When students have
good deductive and inductive reasoning skills, we expect that they will be able to
do PWR well when solving mathematics problems.
One of the critical factors in making a mathematical connection is a deep

understanding of its concept (Dolores-Flores et al., 2019; García-García & Dolores-
Flores, 2018; Silver et al., 2009) and obtaining such conceptual understanding also
needs mathematical connections (Anthony & Walshaw, 2009). Dolores-Flores et
al. (2019) believed that poor understanding of concepts to be the leading cause of
the students’ difficulty in making mathematical connections. As also confirmed
from the interview conducted in this study, almost all of the students experiencing
difficulties in making mathematical connections claimed that they did not
understand the mathematical concepts needed to solve mathematical connection
problems. For that reason, mathematics learning in the classroom should be
focused on how to instill mathematical concepts in students to develop other
mathematical skills.
Lack of familiarity with mathematical connection problems also caused

difficulties in solving the problems. Mhlolo et al. (2012) suggest that most students
lack the opportunity to deepen the understanding of mathematical connections.
Besides, the books used have not yet been explicitly linked to the topics with
various contexts, mathematics with real-life, and thus hinder the students from
developing their mathematical connection skills (Salout et al., 2013). In this case,
Baki, Çathoğlu, Coştu & Birgin. (2009) recommend the need to make changes in
the mathematics class in terms of content and context that must be applied.
The findings of this study have implications for the development of the pedagogy
of mathematics learning. We believe that the factors that cause student difficulties
in making mathematical connections when solving mathematics problems can be
used as a framework for teachers in designing mathematics learning. As stated by
García-García and Dolores-Flores (2020), what students do and communicate
when solving mathematics problems involving mathematical connections is
essential as a reflection to improve the quality of learning. At least the research
findings have implications regarding the importance of teachers developing skills
in facilitating students to make mathematical connections and best practices that
can be applied to develop students’ connection skills.
The teacher’s pedagogical competence in training the students to solve

mathematical connections also needs immediate attention. It is crucial to ensure
that the teacher is well prepared and has confidence that the students have
sufficient initial abilities to develop various mathematical skills. Bowen (2014)
reports that teachers tend to be hesitant about using their knowledge of
mathematical connections because they doubt their student abilities. For this

273
reason, instilling confidence in teachers that their students can develop their
ability in mathematical connection is very important. Besides, lack of assistance
among the teachers regarding the planning and implementation of learning that
facilitates students to make mathematical connections (Monroe & Mikovch, 1994)
is of concern mainly to policymakers in the field of education.
We can make various efforts to improve student ability in mathematical

connections. As stated by NCTM (2000) that mathematical connection is a tool in
problem-solving. This statement means that practicing problem-solving skills in
learning mathematics participates indirectly in students’ practicing mathematical
connection skills. Various studies have reported that some learning models were
effective to develop student problem-solving skills such as Problem-Based
Learning (PBL) (Kartikasari & Widjajanti, 2017; Malasari, Nindisari & Jaenudin,
2017), collaborative problem-based learning strategy (Widjajanti, 2013), as well as
using varied methods in learning such as direct learning, demonstration, practice
and exercise, and also using modified instructional media (Jannah, Apriliya &
Karlimah, 2017). Also, it is crucial to give the students more activities of problem-
solving so that they make a habit of it. Students of all ages are more motivated
with problem situations that involve them and their classmates (Welchman-
Tischler, 1992); use three-dimensional manipulatives (Safi & Desai, 2017); and use
proof approaches (Jiang & O’Brien, 2012). Another strategy that can be applied is
using writing strategies in learning mathematics such as making drawings,
pictures, tables, and graphs; providing a clear explanation of problem-solving
methods and justifications of processes; and doing a reflection on learning
(Haltiwanger & Simpson, 2013). The use of math software such as GeoGebra has
also been recommended to develop students’ mathematical connection skills
(Zengin, 2019)
Although this study succeeded in uncovering the types of mathematical

connection difficulties experienced by students when solving mathematics
problems and their causes, this study has not revealed the relationship between
students’ academic performance levels and difficulties. This limitation provides
an opportunity for future research to focus on uncovering the relationship
between types of mathematical connection difficulties and the level of student
academic performance. Besides, this study’s mathematical problems do not
represent all mathematics topics studied at the high school level. Thus, to
strengthen this study’s findings, it is necessary to replicate this study by using
mathematical problems on other topics.
5. Conclusions and Implications

Based on the study results and discussion, we concluded that the students still
experienced difficulties in making mathematical connections when solving
mathematics problems. These difficulties included making a different
representation as a form of mathematical connections, part-whole relationships
(hierarchical nature of concepts), connections that show A is a procedure for doing
B (logical reasoning), and connections showing that A is a procedure for doing B
(algorithms). In general, the indicators of mathematical connection difficulties
experienced by the students as follows. First, lack of familiarity in solving

274
mathematical connection problems characterized by difficulties in recognizing

mathematical patterns, linking mathematical concepts, and working out
mathematical manipulations. Second, poor understanding of the concept
characterized by difficulties in understanding some mathematical terms,
comprehending the whole problem, identifying essential points of the problem,
applying mathematical formulas. Third, the lack of interest in word problems.
Our research findings are useful for educators in designing mathematics learning
that facilitates the development of students’ mathematical connections skills.
Educators can use our findings to construct a framework to rearrange learning
objectives, adjust the depth of learning topics, select mathematics contents and
contexts, choose innovative strategies, consider the use of technology, even design
assessments that consider mathematical connections. We recommend that the
topics used to teach mathematics contain problem-solving activities that link
some mathematical concepts and procedures and connect mathematics with other
disciplines and real-life situations. Regarding the limitations of this study, we
suggest that future research investigate the relationship between types of
mathematical connection difficulties and the level of student academic
performance. Furthermore, the researchers need to conduct investigations related
to students’ difficulties in making mathematical connections on other topics. We
hope that replications can strengthen our research findings.
6. References
Agustini, R. Y., Suryadi, D., & Jupri, A. (2017). Construction of open-ended problems for
assessing elementary students mathematical connection ability on plane
geometry. Journal of Physics: Conference Series, 895, 1–9. doi:10.1088/1742-
6596/895/1/012148
Albay, E. M. (2019). Analyzing the effects of the problem solving approach to the
performance and attitude of first year university students. Social Sciences &
Humanities Open, 1(1), 1–7. doi:10.1016/j.ssaho.2019.100006
Anthony, G., & Walshaw, M. (2009). Effective pedagogy in mathematics. In J. Brophy
(Ed.), Educational practices series − 19 (pp. 147–164). Geneva, Switzerland: IBE
Publications Unit.
Apino, E., & Retnawati, H. (2017). Developing instructional design to improve
mathematical higher order thinking skills of students. Journal of Physics: Conference
Series, 812, 1–7. doi:10.1088/1742-6596/812/1/012100
Apino, E., & Retnawati, H. (2019). Creative problem solving for improving students’
higher order thinking skills (HOTS) and characters. In E. Retnowati, A. Ghufron,
M. Marzuki, K. Kasiyan, A. C. Pierawan & A. Ashadi (Eds.), Character education for
21st century global citizens: Proceedings of the 2nd International Conference on Teacher
Education and Professional Development (INCOTEPD 2017) (pp. 249–256). London,
England: Routledge.
Appelbaum, P. (2000). Eight critical points for mathematics. Counterpoints, 110, 41-55.
Retrieved from http://www.jstor.org/stable/42975936
Arjudin, A., Sutawidjaja, A., Irawan, E. B., & Sa’dijah, C. (2016). Characterization of
mathematical connection errors in derivative problem solving. IOSR Journal of
Research & Method in Education, 6(5), 7–12. doi:10.9790/7388-0605050712
Baki, A., Çathoğlu, H., Coştu, S., & Birgin, O. (2009). Conception of high school students
about mathematical connections to the real-life. Procedia Social and Behavioral
Science, 1, 1402–1407. doi:10.1016/j.sbspro.2009.01.247
Berry, J. S., & Nyman, M. A. (2003). Promoting students’ graphical understanding of the calculus.

275
Journal of Mathematical Behavior, 22(4), 481–497. doi:10.1016/j.jmathb.2003.09.006

Bogdan, R. C., & Biklen, S. K. (2007). Qualitative research for education: An introduction to
theory and methods. Boston, MA: Allyn and Bacon.
Bowen, B. (2014). Second-career mathematics teachers’ knowledge of mathematical connections.
Journal of STEM Teacher Education, 49(1), 29–44. doi:10.30707/JSTE49.1Bowen
Businskas, A. (2008). Conversations about connections: How secondary mathematics teachers conceptualise
and contend with mathematical connections. (Doctoral dissertation, Simon Fraser University,
Burnaby, Canada). Retrieved from http://ir.lib.sfu.ca/handle/1892/10579
Blum, W., Galbraith, P. L., Henn, H. W., & Niss, M. (2007). Modelling and applications in
mathematics education. New York, NY: Springer.
Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches
(3rd ed.). Thousand Oaks, CA: Sage Publications.
Dolores-Flores, C., Rivera-López, M. I., & García-García, J. (2019). Exploring mathematical
connections of pre-university students through tasks involving rates of change.
International Journal of Mathematical Education in Science and Technology, 50(3), 369–
389. doi:10.1080/0020739X.2018.1507050
Eli, J. A., Mohr-Schroeder, M. J., & Lee, C. W. (2013). Mathematical connections and their
relationship to mathematics knowledge for teaching geometry. School Science and
Mathematics, 113(3), 120–134. doi:10.1111/ssm.12009
English, L. D., & Gainsburg, J. (2016). Problem solving in a 21st-century mathematics
curriculum. In L. D. English & D. Kirshner (Eds.), Handbook of international research
in mathematics education (3rd ed.) (pp. 313–335). New York, NY: Routledge.
García-García, J., & Dolores-Flores, C. (2018). Intra-mathematical connections made by high
school students in performing calculus tasks. International Journal of Mathematical
Education in Science and Technology, 49(2), 227–252. doi:10.1080/0020739X.2017.1355994
García-García, J., & Dolores-Flores, C. (2020). Exploring pre-university students’
mathematical connections when solving calculus application problems.
International Journal of Mathematical Education in Science and Technology, 51(7), 1–25.
doi:10.1080/0020739X.2020.1729429
Hadi, S., Retnawati, H., Munadi, S., Apino, E., & Wulandari, N. F. (2018). The difficulties
of high school students in solving higher-order thinking skills problems. Problems
of Education in the 21st century, 76(4), 520–532. doi:10.33225/pec/18.76.520
Haltiwanger, L., & Simpson, A. M. (2013). Beyond the write answer: Mathematical
connections. Mathematics Teaching in the Middle School, 18(8), 492–498.
doi:10.5951/mathteacmiddscho.18.8.0492
Hodge, B. (2003). Logical thinking in mathematics: From oz to awe! Research and Teaching in
Developmental Education, 19(2), 41–46. Retieved from http://www.jstor.org/stable/42802166
Jannah, R. R., Apriliya, S., & Karlimah, K. (2017). Didactical design material units of
distance and speed to developed mathematical connection in elementary school.
IOP Conference Series: Materials Science and Engineering, 180, 1–9. doi:10.1088/1757-
899X/180/1/012022
Jiang, Z., & O’Brien, G. E. (2012). Multiple proof approaches and mathematical connections. The
Mathematics Teacher, 105(8), 586–593. doi:10.5951/mathteacher.105.8.0586
Kartikasari, A., & Widjajanti, D. J. (2017). The effectiveness of problem-based learning
approach based on multiple intelligences in terms of student’s achievement,
mathematical connection ability, and self-esteem. Journal of Physics: Conference
Series, 812, 1–8. doi:10.1088/1742-6596/812/1/012097
Kenedi, A. K., Helsa, Y., Ariani Y., Zainil, M., & Hendri, S. (2019). Mathematical connection
of elementary school students to solve mathematical problem. Journal on
Mathematics Education, 10(1), 69–80. doi:10.22342/jme.10.1.5416.69-80
Lapp, D. A., Nyman, M. A., & Berry, J. S. (2010). Student connections of linear algebra concepts:
An analysis of concept maps. International Journal of Mathematical Education in Science and
Technology, 41(1), 1–18. doi:10.1080/00207390903236665

276
Leikin, R., & Pitta-Pantazi, D. (2013). Creativity and mathematics education: The state of
the art. ZDM Mathematics Education, 45, 159–166. doi:10.1007/s11858-012-0459-1
Lince, R. (2016). Creative thinking ability to increase setudent mathematical of junior high school by
applying models numbered heads together. Journal of Education and Practice, 7(6), 206–212.
Retrieved from https://iiste.org/Journals/index.php/JEP/article/view/29109/29873
Malasari, P. N., Nindisari, H., & Jaenudin, J. (2017). A development of mathematical
coonecting ability of students in junior high school through a problem-based
learning with course review Horay method. Journal of Physics: Conference Series,
812, 1–7. doi:10.1088/1742-6596/812/1/012025
Mhlolo, M. K. (2012). Mathematical connections of a higher cognitive level: A tool we may use
to identify these in practice. African Journal of Research in Mathematics, Science and
Technology Education, 16(2), 176–191. doi:10.1080/10288457.2012.10740738
Mhlolo, M. K., Venkat, H., & Schäfer, M. (2012). The nature and quality of the mathematical
connections teachers make. Pythagoras, 33(1), 1–9. doi:10.4102/pythagoras.v33i1.22
Monroe, E. A., & Mikovch, A. K. (1994). Making mathematical connections across the
curriculum: Activities to help teachers begin. School Science and Mathematics, 94(7),
371–376. doi:10.1111/j.1949-8594.1994.tb15697.x
Moon, K., Brenner, M. E., Jacob, B., & Okamoto, Y. (2013). Prospective secondary
mathematics teachers’ understanding and cognitive difficulties in making
connections among representations. Mathematical Thinking and Learning, 15(3),
201–227. doi:10.1080/10986065.2013.794322
Mwakapenda, W. (2008). Understanding connections in the school mathematics curriculum.
South African Journal of Education, 28(2), 189–202. doi:10.15700/saje.v28n2a170
NCTM. (2000). Principles and standards for school mathematics. Reston, VA: Author.
Ndiung, S., & Nendi, F. (2018). Mathematics connection ability and students mathematics
learning achievement at elementary school. SHS Web of Conferences, 42(9), 1–5.
doi:10.1051/shsconf/20184200009
Osman, S., Yang, C. N. A. C., Abu, M. S., Ismail, N., Jambari, H., & Kumar, J. A. (2018). Enhancing
students’ mathematical problem-solving skills through bar model visualisation
technique. International Electronic Journal of Mathematics Education, 13(3), 273–279.
doi:10.12973/iejme/3919
Özgen, K. (2016). A theoretical study on the mathematical connection. In W. Wu, M. T.
Hebebci, & O. T. Öztürk (Eds.), International Conference on Research in Education and
Science (pp. 220–230). Bodrum, Turkey: ICRES Publishing. Retrieved from
https://www.2016.icres.net/ICRES_Proceeding_Book.pdf?rnd=333040543
Palmér, H., & Van Bommel, J. (2020). Young students posing problem‑solving tasks: What does
posing a similar task imply to students? ZDM Mathematics Education, 52(4), 743–752.
doi:10.1007/s11858-020-01129-x
Partnership for 21st Century Learning. (2019). Framework for 21st century learning. Retrieved from
http://static.battelleforkids.org/documents/p21/P21_Framework_Brief.pdf
Payton, S. (2019). Fostering mathematical connections in introductory linear algebra
through adapted inquiry. ZDM Mathematics Education, 51(7), 1239–1252. doi:
10.1007/s11858-019-01029-9
Rafi, I., & Retnawati, H. (2018). What are the common errors made by students in solving
logarithm problems? Journal of Physics: Conference Series, 1097(1), 1–9.
doi:10.1088/1742-6596/1097/1/012157
Safi, F., & Desai, S. (2017). Promoting mathematical connections using three-dimensional
manipulatives. Mathematics Teaching in the Middle School, 22(8), 488–492.
doi:10.5951/mathteacmiddscho.22.8.0488
Salout, S. S., Behzadi, M. H., Shahvarani, A., & Manuchehri, M. (2013). Student’s
conception about the relation of mathematics to real-life. Mathematics Education
Trends and Research, 2013(1), 1–7. doi:10.5899/2013/metr-00009
Silver, E. A., Mesa, V. M., Morris, K. A., Star, J., & Benken, B. M. (2009). Teaching

277
mathematics for understanding: An analysis of lessons submitted by teachers

seeking NBPTS certification. American Educational Research Journal, 46(2), 501–531.
doi:10.3102/0002831208326559
Siregar, N. D., & Surya, E. (2017). Analysis of students’ junior high school mathematical connection
ablility. International Journal of Sciences: Basic and Applied Research, 33(2), 309–320. Retrieved
from https://gssrr.org/index.php/JournalOfBasicAndApplied/article/view/7363/3471
Suh, J., & Seshaiyer, P. (2013). Mathematical practices that promote 21st century skills. Mathematics
Teaching in the Middle School, 19(3), 132–137. doi:10.5951/mathteacmiddscho.19.3.0132
Van Zanten, M., & Van den Heuvel‑Panhuizen, M. (2018). Opportunity to learn problem
solving in Dutch primary school mathematics textbooks. ZDM Mathematics
Education, 50(5), 827–838. doi:10.1007/s11858-018-0973-x
Welchman-Tischler, R. (1992). Making mathematical connections. The Arithmetic Teacher,
39(9), 12–17. Retrieved from https://www.jstor.org/stable/41195343
Widjajanti, D. J. (2013). The communication skills and mathematical connections of
prospective mathematics teacher: A case study on mathematics education
students, Yogyakarta State University, Indonesia. Jurnal Teknologi, 63(2), 39–43.
doi:10.11113/sh.v63n2.149
Wijaya, A., Van den Heuvel-Panhuizen, M., Doorman, M., & Robitzsch, A. (2014).
Difficulties in solving context-based PISA mathematics tasks: An analysis of
students’ errors. The Mathematics Enthusiast, 11(3), 555–584.
Zengin, Y. (2019). Development of mathematical connection skills in a dynamic learning
environment. Education and Information Technologies, 24(3), 2175–2194.
doi:10.1007/s10639-019-09870-x

278

Vol. 19, No. 8, pp. 278-296, August 2020
Application of Rasch Model to Develop a

Questionnaire for Evaluating the Quality of
Teaching for Students’ Creativity Development
Thi Le Thuy Bui

Hanoi National University of Education, Hanoi, Vietnam
https://orcid.org/0000-0002-4561-612X
Vyacheslav I. Kazarenkov
Peoples' Friendship University of Russia, Moscow, Russia
https://orcid.org/0000-0002-2808-6526
Van De Tran
Can Tho University of Medicine and Pharmacy, Can Tho, Vietnam
https://orcid.org/0000-0003-0421-5079
Abstract. The purpose of this research is to develop a questionnaire for

evaluating the quality of teaching for creativity development. We use
three methods for creating the questionnaire, namely, interviews,
literature review, and expert consultation. The Rasch model was applied
to assess the psychometric properties of the questionnaire in the first
phase with the participation of 112 students. The results of the study
indicated that all items had a point-measure correlation coefficient
ranging between 0.312 and 0.757, Infit and Outfit MNSQ values were in
the range of 0.62 to 1.54, item difficulty values were in the range from -
1.54 to 1.12. The overall reliability index of the questionnaire was 0.955,
the separation was 4.582, the scale’s number of strata was 6.442.
Cronbach’s alpha coefficient of the scale was 0.963. The results of the
study showed that the questionnaire was in good compliance with the
Rasch model requirements. We used the questionnaire to assess the
current status of teaching for creativity development in the teaching of
Pedagogy in the second phase. This phase involved 300 first- and
second-year students who were studying “Pedagogy” and they were
grouped in 10 groups. A total of 10 teachers were evaluated, 1 teacher
was evaluated by 1 group. The results showed that the quality of
teachers’ teaching for creativity development was at an acceptable level.
We believe that teachers’ preparation according to the evaluation criteria
for teachers’ quality teaching for creativity development is needed to
improve teachers’ teaching for students’ creativity development.
Keywords: Rasch model; Questionnaire; Development; Creativity;

Teaching process

279
1. Introduction
Creativity has a great role for each individual and society and is considered an
important element of participation and contribution to life and society (Loveless,
2002). Creativity is increasingly seen as a vital attribute for work, thinking, and
life in the 21st century (Mehta, Henriksen, 2016). The continuous changes that
modern society is experiencing are raising new demands on higher education on
the way to achieving the goal of initiating and developing creativity as a part of
students' personalities. In this sense, promoting creativity among students is
considered a very important purpose in higher education (Sadeghi, Ofoghi,
2011; Hamed, Preece & Hashim, 2016; Thuy et al., 2019; Bui, Kazarenkov, 2020).
Creativity and innovation in education are not only regarded as an opportunity,
but also a necessity. These are considered an unalienable part of the purpose of
the present and future educational system (Ferrari, Cachia & Punie, 2009).
The issues related to teaching and initiating creativity through education among
students have been studied for over the past twenty years. Modern
psychologists and educators believe that creativity can be taught (Shrivastava,
2016; Beghetto, Kaufman, 2014; Pang, 2015; Kaplan, 2019; Richardson, Mishra,
2018); Tran, Ho & Hurle, 2016, Kazarenkov et al., 2020). Currently, educators are
paying increasing attention to changes in the modern educational process and
the role of teachers in the process of education. In this context, the teaching
process has changed much in recent years and differs widely from what it was in
the past (Biswas, 2011) to become in the form of discussions, seminars, projects,
workshops, etc. The teachers have to play the role of moderators, facilitators,
consultants, and tutors in a modern university. They should help, inspire, and
encourage students whenever needed. Through these roles, the teachers create
the conditions in which the independence, activity, and creativity of students are
manifested. They require them to constantly change and adapt to the
requirements and conditions of modern education. We can say that the teacher is
a decisive factor who decides the education in general and students’ creativity
development in particular.
Teachers’ teaching of creativity is among factors that would motivate the
students to be creative. Although the focus on creativity in the teaching process
has been increased since the 1990s, that did not yield the expected results (Craft,
2005; Beghetto, 2005). According to Shrivastava (2016), one of the main causes of
this is related to teachers. Additionally, there were many studies all over the
world on the formation and development of students’ creativity along with
instruments to measure the different aspects of teaching creativity. However,
there is limited research on psychometric properties analyses of these
instruments which measure the degree of teachers' quality teaching for creativity
development.
One of the modern approaches that allow creating a measuring instrument with
adequate validation and reliability is the Rasch model (Assanovich, 2012). The
Rasch model allows to evaluate the quality of the attributes of the scale within
the theoretical framework of point-measure correlation coefficient (PTMEA
Corr), item’s infit and outfit values, item’s level of difficulty, reliability, and
separation and strata statistics of the questionnaire. Therefore, it can guarantee
the consistency of the of the analyzed factor structure. Additionally, the Rasch

280
analysis provides multiple sources of evidence to test the quality of a scale.

Furthermore, it not only tests theoretical constructs, but also clearly indicates the
items that need to be adjusted or replaced to ensure the overall quality of the
scale. In recent years, the use of this method to assess the psychometric
properties of scales has grown rapidly in various fields. The purpose of this
study is to develop and validate a questionnaire for evaluating teaching for
creativity development based on the Rasch model and using it to assess the state
of teaching for creativity development in the teaching process of discipline
“Pedagogy” at Hanoi National University of Education, Vietnam.
2. Literature Review and Theoretical Framework

2.1. Creativity and Teaching for Creativity Development
Creativity is considered as a process involving the creation of new ideas
(Esquivel, 1995) and is a potentiality that everyone can develop in any field
(Trnka, Zahradnik & Kuška, 2016; Glaveanu, 2018). Creativity refers to people’s
ability to solve problems and transform reality (UNESCO, 2015).
In recent decades, research on creativity has become the central issue of many
different scientific fields. Developing students’ creativity is becoming one of the
most important goals in the modern global educational system (Shaheen, 2010;
Pang, Plucker, 2012). Sternberg and Lubart (1999) place high value on novel and
appropriate products of creativity. Glaveanu (2018) appreciates novelty,
originality, significance, or value in creative results. Creativity arises as a result
of the interrelation between many different internal (intelligence, creative
thinking, abilities, internal motivation) and external (psychological atmosphere
within the group in the process of interpersonal interaction, approaches in the
educational process) factors. Glaveanu (2013) explores the interrelations between
actors, actions, artifacts, audiences, and contexts in the creative process. Walia
(2019) is interested in creative activities and the definition of creativity. Elisondo
and Vargas (2019) pay attention to the relationships between people, processes,
environments, and products. Creative processes involve interrelations between
actors, actions, artifacts, audiences and contexts (Glaveanu, 2013). As Elisondo
(2016) indicated, the process of human creativity is always associated with social
context and other factors.
Recently, researchers have developed a special interest in the relationship
between creativity and the psychological state of people. They believe that
creativity affects human’s health and well-being (Richard, 2010; Corner, Silvia,
2015; Benedek, Bruckdorfer & Jauk, 2019). During and after creativity process,
people feel happier, more active, relieved, relaxed, and satisfied (Silvia, et al.,
2014; Conner, DeYoung & Silvia, 2017). On the contrary, people achieve greater
creative effectiveness when they feel energetic, excited, enthusiastic, and joyful
(Benedek, Bruckdorfer & Jauk, 2019; Elisondo, Vargas, 2019). These findings
suggest that teachers should create positive emotions for students in the
teaching process to increase the effectiveness of creativity development. This is
closely related to building a learning environment that encourages the
development of students’ creativity.
Their creativity largely depends on teacher’s practices. Therefore, the issues of
teaching for creativity development have been attracting a lot of attention by

281
researchers in education around the world. Teaching to develop creativity is a

process of equipping learners with knowledge about the creative process and
giving opportunities for them to express their creativity (NACCCE, 1999). In this
context, researchers believe that encouraging learners to believe in their own
creativity is important in teaching for creativity (Moran, 2010; Ferrari, Cachia &
Punie, 2009). Teaching for the development of creativity is inseparable from the
assessment of (Tran, Ho & Hurle, 2016).
Assessment is considered one of the most important factors to develop students’
creativity (Tran, Ho & Hurle, 2016; Ferrari, Cachia & Punie, 2009). The fact of
teaching at universities shows that assessment of lecturers inhibits the
development of students’ creativity due to many different factors. Traditional
teachers often affect students' creative performance negatively, because they
tend to restrain individual student independence (Ng, 2002). The creative
expression of students in the learning process is considered nonconformist by
teachers (Shrivastava, 2016). The aspect of building a learning environment that
supports the creativity of learners has caught the attention of many researchers
(Beghetto, Kaufman, 2014; Davies et al., 2013; Hong, Chang & Chai, 2014;
Richardson, Mishra, 2018). Some authors concur with the idea that the learning
environment that encourages creativity includes conditions of the physical
environment, learning atmosphere, and student engagement. Hence, the ideal
environment is to give learners a sense of security, comfort, excitement, and
confidence to share ideas and personal thoughts. Other authors focus on
researching creative tools in teaching to develop learners’ creativity. These
include tools for curiosity (Engel, 2013; Starko, 2013; Conklin, 2012), imagination
(Lehrer, 2012), creative thinking (Starko, 2013). Teaching methods to stimulate
learners’ creativity are also discovered by Rankin, Brown (2016). The authors
emphasize the empowerment of the learners and the learners’ willingness to
participate in creative learning. This is seen as the key to the success of teaching
methods for developing learners’ creativity.
2.2. Theoretical Concepts of Teaching for Creativity Development

The number of definitions of creativity is enormous. Each researcher approaches
the nature of creativity differently. But most they believe that creativity is a
process of curiosity-exploring, imagining and thinking based on one’s
knowledge, experiences, emotions and motivations to generate original and
effective products (Vygotsky, 2004; Runco et al., 2012). According to this
definition, the three most important personal traits for the student's creative
development are imagination, curiosity and creative thinking. They have a
strong correlation in the creative process and are embodied in creative products.
Creativity requires inputs of knowledge, creative skills and attitude towards
creativity. Knowledge about creative process, particular area and related areas
are essential factors for creativity (Tran, Ho & Hurle, 2016). Thus, to develop
creativity for students in the teaching process, teachers need to really pay
attention to these factors.
Teaching to develop creativity is a process of equipping learners with
knowledge about the creative process and creating opportunities for them to
express their creativity (NACCCE, 1999). Teaching for creativity development is

282
a form of teaching aimed at developing the creative competence and behavior of

students (Sahlberg, 2009) and is carried out in the teaching process (Gorshunova,
Medvedev & Razdorskaya, 2014). According to these researchers, teachers
should set the goals of developing students' creative competencies in teaching
goals. Teaching for the development of creativity is inseparable from the
assessment of students’ creativity (Tran, Ho & Hurle, 2016).
Any teaching process includes the following factors: goals, content, methods,
tools, forms, environment, teacher, students, results. Therefore, to evaluate
teaching for creativity development, it is necessary to focus on the factors of the
teaching process as well as the goal of the creative development of students.
They are interwoven, mixed, and united in the teaching process and the learning
outcomes. Therefore, evaluation of the quality of teaching for creativity
development includes the following aspects: development of students'
creativity; development of knowledge, skills, attitudes toward the discipline and
interdisciplinary; teaching methods and tools; forms of teaching; classroom
environment; classroom management and assessment of learning outcomes.
2.3. Rasch Model
In the 1950s the concept of the item response theory (IRT) was first proposed by
Frederic Lord (Lord, 1952). IRT includes mathematical models to explore
validity and reliability of the questionnaire construct. Among IRT models, the
Rasch model is most commonly applied and its theoretical basis is a description
of the association between item difficulty and person’s ability (Spinou, et al.,
2017).
Currently, the Rasch model is considered an effective and modern approach for
assessing the validation and reliability of a new scale that is being used in
various scientific fields (Assanovich, 2012). For example, the Rasch model is
used to explore psychometric properties of measurement scales such as
medication counseling quality (Tran, Dorofeeva & Loskutova, 2018); life
(Tennant, McKenna & Hagell, 2004); teachers’ content knowledge (Jüttner,
Boone, Park & Neuhaus, 2013). It is also commonly applied in assessing
psychometric properties of the scales being used today, such as claim evaluation
tools (Austvoll-Dahlgren, et al., 2017) and creative achievement questionnaire
(Wang, Ho, Cheng & Cheng, 2014).
The more society develops, the more of new aspects of scientific research arise.
Meanwhile, the previous tools and scales are no longer suitable for application
to research in new issues. Therefore, the need to develop new scales is
increasing. The role of the Rasch model in modern science is more widely
accepted in various fields. This research contributes to the development of the
teaching evaluation aspect of the development of creativity. The questionnaire is
not only for evaluating the pedagogical activity of teachers but also constructed
for managing the quality of education. Based on this questionnaire, educators,
and education management organizations can have clear directions for
enhancing the creative competence of students as well as the quality of
education in general.

283
3. Methodology
Participants.
The current study was conducted in two phases at Hanoi National University of
Education, Vietnam. In the first phase, 20 teachers and 30 third-year students of
faculty of psychology and education participated in the interviews on the criteria
for assessing the quality of teaching for creativity development and 6
educational experts at 2 universities in Hanoi, Vietnam assessed the quality of
the questionnaire “evaluation of the quality of teaching for creativity
development”. Then the questionnaire was used to survey the assessment of 112
students of faculty of psychology and education. In the second stage, a research
survey was conducted to assess the current situation of teaching for creativity
development in the teaching of Pedagogy. This phase involved 300 first- and
second-year students who were grouped in 10 groups and each group evaluated
one teacher. Thus, a total of 10 teachers were evaluated.
Procedures.
In the interview study we invited teachers who research and teach in
psychological and educational sciences and 30 third-year students of the faculty
of psychology and education. They are selected at random to answer open-
ended question "Which criteria should be used to assess the quality of teaching
for creativity development?". We conducted face-to-face interviews in the
classroom. All responses were carefully recorded and analyzed separately, then
divided by themes.
In the expert consultation method, we sent the questionnaire to the experts via e-
mail. The experts evaluated the structure, content as well as evaluation criteria
of the questionnaire.
In the research to assess the quality of the questionnaire “Evaluation of the
quality of teaching for creativity development” based on the Rasch model, we
invited 112 students of psychology and education. Questionnaires were
distributed to students and were collected as soon as they finished their
assessment.
In the research survey on the current situation of teaching for creativity
development in the teaching of Pedagogy, we have invited 300 first- and second-
year students of different faculties. They came from 10 different groups. Each
group evaluated one teacher who has taught the discipline “Pedagogy”. Student
identification information was not disclosed to ensure the objectivity of the
assessment results.
Design and model.
We used both qualitative and quantitative methods in this study. The qualitative
methods included interviews, expert consultation and analysis and synthesis of
previous studies related to students’ creativity development topics in the
learning process. A phenomenological approach used to explore the perceptions,
perspectives and beliefs of teachers and students about the criteria for assessing
the quality of teaching for creativity development. It allowed us to identify the
specific perspectives of the respondents based on their perceptions and
experience.

284
The quantitative methods involved the Rasch analysis and research survey
method. We assessed the psychological attributes of the questionnaire
“Evaluation of the quality of teaching for creativity development” within the
theoretical framework of point-measure correlation coefficient (PTMEA Corr),
item’s infit and outfit values, item’s level of difficulty, reliability, and separation
and strata statistics based on the Rasch model. It was applied using BIGSTEPS
version 2.82 and jMetrik software version 4.0.6. Additionally, we used the
survey method to assess the current situation of teaching for creativity
development in the teaching of Pedagogy. The obtained results were processed
using IBM SPSS Statistics software version 20 for descriptive statistics
(frequency, mean). Figure 1 shows a model that represents the research
methods.
Figure 1: Model of Research Methods.
4. Research Results
4.1. Developing a Questionnaire “Evaluation of the Quality of Teaching for
Creativity Development” Based on The Rasch Model
Based on the implementation of three methods, namely, interviewing, literature
review, and expert consultations, we developed “Evaluation of the quality of
teaching for creativity development Questionnaire” (see Appendix A). The
questionnaire contains 43 items that reflect the quality of teaching for creativity
development in 7 subscales: subscale I—Development of students' creativity
(items 1–14); subscale II—Development of knowledge, skills, attitudes toward
the discipline and interdisciplinary (items 15–24); subscale III—Teaching
methods and tools (items 25–29); subscale IV—Forms of teaching (item 30);
subscale V—Classroom environment (items 31–34); subscale VI—Classroom
management (items 35–37); subscale VII—Assessment of learning outcomes
(items 38–43). Each item was evaluated on a five-point quality Likert scale: 1-
Very Poor, 2- Poor, 3- Acceptable, 4-Good, and 5-Very good (Brown, 2010). The

285
questionnaire considered as a tool for evaluating teaching for creativity

development that allows us to determine the degree of achievement of teachers
in the development of students’ creative competence.
Table 1: Mean Score, Item’s Difficulty, Fit Statistics and Point Measure Correlation
Coefficient (N=108 students)
Mean Infit Outfit PTMEA
Subscale Item Difficulty
score MnSq MnSq Corr
1 3.56 0.34 0.62 0.62 0.712
2 3.44 0.60 1.36 1.37 0.571
3 3.90 -0.39 1.28 1.27 0.342
4 3.50 0.47 1.03 1.02 0.645
5 3.45 0.57 1.08 1.09 0.716
6 3.38 0.72 0.94 0.93 0.626
I. Development of students' 7 3.43 0.62 0.92 0.91 0.757
creativity 8 3.23 1.01 1.15 1.16 0.540
9 3.19 1.11 0.98 0.98 0.672
10 3.22 1.03 1.47 1.48 0.571
11 3.32 0.83 1.01 1.00 0.668
12 3.41 0.66 1.54 1.52 0.585
13 3.58 0.30 0.88 0.87 0.724
14 3.59 0.28 0.87 0.87 0.635
15 3.56 0.34 0.96 0.97 0.588
16 3.59 0.28 0.89 0.91 0.575
17 3.31 0.85 0.80 0.82 0.627
II—Development of 18 3.33 0.81 0.68 0.69 0.653
knowledge, skills, attitudes 19 3.59 0.28 0.83 0.85 0.589
toward the discipline and 20 3.18 1.12 0.75 0.75 0.566
interdisciplinary work 21 4.06 -0.79 0.67 0.67 0.708
22 3.80 -0.16 1.04 1.02 0.427
23 3.85 -0.29 0.90 0.93 0.488
24 3.55 0.38 1.11 1.10 0.531
25 3.81 -0.20 0.79 0.78 0.692
26 4.04 -0.72 0.89 0.84 0.656
III—Teaching methods and
27 3.75 -0.06 0.78 0.80 0.618
tools
28 3.85 -0.29 0.83 0.81 0.635
29 4.07 -0.81 0.85 0.81 0.618
IV—Forms of teaching 30 4.34 -1.54 1.21 1.20 0.463
31 4.09 -0.86 0.82 0.82 0.685
32 3.96 -0.54 0.96 0.94 0.642
V—Classroom environment
33 3.99 -0.61 0.99 1.02 0.604
34 4.06 -0.77 1.44 1.42 0.437
35 3.79 -0.14 1.39 1.39 0.312
VI—Classroom
36 3.81 -0.18 0.80 0.81 0.629
management
37 4.24 -1.25 0.72 0.72 0.672
38 3.97 -0.56 1.28 1.26 0.506
39 3.75 -0.06 1.42 1.36 0.605
VII—Assessment of learning 40 4.03 -0.70 1.25 1.34 0.628
outcomes 41 4.03 -0.70 0.95 0.93 0.682
42 3.95 -0.52 0.93 0.92 0.675
43 3.94 -0.48 0.88 0.87 0.709

286
This questionnaire was used to survey the assessment of teachers’ teaching for
creativity development by 112 students. Results were analyzed using BIGSTEPS
version 2.82 and jMetrik software version 4.0.6 (for Rasch analysis). For
conducting Rasch analysis, it is required a sample size with at least 100
participants. Therefore, this study sample of 112 students was considered as
appropriate (Jackson, et al., 2020). Person fit statistic was conducted to find out
outliers (misfitting persons) among 112 students. The result of the person fit
statistics yielded that four students (No 38, 98, 105, and 109) were found as unfit
persons because of their Infit and Outfit MnSq values >2.0. Therefore, the
persons were removed from the study data. After eliminating these four
persons, all the remaining students were found to be within the limits of
acceptance. Infit MnSq values of 108 students ranged from 0.28 to 1.90 and
Outfit MnSq value – from 0.27 to 1.90. The analytical results of the psychometric
properties of the questionnaire after removing misfit persons include validation
assessment (item’s difficulty, item fit statistics, point-measure correlation
coefficient (PTMEA Corr), response category statistics) and reliability
assessment (reliability index, separation index, strata index, and Cronbach’s
alpha) (see Tables 1, 2 and 3).
Item polarity was estimated by using the point-measure correlation coefficient
(PTMEA CORR). The acceptable range of values is from 0.3 to 0.8. Under this
condition, the items are working in the same direction to measure a single basic
construct (Allen, 2001). Any item with a PTMEA CORR value outside this range
should be removed from the scale (Bond, Yan & Heene, 2020). Table 1 shows
that the polarity of items ranges from 0.312 to 0.757. All items of the scale are in a
scope consistent with the Rasch model which means that the questionnaire has a
very good polarity of the correlated items.
Infit and outfit mean square (MnSq) values are important statistics when using
the Rasch model. They allow the identification of items that fit the Rasch model
accurately. The MnSq fit statistic values in the range from 0.5 to 1.5 are optimal
(Meyer, 2014). MnSq values below 0.5 or between 1.5–2.0 are considered to be
ineffective for building the scale. However, they do not violate the overall
measuring qualities of the scale (Wright, Linacre, 1994). An item with an MnSq
value greater than 2.0 should be removed from the scale. Table 1 shows that the
Infit and Outfit MNSQ values of all items ranged from 0.62 to 1.54, which
reflects that the questionnaire has very good construct validity. There isn’t any
item that needs editing or omitting.
Item difficulty indicates the relationship between the person’s ability and the
probability of a correct response (Baker, 2001). The range of the item difficulty
value from -2.0 to 2.0 is considered appropriate in the Rasch model. An item
with a difficulty value of less than -2.0 is interpreted as extremely simple, and an
item with a difficulty value of more than 2.0 is considered extremely difficult
(Assanovich, 2014). The results in Table 1 indicate that the difficulty values of all
items were in the range from -1.54 to 1.12. Thus, all 43 items of the scale had an
acceptable difficulty value within the Rasch normative model. Among 43 items,
item 9 which “students choose the most creative ideas to solve problems or
create new products” (1.11) and item 20 which “development of the
interdisciplinary skills” (1.12) proved to be the most difficult items. They reflect

287
subscales, “development of students' creativity” and “development of

knowledge, skills, attitudes of the discipline and interdisciplinary.” This shows
that in teaching for creativity development, process of organizing students to
come up with solutions, ideas to solve problems and choose the most creative
ideas among them is a difficult skill. Besides, the development of
interdisciplinary skills is also a challenge for teachers. Maybe because the
content of the subject is too much, the time for studying the subject is limited
and the teachers don't have good pedagogical skills in organizing creative
classes. Item 30 which “the teacher uses individual and group forms” (-1.54) and
item 37 which “The teacher praises the good behavior and success of the
students” (-1.25) were the easiest items. They reflect subscales: “forms of
teaching” and “classroom management”. This shows that in the teaching
process, teachers often combine individual and group learning forms as well as
promptly praise good behaviors of students.
Table 2: Response Category Statistics of Items for the Questionnaire (N=108 students)
Category Level Threshold Infit MnSq Outfit MnSq
1 Very poor - - -
2 Poor -2.84 0.94 0.94
3 Acceptable -0.95 1.02 1.02
4 Good 0.82 1.04 1.02
5 Very good 2.98 1.03 1.02
The results of response category statistics of items for the questionnaire are
shown in Table 2. Categories statistics found that there was a gradual increase in
the difficulty level from category 1 (very poor) to 5 (very good). In other words,
it is difficult for students to get a category score threshold of 5 rather than 4.
Similarly, category score threshold 4 is more difficult than 3, 3 >2, and 2 >1. The
large difference between the two successive categories is considered as good
(>1.7). Additionally, the categories had good Infit and Outfit statistics, with
acceptable values from 0.94 to 1.04.
Table 3: Reliability, Separation Index and Strata Index, and Cronbach’s Alpha (N=108
students)
Index Overall scale Person
Reliability 0.955 0.961
Separation 4.582 4.984
Number of strata 6.442 6.979
Cronbach’s alpha 0.963 -

288
The results of the reliability index and the separation are shown in Table 3. In
the Rasch model, reliability values are in the range from 0 to 1. If the value is less
than 0.5 the item is considered of unacceptable reliability; 0.5 to 0.6—poor; 0.6 to
0.7—questionable; 0.7 to 0.8—acceptable; 0.8 to 0.9—good; and more than 0.9—
very good reliability. The scale of reliability of this study was 0.955 and the
persons’ reliability was 0.961, which is indicative of the questionnaire having
very good reliability (range greater than 0.9) and items had good internal
consistency in the measurement construct of scale.
According to Fisher (2007), separation values above 3 are considered good. The
result presented in Table 3 shows that the separation of the scale was 4.582,
separation of a person’s ability was 4.984. As suggested by Assanovich (2014), an
index of the number of strata of the scale greater than 1 is considered
appropriate. And, naturally, higher strata values are considered more reliable. In
this study, the scale’s number of strata was separated into six distinct levels
(6.442), strata of person’s ability was 6.979. This indicated a very good level of
the questionnaire’s reliability in our study. Cronbach’s alpha coefficient of the
scale was also at a high level (0.963).
4.2. Application of the Questionnaire in Assessing the Real Situation of
Teaching for Creativity Development in the Teaching of Pedagogy.
In the second stage, we have applied the questionnaire to assess the current
situation of teaching for creativity development in the teaching of Pedagogy.
This phase involved 300 first- and second-year students divided into 10 groups,
1 teacher was evaluated by 1 group (30 students). According to the guidelines
provided by Johannessen, Tufte and Christoffersen (2010), the number of
respondents should not be less than 30 people. The obtained results were
processed using IBM SPSS Statistics software version 20 and are represented by
the mean score of each subscale and each teacher (see Table 4).
Table 4 shows the degree of teaching for creativity development in the teaching
of Pedagogy to be at an acceptable: 3.05. The subscales with the highest score
were IV—Forms of teaching: 3.81 (rank 1); V—Classroom environment: 3.49
(rank 2); VII—Assessment of learning outcomes: 3.23 (rank 3). These are three
aspects in which the teachers performed better than in other aspects. However,
teachers still need to implement them more effectively in the teaching process to
further develop students’ creativity. The subscales with the lowest score were
I—Development of students' creativity (2.73) and III—Teaching methods and
tools (2.96). These are two aspects that need more attention from the teachers to
perform better in the evaluation of the teaching process.
Table 4 also shows the degree of teaching for creativity development of 10
teachers at an acceptable level. We see that they have diverse average scores
ranging from 3.02 to 3.08. However, this difference was negligible. At Hanoi
National University of Education, the discipline "Pedagogy" is a compulsory
common subject for pedagogical all students. Each class usually has from 100 to
120 students with different majors. The creativity development for students in
the teaching process was not focused. Hence, the actual assessment of the
teaching for creativity development is considered necessary to bring

289
effectiveness to future experimental work. The results of detail assessment

including each subscale and items for 10 teachers are shown in Appendix A (see
Appendix A).
Table 4: The Degree of Teaching for Creativity Development in the Teaching of
Pedagogy among 10 teachers (N=300 students)
Teacher (Mean)
Subscale
Subscale
mean
1 2 3 4 5 6 7 8 9 10
I—Development
of students' 2.83 2.75 2.71 2.72 2.71 2.70 2.71 2.74 2.73 2.72 2.73
creativity
II—Development
of knowledge,
skills, attitudes
toward the 3.16 3.14 3.23 3.25 3.20 3.11 3.18 3.18 3.11 3.05 3.16
discipline and
interdisciplinary
work
III—Teaching
methods and 2.88 2.91 2.80 2.99 2.87 2.78 2.99 3.07 3.17 3.11 2.96
tools
IV—Forms of
3.90 3.77 3.73 3.83 3.80 3.93 3.77 3.67 4.03 3.67 3.81
teaching
V—Classroom
3.44 3.45 3.47 3.41 3.43 3.48 3.61 3.62 3.55 3.43 3.49
environment
VI—Classroom
2.99 2.94 2.97 3.14 3.13 3.09 3.22 3.24 3.10 3.08 3.09
management
VII—Assessment
of learning 3.23 3.23 3.23 3.14 3.26 3.30 3.23 3.15 3.24 3.25 3.23
outcomes
Overall scale
mean for each 3.06 3.03 3.03 3.05 3.04 3.02 3.07 3.08 3.07 3.03 3.05
teachers
5. Discussion
The goal of current study is to clarify two research questions: 1) Does the
questionnaire “evaluation of the quality of teaching for creativity development”
conform to the requirements of the Rasch model? And 2) What is the current
status of teaching for creativity development in the teaching of Pedagogy at
Hanoi National University of Education, Vietnam?.
The questionnaire “evaluation of the quality of teaching for creativity
development” contains 43 items. The Rasch model was applied to assess the
quality of the questionnaire using BIGSTEPS version 2.82 and jMetrik software
version 4.0.6. The measurement was carried out within the framework of the
theory of point-measure correlation coefficient (PTMEA Corr), item’s infit and
outfit values, item’s level of difficulty, reliability, and separation and strata
statistics of the questionnaire.

290
The acceptable range of PTMEA Corr values is from 0.3 to 0.8 (Allen, 2001). In
this study 43 items had PTMEA Corr values between 0.312 and 0.757. The MnSq
fit statistic values in the range from 0.5 to 1.5 are optimal (Meyer, 2014). All
items had Infit and Outfit MNSQ values in the range from 0.62 to 1.54. The range
of the item difficulty value from -2.0 to 2.0 is considered appropriate in the
Rasch model (Assanovich, 2014). All items had item difficulty values in the
range from -1.54 to 1.12. The scale reliability of this study was 0.955 and the
persons’ reliability was 0.961. Separation values above 3 are considered good
(Fisher, 2007). The separation of the scale was 4.582, separation of a person’s
ability was 4.984. An index of the number of strata of the scale greater than 1 is
considered appropriate. And, naturally, higher strata values are considered
more reliable (Assanovich, 2014). In this study, the scale’s number of strata was
separated into six distinct levels (6.442), strata of person’s ability was 6.979.
Cronbach’s alpha coefficient of the scale was also at a high level (0.963).
The Rasch analysis showed multiple sources of evidence to test the quality of the
questionnaire. Results indicated that the questionnaire was in good compliance
with the Rasch model requirements. It may be used to measure the quality of
teaching for creativity development and improve teachers’ teaching for
creativity development in higher education.
We used the questionnaire to assess the current status of teaching for creativity
development in the teaching of Pedagogy. The results showed that the degree of
the teachers’ teaching for creativity development at an acceptable level (3.05).
This shows that students' creative capacity has not been developed in the
teaching process.
In order to improve teachers’ teaching for creativity development, we believe
that it is necessary to do the following:
- Build an environment that encourages creative teaching and learning in
universities.
- Organize teacher training on teaching for creativity development.
- Actively organize attractive creative activities associated with teaching and
creative development goals in the classroom, in university as well as in society
for students to participate.
6. Recommendations
Based on the analyzed results, we make the following recommendations for
future research:
- It is necessary to organize the teacher training focusing on methods, tools,
forms, building a classroom environment, assessment of learning outcomes for
creative development.
- Equip students with knowledge and skills about creativity to motivate them to
develop creative capacity in the learning process at the university.
- It is necessary to carry out empirical studies to determine the effectiveness of
teachers’ teaching for creativity development in enhancing learning outcomes
and student developing creativity.

291
7. Conclusions
In this study, the questionnaire “evaluation of the quality of teaching for
creativity development” was created to assess the current status and improve
teachers’ teaching for creativity development. We used three methods to
develop the questionnaire: interviewing, synthesizing documents, and expert
consultation. It consisted of seven subscales: development of students’ creativity;
development of knowledge, skills, attitudes toward the discipline and
interdisciplinary work; teaching methods and tools; forms of teaching; classroom
environment; classroom management; assessment of learning outcomes. The
items are rated on a five-point quality Likert scale.
The Rasch model was applied to assess the quality of the questionnaire using
BIGSTEPS version 2.82 and jMetrik software version 4.0.6. The measurement
was carried out within the framework of the theory of point-measure correlation
coefficient (PTMEA Corr), item’s infit and outfit values, item’s level of difficulty,
reliability, and separation and strata statistics of the questionnaire. The results
showed that the questionnaire conformed closely with the Rasch model’s
requirements and may be used to measure the quality of teaching for creativity
development in different subjects in higher education.
The questionnaire has been applied to assess the current status of teaching for
creativity development in the teaching of Pedagogy at Hanoi National
University of Education, Vietnam. The results showed that the degree of the
teachers’ teaching for creativity development at an acceptable level (3.05).
Therefore, further research is needed to conduct experimental pedagogical work
on the formation of students’ creativity in the teaching of Pedagogy.
8. References
Allen, M. J., & Yen W. M. (Eds.). (2001). Introduction to Measurement Theory. USA:
Waveland Press.
Assanovich, M. A. (2014). The problem of scientific measurement in psychodiagnostics. J.
Grodn. State Medical University, 1(45), 9-13.
Assanovich, M. A. (Ed.). (2012). Clinical psychodiagnostics. Minsk: “Belarus”.
Austvoll-Dahlgren, A., Guttersrud Ø., Nsangi, A., Semakula, D., & Oxman, A. D. (2017).
Measuring ability to assess claims about treatment effects: a latent trait analysis
of items from the ‘Claim Evaluation Tools’ database using Rasch modelling. BMJ
Open, 7(5), e013185. https://doi.org/10.1136/bmjopen-2016-013185
Baker, F. B. (Ed.). (2001). The Basics of Item Response Theory (2nd ed.). USA: ERIC.
Beghetto, R. A. (2005). Does Assessment Kill Student Creativity? The Educational Forum,
69, 254–263.
Beghetto, R. A., & Kaufman, J. C. (2014). Classroom contexts for creativity. High Ability
Studies, 25(1), 53–69. http://dx.doi.org/10.1080/13598139.2014.905247
Benedek, M., Bruckdorfer, R., & Jauk, E. (2019) Motives for Creativity: Exploring the
What and Why of Everyday Creativity. The Journal of Creative Behavior, 1–16.
https://doi.org/10.1002/jocb.396
Biswas, G. K. (2011). Technological Approach in Teaching History, Musings, 2, 127–129.
Bond, T. G., Yan, Z., & Heene, M. (2020). Applying the Rasch Model (4th ed.). London:
Routledge.
Brown, S. (2010). Likert scale examples for surveys. Retrieved from
http://www.extension.iastate.edu/ag/staff/info/likertscaleexamples.pdf

292
Bui, T. L. T., & Kazarenkov, V. I. (2020). Q-Methodology: A New Way to Develop an

Effective Teaching Model for the Development of Students’ Creative Activity.
International Journal of Innovative Technology and Exploring Engineering, 9(3), 1495-
1501. https://doi.org/10.35940/ijitee.C8317.019320
Bui, T. L. T., Kazarenkov, V., Tran, V. D, Dorofeeva, V., & Nguyen, T. (2019). Use of Q-
methodology for studying students’ creative activity. The European proceedings of
social & behavioural sciences, 76(469), 3492-3496. Retrieved from
https://www.futureacademy.org.uk/files/images/upload/SCTCMG2019FA46
9.pdf
Conklin, W. (Ed.). (2012). Strategies for Developing Higher-Order Thinking Skills, Grades 6-
12. Huntington Beach, CA: Shell Educational Publishing.
Conner, T. S., DeYoung, C. G., & Silvia P. J. (2017). Everyday creative activity as a path to
flourishing. Journal of Positive Psychology, 13(2), 181-189.
http://dx.doi.org/10.1080/17439760.2016.1257049
Conner, T., & Silvia, P. (2015). Creative days: A daily diary study of emotion,
personality, and everyday creativity. Psychology of Aesthetics, Creativity, and the
Arts, 9(4), 463-470. https://doi.org/10.1037/aca0000022
Craft, A. (2005). Creativity in schools: Tensions and dilemmas. London: Routledge.
Davies, D., Jindal-Snape, D., Collier, C., Digby, R., Hay, P., & Howe, A. (2013). Creative
Learning Environments in Education—A Systematic Literature Review. Thinking
Skills and Creativity, 8, 80-91. https://doi.org/10.1016/j.tsc.2012.07.004
Elisondo, R. (2016). Creativity is always a social process. Creativity. Theories–
ResearchvApplications, 3(2), 194-210. https://doi.org/10.1515/ctra-2016-0013
Elisondo, R. C., & Vargas, A. (2019). Women’s everyday creative activities: a qualitative
study. Creativity. Theories – Research – Applications, 6(1), 91-111.
https://doi.org/10.1515/ctra-2019-0006
Engel, S. (2013). The Case for Curiosity. Educational Leadership, 70, 36-40.
Esquivel, G. B. (1995). Teacher behaviours that foster creativity. Educational Psychology
Review, 7(2), 185 – 202.
Ferrari, A., Cachia, R., & Punie, Y. (2009). Innovation and Creativity in Education and
Training in the EU Member States: Fostering Creative Learning and Supporting
Innovative Teaching. JRC Technical Notes. Retrieved from
http://ftp.jrc.es/EURdoc/JRC52374_TN.pdf
Fisher, W. (2007). Rating scale instrument quality criteria. Rasch Meas Trans, 21(1), 1087–
1096.
Glaveanu, V. (2013). Rewriting the language of creativity: The five A's framework.
Review of General Psychology, 17, 69-81. https://doi.org/10.1037/a0029528
Glaveanu, V. (2018). Educating which creativity? Thinking Skills and Creativity, 27, 25-32.
Gorshunova, N. K., Medvedev, N. V., & Razdorskaya, O. V. (2014). The Significance of
High School Teacher’s Creativity for Innovation Pedagogical Practice. Journal of
International Scientific Publications, 12, 607-614.
Hamed, P. K., Preece, A. S. D., & Hashim, C. N. (2016). Factors influencing creative
teaching among teachers in Islamic private schools in Kuala Lumpur. Proceedings
of the International Conference on Education towards Global Peace. Retrieved from
http://www.iium.edu.my/capeu2016/wp content/uploads/2017/02/03 1.pdf
Hong, H. Y., Chang, Y. H., & Chai, C. S. (2014). Fostering a collaborative and creative
climate in a college class through idea-centered knowledge-building.
Instructional Science, 42(3), 389–407. http://dx.doi.org/10.1007/s11251-013-9289-
y

293
Jackson, A., Rogerson, M., Le Grande, M., Thompson, D., Ski, C., Alvarenga, M.,
Amerena, J., Higgins, R., Raciti, M., & Murphy, B. M. (2020). Protocol for the
development and validation of a measure of persistent psychological and
emotional distress in cardiac patients: The Cardiac Distress Inventory. BMJ
Open, 10(6). http://dx.doi.org/10.1136/bmjopen-2019–034946.
Johannessen, A., Tufte, P. A., & Christoffersen, L. (2010). Introduksjon til
samfunnsvitenskapelig metode [Introduction to social science methodology] (4th ed.).
Oslo: Abstrakt forlag.
Jüttner, M., Boone. W., Park. S., & Neuhaus. B. J. (2013). Development and use of a test
instrument to measure biology teachers’ content knowledge (CK) and
pedagogical content knowledge (PCK). Educ Asse Eval Acc.
http://dx.doi.org/10.1007/s11092-013-9157-y
Kaplan, D. E. (2019). Creativity in Education: Teaching for Creativity Development.
Psychology, 10, 140-147. https://doi.org/10.4236/psych.2019.102012
Kazarenkov, V., Bui, T., Kazarenkova, T., & Kameneva, G. (2020). Studying the
development factors of students’ creativity based on Q-methodology. Proceedings
of the International Scientific Conference, 1, 333-340.
http://dx.doi.org/10.17770/sie2020vol1.5144
Lehrer, J. (Ed.). (2012). Imagine: How Creativity Works. Boston, MA: Houghton Mifflin
Harcourt.
Lord F. (Ed.). (1952). A Theory of Test Scores. Richmond, VA: Psychometric Corporation.
Loveless, A. M. (Ed.). (2002). Literature Review in Creativity, New Technologies and Learning:
A report for Futurelab. Bristol: Futurelab.
Meyer, J. (Ed.). (2014). Applied measurement with jMetrik. New York: Routledge.
National Advisory Committee on Creative & Cultural Education (Ed.). (1999) All our
futures: creativity, culture and education. Report of the National Advisory Committee
on Creative and Cultural Education. Sudbury: DfEE.
Ng, A. K. (2002). The Development of a New Scale to Measure Teachers’ Attitudes
toward Students (TATS). Educational Research Journal, 17(1), 63-77.
Pang, W. (2015). Promoting creativity in the classroom: A generative view. Psychology of
Aesthetics. Creativity, and the Arts, 9(2), 122-127.
Pang, W., & Plucker, J. A. (2012). Recent transformations in China’s economic, social, and
education policies for promoting innovation and creativity. The Journal of
Creative Behavior, 46, 247–273. http://dx.doi.org/10.1002/jocb.17
Rankin, J., & Brown, V. (2016). Creative teaching method as a learning strategy for
student midwives: A qualitative study. Nurse Education Today, 38, 93–100.
http://dx.doi.org/10.1016/j.nedt.2015.12.009
Richards, R. (2010). Everyday creativity: Process and way of life—four key issues. In J. C.
Kaufman & R. J. Sternberg (Eds.), The Cambridge handbook of creativity (pp. 189-
215). New York: Cambridge University Press.
Richardson, C., & Mishra, P. (2018). Learning Environments that Support Student
Creativity: Developing the SCALE. Thinking Skills and Creativity, 27, 45-54.
https://doi.org/10.1016/j.tsc.2017.11.004
Runco, M. A., & Jaeger, G. J. (2012). The Standard Definition of Creativity. Creativity
Research Journal, 24, 92-96. http://dx.doi.org/10.1080/10400419.2012.650092
Sadeghi, A., & Ofoghi, N. (2011). The psychological factors affecting students' creativity
inside the class (CIC) (case study the University of Guilan, Iran). Procedia Social
and Behavioral Sciences, 15, 263–270. https://doi.org/10.1016/j.sbspro.2011.03.084

294
Sahlberg, P. (2009). The Role of Education in Promoting Creativity: Potential Barriers

and Enabling Factors. In E. Villalba (Ed.), Measuring Creativity (pp. 337-344).
Luxemburg: OPOCE.
Shaheen, R. (2010). Creativity and education. Creative Education, 1, 166–169.
http://dx.doi.org/10.4236/ce.2010.13026
Shrivastava, S. (2016). Creativity and intelligence. International Journal of Recent Advances
in Multidisciplinary Research, 3(3), 339-1344.
Silvia, P. J., Beaty, R. E., Nusbaum, E. C., Eddington, K. M., Levin-Aspenson, H., &
Kwapil, T. R. (2014). Everyday creativity in daily life: An experience-sampling
study of “little c” creativity. Psychology of Aesthetics, Creativity, and the Arts, 8,
183-188. http://dx.doi.org/10.1037/a0035722
Spinou, A., Siegert, R. J., Guan, W., Patel, A. S., Gosker, H. R., Lee, K. K., Elston, C.,
Loebinger, M. R., Wilson, R., Garrod, R., & Birring, S. S. (2017). The development
and validation of the Bronchiectasis Health Questionnaire. Eur Respir J, 49(5):
1601532. http://dx.doi.org/10.1183/13993003.01532-2016
Starko, A. J. (Ed.). (2013). Creativity in the Classroom Schools of Curious Delight. New York:
Routledge
Sternberg, R. J. & Lubart, T. I. (1999). The concept of creativity: Prospects and paradigms.
In R. J. Sternberg (Ed.), Handbook of creativity (pp. 3-15). New York: Cambridge
University Press.
Tennant, A., McKenna. S. P., & Hagell, P. (2004). Application of Rasch analysis in the
development and application of quality of life instruments. Value in health,
7(Suppl 1): S22–S26.
Tran, L. T. B., Ho, N. T., & Hurle, R. J. (2016). Teaching for Creativity Development:
Lessons from a Preliminary Study of Vietnamese and International Upper
(High) Secondary School Teachers’ Perceptions and Lessons Plans. Creative
Education, 7, 1024-1043. http://dx.doi.org/10.4236/ce.2016.77107
Tran, V. D., Dorofeeva, V. V., & Loskutova, E. E. (2018). Development and validation of a
scale to measure the quality of patient medication counseling using Rasch
model. Pharmacy Practice, 16(4), 1327.
https://doi.org/10.18549/PharmPract.2018.04.1327
Trnka, R., Zahradnik, M. Y., & Kuška, M. (2016). Emotional Creativity and Real-Life
Involvement in Different Types of Creative Leisure Activities. Creativity Research
Journal, 28(3), 348-356.
UNESCO (2015). Cultural times The first global map of cultural and creative industries.
Retrieved from
https://en.unesco.org/creativity/sites/creativity/files/cultural_times_the_first
_global_map_of_cultural_and_creative_industries.pdf
Vygotsky, S. L. (2004). Imagination and Creativity in Childhood. Journal of Russian and
East European Psychology, 42, 7-97.
Walia, C. (2019): A Dynamic Definition of Creativity. Creativity Research Journal, 31(3),
237-247. https://doi.org/10.1080/10400419.2019.1641787
Wang, C. C., Ho, H. C., Cheng, C. L., & Cheng, Y. Y. (2014) Application of the Rasch
Model to the Measurement of Creativity: The Creative Achievement
Questionnaire. Creativity Research Journal, 26(1), 62-71.
https://doi.org/10.1080/10400419.2013.843347

295
Appendix A: The questionnaire “Evaluation of the Quality of Teaching for

Creativity Development” and the results of detail assessment including each
subscale and items for 10 teachers (N =300 students)
Item Content Mean SD
I. Development of Students' Creativity

1.1. Development of the ability of curiosity and discovery 2.62 0.51
+ Students ask questions and answer 2.26 0.73
+ The teacher asks questions, students answer 2.97 0.76
1.2. Development of imagination competence 2.48 0.53
+ Creative imagination 2.12 0.73
+ Reproduced imagination 2.83 0.78
1.3. Development of the creative thinking competency 2.35 0.51
+ Students comment on scientific critical ideas 2.25 0.72
+ Students choose the most creative ideas to solve problems or
2.45 0.69
create new products
1.4. Students have creative products 2.56 0.4
+ Creative ideas/solutions 2.42 0.71
+ Creative pictures, diagrams, products 2.42 0.68
+ New questions, views, perspectives 2.84 0.7
1.5. Creative attitude: students desire and have a creative
3.04 0.74
consciousness.
II. Development of Knowledge, Skills, Attitudes toward the
discipline and interdisciplinary
2.1. Development of knowledge of the discipline and
3 0.55
interdisciplinary
+ Deep understanding of the discipline's knowledge 3.42 0.76
+ Development of the interdisciplinary knowledge 2.59 0.74
2.2. Development of skills of the discipline and interdisciplinary 3.49 0.59
+ Development of the discipline's skills 3.51 0.76
+ Development of the interdisciplinary skills 3.18 0.75
+ Development of skills: communication, presentation, self-study,
3.8 0.78
teamwork ...
2.3 Consciousness, civic attitude 3.52 0.59
+ Students work seriously, collaborate with other students and
3.59 0.65
teacher
+ Students are aware of the good or limitations of the problem
3.46 0.89
they have learned to take the right action.
III. Teaching Methods and Tools
3.1. Creative, diverse, integrated methods and tools enable
2.91 0.68
students to deepen knowledge and practice skill development.
3.2. The teacher uses ICT fluently to save time and help students
2.63 0.71
to be more active, positive and creative.
3.3. The teacher uses the students' experiences 3.46 0.8

296
3.4. The teacher uses real-life situations. 3.34 0.8

3.5. The teacher encourages students to discuss and solve
2.46 0.71
problems with multiple solutions.
IV. Forms of Teaching
4.1. The teacher uses individual and group forms 2.37 0.72
V. Classroom Environment
5.1. The teacher and students have a positive relationship during
2.78 0.73
school hours.
5.2. The classroom has a positive and exciting psychological
2.76 0.93
atmosphere.
5.3. The teacher and students listen to, respect, explore, and
support the diverse, different ideas that each student brings to the 2.56 0.75
class.
5.4. The teacher ensures a good temperature and lighting in the
4.06 0.87
classroom
VI. Classroom Management
6.1. Establishing behavioral rules in the classroom. 3.65 0.79
6.2. The teacher observes, embraces students, and responds to
2.38 0.63
students' different needs and abilities.
6.3. The teacher praises the good behavior and success of the
2.68 0.6
students
VII. Assessment of Learning Outcomes
7.1. The teacher uses different types of assessment 3.84 0.78
7.2. There is a combination of teacher assessment with student
3.21 0.98
assessment.
7.3. The teacher provides public assessment criteria and results. 3.37 0.8
7.4. The teacher ensures value, reliability, fairness and efficiency in
3.46 0.66
the assessment process.
7.5. The teacher appreciates creative ideas, products, and valuable
3.48 0.66
practical applications.
7.6. The teacher asks the students to respect each other's ideas. 3.46 0.56
1- Very Poor, 2- Poor, 3- Acceptable, 4-Good, and 5-Very good.

297

Vol. 19, No. 8, pp. 297-312, August 2020
The Challenges of South African Teachers in

Teaching Euclidean Geometry
Simon A. Tachie
University of the Free State, South Africa
https://orcid.org/0000-0003-1535-7312
Abstract. The current study identifies challenges confronting teachers in

the teaching of Euclidean geometry in schools. This qualitative case study
purposefully selected ten schools situated in the Motheo District of
Education, Free State, South Africa. Data was generated using classroom
observations and focus group interviews. The findings revealed that the
majority of the teachers lacked mathematical subject content knowledge
as well as pedagogical content knowledge to teach geometry effectively.
In many cases, redeployment of teachers led to teachers being ill-
equipped to teach specific subjects, such as mathematics.
Recommendations have been made to enhance effective teaching and
learning of mathematics (geometry in this case) in schools.
Keywords: Euclidean geometry; Subject content knowledge; Pedagogical

content knowledge; Teacher challenges; Mathematics teaching
1. Introduction
Since the birth of democracy in South Africa in 1994, education has experienced
curriculum reforms with changes affecting all levels of education, mathematics
included. These changes have retained the review and reform of the mathematics
curriculum, and this has impacted on teachers’ content knowledge as many
teachers, during both their secondary and teacher education, did not study some
of the newly introduced mathematical concepts such as Euclidean geometry.
Morris (1985, p.92) argues that “the root cause of poor geometry teaching lies in
the mismatch between teacher education courses and the needs of the learners in
the school,” meaning that teachers have been trained on some aspects of geometry
which are not related to what they are to teach in a classroom situation. These
teachers, in most cases, find themselves inadequately prepared to teach Euclidean
geometry as expected of them since they are not conversant with the use of
relevant skills and strategies (Luthuli, 1996).
Related studies have increasingly emphasized geometry education research,

which recommends the use of technological tools such as dynamic geometry
software in the teaching and learning of geometry. These studies have indicated
how the use of technologic tools impacts on learners’ geometrical thinking skills

298
(especially on the teaching and learning of geometrical reasoning and proving),

on teachers’ geometric content knowledge, and teacher development for
geometry education (Jones & Tzekaki, 2016). However, little is known about how
South African mathematics teachers use those technological tools since geometry
had not initially been included in the curriculum. To throw more light on the
above issue, teachers graduating from the universities from 2013 had not studied
Euclidean geometry during their formal education, either in their secondary
school education or in tertiary training. The challenges of handling Euclidean
geometry by these teachers are, therefore, more problematic than for those
teachers who have studied the subject and who have not trained adequately for
teaching classroom Euclidean geometry. This under-teaching or non-teaching of
Euclidean geometry poses a severe threat to the performance of the learning of
mathematics, hence the current study. This study aimed to identify all challenges
experienced by these teachers in the teaching of geometry in schools and to find
solutions to the problem at hand. In this case, it was to support mathematics
classroom teachers in the teaching/learning of mathematics in the schools of the
Motheo education district of Free State, South Africa. In light of the preceding,
this primary research question is: What are the main challenges faced by teachers
in teaching/learning of geometry in schools?
2. Teaching of Geometry
Studies have shown that geometry is not only difficult for learners who have to
learn the subject, but similarly for the teachers who have to teach it. Teachers lack
subject content knowledge and also the understanding of other strategies to teach
the subject to perfection since it deals with practical activities. Focus on
geometrical skills is also vital in areas of construction work, architectural design,
and engineering (Ubah & Bansilal, 2019; Alex & Mammen, 2018). Much of the
geometry teaching in the country has been based on immersive and adaptive
instruction and learning where teachers move from a conceptual approach (the
sensible application of procedures) to a procedural approach (calculation
accuracy). Scholars such as Brown (1999) and Khoo and Clements (2001) affirmed
that learners have an inadequate understanding of geometrical concepts,
reasoning, and problem-solving skills. The afore-mentioned statement supports
the analysis of the National Senior Certificate (NSC) diagnostic report per
question, which shows a declining performance of learners in Mathematics paper
2, particularly in Euclidean geometry (Chihanbakwe, 2017).
Brannan et al. (2002) stated that geometry is a branch of mathematics concerned

with the properties of configurations of points, lines, angles, circles, and the most
basic of these figures. It deals with axioms and proofs of theorems through
deductive thinking (Mamali, 2015). Bassarear (2012) defined geometry as “the
study of shapes, their relationships, and properties.” In the same way, Güven and
Kosa (2008) maintain that geometry is the study of space and shape in which more
excellent reasoning capability is necessary to understand the concept well. From
these definitions of geometry, it is quite clear that when dealing with geometry,
the focus is on identifying shapes, measuring different aspects of those shapes,
and categorizing such shapes according to their properties.

299
Geometry originated from several ancient cultures, including Indian, Babylonian,

Egyptian, and Chinese cultures (Ding & Jones, 2006; Jones, 2002). These ancient
cultures used geometry to measure lengths, heights, angles, directions, and
distances. Geometry is an axiomatic and deductive study of mathematics is linked
to critical thinking and logical reasoning. There are different types of geometries
depending on the planes, angles, and approaches they take. Coordinate
(analytical geometry), as an example, is Cartesian-plane driven.
The British Mathematician, Sir Christopher Zeeman saw geometry as that which
comprises those branches of mathematics that exploit visual intuition, the most
dominant of our senses, requiring remembering of theorems, understanding of
proofs, inspiring of conjectures, perceiving of reality, and giving global insight
(Jones, 2002). Indeed, those who are geometrically inclined tend to have a high
level of memory restoration and can easily link or relate events and occurrences,
and this is one of the reasons why Euclidean geometry was introduced as a core
topic in the South African school curriculum.
2.1 The teaching of Geometry in South African schools

According to Kotzé (2007), Euclidean geometry (or – bodies of knowledge
consisting of statements justified by proofs, which depend on mathematical
axioms and an underlying logic) requires educators to assist learners in linking
new knowledge to existing knowledge and develop instructional techniques that
would facilitate cognitive growth and change during teaching and learning.
Studies have shown that geometry occupies a significant role in the teaching of
mathematics in global perspectives. For example, it provides a rich source of
visualization for understanding arithmetic, algebraic, and statistical concepts, as
indicated by Binti et al. (2003). Furthermore, in the scientific world, the importance
of geometry appears naturally in many sectors of the planet, such as the structure
of the solar system, in geological information, rocks and crystals, flowers, and
even in animals for specific purposes. The importance of geometry in the lives of
people cannot be overemphasized. However, many teachers find it challenging to
teach the concept in South Africa due to the lack of subject content knowledge and
lack of cognitive skills. Adolphus (2011) stated that one of the problematic ideas
that teachers struggled to teach in mathematics is Euclidian geometry, because of
the technicalities related to the language, visualization of objects for better
identification of properties, and inadequate conceptual understanding.
Freudenthal (1991) asserted that teaching and learning of mathematics must be
connected to reality, meaning that in the teaching of geometry, teachers should
try to make use of available materials at their disposal to convey the meaning of
geometrical concepts which pose a threat to the learning of geometry in schools.
Researchers such as Freudenthal (1991), Sanni (2007), Van Den Heuvel-Panhuizen
(2003), and Wigley (1994) have disputed the notion that a teaching approach
which connects to reality helps learners develop and apply mathematics to
problem and help makes sense.
It was evident from the assertion of Sanni (2007) that instruction of geometry
concepts was bedevilled regimented mathematics classroom practices such as
checking homework, followed by teacher lecture and demonstration, followed in

300
turn by learner practice in a sequence of classroom instructional activities (Sanni,

2007). Such an approach does not take into account Van Hiele’s (1986) five levels
of geometry thinking, which serve as a rational basis for the teaching of geometry
in schools. These levels are:
Level 1: Recognition /Visual Level: Learners recognize figures by their
appearance.
Level 2: Descriptive/Analytic: Learners recognize/analyse figures by their
properties.
Level 3: Abstract/Relational/Ordering: Learners distinguish between necessary
and sufficient conditions for a concept. They can also form abstract definitions
and classify figures by elaborating on their interrelationships.
Level 4: Formal Deduction: Learners establish theorems within an axiomatic
system. They recognize the difference between undefined terms, definitions,
axioms, and theorems. They can construct original proofs.
Level 5: Mathematical Rigor: Learners understand the relationship between
various systems of geometry. They can describe the effect of adding or deleting
an axiom on a given geometric system and can compare, analyse, and create
proofs under different conditions.
Van Hiele (1986) believes that these five sequential geometric levels are needed
for identification and sorting, developing critical thinking and problem-solving
skills, formulating conjectures, intuition, postulation, deductive reasoning, logical
arguments, and proof formulation. The question that comes to mind is whether
teachers are applying the above level as indicated by Van Hiele (1986), and also
to establish what teachers are currently doing in the teaching and learning of
geometry in schools. Mcyntire’s (2018) views are confirmed when he states that
Van Hiele’s theory of geometrical thinking is the theory behind the teaching of
Euclidean geometry in schools. Many educators still prefer using the traditional
teaching approach in the education of geometry in schools.
South Africa has seen various curriculum reforms as one Minister of Education
replaced another, and that has affected the teaching of Euclidean geometry in
schools. The curriculum reform phenomenon has also affected other countries.
China is one of the countries that underwent curriculum reforms and has seen a
great deal of change in its mathematics curriculum. According to Xei (2005), China
has had to adapt to curriculum reforms for the benefit of the development of
society, and this was done by changing mathematics content taught to elementary
(primary) and middle (junior) learners in the People’s Republic of China. Xei
(2005) explained that the reforms brought both successful experiences and
frustrating lessons in that, through adopting a foreign mathematics curriculum
from the Soviet Union (the 1950s), their cultural identity and national conditions
were compromised. On the other hand, they needed to examine the superiority
and limitation of Chinese mathematics education by looking at the tendencies in
international mathematics education so that China’s curriculum could be
recognized internationally. The current study aimed to establish the applicability
of the China and Soviet Union experience to that of the South African experience.

301
The Chinese Ministry of Education listed six focus areas in mathematics teaching.
One such focus area meant that Euclidean geometry was compulsory for all
learners. The reason for this was that it had been observed that in the traditional
Chinese culture, logical thinking ability had been rare, doing mathematics-related
training, especially about Euclidean geometry, a great need. This indicated the
importance of deductive reasoning, which is extractable from the teaching of
Euclidean geometry for a nation doing it as a core topic in mathematics. However,
many teachers faced challenges in teaching Euclidean geometry as it was newly
introduced into the school curriculum.
In Zimbabwe, Euclidean geometry has suffered the same fate, as was the case in
China. Wessels (2004, p.70) confirmed the fact that “geometry is one of the poor
relatives in the field of mathematics and it is only in the past five years that its
status has improved in Zimbabwe. This common occurrence has been observed
in Zimbabwe’s General Certificate of Education (GCE)”. Students with this
qualification, however, had a deficient level of understanding of Euclidean
geometry as it was not comprehensively taught, which led to challenges being
experienced in teaching Euclidean geometry in Zimbabwean secondary schools.
A similar issue has been observed in South African schools where Euclidean
geometry has seen less emphasis, if any, at some tertiary institutions resulting in
education graduates having not been educated in the topic of Euclidean geometry.
Several Mathematics teaching studies focus on teachers’ content knowledge,

curriculum implementation, instructional strategies, teachers’ competency in
teaching the new mathematics curriculum, and teacher education programs
(Maree & Van der Walt, 2007; Masinga et al., 2007; Strawderman, 2010; Webb,
2010). Some teachers have already qualified as teachers yet did not undergo
training on some of the newly-introduced topics such as Euclidean geometry.
In 2012, the South African curriculum for Grades R–12 saw the introduction of a
new curriculum, which was an amendment of the existing National Curriculum
Statement (NCS) and which did not offer Euclidean geometry as the core topic for
Curriculum and Assessment Policy Statement (CAPS) for Grades R-12. According
to CAPS (DBE, 2011, p.11), “Mathematics is a language that makes use of symbols
and notations for describing numerical, geometric, and graphical relationships. It
is defined as a human activity that involves observing, representing and
investigating patterns and qualitative relationships in physical and social
phenomena and between mathematical objects themselves.”
Teachers who had not studied this topic in their secondary schooling or their
teacher education programs found themselves teaching the subject, but with little
knowledge and understanding. Their preparations were affected because
preparing useful lessons for teaching mathematics depends on how one
understands the mathematics content itself. Brodie et al. (2009) found that there is
always a significant decline in the cognitive levels of mathematics content
whenever a teacher is not comfortable with the topic, and so knowledge transfer
is compromised. The same situation occurred with the teaching of Euclidean
geometry.

302
Turik (2000) contended that inadequate teacher knowledge in teaching such a

topic becomes a challenge. The lack of subject content knowledge of Euclidean
geometry and resources leads to compromising understanding of geometric
concepts, and thus poor lesson preparation and teaching. This lack of subject
content knowledge of Euclidean geometry affects the teachers of the Motheo
Education District, hence the current study.
3. Theoretical Framework
The understanding of geometry and its concepts requires a great deal of basic
knowledge on the part of the person learning it. Saxe (1991) quotes Piaget’s work,
stating that learning involves the elements of discovery and scaffolding to elevate
the learner’s activities to a level. In turn, this indicates that anybody who is trying
to learn an aspect must use concrete objects when introducing and investigating
phenomena to help the person grasp the task at hand quickly and easily
pertaining to the teachers who have not studied Euclidean geometry as their level
of development is unsatisfactory. The participant teachers, therefore, engaged
themselves in collaborative discussions about geometry problems for a better
understanding of some concepts. Thus some of their challenges in teaching the
concept (geometry) were exposed.
Effectiveness in geometry teaching has been the subject of considerable

theorizing. Still, teachers should draw on both subject content knowledge and
pedagogical content knowledge, which is embracing theory and practice gained
from ongoing teaching activities for a better understanding of a concept (Solis,
2009). Much research has been conducted on the validity of Van Hiele’s (1986)
theory and has focused on teachers' emphasis on geometrical reasoning.
Alternatively, Pusey (2003) considered that each of Piaget’s five stages of
development – the sensorimotor, iconic, concrete symbolic, formal, and post-
formal stages – has an important role to play in the learning of geometry. The
current study underpins the constructivist approach based on Piaget’s cognitive
theory as well as Van Hiele’s theory (Piaget, 1962; Van Hiele, 1986). Even though
Van Hiele’s theory is intended for learners’ education of mathematics, it was used
in this study to identify challenges faced by teachers in the teaching of Euclidean
geometry as many experience challenges in solving problems in Euclidean
geometry.
These challenges were identified with pre-test activities completed before the start
of this research. In defense of the teachers, many claimed that they had been
forced to teach mathematics due to redeployment and were neither qualified nor
equipped to teach this aspect of mathematics. This study emphasized building or
constructing knowledge from small existing pieces of the knowledge base. For
concrete geometric understanding, one must have spatial thinking and geometric
abilities which, just like Piaget’s developmental stages in learning, also involves
four phases of spatial developmental thinking (Stephenson et al., 1996)
The primary mission of the Department of Basic Education is to produce

competent teachers. They can offer the necessary services in the community, such

303
as in the Motheo district, through quality teaching and learning (Teacher’s Forum,
2009), calling for the Department to recruit teachers who can teach critical subjects
like mathematics and science. However, the teaching and learning of mathematics
and science in the district has been compromised since some teachers, especially
mathematics teachers, are not equipped to teach specific topics such as geometry
and probability, despite government reform efforts in the development of
learning activities in mathematics and science. Such reforms include the
introduction of geometry and other topics as compulsory topics in the CAPS
curriculum.
4. Research Methodology
The research methodology section highlights the following aspects: general
background, sample, instruments and procedures, data analysis, and ethical
considerations.
4.1 General background

A research approach comprised approaches that are used in educational research
to collect data that may be used for inferences and interpretations (Cohen et al.,
2009; Ayoola, 2017). The primary purpose of the research methodology is to help
understand the process of research. Creswell (2009) stated that choosing the
correct method in addressing a challenge or a problem in any study plays a vital
role in achieving the purpose of the study. In this study, a qualitative research
approach with a case study design was used to identify and detail challenges
facing teachers in the teaching of Euclidean geometry through observation and
focus-group interviews to find tangible solutions to this problem in the district of
Motheo.
4.2 Sample
Ten mathematics teachers were purposively selected from ten schools through a
cluster-sampling technique. The participant teachers were grouped into three
groups, with at least three participants in each group. Convenience sampling
technique was used for the selection of the teachers into their respective groups
to ensure that each group comprises a member from each cluster to elicit different
views based on the problems they experienced in the teaching of geometry in their
various clusters. The other motive was to assist the participant teachers in sharing
their opinions experienced in their individual clusters to generate at least one
thought that represented that particular group during focus-group interviews.
4.3 Instrument and procedures

Observation and focus-group interviews were used for data collection. Class
observations were conducted using the participant teachers in their respective
schools and video recorded. Teachers were observed on the methods they used to
present their lessons and the problems they faced in delivering the content to the
learners. How learners were questioned and how they discussed and debated
with their peers was also noted. The way and manner in which learners responded
to their teachers’ questions and how they solved problems given to them by their
teachers were also noted to see how the teachers’ methods of teaching contributed
to learners’ understanding of geometry concepts. The teachers were later invited
to participate in a collaborative discussion using the recorded videos to examine

304
the phenomenon through discussions and interviews. During focus group-

interviews, when a question was directed to a particular group, each member of
the group was given equal opportunity to respond.
In contrast, other group members attentively listened to their responses. If that

group had no valid or pertinent response, the question was transferred to another
group for their opinion until all questions were exhausted. The data gathered
offered evidence of teachers’ practice in the teaching of Euclidean geometry,
bearing in mind Van Hiele’s levels of geometric thought (cited in Van de Walle et
al., 2013). This paved the way for participants to deliberate on the issues of concern
to identify a reliable solution to the problem identified or discussed.
4.4 Data analysis

In this study, relational content analysis was used. After repeatedly read through
to identify the main themes, the data were examined in-depth to provide detailed
descriptions of the participant teachers. The data were coded, categorized, and
grouped them into themes for interpretation and reporting. Nili et al. (2017) stated
that focus group data can be analyzed quantitatively (e.g., through participants
vote counting) after or in parallel with the primary/qualitative analysis. In this
study, the researcher prepared the data gathered through observation and focus-
group interviews; the transcripts and field notes were also read through to
understand the whole context, bearing in mind that the study aimed to
understand individual teachers’ challenges faced in the teaching of Euclidean
geometry in schools, as demonstrated through data collection. The aim was to
increase the researchers’ understanding of the phenomenon experienced by the
participants. In the next stage, the researcher constituted the content areas
through extracting and bringing together the text about each topic into one text,
which helped the researcher to identify the themes for discussion. Each condensed
meaning unit was extracted and labeled with a code and was sorted into
subcategories and further extracted into categories. The categories were labeled
with content-characteristic names, which were formulated into themes (Nili et al.,
2017).
4.5 Ethical Considerations

Ethical approval was received from the various authorities before the study was
conducted. Ethical clearance was issued from the University of the Free State;
approval letters were obtained from the Free State Department of Education as
well as from the principals of the participating schools where observations were
conducted. Consent forms were signed by participant teachers confirming their
willingness to participate in the study. Further consent forms were received from
the parents of the under-aged learners, indicating their consent for their children
to participate in the study.
5. Research Results
In reporting on this research, the direct responses of the ten (10) participants who
were observed and interviewed are captured to illustrate the findings. The results
are presented according to the following themes: the chalk-and-talk method, the
attitude of the teachers and language used, lack of content knowledge, and
availability of materials.

305
5.1 Chalk-and-talk method of teaching geometry

The participating teachers indicated they faced many challenges in the teaching
and learning of Euclidean geometry in schools. Regarding teaching and learning
strategies, it was evident in the research that the teachers used different teaching
and learning strategies in teaching Euclidean geometry. For instance, some of
them talk and write during teaching and learning without any explanation for the
conceptual understanding of learners. Thus, they only teach without soliciting the
opinion/attention of the learners. They completely dominate the teaching and
learning processes, which is termed as the talk-and-chalk method of teaching.
Responses from the teachers’ focus group interview indicated that the way they
were taught geometry while at schools did not develop a good understanding of
mathematics. According to the teachers, their teachers read from the textbooks
hoping the whole class would understand what was being taught. This was
observed when many of the participating teachers make frequent use of
mathematics textbooks. It seems that teachers were dependent on texts to write
formulae and other theorems on Blackboard before explaining certain concepts to
the learners. In many cases, teachers would request learners to open their
textbooks to check answers to questions, not feeling confident enough in their
answering—this way of teaching modelled how teachers were taught at school
where the textbook is vital. Restricting teaching to just the textbook without
further application and discussion often results in learners’ understanding of not
being fully developed. Lack of knowledge of mathematical concepts and
inadequate training in the teaching of mathematics at school and higher education
level has led to teachers not being well-qualified or experienced in teaching
mathematics, particularly challenging topics such as Euclidean geometry. In other
words, relevant explanations were inadequate for the teaching of geometry in
schools. The following excerpts support the assertion:
Teacher A: As for me, I was forced to teach mathematics since I was
redeployed to my current school. I only have a Grade 12 mathematics
background, which in most cases, I find it difficult to explain some
concepts properly to the learner. I wish the department would send only
teachers with mathematics background from the university or diploma to
handle this critical subject.
This was observed during classroom observation conducted by the researcher and
captured in the video recording. The researcher observed that.
Teacher D: In my college, where I graduated as a teacher, my
mathematics lecturer/teacher always uses telling and the textbook
methods to teach us whereby individual attention was not given because
he always complained about time and finishing of the syllabus. I have stick
to that method as the only way I can teach my learners this geometry since
I was not introduced to different kinds of teaching strategies during my
training.
The teachers indicated that they followed a traditional method of teaching,

drawing of how mathematics was taught at school and college. Also, teachers
were not introduced to other teaching methods during training, and thus their
pedagogical content knowledge is not well-developed to implement during the

306
teaching of Euclidean geometry. The teachers felt that the Department was not
supportive, that the teachers are often redeployed and placed in schools where
they have to teach specific subjects even if they have had little or no training at
higher education levels, and felt that they needed to use traditional chalk-and-talk
methods of teaching being guided by the textbook. It was evident during
observation where teachers were throughout dominant the lessons without given
opportunities to learners to express their views or to indicate if they needed
clarification on problem areas. In some cases, learners were not attentive and lost
focus during teaching and learning. Some teachers had no resources apart from
textbooks, and when they were asked why about other resources and materials,
they indicated that many resources had been stolen; hence the department has
refused to provide them with new resources. The participants also claimed that
their colleagues, who were teaching in other schools also used the chalk-and-talk
methods and did not allow learners to make use of any practical activities that
could help develop their understanding of a concept. The following comment
from one of the teachers in one group, Teacher K supports this claim:
“We wish we could use different methods of teaching geometry that would
make it easy for learners to understand the concepts.”
Two teachers indicated that in their schools, they used one-on-one methods of
teaching geometry using many practical activities. They had acquired the skills
from the technical colleges where activities/projects were encouraged. They
further indicated that they paid attention to the individual styles of learners in the
class, which helped focus attention on their learners when teaching Euclidean
geometry since it involves various practical activities. The excerpts below support
this:
“Sometimes if learners do not understand the concept (geometry),
something I try my best to help them individually by making reference to
the practical activities that I displayed on the blackboard.”
Learning involves the elements of discovery and scaffolding to elevate the

learner’s activities to a particular level. The scaffolding emphasizes the five levels
of geometry thinking and serves as a rational basis for geometry taught in schools.
Also, teachers should align their teaching of geometry to the ideas of Piaget (1962)
and Van Hiele (1986), who remind us that geometry is needed to enhance and
develop learners’ skills of visualization needed for identifications and sorting,
critical thinking, and problem-solving skills, formulating conjectures, intuition,
postulation, deductive reasoning, logical arguments, and proof formulation.
5.2 The attitude of the teachers to teaching geometry and mathematical

language
The research study showed that some of the teachers displayed negative attitudes
towards the teaching of Euclidean geometry since they did not have confidence
in teaching mathematics, having been forced to do so by the Department after
redeployment. The respondents indicated that those teachers who understood
the practical aspects have more confidence in teaching Euclidean geometry than
those without practical experience. Most respondents also agreed that teachers’
methods of teaching and personality have a considerable impact on the
enthusiasm of learners for learning mathematics. Also, the use of mathematical

307
language to express a concept should be learner-friendly. Both these aspects are

essential since they help learners to understand what is being taught. Teachers’
use of mathematical language in the teaching of Euclidean geometry should not
be a threat to learners and not contribute a negative attitude towards the subject.
This sheds more light on the claim by Adolphus (2011, p.144) when he states that
Euclidian geometry is the most problematic area to teach and learn due to the
language used, visualization of objects for better identification of properties, and
lack of conceptual understanding. Freudenthal (1991) believes that teaching and
learning of mathematics geometry need to be precise and should be connected to
reality. This means that in the teaching of geometry, teachers should make use of
available materials at their disposal to convey the meaning of geometrical
concepts which poses a threat to the learning of geometry in schools. Researchers
such as Freudenthal (1991), Sanni (2007), Van Den Heuvel-Panhuizen (2003), and
Wigley (1994) contend that the teaching approach that connects to reality helps
learners develop and apply mathematics to a problem that makes sense to them.
In this regard, one teacher had this to say:
“The way we teach mathematics influences learners’ enjoyment in
mathematics. Therefore, when you teach mathematics without any
practical activities, it does not help learners to understand the concepts,
which contribute to poor performance in the subject. We must do our best
to make the teaching of mathematics enjoyable by making it practical in
your teaching.”
5.3 Lack of content knowledge of teaching and learning of geometry

It also emerged from this study that knowledge of instructional practices for
participant teachers was not up to expected standards. As previously mentioned,
many teachers were forced to teach mathematics even though mathematics was
not their area of specialization, which meant that they lacked subject content
knowledge as well as pedagogical content knowledge that would assist them in
using a variety of instructional practices. The limited understanding and
application of mathematical practices in teaching geometry were also revealed
when teachers were observed; for instance, during lesson observation, it became
apparent that most participant teachers lacked practical skills, such as planning,
demonstration, and organizational skills as well as being able to develop their
learners understanding through involvement. Pedagogical content knowledge is
the integration of subject expertise and skilled teaching and is considered
instrumental in ensuring that teachers deliver effective teaching of geometry. The
observation also revealed that some teachers also apportion the blame on learners
since learners found it difficult to remember some theorems and as well as basic
concepts, which made it difficult for them to teach the concept effectively in
schools.
6. Discussion
In developing learners’ conceptual understanding of Euclidean geometry, Van
Hiele’s (1986) five levels of geometry thinking should be taken into consideration
and followed. It emerged from this study that the knowledge of instructional
practices of participant teachers for teaching and learning of Euclidean geometry
did not embrace the vital five levels and, therefore, both teaching and learning
were poor. Only a few teachers exhibited acceptable standards of teaching. In

308
many cases, deployment has meant that teachers were forced to teach subjects like
mathematics despite having had little or no training. This confirms a statement by
Turik (2000), which alluded to a lack of basics and inadequate teacher background
in mathematics topics, thus engendering negative attitudes towards the latter.
Teaching such a topic was, therefore, a challenge. It was observed that both
teachers’ content knowledge and the pedagogical content knowledge of teaching
mathematics were questionable since many of them could only teach some
geometry concepts with the help of textbooks and were unable to use a variety of
approaches or strategies to apply the mathematical knowledge. Given this, one
teacher had this to say:
“Indeed, a lot of mathematical modelling activities which they think will
play an essential role in teaching mathematics could have been given by
the departmental officials through workshops and seminars. This could
have helped our skills in teaching geometry in schools. We teach those
topics with no prior knowledge simply because we are being forced to teach
due to redeployment. “
This excerpt reveals that lack of support from the Department of Basic Education
created uncertainty among some teachers as they were put in positions where
they were ill-equipped to teach the specific subject, resulting in negative attitudes,
mainly towards the teaching of mathematics. This is in line with the findings by
Maree & Van der Walt (2007); Masinga et al. (2007), Strawderman (2010), and
Webb (2010), who stated that there are several Mathematics teaching studies,
which focus on teachers’ content knowledge, curriculum implementation,
instructional strategies, teachers’ competency in teaching the new mathematics
curriculum and teacher education programs. However, some teachers are
qualified as teachers but have not undergone training on newly-introduced topics
like Euclidean geometry and therefore find it difficult to teach due to lack of
support and training offered by the Department of Basic Education as well as
institutions of higher learning for development.
Practical instruction in geometry teaching, and learning and any topic in

mathematics for that matter, requires a teacher to develop sound mathematical
knowledge as well as instructional skills and strategies making use of useful
resources and activities that guide the teaching activities and further assist in the
effective delivery of the lesson (Luneta, 2014). Teaching without good
instructional skills and modelling activities and sound knowledge of the concept
usually puts the teacher in a challenging situation, and learners tend to lose
interest in the topic. As a result, quality teaching is compromised. It is, therefore,
crucial that mathematics teachers are fully equipped with sound knowledge and
are aware of the teaching environment and their learners. This will ensure that
appropriate modelling activities are used in problem-solving.
Teachers need to adapt their instructional approach or strategy to teaching

various concepts by engaging in a variety of practical ways that assist in
developing learners’ understanding of the concepts under discussion. Teachers
should also try to draw learners’ attention to alternative solutions to mathematical
problems through reflection-on-action. It is believed that teachers are more likely
to reflect on their teaching to improve teaching practice when they understand

309
the curriculum visions (Khoza, 2016). Practical instruction in any topic requires
teachers to develop sound instructional strategies and knowledge of useful
resources and activities (Luneta, 2014) as this allows teachers to gauge the depth
of their learners’ understanding of a concept. This contradicts the findings by
Sanni (2007), which stated that instruction of geometry concepts was bedevilled
regimented mathematics classroom practices such as checking homework,
followed by teacher lecture and demonstration, followed in turn by learner
practice in a sequence of classroom instructional activities. These aspects do not
take into account Van Hiele’s (1986) five levels of geometry thinking, which serve
as a rational basis for the teaching of geometry in schools.
Teachers who are capable of making detailed and explanatory lessons plans can
address misconceptions of mathematical concepts that learners experience in the
classroom. They can explain the difficulties that learners encounter during
teaching and learning by pre-empting them. Knowledge of instructional practices
involves curriculum, tasks, and tools for teaching and incorporates both subject
and pedagogical content knowledge (Ball et al., 2008; Shuman, 1986).
The analysis of the study further revealed that lack of understanding of

mathematical concepts and inadequate training in the teaching of mathematics at
school and higher education level has led to teachers not being well-qualified or
experienced in teaching mathematics, particularly challenging topics such as
Euclidean geometry. This is in line with the assertion by Brown (1999) and Khoo
and Clements (2001) that in many instances, geometry teaching is based on
immersive and adaptive instruction and learning where teachers shift from a
conceptual approach (the sensible application of procedures) to a procedural
approach (calculation accuracy). However, many learners fail to develop an
adequate understanding of geometrical concepts or demonstrate reasoning and
problem-solving skills. This contradicts the primary mission of the Department of
Education, where much emphasis is to produce competent teachers. They can
offer the necessary services in the community, such as in the Motheo district of
the Free State, through quality teaching and learning (Teacher’s Forum, 2009).
7. Conclusions
The findings of the research revealed that teachers’ performance in class was due
to a lack of knowledge of mathematical knowledge and instructional practices.
Teachers face challenges when teaching Euclidean geometry in schools. Thus their
lack of knowledge and application has led to the chalk-and-talk method
predominantly being used guided by textbooks. Teachers felt there was a lack of
support from the Department, little training, and the issue of redeployment,
which means that teachers lacked content as well as pedagogical knowledge due
to being required to teach a subject with which they were not familiar. As a result,
teachers’ attitudes towards mathematics and the teaching of Euclidean geometry,
specifically, were affected. The findings of the study have helped the researcher
to embark on developing training for mathematics teachers in collaboration with
the Department of Basic Education using mathematical modelling activities as
well as relevant skills and strategies in the teaching of geometry in schools for a
better understanding of learners.

310
8. Recommendations
From the findings of this study, it is essential for further research of the same kind
to be conducted on a large scale (that is a quantitative manner) where different
opinions of teachers from other areas can be hard to establish whether or not the
same problems occur when teaching Euclidean geometry in schools. As South
Africa needs suitably-qualified mathematics teachers who can teach the subject
effectively to overcome the problem of poor performance and the high failure rate
in mathematics, further research is required. Findings from such a study could
assist in ensuring quality mathematics education, which will develop the
necessary skills needed by doctors, scientists, and many other scientifically-
oriented professionals.
9. References
Adolphus, T. (2011). Problems of Teaching and Learning of Geometry in Secondary Schools.
Nigeria, Department of Education: River State University of Science and
Technology. Retrieved from http://hdl.handle.net/1893/26189
Alex, J., & Mammen, K. J. (2018). Students’ understanding of geometry terminology
through the lens of Van Hiele theory. Pythagoras, 39(1), pp.1-8.
https://doi.org/10.4102/pythagoras.v39i1.376
Ball, D., Thames, M., & Phelps, G. (2008). Content Knowledge for Teaching What Makes
It Special? Journal of Teacher Education. 59.
https://doi.10.1177/0022487108324554
Bassarear, T. (2012). Mathematics for elementary school teachers (5th Ed.). Cengage Learning.
Binti, N., Tay, I., & Lian, B. (2004). Teaching and Learning of Geometry: Problems and
Prospect. Masalah Pendidikan Jilid, 27, 165-178.
Brannan, D. A., Esplen, M. F., & Gray, J. J. (2002). Geometry. Cambridge University Press.
Brodie, K., Jina, Z., & Modau, S. (2009). Challenges in implementing the new mathematics
curriculum in Grade 10: A case study. African Journal of Research in Mathematics,
Science and Technology Education, 13(1), 19-32.
https://doi.org/10.1080/10288457.2009.10740648
Brown, A. R. (1999). Geometry's giant leap. The Mathematics Teacher, 92(9), 816-819.
https://doi.org/10.5951/MT.92.9.0816
Chihambakwe, T. (2017). Impact of national senior certificate examination diagnostic report on
learner performance in mathematics (Unpublished Masters dissertation). University
of Zululand, Pietermaritzburg
Cohen, L., Manion, L., & Morrison, K. (2009). Research methods in education. Routledge.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches
(3rd Ed.). SAGE Publications.
Department of Basic Education (DBE). (2011). Curriculum and assessment policy statement
(CAPS), Grade 10-12, Mathematics. Government Printing Works.
Ding, L., & Jones, K. (2006). Teaching geometry in lower secondary school in Shanghai, China.
Proceedings of the British Society for Research into Learning Mathematics.
Freudenthal, H. (1991). Revisiting mathematics education: China lectures. Kluwer Academic
Publishers.
Güven, B., & Kosa, T. (2008). The effect of dynamic geometry software on student
mathematics teachers' spatial visualization skills. Turkish Online Journal of
Educational Technology-TOJET, 7(4), 100-107.
http://doi:10.1016/j.sbspro.2010.12.169
Jones, K. (2002). Issues in the teaching and learning of geometry. Routledge.

311
Jones, K., & Tzekaki, M. (2016). Research on the teaching and learning of geometry. In A.
Gutiérrez, G. Leder & P. Boero (Eds.), The Second Handbook of Research on the
Psychology of MathematicsEducation: The Journey Continues (pp. 109-149). Sense.
http://dx.doi.org/10.1007/978-94-6300-561-6_4
Khoo, S. C., & Clements, M. A. (2001). A-level Students' Understanding of Lower
Secondary School geometry. In K. Y. Wong, H. Tairab, & M. Clements, Proceedings
of the Sixth Annual Conference of the Department of Science and Mathematics Education:
Energising Science, Mathematics and Technical Education for all (pp. 213-222).
University of Brunei.
Khoza, S. B. (2016). Is teaching without understanding curriculum visions and goals a high
risk? South African Journal of Higher Education, 30(5), 104-119.
http://doi:10.20853/30-5-595
Kotzé, G. (2007). Investigating shape and space in mathematics: a case study. South African
Journal of Education, 27(1), 19-35.
Luneta, K. (2014). Foundation phase teachers'(limited) knowledge of geometry. South
African Journal of Childhood Education, 4(3), 71-86.
https://doi.org/10.4102/sajce.v4i3.228
Luthuli, D. (1996). Questions, reflection and problem posing as sources of inquiry in
Euclidean geometry. Pythagoras, 40, 17-27.
Mamali, N.R. (2015). Enhancement of learner's performance in geometry at secondary schools in
Vhembe District of the Limpopo Province (Doctoral dissertation).
Maree, K., & van der Walt, M. (2007). Do mathematics learning facilitators implement
metacognitive strategies? South African Journal of Education, 27(2), 223-241.
Masinga, A., Mhlongo, M., & Luneta, K. (2007). Implementing Curriculum change: Addressing
poverty, gender equity and optimizing learning. University of Witwatersrand.
Mcintyre, R. N. I. (2018). Analysing geometry in the Classroom Mathematics and Mind Action
Series mathematics textbooks using the van Hiele levels. (Unpublished Masters
Dissertation). University of Witwatersrand.
Morris, R. (1985). Studies in mathematics education: The education of secondary school
teachers of mathematics. UNESCO.
Nili, A., Tate, M., & Johnstone, D. (2017). A Framework and Approach for Analysis of
Focus Group Data in Information Systems Research. Communications of the
Association for Information Systems, 40(1), 1-21.
https://doi.org/10.17705/1cais.04001
Piaget, J. (1962). The stages of the intellectual development of the child. Bulletin of the
Menninger Clinic, 26(3), 120.
Pusey, E. L. (2003). The Van Hiele model of reasoning in geometry: a literature review.
Retrieved from
https://repository.lib.ncsu.edu/bitstream/handle/1840.16/2275/etd.
pdf?sequence=1&isAllowed=y
Sanni, R. (2007). Teaching geometry in schools: an investigative rather than instructive
process. Pythagoras, 2007(65), 39-44.
Saxe, G. B. (1991). Culture and cognitive development, Studies in mathematical
understanding. Lawrence Erlbaum Associates.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching.
Educational researcher, 15(2), 4-14.
Solis, A. (2009). What matters most in the professional learning of content teachers in
classrooms with diverse student populations. IDRA Newsletter by the International
Development Research Association.

312
Strawderman, V. W. (2010). Math anxiety model. Math Goodies. Retrieved from

https://www.mathgoodies.com/articles/math_anxiety_model
Tirosh, D., & Graeber, A. O. (2003). Challenging and changing mathematics teaching
classroom practices. In A. Bishop, M. K. Clements, C. Keitel-Kreidt, J. Kilpatrick
& F. K. S. Leung (Eds.), Second international handbook of mathematics education (pp.
643-687). Springer.
Ubah, I., & Bansilal, S. (2019). The use of semiotic representations in reasoning about
similar triangles in Euclidean geometry. Pythagoras, 40(1), pp. 1-10.
Van de Walle, J. A., Karp, K. S., & Bay-Williams, J. M. (2013). Elementary and middle school
mathematics: Teaching developmentally (8th Ed). Pearson.
Van Den Heuvel-Panhuizen, M. (2003). The didactical use of models in realistic
mathematics education: An example from a longitudinal trajectory on percentage.
Educational studies in Mathematics, 54(1), 9-35.
https://doi.org/10.1023/B:EDUC.0000005212.03219.dc
Van Hiele, P. M. (1986). Structure and insight. Academic Press.
Webb, D. C. (2012). Teacher change in classroom assessment: The role of teacher content
knowledge in the design and use of productive classroom assessment. In S.J. Cho
(Ed.), Proceedings of the 12th International Congress on Mathematical Education: Topic
Study Group (pp.6773-6782). Springer.
Wessels D. C. J. (2004). Subject didactics mathematics/mathematics education, Tutorial Letter
SDMATO-4/501/2004. Unisa.
Wigley, A. (1994). Models for mathematics teaching. In A. Bloomfield & T. Harries (Eds.),
Teaching and learning mathematics (pp. 22-25). Association of Teachers of
Mathematics.
Xei, M. (2005). Postmodernist, mathematical view and mathematics education. Normal
University Press.

313

Vol. 19, No. 8, pp. 313-329, August 2020
Health Professional Students’ Perceptions and

Experiences of Remote Learning During the
COVID-19 Pandemic
Fatmah Almoayad
Department of Health Sciences, College of Health and Rehabilitation Sciences
Princess Nourah Bint Abdulrahman University, Riyadh, Saudi Arabia
https://orcid.org/0000-0002-8424-5229
Afrah Almuwais
Department of Rehabilitation Sciences, College of Health and Rehabilitation Sciences
https://orcid.org/0000-0002-2774-868X
Samiah F. Alqabbani
Department of Rehabilitation Sciences, College of Health and Rehabilitation Sciences
https://orcid.org/0000-0003-4495-5047
Nada Benajiba*
Department of Health Sciences, College of Health and Rehabilitation Sciences
Department of Basic Health Sciences, Deanship of Preparatory Year
Princess Nourah Bint Abdulrahman University,Riyadh, Saudi Arabia
https://orcid.org/0000-0002-5533-7626
Abstract. This research aims to investigate the learning experiences and

anxiety levels of students attending the College of Health and
Rehabilitation Sciences (CHRS) at Princess Nourah bint Abdulrahman
University (PNU), Saudi Arabia, during the sudden shift to virtual
learning in response to COVID-19. A cross-sectional descriptive study
was conducted among CHRS students (N = 254). Data was collected using
an online questionnaire. Pearson’s correlation coefficient was used to
investigate the relationships between student perceptions of
responsibility and independence in their learning during emergency
remote teaching (ERT), the impact of ERT on learning goal achievement,
and student anxiety levels. Results showed that 66.5% of participants
scored moderate in their perceptions of independence and responsibility
in their learning experiences. In addition, 54.3% of the students achieved
their learning goals. Moreover, 20.9% of students reported that they had
*
Corresponding author: Nada Benajiba; Email: benajibanada@gmail.com

314
not experienced anxiety, while 24.4% had experienced severe anxiety.

There was a significant negative correlation between student anxiety
levels during ERT and their perceptions of responsibility and
independence in learning (r = -0.30, p < 0.001). In conclusion, student
anxiety levels during the sudden shift to ERT moderately influenced
student perceptions of independence and responsibility in their learning
as well as learning goal achievement. Further studies are needed to
explore the types of assistance to provide to students to improve the
learning process during the current COVID-19 pandemic and similar
circumstances where ERT is imposed.
Keywords: Emergency remote teaching; Remote learning; Health

professional students; COVID-19 pandemic
1. Introduction
E-learning refers to education that is delivered using technology either fully or
partially independent from fixed time and space (Hawkins & Oblinger, 2005). E-
learning was established in the late 1990s and has since been gradually integrated
into higher education (Al-Asmari & Khan, 2014; Hubackova, 2015). In 2005, higher
education was expected to evolve progressively from in-person to online, leading
to a revolutionary transformation (Hiltz & Turoff, 2005). E-learning is becoming
an essential part of health professional education: many educational institutions
have incorporated it into their learning strategies (Ellaway & Masters, 2008). In
Saudi Arabia, higher institutions started adopting e-learning at the beginning of
the 21st century (Al-Asmari & Khan, 2014). Since then, the Saudi Ministry of
Education has established the National Centre for e-Learning and nearly every
university in Saudi Arabia has created an e-learning unit or department (Al-
Shehri, 2010). In addition, allied health colleges in Saudi universities also
incorporated e-learning into their curriculums to create blended learning and not
as complete e-learning (Sajid et al., 2016; Zakaria, Jamal, Bisht & Koppel, 2013).
However, in March 2020, the Saudi Ministry of Education imposed emergency
remote teaching (ERT) on all educational institutions across the country (MOE,
2020). By definition, ERT is
‘a temporary shift of instructional delivery to an alternate delivery mode
due to crisis circumstances’ (Hodges, Moore, Lockee, Trust & Bond,
2020).
It was used as a preventive measure to limit the spread of COVID-19 and keep
both staff and students safe. ERT was part of the anticipatory preventive and
precautionary measures taken by the Saudi government, which also included
social distancing and complete lockdown (SPA, 2020). The College of Health and
Rehabilitation Sciences (CHRS) at Princess Nourah bint Abdulrahman University
(PNU) has 13 different allied health profession specialisations (PNU, 2020).
Similar to other allied health colleges at PNU, the CHRS was affected in several
ways by the sudden implementation of ERT. In particular, its practical and clinical
training courses were cancelled; these courses provide students with hands-on
experience to master clinical skills through clinical training in different clinical
settings, practical training, and roleplay simulation. As a replacement, students
were provided with videos and case study reports. Albarrak (2011) demonstrated

315
that e-learning training alternatives, such as virtual patients, videos, and

animations, are common and effective in health profession education. However,
students do not perceive them as sufficient in medical education and prefer to use
them as complementary resources.
Students around the world were concerned about the sudden shift to ERT—it
caused psychological distress and anxiety (Gallagher & Schleyer, 2020; Odriozola-
González, Planchuelo-Gómez, Irurtia & de Luis-García, 2020; Zhai & Du, 2020).
The psychological impact was caused by many factors. A study conducted among
university students in China showed that students experienced feelings of
frustration, anxiety, and betrayal as a result of the loneliness and loss of social
connections, shelter, and homelike environment. These feelings were further
amplified among students who stopped receiving mental health support from
their universities which could increase the negative consequences of their mental
health conditions (Zhai & Du, 2020). Likewise, university closure had a moderate
to severe impact on 50% of students’ mental health at a Spanish university
(Odriozola-González et al., 2020). In the United States, feelings of anxiety were
exacerbated among health professional students who served on the front line
during the COVID-19 pandemic as part of their clinical training (Gallagher &
Schleyer, 2020). The sudden implementation of ERT also caused tension to rise
due to the new technologies deployed in teaching. In addition, e-assessments
became an important component of remote teaching, which is usually challenging
for both students and instructors (Ellaway & Masters, 2008).
Aside from the seemingly inevitable psychological effects of ERT, e-learning has
been reported to have several benefits: it helps support student autonomy,
accommodates different learning styles, motivates student–teacher interaction,
and provides flexibility and accessibility (Algahtani, 2011). Autonomy refers to
students taking responsibility for their own learning, which, according to Drew
(2018), is an important characteristic of learning that facilitates learning goal
achievement and learner development (Algahtani, 2011). However, poorly
designed courses could hinder the learning process (Edwards & McKinnell, 2007),
which could be the problem in the sudden shift to e-learning. Therefore, realistic
expectations should be set and training should be provided to instructors. During
its ERT period, PNU invested a great deal of time and effort to ensure that high-
quality teaching processes and assessments with minimal issues were provided
to instructors and students. It focused on providing support and training,
ensuring that not only were learning materials provided through learning
management systems, such as Blackboard, but also to encourage flexible learning
environments and improve communication and interactions between students
and instructors.
Over the years, studies have reported inconclusive findings on the effectiveness
of e-learning (Algahtani, 2011). During these discussions, it is crucial to consider
the perspectives of both students and instructors as well as the learning context
(Khan, 2005). Researchers should also clarify which aspects of effectiveness they
are assessing and whether they are focusing on accomplishments of learning goals
or learning perspectives (Algahtani, 2011). Moreover, George et al. (2014)

316
evaluated the effectiveness of e-learning for students from five different health
professions and found that it was as effective as or superior to in-person learning.
Consequently, they recommended the adoption of e-learning in a variety of
contexts, particularly where there was a shortage of faculty members or resources.
While e-learning had a positive impact on the knowledge, attitudes, and
satisfaction of health profession students (George et al., 2014; Haslerud, Tulipan,
Gray Jr & Biermann, 2017), its effectiveness in practical skills development was
not demonstrated (George et al., 2014; Ngan, Tang, Chan, Chen & Tang, 2018). For
example, when comparing a group performing hands-on lab work to a group
performing simulation-based lab work remotely, (Corter, Esche, Chassapis, Ma &
Nickerson, 2011) found that while the simulation group achieved better results,
the hands-on group had a higher motivation. Their study concluded that properly
designed and coordinated remote learning could improve learning goal
achievement (Corter et al., 2011). Tinio (as cited in Edwards and McKinnell (2007)
emphasised that when e-learning is carefully designed and implemented, it can
empower students in terms of skills development as well as knowledge
acquisition. Also, some positive outcomes were associated with proper e-learning,
including improved student–instructor and student–student interactions,
improved active and independent learning, improved time management, and the
ability to accommodate different learning styles (Edwards & McKinnell, 2007),
showing that students are taking on a higher level of responsibility for their
learning.
Edwards and McKinnell (2007) identified five crucial areas in e-learning activities:
planning and design, teaching or providing learning support, student assessment
and feedback, facilitating effective learning environments and guidance, and
research and professional activities. PNU has addressed the first four activities;
this study was conducted to address the fifth by investigating learning from the
perspective of students attending CHRS. This research investigates students’
perspectives of learning and anxiety levels during the sudden shift to ERT in
response to the COVID-19 pandemic at the CHRS. Our objectives were to (1)
assess student perceptions of responsibility in learning during ERT, (2) assess
student perceptions of the impact of ERT on learning goal achievement, (3)
investigate student anxiety levels during ERT, and (4) evaluate the correlation
between students’ perceptions of learning and anxiety levels. We hypothesised
that the overall learning experience would be positive and anxiety levels would
be high among students. The findings of this study will contribute to the literature
by helping educational institutions better prepare online learning programmes
during emergencies. It can also help develop appropriate plans to overcome
possible challenges to strengthen the experience of e-learning.
2. Methods
2.1. Study design and subjects
This was a cross-sectional descriptive study. The study population included
students from 13 different applied medical sciences and public health
programmes at the CHRS at PNU. The study sample was calculated to be 251 of
720 students (95% CI; p = 0.5) attending the CHRS. A total of 54 students
participated in the study. Data was collected in May 2020 using an electronic

317
questionnaire distributed to students through the CHRS student mailing list.

Students were recruited using the method of non-probability sampling for a
known population. Students from all study levels were included. Preparatory
year students and students who were not registered for the academic semester
during which the study took place were excluded. Since a non-probability
sampling was used, available primary data from participants was used in this
research until the requested sample size (N=251) was reached This type of
sampling was selected because it was the most convenient method to collect data
during the COVID-19 pandemic lockdown. The study was conducted after
obtaining ethical approval from the IRB committee at PNU (IRB Log Number: 20-
0162). All recruited students gave their consent before taking part in the study.
They were provided information about the study, including its objectives,
voluntarily participation, anonymity, and confidentiality of data collected.
2.2. Study instruments

The questionnaire was developed using previous literature Ghenghesh, Croxford,
Nagaty & Abdelmageed, 2018; Uziak, Oladiran, Lorencowicz & Becker, (2018) to
suit the objectives of this study. It consists of the following four sections. Section
1 focuses on collecting sociodemographic and sample characteristics, including
age, department, programme of study, level of study, and GPA. Section 2
measures student perceptions of responsibility in learning during ERT. Students
answered six questions using a Likert scale (Likert, 1932). The questions focused
on student responsibility and independence in their learning, student–instructor
interactions, time management, and organisation. Participants were categorised
into groups based on their scores: low (range: 6–14), moderate (range: 15–22), and
high (range: 23–30). Section 3 assesses student perceptions of the impact of ERT
on learning goal achievement and learning quality. It consisted of five questions
that were answered using a Likert scale (Likert, 1932). The questions aimed to
determine if ERT improved the quality of their education, enabled them to
complete assignments faster and more efficiently, helped them better understand
the subject material, helped them develop knowledge and skills and improved
their motivation when learning. Students were categorised into three groups
based on their scores: low (range: 5–11.6), moderate (range: 11.7–17.2), and high
(range: 17.3–25). In Sections 2 and 3, total scores were used instead of the average
scores for each item, as recommended by Warmbrod (2014). This was done
because total scores have been reported to provide congruent interpretations of
obtained results. Section 4 assesses student anxiety levels during ERT using the
General Anxiety Disorder-7 (GAD-7) tool (Spitzer, Kroenke, Williams & Löwe,
2006). It is a self-administered test that measures the severity of anxiety using
seven questions. Each question gives a score from 0 (not at all) to 3 (nearly every
day); total scores ranged from 0 to 21. Respondents were categorised into groups
based on their scores: no anxiety (range: 0–4), mild anxiety (range: 5–9), moderate
anxiety (range: 10–14), and severe anxiety (15–21; (Spitzer et al., 2006).
2.3 Validity and reliability

All components of the research tool were tested for validity by six academic
teachers involved in e-learning that were fluent in both English and Arabic. The
tool was piloted with 24 CHRS students before it was shared online; no
amendments were needed as students stated that the questions were clear and

318
easily understood. The questionnaire’s reliability was tested using Cronbach’s α;

coefficient values were 0.91, 0.72, and 0.82 for anxiety, responsibility, and goal
achievement, respectively. This indicated a good internal consistency, as reported
by Nunnally (1975). The questionnaire’s feasibility was evaluated during the pilot
study with 10% of the sample group; minor modifications were needed.
2.4 Statistical analysis

Data analysis was performed using SPSS software (version 22), and results were
presented in terms of frequency and percentage. The Kolmogorov–Smirnov and
Shapiro–Wilk tests were conducted; they showed that the data for anxiety and
perceived learning had an approximately normal distribution. The Pearson
correlation coefﬁcient was applied to assess the correlation between different
studied parameters. One-way ANOVA was used to assess the differences among
the groups. Bonforroni post-hoc analysis was applied to determine which groups
were significantly different (p ≤ 0.05).
3. Results
3.1 General characteristics
A total of 254 students attending the CHRS (mean age: 21.1 ± 1.6; mean GPA: 4.4
± 0.3) participated in the study. Students from the Rehabilitation Science
Department and the Health Sciences Department account for 35.8% and 45.3% of
the total sample, respectively, as they are the two largest departments at the CHRS
(Table 1).
Table 1: General characteristics of the study population (N = 254)
Variable N %
Academic department
Rehabilitation Science 91 35.8
Health Sciences 115 45.3
Communication Sciences 22 8.7
Radiology Sciences 26 10.2
Academic program
Doctor of Physiotherapy 59 23.2
Physiotherapy 10 3.9
Occupational Therapy 22 8.7
Epidemiology 27 10.6
Health Education 31 12.2
Clinical Nutrition 36 14.2
Clinical Psychology 21 8.3
Speech and Swallowing Disorders 11 4.3
Audiology and Balance 11 4.3
Nuclear Medicine Technology 13 5.1
Radiation Therapy 3 1.2
Diagnostic Imaging 10 3.9
Level of study
Entry 81 31.9
Intermediate 79 31.1
Graduate 94 37

319
Grade point average (GPA)

Excellent (> 4.5) 105 41.3
Very Good (range: 3.75–4.5) 128 50.4
Good (range: 2.5–3.75) 14 5.5
Poor (< 2.5) 0 0.0
Mean SD
Age (years) 21.0 ±1.6
3.2 Student independence, responsibility, goal achievement and anxiety

The results in Table 2 show that 66.5% of participants had a moderate perception
of their independence and responsibility in their learning, while 26% had a high
perception. Learning goal achievement was high among 16.1% of participants and
moderate for 54.3% of participants. Anxiety level results revealed that 20.9% had
no anxiety and 24.4% had severe anxiety.
Table 2: Student scores for responsibility and independence in learning, learning goal
achievement, and anxiety levels (N = 254)
Variable N %
Independence and responsibility
Low (range: 6-14) 19 7.5
Moderate (range: 15–22) 169 66.5
High (range: 23–30) 66 26.0
Goal achievement
Low (range: 5–11.6) 75 29.5
Moderate (range: 11.7–17.2) 138 54.3
High (range: 17.3–25) 41 16.1
Anxiety levels
No anxiety (range: 0–4) 53 20.9
Low anxiety (range: 5–9) 90 35.4
Moderate anxiety (range: 10–14) 49 19.3
Severe anxiety (range: 15–21) 62 24.4
Student perceptions of responsibility and independence in their learning during

ERT are presented in Figure 1. Roughly 67% of students selected the response
‘Shifting to remote learning made me more responsible about my own learning’.
Moreover, 74.8% selected ‘Shifting to remote learning helped me to become an
independent learner’ and 71% selected ‘Shifting to remote learning helped me to
work at my own pace’. However, only 30% selected ‘Shifting to remote learning
helped me manage my time more effectively’.

320
Strongly agree Agree Neutral Disagree Strongly disagree
Shifting to remote learning helped me

manage my time more effectively
Shifting to remote learning helped me to

work at my own pace
Shifting to remote learning helped me to

become an independent learner
Shifting to remote learning improved the
interaction between me and the course
instructor
Shifting to remote learning helped me to be
better organized
Shifting to remote learning made me more

responsible about my own learning
0% 20% 40% 60% 80% 100%
Figure 1: Student perceptions of responsibility and independence in their learning

during ERT (N = 254)
In contrast, student agreement on the impact of ERT on learning goal achievement

was lower. Only 54.3% selected the response ‘Shifting to remote learning enabled
me to accomplish assignments more quickly and efficiently’, 13% selected
‘Remote learning motivated better learning’, and 26% selected ‘Remote learning
helped in knowledge and skills development’ (Figure 2). When assessing anxiety
levels, we found that ‘several days’ had the highest selection rate for all questions,
except ‘Being so restless that it's hard to sit still’, where 40% stated that they did
not experience this at all.
Strongly agree Agree Neutral Disagree Strongly disagree
Remote learning motivated better learning
Remote learning helped in knowledge and

skills development
Remote learning helped me understand the

subject’s material well
Shifting to remote learning enabled me to
accomplish assignments more quickly and
efficiently
Shifting to remote learning improved the
quality of my studies
0% 20% 40% 60% 80% 100%
Figure 2: Impact of ERT on goal achievement (N = 254)

321
Not at all Several days Over half the days Nearly every day
Feeling afraid as if something awful might

happen
Becoming easily annoyed or irritable
Being so restless that it's hard to sit still
Trouble relaxing
Worrying too much about different things
Not being able to stop or control worrying
Feeling nervous, anxious, or on edge
0% 20% 40% 60% 80% 100%
Figure 3: Student General Anxiety Disorder-7 scores (N = 254)
3.3 Scores or independence and responsibility, goal achievement, and anxiety

The score results indicated that students’ levels of study significantly affected goal
achievement and anxiety levels (see Table 3). There was a significant difference in
mean GAD-7 score among the three levels (p = 0.024). Post-hoc analysis indicated
that the mean anxiety level was significantly lower for students at the
intermediate level compared to those in the entry level (p = 0.035). There was also
a significant difference in mean learning goal achievement among the three study
levels (p = 0.011), and the mean was higher for intermediate level students than
for graduate level students (p = 0.008).
Table 3: Mean (± SD) scores for independence and responsibility, goal achievement,
and anxiety levels for study levels (N = 254)
Study level
Range Entry Intermediate Graduate p-valuea
(n = 81) (n= 79) (n= 94)
Responsibility and independence 6–30 19.6 ± 3.6 20.4 ± 3.7 19.6 ± 3.2 0.283
Goal achievement 5–25 13.7 ± 3.9 14.7 ± 4.1b 12.9 ± 3.6b 0.011
Anxiety level 0–21 10.6 ± 5.5 c 8.3 ± 5.9c 10.2 ± 5.9 0.024
SD = Standard deviation. ANOVA was used to assess differences between scoring groups
for each study level. Mean scores were compared using the Bonferroni post-hoc test. a =
ANOVA p-value. b = Significant difference in mean goal achievement between students
in the intermediate and graduate levels (p = 0.035). c = Significant difference in mean
anxiety levels between students in the intermediate and entry levels (p = 0.008).

322
Table 4: Mean (± SD) scores for independence and responsibility and goal
achievement for different anxiety levels (N = 254)
Anxiety level
Range None Low Moderate Severe p-valuea
(n = 53) (n = 90) (n = 49) (n = 53)
Responsibility and
6–30 21.3 ± 2.3b,c 20.3 ± 3.6d 19.3 ± 3.4 18.4 ± 3.8 < 0.001
independence
Goal achievement
5–25 16.1 ± 3.2b,c,e 13.9 ± 3.7 d 12.8 ± 4.1 12.1 ± 3.8 < 0.001
of goals
SD = Standard deviation. ANOVA was used to assess differences between scoring groups
for each study level. Mean scores were compared using the Bonferroni post-hoc test. a
= ANOVA p-value. b = Significant difference between students with no anxiety and
moderate anxiety (p = 0.025). c = Significant difference between students with no anxiety
and severe anxiety (p < 0.001). d = Significant difference between students with low
anxiety and severe anxiety (p < 0.001). e = Significant difference between students with no
anxiety and low anxiety (p = 0.007).
Mean student perception scores for independence and responsibility, as well as

goal achievement, are sorted by anxiety level in Table 4. There was a significant
difference in mean independence and responsibility scores among the four
anxiety levels (p < 0.001). Mean scores were significantly higher for students with
no anxiety than moderate (21.3 ± 2.3 vs. 19.3 ± 3.4; p = 0.025) and severe anxiety
(21.3 ± 2.3 vs. 18.4 ± 3.8; p < 0.001). They were also higher for students with low
anxiety than severe anxiety (20.3 ± 3.6 vs. 18.4 ± 3.8; p < 0.001). Differences in mean
learning goal achievement were also significant for the four anxiety levels (p <
0.001; see Table 4). Bonferroni post-hoc analysis revealed that mean scores were
significantly higher in the no anxiety group compared to all other levels. In
addition, there was a significant difference in mean learning goal achievement
between the low and severe anxiety groups (13.9 ± 3.7 vs. 12.1 ± 3.8; p = 0.019).
3.4 Correlation between independence and responsibility, goal achievement,

and anxiety level
Table 5 summarises the correlation between each study variable. Pearson’s r
revealed a strong positive correlation between student perceptions of
independence and responsibility in their learning and learning goal achievement
(r = 0.68; p < 0.001). However, there was a moderate negative correlation between
anxiety level and student perceptions of independence and responsibility (r = -
0.30; p < 0.001) as well as learning goal achievement (r = -0.36; p < 0.001). GPA did
not have a significant correlation with any variables (Table 5).
Table 5: Correlation between independence and responsibility, goal achievement, and

student anxiety levels during ERT (N = 254)
Independence and Goal Anxiety
responsibility achievement level
Independence and 1 .68** -.30**
responsibility
Goal achievement --- 1 -.36**
Anxiety level --- --- 1
Pearson’s r was used to calculate the correlation strength between each set of variables.
**Significant at p < 0.001.

323
4. Discussion
This study assessed the learning process from the CHRS students’ point of view
during the sudden shift to virtual learning in response to the COVID-19
pandemic, measuring aspects related to anxiety, achievement of learning goals
and the perception of independence/responsibility toward their e-learning.
The findings of the study revealed that students considered the effect of the
sudden shift to remote teaching on their responsibilities and their
independence towards their learning to be high. Indeed, agreement on ‘Shifting
to remote learning helped me to become an independent learner’ and ‘Shifting to
remote learning made me more responsible about my own learning’ was reported
as 74.8% and 67.7%, respectively. Student recognition of responsibility towards
learning is an important factor (Joo, Lim & Kim, 2011). Ilic, Hart, Fiddes, Misso
and Villanueva (2013) indicated that there was no difference in student
competency between the blended learning and traditional dictation. However,
Joo et al. (2011) reported that student recognition of their responsibilities in the e-
learning environment was a predictor of learning persistence and learning
flow. Furthermore, the perception of achievement of learning goals was strongly
related to the perception of independence in learning in this study. Yang and Cao
(2013) concluded that self-regulation was related to learning achievement. In the
learning context, self-regulation refers to the degree to which students engage in
the learning process using metacognition and to proper motivation (Zimmerman
& Martinez-Pons, 1988). Moreover, it has also been demonstrated that
achievement of learning goals was a predictor of successful learning (Yang & Cao,
2013).
Motivation is a key element for students to engage in e-learning. Only 13.8% of

the students participating in this study reported that shifting to ERT improved
their motivation towards learning. However, a moderate achievement of goals
was obtained by 54.3%. Several studies have linked students’ motivation to
achievement of learning goals (Pekrun, 2006; Yang & Cao, 2013). Although the
percentage of students agreeing to the statement ’Remote learning motivated
better learning’ was not high, this did not reflect their overall perception
regarding the accomplishment of learning goals. In fact, half of the students
reported agreement on the statement ‘Shifting to remote learning enabled me to
accomplish assignments more quickly and efficiently’. Yang and Cao (2013)
indicated that a task-approach goal was not influenced by intrinsic motivation.
Student assessment is a major concern, and this may explain the higher level of
accomplishment regardless of motivation, as students may have focussed on
getting the task done and proceeding regardless of their sense of motivation.
In contrast, a substantial number of students in this study perceived e-learning as

adversely affecting the quality of their knowledge and skill acquisition as well as
their understanding of the subject material. In fact, it has been well demonstrated
that knowledge acquisition is an essential factor for interpreting the success of an
e-learning experience. Rodríguez-Ardura and Meseguer-Artola (2016) confirmed
that students are more concerned about the quality of knowledge in e-learning
than the quality of the learning management system itself. Some disadvantages of

324
e-learning may contribute to the dissatisfaction of the students in relation to e-

learning effectiveness. It is highly recommended in e-learning strategy that
students should have a high level of self-discipline, and they are expected to
spend more time and effort understanding the subject material on their own.
Moreover, the minimal interaction and direct contact with instructors in the e-
learning environment has a considerable influence on student perception of
effectiveness (Bouhnik & Marcus, 2006). Muilenburg and Berge (2005) indicated
that a lack of social interaction is a major barrier in terms of student perception of
e-learning effectiveness. In our context, the shift towards e-learning was sudden
and without prior planning, which may have increased student concerns for their
education. The suspension of all practical and clinical training may also have
contributed to student disagreement on the quality of their study. The alternatives
provided for practical and clinical training may not have been enough to alleviate
student concerns regarding their clinical-skill acquisition and may also be related
to the higher levels of student anxiety. This study revealed the existence of a
negative and significant correlation between the level of anxiety and the
perception of learning effectiveness; the higher the anxiety, the lower the
perception of learning effectiveness. You and Kang (2014) investigated the impact
of emotions in the academic setting, including anxiety, on perceived academic
control. They reported the existence of a significant relationship between anxiety
and academic control. Perceived control is
‘an individual’s believed capacity to predict and influence events in their
environment’. (Perry & Smart, 1997, p. 12)
It is also considered an important predictor of achievement emotions (Pekrun,

2006) as well as academic motivation and performance (Perry, Hladkyj, Pekrun &
Pelletier, 2001). Conversely, anxiety is one of the most frequent emotions reported
as a result of e-learning (Hara, 2000; Hara & Kling, 2001; You, Kang & Pahng,
2013). The current study assessed anxiety using GAD-7, while anxiety in the
previous studies was part of a comprehensive questionnaire involving other
academic emotions related to e-learning. However, higher anxiety levels had
already been reported by college students in general (Beiter et al., 2015).
Moreover, several studies indicated that anxiety is one of the most frequent
emotions reported due to e-learning (Hara & Kling, 2001). Additionally, concerns
regarding the COVID-19 pandemic could be a contributing source to the overall
anxiety reported by students (Cao et al., 2020; Liu, Liu & Zhong, 2020).
The results showed that the perceived independence/responsibility of students

towards their learning and their achievement of learning goals were significantly
impacted by the level of anxiety. Perry et al. (2001) indicated that perceived
academic control was affected by a student’s anxiety. Additionally, students at the
graduate and entry levels showed higher anxiety than those at the intermediate
level. Similarly, Beiter et al. (2015) reported higher anxiety in students at higher
levels. Furthermore, Moore, Madsen and Trans (2019) demonstrated high anxiety
in first-year medical students. Consequently, the perception of achievement of
learning goals and student responsibility were greater at intermediate levels,
which could be influenced by anxiety. Cao et al. (2020) investigated anxiety levels

325
in college students using GAD-7. They reported that anxiety in students during
the COVID-19 pandemic impacted academic activities and could lead to delays in
their learning. However, they reported lower anxiety levels compared to the levels
obtained in this study, as only 0.9% of their subjects had severe anxiety. However,
Saddik et al. (2020) investigated the anxiety level in relation to e-learning among
university students using GAD-7. The percentages for the anxiety levels were
similar to this study. They indicated that medical students had higher levels of
anxiety before shifting to e-learning; interestingly, they demonstrated that the
anxiety levels were lower after the introduction of e-learning when compared to
non-medical students. Also, female participants in their study demonstrated
higher anxiety levels compared to male participants. However, female
participants demonstrated lower anxiety levels with the shift to ERT in response
to the COVID-19 pandemic. Therefore, during a crisis that causes a shift to e-
learning, such as the COVID-19 pandemic or a similar emergency, educational
institutions should support student mental health along with student learning.
5. Conclusion
Taken together, the findings of this study highlight the connection between
anxiety levels and the perception of the learning experience during the sudden
shift to virtual learning in response to the COVID-19 pandemic among health-
profession students in CHRS (PNU). Anxiety levels were significantly high,
which could be considered a logical outcome of the ERT and the total lockdown
in response to the COVID-19 pandemic. This anxiety had a moderate effect on the
perception of both the independence and responsibility of students towards their
learning as well as on their achievement of learning goals. Furthermore, as 24.4%
of the students reported severe anxiety, it is recommended that there should be a
plan to deliver both specific education and psychological support to help mitigate
any potentially negative consequences on the learning process during the COVID-
19 pandemic. Findings can also be extrapolated to similar emergencies where ERT
is imposed. Additionally, while e-learning is generally encouraged in the case of
health-profession education, well-designed blended learning could be considered
a preferable learning strategy to ensure the benefits of e-learning while
accommodating clinical and practical training. However, further studies are
needed to explore in depth the type of assistance that would improve the student
learning experience during the current COVID-19 pandemic.
6. Strengths and limitations

The study instrument used for data collection was highly reliable, as
demonstrated by good internal consistency: α = 0.91 for anxiety, α = 0.72 for
responsibility and α = 0.82 for achieving goals. Moreover, the survey was
conducted during the COVID-19 pandemic and hence represents precise insight
from CHRS students in relation to their e-learning experience. Lastly, reviewing
the literature, this study seems to be the first to focus on the learning process from
the perspective of health-profession students at Saudi universities. Thus,
evaluating this experience could bring valuable results and provide guidance for
the decision-making process at CHRS concerning which aspects and components
of learning should be prioritised to improve e-learning from a long-term

326
perspective. However, due to the sudden shift, the research could not compare
results to data pre-ERT.
The data was collected through a non-probability sampling for a known

population; the inclusion criteria were clear and students belonged to the same
college (CHRS), which indicates that this sampling procedure would have a
reduced bias. Consequently, the findings of this research can be generalised
beyond the study population.
Acknowledgments: This research was funded by the Deanship of Scientific

Research at Princess Nourah bint Abdulrahman University through the Fast-track
Research Funding Program.
Conflicts of interest/Competing interests: There are no conflicts of interest.
Ethics approval: Ethical approval from the Institutional Review Board/Ethics

Committee at Princess Nourah Bint Abdulrahman University was obtained prior
to conducting the study (IRB Log Number 20-0162).
Consent to participate: The consent to participate was displayed on the first page
of the online questionnaire and included the objectives of the study. Participants
were recruited voluntarily, and anonymity was maintained through all stages of
the research. Participants were assured of the confidentiality of their data and that
it would only be used for research purposes. Participants were informed that by
answering the questionnaire they were giving their consent to participate in the
study.
Author contributions: All the authors (Fatmah Almoayad, Afrah Almuwais,

Samiah F. Alqabbani, and Nada Benajiba) did the following:
- made a substantial contribution to the design of the work and the
acquisition and statistical analysis of the data;
- contributed significantly to the interpretation of the data and the drafting
of the article;
- contributed in collating the supporting literature, adding to and
improving the scientific merit of the study and revising the manuscript
for English language grammar, syntax and flow as well as correct
formatting;
- revised and approved the version to be published; and
- participated sufficiently in the work to take public responsibility for
appropriate portions of the content.
7. References
Al-Asmari, A. M., & Khan, M. S. R. (2014). E-learning in Saudi Arabia: Past, present and
future. Near and Middle Eastern Journal of Research in Education, 2014(1), 2.
https://doi.org/10.5339/nmejre.2014.2
Al-Shehri, A. M. (2010). E-learning in Saudi Arabia:‘To E or not to E, that is the question’.
Journal of family and community medicine, 17(3), 147. https://doi.org/10.4103/1319-
1683.74333

327
Albarrak, A. (2011). E-learning in medical education and blended learning approach.

learning, 13, 14-20. https://doi.org/10.1111/j.1540-4609.2011.00325.x
Algahtani, A. (2011). Evaluating the E ectiveness of the E-learning Experience in Some
Universities in Saudi Arabia from Male Students' Perceptions. (PhD), Durham
University.
Beiter, R., Nash, R., McCrady, M., Rhoades, D., Linscomb, M., Clarahan, M., & Sammut,
S. (2015). The prevalence and correlates of depression, anxiety, and stress in a
sample of college students. Journal of affective disorders, 173, 90-96.
https://doi.org/10.1016/j.jad.2014.10.054
Bouhnik, D., & Marcus, T. (2006). Interaction in distance‐learning courses. Journal of the
American Society for Information Science and Technology, 57(3), 299-305.
https://doi.org/10.1002/asi.20277
Cao, W., Fang, Z., Hou, G., Han, M., Xu, X., Dong, J., & Zheng, J. (2020). The psychological
impact of the COVID-19 epidemic on college students in China. Psychiatry
research, 287, 112934. https://doi.org/10.1016/j.psychres.2020.112934
Corter, J. E., Esche, S. K., Chassapis, C., Ma, J., & Nickerson, J. V. (2011). Process and
learning outcomes from remotely-operated, simulated, and hands-on student
laboratories. Computers & Education, 57(3), 2054-2067.
Drew, S. (2018). The student skills: guide. London, UK: Routledge.
Edwards, A., & McKinnell, S. (2007). Moving from dependence to independence: the
application of e-learning in higher education. Learning, teaching and assessing in
higher education: Developing reflective practice, 68.
Ellaway, R., & Masters, K. (2008). AMEE Guide 32: e-Learning in medical education Part
1: Learning, teaching and assessment. Medical teacher, 30(5), 455-473.
https://doi.org/10.1080/01421590802108331
Gallagher, T. H., & Schleyer, A. M. (2020). “We Signed Up for This!”—student and trainee
responses to the COVID-19 pandemic. New England Journal of Medicine, 25, e96.
https://doi.org/10.1056/nejmp2005234
George, P. P., Papachristou, N., Belisario, J. M., Wang, W., Wark, P. A., Cotic, Z., . . . Car,
J. (2014). Online eLearning for undergraduates in health professions: A systematic
review of the impact on knowledge, skills, attitudes and satisfaction. Journal of
global health, 4(1), 010406-010406. https://doi.org/10.7189/jogh.04.010406
Ghenghesh, P., Croxford, L., Nagaty, K., & Abdelmageed, S. (2018). Students and Teachers
Attitudes and Satisfaction toward E-Learning: A Case Study in Egypt. The Journal
of Middle East and North Africa Sciences, 4(1), 1-17.
https://doi.org/10.12816/0044059
Hara, N. (2000). Student distress in a web-based distance education course. Information,
Communication & Society, 3(4), 557-579.
https://doi.org/10.1080/13691180010002297
Hara, N., & Kling, R. (2001). Student distress in web-based distance education. Educause
Quarterly, 24(3), 68-69.
Haslerud, T., Tulipan, A. J., Gray Jr, R. M., & Biermann, M. (2017). E-learning for medical
imaging specialists: introducing blended learning in a nuclear medicine specialist
course. Acta radiologica open, 6(7), 2058460117720858.
https://doi.org/10.1177/2058460117720858
Hawkins, B., & Oblinger, D. (2005). The Myth about E-Learning. Educause Review, 40(4),
14-15.
Hiltz, S. R., & Turoff, M. (2005). Education goes digital: The evolution of online learning
and the revolution in higher education. Communications of the ACM, 48(10), 59-64.
https://doi.org/10.1145/1089107.1089139

328
Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between
emergency remote teaching and online learning. Educause Review, 27.
Hubackova, S. (2015). History and perspectives of elearning. Procedia-Social and Behavioral
Sciences, 191, 1187-1190. https://doi.org/10.1016/j.sbspro.2015.04.594
Ilic, D., Hart, W., Fiddes, P., Misso, M., & Villanueva, E. (2013). Adopting a blended
learning approach to teaching evidence based medicine: a mixed methods study.
BMC medical education, 13(1), 169. https://doi.org/10.1186/1472-6920-13-169
Joo, Y. J., Lim, K. Y., & Kim, E. K. (2011). Online university students' satisfaction and
persistence: Examining perceived level of presence, usefulness and ease of use as
predictors in a structural model. Computers & Education, 57(2), 1654-1664.
Khan, B. H. (2005). Managing e-learning: Design, delivery, implementation, and evaluation.
London, UK: Idea Group Inc (IGI).
Likert, R. (1932). A technique for the measurement of attitudes. Archives of psychology,
22(140), 55.
Liu, X., Liu, J., & Zhong, X. (2020). Psychological State of College Students During COVID-
19 Epidemic (3/10/2020). http://dx.doi.org/10.2139/ssrn.3552814
MOE. (2020). Distance education in Saudi universities complements the educational
journey of male and female students. Retrieved from
https://www.moe.gov.sa/ar/news/Pages/un-c-2020-75.aspx
Moore, R., Madsen, L. V., & Trans, M. (2019). Stress Sensitivity and Signs of Anxiety or
Depression among First Year Clinical Dental and Medical Students. Open Journal
of Medical Psychology, 9(1), 7-20. https://doi.org/10.4236/ojmp.2020.91002
Muilenburg, L. Y., & Berge, Z. L. (2005). Student barriers to online learning: A factor
analytic study. Distance education, 26(1), 29-48.
https://doi.org/10.1080/01587910500081269
Ngan, O. M. Y., Tang, T. L. H., Chan, A. K. Y., Chen, D. M., & Tang, M. K. (2018). Blended
learning in anatomy teaching for non-medical students: an innovative approach
to the health professions education. Health Professions Education, 4(2), 149-158.
https://doi.org/10.1016/j.hpe.2017.11.001
Nunnally, J. C. (1975). Psychometric theory—25 years ago and now. Educational Researcher,
4(10), 7-21. https://doi.org/10.3102/0013189x004010007
Odriozola-González, P., Planchuelo-Gómez, Á., Irurtia, M. J., & de Luis-García, R. (2020).
Psychological effects of the COVID-19 outbreak and lockdown among students
and workers of a Spanish university. Psychiatry research, 290, 113108.
https://doi.org/10.1016/j.psychres.2020.113108
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions,
corollaries, and implications for educational research and practice. Educational
psychology review, 18(4), 315-341. https://doi.org/10.1007/s10648-006-9029-9
Perry, R. P., Hladkyj, S., Pekrun, R. H., & Pelletier, S. T. (2001). Academic control and
action control in the achievement of college students: A longitudinal field study.
Journal of educational psychology, 93(4), 776. https://doi.org/10.1037/0022-
0663.93.4.776
Perry, R. P., & Smart, J. C. (1997). Effective teaching in higher education: Research and practice.
New York, USA: Agathon Press.
PNU. (2020). College of Health and Rehabilitation Sciences. Retrieved 15/4/2020.
Retrieved from https://www.pnu.edu.sa/en/Faculties/Health-
Rehabilitation/Pages/Home.aspx
Rodríguez-Ardura, I., & Meseguer-Artola, A. (2016). What leads people to keep on e-
learning? An empirical analysis of users' experiences and their effects on

329
continuance intention. Interactive Learning Environments, 24(6), 1030-1053.

https://doi.org/10.1080/10494820.2014.926275
Saddik, B., Hussein, A., Sharif-Askari, F. S., Kheder, W., Temsah, M.-H., Koutaich, R. A., .
. . Hamid, Q. (2020). Increased levels of anxiety among medical and non-medical
university students during the COVID-19 pandemic in the United Arab Emirates
(13/5/2020). medRxiv. https://doi.org/10.1101/2020.05.10.20096933
Sajid, M. R., Laheji, A. F., Abothenain, F., Salam, Y., AlJayar, D., & Obeidat, A. (2016). Can
blended learning and the flipped classroom improve student learning and
satisfaction in Saudi Arabia? International journal of medical education, 7, 281-285.
https://doi.org/10.5116/ijme.57a7.83d4
SPA. (2020, 6/4/2020). Saudi Arabia imposes 24-Hour Curfew Saudi Press Agency.
Retrieved from
https://www.spa.gov.sa/viewfullstory.php?lang=en&newsid=2071013
Spitzer, R. L., Kroenke, K., Williams, J. B., & Löwe, B. (2006). A brief measure for assessing
generalized anxiety disorder: the GAD-7. Archives of internal medicine, 166(10),
1092-1097. https://doi.org/10.1001/archinte.166.10.1092
Uziak, J., Oladiran, M. T., Lorencowicz, E., & Becker, K. (2018). Students' and instructor's
perspective on the use of Blackboard Platform for delivering an engineering
course. The Electronic Journal of e-Learning, 16(1), 1.
Warmbrod, J. R. (2014). Reporting and Interpreting Scores Derived from Likert-Type
Scales. Journal of Agricultural Education, 55(5), 30-47.
https://doi.org/10.5032/jae.2014.05030
Yang, Y., & Cao, L. (2013). Differential influences of achievement approach goals and
intrinsic/extrinsic motivation on help-seeking in e-learning. Knowledge
Management & E-Learning: An International Journal, 5(2), 153-169.
https://doi.org/10.34105/j.kmel.2013.05.011
You, J., Kang, M., & Pahng, P. (2013). Moderating effects of academic emotions between
perceived academic control and self-regulated learning in online courses. Paper
presented at the E-Learn: World Conference on E-Learning in Corporate,
Government, Healthcare, and Higher Education, Las Vegas, NV, USA.
You, J. W., & Kang, M. (2014). The role of academic emotions in the relationship between
perceived academic control and self-regulated learning in online learning.
Computers & Education, 77, 125-133.
Zakaria, N., Jamal, A., Bisht, S., & Koppel, C. (2013). Embedding a learning management
system into an undergraduate medical informatics course in Saudi Arabia: lessons
learned. Medicine 2.0, 2(2), e13. https://doi.org/10.2196/med20.2735
Zhai, Y., & Du, X. (2020). Addressing collegiate mental health amid COVID-19 pandemic.
Psychiatry research, 288, 113003. https://doi.org/10.1016/j.psychres.2020.113003
Zimmerman, B. J., & Martinez-Pons, M. (1988). Construct validation of a strategy model
of student self-regulated learning. Journal of educational psychology, 80(3), 284.
https://doi.org/10.1037/0022-0663.80.3.284

330

Vol. 19, No. 8, pp. 330-349, August 2020
Baseline Assessment in the Elementary

Mathematics Classroom: Should it be Optional or
Mandatory for Teaching and Learning?
Mamsi Ethe Khuzwayo

Cape Peninsula University of technology, Cape Town in Mowbray, South Africa
http://orcid.org/0000-0002-4084-8773
Herbert Bhekumusa Khuzwayo

University of the Western Cape, Cape Town in Belville, South Africa
http://orcid.org/0000-0002-6732-1787
Abstract. Too few mathematics teachers have the expertise to apply

baseline assessment for differentiated learning in mathematics classes.
This empirical study was conducted in a focus group of sixty learners
and three teachers in a grade four mathematics class. It proved that
teachers of mathematics in elementary grades could profitably learn to
conduct baseline assessments; developing activities to benchmark
learners’ abilities and learning needs. The exploratory research design
was utilized to set up the general sample and the focus group, and
mixed research methods; qualitative and quantitative instruments to
collect data and analyzing data. The records of scores obtained by
learners from the assessment activities and the interviews with teachers
were the main sources of data. Data analysis followed quantitative
procedures and presented in figures and categorized them in frequency
distribution tables. Findings highlighted that learners can reach their
optimal levels in learning mathematics in a free and flexible
environment. This paper concludes that the lack of knowledge about
how to conduct baseline and its value is one of the attributes of the high
failure rate of children in the middle class. This article recommends the
consideration of baseline assessment to be an essential aspect of a
continuous professional development program for mathematics
teachers.
Keywords: Assessment; Baseline; Differentiated; Learning; Styles
1. Introduction
The democratic education dispensation in South Africa is in keeping with the
international trend to acknowledge diverse learners’ needs. White paper 6 (DBE,
2011) was promulgated by the Department of Education to enforce the principle
of inclusivity in the teaching and learning programs in mainstream classrooms.

331
Research highlights, however, that implementing principles of inclusivity in

curriculum delivery at the classroom level is a challenge to many teachers
(Donohue & Bornman, 2014; Hazel, Spencer & Robert, 2019). An in-depth review
of the mathematics curriculum innovation called: Curriculum and Assessment
Policy Statement called, (CAPS) informed this work and shows gaps between
the curriculum blueprint and multi-level teaching and learning. Mathematics
teachers are expected to bridge these gaps for multi-level and differentiated
learning to take place in mathematics classes.
According to the principles stipulated in the White Paper 6; mathematics

teachers are expected to perform the following pedagogical roles; design
learning activities that are sensitive to the varying needs of both the subject and
the learners, create a supportive and empowering environment for the learners,
and integrate assessment between teaching and learning (Department of Basic
Education, 2011; 2019). Rhetorically, the attributes and the roles for the
competent teacher related to inclusive education system indicate that: first,
educators must have an understanding of the; purposes, methods, and effects of
assessment and be able to provide helpful feedback to learners. Second, design
and manage both formative and summative assessments in ways that are
appropriate to the level and purpose of the learning. Third, understand how to
interpret and use assessment results to feed into processes for the improvement
of learning programs. Fourth, keep detailed and diagnostic records of
assessment results (Department of Higher Education and Training, 2019, p.58).
The narrative about inclusivity and diversity often centers on the issues of socio-
economic background, physical challenges, and cognitive abilities (Tomlinson,
2013; Aghamdi, Jitendra & Lein, 2019). This article relates issues of the poor
performance of learners in mathematics too, poor preparation of teachers in the
subject content knowledge and pedagogy, to consider diverse learning needs in
mathematics classes. The perspective, which we have adopted in this work
emphasizes that mathematics classrooms are composed of multiple groups of
learners; some do not have difficulties with mathematical knowledge and skills
but who experience challenges with learning styles that are not aligned with
their cognitive structures.
2. Background
The Department of Basic Education (DBE) produced curriculum guidelines for
teachers and school-based curriculum specialist and district-based subject
advisors yet research points to a deteriorating performance among learners in
grade 3 and grade 6 (Department of Basic Education, 2011; Venkat & Spaull,
2015; Department of Basic Education, 2019). Intermediate (middle grades)
learners are often unable to perform necessary, elementary cognitive numerical
and mathematical skills (Mullis, Martin & Hooper, 2017). Venkat and Spaull
(2015, p. 127) revealed the results of the analysis of data gathered by, Trends in
International Mathematics and Science Study (TIMSS) that, 61% of South
African Grade 5 learners could not do basic mathematics, for example,
performing of addition and subtraction of the whole numbers, multiplication by
one-digit numbers, and cannot solve simple word problems.

332
This work recognizes the findings reporting the inadequacy of the initiative of
the DBE to increase the performance of learners in mathematics at elementary
grades. Documents produced by the National Department of Education explain
the initiative called ‘The Foundations of Learning Campaign’, a project
introduced by the Department of Education in 2008 to increase the performance
of all primary school learners in Numeracy/Mathematics to no less than 50%.
The Foundations for Learning Campaign culminated in a national evaluation at
the end of 2011 to assess the Literacy (Languages) and Numeracy (Mathematics)
levels of grade three and six learners in South Africa to determine the impact of
the campaign (Department of Basic Education, 2019).
The analysis of the results of the Foundations of Learning Campaign revealed

that the campaign did not focus on the alternative pedagogical approaches;
instead teachers were merely given material. Research continued to point at the
inconsistencies in the pedagogical approaches, emanating from the unaligned
and uninformed practices in mathematics classrooms nationally (Hazell, Spencer
& Robert, 2019; Henning et al. (2019). In the same, vein (Graven, 2014) agreed
that the campaign did not yield positive results to implement alternative
pedagogical approaches for teaching and learning of mathematics in elementary
classes. Amidst, the challenges that threatened the effective implementation of
the campaign and other innovations to improve the teaching and learning of
mathematics, the main issue of concern is the incompetency of teachers to
address multi-level abilities and diverse learning difficulties experienced by
individual learners in classrooms (Reddy, 2006; Department of Basic Education,
2019; Fleisch, 2008; Schollar, 2008). Research shows that the challenge of
overcrowded classrooms, lack of resources, and rapid changes in the alignment
of the curriculum are the attributes of a high rate of failure in mathematics
(Tomlinson, 2020; Blomeke, Kaiser & Koning, 2020). Spaull and Kotze (2015),
however, argue that the issues of the language comprise the root course of the
high failure rate in mathematics classes in South Africa. Henning et al. (2019) in
the same vein write that there is a great need for the use of home language to
teaching and learning other than the second language which has a profoundly
negative impact upon the acquisition of mathematical concepts in elementary
classes.
Concerns are mounting about the poor performance of learners in mathematics.

The problem identified after the critical synthesis of the findings and
recommendations is that, there seems to be little information said in the
literature regarding accommodating diverse learning styles in the specialized
mathematics pedagogical content knowledge to support the implementation of
inclusive policy in classrooms in South Africa. The curriculum change policy
indicates that the envisaged teacher should:(i) demonstrate an ability to prepare
and develop activities that complement diverse learning styles in the classroom,
(ii) be a subject and phase specialist and, be able to conduct assessments.
The question that this study sought to address was: Is baseline assessment
significant to benchmark diverse learning needs in the elementary mathematics
classes?

333
The problem statement for the study was, to explore the approaches applied by
learners to acquire mathematics content knowledge; concepts, cognitive skills,
and application in the multiplication activities designed by teachers.
Relevant literature posits that the perspective of assessment held by the National
Department was that of integrating assessment to teaching and learning. The
rationale to integrate, to monitor learning progress was to provide necessary
feedback to individual learners in the process of teaching and learning of
mathematics content knowledge (Department of Basic Education, 2011, 2019).
The advocacy document on assessment policy pioneered a view of formative
and developmental assessment and emphasized continuous assessment where a
variety of techniques and methods could be applied to collect or gather evidence
on learners’ performance (Aghamdi, Jitendra & Lein, 2019; Andrade &
Brookhart, 2019; Department of Basic Education, 2019). Teachers of
mathematics are expected to conceptualize in their practices of pedagogy is that
of a continuum that serves to highlight the learning needs of individual learners
concerning the goals and objectives of the mathematics curriculum. The practice
of continuous assessment in South African schools stresses three purposes of
assessment (baseline, formative and summative) in the learning environment.
The most critical purpose of assessment is to identify gaps in learners’ previous
knowledge and identify learners’ levels of cognitive and affective abilities.
According to the guidelines of the Department; for the evaluation to serve the
baseline purpose, teachers are expected to design activities which benchmark
levels of skills, proficiency, expertise, existing knowledge gaps and
misconceptions at the beginning of the new grade, and before embarking on the
new theme of learning context (Henning et al., 2019; Bondie, Dahnke & Zusho,
2019).
In the same narrative, Tiymms (2013), and Tomlinson (2020) highlights the
results of the baseline assessment, which provides the teacher with information
about learners’ abilities and knowledge gaps. Based on that information, the
teacher can structure the learning content, select and align strategies of teaching
and learning with each learner, or a group of learners’ learning needs. A
synthesis of policy documents for teacher qualifications in South Africa shows
that pre-service and in-service programs should equip teachers with
competences in assessment, program designers, and subject/phase specialists
(Department of Basic Education, 2019, Department of higher education, 2019).
Semana and Santos (2018) posit that the proposed new approaches to assessment
and teaching depend on the eagerness and commitment of teachers to track the
progress made by individual learners in the classroom. Proponents of baseline
assessment hold that assessment tasks should provide adequate opportunities
for learners to express their individuality. Research suggests that in a group of
mixed abilities and learning styles, teachers should first have mastered the
knowledge and skills for diagnosing learners’ difficulties along with styles of
learning before making judgments about the capabilities of learners in the
mathematics class (Tomlinson & Tonya, 2013; Sausa & Tomlinson, 2011; Wright,
Martland & Stafford, 2006). For teachers to be competent assessors, they are

334
expected to use these opportunities to develop learning profiles for individual

learners which points to the conditions under, which some learners learn best.
The information gathered during the assessment enables teachers to create and
adjust subject content knowledge and contexts of learning concepts and skills to
suit learners’ interests, experiences, and abilities (Baubeng, Atingane & Amoako,
2019; Goulet-Lyle, Voyer & Wershaffel, 2019; Amoako, Brembong & Bartey,
2019; Kim, 2019; Nguare, Hungi & Mutisya, 2018).
Conceptualized framework for baseline assessment

Advocates of baseline assessment in the context of education assert that the
principal reason for a baseline in teaching and learning is to get to know
children at the entry-level of a new phase of schooling (Tiymms, 2013; Nguare,
Hungi & Matisya, 2018; Tomlinson, 2020). The purpose of baseline assessment,
according to the Department of Basic Education (2019) is to help teachers to
develop learning activities that are inclusive of the differentiated learning styles,
and to identify the special needs of children at an early stage, so that the
remediation program can be put in place. Tomlinson and Moon (2013, p. 182)
agree that baseline assessment is supposed to be at the heart of curriculum
innovations since it establishes a baseline for gauging future progress, not only
for individual learners but for the entire class. According to some researchers,
baseline assessment may be organized in various forms which, could be: a test
written by all learners in a class at the beginning of the learning program, which
could be carried out in 20 to 25 minutes. Alternatively, baseline assessment
could be carried out through easily manageable activities that are carefully
developed to measure subjective and objectives values and indicators necessary
for future development and progress (Henning et al., 2019; Kim, 2019; Nguare,
Hungi & Mutisya, 2018).
4. Theoretical framework
The theoretical framework for the research project was established from the
principles asserted in the theory of multiple intelligences, Multi-levels, and
differentiated learning (Tomlinson, 2020; Bondie, Dahnke & Zusho; 2019;
Tomlinson, 2013; Davis, Christodoulou, Seider & Gardner, 2011). These theories
explain the formation or composition of a classroom as a community of
individuals who come from different socio-cultural and economic backgrounds,
and individuals who have bits of intelligence, experiences, and interests are
nurtured.
Dryden and Vos (2005, p.146), indicate that bits of intelligence are certain traits
and propensities for learning specifics skills that everyone possesses at birth. The
development of the potential bits of bits of intelligence in learning is determined
by how they are nurtured to become sets of skills and abilities. According to
Sternberg (1985) in Dryden and Vos (2005, p. 146), intelligence takes three forms:
analytical, creative, and practical. Gestalt Psychologists uphold a similar
theoretical view about the development of cognitive and affective skills: they
point out the importance of interest, experience, configuration, patterns in the
stimuli, and relations (Bondie, Dahnke & Zusho, 2019).

335
Multi-level and multi-intelligence theories advert to the critical principles to be

reckoned with when identifying traits and propensities of learners: (i) flexibility
in the organizations of learning material (ii) reading and comprehension of the
language of teaching and learning and (iii) familiarity with the context of in
which learning of content knowledge takes place (Wright, Martland & Stafford,
2006).
5. Methodology and Research Design

This study employed a mixed-method research paradigm drawing from the
ideas and views of Creswell and Creswell (2018, p.265) quantitative and
qualitative methods allow the researcher to gather numerical and narrative data
from the same sample. According to Henning, Van Rensberg & Smit (2013, p.35)
qualitative, the methods of data collection can be personal narratives or
participatory observation. The phenomenological case design was appropriate
for the selection of participants, managing the phases of qualitative data
collection and data analysis.
This study had two sources of data gathered through the quantitative and
qualitative procedures, the learners’ scores comprise quantitative and narrations
of the qualitative data from interviews.
Sampling
A purposive sampling strategy was used to select participants in the sample and
focus groups. The purpose was to gather reliable and valid data upon which
findings could address the research problem and the purpose of the study.
Grade 4 teachers from each of the three schools selected twenty mathematics
learner books, and the total number of books analyzed in this study was 60. The
rationale for choosing class four learners in this sample is that class four is an
entry grade in the middle schooling or intermediate phase.
Instruments
Tymms (2013, p.233) suggests three types of instruments for gathering data:
employing, observation schedule and assessment activities for quantitative data
and in-depth interviews for qualitative data. Observations alone, however,
cannot provide objective comparative data, nor can they probe indicators of
individual learners’ learning styles (Tymms, 2013). Observations and assessment
activities were used in this study to complement subjective and objective data
and to ensure that good productive validity was monitored. In the same vein,
Bondie et al., (2019) state that in mathematical learning, particularly, observation
may not reveal indicators, hence carefully chosen activities are required to
supplement data. In-depth interviews were conducted after the collection of data
from the learner workbook, and the results of baseline assessment activities were
analyzed.
6. Data collection procedures

Ethics for this study were considered, and that entailed permission from
gatekeepers, the Kwazulu-Natal Department of Education, and school
principals. The consent of the participants was given in writing. Anonymity and
confidentiality of all participants were assured.

336
First stage data collection: Meetings were arranged with three grade four
mathematics teachers in three different primary schools in a selected township
in the Northern KwaZulu-Natal province, respectively. Schools were visited on
different days during the second term. The topic which was chosen was on
multiplication according to the school’s work schedules. It was selected because
it was part of the work which was covered during the term in which the study
was conducted. Schools A, B, and C were using the same workbook. Learners
completed the same activities in the grade 4 learners’ book as prescribed by the
Department of Basic Education.
Procedures of data analysis

The quantitative data was coded, categorized, and organized according to the
trends and patterns manifested. Trends from the coded data were identified
through the observed frequency showing similarities from each category. The
categories focused on the descriptions of techniques from the Multiple
Intelligences, and they were of assistance for aligning learners’ differentiated
approaches to solving problem applying mathematical operations in different
contexts of the activities. The pie chart and tables were generated to present the
summaries of data. Qualitative data was also coded and classified under
categories and thereafter patterns of thought manifested were organized into
themes.
Data gathered from learners’ Mathematics learner books.

The first set of data was gathered from the analysis of learners’ performance in
three multiplication activities. Recorded data were classified under the
categories in the pie chart below, indicating difficulties experienced by learners
with the multiplication activities in grade 4 learner books.
Categories of learning difficulties experienced by learners with

multiplication
N=60 difficulties with concept
"multiply" 25%
diffuclties with
20% calculating skills in
25% multiples 40%
15% difficulties with symbol
"X" in mathematics 15%
40%
difficulties with reading
instructions from the
learner book 20%
Figure 1: the pie chart showing the types of difficulties recorded from learners’
performance in the multiplication activities.
Data gathered from baseline assessment activities.

The second phase of data collection focused upon learner performance in
learning activities designed by teachers under the supervision of the researcher.

337
The researcher and teachers were guided by three critical principles

recommended in the Department of Basic Education (2019), Tomlinson (2020),
and Bondie et al. (2019) where they developed and organized activities on
worksheets: flexibility in the organizations of learning material, reading, and
comprehension of the language of teaching and learning and familiarity with the
context of Education. Learners were allowed to choose worksheets which they
liked the most as long as they had the groupings of two and four. Below is the
set of activities developed per differentiated learning styles for grade four
learners, respectively.
The learning outcome for the activities: by the end of the learning activities,
learners should be able to:
Differentiate between continuous addition and multiplication, count items in
groupings, and to understand the concept of times, multiples through repeated
counting.
Assessment Criteria or expected performed knowledge and skills from the

activities were to; (i) apply the concept multiple as groupings (ii) Substitute
items with symbols Repeating counting of 1 grouping in the context of times to
develop tables of 2 and 4 on their own
Instructions:
• Cut a picture from the worksheet and paste it on the sheet of paper provided
and write two on top
• Next to the margin write numbers 1,2,3,4,5,6,7
• Count any grouping of two you see on the picture or objects you have chosen
using the numbers 1,2,3 -7
Count any grouping of four you see on your picture or items using the numbers
1, 2, 3-7
Visual-spatial learners: used objects to count in multiples of 2 and 4. They scored

the groupings on the one object repeatedly until they completed developing
their tables. According to Davis et al. (2011), the visual-spatial learning style is
characterized by an interest in working with real objects and pictures or
photographs. The physical object e.g., counters and pictures, as well as
drawings, were provided.
For example:
2 Ears of the dog (x) multiple 4 legs of the dog
Counting in (times) totals
One time =ears 1x2= ----- -- ears one time = 1x4= legs
Two times = ears2x2= ------- ears Two times= 2x4=legs
Three times = ears 3x2 =-------ears Three times= 3x4= legs
Four times = ears 4x2= ------ears Four times= 4x4= legs
Five times = ears 5X2= ------ears Five times= 5x4=legs
Six times = ears6x2= -------ears Six times = 6x4=legs
Seven times7x2 = ---------ears Seven times = 7x4=legs

338
Bodily-Kinesthetic Learners: used baking pans and cookies to develop their

tables of the multiples of 2’s and 4’s. Davis et al. (2011) describes bodily-
kinesthetic learning styles, to be physical movements. Enjoy creating things with
his or her hands, remember by doing, rather than hearing or seeing.
The game was about sorting cupcakes into boxes. Sorting began with
One box of 2cupcakes, 1x2= ---------
Two boxes of 2 cupcakes 2x2= ---------
Three boxes of 2cupcakes 3x2= --------
Four boxes of 2 cupcakes 4x2= ---------
Five boxes of 2 cupcakes 5x2= ---------
Six boxes of 2 cupcakes 6x2= --------
Seven boxes of 2 cupcakes 7x2= --------
One box of 4 cupcakes 1x4= ------cupcakes

Two boxes of 4 cupcakes 2x4= ------- cupcakes
Three boxes of 4 cupcakes 3x4= ------ cupcakes
Four boxes of 4 cupcakes 4x4= --------cupcakes
Five boxes of 4 cupcakes 5x4= --------cupcakes
Six boxes of 4 cupcakes 6x4= -------- cupcakes
Seven boxes of 4 cupcakes 7x4= -------- cupcakes
Verbal Linguistic and Interpersonal learners were allowed to solve problems

by reading story sums, pictures, and drawing they have created. Davis,
Christodoulou, Seider & Gardner (2011) asserted that the following interests and
capabilities characterize the verbal-linguistic style of learning: Interpret pictures,
graphs, and charts well; enjoy drawing, painting, and the visual arts, as well as
recognizing patterns easily.
The data gathered through observation of learners' patterns of behavior and
identified traits and propensities of learners demonstrated by individual
learners during the learning activities, were classified under trends using Davis,
Christodoulou, Seider & Gardner (2011) descriptions of visual-spatial learning
styles, verbal-linguistic learning style, interpersonal and body kinesthetic
learning styles in the multiple intelligences theory.
The summary of data on the bar graph indicates the frequency of learners in the
sample of 60 learners from three schools under each of the three learning styles,
respectively.

339
frequency distribution of learners learning styles (n=60)
60%
50%
40%
30%
20%
10%
0%
bodily kenasthetic verbal linguistic visual spatial learning
learning style learning style style
Figure 2: The bar chart shows the frequency distribution of behavioral patterns
indicating learning styles identified from the combined group of grade four learners
in the sample.
Data gathered concerning the expected performance stated in the performed
goals or standards of the learning outcome.
The data gathered through analytic rubrics comprised a rating of the
performance of individual learners against performance objectives (criteria and
the learning outcomes). The objectives or criteria set the expected achievement of
learners in the mastery of subject content knowledge, cognitive skills, and
practical or applications.
The summary of raw scores obtained by learners under each category was
classified in table 1 below. The data recorded during the learning and
assessments were classified under each type of event. Individual Learners’
learning styles were identified from the scores they obtained from each activity.
The Performance was analyzed to understanding the class composition in terms
of learning techniques
Table 1: shows the analysis of the feedback to the performance of diverse learning
styles on each criterion, respectively.
Number of Analytic assessment rubrics with descriptors

entries 60 Achieved (A): (abilities to understand the concepts of groupings and
(N=60) multiples in the context of time/s (first time, the second time, etc.), are
demonstrated. The skill of identifying clusters counting, and
calculating accurately are performed. Abilities two substitute words
with numerical symbols in performing calculations following
instructions were displayed.
Partial achieved (PA): abilities to understand concepts of clusters and

multiples have partially improved; however, more practice is required.
The skills of identifying groupings, counting, and calculation are still
essential, and more training is essential. Guided reading to
comprehend instructions is necessary. The abilities to develop a table
of 2 and 4 have been mastered.
Not Achieved (NA): (abilities to understand the concepts of grouping

and multiples

340
concept of ‘multiply’ skills of counting and substitution

calculations with symbols
Verbal- achieved
achieved partial achieved
linguistic &
interpersonal
Bodily- achieved partial achieved achieved
Kinesthetic
Visual-spatial
achieved achieved achieved
Data Gathered from In-depth Interviews with three Teachers of 25 learners.

The following three questions were asked to three teachers participating in the
sample. The first question asked: What can be the reasons for these learners’ poor
performance in these multiplication activities?
This question was asked during the analysis of three activities carried out by
learners in the learner workbook. This is what teachers had to say:
Teacher A: “Mathematics is a difficult subject to those learners who are less
gifted, as a teacher; I cannot create something that is not there.”
Teacher B: “learners who experience difficulties are, those who are lazy to think
and to reason even when you give them simple math problems to solve.”
Teacher C: “CAPS learner books and teachers’ guides provide activities that
learners should do in the first term; therefore we steak to them.” The second
question was asked: Was your participation in baseline assessment activities of value
to you as the mathematics teacher in grade four?
This question was asked to teachers after the activities.

Teacher A: “Yes, it was. It has been my view that not all learners are intelligent
to grasp the knowledge of Mathematics, as the teacher of Mathematics in this
over the years, some learners take time to differentiate between addition and
multiplication.”
Teacher B: "I am a new teacher, and I am teaching grade four for the first time. I
learned from inclusive education that we should attend to individual learners
learning barriers. I, also know about the theory of Gardiner. The problem,
however, is that I did not have practical experience. This research has trained me
in something of value. The challenge though, that our curriculum is restricted to
learners’ mathematics workbooks.”
Teacher C: "I think, participating in this research was a golden opportunity for
me. From this experience, I know that learners are judged unfairly by us,
teachers. Honestly, I have not bothered about my slow learners. I will use this
experience to continue to support my learners in the class.”The third and the last
question asked: Do you think are the advantages of considering different learning
styles in the teaching and learning of Mathematics in grade 4?
The responses which we got from the teachers were the following: Teacher A:
“The excitement and joy I witnessed in learners indicated to me that they loved
what they were doing. I have also noticed that their participation and
commitment to the activities improved”
Teacher B: “I have learned that over the years, I have been unfair in my
judgment of learners, because due to my lack of knowledge about their learning

341
styles, I concluded they have learning barriers and, there is no miracle to change
those barriers. The heads of the department do not have a solution, and we
subsequently promote them to the next grade with their barriers.”
Teacher c: The mathematics learner workbooks do not provide activities based
on these different learning styles. The assessment activities should not be the
event of one size fits all.” This thing of barriers and inclusivity and learner
should be left behind; I consider this as just a slogan of the DBE. But what we
have experienced worth it.
7. Findings
Results of baseline assessment expose diverse learning styles other than
learning difficulties.
The traits and propensities presented in the bar chart, figure2, and andtable1
indicate a change in the learners ‘performance, to the four criteria of the
multiplication activities, respectively. The activities exposed learners to
opportunities to master knowledge and skills of multiplication at their own pace
and learning styles. The results highlighted the population of learners in the
sample of 60 represented the demographics of the diverse learning styles in the
bar chart, figure 2. The performance which could have been declared weak in
the four aspects of multiplication appeared to have improved drastically.
Improvements in learners’ performance highlight the importance and usefulness
of baseline assessment in identifying the learning styles of learners in
mathematics class. The analysis of the learners’ performance provided informed
the development of activities that accommodated diverse capabilities, interests,
and frameworks for learning mathematics skills, knowledge, and logos.
The positive results obtained from a variety of activities confirmed assertions
made by Kim (2019) and Tiymms (2013) that baseline assessment tasks provide
adequate opportunities for learners to express their individuality in the learning
process.
Planning and development of diverse activities offer learners opportunities to

master multiplication skills at their level of cognitive processing.
Learners who met all four criteria were considered to have mastered
multiplication set out in the learning outcome. This finding confirms Piaget’s
assertion that mastery of skills begins with the coding of concepts in the short
term memory before proceeding to the working memory in the mind where
meaning and understanding are accomplished Bondie et al. (2019). Such positive
results endorse the principles of multi-level abilities teaching and multiple
intelligences that learning of any knowledge and skills is determined by;
flexibility of organization of material and familiarity with context (Tomlinson,
2020; Baubeng, Atingane & Amoako, 2019).
This finding suggests strongly that the results of the baseline assessment are the
performance indicators (PI) for differentiated learning (Tymms, 2013)
Findings were elucidated from data gathered from in-depth interviews with
participating teachers.
The findings that emerged during the process of data analysis and its collection
highlighted various issues about the learners and teachers participated in the
sample. These issues were translated into themes for discussion.

342
Theme 1: Inadequate theoretical and practical knowledge of baseline assessment

The responses of the participants to the main question highlighted that their
knowledge of baseline assessment did not resonate with the principles of
inclusivity and differentiated learning advocated in the White Paper 6 (DBE,
2011). The responses of teacher, A, and C to the question on the reasons for the
high rate of failure in mathematics in the intermediate phase (middle classes)
highlighted the lack of knowledge of the varying capabilities in the learning
environment. This finding, therefore, answered the question asked in this article
on how teachers in elementary mathematics classes identify differentiated
learners’ learning styles in mastering conceptual knowledge of multiplication
and the application of numerical cognitive skills. The repeated mentioning of the
Activities from textbooks as a means of baseline assessment indicated that
participants were not certain about the rationale for conducting baseline
assessment as part of implementing inclusive policy in teaching and learning.
Theme 2: Misconstrued understanding of differentiated learning styles versus

learning difficulties
Analysis of responses highlighted that participants held the perception that
differentiated learning styles refer to learning difficulties. The misconception of
differentiated learning styles led other learners to state that they failed because
they could not fit into the mainstream teaching and learning styles preferred by
the teacher in the mathematics class; something highlighted in the utterance
made by one participant: “learners who experience difficulties are those who are
lazy to think and to reason even when you give them simple math problems to
solve.”
Theme 3: dependence on textbooks and curriculum guidelines for assessment

activities.
The participants highlighted reliance on the prescribed learners’ mathematics
books and curriculum guidelines when conducting assessments. Textbook based
activities in this argument are generic and are not suitable for implementing
inclusivity in the mathematics classroom. The findings of this study confirm that
teachers’ creativity and flexibility could bring about fruitful intervention to
promote equity and equal learning opportunities for all learners in the learning
environment. The tendency of the Department of education to prescribe
assessment tools for all learners in the grade is proven in this study to be an
obstacle to teachers’ critical and creative thinking about what their contextual
reality demands. The study also, proved that teachers are capable of developing
activities that suit their learners' experience and interests, and capabilities as
well.
8. Discussion
This section discusses the synthesis of the findings of the research concerning the
question, which was the main focus of the collection and analysis of data. The
question sought to Is baseline assessment a reliable procedure to benchmark
diverse learning needs in the elementary mathematics classes?

343
The findings presented in this paper highlighted the possibility of improving the
performance of learners in mathematics through careful planning and
organizing activities driven by the abilities, needs, and interests of the learners.
The findings of the research proved that baseline assessment activities provided
diverse opportunities for learners to demonstrate their capabilities and interests.
The activities were able to benchmark the interests, needs, and abilities which
can guide further development of learning activities for the learners. According
to Tomlinson (2020) baseline in the form of the formative assessment provide
teachers feedback about the diverse needs and abilities to be considered when
designing teaching and learning activities in the subject. The results of the study
highlighted that, despite the argument of Henning et al. (2019 regarding the
negative impact on the learning of mathematical content knowledge, the issue of
pedagogical approach proved by this research to be the obstacle to learners with
special learning techniques to demonstrate their capabilities.
The success in the improvement of underperforming learners scored through the

baseline assessment activities confirmed the principle of a continuum in a
performance highlighted by Armstrong (2009 p.146) that; flexibility in the
organization of learning and assessment material and familiarity with the
context are critical principles in differentiated learning styles. The evidence in
Tables 1 and figure 2 proved that active involvement with objects engages
learners in working with mathematical logos made learning of multiple of four
practical and contextualized. Learners reckoned that logos are the mathematical
language to communicate knowledge in different contexts. This finding was the
realization of the idea that productive and meaningful learning of concepts and
experience is possible in a framework that is familiar to learners.
The results and the findings presented in this paper highlight the value of
baseline assessment to benchmark learning needs which prevail in mathematics
class. The results of the baselines assessment were of assistant to teachers to
develop mathematics teaching and learning activities in elementary classes.
Teachers participating in the sample were eager to cooperate in learning about
strategies of baseline assessment to identify distinct learning styles. The inability
of mathematics teachers to develop baseline assessment activities was
interpreted as a threat to inclusivity in learning, the result of which is the
disadvantaging of other learners. The proposals and recommendations made on
the grounds of this finding are two-fold: empowering mathematics teachers with
strategies (i) to develop baseline activities and (ii) to analyze results according to
learning styles.
9. Limitations
Even though, the fact that the findings shared in this article are of value and
importance it is worth noting, however, that the sampling focused on three
schools in a district with over 50 primary schools. The researcher could not
access other schools due to time and financial constraints.

344
10. Conclusion and recommendation

The analysis of the principles underpinning baseline assessment and the
findings of this study, lead to the following conclusions about the process of
benchmarking learners' approaches to acquiring mathematical knowledge. First,
the principles of the theory of Multiple Intelligence which guided assisted this
study to understand the implications of baseline assessment in practice. The
principle of uniqueness of learners in the process of learning suggested by the
advocates of differentiated learning techniques was confirmed by the results and
the findings in the work (Dryden & Vos, 2005; Bondie, Dahnke & Zusho, 2019).
Second, in addition to focusing on the reliability of the results, was that
performance proved that teachers should reckon; flexibility in the organization
of learning material contextualized learning of mathematics concepts, skills, and
operation abilities, particularly for the intermediate phase (middle phase of
school. Third, upon reflection, baselines activities appeared to be developing
interest and positive attitude in learners toward mathematics. While
acknowledging, the concerns of the rapid and radical curriculum change since
1997, and a lack of resources, and parental support highlighted by the
Department of Basic Education (2019), Spaull & Kotze (2015), the findings of this
research, however, proved that consideration of differentiated learning
techniques is lacking in mathematics teaching and learning, in some primary
schools.
This study recommends that future studies could use the findings highlighted
by this research to study, the application of baseline assessment to identify
teaching techniques in mathematics classes on a larger scale and highlight other
learning styles that are not part of this article. The study recommends the serious
consideration of baseline assessment to be an essential aspect of a continuous
professional program for mathematics teachers. Furthermore, the research
highlighted that learners can reach their optimal levels in learning mathematics
in a free and flexible environment. This paper concludes that the lack of
knowledge about how to conduct baseline and its value is one of the serious
attributes of the high failure rate of children in grade four. These findings
highlighted in the article suggest that the department of education in South
Africa should provide the teacher of mathematics in elementary classes with
pedagogical content knowledge and training to handle issues of differentiated
learning in mathematics class; this does not mean remedial teaching but an
ability to develop activities to accommodate all styles of learning.
According to the findings of the research, the claim can be made that
pedagogical strategies proposed for integrating teaching, learning, and
assessment have not found their way into the practices of teachers in classrooms.
Teachers in this study were seldom able to develop and plan learning and
teaching activities sensitive to learners’ diverse learning styles. The positive
results indicate that baseline assessment could be of assistance to mathematics
teachers to establish diverse learning styles in their classrooms; so that they can
widen opportunities in their teaching of mathematics conceptual knowledge and
algorithms. It was proven in the study that the principle of inclusivity in
education promulgated by the Department of Basic Education (2011) in the

345
White paper 6 regulations on inclusive education are not considered in the

planning and organization of learning material, and subsequently, it is not
implemented due to the lack of empowerment of mathematics teacher.
The first activity provided learners with the different settings to make their own
choice and to count one cluster of objects in cycles and record the cycles of
counting and the increase they observed in the rise in numbers.
This paper recognizes the demographics of diverse learning styles in

mathematics classes, and the question was how the demographics contributed to
the poor performance of learners in grade four in mathematics learning. The
findings of the study indicated that learners’, performance was focused on
sound, weak and poor categories. The results presented in this work point to the
inadequacies of mathematics teachers in the entry grade of intermediate phase to
plan, and organize learning, and assessment activities to accommodate diverse
learners' needs in mathematics classes.
The data and the findings presented in this article point out that a new and
viable approach to teaching and learning of Mathematics knowledge and skills
is necessary.
11. References
Aghamdi, A., Jitendra, A. K., & Lein, A. E. (2019). Teaching students with mathematics
disabilities to solve multiplication and division word problems: the role of
schema-based instruction. Mathematics Education, 52(1), 125-137.
https://doi.org/10.1007/s11858-019-01078-0
Amoako, I., Brembong, D. A., & Bortey, J. (2019). Knowledge of Formative Assessment
practices among senior High School Mathematics Teachers in Ghana. Open
Journal of Social Science Research, 3(3), 8-13.
Andrade, H. L., & Brookhart, S. M. (2019). Classroom assessment as the co-regulation of
learning. Assessment in Education: Principles, Policy & Practice, 27(4), 350-372.
https://doi.org/10.1080/0969594X.2019.1571992
Baubeng, I. Atingane, A. B., & Amoako. (2019). Practices, challenges, and Perceived
Influence, of Classroom Assessment on Mathematics Instruction. International
Journal of Assessment Tools in Education, 6(3), 476-486.
Blomeke, S., Kaiser, G., & Koning, J. (2020). Profiles of mathematics teachers’ competence
and their relation to instructional quality. Mathematics Education, 52(2), 329-342.
https://doi.org/10.1007/s11858-020-01128-y
Bondie, R, S., Dahnke, C., & Zusho, A. (2019). How does changing "One Size Fits All" to
differentiated Instruction Affect Teaching? Review of Research in Education (chapter
12, 43(1), pp. 336-362). SAGE Publication.
Davis, K., Christodoulou J. Seider, S., & Gardiner H. (2011) The Theory of Multiple
Intelligences. In R. J. Sternberg & S. B. Kaufman (Eds.) Cambridge Handbook of
Intelligence. Cambridge University Press: Cambridge.
Department of Basic Education. (2019). Mathematics Teaching and Learning Framework for
South Africa: Teaching and Learning mathematics for understanding. Department
Printers: Pretoria RSA.
Department of Higher Education. (2019). Minimum Requirement for Teacher Qualification in
South Africa. Pretoria. Government Press.
Department of Basic Education. (2011) South African curriculum and assessment policy
statements, Department of Education, Pretoria.

346
Donohue, D. K., & Bornman, J. (2014). The challenges of realizing Inclusive Education in
South Africa, South African Journal of Education, 34(2),1-14.
Dryden, G., &Vos, J. (2005) (Eds.). New learning Revolution. United Kingdom: Network
Educational Press Ltd.
Fleisch, B. (2008). Primary education in crisis: Why South African schoolchildren underachieve
in reading and mathematics. Juta, Cape Town.
Goulet-Lyle, M., Voyer, D., & Wershaffel, L. (2019). How does imposing a step-by-step
solution method impact students' approach to mathematics word problem-
solving? Mathematics Education, 52(1), 139-149. https://doi.org/10.1007/s11858-
019-01098-w
Hazell, E., Spencer, G., & Robert, N. (2019). Improving Grade R mathematics teaching in
South Africa: Evidence from an impact evaluation of a province-wide
intervention.
Henning, E., Ehlert, A., Balzer, L., Ragpot, L., Herholdt, R., & Fritz, A. (2019) Marko-DSA:
Assessment of number concept development. University of Johannesburg printers:
Johannesburg RSA.
Henning, E., van Rensburg, W., & Smit, B. (2013). Finding your way in qualitative research.
Pretoria: Van Schaik
Kim, H. J. (2019). Teacher learning opportunities provided by implementing formative
assessment lessons: Becoming responsive to student mathematical
thinking. International Journal of Science and Mathematics Education, 17, 341–361.
Mullis, I. V. S., Martin, M. O., & Hooper, M. (2017). Measuring changing educational
context in a changing world: Evolution of TIMMS and PIRLS questionnaire. In
M. Rosen, K. Y. Hansen & U. Woff (Eds). Cognitive abilities and educational
outcomes.
Nguare, M. W., Hungi, N., & Mutisya, M. (2018). Assessing Learning: How can
classroom-based teachers assess students’ competences in Numeracy. Journal of
Assessment in Education, 26(2), 222-244.
https://doi.org/10.1080/0969594X.2018.1503156
Reddy, V. B. (2006). Mathematics and science achievement at South African schools in TIMMS
2003. Cape Town.
Schollar, E. (2008). The primary mathematics research project 2004–2007: Towards evidence-
based educational development in South Africa. Primary Mathematics Project,
Johannesburg.
Semana, S., & Santos, L. (2018) Self-regulation capacity of middle school students in
mathematics. Mathematics Education, 50(6), 743-755.
https://doi.org/10.1007/s11858-018-0954-0
Spaull, N., & Kotze, J. (2015). ‘Starting behind and staying behind in South Africa
International Journal of Educational Development, 41, 13–24.
Tomlinson, C. A. (2020). Fulfilling the promise of the differentiated classroom: Strategies and
tools for responsive teaching. Alexandria. Retrieved from
http://www.ascd.org/publications
Tomlinson, C. A. & Moon, T. (2013). Differentiation and classroom assessment. In J. H.
McMillan (Ed.), SAGE handbook of research on classroom assessment (pp. 415–430).
Los Angeles: SAGE.
Tymms, P. (2013). Baseline Assessment and Monitoring in Primary schools: Achievement,
Attitudes, and Value-added indicators. New York: Routledge.
Veldhuis, M., & van den Heuvel-Panhuizen, M. (2019). Supporting primary school
teachers’ classroom assessment in mathematics education: effects on student
achievement. Mathematics Education Research Journal, 32, 449–471.
https://doi.org/10.1007/s13394-019-00281-2

347
Venkat, H., & Spaull, N. (2015). What do we know about primary teachers' mathematical
content knowledge in South Africa? An analysis of SACMEQ 2007. International
Journal of Educational Development, 41, 121-130.
https://doi.org/10.1016/j.ijedudev.2015.02.002
Wright, R. J. Martland, J., & Stafford, A. K. (2006). Early numeracy: Assessment for teaching
and intervention. London: Paul Chapman Publishing Ltd.

348
Appendix A: Activities of multiplication problems for baseline assessment
Activity One: multiples of two
⚫ Problem is counting objects in pairs or in the groupings of two, in a

number of times and recording solutions or number patterns.
⚫ Example:
Counting a pair
Once (1x2 objects)
Counting a pair twice (2 times x2 objects)
Activity Two: Multiples of four
⚫ Problem is counting objects in the groupings of four
Intelligences Techniques of solving skills knowledge

problems. Material chosen
Verbal Linguistic Picture/ diagrams / Counting Number

Reading words and words Sorting patterns
listening Organising Concepts of
recognising increase and
Bodily Kinaesthetic Games, role playing multiplication
Move around,
process information
through bodily
sensations
Visual spatial
Imagine things, Pictures/ drawings
working with visual
images and pictures
Appendix B: Observation schedule
Pseudonyms Interests Technique for solving Performance

for students mathematics problems
(e.g. games, drawing)
Learner R
Learner V
Learner M
Learner P

349
Appendix C: in-depth interview questions
(i) How do you assist learners who are not performing well in your
mathematics class?
(ii) What can be the reasons for these learners’ poor performance in these
multiplication activities
(iii) Was your participation in baseline assessment activities of value to you as
the mathematics teacher in grade four?
(iv) Do you think are the advantages of considering different learning styles in
the teaching and learning of Mathematics in grade 4?

350

Vol. 19, No. 8, pp. 350-370, August 2020
Authentic Videos in Teaching English to

Engineering Students at Universities
Nataliia Saienko and Mariana Shevchenko

National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic
Institute”, Kyiv, Ukraine
https://orcid.org/0000-0001-8898-5198
https://orcid.org/0000-0002-5881-0263
Abstract. The article provides the study of authentic videos effectiveness

in teaching English to future engineers, focusing on listening and
speaking skills advancement. Being created primarily for native
speakers, authentic videos (movies, TV shows, and programs) present
real-life issues and communication examples. Therefore, such records
may become indispensable for engineering students, who often have
limited English vocabulary. The 6-week research was conducted at the
National Technical University of Ukraine “Igor Sikorsky Kyiv
Polytechnic Institute” in February-March 2020. The sample was 50 full-
time 3rd-year engineering students. During the research, the
participants received surveys and tests. The obtained post-experimental
test results revealed that the introduction of authentic videos in the
English course regularly helps improve the technical students’ listening
and speaking skills by almost two times. The average total mark,
comprising the outcomes of listening, reading, writing, and speaking
tasks, increased by 1.5 times. In the post-experimental questionnaires,
the students stated that authentic videos advanced their English
proficiency significantly and improved their understanding of the
studied topics, including specialty-related vocabulary and issues. The
vast majority of engineers-to-be expressed the desire to continue the
work with videos at home. The study allows us to conclude that
authentic videos are useful in English teaching since they help boost
engineering students’ skills, motivation, positive characteristics
(concurrently amending their weak points), knowledge in the field of
specialty, and increase their interest in studying English. The research
results may be employed for further studies concerning English
teaching with videos at university and the subsequent lifelong learning
of English.
Keywords: Authentic video; Teaching; English; Engineering student;

University

351
1. Introduction
Present times require up-to-date forms of study at university, especially a
technical one. Students need a unique approach to get modern education and
relevant expertise that will be flexible and acceptable in the constantly-
developing scientific field in the time of the ever-changing world. Besides, future
professionals should acquire lifelong learning skills as they are among the most
needed ones now for career advancement. Teaching English requires keeping up
with current trends. Recently, the ongoing shift towards a mainly-
communicative approach to teaching English at non-linguistic faculties has
become visible. Therefore, it is important to analyze personality types and traits
of technical students and find suitable means and aids to boost their incentive to
improve their skills and knowledge of English at university and after
graduation.
There is a current trend towards adding audio-visual materials in the classroom
of both humanities and technical students to boost the teaching/studying
process (Malhotra & Verma, 2020; Muslem et al., 2017). Among others,
engineering students express a positive attitude to the use of videos in class
(Babu H et al., 2019; Månsson, Löfgreen & Warfvinge, 2017; Sun, 2020). During
the training of future engineers, videos are mainly used for demonstration of a
device, a technological process, or an issue and not for teaching English (Zabalza
et al., 2020; Zhou, 2005). However, relevant authentic videos used in English
lessons as educational aids can also potentially improve the levels of foreign
language proficiency of the students in question and make such classes more
effective. Yet, this aspect has not been sufficiently studied, making our research
opportune.
Scientists from different countries across the globe are investigating the positive
and negative sides of videos as a supplement to foreign language teaching. The
role of video records in teaching English oral speech at university has been
covered widely in scientific works. The findings are provided further. Polat and
Erişti (2019) examined the impact of authentic video aids on foreign language
listening skill development. According to the scholars, authentic videos
demonstrate the ‘real’ language (spoken by native speakers of English) and give
communication samples, and thus have a highly effective influence on the
development of English listening skills. Also, they relieve the foreign language
listening anxiety of the students who have low levels of English proficiency.
Ismaili (2013) explored the beneficial nature of films for the work of students in
the EFL class. The results of the investigation showed a direct connection
between motion pictures and the boost of students’ listening comprehension and
communication skills development. Kalra (2017) described the same outcomes of
the study. It was determined that films in English are motivating aids that
present the speech in a close-to-authentic way, as it is done by the native
speakers; thus, they significantly contribute to the enhancement of students’
listening and speaking comprehension. Wang (2015) believes that authentic
videos stimulate students’ proactivity in English lessons and the incentive to
study the foreign language, focus their attention on the educational means, and

352
depict realistic circumstances and foreign cultures. In the researcher’s scientific

work, movies were mentioned as handy learning materials. However, it was
recommended that teachers should design many varieties of classroom activities
to take full advantage of videos. Hadijah (2016) expressed a similar thought
about the importance of abundant techniques application in teaching English
with multimedia. That means that videos should be used in diverse, compelling,
and meaningful ways to be considered advantageous educational aids in the
English classroom.
The employment and usefulness of authentic videos in English lessons may

depend on the learners’ personality traits and skills. Tordai and Holik (2018)
have analyzed the engineering sector and pointed out that engineering
informatics students display a low level of social skills and activity, implying
that most of them are introverted and inactive. This may complicate the foreign
language study at university. It has been recommended to augment the
mentioned skills by oral interaction and presentations. However, no possible
ways of advancing their communication and collaboration skills using videos
were specified.
The foregoing literature review allows us to conclude that the subject matters of
audio-visual materials use in foreign language lessons of humanities students
and the communicative approach to teaching have been examined by various
researchers, creating prerequisites for further study. Yet, there is a pressing need
for an in-depth analysis of authentic video clips efficacy in teaching English to
students of technical faculties. Among such are engineering students. In their
lessons at university, videos are mainly used for specialty-related purposes, not
for teaching English. However, authentic videos used in English classes may
provide the opportunity to teach general and specialized English vocabulary,
scientific concepts, issues, slang, and idioms that appear and are updated almost
every day, being in permanent use by native speakers. The examination of the
authentic videos use effectiveness for English studying (at university and after
graduation) of engineers-to-be should be conducted together with the study of
engineering students’ characteristics, and their future employers’ requirements
for them.
3. Engineering Specialists’ Place in the Modern World

Before the examination of the role and efficiency of authentic videos in English
lessons of technical students, it is crucial to identify the personal characteristics
of such individuals. The present article deals with engineering students since
their specialty is one of the most in-demand majors at present and is regarded as
a top career. Among other things, modern engineers pay special attention to the
development of energy-saving technologies and the creation of
environmentally-friendly devices and manufacturing processes. Due to the
specified line of work, engineering skills are viewed as ‘profitable’ or ‘valuable’
worldwide now (Freitas et al., 2012, p. 2). It often triggers off ‘brain drain’
(highly trained or qualified individuals emigration), which is a big issue for
every country. Each state wishes to retain their brilliant minds to be a worthy
competitor in the field of technology and scientific innovation. Skilled migration

353
of engineers may provoke deterioration or even collapse of specific technology

sectors of a country. Consequently, a decline in the country’s scientific value,
socio-economic development, as well as economic and financial losses may
happen. The outflow of engineers can occur for various reasons: low wages,
underestimation by management, search for better prospects, superior training
and working conditions, etc.
At present, the issue of education and preventing the migration of specialists in

industries that are rapidly gaining value in the international arena, such as
environmental engineering, is urgent. Such specialists are engaged in preserving
the environment and developing new technologies to improve people’s lives
and even save them. Now, almost every country has a plan for the development
and large-scale implementation of ‘green’ technologies, based on renewable
sources (solar-, wind-, wave energy, biomass, etc.). It is done by engineers in
various fields. Therefore, it is essential to update the university education of
engineering students that are non-native speakers of English per the needs of
today, showing the real possibilities of applying the acquired knowledge,
including English. Often engineering students view it as the possible way to
immigrate to other countries that offer them more career prospects. However,
teaching English with modern means can help them find many opportunities
without a radical change in their lives. It can help gain new expertise,
knowledge, and practice even in the students’ homeland, opening the world to
them from a different perspective, including the world of science. Moreover,
adequately studied English – utilizing modern audio-visual materials currently
available in abundance – helps engineers-to-be gain access to the global scientific
community of innovators without effort, promoting mobility, i.e., the
‘circulation’ of specialists within a global labor market (Freitas et al., 2012), and
lifelong learning of English.
4. Relation between Engineering Students’ Characteristics and

Authentic Videos Effectiveness in English Lessons
The new tendencies have a significant impact on student-engineers’
personalities and cognitive characteristics. Among them, researchers (Kline &
Lapham, 1992; Tordai & Holik, 2018; Van Der Molen et al., 2007) uniformly
determine such most distinctive ones as capability of independent (autonomy)
and dependent work, emotional stability (anxiety control), tough-mindedness,
conscientiousness (i.e., showing accuracy and precision in activities),
responsibility and reliability, moderate cooperation, as well as a low level of
such social skills as effective communication and self-expression.
Engineering students, who have work experience, have defined skills vital for
their future profession. The highest ratings have problem-solving, causal
relationships understanding, ability to apply knowledge, work precision,
adaptation to change, goal orientation, and flexibility (Tordai & Holik, 2018,
pp. 35–37). Graduates emphasized that communication skills, problem-solving,
and interpersonal skills (cooperation) are among the most valued generic
competencies for employability (Markes, 2006, p. 18; Tordai & Holik, 2018,
p. 37). The students viewed this expertise as more critical in the engineering

354
profession than hard skills. At the same time, the vast majority of respondents
said that usually the least important skill for their success in the professional
field is written communication.
Skills and Competencies of Engineering Students to Be Advanced

There is no doubt that students’ abilities, opinions, and needs are essential for
differentiated English teaching at university. However, that is the employers
who dictate what is needed in engineering branches at the moment. Thus, they
indirectly determine the competencies and skills to be promoted in the course of
English teaching. The following are skills and competencies of engineering
students that can be developed and advanced simultaneously with four basic
language skills in the course of English teaching with authentic videos at
university.
A thorough analysis revealed that employers in the various branches of

engineering usually require their employees to comply with standards and have
the following competencies (Eldeen et al., 2018, p. 965; Yusoff et al., 2012): inter-
personal skills; intra-personal skills; basic (computer) skills; specific
(professional) knowledge.
Considering the ranking of the employability skills currently required by

worldwide engineering industries, university lecturers of English can prioritize
the student skills that should be improved in the course of the foreign language
study, as well as choose the most suitable means of teaching.
Understanding and being able to speak English and other languages are among
the top benchmarks of a modern specialist in any technical field. That defines
the vital role of English for engineers. The key criteria that employers in
engineering branches utilize for worldwide staff selection were gathered (Yusoff
et al., 2012, pp. 498–499), categorized, and are provided further. It is possible to
develop and advance the majority of them with the introduction of authentic
videos in English language lessons of engineering students on an ongoing basis:
communication skills; teamwork; knowledge of contemporary issues;
professionalism; problem-solving and decision-making skills; competency
(making use of modern engineering devices and software, continuous
acquisition of in-depth technical competence in a particular engineering branch);
lifelong learning (recognition of the importance of engagement in lifelong
learning, and setting personal goals).
The investigation results determine the course of English language teaching at a

technical university. It is clear that to train successful and versatile engineering
specialists, it is essential to incorporate the development of non-technical
competencies and soft skills across curricula (Itani & Srour, 2016; Riemer, 2003)
and motivate students for lifelong learning, using modern aids. Non-technical
competencies and soft skills are the interpersonal attributes you need to succeed
in the workplace: communication abilities (listening, presentation, public
speaking, storytelling, persuasion, verbal communication, reading body
language), critical thinking (creativity, flexibility, curiosity, logical thinking,

355
problem-solving, research, innovation), leadership (giving clear feedback,

delegation, decision-making), positive attitude (cooperation, confidence,
patience), and teamwork.
Lifelong Learning of Engineering Students

It is important to remember that an individual’s learning usually happens not
solely in a formal ambience (university classroom). It occurs at every moment of
their life, including non-formal (professional associations) and informal (library,
museum, cinema, family, etc.) settings. As regards lifelong learning, it is a
continuous process, among key features of which are diversity, flexibility, and
availability at any time and place. Learning opportunities can be found
everywhere. The mentioned kind of learning emboldens people to acquire all the
knowledge and skills they may require throughout their lives (both personal
and professional) and apply them with creativity, assurance, and satisfaction in
any circumstances and environments (Djebbari & Djebbari, 2018, p. 9). In the
process of continuous learning, people should regularly update their
competencies, not allowing their knowledge and skills to become obsolete. For
instance, via utilizing mobile-assisted technologies that can be integrated into an
English course at university and used by students anywhere off-campus.
Technologies were proven to be a powerful tool to develop self-directed learning
skills when adequately used (Saienko & Lavrysh, 2020).
To be considered a successful lifelong learner, a person must possess the

following traits (Djebbari & Djebbari, 2018, p. 10): be interdependent and
independent (autonomous), logical and analytical, methodical and disciplined,
curious, motivated to learn and grow, responsible for their learning and its
evaluation, able to think critically and respond to various situations, as well as
willing to collaborate with others. Well-developed communication skills, higher-
order thinking skills, and information-seeking and retrieval skills are also vital.
That makes lifelong learning perfect for engineering students, taking into
account their characteristics mentioned in the article earlier.
Turning English language classrooms into ones with lifelong learning objectives
may emphasize technical students’ English language proficiency, expand their
experiences through attaching state-of-the-art technology; boost creativity,
initiative, responsiveness, adaptability, innovation, and ability to change. Thus,
to prepare engineering specialists with high skills in English, the lecturers need
to not only provide the students with theoretical and practical knowledge of the
foreign language but also arouse their interest in the studied subject. Besides,
English lecturers should inspire technical students for continuous learning: at
first – off-campus, and after graduation – as lifelong learning. That can be done
only via the employment of modern studying aids and demonstration of their
efficiency and usage variety.
Authentic Videos in English Lessons of Engineering Students

To encourage engineering students to be proactive while doing activities aimed
to develop or improve the aforementioned soft skills, it is better to use authentic
videos most diversely. Such videos promote active study in English lessons and
represent one of the main elements of student-centered learning environments.

356
That is because such a type of teaching usually allows students to get authentic
tasks and gain relevant learning experiences (Elen et al., 2007, p. 105).
By the word ‘authentic,’ a non-educational character of any tool is meant.

Authentic aids are created, in the first place, for native speakers primarily by
individuals speaking a language as their mother tongue. Verbal, audio-visual,
and written elements (e.g., subtitles) of authentic videos reflect the real-life
communication examples (Polat & Erişti, 2019, p. 137). That is the reason why
the rate of speech and vocabulary heard on them are never artificially reduced to
be more comprehensible (Shahani et al., 2014, p. 44), unlike on the records
created purposefully for language teaching (for non-native speakers). The
examples of such materials are present in abundance in any format everywhere
now – movies (and their fragments), TV shows, programs, etc. on the Internet
platforms (YouTube, Netflix, Hulu, Amazon Prime), DVD/Blu-Ray disks, in the
form of digital releases. Foreign language lecturers often discard authentic
videos in English. It happens since they subjectively regard such records as too
time-consuming and difficult-to-understand for their technical students, who
presumably have a limited volume of general English. The mentioned foreign
language vocabulary may be limited in terms of shortenings (e.g., such as ‘ad’
for advertisement, ‘vid’ for video), slang, idioms, and, especially, abbreviations
and acronyms used by English-speaking countries in written and oral
communication sometimes even more often than their full forms (e.g., ASAP – as
soon as possible, CEO – Chief Executive Officer). However, they are just as
needed as the standard dictionary. If they are commonly used by native
speakers of English, they are most definitely utilized in the workplaces, and so,
by the potential foreign colleagues of the now-students.
To meet the requirements of the employers while simultaneously taking into

account the personality characteristics of engineering students, an English
language lecturer needs to set three aims of teaching with authentic videos
(Wang, 2015, pp. 23–24). Firstly, it is necessary to facilitate the development of
the students’ language skills (primarily, listening and speaking). It is a fact that
students get tired much quicker of strictly listening than when the audio is
accompanied by some visual. Videos can enhance their attention focus on the
audio material, providing much more information through the visual and
auditory receptors concurrently. Besides, authentic video aids can create a more
realistic language learning environment and stimulate students’ interest in
English studying and improvement of their comprehensive linguistic
competence. Since the development of technical students’ language abilities is
generally connected with their incentive and interest in what they are studying,
it is motivation, namely instrumental and integrative, that plays a vital role in
the foreign language training of future engineers (Saienko, 2017). It is confirmed
by the cognitive theory of motivation, according to which our expectations
guide our behaviors, usually, in ways that would lead to desirable outcomes.
Secondly, the cultivation of the students’ intercultural communication
competence is essential. When an authentic video is shown in the classroom, it
does not only present information about the language but also demonstrates the
culture of English native speakers and ultimately serves as a support for

357
students’ communicative competence in English. And finally, videos in question

have to develop students’ aesthetic values and ability to appreciate English
artistic videos (including movie fragments) in their foreign language lessons.
Technical students should understand that the not-inherently-technical character
of a film or show does not downplay its significance and does not mean pure
entertainment in the class. Authentic videos exist in multiple genres, and various
engineering subject matters can be found frequently in virtually any of them,
even in Hollywood movies. Clips selected correctly (meaning those that depict
an aspect of the students’ field of specialization – an issue, a process, a device,
etc.) can expedite engineering students’ deep thinking, critical review, and
technical discussions.
Of all the mentioned characteristics of engineering students and the

requirements of employers in their professional sphere, the main emphasis in
the majority of examined scientific works (Shevchenko, 2015a; Reddy, 2019;
Riemer, 2003; Van Der Molen et al., 2007) is on the importance of paying more
attention to the development and enhancement of communication abilities
(interpersonal skills) of students. The reason for that is that engineers, being
typical realistic individuals, prefer to work with things rather than ideas or
people and enjoy physical labor more than mental activities, and so, have a
lower level of social skills than humanities students. It highlights the
significance of technical students’ English oral speech advancement at
university.
Communication skills improvement can be put into action through problem-

based learning, case study, student presentations, and projects on the subject
matters of lessons structured around video clips that are thematically-relevant to
students’ field of specialization. Discussions of compelling and relevant
engineering issues seen on video records help to boost students’ oral interaction
using general English vocabulary and specialized terminology. At the same
time, reflection on the lesson’s topic promotes students’ critical thinking and,
ultimately, advances their proficiency in interaction with a degree of fluency and
spontaneity. It is recommended to organize it in the form of cooperative learning
(teamwork and collaboration) to stimulate proactive rather than reactive English
study with feedback and encourage active participation in the foreign language
learning process. It creates the basis of the student-centered approach to
education, which, in turn, motivates technical students and prepares them for
the employment challenges of today.
Authentic videos in English are multifunctional. They can serve the English
teaching process at university in various ways: to improve engineering students’
listening and speaking skills (vocabulary practice), information recall,
prediction, writing, knowledge of speech structure and grammar, teamwork,
and even English pronunciation. In the auditorium, authentic video clips
stimulate discussion (of a wide range of depicted issues/situations/subjects)
with the justification of one’s viewpoint, argument provision and persuasion,
critical thinking, as well as problem-solving. Moreover, the videos under
consideration may also serve as the basis for such subsequent independent work

358
of students as preparation of presentations and clear and logical speeches on the

lesson’s topic or a related one. Assignments based around video records
introduce interest and incentive boost in the English language lessons, showing
the authenticity of specialty-relevant topics/issues, learning material, and the
studied foreign language vocabulary. The information above demonstrates the
productivity of video records for the introduction of the communicative
approach to teaching English: they create the conditions for focusing on
communication in class via showing a real conversation process and promoting
learning-by-doing after the viewing; provide rich and meaningful input in the
form of authentic language discourse, and stimulate collaborative learning
(Shevchenko, 2015a).
The role of authentic videos in teaching English at technical universities has

been examined previously (Shevchenko, 2015b). The current article’s topic is its
logical continuation, studying the effectiveness of the use of authentic videos in
teaching English to a specific group of technical students – future engineers. The
research is done via exploring the characteristics of engineering students, the
requirements for them from employers, and the features of authentic videos
utilization in English lessons at universities to achieve positive results. However,
the matter would not be thorough enough without an experimental verification
of the obtained results.
5. Aim of the Study

This article aims to study and experimentally verify the effectiveness of using
authentic videos in teaching English to engineering students at universities.
Hypothesis
We hypothesize that introduction of authentic videos in the English course
regularly will improve engineering students’ listening and speaking skills.
6. Materials and Methods

Research Design
The study employs a mixed-methods research approach. The quantitative
research method relied on collecting and processing data through the use of pre-
and post-experimental questionnaires and tests (B2 level). The qualitative
research method focused on interpreting the information. The theoretical data
collected and presented in the current work were experimentally verified. The
results are provided further in the article.
Participants and Procedure

For the 6-week experiment in February-March 2020, two groups of 3rd-year
engineering students of the Institute of Energy Saving and Energy Management
(IEE) of the National Technical University of Ukraine “Igor Sikorsky Kyiv
Polytechnic Institute” were selected. Each group had 25 full-time students with
an equal number of male and female persons. The level of proficiency of the
majority of the students chosen for the experiment was B2 (Upper-Intermediate).
Group 1 was experimental, and therefore was studying with authentic videos as
educational aids. In its turn, Group 2 (a control one) was utilizing mainly

359
conventional printed materials – on the same topics as Group 1 – and only

several times they listened to audio recordings when it was needed for the
coursebook exercise doing. Each group was provided with various tasks to
check their comprehension throughout the experimental study.
Before and after the experiment, the engineering students received links to
Google Forms with two questionnaires prepared by us, both with open-ended
and closed-ended questions. Group 1 and Group 2 also received two printed
tests (pre- and post-experimental) – each was the same for both groups. The pre-
experimental test was prepared by us, taking into account the terminology the
students had come across in their coursebook (‘Cambridge English for
Engineering’ by Mark Ibbotson). In the post-experimental test, we used the
information and vocabulary studied during the experiment and the lexis from
the coursebook. The tests contained various tasks for students with B2 level. Our
identification of the students’ English competence was conducted per
Cambridge B2 First (FCE) exam format. That is, our pre- and post-experimental
check of the students’ knowledge of English consisted of Listening, Reading and
Use of English, Writing, and Speaking (interviewing).
The groups’ pre-experimental surveys were identical, whereas the post-

experimental ones were different for each group, aiming to clarify students’
opinions about their learning experience during the experiment. The pre-
experimental questionnaire (Appendix 1) aimed to get the students' views on
different aspects of their English lessons and thoughts concerning the level of
their skills development at that moment. After the survey, both groups passed a
pre-experimental test to check their listening and reading comprehension, as
well as writing skills. Following the test, students’ speaking skills were analyzed
during an interview on the subject of their specialty, corresponding to the
description of B2 level by Common European Framework of Reference for
Languages (CEFR). Listening comprehension in the tests was checked using
short podcasts available in free access on the Internet, for instance, on the British
Council website.
Preliminary test results showed an approximately equal level of English

knowledge of both groups. Engineering students of Group 1 (experimental)
were studying with the use of authentic videos in English and performed tasks
based on the information from them to check listening and reading
comprehension, as well as the use of English, grammar, speaking, and writing.
Each activity was a logical continuation of the previous one, differing in
difficulty level and varying in the form of fulfillment (individual-, pair-, team-,
the-whole-group work). That was done to develop and improve as many skills
as possible while corresponding to engineering students’ characteristics and
levels of English proficiency. The selected authentic video aids were fragments
of feature films (clips from top-rated Hollywood movies thematically relevant to
their specialty) and short scientific programs related to the lessons’ topics:
ecological catastrophes, energy conservation, and modern technologies. As the
research results provided in the current article imply, engineering students’
social skills are usually the least developed ones. Therefore, during the

360
experiment, special attention was paid to the advancement of oral

communication skills, i.e., listening and speaking.
After the experiment, the procedure was the same. A separate post-experimental
questionnaire for each group (Appendix 2; Appendix 3) had been prepared on
Google Forms to ask the students about their study during the experiment.
Besides, there was a post-experimental printed test followed by oral interaction
(an interview) with the lecturer to check whether the engineering students’
common and specialized knowledge, as well as listening-, reading, writing, and
speaking skills had improved.
Data Collection and Data Analysis Tools

All the pre- and post-experimental questionnaires and tests were prepared by us
and are protected by copyright. The analysis of the data contained two stages.
The students’ results of pre- and post-experimental tests were checked,
analyzed, and summarized by us in the form of a comparison table of tests
results provided further in the article. The surveys of students were conducted
with the help of Google Forms service, and the answers were automatically
systematized, analyzed, and graphically presented. The questionnaires
comprised both closed-ended and open-ended questions.
Ethical Issues
The students were informed about the aim and procedure of the experiment and
gave their agreement for the participation and academic achievement
processing. Students could refuse to participate at any stage of the experiment
without any academic consequences. The procedure of the research had been
approved by the Scientific Council of the Faculty of Linguistics, Igor Sikorsky
Kyiv Polytechnic Institute. The experts ensured that the investigation was
performed in proper conditions and confirmed the reliability of the obtained
results.
7. Research Results
After six weeks, a post-experimental B2 level test (the same for both groups) was
conducted. The purpose was to check students’ recall of the studied topics,
listening and vocabulary comprehension, and ability to put it in use (speaking
and writing tasks). Most attention was paid to testing students’ listening and
speaking skills to confirm or refute our hypothesis about the effectiveness of
authentic videos introduction in the English course regularly for improvement
of engineering students’ listening and speaking skills. A comparison of the pre-
and post-experimental tests results is presented in Table 1.
Table 1: Comparison of pre- and post-experimental tests results

Listening Reading Writing Speaking Total
average mark ratio per activity
max. 3 max. 2 max. 2 max. 3 max. 10
G1 G2 G1 G2 G1 G2 G1 G2 G1 G2
Pre-exp. test 1.32 1.16 1.56 1.64 1.28 1.24 1.4 1.28 5.56 5.32
Post-exp. test 2.48 1.2 1.8 1.8 1.52 1.28 2.4 1.32 8.20 5.60

361
The positive change of Group 1’s (G1) results was noticeable: the average total
mark during the pre-experimental test was 5.56 out of 10 and became 8.20 after
the experiment. The average listening and speaking ratios increased the most, by
1.9 and 1.7, confirming the benefits of using authentic videos to advance these
skills of engineering students. Reading and writing skills ratios of G1 increased
concurrently – by 1.15 and 1.19, respectively. Group 2’s (G2) outcomes did not
show significant changes: 5.32 (pre-experimental total average mark ratio) and
5.60 (post-experimental one). Among G2’s skills, only the improvement of
reading comprehension was apparent – by 1.1, whereas the other three
(listening, writing, and speaking) demonstrated growth within the range of
1.031–1.034.
Before and after the experiment, all students (G1 and G2) were asked to fill out
the questionnaires (Appendix 1; Appendix 2; Appendix 3). They had been
prepared by us and posted on Google Forms, comprising both closed-ended and
open-ended questions. It was done to discover their opinions concerning
different aspects of work in English lessons. They displayed contrasting
answers. Both groups expressed their neutral satisfaction with the amount and
diversity of tasks aimed to improve their communication skills before the
experiment. The level of motivation to do some further English study off-
campus was average (50%–60%). However, after the experimental study with
authentic videos and various assignments for boosting their social (oral
interaction) skills, as well as language-in-use practice, Group 1 voiced other
opinions. Qualitative data have shown that 88% of G1 students expressed the
desire to continue the work with videos at home. 90% stated that exercises with
authentic videos used as teaching aids had improved their understanding of the
studied topics, including specialty-related vocabulary and issues. Furthermore,
94% believed that assignments with content prediction, role-plays, and
argumentation-based discussions using the lesson’s lexis had advanced their
English proficiency significantly. Yet, the control group’s responses did not
show a similarly high percentage of satisfaction. According to them, their
incentive to continue English study at home was 64%, the ease of
comprehension averaged 70%, and the ease of speaking tasks fulfillment
amounted to 60%.
The results of the research and experimental verification demonstrate the

considerable influence of authentic video records on various aspects of English
teaching. The similarity of the positive outcomes of researchers from different
countries confirms the productive nature of authentic video records. In a short
time, they have proved to be efficient for the advancement of engineering
students’ competencies and skills, being just the right aids for their personal
qualities (characteristics) and to their liking. Thus, the current study has the
potential for further examination of authentic videos’ long-term effectiveness in
teaching engineering students English at university and their role in the
students’ lifelong learning of English.

362
8. Discussion and Recommendations

Engineering students usually have such qualities as capability of independent
and dependent work, emotional stability and tough-mindedness,
conscientiousness, responsibility and reliability, moderate cooperation, a low
level of such social skills as effective communication and self-expression. The
students majoring in engineering (with work experience) and graduates regard
oral interaction skills, problem-solving, ability to apply knowledge, and
interpersonal skills as the most valued generic competencies for employability in
their professional sphere, while written communication skills are believed to be
the least significant (Kline & Lapham, 1992; Tordai & Holik, 2018; Van Der
Molen et al., 2007). These fully correspond to the key criteria that employers in
engineering branches utilize for comprehensive staff selection: communication
skills (native and foreign languages), teamwork, knowledge of contemporary
issues (current events and technologies), professionalism, problem-solving and
decision-making skills, critical thinking, competency (general and in a specific
engineering field), motivation, lifelong learning. It is to the improvement of
them the most attention should be paid in English lessons with authentic audio-
visual materials.
Authentic videos are records created primarily by individuals speaking a

language as their mother tongue, in the first place, for native speakers of some
language, not for educational purposes. On such videos, verbal, audio-visual,
and written elements reflecting the real-life communication examples are
presented, so their speech rate and vocabulary are never specially reduced to be
more understandable for non-native speakers. The most popular among
authentic videos are feature films (and movie clips), TV shows, and programs,
which are available on the Internet platforms, DVD/Blu-Ray disks, and in the
form of digital releases. An English language lecturer’s aims of teaching with
authentic videos are the facilitation of the students’ language skills development
(primarily, listening and speaking); cultivation of students’ intercultural
communication competence; development of students’ aesthetic values and the
ability to appreciate authentic videos in English lessons along with the
understanding that the not-inherently-technical nature of a movie or show does
not downplay its significance in class since engineering issues can be found in
them frequently.
The advancement of communication skills (oral speech) of engineering students

is believed to be paramount. Such skills can be improved through problem-
based learning, a project method, discussions of compelling and relevant
engineering issues seen on authentic videos, reflection on the lesson’s topic
(promotion of critical thinking); student presentations, and projects on the
subject matters of lessons, using general English vocabulary and specialized
terminology (Saienko, 2017; Shevchenko, 2015b). We recommend organizing the
work in class in the form of cooperative learning (teamwork) to stimulate
proactive English study.
The use of authentic videos in English lessons should advance approximately

90%–95% of the competencies and skills of students, prioritized by employers in

363
the engineering sphere around the globe. Simultaneously, they should boost the
positive characteristics of engineers-to-be and amend their weak points.
The experimental verification of the article’s topic confirmed the effectiveness of

authentic videos in English lessons of engineering students. The study of the
experimental group was organized with the use of authentic videos, taking into
account the characteristics of engineering students (skills and personal traits).
The results of the post-experimental test demonstrated the noticeable rise of the
experimental group’s recall of the studied topics, listening and vocabulary
comprehension, and ability to put it in use in oral and written form, which is in
line with other studies (Ismaili, 2013; Kalra, 2017; Polat & Erişti, 2019).
Simultaneously, the outcomes of the control group, who studied without videos,
remained practically unchanged.
Also, the majority of the experimental group’s engineering students mentioned

the significant increase of satisfaction with their English proficiency
advancement, improvement of their understanding of the studied topics with
terminology and issues relevant to the specialty, and expressed the desire to
continue the work with videos at home, as well. The opinions of the control
group were far from equally unanimously high.
These results are supported by other studies. For example, Ismaili (2013), after
the experimental verification of the effectiveness of using movies in the EFL
classroom, revealed the improvement of students’ listening comprehension and
speaking ability, increased English vocabulary, as well as learning interests and
motivation through demonstration of real-life situations and conversations. The
findings of Ardiansyah (2018) also proved the positive influence of authentic
videos implementation on the students’ listening skills. According to the
researcher, such audio-visual aids may bring the content and the studied subject
matter to life for students, enabling them to make important connections in the
world beyond the classroom.
Based on our experience of using authentic videos in the English classroom, we

recommend some practical ways to boost engineering students’ collaboration
and oral speech using authentic video clips in English. They may be taken from
feature films but are always thematically relevant to the students’ specialty.
• Translation and Putting in Order (this activity draws on students’ linguistic
repertoires – translanguaging; the task’s goal is to enrich engineering
students’ vocabulary and knowledge of English dialogue structure and
develop logical thinking). Each group of students receives from their lecturer
a set of papers. On each, a sentence is written in their native language. These
are the dialogue lines from the authentic (English) video, which the lecturer
has translated into the learners’ language. The students should translate
them back into English and quickly put in the right order. After that,
everybody watches the record in English and checks if their translation and
the order of the lines were correct.
• Dialogue Guessing (this activity aims to develop students’ prediction,
writing- and speaking skills, as well as provide the opportunity to practice

364
English in use). Students watch a muted fragment with actively changing

events or facial expressions/body language on screen. They need to write
down possible dialogues of the characters, utilizing the handouts with the
word-list in English they have received from their teacher. Afterward, they
can read their dialogues synchronously with the video (still played without
sound). Then, the video is replayed once more unmuted, and everyone
checks whether they were right.
• Sequence of Events (the assignment improves engineering students’
prediction and oral communication skills, and expands their general and
technical English vocabulary). Students work in pairs. They get cards on
which the events from the video are written in English. The pair reads them
and consults with their foreign language lecturer if there is a problem with
the understanding of some words or collocations. Then, one of the pair sits
their back to the screen, on which the video will be played, and the other one
sits facing the screen. After the record has been set going (without sound) by
the teacher, the latter student tries to explain what they see, and the former
looks for the suitable card with an event and puts one card after the other to
form the logical succession in the end. When all is done, the lecturer plays
the clip with sound, and students check their accuracy.
• Recall and Putting in Order (the exercise is useful for training technical
students’ recall ability). The work can be done with the whole group, teams,
or pairs. Having watched an authentic video (a movie fragment) in English,
students receive a set of event cards which they should arrange in order.
When the whole group is doing the task, one student reads their card aloud,
and someone in the class reads theirs if they think that their card is a logical
continuation. In pairs and teams, the work is the same but with the
possibility of time limitation.
• Shadow Reading (the task is beneficial for students’ pronunciation practice
in English, especially if repeated several times). The lecturer chooses a video
in English with two or more characters. After the students have watched the
clip and familiarized themselves with the plot, vocabulary, and grammar
used, they get the printed transcript from the teacher and try to do the
dubbing of the same video (played muted), i.e., do shadow
reading (speaking at the same time with the characters on screen). While
performing such an assignment, engineering students will be replicating key
features of native speakers’ connected speech, intonation, speech rate, etc.
We recommend dividing the group into sub-groups, depending on the
number of roles to be played.
9. Conclusion
The obtained positive results demonstrate the practicality of authentic video
records in teaching engineers-to-be English, boosting their positive
characteristics, and amending the weak points. The outcomes of the study prove
our hypothesis about the effectiveness of authentic videos in improving
engineering students’ listening and speaking skills when such audio-visual
materials are introduced in the English course regularly. Authentic videos
increase students’ motivation and interest in studying English, as well as their
knowledge in the field of specialty, providing examples of real-life situations,

365
communication, and issues. Moreover, videos in question stimulate students’

critical thinking, various kinds of oral interaction, and collaboration. However,
to take full advantage of videos in lessons, the foreign language lecturers should
bear in mind the importance of multimedia use in diverse, compelling, and
meaningful ways, always together with the variety of classroom activities
(tasks), and necessarily per the technical students’ specialty. In that way, the
authentic video records will be considered advantageous educational aids and
will bring about the desirable outcomes of the study. The conducted research
creates an opportunity for further examination of authentic videos’ long-term
usefulness in teaching the foreign language at university and their role in each
engineering student’s lifelong learning of English.
10. References
Ardiansyah, L. D. S. (2018). Using videos in the teaching of listening. Jurnal Ilmiah
Mandala Education, 4(1), 290–304. doi:10.36312/jime.v4i1.552
Babu H, R., Buddayya, R., & Gujjarappa, N. L. (2019). Benefits of videos in YouTube for
the undergraduate students in engineering and technology in
India. Webology, 16(2), 57–71. doi:10.14704/web/v16i2/a190
Djebbari, Z., & Djebbari, H. (2018). Promoting innovation and change in English
education: Towards a philosophy of lifelong learning. International Journal of
Linguistics, Literature and Translation (IJLLT), 1(4), 8–11.
doi:10.32996/ijllt.2018.1.4.2
Eldeen, A. I. G., Abumalloh, R. A., George, R. P., & Aldossary, D. A. (2018). Evaluation
of graduate students employability from employer perspective: Review of the
literature. International Journal of Engineering & Technology, 7(2.29), 961–966.
doi:10.14419/ijet.v7i2.29.14291
Elen, J., Clarebout, G., Léonard, R., & Lowyck, J. (2007). Student-centred and teacher-
centred learning environments: what students think. Teaching in Higher
Education, 12(1), 105–117. doi:10.1080/13562510601102339
Freitas, A., Levatino, A., & Pécoud, A. (2012). Introduction: New perspectives on skilled
migration. Diversities. Skilled Migration and the Brain Drain, 14(1), 1–7. Retrieved
from http://newdiversities.mmg.mpg.de/?page_id=2024
Hadijah, S. (2016). Teaching by using video: Ways to make it more meaningful in EFL
classrooms. Proceedings of the Fourth International Seminar on English Language and
Teaching (ISELT-4), 4(2), 307–315. Retrieved from
http://ejournal.unp.ac.id/index.php/selt/article/view/6990
Ismaili, M. (2013). The effectiveness of using movies in the EFL classroom – A study
conducted at South East European University. Academic Journal of
Interdisciplinary Studies, 2(4), 121–132. doi:10.5901/ajis.2012.v2n4p121
Itani, M., & Srour, I. (2016). Engineering students’ perceptions of soft skills, industry
expectations, and career aspirations. Journal of Professional Issues in Engineering
Education and Practice, 142(1), 04015005. doi:10.1061/(ASCE)EI.1943-5541.0000247
Kalra, R. (2017). The effectiveness of using films in the EFL classroom: A case study
conducted at an International University in Thailand. Arab World English Journal,
8(3), 289–301. doi:10.24093/awej/vol8no3.19
Kline, P., & Lapham, S. L. (1992). Personality and faculty in British universities.
Personality and Individual Differences, 13(7), 855–857. doi:10.1016/0191-
8869(92)90061-s
Malhotra, R., & Verma, N. (2020). An impact of using multimedia presentations on
engineering education. Procedia Computer Science, 172, 71–76.
doi:10.1016/j.procs.2020.05.011

366
Månsson, J., Löfgreen, J., & Warfvinge, P. (2017, November 22–23). Effective use of video in
engineering education. Paper presented at the 6:E Utvecklingskonferensen För
Sveriges Ingenjörsutbildningar [6th Development Conference for Swedish
Engineering Education], Chalmers Tekniska Högskola [Chalmers University of
Technology], Göteborg, Sweden. Retrieved from
https://www.lth.se/fileadmin/cee/Documents/Effective_use_of_video_in_eng
ineering_education_FINAL.pdf
Markes, I. (2006). A review of literature on employability skill needs in engineering.
European Journal of Engineering Education, 31(6), 637–650.
doi:10.1080/03043790600911704
Muslem, A., Mustafa, F., Usman, B., & Rahman, A. (2017). The application of video clips
with small group and individual activities to improve young learners’ speaking
performance. Teaching English with Technology, 17(4), 25–37. Retrieved from
https://www.researchgate.net/publication/320805845_The_application_of_vid
eo_clips_with_small_group_and_individual_activities_to_improve_young_learn
ers’_speaking_performance
Polat, M., & Erişti, B. (2019). The effects of authentic video materials on foreign language
listening skill development and listening anxiety at different levels of English
proficiency. International Journal of Contemporary Educational Research, 6(1), 135–
154. doi:10.33200/ijcer.567863
Reddy, K. J. (2019). Relevance of listening and speaking skills for engineering students in
their professional career. Language in India, 19(11), 150–159. Retrieved from
http://www.languageinindia.com/nov2019/kattajanlisteningspeakingskillseng
ineeringstudents.pdf
Riemer, M. J. (2003). Integrating emotional intelligence into engineering education. World
Transactions on Engineering and Technology Education, 2(2), 189–194. Retrieved
from
http://www.wiete.com.au/journals/WTE&TE/Pages/Vol.2,%20No.2%20(2003
)/Riemer51.pdf
Saienko, N. (2017). Cognitive development of students in foreign language acquisition.
Advanced education, 7, 4–8. doi:10.20535/2410-8286.77570
Saienko, N., & Lavrysh, Y. (2020). Mobile assisted learning for self-directed learning
development at technical university: SWOT analysis. Universal Journal of
Educational Research, 8(4), 1466–1474. doi:10.13189/ujer.2020.080440
Shahani, S., Tahriri, A., & Divsar, H. (2014). EFL learners’ views towards video materials
and viewing techniques. International SAMANM Journal of Business and Social
Sciences, 2(1), 42–60. Retrieved
from https://www.researchgate.net/publication/285581685_EFL_LearnersVie
ws_towards_Video_Materials_and_Viewing_Techniques
Shevchenko, M. V. (2015a). Communicative approach to teaching English at technical
universities. Advanced Education, 3, 101–107. doi:10.20535/2410-8286.44302
Shevchenko, M. V. (2015b). The role of authentic videos in teaching English at technical
universities. Advanced Education, 4, 66–70. doi:10.20535/2410-8286.57306
Sun, L. (2020). Students’ attitudes and perspectives towards using videos in technology
assisted English class. Advances in Educational Technology and Psychology, 4(1), 1–
11. doi:10.23977/aetp.2020.41001
Tordai, Z., & Holik, I. (2018). Student’s characteristics as a basis for competency
development in engineering informatics education. International Journal of
Engineering Pedagogy (iJEP), 8(4), 32–42. doi:10.3991/ijep.v8i4.8133
Van Der Molen, H. T., Schmidt, H. G., & Kruisman, G. (2007). Personality characteristics
of engineers. European Journal of Engineering Education, 32(5), 495–501.
doi:10.1080/03043790701433111

367
Wang, Z. (2015). An analysis on the use of video materials in college English teaching in
China. International Journal of English Language Teaching, 2(1), 23–28.
doi:10.5430/ijelt.v2n1p23
Yusoff, Y. M., Omar, M. Z., Zaharim, A., Mohamed, A., & Muhamad, N. (2012).
Formulation in evaluating the technical skills of engineering graduates. Procedia
- Social and Behavioral Sciences, 60, 493–499. doi:10.1016/j.sbspro.2012.09.413
Zabalza, I., Aznar, M., Uche, J., Zalba, B., Peña, B., Marín, J. M., Llera, E. M., & Usón, S.
(2020, July). Students’ feedback on the use of educational videos to support the
study of Thermodynamics. EDULEARN20 Proceedings, 3832–3840.
doi:10.21125/edulearn.2020.1047
Zhou, J. (2005, June). Effective use of videos to enhance the teaching and learning of
Environmental Engineering. 2005 Annual Conference Proceedings. 2005 Annual
Conference, Portland, Oregon. doi:10.18260/1-2--14893
Appendix 1
Pre-Experimental Questionnaire “English Studying (Engineering Specialties)”
1. Your sex:
• male
• female
2. Name of your department: ______ .
3. Which activity in English class do you find most difficult? (select no more than 2)
• listening
• reading
• writing
• speaking
4. Which task is usually the most difficult for you to do during tests in English lessons?
(select no more than 2)
• listening
• reading
• writing
• speaking
5. Which task do you like to do the most in English class? (select no more than 2)
• listening
• reading
• writing
• speaking
6. Which task do you dislike to do in English class? (select no more than 2)
• listening
• reading
• writing
• speaking
7. What knowledge do you think you lack the most to be able to speak English easily?
(select no more than 2)
• listening
• reading
• writing
• speaking
8. Do you like the way English is taught at the university now?
• yes
• no
9. Are you satisfied with the amount and diversity of tasks aimed to improve your
communication skills in English lessons now?

368
• yes
• no
10. How often do you do some further study of English at home?
• very often
• often
• rarely
• never
11. What would you change in terms of studying English at university now?
12. Does your English lecturer use videos in class to present the educational material?
• yes
• no
13. How often does your English lecturer use English-language videos in class?
• every week
• several times a month
• several times a semester
• several times a year
• videos in English are not used at all
14. Would you be interested in studying English with the help of authentic videos in
English relevant to your specialty, such as fragments from movies, shows, programs?
• yes
• no
15. Would watching authentic videos in English encourage you to improve your
diligence and the absence of missed (for no good reason) English lessons?
• yes
• no
Appendix 2
Post-Experimental Questionnaire of the Experimental Group “English
Studying with Authentic Videos”
1. Your sex:
• male
• female
3. Did you enjoy studying English using authentic videos?
• yes
• no
4. Were you satisfied with the amount and diversity of tasks aimed to improve your
skills in English lessons with authentic videos?
• yes
• no
5. In your opinion, which skill did you improve the most while studying with the use of
authentic videos? (select no more than 2)
• listening
• reading
• writing
• speaking
6. In your opinion, have you improved your knowledge of issues in the specialty field in
the process of studying with authentic videos?
• yes
• no
7. After the experiment, you believe it is easier for you to recall in detail the information
studied in English class ______.

369
• after reading a text

• after watching a video
8. After the experiment, can you say it has become easier for you to speak English?
• yes
• no
9. Did authentic videos used in English class help you to improve your competency?
They ______. (select as many as you wish)
• helped to expand vocabulary
• helped to improve comprehension of oral speech
• helped to improve the quality of my English speaking
• helped to improve knowledge of grammar and its use
• helped to improve writing
• helped to improve reading
• helped to improve knowledge of my specialty
• helped to improve knowledge of the foreign culture
• did not help me to improve anything
10. In your opinion, after lessons with authentic videos, your English proficiency ______.
• has increased significantly
• has increased slightly
• has not increased
11. Has your interest in studying English grown after lessons with authentic videos?
• yes
• no
12. After studying English using videos, what kind of activity in class would you prefer
in the future?
• studying with the predominant use of authentic videos in English
• using only printed materials (textbook, handouts, etc.) while studying
• studying with the use of printed materials and videos (50/50)
13. Do you plan to watch authentic videos in English at home, as well, to improve your
English proficiency?
• yes
• no
14. What did you like most about the English lessons with authentic videos?
15. What did you dislike about English lessons with authentic videos?
16. Do you have any recommendations for improving English lessons with authentic
videos?
Appendix 3
Post-Experimental Questionnaire of the Control Group “English Studying
with Text Information”
1. Your sex:
• male
• female
3. Did you enjoy studying English using text information about engineering issues?
• yes
• no
4. Were you satisfied with the amount and diversity of tasks aimed to improve your
skills in English lessons with text information?
• yes
• no

370
5. In your opinion, which skill did you improve the most while studying with the use of
text information? (select no more than 2)
• listening
• reading
• writing
• speaking
6. In your opinion, have you improved your knowledge of issues in the specialty field in
the process of studying with the text information provided in English class?
• yes
• no
7. After the experiment, is it easy for you to recall in detail the information studied in the
English class?
• yes
• no
8. After the experiment, can you say that it has become easier for you to speak English?
• yes
• no
9. Did the text information provided in the English class help you to improve your
competency? It ______. (select as many as you wish)
• helped to expand vocabulary
• helped to improve comprehension of oral speech
• helped to improve the quality of my English speaking
• helped to improve knowledge of grammar and its use
• helped to improve writing
• helped to improve reading
• helped to improve knowledge of my specialty
• helped to improve knowledge of the foreign culture
• did not help me to improve anything
10. In your opinion, after studying with the text information provided in English lessons,
your English proficiency ______.
• has increased significantly
• has increased slightly
• has not increased
11. Has your interest in studying English grown after lessons with the text information
provided?
• yes
• no
12. After studying English using text information about engineering issues, what kind of
activity in class would you prefer in the future?
• continue using only printed materials (textbook, handouts, etc.) while studying
• studying with the use of printed materials and videos (50/50)
• studying with the predominant use of authentic videos in English
13. Do you plan to continue studying English at home via reading text information on
your specialty, similar to the one provided in class?
• yes
• no
14. What did you like most about English lessons with text information about
engineering issues?
15. What did you dislike about English lessons with text information about engineering
issues?
16. Do you have any recommendations for improving your English lessons?

371

Vol. 19, No. 8, pp. 371-392, August 2020
Quality Management of Educational Activities in

the Training of Specialists in the Field of Health
Care: the Case of Ukrainian Medical HEIs
Svitlana V. Gordiichuk
Zhytomyr Medical Institute of Zhytomyr Regional Council, Zhytomyr, Ukraine
https://orcid.org/0000-0003-4609-7613
Liudmyla M. Kalinina
Institute of Pedagogy of the National Academy of Pedagogical Sciences, Kyiv,
Ukraine
https://orcid.org/0000-0003-0534-6089
Irena E. Snikhovska
Zhytomyr Ivan Franko State University, Zhytomyr, Ukraine
https://orcid.org/0000-0001-6775-274X
Olga V. Goray
Zhytomyr Medical Institute of Zhytomyr Regional Council, Zhytomyr, Ukraine
https://orcid.org/0000-0002-2389-1366
Abstract. The paper addresses the current background, shortcomings

and improvement perspectives for the quality management system in
training nurses, paramedics and midwives in HEIs of Ukraine. To reach
the aim of the study, the authors analysed the legal and regulatory
documents, website documentation and electronic resources of 23
medical HEIs. The methods of mathematical data processing and
criterion analysis were used to analyze the effectiveness of the quality
management system implementation medical HEIs. The survey of the
managerial and administrative staff (n=398) was conducted. The aim of
the survey was to gain insight into the internal quality management
principles and procedures in medical higher educational institutions of
Ukraine, to identify the main tools used for internal auditing of
educational quality, the measures taken to assess the education quality
at the institutional level. The research findings are discussed with due
regard to the surveyed managerial staff needs and quality assurance
guidelines in medical education. The results of the article have a few
significant implications for practice of quality management system in
medical colleges and its perspectives for enhancing Ukrainian higher
education system. The conclusion was made that with introducing an
effective quality management system in medical HEIs, continuous
monitoring of quality assurance by HEI leaders, commitment to quality

372
as an organizing factor in education management, future medical

specialists will be competitive, demonstrating a sustainable growth in
academic performance, developing relevant practical skills and
professional mindset.
Keywords: Quality management; Quality assurance; Medical higher

education institution; Total quality management; Competence
1. Introduction
In this article, we shall probe into QM practices in Ukrainian medical HEIs. This
study aims to substantiate the rationale for the current background and
improvement perspectives for the quality management in training nurses,
paramedics and midwives in HEIs of Ukraine in connection with the shift of the
education paradigm to instill the culture of quality. We shall highlight the
premises and goals of the QMS at medical HEIs in Ukraine and illuminate the
insight of its implementation drawing on the authors’ model of the QMS of
educational activity. The findings should help HEI leaders pursue the idea of
quality awareness and develop the framework for quality management in their
department, college, institute, or university.
Currentlly, we witness an unprecedented drive for innovation and quality in

education worldwide, an urgent need to enhance the competitive advantage of
higher education institutions (HEIs). The effective Strategic Program for
European Cooperation in Education "Education and Training 2020" directs EU
member states to create high-quality lifelong national education systems for the
varied population segments. Quality education underpins the development of
sustainable societies and personal development, professional careers and
safeguards against unemployment and poverty. In the European context, this
will not only contribute to the EU’s economic and social goals, but will have a
positive impact on its member states’ socio-oriented capital. The outlined
problem has a priority social significance as evidenced by the pandemic on a
global scale, and undoubtedly is relevant for Ukraine and for all countries
without exception.
According to world practice, one of the most effective methodologies for

achieving this goal is the use of total quality management (TQM), which
includes a set of strategies. In this regard, the category of quality of medical
education is decisive as a result of quality management of educational activities
in the training of specialists in the field of health care. The strategy for the
development of quality higher medical education in Ukraine draws on global
and European trends to enhance its essence, mechanisms and results of its
acquisition, which in many countries has become the basis for a resilient nation
and accelerator of their economic prosperity. Ukrainian researchers are well
aware and take account of the international experimental studies conducted
worldwide, in particular the correlation between the level of the national
education system and the competitiveness of the country, thus the need to
pursue high-quality higher education (including medical education) is obvious.
Morover, given the falling of Ukraine's position in international rankings, in

373
particular the Human Development Index, the Freedom of Education Index

(Miller et al., 2020), the Global competitiveness Index, etc., the quality of
Ukrainian higher education should become an organizing factor.
The recent years have witnessed a surge of efforts to upgrade educational

services in Ukraine to international standards of excellence. According to
Ukraine’s legislative documents, the National Agency for Higher Education
Quality Assurance (NAQA) is responsible for setting out requirements for
quality and transparency for institutions, programs, and standards, implements
state policy in the field of higher education and withstands modern challenges.
To actualize these priorities and the ESG-2015 standards developed within the
frame of the Bologna Process (Bologna Process Implementation Report, 2018;
ESG, 2015) as well as European best practices [QUAERE, 2018a; QUAERE,
2018a], NAQA has developed the Regulation on Study Programs Accreditation
(Regulations, 2019) and it was adopted in August 2019 by the Ukrainian Cabinet
of Ministers.
The procedure of the new study programs accreditation began in September

2019 and is currently in progress, carried out by the NAQA experts. The
establishment of the National Agency was an important stage in the reform of
Ukrainian higher education, aimed at aligning the accreditation procedure with
that of the European higher education and research area (EHEA), developing
quality assurance system at both national and institutional levels (Strategy,
2020). NAQA is performing some of the key powers of the Ministry of Education
and Science of Ukraine. The first role of NAQA is viability analysis of the study
programs at higher educational institutions, responsibility for their evaluation
and accreditation according to measured criteria, issuing a relevant opinion and
recommending measures for improvement, etc. Apart from that, the institutional
accreditation and accreditation of independent accreditation agencies are an
essential function of NAQA, as well as promoting academic integrity, ensuring
transparency in university rankings, fostering internal quality assurance of
Ukrainian universities (Sydorenko, 2016). In the renewed accreditation process,
the ability of higher education institutions to provide a quality teaching and
learning process based on the principles of university autonomy is studied and
evaluated (Stukalo & Dluhopolsky, 2020).
The purpose of this study was to determine the state of the quality management
system of educational activities in the training of specialists in the field of health
care in Ukraine during 2014-2019 and to monitor the implementation of the
quality management system among medical HEIs training future nurses,
midwives, paramedics. To reach the aim of the study, the authors analysed the
legal and regulatory documents, website quality management documentation
and electronic resources of 23 medical public HEIs.
The hypothesis of the study was as follows: the competitiveness of the

healthcare professionals depends on the quality of educational activities of the
institution of higher learning and the level of formation of general and special
competencies. Competence is viewed as a basis for the preparedness acquired in

374
the process of professional training in the field of health care, as well as a

component of quality medical education. The data used for the study to test the
hypothesis were obtained through questionnaire sent to the management staff of
medical HEIs in Ukraine. The hypothesized relationships were tested with data
collected from 356 respondents by using the Spearman's rank correlation
method.
2. Literature review
In this section, the international scope of the quality management philosophy is
highlighted, as well as the quality assurance context in Ukraine is outlined.
Extensively used in business, manufacturing and services, the total quality
management (TQM) methodology provides effective tools to manage change in
education (Sallis, 2015) to produce educational services in the best, effective and
efficient manner by way of enhancing managerial control over HEI quality
assurance (Lubwama, Onen, & Kasenene, 2017). Exemplifying the Greek case,
Psomas and Antony (2017) emphasize that all educational activities should be
aimed at ensuring excellent academic performance, while HE quality
management must arise from the entire academic community awareness.
Exploring TQM in emerging economies, London argues that in managing for
quality, an achievement-raising factor the concept of quality itself may be
reconceptualized as a way of enforcing the QM practices and strategies to be
adopted for effective management (London, 2006). As Sciarelli et al. show, to
implement QM properly and achieve its effectiveness, HEIs shoud establish both
soft and hard QM practices, adopting the multidimensional approach of QM in
HE which could help directors understand the interdependencies of soft and
hard quality practices (Sciarelli et al., 2020).
An important point is made by Tarí and Dick (2016) regarding the trends in
quality management research at HEI. In particular, the authors make efforts at a
thorough analysis to show that the indispensable prerequisites for high quality
in higher education are a long-term planning, formulating yearly quality
programs and executing such quality programs to the letter annually so that the
vision of the higher education institutions can be accomplished. The
implementation of TQM practices is viewed as a crucial factor in reviving HEI
by way of entrenching awareness about quality into all undertaken procedures
(Johnson, 2019). According to Karuppusami and Gandhinathan, critical success
factors present essential constructs proceeding from the statistical analysis can
be carried out further to guide the researchers in using the reliable set of factors
for empirical studies (Karuppusami & Gandhinathan, 2006). To this end, Tarí
and Dick identify several critical success factors, namely: organizational
management, communication as a means of enhancing quality, the involvement,
training and professional development of the employees, and university
recognition and culture. All these must be given proper consideration by HEI
before total quality can be achieved or become successful (Oluwafemi &
Laseinde, 2020). Also, in our research we take account of the literature review
done by Sila and Ebrahimpour, who identified twenty five critical success
factors of TQM based on survey studies conducted in different countries and
published in a variety of journals (Sila & Ebrahimpour, 2002).

375
In the European Association for Quality Assurance Agencies context,

accreditation of the study programs includes some institutional elements, such
as the functioning of the internal quality assurance system, objectivized with due
regard to a robust and reliable education management system to ensure the
quality of provided educational services (Al-Ababneh & Alrhaimi, 2020; Sari,
Hadiyat, & Beatrice, 2015).
However, in medical HEIs there are numerous concerns as regards the stage-by-
stage integrated learning outcomes framework, assessment mechanisms (Macha,
2018). As Cumming (2010) notes, “application of the Bologna principles to
medicine can help to drive educational development and quality enhancement
in European medical education”. Patrício and Harden (2010) argue that medical
education in the Bologna Process serves as a catalyst for necessary changes.
The overview of management practices in Ukrainian medical HEIs convincingly

demonstrates the drive to the strategic and information management, quality
assurance system development at the institutional level and their efficient
practical implementation (Mazurkiewicz, Liuta & Kyrychenko, 2017; Kovtun &
Stick, 2009). To achieve this goal, a TQM methodology in higher education has
been introduced (Knutson & Kushnarenko, 2015). The institutional policy and
mission on education quality as well as the procedures and tools, progress
controls and performance indicators (European Standards and Guidelines, 2015;
EHEA Report, 2018 Bucharest Communique, 2012) of internal QA system of
education are being elaborated in Ukraine (Kuruch, 2018;Gordiichuk,
2018;Gordiichuk, 2019a, Mazurkiewicz, 2016; Finikov, 2018).
In Ukrainian medical colleges and schools, after their transition to a system of

professional pre-university education, efforts are made to create more integrated
reinforced curricula similar to those at the European medical schools, however
the model is currently under construction and has yielded no feasible results yet
(Kondrashov, 2018). That is why the creation of a QMS for educational activities
at the medical college has become urgent in recent years (Gordiichuk, 2017). In
Ukrainian context, the evaluation approach based on QA is used, and the
prerequisite of QMS is paramount to understand for an effective operation in the
sphere of higher education in Ukraine (Bilokonenko, 2018).
It is grounbreaking that NAQA has achieved international recognition through a

full membership in the International Network for Quality Assurance Agencies in
Higher Education (INQAAHE), affiliation with the European Association for
Quality Assurance Agencies (ENQA) and was included in the relevant list of the
European Quality Assurance Register for Higher Education (EQAR). It gives
every reason to consider the approaches towards accreditation in Ukraine to be
consistent with the requirements common in European practice.
3. Methodology
At the level of methodology, the subject of consideration is the managerial
activities of medical HEIs in Ukraine to ensure quality medical education. The
systemic analysis in higher medical education also requires: policy and mission

376
of the institution to ensure quality education, organizational structure of quality

management of educational activities; procedures, tools, success factors and
indicators of effectiveness.
The study was a succession of three stages: Stage 1 (the initial stage), Stage 2 (the
intermediate stage), and Stage 3 (the final stage). At different stages of the
research the following methods were used to address the tasks set: Stage 1:
theoretical analysis of literature, content analysis of the basic concepts, synthesis
and summation of the relevant normative and legal documents in EHEA
regulating the quality management of educational activity were carried out, as
well as drawing on the relevant research done by the European scholars
Wibisono (2018), Camilleri (2017), Fonseca (2015) and Ukrainian researchers
Kovtun and Stick (2009). Also, modeling was performed at this stage to develop
a framework for the survey of medical HEIs; Stage 2: empirical observations,
questionnaires, testing to determine the indicators for the model, experimental
survey to test the effectiveness of the authors’ method of quality management;
Stage 3: prognostic – independent expert evaluations for the purpose of
implementation and approbation of the developed criteria for assessing quality;
and statistical – methods of mathematical processing of quantitative research
data. In this study we focused on the effectiveness of the health care specialists
training using Spearman's rank correlation method, which allowed determining
the strength and correlation direction between the formed competencies in
academic disciplines and methods of mathematical processing of the data set on
the rank correlation coefficients.
At the initial stage of the research, the unit of analysis was taken to be the
educational achievements of medical students, which they obtained by
mastering basic disciplines. At the intermediate stage of the study, the unit of
analysis was the relation between performance in basic and fundamental
disciplines. At the final stage after the introduction of the content of education,
the unit of analysis was the formation level of professional competencies in
medical education.
To address the existing state of implementing the quality management system of

educational activities in medical HEIs, a survey of the managerial and
administrative staff was conducted on the principles and procedures of the
quality assurance system of educational activities at the institutional level. The
total number of respondents was 398, including 23 rectors of medical colleges, 63
vice-rectors, 89 heads of departments, 24 experts in teaching methodology and
199 chairpersons of final examination boards. The main sample inclusion
criterion was to be a representative of a medical HEI. The questionnaire used for
the study was a modification for medical institutions of a questionnaire
developed by the International Charitable Foundation "International Foundation
for Educational Policy Research" which the Ukrainian Catholic University
conducted within the framework of the project "Educational Initiatives Space"
(Finikov and Tereshchuk, 2018). The questionnaire took into account the
specifics of medical colleges and management of educational activities, training
of specialists in health care.

377
The survey (Appendix 1) comprised 28 questions, which were subdivided into

three clusters and provided for selective, constructive and alternative answers.
The first cluster aimed at collecting the general information on HEI, its teaching
staff, the availability of structural units responsible for the implementation of the
internal quality management system of educational activities at the medical
institution of higher learning. The questions of the second cluster made it
possible to gain insight into the presence of the internal quality assurance system
in medical HEIs, the main tools used for internal auditing of educational quality,
the measures taken to assess the education quality in medical HEIs. The third
cluster included the questions that determined the feedback from the main
stakeholders.
The second objective of the intermediate stage of the study was to analyze the
relevant information and the data on quality management of educational
activities, available on the official websites of higher medical education
institutions participating in the survey. Further, a comparative analysis yielded
the results about the actual state of implementing the internal quality
management system in medical institutions as compared with the data provided
by the medical colleges’ management. In total, the structure and content of the
official webpages of 23 medical schools and colleges throughout Ukraine were
analyzed.
The conceptual component of the authors’model allowed implementing the

concept of the overall quality management of educational activities in medical
educational institutions. The methodological and fit-for-purpose component of the
quality assurance system of educational activities in medical educational
institutions provided for the formation of strategy as well as strategic and
tactical goals and objectives of quality assurance of educational activities in
medical colleges. The subjective and contentual component of the model ensured a
quality management of the educational activities in the medical college on the
basis of national educational standards and quality management system. The
activity and procedural component of the model made it possible to gain insight
into the compliance of educational services provision by the medical institution
of higher learning with licensing and accreditation requirements in higher
education and a reasonable provision of resources for the teaching and learning
process. This component of the model was considered from the standpoint of
system management in terms of the quality assurance using the system-based,
the process-based, the situation-based, the innovative, the informational, the
competency-based, the resource, the creative, and the standardized approaches,
as well as relying heavily on internationalization and academic integrity. In its
turn, the evaluational component of the system of quality assurance of
educational activities of the medical college provides for an analysis of the
effectiveness of the quality of educational activities of the medical college and
the quality level of medical education.
To analyze the effectiveness of the quality management system of educational

activities in medical colleges, the academic performance of the degree-seeking
students in medical education during their programme’s life cycle from 2014 to

378
2019 in 4 medical colleges was analyzed, in particular: Zhytomyr Medical

Institute, Berdychiv Medical College, Mykolayiv Basic Medical College and
Vinnytsia Medical College named after acad. D.K. Zabolotny. Basically, the key
indicators taken account of in the study were as follows. Indicator 1: The
prospective students’ success in the subject of biology as a major discipline when
entering the medical college, as attested by the completion certificate of the basic
general secondary education or the certificate of complete general secondary
education; Indicator 2: The prospective students’ success rate in the subject of
biology of based on the results of entrance examinations (for persons who were
enrolled on the ground of the basic general secondary education) and the results
of external independent evaluation (for persons who were enrolled on the
ground of the basic general secondary education); Indicator 3: The students’
success rate in fundamental disciplines: "Human Anatomy", "Physiology",
"Medical Biology", "Fundamentals of Medical Genetics" belonging to the cycle of
general training; Indicator 4: The students’ success rate in professionally-
oriented disciplines (majors): "Internal Medicine", "Surgery", "Pediatrics",
"Obstetrics", "Gynecology", "Nursing in Internal Medicine", "Nursing in
Surgery", "Nursing in Pediatrics" "Nursing in obstetrics", "Nursing in
gynecology". Indicator 5: The undergraduate students’ success rate in passing
the mid-tests both in the subject cluster "Internal Medicine", "Surgery",
"Fundamentals of Nursing" and "Patient Care", "Pediatrics", "Obstetrics and
Gynecology" within the framework of the national integrated exam KROK-M.
In total, in the course of the final stage of the study, the academic performance of
895 students was analyzed, including 284 students who were enrolled in the
medical college on the ground of the basic general secondary education (111
people studied at the department “General Medicine”, the qualification of a
paramedic; 173 people – at the department of "Nursing", the qualification of a
nurse), and 611 people who entered the program on the score of complete
general secondary education (246 people studied at the department "General
Medicine", the qualification of a paramedic, 206 people – at the department
"Nursing", the qualification of a nurse; 159 people – at the department
"Obstetrics", the qualification of a midwife).
To explore the effectiveness of the quality management system of educational

activities in medical colleges, the success rates of each student in the course of
their educational life cycle were compared by determining the Spearman
correlation coefficient (the direct and inverse strong correlations ± 0.1 – ± 0.29,
average strength ± 0.3 – ± 0.69, weak ± 0,7 – ± 0.99, ± 1 complete dependence).
The reliability of the received results was verified by means of Student's
criterion. Due to the methodology used in our empirical analysis it should be
emphasized that the following findings reflect the relevance of quality assurance
management in medical HEIs.
4. Results
Based on these findings, further conclusions can be drawn. In the course of the
research, we determined that the management of 23 medical schools and
colleges claims to have its own internal system for quality assurance of

379
educational activities and the implementation of procedures and measures

provided for by the regulatory documents. At the same time, the heads of
medical HEIs that took part in the survey noted that at the institutional level
there are only elements of internal quality assurance systems for educational
activities. Moreover, rectors ad vice-rectors of 22 medical schools and colleges
expressed their intention to build local quality assurance systems. In our
opinion, the resulting imbalance of answers is associated with a lack of
awareness, motivation, interest in the need to implement internal QA system to
ensure the quality of educational activities by the top management of medical
HEIs.
According to the research data, it was shown that in all the surveyed medical
HEIs there is no separate structural units responsible for the management and
quality assurance of educational activities. The structural units that are in charge
of periodical monitoring of the quality of educational services are as follows: the
Department of Teaching and Learning, the Department of Practice, the Final
Exams Commission, and administrative officials (Vice-Rectors for Educational
Work, heads of departments, the chairpersons of final examboards, experts in
methodology, etc.).
The management of medical education institutions uses various audit methods

and plans measures to eliminate the identified shortcomings, clarifies
problematic issues, but does not take into account the results of self-assessment
of their own activities by teachers. The heads of medical schools and colleges
unanimously agree as regards the necessity for consistent review of educational
programs. For a qualitative revision and correction of the content of the
educational program, the opinions of stakeholders are monotored: teaching staff
(52.2%); HEIs administration (43.5%); students’ parents (56.6%); employers
(39.2%); students (34.8%); professional public associations, unions (30.5%),
entrants (13.1%). Notably, analyzing the respondents' answers to the measures
taken as a result of the review of educational programs, we found that the
removal of unnecessary subjects is systematically or periodically carried out in
21% of institutions; 26% of medical colleges and schools introduce the new
compulsory subjects based on the results of the audit; 78% transfer disciplines to
another year of study; 39% expand the list of the student’s elective disciplines;
47% develop new educational programs.
5. Discussion
One of the problematic issues of procedures and measures to ensure the quality
of educational activities is to assess the quality of teachers’ work. The basic
measures taken during the audit of the teaching staff activities at medical HEIs
are as follows: the use of systematic or periodic surveys of students on the
quality of teaching (91%); conducting open lectures with their subsequent
evaluation by the colleagues and administrators (95%); conducting the final
control in the discipline (test / exam) by another teacher (73%); executing the
rector's test papers (94%); monitoring the volume and quality of the educational
and methodological support of the discipline developed by the teacher
(textbooks/manuals, workshops, tests, etc.) (92%); evaluation of the teacher's

380
research activity (number of publications, participation in scientific conferences,

etc.) on quality assurance of education (90%); assessment of the teacher’s
extracurricular activity (arranging academic competitions and contests,
supervising students’ reserch, etc.) (93%). Thus, the management of medical
HEIs with different degree of frequency (systematically/periodically) tries to
audit the teachers’ educational, methodological, practical, educational, scientific
activities.
Of a particular concern is consistency and systemic character in the use of

information obtained during the audit to improve the quality of educational
activities, as we found the following frequency of measures based on the results
of teaching quality assessment: an personal interview with the head of medical
institution (systematically 69.5%, periodically 21.7%), formation and making
public the teachers’ rankings of HEIs based on the results of the teaching quality
audit (systematically 34.9%, periodically 47.9%); the salary differentiation
depending on the results of the audit of the teaching quality (systematically
17.3%, periodically 8.9%); introducing financial incentives in the form of
awarding the best performing teachers based on the results of the audit of the
teaching quality in medical schools and colleges (systematically 52.1%,
periodically 34.7%); mandatory teacher’s certification at the advanced training
courses, extention courses, etc. based on the results of the audit of the teaching
quality in medical HEIs (systematically 60.8%, periodically 8.7%); dismissal of
teachers based on the results of the audit of the teaching quality in medical
schools and colleges (systematically 4.5%, periodically 13%). Thus, as a rule, the
main incentives to increase the motivation and quality of teaching are financial
rewards, while in the identification of shortcomings in teaching and learning
process the management of medical HEIs focuses only on individual interviews
and recommendations for the training enhancement (Gordiichuk, 2019).
In our opinion, one of the criteria for the effectiveness of the internal quality
assurance system of educational activities is the extent to which the general and
special competencies in medical education are formed. We also found out what
the tools are and how often they are implemented to audit learning outcomes.
Thus, it is determined that 86% of medical colleges and schools systematically
carry out internal measurement of the competence formation level by
conducting rector's tests, internal testing and review procedures, etc.; 47.8% of
medical HEIs systematically and 30.4% periodically conduct external evaluation
of the quality of students learning outcomes; 43.4% of the surveyed medical
HEIs conduct surveys of graduates and maintain feedback with them; the top
management of 30.4% of medical colleges and schools systematically and 56.5%
periodically conduct surveys among the employers (enterprises, institutions,
etc.) in order to obtain an assessment on the alumni employees’ skills, abilities,
knowledge quality.
The subsequent part of the study was to analyze the completeness of the
published information on the HEIs official websites. The criteria for the analysis
were as follows: the availability of regulatory documents (an organization
charter, a corporate agreement, information on the activities of structural units,

381
provisions on the organization of the educational process, regulations on the

internal quality assurance of educational activities, information on degree
programs and relevant study programs offered in medical HEI; the details as
regards the admission and enrollment, etc.); information about the department
or structural unit in charge of monitoring the quality of education; interaction
with stakeholders; the results of the annual surveys of students and teaching
staff; information on the conducting of educational activities at medical
education institutions.
The information published on the official websites of medical HEIs basically

highlights the history of the institution, the composition of the administration,
departments as structural units of medical institutions, cycle subject
commissions as structural units responsible for methodological activities,
licensing and accreditation, enrollment, etc. The websites of 17% medical schools
and colleges do not provide any information about the organization charter, the
corporate agreement, internal provisions regulating teaching and learning,
methodological, educational, practical and scientific activities. Only a scanned
copy of the regulations on the educational process is posted on the official
websites of 15 medical education institutions (65%). In our perspective, the
following aspects are particularly striking and presented a challenge for the
objective dta collection. Analyzing the official websites of 23 medical HEIs that
train nurses, midwives, paramedics, we found that only the websites of the two
of them (8.5%) published the provisions on the internal quality assurance
system, the websites of still two more medical colleges and two medical schools
(17.3%) contain provisions on internal control, on internal school control, on
monitoring the quality of the educational process, on conducting rector's tests,
which is an element of the internal quality assurance system of education. No
medical education institution has a separate structural unit responsible for
quality assurance. Moreover, there was no information in terms of feedback
from health care institutions in charge of internship training, nor stakeholders
(employers, students and their parents, alumni, etc.). Furthermore, we did not
find any ratings assessing the educational activities of the HEI teaching staff (in
one institution of medical education there was a provision in place on the rating
of teachers, the website of another institution contained information on the
regulations of preparation for certification of teachers). The websites of 12
medical schools and colleges (52%) provided the documents informing about
educational programs for training students on the basis of both complete and
basic general secondary education.
Analysis of the effectiveness of the implementation of the quality assurance

system of educational activities in the Zhytomyr Medical Institute” revealed the
presence of strong direct correlations at the Departments of the General
Medicine, the Department of Nursing and the Department of Obstetrics (Table
1):

382
Table 1: Correlation between performance, of competence formation level and

medical education quality in Zhytomyr Medical Institute
Department Discipline Competencies formed Correlation
by the discipline strength
General Internal Medicine 0.77
Medicine Human Anatomy Surgery 0.76
Gynecology 0.77
Pharmacology and Internal Medicine 0.73
Medical Prescription Obstetrics 0.75
Infectology Microbiology 0.75
Internal Medicine 0.76
Patient Care
Gynecology 0.72
Nursing Nursing in Pediatrics 0.78
Human Anatomy Nursing in Surgery 0.85
Nursing in Gynecology 0.73
Pharmacology and Nursing in Surgery 0.79
Medical Prescription Nursing in Pediatrics 0.77
Fundamentals of Nursing Nursing in Obstetrics 0.75
Obstetrics Pharmacology and Gynecology 0.86
Medical Prescription
A comparative analysis of other indicators revealed the presence of direct

correlations of the average strength between the studied phenomena in students
of the three departments. The analysis of the results of student success in the life
cycle in Berdychiv Medical College at the Department of General Medicine, the
Department of Nursing gave an opportunity to establish strong direct
correlations between competencies formed by general training in the disciplines
and students’ competencies in the fundamental disciplines (Table 2):

medical education quality in Berdychiv Medical College
Department Discipline Competencies formed by Correlation
the discipline strength
General Medicine Human Anatomy 0.72
Biology
Surgery 0.76
Internal Medicine 0.77
Human Anatomy Surgery 0.70
Nursing Human Anatomy 0.70
Biology
Medical Biology 0.77
Human Anatomy Nursing in Pediatrics 0.90
Pharmacology and Nursing in Gynecology 0.76
Nursing in Pediatrics 0.90
Physiology Nursing in Obstetrics 0.75
Nursing in Gynecology 0.81
Nursing in Internal 0.72
Biology
Medicine
The comparative analysis of student performance based on the implementation

of the model of quality assurance of educational activities at the Department of

383
General Medicine, the Department of Nursing, the Department of Obstetrics at

Mykolayiv Basic Medical College yielded strong correlations between
competencies formed by the fundamental disciplines (Table 3):

medical education quality in Mykolayiv Basic Medical College
General Medicine General Care Gynecology 0.74
Pharmacology and Gynecology 0.74
Medical Prescription Pediatrics 0.73
Nursing Medical Biology 0.76
Biology
Human Anatomy 0.74
Nursing in Internal 0.73
Human Anatomy Medicine
Nursing in Pediatrics 0.76
Pharmacology and Nursing in Obstetrics 0.72
Obstetrics Pharmacology and Surgery 0.76
General Care Obstetrics 0.79
Analysis of efficiency indicators of introducing the system of quality assurance

at Vinnytsia medical college named after acad. D.K. Zabolotny enabled to reveal
that the educational achievements of medical students who studied at the
Department of General Medicine, the Department of Nursing, the Department of
Obstetrics showed strong direct correlations between the competencies formed
at the level of general education disciplines and the competencies formed at the
level of professionally-oriented clinical disciplines (Table 4):

medical education quality in Vinnytsia medical college named after acad. D.K.
Zabolotny
General Medicine Biology Medical Biology 0.73
Human Anatomy Gynecology 0.71
Nursing Nursing in Pediatrics 0.73
Human Anatomy Nursing in Internal 0.73
Medicine
Pharmacology and Nursing in Obstetrics 0.73
Physiology Nursing in Pediatrics 0.73
Obstetrics Pharmacology and Pediatrics 0.74
Physiology Internal Medicine 0.71
Patient Care Obstetrics 0.74
Among other indicators analyzed, in the departments where the training of

future paramedics, midwives, nurses is provided, should be mentioned the

384
presence of direct orientation correlations of medium (86%) and weak (2%)

strength.
Thus, based on the results of the study, we have drawn some inferences about
the state of implementation of quality assurance in Ukrainian medical HEIs. The
management of medical schools and colleges only declares the implementation
of an internal quality assurance system or its elements, but a detailed analysis
drawing on the answers of respondents and monitoring of official websites
shows that most medical HEIs do not have the necessary information,
methodological, incentive and human resources to implement effective quality
assurance system at the institutional level.
In the medical HEIs that participated in the study, there are no structural units
responsible for quality management of education, such functions are
additionally assigned to certain administrators, usually vice-rectors in charge of
educational work or experts in methodology, who have a number of other
functions, and therefore the process of internal quality assurance is of a formal
nature, it is reduced to conducting rectoral tests and assessment analyses of the
students’ grades. The prevailing number (86.0%) of medical schools and colleges
note that the process of introducing the internal quality assurance system of
educational activities requires the elaboration of the pattern of local quality
assurance system that would take into account the uniqueness of a medical
institution, would be adapted to the specifics of the medical school or college
learning environment and would be made public in the form of methodical
recommendations. The management of 20 medical schools and colleges (86%)
admit that the process of introducing internal quality assurance systems requires
the creation of a certain model of a local quality assurance system.
Medical education institutions have taken on board the commitment to study of

the interests and opinions of stakeholders (students, entrants, teachers,
administration, governing bodies, students’ parents, employers, professional
and public organizations, etc.). However, even if there is a quality feedback from
the stakeholders, in most cases, the obtained results are not taken into account
when reviewing educational programs. The problem of quality and adequate
response to the results of the quality audit of the existing educational programs
is provided by the educational standard for training students in the field of
health care is a unified qualifying exam and its component, namely the licensed
comprehensive qualifying exam "KROK". The elimination of the "unnecessary"
disciplines, the introduction of new compulsory disciplines, the expansion of the
list of electives is not virtually realized in medical HEIs. The possible reason is
that the State Institution "Testing Center at the Ministry of Health of Ukraine"
forms the pool of test tasks on the basis of the estimated curriculum for the
training of medical personnel. In doing so, the cycles of natural sciences and
professional training are deemed obligatory.
Medical HEIs have elements in place for assessing the quality of teachers' work,
but the results of such audits are closed and are not published on websites. In
addition, the main activities carried out based on the results of the audit are

385
individual interviews of the head and the teacher. Only 47.0% of medical
colleges and schools out of 23 use the technological innovations in educational
activities, which indicates the inadequacy of information and educational
environments and the need for its introduction for both external and internal
quality assurance system of the educational process, which was evidenced in the
conditions of quarantine COVID-19.
An analysis of the official websites of medical schools and colleges gives the
ground to conclude that most sites are overloaded with reports about activities
and events at educational institutions, but they have minimum information for
students, applicants and other stakeholders to ensure quality, content of
educational programs and disciplines, as well as the evaluation criteria.
Taking a closer look at the effectiveness indicators of the quality assurance

system, it should be noted that the correlations between the academic
performance data indicate the systemic dependence of the of general and special
competencies formation in students on managerial decisions on educational
quality, pedagogical skills and awareness of the faculty, as well as commitment
to the efficient management of HR, information, innovation, methodological,
and financial resources.
6. Conclusion
Based on the findings of the study, we drew some conclusions about the state of
the QMS and QA implementation in medical HEIs of Ukraine. With this current
study, we determine that there is a discrepancy between the information on the
implementation of internal quality assurance system declared by the
management of medical colleges and schools; issues of ensuring the quality of
educational activities are dealt with by officials at different levels of the
management structure. The major issues to be addressed are: 1) a consistent
coordination and monitoring of quality management; 2) the provision of quality
feedback from the HEI to all stakeholders for reviewing the educational
programs. It should be emphasized that the major recommendations for medical
HEIs are to improve the management of educational activities, the
implementation of the internal QA system, the monitoring of the key
competencies, the quality feedback from stakeholders. In addition, the
implementation of a QA model of educational activities at medical HEIs that
train nurses, midwives, paramedics provides for the development of a consistent
formation of the professional mindset. It is demonstrated by strong and medium
direct correlation links between the competencies’ formation in both
fundamental and major disciplines in the educational programs. The data
obtained during the study and analyzed and generalized results provided an
opportunity to determine the state of the quality management system of
educational activities in the training of health care professionals in Ukraine
during 2014-2019.
7. Limitations of the study

The overriding limitation of this study was that even though medical HEIs have
the tools for assessing the quality of teachers' work, the results of such audits are

386
not in open access on websites. Another limitation was the scarcity of feedback
from the stakeholders, the obtained feedback is not taken into account when
reviewing educational programs majoring in nursing.
8. Directions for future research

Further research prospects may be monitoring studies of the effectiveness of
quality assurance systems in Ukrainian HEIs in 2020-2021, in particular to find
how the principles of total quality management are applied in comparison with
the survey of 2014-2019, what best practices the HEI leaders benchmarked and
adopted to assure quality; whether all relevant stakeholders such as students
and employees are engaged in managing quality at the HEI.
9. References
Al-Ababneh, H., & Alrhaimi, S. (2020). Modern Approaches to Education Management
to Ensure the Quality of Educational Services. TEM Journal, 9(2), 770‐778.
Bilokonenko, H. V. (2018). Tools for Designing, Monitoring and (Self)Evaluating the
Efficiency of Internal Quality Assurance System by Hei. Adaptive Management:
Theory and Practice. Series Pedagogics, 5(9).
Bucharest Communique. (2012). Making the Most of Our Potential: Consolidating the
European Higher Education Area, Bucharest Communique 2012. Retrieved from
http://www.ehea.info/Upload/document/ministerial_declarations/Bucharest
_Communique_2012_610673.pdf
Camilleri, F. A. (2017). Standardizing Management Systems for Educational
Organizations: Implications for European Higher Education, 12th European
Quality Assurance Forum, Riga, Latvia.
Cumming, A. (2010). The Bologna process, medical education and integrated learning.
Medical Teacher, 32(4), 316-318. https://doi.org/10.3109/01421590903447716
Finikov, T. V., & Tereschuk, V. I. (2018). Lokalni systemy upravlinnya yakistyu: svitovyi
dosvid ta ukrayinski praktyky pobudovy [Local quality management systems –
international and Ukrainian practices]. Kyiv: Takson. Retrieved from
http://www.edupolicy.org.ua/_dx/assets/images/Analit18ua_08-5mm.pdf
Fonseca, L. M. (2015). From quality gurus and TQM to ISO 9001:2015: a review of several
quality paths, Int. J. Qual. Res., 9, 167–180.
Freedom of Education Index. (2016). Worldwide report 2016 on freedom of education. OIDEL:
Fondazione Novae Terrae. Retrieved from
http://oidel.org/doc/FEI_complet2.pdf
Future of Europe: Towards a European Education Area by 2025. (November 14, 2017). (Press
Release) Retrieved from https://ec.europa.eu/commission/
presscorner/detail/en/IP_17_4521
Gordiichuk, S. V. (2017). Ensuring the quality of educational activities at the institutional
level. Pedagogical sciences: a collection of scientific works, 139-144. (In Ukrainian).
Gordiichuk, S. V. (2018). Substantiation of internal quality assurance of educational
activity and quality of higher education in Zhytomyr Medical Institute in the
conditions of European higher school. Bulletin of Luhansk National University
named after T. Shevchenko (Pedagogical sciences), 6(320), 103-110. (In Ukrainian).
Gordiichuk, S. V. (2019a). Management of the internal system of quality assurance of
educational activities in medical education institutions. Bulletin of the National
University "Chernihiv Collegium" named after T.G. Shevchenko, 5(161), 48-52. (In
Ukrainian).

387
Gordiichuk, S. V. (2019b). Management of the quality of educational activities of a

medical educational institution on the basis of an innovative approach. Bulletin
of Zhytomyr I. Franko State University, 2(97), 35-42. (In Ukrainian).
Johnson, D. (2019). Assessing Quality Assurance Practices in Institutions of Higher
Learnings. International Journal of Learning, Teaching and Educational Research,
18(12), 30-45. https://doi.org/10.26803/ijlter.18.12.3
Karuppusami, G., & Gandhinathan, R. (2006). Pareto analysis of critical success factors of
total quality management: A literature review and analysis. The TQM Magazine,
18(4), 372-385. https://doi.org/10.1108/09544780610671048
Knutson, S., & Kushnarenko, V. (2015). Ukraine: The New Reforms and
Internationalization. International Higher Education, 79, 18-19.
https://doi.org/10.6017/ihe.2015.79.5850
Kondrashov, N. (2018). Interaction of the higher education and school in the future
teacher to continuing professional education. Indywidualistyczny і wspolnotowy
obraz ksztalcenia doroslych w swiecie – terazniejszosc і perspektywy. Monografia
wieloautorska; redakcja naukowa Leszek Ploch, Matgorzata Myszka, Aneta
Nieweglowska, Malgorzata Wisniewska: Siedlce (pp. 139-147).
Kovtun, O., & Stick, S. (2009). Ukraine and the Bologna Process: A Case Study of the
Impact of the Bologna Process on Ukrainian State Institutions, Higher Education
in Europe, 34(1), 91-103. https://doi.org/10.1080/03797720902747066
Kuruch, A. V. (2018). Ensuring the quality of higher education in Ukraine: the challenges
of today. Science and Education a New Dimension. Pedagogy and Psychology,
VI(70)(170), 27-32. https://doi.org/10.31174/SEND-PP2018-170VI70-06
London, N. A. (2006). The management of quality in education in an emergent society: a
case study. Journal of Education Policy, 1(3), 303-
317. https://doi.org/10.1080/0268093960110303
Lubwama, J., Onen, D., & Kasenene, E. S. (2017). Influence of Management on Quality
Assurance in National Teacher's Colleges. International Journal of Learning,
Teaching and Educational Research, 16(11), 138-156.
Macha, W. (2018). Incompatible With Bologna? The Two-Cycle Degree Structure in
Medical Education in the EHEA. Education Policy, Dec. 11. Retrieved from
https://wenr.wes.org/2018/12/two-cycle-degree-structure-in-medical-
education-in-the-ehea
Martin, M. (2018). Internal Quality Assurance: Enhancing higher education quality and
graduate employability. Paris: IIEP-UNESCO.
Mazurkiewicz, M. (2016). Quality Assurance in Higher Education Institutions in Ukraine
2016 through the prism of European Guidelines and Standards ESG 2015. Wrocław:
EXANTE. Retrieved from http://quaere.pwr.edu.pl/wp-content/uploads/QA-
in-Ukraine-reports.pdf
Mazurkiewicz, M., Liuta, O., & Kyrychenko, К. (2017). Internal Quality Assurance
System for the Higher Education: Experience of Ukraine and Poland. Business
Ethics and Leadership, 1(4), 74-83. https://doi.10.21272/bel.1(4).74-83.2017.
Mission and Strategy. (2020). The National Agency for Higher Education Quality
Assurance (Ukraine). Retrieved from https://en.naqa.gov.ua/?page_id=642
Oluwafemi, I., & Laseinde, T. (2020). Useful Total Quality Management Critical Success
Fundamentals in Higher Education Institution. Human Systems Engineering and
Design, II, 1066-1074. https://doi.10.1007/978-3-030-27928-8_158.
Patrício, M., & Harden, R. M. (2010). The Bologna Process – A global vision for the future
of medical education. Med Teach, 32(4), 305-315.
https://doi.org/10.3109/01421591003656123

388
Psomas, E., & Anthony, J. (2017). Total quality management elements and results in
higher education institutions: The Greek case. Quality Assurance in Education,
25(2), 206-223. https://doi.org/10.1108/QAE-08-2015-0033
Sallis, E. (2015). Total Quality Management in Education (3rd Edition). Routledge.
Sari, Y., Hadiyat, M. A., & Beatrice, C. (2015). The design and implementation of Lean
Quality Management System National, Conference Proceedings of Production
System XI and Quality Engineering Management, VI (pp. B123–B136).
Sciarelli, M., Gheith, M. H., & Tani, M. (2020). The relationship between soft and hard
quality management practices, innovation and organizational performance in
higher education. The TQM Journal. https://doi.org/10.1108/TQM-01-2020-0014
Sila, I., & Ebrahimpour, M. (2002). An investigation of the total quality management
survey based research published between 1989 and 2000: A literature review.
International Journal of Quality & Reliability Management, 19(7), 902-970.
https://doi.org/10.1108/02656710210434801
Standards and Guidelines for Quality Assurance in the European Higher Education Area.
(2015). Кyiv: CS Ltd.
Strategy of the National Agency For Higher Education Quality Assurance to 2022. (2020)
Retrieved from https://en.naqa.gov.ua/wp-
content/uploads/2020/04/Strategy-to-2020.pdf
Stukalo, N., & Dluhopolskyi, O. (2020). Peculiarities оf Educational Programs
Accreditation in Pandemic Times (Case оf Ukraine). Implementation of European
Standards in Ukrainian Educational Research: Conference proceedings of the
materials presented at the Forth UERA Conference, June 26, 2020. Drohobych:
Тrek.
Sydorenko, N. (2016). Internal quality assurance of higher education in Ukraine as social
and educational priority. Public Administration and Local Government, 4(31), 81–86.
Tarí, J. J., & Dick, G. P. M. (2016). Trends in quality management research in higher
education institutions. Journal of Service Theory and Practice, 26(3), 34-70.
https://doi.org/10.1108/JSTP-10-2014-0230
The European Higher Education Area in 2020: Bologna Process Implementation Report
(2018). Retrieved from http://www.ehea.info/page-ministerial-conference-
paris-2018
Wibisono, E. (2018). The new management system ISO 21001:2018: What and why
educational organizations should adopt? International Seminar on Industrial
Engineering and Management, Makassar, Indonesia, 11.

389
Appendix 1
Questionnaire
for heads or deputies for educational work (vice-rectors) of medical higher
education institutions
Dear colleagues!
This questionnaire addresses the monitoring of the internal quality assurance system of
education in medical colleges / institutes / academies. The results of the questionnaire
will be used for scientific purposes to improve the quality management system of
educational process.
1. Indicate the form of ownership of the HEI, which you head or in which you
work (public or private).
2. Please name the management structures in the HEI that you head or in which
you work.
3. Please indicate the number of teachers who work in your HEI (as the main
employment).
4. Please indicate the number of teachers working in your HEI.
5. Please indicate the number of students studying in your HEI.
6. Does your educational institution have an internal quality assurance system?
7. Are there elements of the internal quality assurance system of education in
your HEI?
8. Does the HEI you head or work in intend to establish an internal quality
assurance system of educationalactivity? If not, what hinders your intentions?
9. Please indicate which elements of the internal quality assurance system of
education (principles and procedures) are used in the HEI that you head or in
which you work (please give one answer in each line):
9.1 ensuring the quality of educational programs;
9.2. ensuring the quality of teaching and assessment;
9.3. ensuring the quality of learning outcomes;
9.4. ensuring the quality of students' work.
10. What difficulties did the HEI, which you lead or work in, face in the process
of implementing the internal system of quality assurance of education and/or its
elements?
11. Please, name the main documents that regulate the functioning of the
internal quality assurance system of education and its individual elements in the
HEI, which you head or in which you work (regulations, orders of the rector, etc.
(please specify)?
12. Has the HEI which you lead or in which you work, received a certificate
certifying the compliance of the quality management system with the
requirements of the international standard ISO 9001: 2015?
13. Do you carry out preparatory work to obtain a certificate proving compliance
of the quality management system with the requirements of the international
standard ISO 9001: 2015?
14. Is there a separate structural unit in the HEI that you head or work in that is
responsible for managing the quality of education?

390
1. Yes (Please indicate the name of the unit) (Please indicate the title of its
head) (Indicate if this is a separate position) (Additional function of a certain
administrator)
2. No.
15. Are the functions of quality management of education assigned to certain
structural units of the HEI that you head or in which you work?
16. How were the interests and opinions of stakeholders / stakeholders studied
when reviewing the set of educational programs (specialties)? (Please give one
answer in each line):
16.1. students;
16.2. entrants;
16.3. faculty;
16.4. HEI administration;
16.5. management bodies of the HEI;
16.6. students’ parents / entrants’ parents ;
16.7. employers ;
16.8. professional, public associations, unions, agencies, etc.
17. What tools are used to audit / evaluate the quality of existing educational
programs? (Please give one answer in each line):
(Systematically / Periodically / Not implemented / There was no need for
implementation)
17.1. Ongoing surveys of students on the quality of educational programs in
general;
17.2. Survey of students on the level of their satisfaction with the content of
curricula;
17.3. Survey of employers on the quality of existing educational programs;
17.4. Creating an expert panel of professionals-practitioners in this field;
17.5. Evaluating the content of plans / programs with the help of graduates
of this specialty / educational program;
17.6. Comparison of the content of educational programs with similar
programs of Ukrainian HEIs;
17.7. Comparison of the content of educational programs with similar
programs of foreign HEIs.
18. What activities are carried out in the HEI, which you head or in which you
work, based on the results of the audit? (Please give one answer in each line)?
implementation)
18.1. removal of unnecessary disciplines
18.2. introduction of new compulsory disciplines
18.3. transfer of disciplines to another course
18.4. expansion of the list of elective disciplines
18.5. creation of new educational programs
19. Is there an audit / evaluation of the quality of teachers' work in the HEI that
you head or in which you work?
1. Yes 2. No.
20. What tools are used in the audit / evaluation of the quality of teachers'
work? (Please select one answer in each line)?
20.1. ongoing surveys of students on the quality of work of specific teachers;

391
20.2. conducting open classes with their subsequent evaluation by teachers

and administrators;
20.3. conducting final control in the discipline (test/exam) by another
teacher;
20.4. rectoral tests;
20.5. assessment of the volume and quality of the educational and
methodological support of the discipline developed by the teacher
(textbooks / manuals, workshops, tests, etc.);
20.6. evaluation of scientific activity of the teacher (number of scientific
publications, participation in scientific conferences, etc.) on issues of quality
assurance of education;
20.7. assessment of extracurricular activity of the teacher (conducting
competitions and contests, tutorials on students’ research, etc.).
21. What actions are taken based on the results of the audit of teaching quality?
(Please give one answer in each line):
(Systematically / Periodically / Not implemented)
21.1. interview with the head /rector;
21.2. publication of teachers’ ratings
21.3. wage differentiation depending on the results of the quality audit;
21.4. awarding the best teachers;
21.5. obliging the teacher to pass advanced training courses, trainings, etc.;
21.6. dismissal of a teacher.
22. Is there an audit / evaluation of learning outcomes in the HEI you lead or
work in?
1. Yes.
2. No.
23. What tools are used in the audit of learning outcomes? (Please give one
answer in each line)?
(Systematically / Periodically / Not implemented)
23.1. Carrying out internal measurements of the level of competence
formation (rector's control, internal testing, internal reviewing, etc.)
23.2. Carrying out external measurements of the level of formation of
competencies (external review of the course / diploma, tests, etc.)
23.3. Receiving feedback from graduates on the application of the level of
competencies
23.4. Receiving assessment from organizations and enterprises where
graduates work, by means of feedback.
24. What measures are taken as a result of the audit of the quality of acquired
professional (special) competencies of graduates of the institution? (Please give
one answer in each line)?
implementation)
24.1. Considering to keep the educational program (speciality)
24.2. Analysis of the staff that provides teaching in this educational program
(speciality)
24.3. Replacement of the head (curator) of the program (speciality)
25. What are the procedures for reviewing student complaints in the HEI that
you head or in which you work?

392
26. What documents regulate the procedures for reviewing the complaints of
students in the HEI, which you head or in which you work?
27. Can the students’ complaints in the HEI, which you head or in which you
work, result in:
27.1. correction of the content of the educational program
27.2. correction of the content of the curriculum
27.3. replacement of a teacher
27.4. transferring the discipline from the curriculum (one year / several
years later / earlier)
28. Does your HEI use an electronic environment for student learning
(MOODLE, services and information tools, etc.)? (Please select one answer)?
1. Yes.
2. Not yet, but we are at the stage of choosing a system and its
implementation.
3. No.
Date___________________________
Responsible for the survey:

Vice-Rector for Academic Affairs: Svitlana Gordiichuk.
Contact phone: (097) 2887464, e-mail: stepanovasvg77@gmail.com

393

Vol. 19, No. 8, pp. 393-408, August 2020
Use of Augmented Reality to Improve Specific

and Transversal Competencies in Students
Esteban Vázquez-Cano
National University of Distance Education, Madrid, Spain
https://orcid.org/0000-0002-6694-7948
Verónica Marín-Díaz
University of Cordoba, Córdoba, Spain
https://orcid.org/0000-0001-9836-2584
Wellington Remigio Villota Oyarvide

Santiago de Guayaquil Catholic University, Guayaquil, Ecuador
https://orcid.org/0000-0002-0081-4704
Eloy López-Meneses
Pablo de Olavide University, Seville, Spain
https://orcid.org/0000-0003-0741-5367
Abstract. Work in higher educational centres implicates the use of

different types of methodology. The digital society demands students
with a digital competence, and it is the responsibility of the university
institution to ensure they succeed in obtaining it. This article presents an
innovative immersive experience developed in a seminar on emerging
technologies with 231 university students, and records their perceptions
of some augmented reality apps used on the Social Education degree
course and the Social Work and Social Education joint degree course at
Pablo de Olavide University in Seville (Spain) in the academic years
2016-2018. A qualitative exploratory descriptive study was used to
analyse the experience. The study presents the most important findings
of this experience with augmented reality at the university, and
proposes some didactic uses for the most interesting of these apps, with
attention to their usability, ease of use and capacity to help early-stage
learners to acquire new knowledge. Students considered that AR apps
would provide educational scenarios that were more stimulating,
collaborative and interactive, and would foment a more open type of
education (N= 159/68.8%). The principal conclusion is using this
technology in the psychosocial treatment of problems could help social
area professionals to sharpen their competences and, at the same time,
reinforce support and improve the conditions and treatments of aspects
identified in these at risk groups.
Keywords: Augmented reality; Specific and transversal competences;

University students; Digital competence; App

394
1. Introduction
The development of specific and transversal competencies in higher education is
one of the fundamental objectives of the European Credit Transfer and
Accumulation System (ECTS). This type of training approach in higher
education subjects allows us to deal with things in a transversal and
interdisciplinary way, such as content and skills for treatment, creation and
dissemination, as well as the reflections and conclusions derived from the entire
teaching-learning process. In this context, Augmented Reality (AR) emerges as a
catalyst tool for content and competencies with great didactic potential. With
AR, we can design more enriched educational environments that bring the
student closer to realities and situations in an immersive and video-simulated
way, allowing them a greater understanding and reflection in inquiry in their
learning. For this, it is necessary for teachers to acquire new roles that would
enable didactics and evaluation under the new learning scenarios: tutor,
counsellor, designer of mediated learning situations, evaluator, and content
creator, among others. In this research, we propose and analyse new ways of
dealing with content and competences in a combined way with the use of
different augmented reality tools in higher education through the analysis of an
educational experience at the Pablo de Olavide University (Seville / Spain).
2. Augmented technology: new possibilities for innovation in education

The technological imperative in education mirrors a more general evolution and
digitalisation in society and the need to acquire new competences (Kettil, 2019).
Today, the use of information and communication technologies (ICT) in
education facilitates the shared creation of knowledge through learning
communities (Vazquez-Cano, León-Urrutia, Parra-González & López-Meneses,
2020). For its part, the European Commission (2018) has long considered that
young citizens must possess certain key competencies to prepare them for adult
life, to enable them to be active participants in society and to encourage
continuous learning throughout their lifetimes. Digital competence is vital for
activating today’s citizens and must be adopted by all education systems across
all areas (curricula, resources and support for training, updating competences by
continuous learning, teacher training, equality, special needs, educational
policies…).
A connectivity and ubiquity model of online learning could also foment a digital
attitude based on sharing, constructing and exchanging digital resources that
places collective intelligence at the service of knowledge and the enrichment of
the educational community (Moreno, Leiva & López-Meneses, 2017;
Rauschnabel, Felix & Hinsch, 2019). Technology is driving constant continuous
transformation in people’s lives, in the way they search for information, interact
with others and generate content, as well as in resolving everyday problems;
and technology used in education provides benefits and possibilities that affect
traditional content transmission methods (López-Belmonte, Pozo, Morales-
Cevallos & López-Meneses, 2019).
At the same time, a wide range of technologies is emerging in education, such as

augmented reality, which is having a major impact on this field especially in

395
higher education (Barroso & Gallego, 2017; Fernández, 2017; López-Belmonte et

al., 2019; Moreno & Leiva, 2017).
The Horizon Reports of 2015 and 2016 predicted that AR would be used
extensively in education (3 to 5 years) in the future. Different authors and
reports (Barroso & Gallego, 2017; Fernández, 2017; López-Belmonte et al., 2019;
Moreno & Leiva, 2017) have stated that AR would be potentially adopted in the
next years in all educational stages. What is more, this emerging technology is
spreading thanks to the use of mobile digital devices that enable everyone to
access AR (Aznar-Díaz, Romero-Rodríguez & Rodríguez-García, 2018; Blas,
Vázquez-Cano, Morales & López, 2019). Several authors (Bursalia & Yilma, 2019;
Cabero & Barroso, 2016; Nadolny, 2017; Villalustre, 2020) have shown how AR
represents the environment that integrates the real and the virtual worlds,
combining digital and physical information in real time through technological
devices. Augmented technology in education has huge potential, such as its
application in the various learning stages (Bacca, Baldiris, Fabregat, Graf, &
Kinshuk, 2014; Garay, Tejada & Maiz, 2017) its capacity to transfer experiences
to different areas of education work with collaborative and constructivist
methodologies (Blas et al., 2019; Cochrane, Narayan & Antonczak, 2016) create
simulated scenarios (Fabregat, 2012) enrich the use of printed material with a
range of resources (Moreno & Leiva, 2017), and transform the student into a
technology designer (Cabero & Barroso, 2016).
This immersive technological scenario also facilitates the creation of a

constructivist educational context, and invigorates active teaching
environments. Jee, Lim, Youn, and Lee (2014) stated that the tools traditionally
used in the classroom can now be used in tandem with AR to draw daily reality
closer to education, make learning more interesting and stimulating, and more
relevant to students’ everyday lives. Cabero, Leiva, Moreno, Barroso and López-
Meneses (2016) pointed to AR’s potential to activate students’ cognitive learning
processes, develop cognitive and spatial skills regardless of age and academic
level, and provide learning scenarios that are more motivating, collaborative and
interactive.
RA is an educational instrument that allows students to be absorbed in

immersive environments in which the simulated context is mixed with real
objects and supports that certain more abstract contents could be explained and
visualised in a more visual and enriched form. AR was first introduced as a
training tool for airline and air force pilots during the 1990s (Caudell & Mizell,
1992). Since Tom Caudell coined the term augmented reality in the early 1990s
(Lee, 2012), this technology has experienced a great application in educational
settings.
According to Dunleavy, Dede and Mitchell (2009), AR's most significant

advantage is its “unique ability to create immersive hybrid learning
environments that combine digital and physical objects, thereby facilitating the
development of processing skills such as critical thinking, problem solving, and
communicating through interdependent collaborative exercises” Augmented

396
reality has a wide variety of fields of applications, such as art (Amakawa &
Westin, 2018), maths (Cai et al., 2019), medicine (ChanLin, ChiChan & Wan,
2019), tourism (Kourouthanassis, Boletsis, Bardaki & Chasanidou, 2015),
entertainment or education (Villalustre, 2020). In this line and according to
Chiang, Yang, and Hwang (2014a), AR technology provides relevant
information, guidance to the students and supports motivation. On the other
hand, this method is also perceived by students as more satisfying than
classroom lessons (Marín-Díaz, 2017b; Villalustre, 2020). By displaying virtual
elements alongside real objects, AR facilitates the observation of events which
cannot easily be observed with the naked eye. Among other reasons, this is due
to the ease of accessing information offered by this tool, as it is normally
accessed by way of mobile devices (Murat & Gökçe, 2017). In this sense, its
combination with apps on mobile digital devices, such as a smartphone, allows
teachers and students to have an educational tool inside and outside the
classroom with which to simulate different educational experiences (Vázquez-
Cano & Sevillano-García, 2018). As Chang, Wu, and Hsu (2013) and Jee et al.
(2014) establish, AR could enhance students’ motivation and promote situated
learning, which may, in turn, result in students making more informed
decisions.
In short, its use in education, as pointed out by different authors (Chang &
Hwang, 2018; Pejoska-Laajola, Reponen, Virnes & Leinonen, 2017; Rauschnabel,
He & Ro, 2018), could provide different possibilities such as: a) To identify
relevant information and avoid information that hinders the assimilation of
significant information; b) Locate in reality the fundamental constitutive
elements that allow the student to understand how it works; c) Being able to
observe an object from different points of view and from different perspectives;
d) To promote ubiquitous and mobile learning; e) Be able to have simulated and
safe laboratory practices; f) Provide students with diverse learning systems that
complement other, such as audiovisual and printed proposals; g) Turning the
student not only into a content viewer, but also in the creation of proposals
based on augmented reality; h) Complement other didactic approaches such as
the Flipped Classroom methodology. AR employment outside the formal
educational area can provide the citizen, in general, with great educational and
entertainment possibilities (experiments, museums, etc.).
However, some authors (Cai, Liu, Shen, Liu, Li & Shen, 2019; Han, Jo, Hyun &
So, 2015; Fernandez-Robles, 2018; Marín-Díaz, 2017a; Villalustre, 2020) stress
AR’s limitations: teachers’ shortcomings in technological competence, the lack of
resources and the failure to provide clear objectives for learning with AR.
However, the limitations or disadvantages of AR are not the objective of this
article(). The principal aim is to detect the possibilities of using AR in mobile
applications with university students as this could improve or achieve the
competencies needed for the final grade. This answers the hypothesis of the
principal project that this article proposes: the use of AR with students will
allow to detect the needs of students by using it to improve the student’s
competencies and create social attitudes for professional activities.

397
2. Materials and method

This is a qualitative exploratory descriptive study. The methodological concept
is based on a virtual collaborative investigation-action performed by university
students to foment their generic and specific competences in the European
Higher Education Area (Bogdan & Biklen, 1992; Pool-Cibrian & Martínez-
Guerrero, 2013). One of its objectives was to know the most useful and
stimulating AR apps for educational purposes among those used in a seminar on
immersive technologies in the academic years 2016-2017 and 2017-2018 for the
university degree course in Social Education and the joint degree in Social
Education and Social Work.
The qualitative analysis was based on a coding and categorisation process in two
stages, descriptive and interpretative. The procedure was organised in three
phases: Phase 1: ’Segmentation and identification of the units of meaning, and
grouping in descriptive categories’; Phase 2: Construction of a system of
emerging thematic nuclei and metacategories’; Phase 3: ‘“Identification of
qualitative dominions (sequential and transversal analysis of the
metacategories’.
2.1 Objectives
The objectives were: 1) To know the new AR apps in educational settings, and
the pedagogical potential of these AR apps in learning contexts. 2) To generate
proactive attitudes in students towards augmented technologies. 3) To develop
students’ competencies in the use of these AR apps in didactic settings.
2.2 Procedure
This innovative immersive experience was developed in a seminar on emerging
technologies in January and February 2016-2017, and again in 2017-2018. The
two courses were Information Technologies and Communication in Social
Education, both of which formed part of the first-year curriculum of the degree
course in Social Education and the joint degree course in Social Education and
Social Work, in the Faculty of Social Sciences at Pablo de Olavide University in
Seville.
The AR apps used in the practical sessions with the students in the seminar
were:
• Anatomy 4D. This AR app enables the student to explore inside the human
body on a virtual journey to study the body’s organs and systems (see Figure
1).
• Quiver. This is an augmented reality- and virtuality-based app that enables
the user to colour printed sheets. These sheets can be photographed on any
mobile device and then transformed, as the students generate augmented
scenarios that enhance their learning processes (see Figure 2).
• Chromville. This app is similar to Quiver, with the same dynamic of printed
sheets for colouring and immersive technology. The sheets act as markers for
the creation of augmented fantasy settings via the mobile device’s camera
(see Figure 1).

398
• Zookazam. With this app, the user can select from a wide range of animal
species from our real habitat to create scenes from fables. It is similar in
dynamic to the previous two and is available from Apple’s app store and
Google Play (see Figure 1).
2.3 Sample
The sample consisted of 231 students attending Pablo de Olavide University in
Seville, with the following characteristics: 60 students (8 men, 58 women) in
2016-2017, and 58 students (5 men, 53 women) in 2017-2018, studying ICT and
Social Education as part of their first-year joint degree course in Social Education
and Social Work; 57 students (6 men, 51 women) in 2016-2017, and 56 students (4
men, 52 women) in 2017-2018, studying ICT and Social Education as part of their
first-year degree course in Social Education.
The possible existence of a bias in the population with regard to the gender
variable could be detected, but as Cheng and Chang (2006) and Gialamas,
Nikiolopoulu, and Koutromanos (2013) point out, studies in the field of social
sciences present an eminently feminised profile, affecting this variable
worldwide, that is why we consider the non-existence of said bias in our sample
2.4 Instruments
At the end of February in both years, the students completed a questionnaire,
The didactic use of AR, available at: https://goo.gl/forms/STik3sI9KdPzZi773)
on challenges and didactics uses of AR. The questionnaire design was based on
the theoretical considerations discussed by Barroso and Gallego (2017). To
design the questionnaire, we have used a modified version, the two-round
‘Modified Delphi’. In its implementation, we must pay special attention to a
series of aspects: ensuring the anonymity of the participants, and, more
specifically, their answers; using different iterations; establishing feedback
control by the coordinating group; and using statistical techniques in the
analysis of the responses (Rowe & Wright, 1999).
To implement the Delphi method, five phases were applied:

1. Drafting of the first list of topics, with the descriptors that could be included.
2. First round of the Delphi study.
3. Analysis of the results obtained and the drafting of a new list.
4. Second round of the Delphi study.
5. Analysis of the results obtained and drafting of the validation scale.
Twenty-one participants were asked to evaluate them according to a scale of 1 to
5, where 1 is equivalent to not at all important and 5 is very important. They
were also asked to give their opinion on a series of aspects: a block of contents
that they would eliminate or include or any items they would suggest
eliminating, including or modifying within the proposed blocks of contents. The
final mean and standard deviation of the dimensions of the questionnaire are
shown in Table 1.

399
Table 1: Mean and standard deviation of the dimensions of the questionnaire.

Questionnaire dimensions M SD
Augmented Reality: app characteristics 4.30 0.73
Augmented Reality: educational functionalities 4.08 0.65
Augmented Reality: advantages and limitations 4,21 0.64
One of the aims of the survey was to collate the students’ opinions on the most
useful and stimulating AR apps used in the immersive technology seminar. Our
study specifically analysed the impressions of the 231 students of the AR apps
used in the classroom, and their responses to an open question on the
advantages of the didactic use of AR at university (see images 1 and 2).
Image 1. Students using Augment and Anatomy 4D
Image 2. Students using Quiver

400
4. Results
This section presents the results of the analysis and interpretation of the
contributions by the 231 students on the Social Education degree course and the
Social Work and Social Education joint degree course between 2016-2018 with
regard to the most useful and stimulating AR apps that they experimented with
during the practical sessions of a seminar on emerging technologies.
According to the students on the Social Work and Social Education joint degree
course in 2016-2017, the best AR apps to apply to significant learning processes
were: Quiver (45%); Zookazam (23%), Chromville (17%) and Anatomy 4D (15%).
The majority of students stated that Quiver was the most interesting from a
didactic point of view, as it was easy to use, and highly intuitive and appropriate
for young learners, enabling them to colour in the sheets then convert them into
animated objects. Second-placed Zookazam was also easy to use and fast and
helped students to learn about a wide range of animals. The students on the
Social Education degree course (2016-2017) who tried out the AR apps during
the emerging technologies seminar classified them for their motivational
qualities as follows: Quiver (44%), Zookazam (23%), Chromville (14%) and
Anatomy 4D (19%) (see images 1 and 2).
They stated that Quiver and Zookazam were the most interesting and
educational for ease of use, usability and user-friendliness, as well as their clear
orientation to educational processes. The results for the students on the Social
Work and Social Education joint degree course in 2017-2018 were similar to
those of their counterparts a year earlier for Quiver (45%) and Zookazam (24%).
However, they differed from the previous year by placing Anatomy 4D in third
place ahead of Chromville, at 17% and 14%, respectively.
The students on the Social Education degree course in 2017-18 scored Quiver at
43% and Zookazam at 21%, as the most interesting for use in a socio-educational
setting due to their usability and applicability to a range of contexts for young
learners.
Figure 1 presents the results of the frequencies for 2016-2018 in order to observe
the oscillations about the most useful and innovative AR apps for educational
processes that were tested by the participants in the emerging technologies
seminar.

401
Figure 1. Comparison of frequencies for students at the emerging technologies

seminar in 2016/17 and 2017/18.
It can be inferred that the students in both years who experimented with the AR
apps at the emerging technologies seminar, as part of the course on ICT and
Social Education, considered Quiver and Zookazam to be the best in terms of
usability and ease of use, and were deemed the most appropriate for early
learners to acquire new knowledge.
Students on both degree courses considered that these AR apps would provide
educational scenarios that were more stimulating, collaborative and interactive,
and would foment a more open type of education (N= 159/68.8%). Similar
results were found in teacher training for primary education (Moreno & Leiva,
2017; Nielsen, Brandt & Swensen, 2016; Villalustre, 2020). The motivation to
create AR apps for educational processes has also been cited in diverse studies
(Chiang, Yang & Hwang, 2014a; Cochrane, Narayan & Antonczak, 2016) and
follows the same line as these results. Nevertheless, both studies considered that
the focus of the students’ attention should never be on how well they can handle
the technology, but on the didactic function at the heart of that technology, in
order for such apps to be applied correctly in both the learning and professional
context (Cabero-Almenara, Vázquez-Cano & López-Meneses, 2018; Rahman,
Ling & Yin, 2020).
The results from the students’ perceptions on the relation between the
advantages and disadvantages of AR for the educational environment can be
observed in the Atlas-Ti network in Figure 2.

402
Figure 2. Atlas-Ti network. Advantages of AR applied to university student learning
5. Discussion
Firstly, the present study’s findings are in line with those in other research
(Aznar-Díaz et al., 2018; Cabero, Llorente & Gutiérrez-Castillo, 2017; Cozar, de
Moya, Hernández & Hernández, 2015; Villalustre, 2020), in that the students
reacted favourably towards the use of this type of emerging technologies
because they considered them to be good motivational tools for learning (Chiang
et al., 2014). AR apps also foment a proactive environment in teaching
(Rauschnabel et al., 2019) and produce a high level of satisfaction among
students (Chen, 2019). Likewise, as other investigations have testified (Barroso &
Gallego, 2017; Cabero et al., 2018), AR is useful for developing emerging
competences in ICT, strengthening group work, and for discovering new useful,
immersive didactic resources in social education and social work scenarios that
were previously unknown to the majority of the students; this can generate new
educational processes based on an investigative, constructivist and ubiquitous
perspective. In this sense, we wish to emphasise that AR-based didactic activities
are especially useful for pre-university and degree-level education, particularly
in the Humanities and Social Sciences, because they allow students to access
content in a different way to that frequently presented in a one-directional
printed form that does nothing to motivate them or raise their ability for abstract
thinking on the subject. AR apps allow content to be visualised in a way that is
more creative, dynamic and real, which makes the teaching of a subject more
attractive, innovative and stimulating for students at all levels of education
(Marin-Díaz, 2017a,b; Moreno & Leiva, 2017; Villalustre, 2020).

403
The analysis of the AR didactic experience with the university students provided
us with an initial conclusion that the use of objects based on augmented
technology in higher education arouses considerable interest in students. Our
study is in line with others that corroborate a high degree of satisfaction among
students who use this technology and demonstrate a considerable increase in
motivation when they are immersed in educational activities involving AR (Yip,
Wong, Yick, Chan & Wong, 2019).
In line with Fernández-Robles (2018), AR can be presented as a genuinely useful

technology for training students at university, allowing them to work with
active constructivist methodologies, providing them with the opportunity to
visualise objects from different perspectives, facilitating the acquisition of
knowledge that is difficult to access, enabling the presentation of simulated
scenarios and enriching traditional printed material (Moreno, Franco-Mariscal &
Franco-Mariscal, 2018). This aspect could be very useful for students with any
kind of disabilities. People with disabilities that negatively impact their social
skills can use virtual reality to practise and improve their social skills (Bridges,
Robinson, Stewart, Kwon & Mutua, 2019; Cascales-Martínez, Martínez-Segura,
Pérez-López & Contero, 2017).
At the same time, the opinions of the Social Sciences students in this study show
that the use of AR-based activities generates deeper reflection and a more
positive attitude, and boosts the cognitive processes applied to the content in the
Social Education and Social Work degree courses that they were taking. Also, it
was interesting to see how students not only found AR to be applicable to the
teaching-learning process, but also discovered how useful it was in the
development of their own professional competencies as educators and social
workers. In terms of applying AR as professionals in education and social work,
the students emphasised that AR could be a particularly interesting tool in the
treatment of drug addiction and its prevention among teenagers, in managing
dementia-related disabilities in older people, and in the education of infants and
adolescents within formal and informal educational settings. As Lindsey Getz
(2018, p.6) points out, authors such as Patrick Bordnick are “currently exploring
the ways in which we are training the next generation of social workers to
prepare for disaster scenarios. If they never have any previous exposure, it's
quite possible that they will present with anxiety when exposed to extreme
situations or from a social work perspective, imagine the possibilities involved
when students get an opportunity to interview a family and assess their needs
post trauma—all in a virtual world.”
on the other hand, without adequate teacher training in emerging technologies,

their use in the classroom could even be counterproductive, acting as a brake on
the development of competencies and content in the curriculum (Barroso &
Gallego, 2017; Blas et al., 2019; Marín-Díaz, 2017a,b). Likewise, it is important to
be mindful of the resources available to educational institutions and students in
order not to widen the digital divide or discriminate against students for lack of
finance or access to technology. It is also important to ensure that this
technology can be used for, and adapted to, the various special needs of students

404
in the classroom. In this sense, the ease of use of virtual reality applications on
smartphones favours the development of immersive learning spaces and
provides access (Fombona, Vázquez-Cano & Del Valles, 2018). What seems more
important than the economic cost is the need to train teachers and professors in
the new digital skills with this technology in order to take advantage of the
educational and social potential of augmented reality applications to develop
generic and transversal competencies among students (Blas et al., 2019; Marín &
Sampedro-Requena, 2019).
6. Conclusions
The principal conclusion of this research is that using this technology in the
psychosocial treatment of these problems could help professionals to sharpen
their competences and, at the same time, reinforce, support and improve the
conditions and treatments of aspects identified in these at risk groups. In
addition, AR was found to be useful for building emerging competences in ICT,
for bolstering group work and for discovering new immersive didactic
resources in social education and social work settings that the majority of
students had not previously known about; such resources could help develop
new educational processes from an investigative, constructivist and ubiquitous
perspective. In this sense, the study emphasises that AR-based didactic activities
can be particularly useful in pre-university and higher education settings in the
fields of the Humanities and the Social Sciences, since they allow students to
access content that improves on the one-directional printed form that often fails
to stimulate or encourage students to think in a more abstract way about the
subject. AR resources enable students to visualise content in a form that is more
creative, dynamic and real, and teachers to present a didactic that is more
attractive, innovative and motivational at all levels of education.
Despite these apparent benefits, we believe it is vital to go further in generating

proposals and models for AR integration, with the development of AR firmly
grounded in the educational context, taking into account subject areas and
resources available, as well as the extent of teacher training in such technologies.
7. Limitations
This study has been approached from a qualitative approach of descriptive
research in which the main purpose has been to know the opinion and
perception of the university students about the educational possibilities of
augmented reality for the development of content and competences in higher
education. Future studies should contemplate the evaluation of the academic
performance of students with the use of statistical inference techniques.
8. Acknowledgement
This study is part of the Project: Design, implementation and evaluation of mixed
reality materials for learning environments (PID2019-108933GB-I00) funded by the
Spanish Ministry of Science and Innovation.

405
9. References
Amakawa, J., & Westin, J. (2018) New Philadelphia: using augmented reality to interpret
slavery and reconstruction era historical sites. International Journal of Heritage
Studies, 24(3), 315-331. https://doi.org/10.1080/13527258.2017.1378909
Aznar-Díaz, I., Romero-Rodríguez, J. M., & Rodríguez-García, A. M. (2018). Virtual
Reality mobile technology in education: a review of the state of scientific
literature in Spain. EDMETIC, Revista de Educación Mediática y TIC, 7(1), 256-274.
https://doi.org/10.21071/edmetic.v7i1.10139
Bacca, J., Baldiris, S., Fabregat, R., Graf, S., & Kinshuk, J. (2014). Augmented Reality
Trends in Education: A Systematic Review of Research and Applications.
Educational Technology & Society, 17(4), 133-149.
Barroso, J., & Gallego, O. (2017). Learning resource production in Augmented Reality
supported by education students. EDMETIC, Revista de Educación Mediática y
TIC, 6(1), 23-38. https://doi.org/10.21071/edmetic.v6i1.5806
Blas, D., Vázquez-Cano, E., Morales , M., & López, E. (2019). Use of augmented reality
apps in University classrooms. Campus Virtuales, 8(1), 37-48.
Bogdan, R., & Biklen, S.K. (1992). Investigación Cualitativa de la Educación. Needham
Heights, MA: Allyn and Bacon.
Bridges, S. A., Robinson, O. P., Stewart, E. W., Kwon, D., & Mutua, K. (2019).
Augmented reality: Teaching daily living skills to adults with intellectual
disabilities. Journal of Special Education Technology, 35(1), 3-14.
https://doi.org/10.1177/0162643419836411
Bursalia, H., & Yilma, R. M. (2019). Eﬀect of augmented reality applications on
secondary school students' reading comprehension and learning permanency.
Computers in Human Behavior, 95, 126-135.
https://doi.org/10.1016/j.chb.2019.01.035
Cabero, J., & Barroso, J. (2016). The educational possibilities of Augmented Reality.
Journal of New Approaches in Educational Research, 5(1), 46-52.
https://doi.org/10.7821/naer.2016.1.140
Cabero, J., Leiva, J. J., Moreno, N., Barroso, J., & López-Meneses, E. (2016). Realidad
Aumentada y Educación. Innovación en contextos formativos. Octaedro: Barcelona.
Cabero, J., Llorente, M. C., & Gutiérrez-Castillo, J. J. (2017). Evaluation by and from
users: learning objects with Augmented Reality. RED, Revista de Educación a
Distancia. 53, Artic. 4. http://dx.doi.org/10.6018/red/53/4
Cabero, J., Vázquez-Cano, E., & López Meneses, E. (2018). Use of Augmented Reality
Technology as a Didactic Resource in University Teaching. Formación
Universitaria, 11(1), 25-34. https://doi.org/10.4067/S0718-50062018000100004
Cabero-Almenara, J., Barroso-Osuna, J., Llorente-Cejudo, M. C., & Fernández-Martínez,
M. (2019). Educational uses of Augmented Reality (AR): Experiences in
Educational Science. Sustainability, 11(18), 4990.
https://doi.org/10.3390/su11184990
Cai, S., Liu, E., Shen, Y., Liu, Ch., Li, S., & Shen, Y. (2019). Probability learning in
mathematics using augmented reality: impact on student’s learning gains and
attitudes. Interactive Learning Environments,
https://doi.org/10.1080/10494820.2019.1696839
Cascales-Martínez, A., Martínez-Segura, M.-J., Pérez-López, D., & Contero, M. (2017).
Using an augmented reality enhanced tabletop system to promote learning of
mathematics: A case study with students with special educational needs. Eurasia
Journal of Mathematics, Science and Technology Education, 13(2), 355–380.
https://doi.org/10.12973/eurasia.2017.00621a

406
Caudell, T. O., & Mizell, D. W. (1992). Augmented reality: An application of heads-up

display technology to manual manufacturing processes. Proceedings of the
Twenty-Fifth Hawaii International Conference on System Sciences, 2, 659-669.
10.1109/HICSS.1992.183317
Chang, H., Wu, K., & Hsu, Y. (2013). Integrating a mobile augmented reality activity to
contextualize student learning of a socio scientific issue. British Journal of
Educational Technology, 44(3), 95-99. https://doi.org/10.1111/j.1467-
8535.2012.01379.x
Chang, S., & Hwang, G. (2018). Impacts of an augmented reality-based flipped learning
guiding approach on students’ scientific project performance and perceptions,
Computer & Education, 125, 226–239.
ChanLin, L., ChiChan, K., & Wan, Ch. (2019). An epistemological assessment of learning
nutritional information with augmented reality. The Electronic Library, 37(2), 210-
224. https://doi.org/10.1108/EL-06-2018-0128
Chen, Y. (2019). Effect of Mobile Augmented Reality on Learning Performance,
Motivation, and Math Anxiety in a Math Course. Journal of Educational
Computing Research, 57(7), 1695-1722.
https://doi.org/10.1177/0735633119854036
Chiang, T. H., Yang, S. J., & Hwang, G. (2014). An augmented reality-based mobile
learning system to improve students' learning achievements and motivations in
natural science inquiry activities. Journal of Educational Technology & Society.
17(4), 352-365
Cochrane, T., Narayan, V., & Antonczak, L. (2016). A Framework for Designing
Collaborative Learning Environments Using Mobile AR. Journal of Interactive
Learning Research, 27(4), 293–316.
Cozar, R., de Moya, M., Hernández, J., & Hernández, J. (2015). Emerging Technologies in
Social Sciences Teaching. An Experience Using Augmented Reality in Teacher
Training. Digital Education Review, 27, 138-153.
Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive
participatory augmented reality simulations for teaching and learning, Journal of
Science Education and Technology, 18(1), 7-22. https://doi.org/10.1007/s10956-
008-9119-1
European Commission (ed.) (2018). Proposal for a council recommendation on key
competences for lifelong learning. Brussels. Retrieved from https://bit.ly/2YsyGNz
Fabregat, R. (2012). Combining Augmented Reality and Adaptive E-Elearning Platforms.
Enl@ce Revista Venezolana de Información, Tecnología y Conocimiento, 9(2), 69-78.
Fernández-Robles, B. (2018). The use of objects of learning of augmented reality in the
universityeducation of primary education. International Journal of Educational
Research and Innovation (IJERI), 9, 90-104.
Fombona Cadavieco, J., & Vázquez-Cano, E. (2017). Posibilidades de utilización de la
Geolocalización y Realidad Aumentada en el ámbito educativo. Educación XX1,
20(2), 319-342. https://doi.org/10.5944/educxx1.19046
Fombona Cadavieco, J., Vázquez-Cano, E., & Del Valle Mejías (2018). Análisis de la
geolocalización y realidad aumentada en dispositivos móviles, propuestas socio-
educativas relacionadas con el entorno y las salidas de campo. Profesorado.
Revista de Currículum y formación del profesorado, 22(4), 197-222.
https://doi.org/10.30827/profesorado.v22i4.8413
Fombona, J., & Vázquez-Cano, E. (2017). Possibilities of using geolocation and
augmented reality in education. Educación XX1, 20(2), 319-342.
https://doi.org/10.5944/educXX1.10852

407
Garay, U., Tejada, E., & Maíz, I. (2017). Valuation of the Educational Objects with
Augmented Reality: Experience with Posgraduates Students. Píxel-Bit. Revista de
Medios y Educación. 50, 19-31. https://doi.org/10.12795/pixelbit.2017.i50.01
Getz, L. (2018). Social Services Innovations: A Whole New World—Virtual Reality in
Social Work. Social Work Today, 18(2), 6.
Han, J., Jo, M., Hyun, E., & So, H. (2015). Examining young children’s perception toward
augmented reality-infused dramatic play, Education Technology Research
Development. 63, 455-47. https://doi.org/10.1007/s11423-015-9374-9
Jee, H. K., Lim, S., Youn, J., & Lee, J. (2014). An augmented reality-based authoring tool
for E-learning applications. Multimedia Tools and Applications, 68, 225-235.
https://doi.org/10.1007/s11042-011-0880-4
Ketil, B. (2019). Understanding social and cultural aspects of teachers’ digital
competencies. Comunicar, 61, 9-19. https://doi.org/10.3916/C61-2019-01
Kourouthanassis, P., Boletsis, C., Bardaki, C., & Chasanidou, D. (2015). Tourists
responses to mobile augmented reality travel guides: The role of emotions on
adoption behavior. Pervasive and Mobile Cumpoting, 18, 71-87.
https://doi.org/10.1016/j.pmcj.2014.08.009
Lee, L. (2012). Augmented reality in education and training. TechTrends, 56(2), 13-21.
https://doi.org/10.1007/s11528-012-0559-3
López Meneses, E., Leiva, J. J., & Vázquez-Cano, E. (2017). The impact of personal
interactions on the stress of school administrators: The validation and
application of an assessment tool. Intangible Capital, 13(3), 499-522. doi:
http://dx.doi.org/10.3926/ic.993
López-Belmonte, J., Pozo-Sánchez, S., & Fuentes-Cabrera, A. (2019). Techno-pedagogical
resources to support teaching: Augmented reality as a dynamic tool for the
substitute teacher. International Journal of Educational Research and Innovation
(IJERI). 12, 122-136.
Marín, V., & Requena, B. (2019). La Realidad Aumentada en Educación Primaria desde
la visión de los estudiantes. Alteridad, 15. 61-73. doi: 10.17163/alt.v15n1.2020.05.
Marín, V., & Sampedro-Requena, B. (2020). La Realidad Aumentada en Educación
Primaria desde la visión de los estudiantes. Alteridad, 15(1), 61-73.
https://doi.org/10.17163/alt.v15n1.2020.05
Marín-Díaz, V. (2017a). The relationships between Augmented Reality and inclusive
education in Higher Education. Bordón, 69(3). 125-142.
https://doi.org/10.13042/Bordon.2017.51123
Marín-Díaz, V. (2017b). The Augmented Reality in the educational sphere of student of
degree in childhood education. Case study. Pixel Bit, Revista de Medios y
educación, 51, 9-24. http://dx.doi.org/10.12795/pixelbit.2017.i51.01
Moreno, N. M., Leiva, J. J., & López-Meneses, E. (2017). La realidad aumentada como
tecnología emergente para la innovación educativa. Notandum. 44-45, 125-140..
Moreno, N., & Leiva, J. J. (2017). Formative experiences in the educational use of
augmented reality with students of primary education degree at the University
of Malaga. EDMETIC, Revista de Educación Mediática y TIC, 6(1), 81-104.
https://doi.org/10.21071/edmetic.v6i1.5809
Moreno, N., Franco-Mariscal, R., & Franco-Mariscal, A. (2018). Augmented Reality in
Chemistry: An Experience in Secondary Education Through Elements 4D.
Journal of Science Education, 19(2), 71-94.
Murat, A., & Gökçe, A. (2017). Advantages and challenges associated with augmented
reality for education: A systematic review of the literature. Educational Research
Review, 20, 1-11. https://doi.org/10.1016/j.edurev.2016.11.002

408
Nadolny, L. (2017). The design of cognitive tasks in blended augmented reality and print
documents. British Journal of Educational Technology, 48, 814-823.
https://doi.org/10.1111/bjet.12462
Nielsen, B., Brandt, H., & Swensen, H. (2016). Augmented Reality in science education-
affordances for student learning. Nordic Studies in Science Education, 12(2), 157-
174. https://doi.org/10.5617/nordina.2399
Pejoska-Laajola, J., Reponen, S., Virnes, M., & Leinonen, T. (2017). Mobile augmented
communication for remote collaboration in a physical work context. Australasian
Journal of Educatinal Technology, 33(6), 11–25. https://doi.org/10.14742/ajet.3622
Pool-Cibrian, W. J., & Martínez-Guerrero, J. I. (2013). Self-efficacy and Use of Strategies
for Self-regulated Learning in University Students. Revista Electrónica de
Investigación Educativa, 15(3), 21-37.
Rahman M. A, Ling L. S., & Yin O. S. (2020) Augmented Reality for Learning
Calculus: A Research Framework of Interactive Learning System.
Computational Science and Technology. Technology, 603, 491-499.
https://doi.org/10.1007/978-981-15-0058-9_47
Rauschnabel, P., He , J., & Ro, Yo (2018). Antecedents to the adoption of augmented
reality smart glasses: a closer look at privacy risks. Journal of Business Research,
92, 374-384. https://doi.org/10.1016/j.jbusres.2018.08.008
Rauschnabel, P. A., Felix, R., & Hinsch, Ch. (2019). Augmented reality marketing: How
mobile AR-apps can improve brands through inspiration. Journal of Retailing and
Consumer Services, 49, 43-53. https://doi.org/10.1016/j.jretconser.2019.03.004
Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool. Issues and
analysis. International Journal of Forecasting, 15(4), 353-375.
https://doi.org/10.1016/S0169-2070(99)00018-7
Vázquez-Cano, E., & Sevillano-García, M. L. (2018). Ubiquitous Educational Use of
Mobile Digital Devices. A General and Comparative Study in Spanish and Latin
America Higher Education. Journal of New Approaches in Educational Research, 7,
105-115. https://doi.org/10.7821/naer.2018.7.308
Vázquez-Cano, E., León-Urrutia, M., Parra-González, M. E., & López-Meneses, E. (2020).
Analysis of interpersonal competences in the use of ICT in the Spanish
University Context. Sustainability, 12(2), 476.
https://doi.org/10.3390/su12020476
Villalustre, L. (2020). Propuesta metodológica para la integración didáctica de la realidad
aumentada en Educación Infantil. EDMETIC, Revista de Educación Mediática y
TIC, 9(1), 170-187. https://doi.org/10.21071/edmetic.v9i1.11569
Yip, J., Wong, S., Yick, K., Chan, K., & Wong, K. (2019). Improving quality of teaching
and learning in classes by using augmented reality video. Computers & Education,
128, 88-101. https://doi.org/10.1016/j.compedu.2018.09.014

409

Vol. 19, No. 8, pp. 409-423, August 2020
How School Culture and Teacher’s Work Stress

Impact on Teacher’s Job Satisfaction
Susan Febriantina, Suparno, Marsofiyati

Universitas Negeri Jakarta, Indonesia
http://orcid.org/0000-0001-7661-8227
http://orcid.org/0000-0002-5518-0489
http://orcid.org/0000-0002-6842-6046
Rusi Rusmiati Aliyyah

Universitas Djuanda Bogor, Indonesia
http://orcid.org/0000-0002-4908-4677
Abstract. This study examined the impact of school culture and teachers’
stress on satisfaction among vocational teachers. Vocational school as a
school with the development of work-ready skills, combine learning dual
system by learning in school and work in the industry, needs to be
examined as the effectiveness of teacher abilities and overall education
policy. Teacher satisfaction in this study is regarding teachers' feeling level
of pleasure as an upbeat assessment of the work and its environment
(school). Data were collected through a survey into 142 teachers of
vocational high school in Greater Jakarta, Indonesia. Data were analysed
by path analysis to determine the effects among variables and processed by
SPSS 24. The results showed that school culture which now impacts teacher
stress equal to 1,795%, the contribution of teacher stress which is directly
teacher’s job satisfaction equal to 43,296%, the contribution of school
culture to teacher’s job satisfaction skill through teacher stress is 5,198%.
The findings are that teacher’s job satisfaction is directly influenced by the
teacher’s pressure with a dominant influence. Physical condition is the
leading indicator in developing teacher’s job satisfaction skills for school in
policy-making as well as a teacher. Simultaneously school culture and
teacher’s stress affect teacher’s satisfaction skills by 52.3% and 47.7%
influenced by other variables. This study also recommends further research
to develop variables on aspects of teacher’s competence and creativity in
measuring teacher satisfaction skills, both in direct and indirect influence.
Keywords: School culture; Teacher stress; Teacher satisfaction; Vocational

teacher
1. Introduction
The role of the teachers as one of the crucial human resources in the school
organisation will significantly determine the success of achieving the goals of the

410
school organisation. One of the main reasons for this statement is that the success
of teaching and learning at school will create a qualified human resource. The
teachers are ever considered as the most critical agents in the school system, for
they are the main actor who deals with students as an educational object (Baumert
et al., 2013). Recently, the teacher has several demands related to inside and outside
school tasks which he must carry out well. Internal functions are associated with
the role of the teacher as a designer and facilitator in the learning process, which
requires high creativity. Moreover, the students today called the millennial
generation are no less creative than their teachers in any kind of understandings of
the materials and technology (Kotz, 2016). This phenomenon can be one challenge
for teachers nowadays to create effective learning methods and approaches. The
second one is related to additional tasks which are non-academic tasks, such as
fostering extracurricular activities, attending meetings, self-upgrading through
teacher training, and helping school leaders in achieving school goals through
school accreditation which quite burdens for some teachers.
The ability of a school organisation that has a competitive advantage is also urgent
to be able to compete with other schools, both public and non-public school. In
addition to this, the school organisation must also have a good synergy among
education elements included principals (Borman & Maritza Dowling, 2008),
teachers and staff, students, parents, society, government, and another business
world (Erichsen & Reynolds, 2019). Thus, the school organisation system will run
effectively and resulting in a quality school.
On the other hand, each teacher has the characteristics and uniqueness in
determining their perceptions, views, seen, and feelings towards all their needs and
desires in the school organisation. Therefore, the school management needs to pay
attention to these conditions for the teachers having reasonable job satisfaction
(Ryan et al., 2017). To support the synergy among school elements which ultimately
also affect the achievement of school goals, the school has to increase its teachers'
satisfaction. There are three kinds of teachers in Indonesia based on status, which
perhaps will be different in their satisfaction. The first is a public teacher who is
paid by the government with a standardized government salary. In the capital city
of Indonesia, Jakarta general teachers get the highest salary based on provincial
minimum wages compared to teachers in other regions. One of the reasons is that
they received a regional performance allowance which does not receive by another
teacher in the other areas. This regional performance allowance can reach 8 to 10
million per month. Teachers in Jakarta regions have higher work pressure and
public attention than others. The second is a non-general teacher whose salary is
paid by a foundation. The wage of this kind of teacher is varied based on foundation
ability and regional minimum wage. The last is an honorary teacher whose
compensation could be paid by a school both public and non-general school, but
the number is still far from the two one. The interesting thing in this study, that we
explore teachers’job satisfaction among capital city public teachers who claimed to
get the highest salary than another. Teacher’s job satisfaction is not only related to
the compensation but also feelings, as well as promotions, and many other
indicators. It is an individual thing that has a different level of job satisfaction
regarding each adopted value system (Cirocki & Farrell, 2019). The more tasks in
their work following their assumed value, the more satisfied with their work the

411
teacher will be. Vice versa, the more functions in their careers incompatible with
their adopted value system, the more dissatisfied with the outcome the teacher will
be. The previous study explains many factors influencing teacher’s job satisfaction
such as school culture (Zavyalova & Kucherov, 2010), work stress (Rizwan,
Waseem & Bukhari, 2014) (Riaz et al., 2016) (Eichinger, 2000), principal leadership
(García Torres, 2019), work motivation (Sohail et al., 2014), work loyalty(Waqas et
al., 2014), work performance (Jalagat, 2016), compensation and empowerment
(Asmawi-, 2017), and teacher’s self-efficacy (Sun & Xia, 2018). This present article
aims to enrich the study of job satisfaction relate to school culture and teacher’s
work stress.
2.1. School Culture
Hoy (1990) states that school culture is talking about feeling, character, and
organisational ideology within school complexity. It is relevant to the
professional community, organisation learning, and trust (Louis, 2006). Some
other previous studies rarely used the phrase “school culture” which may be
partly due to organisational culture. They find that school as an education
organisation is a system within its culture concepts. Schein (2004) defines some
patterns of belief and perception of students, teachers, staff, princip al,
stakeholders, and school values as school culture. Organisational culture is
considered more applicable and related to measurement problems that
accompany it, especially when the quantitative approach used, besides it has
its roots in the field of sociology (Lee & Louis, 2019). As a systemic
organisation in education level, the school has some beliefs, ideology, and
special characters that may be different from one other school. Every school
leader needs to ensure their school culture can control school beliefs,
personalities, and habits (Gruenert & Whitaker, 2015). Thus, school culture
within their characteristics existing in schools relates to value, norm s, morals,
and behaviours based on the school itself (Deal & Peterson, 2016) .
School culture includes norms, values, beliefs, rituals, traditions that
distinguish one school from another school. This system is often formed by
what people think and how they act. The people of the school organisation
consist of the principal, teachers and staff, students, student’s parents, school
committees, and some stakeholders. School culture functions as a school
framework in acting, interacting, thinking, and feeling, including individual
discussion way, school physical appearance, and how they serve their social
standards, values, expectations, and behaviours (Guthrie & Schuermann,
2011).
Previous researchers have studied the impact of school culture on teachers ‘job
satisfaction. For example, Bellou (2010) found certain cultural traits as job
satisfaction amplifiers. Some other researchers in corporate culture agree that
organisational culture influences job satisfaction. One example is on school
culture, which will give many impacts on teacher’s job satisfaction (Belias,
Koustelios, Vairaktarakis & Sdrolias, 2015). The study about the school, as an
organisation, and its culture has not been so much recently. Hence, in terms of
the school level, this present study also hypothesized that school culture
impacts on teacher’s satisfaction.

412
2.2. Teacher’s Job Stress

Uncomfortable conditions at work may cause teachers frustrated and
depressed. They even feel angry, worry, and negative thinking about their
careers. If this happens, teachers cannot control the problems which
undoubtedly affect their job. These bad feelings among teachers are what we
call teacher job stress (Chris, 2001). Previous studies showed that many factors
cause the teacher’s job stress. One clear example is that relates to unsatisfied
salary, low promotion, unclear career, problems with principle and colleagues,
insufficient appreciation of teaching, and inadequate learning facilities (Abel
& Sewell, 1999). Teacher’s issues not only relate to the students but also
connect to partners, as well as environment and workloads. Another previous
study, Liu & Ramsey (2008) stated that teachers’ work stress is influenced by
workload and students' misbehaviour. If the teachers cannot solve their
pressure well, as an effect, they are a bad time for planning and preparing a
heavy teaching workload. Furthermore, if they still on that stress without any
solutions, perhaps it may cause some physical and physiological pain. Chaplin
(2008) explains three factors identified as teacher’s work stress are
management system, overlapping, and inadequate support. Some other factors
also reveal that inappropriate student’s behaviour, too many jobs, school rules
system, poor colleagues, and the way how school leaders give supports are
some other causes making teacher’s job stress (Wilson, 2002). Uniquely, what
founded is that several studies state about female teachers who have more
work stress than male teachers.
Klassen and Chiu (2010) who did previous research about it, stated that there
are differences between male and female teachers in facing stress during
teaching. The first difference tends to relax and ignore the problems, while the
second is becoming it into huge problems. Female teachers indicated more
detailed in designing the learning process in the classroom; they also are very
strict in conducting class management and roles. Then, when they could not
manage it correctly, most of them think these as big problems. Even though
they consider those big problems will not end when some administrative tasks
and other assignments outside of their teaching obligations increase and
accumulate. Stress in any kind of terms has impacts on job satisfaction,
included at school. Teachers’ work stress influences on their job satisfaction as
a teacher (Troesch & Bauer, 2017). Yin et al., (2019) also stated that the
expression of naturally felt emotions reduced teaching satisfaction. This study
presents the hypothesis that teacher stress impacts on teacher satisfaction.
2.3. Teacher’s Job Satisfaction

Most of the researchers used job satisfaction as the basic theory of teacher
satisfaction. Positive feelings, including positive emotions and happy for
doing the works defined as job satisfaction (Locke, 1976) said. It means that
when we think, we have feelings about what we believe. Then when we have
feelings, we think about what we feel. Regarding this statement, we can find
that teachers' satisfaction, including the intrapersonal judgment process,
which appears from positive emotions. However, this only measures the
affective condition of the teacher, nor fully measure teaching satisfaction (Ho
& Au, 2006). Job satisfaction describes the emotional state of a person

413
associated with positive and negative assessments from within the individual
of his work (Ma & MacMillan, 1999). Chiu & Chen (2005) mentioned that the
teacher’s job satisfaction derived from intrinsic and extrinsic factors, and both
of them are complex concepts. The first one relates to the level of the work,
and the second one relates to the level of job conditions, school policies, and
other factors which no relations with the job. On the other side, Demirtas (2010)
explains the teacher’s job stress as a positive emotional state resulting from
teaching experience. Not much research demonstrates the influence of both
school culture and teachers' work stress on teachers' job satisfaction at school.
However, by a separate study of two different variables, this study
hypothesizes that both school culture and teacher anxiety impact on teachers’
satisfaction. Thus, based on the model, the conceptual framework of this
present study can be seen in Figure 1
Figure 1. The Framework of The Conceptual
3. Method
This study was carried out through a survey method, using questionnaires as
the main instrument (Appendix). The population was 240 vocational teachers
in Jakarta Indonesia with 142 teachers as the sample. Both public and private
vocational teachers including in this research, and Isaac, and Michael formula
are also used to determine the significance level of 5%. This study used a
simple random sampling technique in determining the sample. The sampling
technique choosing is as regarding Chauvet & Do Paco (2018) that this
technique is for all populations with a similar opportunity. The study used a
questionnaire to collect the data, as this is a survey study which analyed by
using Confirmatory Factor Analyses (CFA).
We use the Lilliefors test, as stated by Malhotra (2009), to test the normality of
regression estimation of the data. In this test, generally, the data is normally
distributed if the significance value is higher than 0.05, while below than this
value indicates not normally distributed. We also use ANOVA to test the
significance of the coefficient of regression and its correlation. Meanwhile to
try the validity test, that each item arithmetic should larger than 0.30, we use
the Product Moment correlation between Pearson and the reliability test of
Cronbach’s’ Alpha, that each item should be higher or same as 0.5 (Hair et al.,
2010; Eisingerich & Rubera, 2010). To analyse the impact of patterns among
endogenous variables, this study uses path analyses. According to Hair et al.,
(2010) when p-value < 0.05, the independent variable will be significantly
impacted by the dependent variable.

414

In Table 1, we can see the characteristics of respondents in this study. The sample
consisted of 60 male vocational teachers (41.4%) and 82 female vocational teachers
(58.6%) around Jakarta regions. It indicates that most of the Indonesian teachers are
dominated by female teachers who are assumed to have more work stress than the
other one. Most of the respondents are aged 25 to 34 years old (42.1%). This age
indicates a productive age for teachers to full of their performance at school, but it
is still regarded as unstable in determining teacher’s satisfaction perception. The
educational background of the respondents is a bachelor's degree (83.2%) which
describes the educational experience of Indonesian teachers generally. 85.7% of the
respondent is also married, which indicates that the respondents with multiple
roles as a teacher, wife or husband, father, and mother, have a significant impact on
their satisfaction at school. Different from married teachers, unmarried teachers or
teachers with divorce background and a single parent may have a different level on
how much job satisfaction they have.
Table 1. Characteristics of Respondent

Demographic factors number Per cent (%)
Sex:
Male teachers 60 41.4
Female teachers 82 58.6
Age (years old):
< 25 21 15.0
25 – 34 59 42.1
35 – 44 28 20.0
45 – 54 20 14.3
> 55 14 8.6
Education:
Bachelor 124 83.2
Master 18 16.8
Marital Status:
Single 12 7.9
Married 121 85.7
Divorce 9 6.4
Total n=142 100.0
School culture as an independent variable is the primary data obtained through a

questionnaire tool using a Likert scale. The instrument with four indicators
contained 18 statements that tested its validity and reliability namely school values,
school beliefs, school assumptions, and school norms (Table 2).

415
Table 2. School Culture

Cronbach’s
Corrected
Alpha when Total
Dimension Indicator Number item-total %
numbers Score
correlation
deleted
1 0.143 0.824 506
2 0.045 0.831 523
School Values 134.22
3 0.618 0.798 519
4 0.579 0.803 486
5 0.163 0.824 516
6 0.605 0.803 562
School Beliefs 7 0.367 0.813 503 136.87
8 0.513 0.805 503
School 9 0.318 0.817 574
Culture 10 0.613 0.799 508
School
11 0.564 0.804 578 152.25
Assumptions
12 0.292 0.818 526
13 0.414 0.811 416
14 0.601 0.798 465
15 0.483 0.807 501
School Norms 16 0.526 0.804 466 123.34
17 0.313 0.816 357
18 0.270 0.818 325
Based on the data above, school assumptions with a total score of 152.25 % places
the highest level as the most influential factors on school culture while the lowest
total rank is the school norms indicator. School assumptions relate to how the
teachers perceive their school system assumptions, including environmental
culture, rules and regulations, etiquette, and spiritual culture. School beliefs define
as an eclectic mix of practical rules, generalisations, opinions, and structured
expectations. The more it is positioned at the centre of the belief system, the less this
belief is subject to change (Herman et al., 2008). School values relate moral order
hold by a school including the commitment to fairness, consistency, care ethics, and
inclusion. School norms relate to legal principles, rules, humanity, and inspirations
(Bipath & Moyo, 2016).
In addition to school culture, this research also measures teacher stress which
measured by three indicators consisted of emotions, thinking processes, and
physical conditions, as shown in Table 3.

416
Table 3. Teacher’s Stress
Cronbach’s
Corrected item- Total
Dimension Indicator Number Alpha when %
total correlation Score
item deleted
1 0.464 0.525 377
2 0.132 0.570 309
3 0.340 0.538 298
Emotions 85.49
4 0.091 0.572 434
5 0.320 0.538 311
6 -0.056 0.604 483
7 0.372 0.527 348
8 0.363 0.529 278
Teacher's Thinking 9 0.185 0.561 290
10 0.418 0.519 303 78.91
Stress Process
11 -0.062 0.603 532
12 0.458 0.518 253
13 0.231 0.554 411
14 0.108 0.575 293
Physical 15 -0.322 0.630 445
93.20
Condition 16 0.240 0.553 339
17 0.205 0.558 421
18 0.184 0.561 459
According to the data, physical conditions became the highest indicators in

teacher’s stress, while the thinking process is the lowest rank. Physical conditions
related to stress symptoms, such as feeling the pain of some parts of the body,
headache, stomachache, back pain, and other physic problems. The data above
surprisingly show us that most of the teachers feeling pain physically when they
are stress in their works. These physical problems are mostly found in school
teachers who represent an occupational group within a high prevalence of neck
and/or shoulder pain, and low back. (Yang, Ge, Hu, Chi & Wang, 2009) Yang et al.,
(2009) state that occupational stress and strain cause worsening physical and
mental conditions for teachers. Emotions at work also received attention, since it
relates to the activation of the cardiovascular system, stress, exhaustion, and other
physical symptoms, such as headache. In general, the factors that cause teacher’s
emotions are emotional links between school leaders and teachers, students,
partners (Nono et al., 2007). Idris (2011) also states that teachers who are trapped
in negative emotions, such as angry, and depression may intend to feel
dissatisfaction with their job, and leave their organisation.
The primary dependent variable in this study is teacher satisfaction which
measured by some indicators such as the teacher’s feelings, teacher’s work, salary,
supervision, teachers’ partner, and teacher’s promotion. The highest rank of
teacher’s wages shows that the respondents have a higher perception of it than
others. They considered that salary is the most factor in determining their job
satisfaction (see Table 4).

417
Table 4. Teacher Satisfaction

Corrected Cronbach’s
Total
Dimension Indicator Number item-total Alpha when %
Score
correlation item deleted
1 441 0.785 441
Teachers’
2 549 0.795 549 16.64
feeling
3 468 0.784 468
4 524 0.775 524
5 478 0.777 478
6 560 0.772 560
Teachers’ 7 468 0.791 468
16.52
work 8 576 0.783 576
9 416 0.776 416
10 335 0.788 335
11 502 0.779 502
12 596 0.777 596
13 572 0.768 572
Teachers' Salary 19.59
14 610 0.772 610
Job
15 510 0.782 510
Satisfaction
16 460 0.774 460
Supervisor 17 382 0.788 382 14.30
18 411 0.775 411
19 502 0.782 502
20 506 0.786 506
Teachers’ 21 462 0.788 462
17.14
partner 22 539 0.778 539
23 494 0.796 494
24 500 0.788 500
25 506 0.775 506
26 489 0.777 489
Teachers’
27 452 0.788 452 15.81
promotion
28 452 0.776 452
29 409 0.784 409
Based on the normality test, teacher’s satisfaction (Y) has a significance value of
0.764, while the significance of school culture (X1) of 0.918, and teacher’s stress (X2)
of 0.985. Therefore, using the normality Shapiro-Wilk to the significant value of
the residual standard data, we found that the data generally normal since the
significance of those three variables is more than 0.05. The table also means that H0
is accepted; hence the regression model has fulfilled the normality assumption. The
linearity of school culture and teacher’s satisfaction also indicates the significance
value of the standard residual of 0.639. Since this is more than 0.05, it means that
school culture has a linear effect on teacher’s satisfaction. The linearity test of
teacher stress and teacher’s satisfaction results in a significant value on the residual
standard of 0.510 is greater than 0.50. Thus, the teacher stress and teacher’s
satisfaction have a linear effect. The result also shows the experimental data of
school cultures (X1), teacher stress (X2), and teachers’ satisfaction (Y) into 142
vocational teachers using validity tests with the Product Moment correlation of

418
Pearson. Since we found that r arithmetic > 0.50, it means each item can be
measured. Cronbach’s Alpha shows that the value of school cultures is 0,85, teacher
stress is 0,88, and teacher satisfaction is 0,83. Hence, the measuring instrument used
is valid and reliable by using a statistic reliability test. To know the direct influence
between school culture and teacher’s work stress, we put school culture (X1) as an
exogenous variable and teacher’s work stress (X2) as endogenous variables on
hypothesis testing between variable structure and substructure 1 with structural
equation X2 = ρ21+ε1. This calculation uses SPSS 24, and we can see the result in
Table 5.
Table 5. Coefficients Sub-Structure Test 1; Influence of School Culture to Teacher’s Work Stress
Coefficients Coefficients t F Sig.
Model B Std. Error Beta
1 (Constant) 53.250 3.953 13.472 2.533 0.000
School Culture -0.099 0.062 -0.134 -1.591 0.114
a. Dependent Variable: Teachers’ Work Stress (TWS)
According to those calculations, the value of path coefficient ρ21 is -0.134, hence the
structural equation form is X2 = -0,134 X1 + ε1. With R22.1 is 0.18, the magnitude of
variables that affect the model into endogenous variables of teacher work stress is
ε1= 0.82.
Figure 2. Path Coefficient Substructure Model 1
Further analysis to study the indirect influence of school cultures (X1) on teachers’
satisfaction (Y) which is mediated by teacher stress (X2) uses SPSS 24, and the result
as shown in Table 6. The model path coefficient analysis of this substructure 2 is
X3=ρ31X1+ρ32X2+ε2.
Table 6. Test Coefficients Sub-Structure 2. Influence of School Culture, Teachers’

Work Stress toward Teachers’ Satisfaction
Coefficients Coefficients t F Sig.
Model B Std. Error Beta
1 (Constant) 136.193 6.309 21.587 62.4 0.000
School Culture (SC) 0.147 0.066 0.140 2.249 0.026
Teachers’ Work -0.942 0.089 -0.658 -10.547 0.000
Stress (TWS)
a. Dependent Variable: Teachers’ Job Satisfaction (TJS)

419
Based on those calculations, the value of path coefficient ρ31=0.140 and ρ32=-0.658,
hence we found that X3 = 0.14X1 – 0.658X2 + ε2. Since the value of R23.12 is 0.477,
then the magnitude of other variables that affect outside the model towards
endogenous variables X3 is ε2= 0.523.
Figure 3. Coefficient of Path Substructure Model 2
We also analyse total influence which is a combination of both previous effects.

According to the coefficient value, as shown in Table 7, we can conclude that all of
these paths are significant.
Table 7. Path Coefficient, Direct and Indirect Impact

Causal effect
Impact Traversed Variable total
Direct X2 Indirect
0.14 - - 0.14
SC to TJS (-0.134)
0.088 0.228
(-0.658)
TWS to TJS -0.658 - - -0.658
SC to TWS -0.134 - - -0.134
Regarding the analysis above, we found the form as follows:

a. School culture impacts on teacher’s stress directly and significantly as 1.795%.
b. Teacher’s stress impacts on teacher’s job satisfaction directly and significantly
as 43.396%.
c. School culture impacts on teacher’s job satisfaction mediated by teacher stress
as 5.198%
Based on some data above, we can reveal that school culture components like
beliefs, norms, assumptions, and values having a significant and positive influence
on teachers ‘job satisfaction. Furthermore, as shown in Table 1, school beliefs have
a higher impact on it. If organisational culture goes up, then the job satisfaction of
the teacher and the faculty members of the universities will also go up. School
culture has many more influences on school performance by impacting the
psychological of individual teachers and groups (Zhang & Li, 2013). On the other
side, De Ruyter, Wetzels and Feinberg (2001) revealed that job stress impact on job
satisfaction directly. The ambiguity of stress and conflict stress significantly impact
on job satisfaction. In addition to this, teachers’ workload at school, causing
physical symptoms effects on job satisfaction (De Simone, Cicotto & Lampis, 2016).

420
Educational change and current curriculum initiatives caused teachers to work

stress which impacts their job satisfaction (Moriarty, Edmonds, Blatchford &
Martin, 2001). Thus, as shown in Table 4, both school culture and teachers' work
stress can influence teachers’ job satisfaction.
5. Conclusion
The result of the study concluded that school culture and teachers’ work stress
significantly impact on teachers’ job satisfaction. These school culture indicators
consist of teachers’ feelings, career, salary, supervision, partner, and promotion.
High and conducive school beliefs, norms, assumptions, and values will directly
impact teacher’s feelings whether they are pleased or not to work as a teacher.
Teachers’ emotions, physical symptoms, and thinking processes are kinds of
teacher’s work stress that also can impact on their job satisfaction. Thus, to increase
teachers’ job satisfaction, effective school management not only needs to build and
create positive school culture but also analyses as well as solve teachers’ work stress
concisely.
This finding is also could be a study in developing teachers’job satisfaction in
theoretical aspect studies, namely the development of scientific studies for teacher’s
job satisfaction, as well as practical in developing teacher’s quality. In developing
teacher’s satisfaction skills among vocational teachers, it was found that physical
aspects as the leading indicator of teacher’s stress variables. Thus, both the
government and school need to analyse and evaluate teacher development policies.
It is also an input for the government in taking the system of developing vocational
teacher competencies, especially in optimizing and synchronizing between
vocational schools and industries so that they become pro.
6. Acknowledgment
We would like to express our gratitude to the Faculty of Economics, Universitas
Negeri Jakarta for funding this publication.
7. References
Abel, M. H., & Sewell, J. (1999). Stress and burnout in rural and urban secondary school
teachers. Journal of Educational Research, 92(5), 287–293.
https://doi.org/10.1080/00220679909597608
Asmawi, M. (2017). The Effect Of Compensation, Empowerment, And Job Satisfaction On
Employee Loyalty. International Journal of Scientific Research and Management, 5(12),
7590–7599. https://doi.org/10.18535/ijsrm/v5i12.03
Baumert, J., Kunter, M., Blum, W., Klusmann, U., Krauss, S., & Neubrand, M. (2013).
Cognitive activation in the mathematics classroom and professional competence of
teachers: Results from the COACTIV project. Cognitive Activation in the Mathematics
Classroom and Professional Competence of Teachers: Results from the COACTIV Project,
1–378. https://doi.org/10.1007/978-1-4614-5149-5
Belias, D., Koustelios, A., Vairaktarakis, G., & Sdrolias, L. (2015). Organisational Culture
and Job Satisfaction of Greek Banking Institutions. Procedia - Social and Behavioral
Sciences, 175, 314–323. https://doi.org/10.1016/j.sbspro.2015.01.1206
Bellou, V. (2010). Organisational culture as a predictor of job satisfaction: The role of gender
and age. Career Development International, 15(1), 4–19.
https://doi.org/10.1108/13620431011020862

421
Bipath, K., & Moyo, E. (2016). Principals Shaping School Culture for School Effectiveness in
South Africa. Journal of Social Sciences, 48(3), 174–186.
https://doi.org/10.1080/09718923.2016.11893581
Bono, J. E., Foldes, H. J., Vinson, G., & Muros, J. P. (2007). Workplace Emotions: The Role of
Supervision and Leadership. Journal of Applied Psychology, 92(5), 1357–1367.
https://doi.org/10.1037/0021-9010.92.5.1357
Borman, G. D., & Maritza Dowling, N. (2008). Teacher attrition and retention: A meta-
analytic and narrative review of the research. Review of Educational Research, 78(3),
367–409. https://doi.org/10.3102/0034654308321455
Chaplain, R. P. (2008). Stress and psychological distress among trainee secondary teachers
in England. Educational Psychology, 28(2), 195–209.
https://doi.org/10.1080/01443410701491858
Chauvet, G., & Do Paco, W. (2018). Accurate balanced random imputation for sample
survey data. Computational Statistics and Data Analysis, 128, 1–16.
https://doi.org/10.1016/j.csda.2018.06.006
Chiu, S. F., & Chen, H. L. (2005). Relationship between job characteristics and organisational
citizenship behaviour: The mediational role of job satisfaction. Social Behaviour and
Personality, 33(6), 523–540. https://doi.org/10.2224/sbp.2005.33.6.523
Chris, K. (2001). Teacher stress: Directions for future. Educational Review, 53(1), 27–35.
https://doi.org/10.1080/0013191012003362
Cichoki, A., & Farrell, T. S. C. (2019). Professional development of secondary school EFL
teachers: Voices from Indonesia. System, 85, 102111.
https://doi.org/10.1016/j.system.2019.102111
Deal, T. E., & Peterson, K. D. (2016). Shaping school culture. CA: John Wiley & Sons.
De Ruyter, K., Wetzels, M., & Feinberg, R. (2001). Role stress in call centres: Its effects on
employee performance and satisfaction. Journal of Interactive Marketing, 15(2), 23–35.
https://doi.org/10.1002/dir.1008
De Simone, S., Cicotto, G., & Lampis, J. (2016). Occupational stress, job satisfaction and
physical health in teachers. Revue Europeenne de Psychologie Appliquee, 66(2), 65–77.
https://doi.org/10.1016/j.erap.2016.03.002
Demirtaş, Z. (2010). Teacher’s job satisfaction levels. Procedia - Social and Behavioral Sciences,
9, 1069–1073. https://doi.org/10.1016/j.sbspro.2010.12.287
Eichinger, J. (2000). Job stress and satisfaction among special education teachers: Effects of
gender and social role orientation. International Journal of Disability, Development, and
Education, 47(4), 397–412. https://doi.org/10.1080/713671153
Erichsen, K., & Reynolds, J. (2019). Public school accountability, workplace culture, and
teacher morale. Social Science Research, (December 2018), 102347.
https://doi.org/10.1016/j.ssresearch.2019.102347
García Torres, D. (2019). Distributed leadership, professional collaboration, and teachers’
job satisfaction in US schools. Teaching and Teacher Education, 79, 111–123.
https://doi.org/10.1016/j.tate.2018.12.001
Guthrie J. W., & Schuermann P. J. (2011). Components of high performing school culture. In: J
Guthrie (Ed.): Leading Schools to Success: Constructing and Sustaining High-Performing
Schools (pp. 39-72). Thousand Oaks: Sage.
Hermans, R., Tondeur, J., van Braak, J., & Valcke, M. (2008). The impact of primary school
teachers’ educational beliefs on the classroom use of computers. Computers and
Education, 51(4), 1499–1509. https://doi.org/10.1016/j.compedu.2008.02.001
Ho, C.-L., & Au, W.-T. (2006). Measuring job satisfaction of teachers. Educational and
Psychological Measurement, 66(1), 172–185.
https://doi.org/10.1177/0013164405278573

422
Hoy, W. K. (1990). Organisational Climate and Culture: A Conceptual Analysis of the School
Workplace. Journal of Educational and Psychological Consultation, 1(2), 149–168.
https://doi.org/10.1207/s1532768xjepc0102_4
Idris, M. A., & Dollard, M. F. (2011). Psychosocial safety climate, work conditions, and
emotions in the workplace: A malaysian population-based work stress study.
International Journal of Stress Management, 18(4), 324–347.
https://doi.org/10.1037/a0024849
Jalagat, R. (2016). Job performance, job satisfaction, and motivation: a critical review of their
relationship. International Journal of Advances in Management and Economics, 5(6), 36–
42. Retrieved from
https://www.managementjournal.info/index.php/IJAME/article/view/64
Klassen, R. M., & Chiu, M. M. (2010). Effects on Teachers’ Self-Efficacy and Job Satisfaction:
Teacher Gender, Years of Experience, and Job Stress. Journal of Educational
Psychology, 102(3), 741–756. https://doi.org/10.1037/a0019237
Kotz, P. E. (2016). Reaching the Millennial Generation in the Classroom. Universal Journal of
Educational Research, 4(5), 1163–1166. https://doi.org/10.13189/ujer.2016.040528
Lee, M., & Louis, K. S. (2019). Mapping a healthy school culture and linking it to sustainable
school improvement. Teaching and Teacher Education, 81, 84–96.
Liu, X. S., & Ramsey, J. (2008). Teachers’ job satisfaction: Analyses of the Teacher Follow-up
Survey in the United States for 2000-2001. Teaching and Teacher Education, 24(5),
1173–1184. https://doi.org/10.1016/j.tate.2006.11.010
Louis, K. S. (2006). Changing the Culture of Schools: Professional Community,
Organisational Learning, and Trust. Journal of School Leadership, 16(5), 477–489.
https://doi.org/10.1177/105268460601600502
Ma, X., & MacMillan, R. B. (1999). Influences of workplace conditions on teachers’ job
satisfaction. Journal of Educational Research, 93(1), 39–47.
https://doi.org/10.1080/00220679909597627
Moriarty, V., Edmonds, S., Blatchford, P., & Martin, C. (2001). Teaching young children:
Perceived satisfaction and stress. Educational Research, 43(1), 33–46.
https://doi.org/10.1080/00131880010021276
Riaz, M., Ahmad, N., Riaz, M., Murtaza, G., Khan, T., & Firdous, H. (2016). Impact of Job
Stress on Employee Job Satisfaction. International Review of Management and Business
Research, 5(4), 1370–1382. Retrieved from
http://www.irmbrjournal.com/paper_details.php?id=610
Rizwan, M., Waseem, A., & Bukhari, S. A. (2014). Antecedents of Job Stress and its impact
on Job Performance and Job Satisfaction. International Journal of Learning and
Development, 4(2), 187–203. https://doi.org/10.5296/ijld.v4i2.6097
Ryan, S. V., von der Embse, N. P., Pendergast, L. L., Saeki, E., Segool, N., & Schwing, S.
(2017). Leaving the teaching profession: The role of teacher stress and educational
accountability policies on turnover intent. Teaching and Teacher Education, 66, 1–11.
Schein, H. E. (2004). Organisational Culture and Leadership (third). San Francisco: Jossey-Bass
A Wiley Imprint. https://doi.org/10.1136/jcp.10.4.354
Sohail, A., Safdar, R., Saleem, S., Ansar, S., & Azeem, M. (2014). Effect of Work Motivation
and Organisational Commitment on Job Satisfaction: (A Case of Education Industry
in Pakistan). Global Journal of Management And Business Research, 14(6), 41–46.
Gruenert, S., & Whitaker, T. (2015). School culture rewired: How to define, assess, and
transform it. Alexandria, VA: Association for Supervision and curriculum
development.

423
Sun, A., & Xia, J. (2018). Teacher-perceived distributed leadership, teacher self-efficacy, and
job satisfaction: A multilevel SEM approach using the 2013 TALIS data. International
Journal of Educational Research, 92(April), 86–97.
https://doi.org/10.1016/j.ijer.2018.09.006
Troesch, L. M., & Bauer, C. E. (2017). Second career teachers: Job satisfaction, job stress, and
the role of self-efficacy. Teaching and Teacher Education, 67, 389–398.
Waqas, A., Bashir, U., Sattar, M. F., Abdullah, H. M., Hussain, I., Anjum, W., … Arshad, R.
(2014). Factors Influencing Job Satisfaction and Its Impact on Job Loyalty.
International Journal of Learning and Development, 4(2), 141–161.
https://doi.org/10.5296/ijld.v4i2.6095
Wilson, V. (2002). Feeling the strain: An overview of the literature on teachers’ stress
(Research Report No 109). Edinburgh, UK: The Scottish Council for Research in
Education. Retrieved from http://docs.scie-
socialcareonline.org.uk/fulltext/screteastress.pdf
Yang, X., Ge, C., Hu, B., Chi, T., & Wang, L. (2009). Relationship between quality of life and
occupational stress among teachers. Public Health, 123(11), 750–755.
https://doi.org/10.1016/j.puhe.2009.09.018
Yue, P., Liu, F., & Li, L. (2012). Neck/shoulder pain and low back pain among school
teachers in China, prevalence and risk factors. BMC Public Health, 12(789), 1-8.
https://doi.org/10.1186/1471-2458-12-789
Yin, H., Huang, S., & Chen, G. (2019). The relationships between teachers’ emotional labour
and their burnout and satisfaction: A meta-analytic review. Educational Research
Review, 28, 100283. https://doi.org/10.1016/j.edurev.2019.100283
Zavyalova, E., & Kucherov, D. (2010). Relationship between organisational culture and job
satisfaction in Russian business enterprises. Human Resource Development
International, 13(2), 225–235. https://doi.org/10.1080/13678861003703740
Zhang, X., & Li, B.(2013). Organisational Culture and Employee Satisfaction: An
Exploratory Study. International Journal of Trade, Economics and Finance, 4(1), 48-54.
https://doi.org/10.7763/IJTEF.2013.V4.259

Appendix
Questioners
How School Culture and Teacher’s Work Stress Impact on
Teacher’s Job Satisfaction
Note
SA : Strong Agree
A : Agree
N : Neither Agree nor Disagree
D : Disagree
SD : Strong Disagree
1. School Organisation
Number Statements SA A NAND D SD

1. School support the works with values
School leader asked me to have the
2. initiative to work
School leader support me to increase the
3. creativity
School leader support me to have
4. innovation in sharing the ideas
The school support me to decide any
decisions relates to my responsibility as a
5.
teacher
6. I do not understand my school values
7. I am ready to take the risk in my duty
The school support me to believe that the
8. result never betrays the effort
9. The school give me a chance to my best

10. The school apply the discipline strictly
The school convince me about my next
11. carrier
12. The school motivates the teachers well
The school suggested the teachers be
13.
always productive in reaching school goals
The school support the teachers to be
14. creative in doing the duty
The school applies well system in
15.
preventing the pressure
The school roles support the goal
16.
enrichment
The school roles are relevant to school
17.
goal
The school roles support me to do better
18.
in my duty
19. The school roles made me stress
20. The school roles are relevant to my value
2. Teacher’s Stress Work

1. I feel high tension facing my works
I cannot control my emotions when
2.
facing my works
I always feel angry when my students
3.
ignore me
I have difficulties in working, but I am
4. afraid to tell them about my principles
nor my partners
I enjoy and feel comfortable in my
5.
workings
I have no emotional distractions in
6.
doing my job
7. I feel dizzy in facing my works
8. I had diarrhea in facing my works
I cannot sleep well because thinking of
9.
my works
10. I feel very tired when teaching
I do not feel any pain in facing my
11.
jobs
I always take care of my health even
12.
though I have many works
I have a stomachache both at work
13.
and afterward
I think positively about the policies
14.
given to works
I felt that the demands on the quality of
15.
my works were unreasonable
16. I suspect things were badly planned
I am passionate and excited to think

17.
about the school progress
I am not excited to think about my job
18.
19. Unwell explained authority or
responsibility makes me feel
depressed
20. I felt the workload assigned to me was
reasonable
3. Teacher’s Job Satisfaction

1. I feel happy to work at this school
2. I felt the school atmosphere was fun
3. My school environment was fun
4. I feel unhappy to work at this school
I feel happy when I meet students who are
5.
ready to study
6. I do not like facing naughty students
7. My working is relevant to my
expectations
I feel this job is what I dreamed of
8.
long ago
My job is relevant to my educational
9.
background, competency, and skills
10. My job gives me a better challenge
11. I understand my jobs and
responsibility well
12. Working at this school made my days
13. This school is my best place to work
My works are interesting and
14
challenging
The salary I got is relevant to my
15.
workload
16. The salary I got was huge
The salary I got is relevant to my
17. educational background, experiences,
and skills
18. I always feel being supervised at work
19. I was annoyed by the school
supervisor
20. The school superviser is good
21. The school leader supervise as needed
The school leader support me to
22.
increase my skills
23. My partners support me well
24. In general, the relationship among
partners is harmony
25. The conflict with school partner made me
difficult to work
26. The school give some awards to the
achievement teachers
27. The opportunity to get the promotions at
this school tends to be easy
28. I feel my school appreciate me well in
working
29. I think the school did not give enough
rewards if I do well job
30. When I complete my job well, my school
leader praises me well too
424

Vol. 19, No. 8, pp. 424-446, August 2020
Investigating the Quality of University Education:

A Focus on Supply Chain Management
Joash Mageto*, Rose Luke and Gert Heyns

Department of Transport and Supply Chain Management,
University of Johannesburg, Johannesburg, South Africa
https://orcid.org/0000-0002-3678-2986
https://orcid.org/0000-0003-1915-6956
https://orcid.org/0000-0003-0881-6525
Abstract. While the quality of higher education has received attention

from researchers globally, its focus has been mainly on universities in
‘developed country’ contexts. There has been limited research on the
quality of higher education in African universities, and even less that has
focused on the discipline of supply chain management. In this paper, the
quality of supply chain education of five universities in Kenya is
investigated using a SERVPERF model. Data were collected through a
structured questionnaire administered to 781 students enrolled for a
supply chain management qualification. The data were analysed through
confirmatory factor analysis and one-way analysis of variance. The
findings are fourfold: (1) the service quality of supply chain education is
at medium level in terms of the SERVPERF metrics; (2) four dimensions
were identified by students as the most important aspects of service
quality: course-centeredness, academic and support staff helpfulness,
service excellence and learning facilities; (3) a four-factor SERVPERF
model of supply chain service quality is developed and (4) service quality
was significantly different across the selected universities. Although the
service quality was rated at a medium level overall, the variability in
quality across universities should alert educators and management to the
need for a coordinated effort to improve particular aspects of students’
learning experiences. The study contributes to the body of knowledge by
establishing that SERVPERF is a four-factor model in the higher
education sector.
Keywords: Service quality; Supply chain education; Factor analysis;

SERVPERF; Analysis of variance
1. Introduction
Increased competition among universities, globalisation, and the continued
waning of government funding has forced higher education institutions to offer
excellent service to attract and maintain a sustainable student base (Mahmoud &
Khalifa, 2015; Shabani, Okebukola & Oyewole, 2014; Yusoff, McLeay &

425
Woodruffe-Burton, 2015). To create memorable encounters for students in higher

education, universities measure service quality in their various programmes to
identify problem areas and to improve (Abdullah, 2005; Yusoff et al., 2015). The
instruments used to measure service quality in higher education include
SERVQUAL (Galeeva, 2016; Parasuraman, Zeithaml & Berry, 1988), SERVPERF
(Cronin & Taylor, 1992; Luke & Heyns, 2018), and HEdPERF (Abdullah, 2006;
Silva, Moraes, Makiya & Cesar, 2017). Brochado (2009) compared all the three
instruments and concluded that SERVPERF and HEdPERF are better placed to
measure service quality in higher education because they offer excellent
measurement capability, have high reliability, and explained variance. Previous
studies have measured service quality institution-wide, that is, surveying the
general student population to collect their perceptions on the service quality
dimensions (Brochado, 2009; Abdullah, 2006; Teeroovengadum, Kamalanabhan
& Seebaluck, 2016). Studies that have examined supply chain education have
generally focused on research methods used (Sun & Song, 2018), supply chain
skills (Lorentz, Töyli, Solakivi & Ojala, 2013), and service quality in supply chain
education (Luke & Heyns, 2018). Previous studies have left a lacuna regarding the
current state of service quality in supply chain education. Academics and
practitioners are also unfamiliar with the service quality aspects that supply chain
students value, as well as whether there are any service quality differences across
universities. Therefore, academics and educationalists must measure the service
quality of supply chain education regularly, especially given the pivotal role that
supply chain management plays in the contemporary business environment.
Previous studies have observed that, in some cases, supply chain graduates are
ill-prepared to handle current supply chain issues, based on assessments of the
modules taught (Leon & Uddin, 2016). Also, there is a limited understanding of
quality issues in supply chain education at the tertiary level (Sun & Song, 2018).
Finally, perspectives on the quality of supply chain education from students have
not been investigated adequately (Yusoff et al., 2015), although studies from single
universities, covering all faculties, have been highlighted (Nadiri, Kandampully
& Hussain, 2009). The current study furthers the discourse on the quality of
university education by (1) focusing supply chain education in a developing
country, (2) drawing a student perspective of the quality of supply chain
education and (3) applying the SERVPERF model making possible for comparison
with similar future studies. Therefore, the study aims at investigating student
perspectives on the service quality of supply chain education in selected public
universities using the SERVPERF instrument. As such, this study answers the
following questions: 1) What is the level of service quality of supply chain
education?, 2) What aspects of service quality do supply chain students value?,
and 3) Is the service quality of supply chain education uniform across
universities? This study is likely to enhance the supply chain body of knowledge
regarding the quality of training; from both student and developing country
perspectives.
2. Literature review
In this section literature review is conducted on service quality of supply chain
education, service quality models, and related works. The literature review helps

426
to discuss the main research concepts by presenting what is known and identify
the gap that this study will help fill. The next section presents a discussion on the
service quality of supply chain education.
2.1 Service quality of supply chain education

A supply chain education (SCE) can refer to systematic instruction in the area of
supply chain management to meet the requirements of the business environment.
It includes designing a curriculum and continuous improvement of the methods
of instruction, to transfer knowledge in an excellent manner (Sun & Song, 2018).
Currently, professional SCE is offered by tertiary institutions, which award
diplomas and degree qualifications in Kenya (Commission of University
Education (CUE), 2018). Globally, supply chain management as a discipline has
developed tremendously over the last three decades owing to factors like
globalisation, innovation, and advancements in technology (Akbari, 2018). The
growth has also been observed in developing countries, and universities have
responded by training students to handle supply chain management (SCM)
related issues in the business environment. The rapid changes are still ongoing,
implying that SCM professionals should be well trained, to be able to handle the
increasing complexity (Sun & Song, 2018). The challenge is that, although the
quality of business education has been examined (Yusoff et al., 2015), researchers
have not adequately interrogated the service quality of supply chain education
from the perspective of a student (Sun & Song, 2018).
The quality of education can broadly be defined as the relevance of the training to
industry requirements, in terms of the employability and entrepreneurship of
trainees (Mittal, Garg & Yadav, 2018; Mohamedbhai, 2014). Quality of education
depends on many factors including infrastructural facilities such as the
availability of furniture and adequate classroom facilities, the availability of Wi-
Fi, a library equipped with relevant information and search systems, ICT-
equipped classrooms (with projectors, public address systems, and other suitable
teaching aids), computer labs with appropriate learning applications, and
adequate provision of light (Verma & Prasad, 2017; Amini-Philips & Mukoro,
2016). Thus, adequate infrastructure is critical to the quality of education provided
by tertiary institutions (Mittal, et al., 2018; Yusoff et al., 2015). Therefore, having
the right educational infrastructure is likely to improve student-teacher
interaction, which might result in better educational outcomes (Sun & Song, 2018).
Also, academic staff in higher education institutions who conduct teaching and
research as well as administration should have the right skills to promote high-
quality university education (Lubwama, Onen & Kasenene, 2017).
2.2 Service quality models

Service quality as a construct has received attention in both the manufacturing
and service industries for decades. Seth, Deshmukh, and Vrat (2005) identified 19
different service quality models including SERVQUAL, technical and functional
quality model, attribute service quality model, SERVPERF, ideal value model of
service quality, and PCP attribute model. Among researchers and practitioners,
Parasuraman, Zeithaml, and Berry (1985) presented significant guidance on how
the service quality construct can be measured in firms. They decomposed service
quality into five dimensions, which are tangibility, responsiveness, reliability,

427
assurance, and empathy. They argued that service quality could be measured by
finding the gap (expectations minus perceptions) between customer expectations
and the perceived service delivered at the firm’s facilities using the SERVQUAL
(service quality) model (Seth et al., 2005). A negative value will imply that
customer expectations have been met, while a positive value indicates that they
have not. Although the SERVQUAL model has had many different applications
over the years (Parasuraman et al., 1985), some scholars (Mahmoud & Khalifa,
2015; Yusoff et al., 2015) have argued that the model cannot be applied in every
industry. Thus, new models have been developed for application to specific
industries, such as education.
The service quality of an educational institution can be measured using different
metrics, including SERVQUAL, SERVPERF, and HEdPERF (Abdullah, 2005;
Mahmoud & Khalifa, 2015). The selection of a model is subjective, although
Rodrigues, Barkur, Varambally, and Golrooy Motlagh (2011) claim that
SERVPERF metrics are better when measuring service quality in more intensive,
service-orientated contexts, such as higher education institutions because they are
performance-based. In agreement, Cronin and Taylor (1992) observe that, when
measuring service quality in higher education institutions, the SERVPERF metrics
are likely to offer reliable estimations with less bias when compared to those of
SERVQUAL. However, Brochado (2009) argued that SERVPERF and HEdPERF
produce almost similar results; thus, any of them can be applied to measure
service quality in higher education institutions. Besides, Souca (2011) claimed that
SERVQUAL measures customer satisfaction rather than service quality.
Therefore, Jain and Gupta (2004) find the SERVPERF model to be
psychometrically sound with greater “instrument parsimoniousness” and is
therefore appropriate for adjudicating an organisation’s overall service quality.
SERVPERF is a performance-based method used to measure service quality
(Cronin & Taylor, 1992). They argued that service quality should be measured
using an attitude scale, preferably the performance-based SERVPERF metric,
which is better than that of SERVQUAL. The SERVPERF metric has 22 items that
are measured using an attitude scale as opposed to that of SERVQUAL, which
measures the gap between expectations and perceptions using 44 items (Cronin
& Taylor, 1992; Yusoff et al., 2015). The 22 items are the indicators of the five
dimensions of service quality, that is, tangibility, reliability, responsiveness,
assurance, and empathy. In the performance-based SERVPERF model, service
quality is measured using the following formula provided by Seth, Deshmukh,
and Vrat (2005):
𝑘
𝑆𝑄 = ∑ (𝑃𝑖𝑗 )
𝑗=1
Where:
SQ = Overall service quality;
k = the number of attributes;
Pij = performance perception of stimulus (i) , in relation to attribute (j).
The relevance of using SERVPERF metrics to measure the service quality of higher
education has received considerable attention from researchers, who, in most
cases, have favoured SERVPERF over SERVQUAL (Rodrigues et al., 2011; Yusoff

428
et al., 2015). Many studies have used SERVPERF to measure service quality in
higher education, for example, Brochado (2009), Yusoff et al. (2015), and
Mahmoud and Khalifa (2015). The users of SERVPERF favour it because it
provides results that can guide future decisions, given that it assesses perceived
performance; provides the real state of service quality, as perceived by customers;
uses fewer indicators compared to SERVQUAL; is likely to be less biased; explains
more variance; and holds higher validity and reliability (Cronin & Taylor, 1992;
Mahmoud & Khalifa, 2015; Yusoff et al., 2015). Based on the reviewed literature,
this study selected the SERVPERF metric to measure the service quality of supply
chain education in universities.
2.3 Related work

Several studies have considered service quality in higher education, however,
searches through several databases, including SCOPUS, ScienceDirect,
EBSCOHost and SABINET using the terms “quality”, “supply chain” and
“education” have revealed few related studies. The “quality” term was dropped
and the results indicate that the majority of research relates to curriculum
development and the identification of skills to incorporate in supply chain
education (Mageto & Luke, 2020; Purnomo, et al., 2020; Luke & Heyns, 2019;
Colicchia, et al., 2018; Lorentz, Töyli, Solakivi & Ojala, 2013; Gonzalez et al., 2008;
Sauber et al., 2008). Others highlight the importance of supply chain education
(Erturgut & Soysekerci, 2011), building specific skills into supply chain education
(Pekkanen, et al., 2020), tools for supply chain education (Holweg & Bicheno,
2002), and educational content development for modern supply chains (Li, 2020).
Because so few studies have addressed the specific issue of the service quality of
supply chain education, previous studies on the service quality of higher
education, in general, were consulted to form a framework for this research. In
this regard, there are numerous studies to rely on. Early works by Gustafsson &
Larsson (1997) used two total quality models, namely the Total Perceived Quality
Model and Quality Model in conjunction with the SERVQUAL model to describe
the service quality elements in higher education. Thus, the current study serves to
bridge the gap that has been left in the literature regarding the quality of supply
chain education.
3. Methodology
In this study, the service quality of supply chain education is described from the
perspective of the student, to help guide university academics and stakeholders
in identifying areas for improvement. This section presents a discussion on
research design, respondents, instrument, data collection, and statistical analysis,
in that order.
Research design
The study followed a survey research design. A survey design was selected
because (1) it was best suited to answer the research questions at hand, (2) it
allowed the collection of data from a large sample of the student population
within a short time when compared to the qualitative approach (Ponto, 2015), and
(3) it was easy to convert the students’ opinions and attitudes towards the quality
of supply chain education they receive into numbers.

429
Respondents
The university management, lecturers, and their support staff are service
providers, while the students are considered to be the customers. A study on
service performance was likely to benefit from surveying the customers; as such,
the respondents were students enrolled for a supply chain management
qualification. This means that students targeted were at least in their second year
of study. This is because first-year students take general modules and only start
specialisation in the second year of study.
Instrument
The questionnaire items were adopted from the SERVPERF instrument used by
Luke and Heyns (2018), which had been adapted from the Cronin and Taylor
(1992) instrument. The questionnaire items focused on service performance and
collected student perceptions across five dimensions, that is tangibility (four
items), reliability (five items), responsiveness (four items), assurance (four items),
and empathy (five items). All of the items were measured on a five-point Likert-
type scale (where 1=strongly disagree and 5=strongly agree). The respondents
were asked to rate the extent to which they agreed with the statements. Only
perception data was collected, as was also done by Luke and Heyns (2018) in
South Africa. The instrument used to collect data for this study is an industry-
accepted SERVPERF model and had been utilised over the years. However, given
the significance of a pilot phase in research (In, 2017), the instrument was piloted
on 50 students. There was no issue identified on all the 22 items of the instrument.
As such, the pilot data was included in the main research.
Data collection
A paper-based survey was conducted between May and August 2018 at five
selected major universities offering supply chain-related courses. The survey was
part of a wider study on supply chain skills gaps.
During data collection, lecturers were requested to allow the students some time
at the end of their lectures to respond to the questionnaire. The questionnaires
were distributed in the last 20-30 minutes of the lecture and collected at the end.
The students were requested to volunteer to complete the questionnaire, and the
objectives of the study were explained to them. A total of 2 000 questionnaires
were distributed across five universities.
Statistical analysis
The number of returned valid questionnaires was 781, resulting in a 39 per cent
response rate, which was comparable with other logistics surveys, as observed by
Wagner and Kemmerling (2010). Nonresponse bias was tested according to the
procedure provided by Armstrong and Overton (1977). Nonresponse bias was
ruled out as there was no statistical difference in responses received early in the
survey, and those collected in the last two weeks. To establish the level of service
quality, descriptive statistics were applied, specifically the mean rating and
standard deviation in SPSS version 26. An exploratory factor analysis (EFA) was
conducted to identify the latent factors as well as the service quality aspects that
supply chain students value. A confirmatory factor analysis (CFA) was performed
in Analysis of Moment Structures (AMOS) version 26, which established a four-

430
factor SERVPERF model. Finally, service quality differences across the surveyed
universities were tested using one way ANOVA. The results are presented in the
next section.
4. Results
The five universities were labelled A, B, C, D, and E, and actual names were
withheld as only a general permit to conduct research had been obtained from the
Kenyan National Commission of Science, Technology and Innovation.
Respondents per university were distributed as 26.6 per cent, 11 per cent, 37.4 per
cent, 21.4 per cent and 3.6 per cent for A, B, C, D, and E, respectively. The majority
(52.1%) of the respondents were male. This may be indicative of the overall
student distribution within the institutions surveyed. Age distribution revealed
that 77.7 per cent of the respondents were 18 to 27-year olds with 7.8 per cent and
0.4 per cent of the population being in the 28-37, and above 38 years age groups,
respectively. The age distribution was expected since students at this level are
generally young. Among the respondents, only 25 per cent were employed with
the majority being unemployed. This huge unemployment status was expected
since most students only secure jobs after obtaining their qualifications. The
unemployed proportion might be an indicator of vast youth unemployment in
Kenya (Hall, 2017). The respondents with jobs were employed in various
positions, including in logistics and supply chain management (49.3%),
accounting and finance (13.4%), customer service (15%), and marketing (13.4%);
while 8.9 per cent indicated that they were self-employed.
The descriptive statistics of the SERVPERF dimensions per university, and
overall, were examined, and the results presented, as shown in Table 1. The
highest-ranked item overall is under the reliability dimension, “Courses are
taught by highly knowledgeable teaching staff” (M=4.15; SD=0.820). This item
was also highly ranked in universities A (M=4.20; SD=0.796), B (M=3.86;
SD=0.939) and C (M=4.23; SD=0.803). The highest-ranked item in university D
was “Library (including online) has the latest literature in your area of interest”
(M=4.23; SD=0.913), and “Assignment, test and exam feedback and marks are
accurately captured and recorded” (M=4.48; SD=0.814) in university E. The
lowest-ranked item overall was also under the reliability dimension, “When
something is promised by a certain time, it is always provided” (M=3.10;
SD=1.093) signifying the variability of perceptions across universities regarding
the dimension. The lowest-ranked items per university included “Assignment,
test and exam feedback and marks are made available timeously” (M=2.66;
SD=1.407) in university B; “The lecture venues have modern and latest
equipment” (M=2.94; SD=1.135) in C; and “When students have problems, staff
are polite, even if not able to help” (M=2.83; SD=1.149) in university D. The results
presented so far indicate that there is service quality variability across universities.
Further results presentation qualifies this claim.

431
Table 1: SERVPERF dimensions - Descriptive statistics and reliability

Overall A B C D E
Std. Std. Std. Std. Std. Std.
SERVPERF scale items Mean
Deviation
Mean
Deviation
Mean
Deviation
Mean
Deviation
Mean
Deviation
Mean
Deviation
Tangibility
The lecture venues have modern and latest equipment 3.12 1.104 3.23 1.079 2.87 1.200 2.94 1.135 3.34 .961 3.95 .722
The appearance of the physical facilities is attractive 3.31 1.080 3.39 1.013 3.25 1.057 3.02 1.106 3.71 .993 3.77 .922
The course material is up to date and relevant 3.90 .988 3.91 .911 3.46 1.172 3.86 1.047 4.17 .803 4.09 .750
Library (including online) has the latest literature in your area of interest 3.78 1.122 3.81 1.048 3.26 1.225 3.65 1.174 4.23 .913 3.59 .908
Reliability
When something is promised by a certain time, it is always provided 3.10 1.093 3.16 1.004 2.94 1.141 3.03 1.182 3.13 1.001 3.73 .935
When students have problems, staff are polite, even if not able to help 3.13 1.148 3.09 1.091 3.04 1.131 3.32 1.167 2.83 1.149 3.68 .894
Courses are taught by highly knowledgeable teaching staff 4.15 .820 4.20 .796 3.86 .939 4.23 .803 4.09 .805 4.32 .646
The teaching staff respects lecture and exam schedules 4.08 .913 4.04 .897 3.66 1.009 4.20 .878 4.10 .908 4.38 .669
Assignment, test and exam feedback and marks are accurately captured and recorded 3.71 1.127 3.64 1.124 3.10 1.203 3.86 1.030 3.74 1.165 4.48 .814
Responsiveness
Students are informed of schedules and changes in schedules in advance 3.75 1.023 3.74 1.015 3.32 1.132 3.81 1.030 3.82 .942 4.18 .733
Service hours of learning facilities accommodate all students 3.74 1.028 3.70 1.048 3.51 1.142 3.80 .992 3.74 1.025 4.09 .750
Assignment, test and exam feedback and marks are made available timeously 3.52 1.136 3.48 1.085 2.66 1.407 3.58 1.048 3.80 1.025 4.09 .750
Administrative staff are quick to respond to student requests 3.19 1.161 3.24 1.134 2.78 1.207 3.32 1.134 3.05 1.199 3.59 .854
Assurance
The behaviour of teaching staff instils confidence in you 3.76 .967 3.74 .965 3.58 .885 3.78 .967 3.80 1.028 4.05 .740
Students are able to trust the administrative staff 3.54 1.022 3.55 .979 3.23 1.108 3.63 .988 3.48 1.085 3.81 .814
Administrative staff is friendly and polite 3.46 1.065 3.52 1.043 3.22 1.105 3.64 1.001 3.16 1.118 3.76 .944
Teaching staff is dependable 3.83 .949 3.83 .958 3.48 1.108 3.92 .883 3.84 .940 3.86 .910
Empathy
Teaching staff are approachable to assist with coursework queries 3.93 .940 3.89 .937 3.65 1.076 3.99 .912 3.99 .922 4.27 .703
Staff members give students individual attention when needed 3.66 1.038 3.62 1.041 3.49 1.152 3.76 .989 3.56 1.066 4.14 .774
Staff has students' best interest as a major objective 3.65 1.006 3.59 1.025 3.41 1.165 3.80 .933 3.54 1.004 4.05 .785
Staff understands the specific needs of students 3.52 1.058 3.52 1.005 3.29 1.143 3.63 1.041 3.38 1.098 4.00 .873
The department’s operating hours are convenient to students 3.88 .938 3.88 .934 3.67 .957 3.91 .943 3.88 .927 4.23 .813
Source: Research data

432
The five SERVPERF dimensions, as identified in the literature, were ranked based
on the mean statistic. Overall, empathy (M=3.73) was ranked highest, as well as
in universities A, B, and C and second in D and E. In contrast, tangibility (M=3.53)
was ranked lowest overall, as well as in universities C and E. Reliability was
ranked third by universities A, B, and C. The findings imply that the
responsiveness dimension required attention in universities A and B; tangibility
needed attention in C and E; while assurance required attention in D, although it
was also ranked second overall and in A, B and C. The dimension rank statistics
are shown in Table 2. The overall service quality score was calculated based on
the full SERVPERF model items and presented in Table 2. The findings indicate
that, in general, the service quality of supply chain education in universities is
moderate, with students from the only surveyed private university revealing
higher service quality levels. The findings further indicate that the service quality
of supply chain education at university B is relatively lower than that of its peers,
and this may call for attention from the management.
Table 2: Mean statistic and rank per dimension
Overall A B C D E
SERVPERF scale items Mean Rank Mean Rank Mean Rank Mean Rank Mean Rank Mean Rank
Empathy 3.73 1 3.701 1 3.514 1 3.815 1 3.669 2 4.136 2
Assurance 3.65 2 3.661 2 3.377 2 3.741 2 3.572 5 3.869 4
Reliability 3.64 3 3.628 3 3.333 3 3.721 3 3.584 4 4.142 1
Responsiveness 3.55 4 3.540 5 3.056 5 3.631 4 3.607 3 3.988 3
Tangibility 3.53 5 3.590 4 3.210 4 3.370 5 3.870 1 3.850 5
Service quality score 3.62 3.620 3.300 3.660 3.660 4.000
The reliability of the SERVPERF items was examined using the Cronbach’s alpha
per dimension resulting in, tangibility (0.805), reliability (0.724), responsiveness
(0.792), and empathy (0.866) all being above the minimum acceptable value of 0.6.
Assurance was 0.528, indicating low internal consistency in this dimension. All
the dimensions had significant and moderate positive correlations, signifying that
the scales measured service quality in the same direction. These are shown in
Table 3.
Table 3: Correlations
Dimensions Tangibility Reliability Responsiveness Assurance Empathy
Tangibility 1
Reliability .580** 1
Responsiveness .567** .703** 1
Assurance .432** .521** .550** 1
Empathy .512** .658** .685** .575** 1
** Correlation is significant at the 0.01 level (2-tailed).

433
Service quality model development

The theoretical model underpinning this study is the SERVPERF as presented by
Cronin and Taylor (1992). To understand the current data better and to gain more
insights, factor analysis was performed to identify the pattern structure of the
items. Factor analysis was conducted on all of the 22 SERVPERF metrics using
principal component analysis and an equamax rotation technique by suppressing
all indicators with a less than 0.5 factor-loading. The Kaiser-Meyer-Olkin measure
of sampling adequacy was 0.944, and Bartlett’s test of sphericity was statistically
significant (p-value < 0.001), revealing that the data was suitable for factor
analysis. Four factors were extracted, based on Eigenvalues greater than 1.
The overall variance explained by the four factors was 62.303 per cent, revealing
that the data explained more than 50 per cent of the variance (the factors extracted
are shown in Table 4). The reliability of all of the four extracted factors was tested,
resulting in Cronbach’s alpha (α) values for factors 1, 2, 3, and 4 as 0.802, 0.761,
0.821, and 0.895, all well above the threshold value of 0.6 (Cronin & Taylor, 1992;
Pallant, 2010), as illustrated in Table 4. The values reveal high internal consistency
and uni-dimensionality of the latent variables. Three indicators; “When students
have problems, staff are polite, even if not able to help”; “The behaviour of
teaching staff instils confidence in you”; and “The department’s operating hours
are convenient to students”, were excluded from further analysis due to cross-
loading problems. This exclusion did not prejudice the results. The overall scale
reliability for all of the 19 retained indicators resulted in α = 0.931, indicating uni-
dimensionality of the scale. The indicator loadings were generally strong, ranging
from 0.503 to 0.830. The extracted factors 1, 2, 3, and 4 were interpreted based on
the indicators loading onto them, and previous literature (Mahmoud & Khalifa,
2015; Yusoff et al., 2015) as facilities, course-centeredness, service excellence, and
academic and support staff helpfulness respectively. The interpretation helped to
identify the factors that students value regarding service quality in higher
education.
Table 4: Rotated Component Matrix

Factor Eigenvalue % of variance Reliability
loading explained (α)
Factor 1 9.835 18.616 0.802
Facilities
T1 The lecture venues have .820
modern and latest
equipment
T2 The appearance of the .830
physical facilities is
attractive
T4 Library (including online) .648
has the latest literature in
your area of interest
RL1 When something is .538
promised by a certain
time, it is always provided

434
Factor 2 1.524 15.042 0.761

Course centeredness
T3 The course material is up .600
to date and relevant
RL3 Courses are taught by .685
highly knowledgeable
teaching staff
RL4 The teaching staff respects .701
lecture and exam
schedules
E1 Teaching staff are .602
approachable to assist
with coursework queries
Factor 3 1.327 14.637 0.821
Service excellence
RL5 Assignment, test and .690
exam feedback and marks
are accurately captured
and recorded
RS1 Students are informed of .536
schedules and changes in
schedules in advance
RS2 Service hours of learning .555
facilities accommodate all
students
RS3 Assignment, test and .692
exam feedback and marks
are made available
timeously
RS4 Administrative staff are .621
quick to respond to
student requests
Factor 4 1.021 14.007 0.895
Academic and support staff
helpfulness
A2 Students are able to trust .602
the administrative staff
A3 Administrative staff is .672
friendly and polite
A4 Teaching staff is .503
dependable
E2 Staff members give .706
students individual
attention when needed
E3 Staff has students' best .744
interest as a major
objective
E4 Staff understands the .742
specific needs of students
A confirmatory factor analysis (CFA) was run using the analysis of moment
structures (AMOS) 26 to statistically test the linkages between the four latent

435
variables and indicators based on the SERVPERF model structure (Byrne, 2010).
The Chi-Square value was statistically significant, and the value 𝑋2/degree of
freedom (df) was larger than 5, signifying that the data was not fit for the specified
model. To improve the model, three low loading indicators were eliminated
stepwise while monitoring improvements of the model fit indices. Elimination of
the three indicators did not prejudice the theoretical underpinning of the model.
Modification indexes were examined and used to improve model fit. The final
model parameters and fit indexes are shown in Table 5, and the final structural
model in Figure 1. All the model estimates were statistically significant as the
critical ratios (CR) were greater than 1.96, as illustrated in the model parameters
in Table 5. The model Chi-square value was statistically significant, implying that
the data was not fit for the specified model. Other fit indexes, that is, the goodness-
of-fit index (GFI), normed-fit index (NFI), and comparative-fit index (CFI), were,
however, all above 0.9, indicating that the specified model was fit for the data
(Mahmoud & Khalifa, 2015; Schreiber, Nora, Stage, Barlow & King, 2006). The root
mean square error of approximation (RMSEA) value of 0.08 indicated that the
final model was fit, as advised by Schreiber et al. (2006), though marginally.
Table 5: CFA four-factor model parameters and fit statistics

Observed
Estimate S.E. C.R. P
variable Latent variable
SSQT4 <--- Facilities 1,000
SSQT2 <--- Facilities 1,433 ,098 14,545 ***
SSQT1 <--- Facilities 1,533 ,105 14,634 ***
SSQE1 <--- Course 1,000
SSQRL4 <--- Course ,990 ,064 15,521 ***
SSQRL3 <--- Course ,886 ,057 15,530 ***
SSQT3 <--- Course ,852 ,063 13,435 ***
SSQRS4 <--- Service 1,000
SSQRS3 <--- Service 1,027 ,052 19,811 ***
SSQRS2 <--- Service ,742 ,047 15,943 ***
SSQRL5 <--- Service ,873 ,051 17,159 ***
SSQE4 <--- Helpfulness 1,000
SSQE3 <--- Helpfulness ,988 ,039 25,292 ***
SSQE2 <--- Helpfulness ,925 ,040 23,021 ***
SSQA3 <--- Helpfulness ,886 ,043 20,448 ***
SSQA2 <--- Helpfulness ,882 ,041 21,370 ***
Chi-square 561.893, df=95; p<0.05
GFI 0.913>0.9
NFI 0.910>0.9
CFI 0.924>0.9

436
Source: research data

Figure 1: Supply chain education service quality model
The four-factor model obtained, as well as the reduced number of indicators from
the original 22, was found to be consistent with previous studies that have

437
investigated service quality in higher education using the SERVPERF model

(Mahmoud & Khalifa, 2015). By implication, the complexity of the model varies
from one higher educational context to another.
Correlations between the four extracted factors were tested. All the correlations
were statistically significant, positive, and substantive, as presented in Table 6.
The correlations are indicative of the interrelationships between the extracted
factors, implying that to achieve expected service quality levels, the four factors
should be addressed jointly.
Table 6: Dimension correlations
Factor extracted Facilities Course service Academic
centeredness excellence and
support
staff
helpfulness
Facilities 1
Course centeredness .569** 1
Service excellence .603 ** .634** 1
Academic and support staff .571 ** .642** .720** 1
helpfulness
**. Correlation is significant at the 0.01 level (2-tailed).
The difference in student perceptions on the SERVPERF scale between the various
universities was tested using one-way ANOVA. The results revealed a statistically
significant difference in all of the four extracted factors, that is, Facilities (F4,748
=10.874, ρ<0.05), Course centeredness (F4,746 =7.233, ρ<0.05), service excellence
(F4,753 =11.686, ρ<0.05) and academic and support staff helpfulness (F4,750 =5.138,
ρ<0.05). The post hoc tests were conducted using the Tukey test to reveal the
specific groups that had significant differences. The test results were examined for
each of the four factors. Under facilities, specific differences were observed
between all the universities, except between university A and E (ρ=0.602),
university B and C (ρ=0.546), C and E (0.065), and D and E (ρ=1.00). The specific
differences are illustrated in the post hoc test Table B in the Appendix.
Table 7: Results of the one-way ANOVA

Sum of
Squares df Mean Square F Sig.
Facilities Between Groups 31.264 4 7.816 10.874 .000
Within Groups 537.625 748 .719
Course Between Groups 13.653 4 3.413 7.233 .000
centeredness Within Groups 352.036 746 .472
Service Between Groups 31.025 4 7.756 11.686 .000
excellence Within Groups 497.116 749 .664
Academic and Between Groups 13.854 4 3.464 5.138 .000
support staff Within Groups 502.917 746 .674
helpfulness

438
The specific significant differences were identified from the post hoc tests for each
of the factors. For facilities, University A was significantly different from B and C;
B was significantly different from D and E; and C was significantly different from
D and E, as illustrated in Table A (Appendix).
The extracted factors were ranked using the mean statistic. Course-centeredness
is ranked top, while Facilities were ranked lowest. This might imply that service
quality issues related to facilities require immediate attention to improve the
teaching and learning of supply chain management in universities. To better
understand the meaning of the responses, as shown in Table 8; the service quality
levels were categorised as: low (<3.0), medium (≥3.0<4.0), and high (≥4.0),
according to Rodrigues et al. (2011). As such, the overall service quality of supply
chain education is at medium (Mean = 3.637) level.
Table 8: Factor rankings
Extracted factors Mean Std. Deviation
Course-centeredness 4.0223 .69827
Academic and support staff helpfulness 3.6105 .83008
Service excellence 3.5846 .83749
Facilities 3.3303 .86977
5. Discussion
The objectives of this study were: firstly, to determine the level of service quality
of supply chain education in universities; secondly, to establish service quality
factors that supply chain students consider important in universities and model
the factors based on the SERVPERF model; thirdly, to establish whether there
were service quality differences across the selected universities. Each of the
objectives is discussed in the following paragraphs.
The measurement of the service quality of higher education is essential in
identifying the areas or dimensions that students are not satisfied with, to take
corrective action. Yusoff et al. (2015) established that service quality is a significant
determinant of customer satisfaction in higher education. Overall, the service
quality of supply chain education is rated at a medium level. This finding agrees
with other studies that have investigated service quality in higher education,
although the contexts differ, given that some originate from developed countries
(Rodrigues et al., 2011; Galeeva, 2016). Whilst service quality relating to the course
centeredness was high, signifying that the content and instructional methodologies
were up to date, universities should improve facilities. Facilities may include
adequate and conducive lecture venues, modernised libraries as well as the
availability of recreational facilities. The finding that universities are struggling
with inadequate facilities, that are key to offering high-quality education,
supports Munene’s (2016) observation that many public universities do not have
adequate facilities.
Underlying dimensions of service quality from the perspective of a student were
identified as 1) course-centeredness, 2) academic and support staff helpfulness, 3) service
excellence, and 4) facilities. This finding supported the observations of Sultan and
Yin Wong (2012) who identified academic, administration, and facilities as core

439
aspects of service quality in higher education as well as Mahmoud and Khalifa

(2015), who identified faculty individualised attention, support staff helpfulness,
and support staff empathy as critical factors for students. However, the current
model is more enriched having identified four factors including course-related
materials, general service excellence, and staff helpfulness when compared to
Sultan and Yin Wong (2012) and Mahmoud and Khalifa (2015) models. Course-
centeredness refers to the service quality elements that relate to supply chain
modules; such as the relevance of the learning materials as well as the quality of
the teaching staff in this area. The extraction of course-centeredness as one of the
factors of service quality supports Abdullah’s (2005) study, in which course
content was also identified as a service quality aspect in higher education. The
students rated this factor highly, implying that highly qualified staff teach the
supply chain courses and that the course materials are up to date. The finding
supported the claim by Purgailis and Zaksa (2012), that study content is a
significant consideration when assessing the service quality of higher education.
Academic and support staff helpfulness was identified as a measure of service
quality in supply chain education. Staff helpfulness generally refers to the
readiness to understand the individual student’s needs and meeting them. In
support of this study’s findings, Yusoff et al. (2015) identified staff helpfulness as
one of the critical factors that influence student helpfulness perceptions. The
contributions of staff to service quality was also expressed by Mahmoud and
Khalif (2015), who categorised it as faculty individualised attention, and support
staff helpfulness and empathy, which they termed as the main dimensions of
service quality in higher education. Besides, the finding on the excellence of
academic staff as a measure of quality in higher education supports
Atwebembeire, Musaazi, Ssentamu and Malunda (2018) study that established a
positive link between academic staff excellence and quality of university
education. The students rated the service excellence aspect as being of average
quality, implying that it requires attention from management. Issues affecting
service excellence might include not keeping to schedules of service hours;
administrative staff not responding adequately to student requests; as well as late
feedback and low accuracy, especially in tests and assignments. Finally, the
facilities aspect of service quality was the lowest rated, possibly echoing the
various reports on higher education issues, as observed by Munene (2016).
Facilities may relate to inadequate library resources, including online access,
unattractive lecture rooms, a lack of appropriate furniture, low lighting, and a lack
of equipment like projectors. Inadequate facilities affect service quality negatively,
as also claimed by Brochado (2009) and Yusoff et al. (2015).
In addition to identifying the factors that students value, a four-factor supply
chain education service quality model was developed. The model is based on 16
indicators, which were clustered into facilities (3 indicators), course centredness (4
indicators), service excellence (4 indicators), and academic and support staff helpfulness
(5 indicators) as opposed to the 22 in Cronin and Taylor (1992) study. The
indicators identified are likely to be the most relevant measures of the service
quality of supply chain education. The resultant model is expected to encourage
research on this area given that the SERVPERF model applied in developed
countries (for example, Cronin and Taylor model) may not be used or fit directly
in another context, more especially in a developing country context.

440
The differences in service quality between the universities were established using
a one-way ANOVA test. There were statistically significant differences in service
quality between the universities, as tested for each of the factors that students
value. The test factors were course-centeredness, academic and support staff
helpfulness, service excellence, and facilities. The differences indicate that
universities deliver varying levels of service quality to supply chain students. By
implication, there is little uniformity in terms of the quality of course materials,
learning facilities, staff helpfulness, or commitment to service excellence.
However, it is also essential to observe that there could be other factors that were
not surveyed in this study, which could have contributed to the differences; for
example, class sizes, the appearance of facilities, and the general perceptions of
support staff readiness (Brochado, 2009). It is worth highlighting that universities
can improve overall service quality by addressing the four factors identified
jointly, as opposed to individually.
6. Conclusion, implications and future research

The measurement of service quality is important to higher education managers as
it has a direct influence on student satisfaction as well as on performance (Yusoff
et al., 2015). In this study, the service quality level of supply chain education from
the perspective of a student is considered to be medium. This could imply that the
students are generally indifferent to the service they receive from the universities.
This finding should inform management that supply chain students might not be
impressed with the overall service quality, especially in areas related to learning
facilities, service assurance, and empathy. The areas mentioned above are
recommended as focus areas for university management to improve students’
perceptions of service quality.
The generic five dimensions of service quality, as expounded by Cronin and
Taylor (1992), might not be directly applicable to every context. As such, supply
chain students identified four factors or dimensions that influence their
perceptions of service quality, that is, course-centeredness, staff helpfulness,
service excellence, and facilities. Therefore, supply chain education managers
could be guided more specifically on areas or factors that students value and on
which they can consequently focus to improve student perceptions of service
quality. The differences might imply that supply chain students experience a
highly varied service quality environment across different universities. Supply
chain education managers are called upon to coordinate efforts to improve the
learning experience of students, especially in terms of facilities and commitment
to service.
This study’s contribution to the body of supply chain skills knowledge can be seen
as four-fold. First, the service quality of supply chain education has been
established using the SERVPERF tool. Second, four aspects of service quality
considered as essential to students were identified; third, a modified SEVPERF
model of supply chain education service quality has been developed; and fourth,
service quality was found to be different from one institution to the other, based
on the four extracted factors that students value, calling for a coordinated
approach in the management of supply chain education in Kenya.

441
The current research findings might be limited to the selected context; however,
given the convincing inferential statistics obtained, it can be generalised to all
universities in Kenya. Based on the findings of this study there are opportunities
for further research, for example, (1) advancing theory on quality of supply chain
education through a comparative analysis of students’ perceptions from other
contexts among the developing countries; (2) application of different research
methodologies and instruments such as HEdPERF, and comparing results across
universities and countries; and (3) replications of the study in future to monitor
changes in service quality of supply chain education in the same context. This will
measure whether managerial decisions are taken in the universities to improve
the service quality of supply chain education.
7. References
Abdullah, F. (2005). HEdPERF versus SERVPERF: The quest for ideal measuring
instrument of service quality in higher education sector. Quality Assurance in
education, 13(4), 305-328. https://doi.org/10.1108/09684880510626584
Abdullah, F. (2006). The development of HEdPERF: a new measuring instrument of
service quality for the higher education sector. International Journal of Consumer
Studies, 30(6), 569-581. https://doi.org/10.1111/j.1470-6431.2005.00480.x
Akbari, M. (2018). Logistics outsourcing: a structured literature review. Benchmarking: An
International Journal, 25(5), 1548-1580. https://doi.org/10.1108/BIJ-04-2017-0066
Amini-Philips, C., & Akpoyowaire, M. S. (2016). Policy of carrying capacity and access to
university education in Nigeria: issues, challenges and the way forward.
International Journal of Learning, Teaching and Educational Research, 15(9), 55-70.
Armstrong, J. S., & Overton, T. S. (1977). Estimating nonresponse bias in mail surveys.
Journal of marketing research, 14(3), 396-402. Retrieved from
https://repository.upenn.edu/marketing_papers/17
Atwebembeire, J., Musaazi, J., Ssentamu, P. N., & Malunda, P. (2018). Performance
monitoring and quality teaching and research in Private Universities in Uganda.
International Journal of Learning, Teaching and Educational Research, 17(10), 70-85.
Brochado, A. (2009). Comparing alternative instruments to measure service quality in
higher education. Quality Assurance in education, 17(2), 174-190.
https://doi.org/10.1108/09684880910951381
Byrne, B. M. (2010), Structural Equation Modeling with AMOS: Basic Concepts, Applications,
and Programming, Routledge, Taylor & Francis Group, LLC, New York, NY.
Colicchia, C., Creazza, A., & Strozzi, F. (2018). Citation network analysis for supporting
continuous improvement in Higher Education. Studies in Higher Education, 43(9),
637-1653.
Commission of University Education. (2018). Approved Academic Programmes Offered
Universities in Kenya November 2018. Retrieved from
http://www.cue.or.ke/index.php/approved-academic-programmes.
Cronin Jr, J. J., & Taylor, S. A. (1992). Measuring service quality: a reexamination and
extension. Journal of marketing, 56(3), 55-68. https://doi.org/10.2307/1252296
Erturgut, R., & Soysekerci, S. (2011). Professional manager education on logistics and
supply chain management. Procedia - Social and Behavioral Sciences, 15, 2771-2775.
Galeeva, R. B. (2016). SERVQUAL application and adaptation for educational service
quality assessments in Russian higher education. Quality Assurance in Education,
24(3), 329-348. https://doi.org/10.1108/QAE-06-2015-0024

442
Gonzalez, M. E., Quesada, G., Gourdin, K., & Hartley, M. (2008). Designing a supply chain
management academic curriculum using QFD and benchmarking. Quality
Assurance in Education, 16(1), 36-60.
Gustafsson, M., & Larsson, J. (1997). Service quality in higher education. [Online]
Retrieved http://www.diva-portal.org/smash/get/diva2:645830/fulltext01.pdf
Hall, S. (2017). Youth Employment in Kenya. Retrieved from
https://www.britishcouncil.co.ke/sites/.../ng_kenya_youth_employment_in_k
enya.pdf
Holweg, M., & Bicheno, J. (2002). Supply chain simulation – a tool for education,
enhancement and endeavour. International Journal of Production Economics, 78(2),
163-175.
In, J. (2017). Introduction of a pilot study. Korean journal of anesthesiology, 70(6), 601-605.
Jain, S. K., & Gupta, G. (2004). Measuring Service Quality: SERVQUAL vs. SERVPERF
Scales. VIKALPA, 29(2), 25-37.
Leon, S., & Uddin, N. (2016). Finding supply chain talent: an outreach strategy. Supply
Chain Management: An International Journal, 21(1), 20-44.
https://doi.org/10.1108/SCM-02-2015-0084
Li, L. (2020). Education supply chain in the era of Industry 4.0. Systems Research and
Behavioral Science, 37(4), 579-592.
Lorentz, H., Töyli, J., Solakivi, T., & Ojala, L. (2013). Priorities and determinants for supply
chain management skills development in manufacturing firms. Supply Chain
Management, 18(4), 358-375. https://doi.org/10.1108/SCM-03-2012-0111
Lubwama, J., Onen, D., & Kasenene, E. S. (2017). Influence of Management on Quality
Assurance in National Teacher’s Colleges. International Journal of Learning,
Teaching and Educational Research, 16(11), 138-156.
Luke, R., & Heyns, G.J. (2018). Determining the quality of supply chain management education
at a South African university using an adapted SERVPERF model. International
Business Conference Proceedings 2018, September 2018. North-West University,
Mauritius.
Luke, R., & Heyns, G. J. (2019). Skills requirements in South African supply chains: a
higher education perspective. South African Journal of Higher Education, 33, 156-170.
Mageto, J., & Luke, R. (2020). Skills frameworks : a focus on supply chains. Journal of
Transport and Supply Chain Management, 14(1), 1-17.
Mahmoud, A. B., & Khalifa, B. (2015). A confirmatory factor analysis for SERVPERF
instrument based on a sample of students from Syrian universities. Education+
Training, 57(3), 343-359. https://doi.org/10.1108/ET-04-2014-0038
Mittal, R. K., Garg, N., & Yadav, S. K. (2018). Quality assessment framework for
educational institutions in technical education: a literature survey. On the Horizon,
26(3), 270-280. https://doi.org/10.1108/OTH-08-2017-0066
Mohamedbhai, G. (2014). Massification in Higher Education Institutions in Africa: Causes,
Consequences and Responses. International Journal of African Higher Education,
1(1). https://doi.org/10.6017/ijahe.v1i1.5644
Munene, I. (2016). Kenya’s universities are in the grip of a quality crisis. Retrieved from
https://theconversation.com/kenyas-universities-are-in-the-grip-of-a-quality-
crisis-54664.
Nadiri, H., Kandampully, J., & Hussain, K. (2009). Students' perceptions of service quality
in higher education. Total Quality Management, 20(5), 523-535.
https://doi.org/10.1080/14783360902863713
Pallant, J. (2010). SPSS Survival Manual (4th edition). Berkshire.

443
Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1988). Servqual: A multiple-item scale
for measuring consumer perc. Journal of retailing, 64(1), 12. Retrieved from
https://www.msi.org/reports/servqual-a-multiple-item-scale-for-measuring-
customer-perceptions-of-servic/
Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1985). A conceptual model of service
quality and its implications for future research. Journal of marketing, 49(4), 41-50.
https://doi.org/10.2307/1251430
Ponto, J. (2015). Understanding and evaluating survey research. Journal of the advanced
practitioner in oncology, 6(2), 168-171. Retrieved from
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4601897/
Purgailis, M., & Zaksa, K. (2012). The impact of perceived service quality on student
loyalty in higher education institutions. Journal of Business Management, 4(6), 138-
152. Retrieved from https://www.riseba.lv/sites/default/files/inline-
files/jbm2012.pdf#page=138
Purnomo, M. R. A., Anugerah, A. R., & Taradipa, B. (2020). Sustainable Supply Chain
Management Framework in a Higher Education Laboratory Using Intuitionistic
Fuzzy Cognitive Map. Journal of Industrial Engineering & Management, 13(2), 417-
429.
Rodrigues, L. L., Barkur, G., Varambally, K. V. M., & Golrooy Motlagh, F. (2011).
Comparison of SERVQUAL and SERVPERF metrics: an empirical study. The TQM
Journal, 23(6), 629-643. https://doi.org/10.1108/17542731111175248
Sauber, M. H., McSurely, H. B., & Tummala, V. M. R. (2008). Developing supply chain
management program: a competency model. Quality Assurance in Education, 16(4),
375-391.
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural
equation modeling and confirmatory factor analysis results: A review. The Journal
of educational research, 99(6), 323-338. https://doi.org/10.3200/JOER.99.6.323-338
Seth, N., Deshmukh, S. G., & Vrat, P. (2005). Service quality models: a review. International
journal of quality & reliability management, 22(9), 913-949.
https://doi.org/10.1108/02656710510625211
Shabani, J., Okebukola, P., & Oyewole, O. (2014). Quality Assurance in Africa: Towards a
Continental Higher Education and Research Space. International Journal of African
Higher Education, 1(1). https://doi.org/10.6017/ijahe.v1i1.5646
Silva, D. S., Moraes, G. H. S. M. D., Makiya, I. K. & Cesar, F. I. G. (2017). Measurement of
perceived service quality in higher education institutions: A review of HEdPERF
scale use. Quality Assurance in Education, 25(4), 415-
439. https://doi.org/10.1108/QAE-10-2016-0058
Souca, M. L. (2011). SERVQUAL - Thirty years of research on service quality with implications
for customer satisfaction. s.l., Cluj-Napoca: Babes Bolyai University.
Sultan, P., & Yin Wong, H. (2012). Service quality in a higher education context: an
integrated model. Asia Pacific Journal of Marketing and Logistics, 24(5), 755-784.
https://doi.org/10.1108/13555851211278196
Sun, L., & Song, G. (2018). Current state and future potential of logistics and supply chain
education: a literature review. Journal of International Education in Business, 11(2),
124-143. https://doi.org/10.1108/JIEB-10-2017-0039
Teeroovengadum, V., Kamalanabhan, T. J., & Seebaluck, A. K. (2016). Measuring service
quality in higher education: Development of a hierarchical model (HESQUAL).
Quality Assurance in Education, 24(2), 244-258. https://doi.org/10.1108/QAE-06-
2014-0028

444
Verma, S., & Prasad, R. K. (2017). The MEQUAL scale: measure of service quality in
management education. International Journal of Comparative Education and
Development, 19(4), 193-206. https://doi.org/10.1108/IJCED-12-2016-0024
Wagner, S. M., & Kemmerling, R. (2010). Handling nonresponse in logistics research.
Journal of Business Logistics, 31(2), 357-381. https://doi.org/10.1002/j.2158-
1592.2010.tb00156.x
Yusoff, M., McLeay, F., & Woodruffe-Burton, H. (2015). Dimensions driving business
student satisfaction in higher education. Quality Assurance in Education, 23(1), 86-
104. https://doi.org/10.1108/QAE-08-2013-0035

445
Appendix
Table A: Multiple Comparisons
Tukey HSD
Mean 95% Confidence Interval
Dependent (I) Name of (J) Name of Difference (I-
Variable University University J) Std. Error Sig. Lower Bound Upper Bound
Facilities A B .31119* .11093 .041 .0078 .6145
C .23540* .07775 .021 .0228 .4480
D -.22137 .08978 .100 -.4669 .0241
E -.36481 .19029 .309 -.8852 .1555
B A -.31119* .11093 .041 -.6145 -.0078
C -.07578 .10616 .953 -.3661 .2145
D -.53256* .11526 .000 -.8477 -.2174
E -.67600* .20356 .008 -1.2326 -.1194
C A -.23540* .07775 .021 -.4480 -.0228
B .07578 .10616 .953 -.2145 .3661
D -.45677* .08381 .000 -.6860 -.2276
E -.60021* .18755 .012 -1.1131 -.0874
D A .22137 .08978 .100 -.0241 .4669
B .53256* .11526 .000 .2174 .8477
C .45677* .08381 .000 .2276 .6860
E -.14344 .19285 .946 -.6708 .3839
E A .36481 .19029 .309 -.1555 .8852
B .67600* .20356 .008 .1194 1.2326
C .60021* .18755 .012 .0874 1.1131
D .14344 .19285 .946 -.3839 .6708
Course A B .34522* .08989 .001 .0994 .5910
centeredness C -.05945 .06309 .880 -.2320 .1131
D -.09327 .07262 .701 -.2919 .1053
E -.27586 .15747 .403 -.7065 .1547
B A -.34522* .08989 .001 -.5910 -.0994
C -.40466* .08609 .000 -.6401 -.1693
D -.43849* .09330 .000 -.6936 -.1834
E -.62108* .16801 .002 -1.0805 -.1617
C A .05945 .06309 .880 -.1131 .2320
B .40466* .08609 .000 .1693 .6401
D -.03383 .06786 .988 -.2194 .1517
E -.21642 .15533 .632 -.6412 .2083
D A .09327 .07262 .701 -.1053 .2919
B .43849* .09330 .000 .1834 .6936
C .03383 .06786 .988 -.1517 .2194
E -.18259 .15944 .782 -.6186 .2534
E A .27586 .15747 .403 -.1547 .7065
B .62108* .16801 .002 .1617 1.0805
C .21642 .15533 .632 -.2083 .6412
D .18259 .15944 .782 -.2534 .6186
Service excellence A B .50379* .10707 .000 .2110 .7966
C -.11183 .07471 .565 -.3161 .0925
D -.07152 .08583 .920 -.3062 .1632
E -.54877* .18675 .028 -1.0594 -.0381
B A -.50379* .10707 .000 -.7966 -.2110
C -.61562* .10250 .000 -.8959 -.3353
D -.57531* .11086 .000 -.8785 -.2722
E -1.05256* .19950 .000 -1.5981 -.5070
C A .11183 .07471 .565 -.0925 .3161
B .61562* .10250 .000 .3353 .8959
D .04031 .08006 .987 -.1786 .2592
E -.43693 .18417 .124 -.9405 .0667
D A .07152 .08583 .920 -.1632 .3062
B .57531* .11086 .000 .2722 .8785
C -.04031 .08006 .987 -.2592 .1786
E -.47725 .18895 .086 -.9939 .0394
E A .54877* .18675 .028 .0381 1.0594
B 1.05256* .19950 .000 .5070 1.5981
C .43693 .18417 .124 -.0667 .9405
D .47725 .18895 .086 -.0394 .9939
A B .25662 .10736 .119 -.0370 .5502
C -.12032 .07536 .500 -.3264 .0857

446
Academic and D .10725 .08671 .730 -.1298 .3443

support staff
helpfulness E -.34617 .18817 .351 -.8607 .1684
B A -.25662 .10736 .119 -.5502 .0370
C -.37693* .10293 .002 -.6584 -.0955
D -.14936 .11151 .667 -.4543 .1556
E -.60279* .20081 .023 -1.1519 -.0537
C A .12032 .07536 .500 -.0857 .3264
B .37693* .10293 .002 .0955 .6584
D .22757* .08116 .041 .0056 .4495
E -.22586 .18568 .742 -.7336 .2819
D A -.10725 .08671 .730 -.3443 .1298
B .14936 .11151 .667 -.1556 .4543
C -.22757* .08116 .041 -.4495 -.0056
E -.45342 .19057 .122 -.9745 .0677
E A .34617 .18817 .351 -.1684 .8607
B .60279* .20081 .023 .0537 1.1519
C .22586 .18568 .742 -.2819 .7336
D .45342 .19057 .122 -.0677 .9745
*. The mean difference is significant at the 0.05 level.

447

Vol. 19, No. 8, pp. 447-458, August 2020
Exploring the Content Knowledge of Accounting

Teachers in Rural Contexts:
A Call for a Decoloniality Approach
Habasisa Vincent Molise

Sol Plaatje University, Kimberly, South Africa
https://orcid.org/0000-0002-3971-0718
Abstract. The purpose of this paper was to investigate deficiencies in the

teaching and learning of financial literacy, which is the accounting part of
the subject economic and management sciences in Grade 9 in South
Africa. The content knowledge of Grade 9 accounting teachers at rural
schools compared to those at urban schools, remains a challenge, as many
of the former lack qualifications. Teachers at rural schools are considered
to have inadequate content knowledge of accounting, owing to a lack of
training and continuous professional development. The sample
comprised 89 Grade 9 teachers. The study used secondary data that was
collected with a questionnaire on different accounting topics to diagnose
the difficulties teachers experience teaching accounting content. The
items of the questionnaire were clustered into three constructs: analysis
and recording of transactions on the accounting equation, understanding
accounting concepts and connections between topics and application of
accounting principles, and analysis of transactions in subsidiary journals.
The results show that there are deficiencies relating to teachers' content
knowledge and competence. This finding suggests that teachers in rural
areas should undergo professional development to prepare them to teach
learners at rural schools. This development should include establishing
professional learning communities for sharing scarce instructional
resources between various schools.
Keywords: Accounting education; Accounting teachers; Content

knowledge; Competence; Decoloniality approach
1. Introduction
The curriculum practice of economics and management sciences (EMS) remains a
challenge to teachers, mainly due to their lack of content knowledge (Pereira &
Sithole, 2020; Alhassan, Abukari & Samari, 2018). Teachers have inadequate
content knowledge because of teaching subject mismatches, because they lack of
qualifications, and did not major in accounting (Ward, He, Wang & Li, 2018;
Modise, 2016). Accounting teachers in rural schools do not possess sound
knowledge of the subject content they teach (Ward et al., 2018; Ngwenya, 2014).

448
These teachers are not professionally supported to improve their content

knowledge and practice, which compromises the quality of teaching in accounting
(Mohangi, Krog, Stevens & Nel, 2016). According to Hlalele (2012), rural schools
are regarded as deficient, because teachers are incorrectly placed and they teach
subjects they did not specialise in during training. In most cases, teachers are
unwilling to work in rural schools because they struggle with transport from their
places of residence to these schools (Manda, 2014).
I decided to investigate deficiencies in the content knowledge of accounting

teachers at rural schools. Kwarteng (2016) suggests that all teachers ensure they
have mastered the content knowledge they teach. I argue that ensuring this
mastery is the responsibility of subject advisors/learning facilitators (Diseko &
Modiba, 2018). Subject advisors are Department of Basic Education officials who
provide professional development support to in-service teachers in the form of
workshops. Therefore, subject advisors have a vested responsibility in identifying
the content gaps that teachers may have regarding the teaching of accounting
(Gegenfurtner, Lewalter, Lehtinen, Schmidt & Gruber, 2020), and should support
teachers by providing the necessary training/workshops. Mukeredzi (2015)
explains that professional development of teachers is considered crucial if
teachers are to fulfil the requirements for quality teaching and learning of
accounting, especially regarding financial literacy, which is part of accounting,
and which is being neglected by teachers in their teaching (Ngwenya & Arek-
Bawa, 2019).
According to Ngwenya and Arek-Bawa (2020), effective teaching requires

teachers to have a deep understanding of the concepts of the subject they teach, in
this case, accounting equations, accounting concepts and accounting principles.
This view is supported by Modise (2016), who asserts that, “an understanding of
accounting concepts and how they are related and organized, enables teachers to
draw on the content knowledge of the subject in teaching”. However, Ngwenya
(2019) asserts that teachers are left to struggle with content knowledge challenges
and receive little support. Therefore, I argue that rural school teachers are not
prepared for their lessons and do not clearly understand what is expected of them
in every lesson.
Akpanobong and Asuquo (2015) assert that effective teaching of accounting

requires knowledge integration and skills across topics, and the ability to
manipulate accounting equations (Matos, 2020). Findings of a study by Ngwenya
and Maistry (2012) indicate that teachers lack the necessary accounting skills to
teach the content, which provides learners with few opportunities for logical
reasoning in accounting (Hine & Thai, 2019). Similarly, Manda (2014) confirms
that this shortcoming is problematic, as it leaves accounting lessons being taught
with a deficit approach, and compromises quality teaching in the subject.
Ngwenya’s findings corroborate that “teachers who lack subject content
knowledge are ill-equipped” to explain and present topics in ways that make
sense to learners (2014). I argue, therefore, that when teachers lack subject content
knowledge, they tend to promote surface learning, instead of deep learning.

449
Da Costa (2020) maintains that teachers with adequate content knowledge are able
to make important connections among topics, including using context-based
examples when explaining the content to learners (Ngwenya & Arek-Bawa, 2020;
Shepherd, 2013). In turn, Asadullah (2016) found that rural school teachers
struggle to show how the content they teach connects to the learners’ real-life
experiences (Bietenbeck, Piopiunik & Wiederhold, 2018). I argue that there is a
need to reconsider the way teaching and learning of accounting are
conceptualised, especially in rural schools, to deal with teachers’ content
knowledge deficiencies.
2. Decoloniality as a theoretical lens

This paper is grounded on decoloniality, which serves as a theoretical lens.
Decoloniality was used as an approach to deconstruct and dismantle the
narratives and discourses that exist in the curriculum practice of accounting
teachers in rural contexts, and the main challenges they face in their classrooms
(Dube & Molise, 2018). Decoloniality is a vehicle for human emancipation, as it
considers suppressed voices (Ndlovu-Gatsheni, 2013); in this case, it is meant to
rehumanise accounting teachers, who have been subject to systemic oppression
by colonisation, which is prevalent in rural classrooms (Vorster & Quinn, 2017).
This view is corroborated by Luckett (2016), who explains that teachers are
usually forced to accept and implement the curriculum by district officials,
without the opportunity to provide inputs or to share best practices they have
acquired from teaching in rural contexts. However, Zipin, Fataar and Brenann
(2015) state that subject advisors' knowledge is considered to be the only powerful
knowledge (Adam, 2020), because of the powerful status advisors assume in
relation to their rural school-based counterparts (Exley, Whatman & Singh, 2018).
Therefore, using a decoloniality approach in this study allowed teachers to speak
openly about the challenges they faced in teaching accounting.
The researcher is of the view that some challenges teachers face relate to subject
allocation mismatches, lack of suitable qualifications, and ineffective support by
the district, which are drawbacks that position teachers to fail to execute the
accounting curriculum as intended by the Department of Basic Education
(Lockley, 2018). Decoloniality is also about gaining and building curriculum
consciousness on these shortcomings, which, independently and progressively,
encourages optimum implementation and curriculum delivery in rural contexts,
so that a Eurocentric environment, similar to that of the oppressor, is
circumvented (Seehawer, 2018).
In this paper, decolonising the accounting curriculum meant using local, relevant
examples of accounting and applying knowledge of accounting in the teachers’
contexts, or reorienting certain accounting concepts to address content problems,
such as analysis and recording of transactions on the accounting equation,
understanding accounting concepts, linking topics, applying accounting
principles and analysing transactions in subsidiary journals (South Africa,
Department of Basic Education, 2011). Le Grange (2016) concludes that
decolonised curriculum practice for accounting in rural contexts would require
discarding the wrong writings of the oppressor, and interrogating distortions of

450
people's life experiences, negative labelling, deficit theorising and genetically or

culturally deficient models that pathologise rural accounting teachers; retelling
stories of the past and envisioning the future (Ndlovu-Gatsheni, 2013). This idea
is supported by Fataar (2012), who calls for reinvigorating production of
knowledge that is relevant, effective, and empowering for accounting teachers in
rural contexts.
Therefore, to counter the Eurocentrism of accounting teaching in rural schools, we

need to decolonise the curriculum and content in accounting classrooms (Hlalele,
2012). I hold the view that doing so will maintain a social order and rules through
which an effective culture of accounting teaching can evolve; foster an
understanding of rural teachers’ consciousness; facilitate a critical emancipatory
approach to solving accounting problems teachers experience in accounting
lessons; and produce material and capacities for accounting teachers to know how
the content should be presented effectively in their classrooms (Pereira & Sithole,
2019). The knowledge system that was geared to promote colonisation of the
accounting curriculum paid little attention to affirming the indigenous
knowledge of rural teachers, which resulted in “book knowledge” production
(Seehawer, 2018). Salami and Okeke (2017) argue that knowledge production is
not about memorising facts, but is, instead, a learning experience that arises from
contextualised experiences. I argue in this paper that decolonising accounting
teaching should incorporate inclusion, recognition, and affirmation of rural
contextual knowledge, which is seldom considered by subject advisors when they
provide opportunities for the professional development of accounting teachers,
so that teachers can execute curriculum effectively during their lessons.
3. Unpacking realities in the teaching of accounting in rural contexts

“Rurality” has always been associated with “lack and characterized by loss”
(Alhassan, et al, 2018; Hlalele, 2012). However, rural schools have always
survived, regardless of the material conditions of teaching, such as lack of
resources, which exist in that context. Thus, accounting teachers need to view
these shortcomings as an opportunity to try out new ways of teaching, instead of
complaining about the support they receive, to ensure effective curriculum
delivery (Pereira & Sithole, 2020). For instance, accounting teachers need to ensure
that the curriculum they teach is relevant to the context in which they teach, by
taking advantage of the unique experiences of the learners, to utilise learners’
prior knowledge about accounting concepts (Lindsjö, 2018). I argue that, unless
the content taught to learners in rural contexts is made relevant to their real-life
experiences, then teaching and learning will remain deficient.
According to Qhosola (2015), accounting subject content knowledge includes

“knowledge of facts, concepts, procedures, and the relationships among them;
knowledge of the ways that subject ideas can be represented (Ward et al., 2018;
Manda, 2014), and knowledge of the subject as a discipline” (Ngwenya, 2019;
Kwarteng, 2016. Research has shown that the lack of subject content knowledge is
likely to influence the way teachers teach content in their lessons (De Lange, Khau
& Athiemoolam, 2014). Lindsjö (2018) emphasises that teachers’ subject content
knowledge is important for ensuring that relevant content is taught to the learners

451
(Ngwenya & Arek-Bawa, 2019). This means that the mastery of content
knowledge is key to ensuring effective teaching and learning in the accounting
curriculum.
Quan-Baffour and Arko-Achemfuor (2009) assert that, “accounting teachers are

not in touch with market expectations. In addition to teaching accounting topics,
teachers are expected to have a broad range of knowledge, skills, and abilities
(Ward et al., 2018)”. If learners perform well, the honour goes to the school and
teachers; however, if they perform badly, the blame ultimately falls on the teacher
alone (Da Costa, 2020). I argue that, if teachers receive proper training in, for
instance, workshops on the content, they are likely to teach effectively, with
confidence, and can motivate learners to like their subjects and work harder and,
subsequently, improve accounting results. Mastery of the content can reduce
anxiety in accounting teachers and their learners, because both parties become
more familiar with the course material and incorporate new methods of teaching
and learning accounting (Hine & Thai, 2019). Omotayo (2014) maintains that it is
important for accounting teachers to understand the motives, expectations, and
preparedness of learners (Matos, 2020), so that teachers can develop learning
environments that promote high-quality learning outcomes. I am of the view that,
if teachers are to satisfy the requirements of curriculum reform in accounting, they
need to refine their content knowledge.
4. Research design and methods

A quantitative research approach that incorporated a content-based questionnaire
was used to investigate the content knowledge deficiencies of accounting
teachers. The participants, data collection instruments and analysis are described
as follows.
4.1 Participants
The data were generated through the administration of a questionnaire to Grade
9 EMS teachers from 15 schools in Thabo Mofutsanyane education district in the
Free State province. The questionnaire consisted of two sections. Section A
comprised 25 accounting questions, which were structured so that different
accounting topics were assessed, to diagnose whether teachers experienced
difficulties in understanding the content of accounting. Section B investigated the
biographical characteristics of teachers. A total of 89 Grade 9 teachers participated:
45 men and 44 women.
4.2 Instrumentation
The content-based questionnaire was used to generate data on the content
challenges that accounting teachers experience as a result of lack of content
knowledge. The questionnaire was used to identify the specific problem areas of
accounting content, so as to understand the kind of support teachers need to
improve their content knowledge. The questionnaire was structured in a way that
that ensured that respondents understood the questions, and how they were
supposed to respond to questions. To ensure clarity, questions were arranged
from simple to complex questions phrased at different cognitive levels.

452
The data were collected by administering the questionnaire at the selected schools,
which were sampled because the EMS curriculum was taught at these schools. It
was easy to administer the questionnaire at schools, because the centres were
easily accessible to teachers in the location and their neighbouring schools. I
involved other people to help administer and invigilate the questionnaire at
certain schools, and the university involved always had a representative
overseeing the specific centres.
The content-based questionnaire was structured such that it addressed three main
constructs of Grade 9 accounting themes. From the themes, the following
constructs were developed to determine the content knowledge level of
accounting teachers: analysis and recording of transactions on the accounting
equation; understanding of accounting concepts and connections between topics;
application of accounting principles and analysis of transactions in the subsidiary
journals. These three constructs served as important elements in assessing the
teachers' content knowledge.
The data collected through the questionnaire instrument were captured in

Microsoft Excel files which were imported into the Matlab software package for
analysis. In this paper, I use a quantitative approach. I calculated the frequency
for each answer given per question, and express it as percentages. I use frequency
table to visualise the frequencies.
4.3 Validity and reliability

The accounting content items included in the questionnaire were selected to
assess different cognitive levels. The questionnaire was moderated by the senior
subject advisor for accounting in the Free State province. Furthermore, the
questionnaire was administered under the supervision of the researcher and
research assistants.
5. Findings
This section will discuss the analysis and interpretation of data that were gathered
through the content-based questionnaire that was organised according to the
following themes: analysis and recording of transactions on the accounting
equation; understanding of accounting concepts and connections between topics;
application of accounting principles and analysis of transactions in the subsidiary
journals.
Table 1: Frequencies of applicable content knowledge

%
Transactions based on % correct
Themes/Constructs incorrect
accounting content responses
responses
Theme 1: Analysis and Profit mark-up 27.5% 72.5%
recording of transactions on Bad debts 27.5% 72.5%
the accounting equation Debtors/creditors 32.5% 67.5%
Average % per construct 29.2% 70.8%
Assets 10.9% 89.1%
Income 12.5% 87.5%

453
Theme 2: Understanding Expenses 17.5% 82.5%

accounting concepts and the Equity 27.5% 72.5%
connection between topics Liabilities 50% 50%
Theme 3: Application of Posting to ledger 17.5% 82.5%
accounting principles and Drafting of final
62.5% 37.5%
analysis of transactions in accounts
subsidiary journals Subsidiary journals 77.5% 22.5%
The frequencies of correct and incorrect responses regarding applicable content

knowledge are displayed in Table 1, in decreasing order. Furthermore, Table 1
addresses the three themes, which are expressed as percentages of the total
number of 89 respondents, organised as follows: analysis and recording of
transactions on the accounting equation; understanding of accounting concepts
and connections between topics; and application of accounting principles and
analysis of transactions in the subsidiary journals.
The first theme provides data on the analysis and recording of transactions in the
accounting equation. Theme 1 is based on transactions relating to the content of
profit mark-up, bad debts, and debtors/creditors. On average, 70.8% of the
respondents gave incorrect answers, while 29.2% gave correct answers under this
theme. The first transaction required the respondents to calculate the profit mark-
up, and 72.5% gave incorrect answers, while 27.5% calculated the profit mark-up
correctly. The second transaction required the respondents to calculate the
amount to be written off as bad debts, and 72.5% gave incorrect answers, while
27.5 calculated the amount to be written off as bad debts correctly. The third
transaction required the respondents to record the debtor's amount in the general
journal by showing the account to be debited/credited; 67.5% gave the incorrect
answer while 32.5% recorded the correct journal entry.
The second theme provides data on teachers' understanding of accounting

concepts, and their ability to link topics. Theme 2 is based on transactions relating
to the content of assets, income, expenses, equity, and liabilities. On average,
76.3% of the respondents gave incorrect answers, and 23.7% gave correct answers
on all the content in this theme. The first transaction required respondents to
define what assets are, and 89.1% gave incorrect answers, while 10.9% defined the
concept of assets correctly. The second transaction required the respondents to
classify whether rental income is an expense or income, and 87.5% gave incorrect
answers and 12.5% classified the amount received from leasing a property
correctly as rent income. The third transaction required respondents to record
insurance on the accounting equation, and 82.5% gave the incorrect answer, while
17.5% recorded insurance correctly as part of expenses on the accounting
equation. The fourth transaction required the respondents to define the concept
of equity; 72.5% gave incorrect answers, while 27.5% defined equity as an
accounting concept correctly. The fifth transaction required the respondents to
classify whether a loan is an asset, equity, or liability, and 50% gave incorrect
answers, and 50% classified a loan correctly as a liability.

454
The third theme provides data on the application of accounting principles and
analysis of transactions in the subsidiary journals. Theme 3 is based on
transactions relating to the content of the posting to the general ledger, drawing
of final accounts, and subsidiary journals (cash receipt and cash payment journal).
On average, 47.5% of the respondents gave incorrect answers, with 52.5% giving
correct answers on this theme. The first transaction required respondents to post
the transaction for the purchase of new machinery to the general ledger, and
82.5% of the respondents gave incorrect answers, while 17.5 posted the equipment
account correctly in the general ledger. The second transaction required
respondents to balance off the debtors' control account; 37.5% gave incorrect
answers and 62.5% correctly balanced off the debtors' control account, with clear
identification of balance carried down and balance brought down. The third
transaction required respondents to analyse and record a transaction relating to
rent received from a tenant in the cash receipt journal; 22.5% gave incorrect
answers and 77.5% recorded the rent income amount correctly in the cash receipt
journal, by transferring the rent amount to the sundry accounts column.
5.1 Discussion of findings

This paper investigated deficiencies in the teaching and learning of financial
literacy, which is the accounting part of EMS in Grade 9. The findings from the
thematic analysis presented in Table 1 confirm that the respondents had
inadequate accounting content knowledge. The implication of this finding is
explained by Omotayo (2014), namely, that teachers’ content knowledge and
learners’ academic performance are positively related (Matos, 2020; Ngwenya &
Arek-Bawa, 2019). The teachers’ content knowledge, in the first instance,
influences teachers’ performance, which, in turn, enhances learners’ achievement
(Diseko & Modiba, 2018). I concur that there is a logical link between these two
variables.
Akpanobong and Asuquo (2015) report that teachers’ content knowledge is a

good predictor of students’ achievement in school subjects (Ngwenya, 2019). This
finding is in agreement with that of Qhosola (2015), who found that teachers’
mastery of their content knowledge is consistent with learners’ achievement in
accounting (Pereira & Sithole, 2020). Other factors may also influence teachers’
lack of content knowledge – these factors are beyond the scope of this study. The
length of service of teachers does not guarantee mastery of content knowledge
and learner achievement. Instead, continuous training that teachers receive helps
them to apply and display the proficiency acquired from the training/workshops
and, thus, leads to quality teaching and learning in their classes (Ngwenya &
Arek-Bawa, 2020). Certain motivational factors, such as financial (salary and other
emoluments) and non-financial matters (contentment, passion for teaching, or an
interest in the profession) may encourage teachers to be more productive, which
may lead to an improvement in academic performance of learners (Ward et al.,
2018).
While Shepherd (2013) focused on the qualifications of teachers, the findings of

this paper are unique, because they reveal the content knowledge challenges
experienced by accounting teachers (Bietenbeck et al., 2020) and how teachers

455
apply their understanding of accounting content on topics, such as analysis and

recording of transactions on the accounting equation; understanding accounting
concepts and connections between topics; and application of accounting
principles and analysis of transactions in subsidiary journals.
5.2. Ways of improving teacher competence in rural school

Several studies have revealed that the poor performance of Grade 9 rural school
accounting learners compared to those at urban schools, is caused by unqualified
teachers. After all, “they have inadequate content knowledge of Accounting
(Ngwenya, 2014) probably, because they were not properly trained and equipped
to teach it competently”. Newly qualified teachers do not receive the necessary
support and, thus, they lack career development opportunities (Gegenfurtner et
al., 2020), especially those at rural schools, which compromises effective teaching
to a greater extent in rural than in urban schools. There is a need for professional
development workshops meant specifically for rural school teachers, to address
their context-based challenges (Diseko & Modiba, 2018). These workshops should
include establishing professional learning communities for rural school teachers,
to enable them to share instructional resources and best teaching practices
(Lindsjö, 2018). In addition to advanced training, “rural teachers should also have
access to distance learning, seminars, and workshops” (Diseko & Modiba, 2018).
Implementing these measures will provide rural teachers with opportunities to

develop an interest in teaching under different conditions and in various cultures.
Thus, the “emphasis on background and adequate early exposure for prospective
teachers to the rural environments in which they will teach is crucial for teaching
in rural schools” (Mukeredzi, 2015). I strongly recommend that rural school
teachers facilitate learning in a learner-centred way, in order to accommodate the
diverse needs of learners. To do this effectively, teachers should use learner-
centred strategies and methods that fit the instructional challenges in the rural
context best (Modise, 2016; Qhosola, 2015).
6. Conclusion
This paper investigated deficiencies in the teaching and learning of financial
literacy, which is the accounting part of EMS in Grade 9. In this paper, data were
organised and the findings were discusses per the constructs that were identified.
It is evident from the discussion that these constructs are intricately interwoven
and, therefore, it is imperative to integrate the different knowledge areas,
especially when developing possible solutions for the problems experienced by
teachers. This integration can be achieved by establishing the prevalence and
impact of the different causes of each problem that was identified. The implication
is that diagnosing accounting teachers' problems needs to be done accurately, and
identification must be approached thoughtfully. A detailed analysis of the
findings should assist in developing teacher development material that is
responsive and relevant to addressing the challenges of teacher competence. The
study recommends that teachers from rural and urban schools work together by
establishing professional learning communities. By enabling rural and urban
teachers to work together by setting up a support group, the isolation that teachers
often experience can be ended. Teachers working together can find solutions to

456
the common problems they face by sharing resources and insight. Using social
networking sites for teacher support and learner support could provide
continuous professional development (workshops) for accounting teachers.
A limitation of this study is that the data was generated through the use of a
content-based questionnaire alone. If interviews could have been conducted,
much richer data could have been collected by eliciting respondents’ own views
on what informs their content knowledge deficiencies in accounting.
7. References
Adam, T. (2020). Between social justice and decolonisation: Exploring South African
MOOC designers’ conceptualisations and approaches to addressing injustices.
Journal of Interactive Media in Education, 1(7), 1–11.
https://doi.org/10.5334/jime.557
Alhassan, S. N., Abukari, M. A., & Samari, J. A. (2018). Exploring the pedagogical content
knowledge of experienced senior high schools' science teachers on “states of
matter and its changes”. East African Journal of Educational Research and Policy, 15,
105–122.
Akpanobong, U. E., & Asuquo, E. E. (2015). Teacher quality and students’ achievement in
financial accounting in senior secondary schools in Akwa Ibom State, Nigeria.
International Journal of Education and Research, 3(9), 177–188.
Asadullah, S. (2016). Effects of teacher training on secondary teachers’ mathematical
content knowledge in Dhaka, Bangladesh. International Journal of Innovation and
Economic Development, 2(5), 40–55. https://doi.org/10.18775/ijied.1849-7551-
7020.2015.25.2005
Bietenbeck, J., Piopiunik, M., & Wiederhold, S. (2018). Africa’s skill tragedy: Does teachers’
lack of knowledge lead to low student performance? Journal of Human Resources,
53(3), 553–578. https://doi.org/10.3368/jhr.53.3.0616-8002R1
Da Costa, D. A. (2020). Knowledge to teach and knowledge for teaching in teacher
education research. Pedagogical Research, 5(3), em0059.
https://doi.org/10.29333/pr/7936
De Lange, N., Khau, N., & Athiemoolam, L. (2014). Teaching practice at a rural school?
And why should we go there? South African Journal of Higher Education, 28(3), 748–
766.
Diseko, R., & Modiba, W. (2018, June 25). Authentic online assessment in assessing basic
accounting content knowledge of Grade 10 learners. In T. Bastiaens, J. Van Braak,
M. Brown, L. Cantoni, M. Castro, R. Christensen … O. Zawacki-Richter (Eds.),
EdMedia World Conference on Educational Media and Technology. Proceedings.
Association or the Advancement of Computing in Education. Retrieved from
https://www.learntechlib.org/primary/p/184556
Dube, B., & Molise, H. V. (2018). The church and its contributions to the struggle to liberate
the Free State province. Journal for Contemporary History, 43(1), 160–177.
https://doi.org/10.18820/24150509/JCH43.v1.8
Exley, B., Whatman, S., & Singh, P. (2018). Postcolonial, decolonial research dilemmas:
Fieldwork in Australian indigenous contexts. Qualitative Research, 18(5), 526–537.
https://doi.org/10.1177/1468794118778611.
Fataar, A. (2012). Pedagogical justice and student engagement in South African schooling:
Working with the cultural capital of disadvantaged students. Perspectives in
Education, 30(4), 52-75. Retrieved from
https://journals.ufs.ac.za/index.php/pie/article/view/1782.

457
Gegenfurtner, A., Lewalter, D., Lehtinen, E., Schmidt, M., & Gruber, H. (2020). Teacher
expertise and professional vision: Examining knowledge-based reasoning of pre-
service teachers, in-service teachers, and school principals. Frontiers in Education,
5, 59. https://doi.org/10.3389/feduc.2020.00059
Hine, G., & Thai, T. (2019). Pre-service mathematics teachers’ self-perceptions of readiness
to teach secondary school mathematics. Mathematics Teacher Education and
Development, 21(2), 64–86.
Hlalele, D. (2012). Exploring rural high school learners’ experience of mathematics anxiety
in academic contexts. South African Journal of Education, 32, 267-278.
Le Grange, L. (2016). Decolonization of the university curriculum. South African Journal of
Higher Education, 30(2), 1-12. http://dx.doi.org/10.20853/30-2-709
Lindsjö, K. (2018). Contextualizing the quality of primary education in urban and rural
contexts: The case of Iringa Region, Tanzania. Norsk Geografisk Tidsskrift/Norwegian
Journal of Geography, 72(4), 234-247.
https://doi.org/10.1080/00291951.2018.1492962
Lockley, P. (2018). Open initiatives for decolonising the curriculum. In, G. K. Bhambra, K.
Nişancıoğlu, & D. Gebrial (Eds.), Decolonising the university (pp. 145–173). London:
Pluto Press.
Luckett, K. (2016). Curriculum contestation in a post-colonial context: a view from the
South. Teaching in Higher Education, 21(4), 415-428.
https://doi.org/10.1080/13562517.2016.1155547
Manda, D. C. (2014). An investigation on the shortage of accounting teachers and its effect
on high schools pass rates in Vhembe District Limpopo province, South Africa.
Journal of Social Sciences, 41(3), 433-440.
https://doi.org/10.1080/09718923.2014.11893378
Matos, J. M. (2020). Construing professional knowledge of secondary school teachers of
mathematics: A historical perspective. Pedagogical Research, 5(3), em0058.
https://doi.org/10.29333/pr/7898
Modise, A. M. (2016). Pedagogical content knowledge challenges of accounting teachers.
International Journal of Educational Sciences, 13(3), 291-297.
https://doi.org/10.1080/09751122.2016.11890464
Mohangi, K., Krog, S., Stephens, O., & Nel, N. (2016). Contextual challenges in early
literacy training and learning in Grade R rural schools in South Africa. Per
Linguam, 32(1), 71-87. http://dx.doi.org/10.5785/32-1-646
Mukeredzi, T.B. (2015). Creating space for pre-service teacher professional development
during practicum: a teacher educators’ self-study. Australian Journal of Teacher
Education, 40(2), 126–145.
Ndlovu-Gatsheni, S. J. (2013). The entrapment of Africa within the global colonial matrices
of power: Eurocentrism, coloniality, and de-imperialization in the twenty-first
century. Journal of Developing Societies, 29(4), 331–353.
https://doi.org/10.1177/0169796X13503195
Ngwenya, J. C. (2014). Accounting teachers' understandings and practices of teaching and
assessment in the context of curriculum change. Alternation, 21(1), 171–189.
Ngwenya, J. C. (2019). Accounting teachers’ experiences of providing feedback in a
selected rural school in South Africa: possibilities for communal feedback. South
African Journal of Education, 39(S2), S1–S12.
https://doi.org/10.15700/saje.v39ns2a1502
Ngwenya, J., & Arek-Bawa, O. (2019). Exploring the quality of Grade 12 accounting
education textbooks. The Journal for Transdisciplinary Research in Southern Africa,
15(1), a662. https://doi.org/10.4102/td.v15i1.662

458
Ngwenya, J. C., & Arek-Bawa, O. (2020). Assessment in textbooks: exploring cognitive

demand in first-year accounting textbooks. South African Journal of High Education,
34(1), 176-196. http://dx.doi.org/10.20853/34-1-3370
Ngwenya, J. C., Maistry S. M. (2012). Teaching and assessment in Accounting: An
exploration of teachers’ experiences in a rural KwaZulu-Natal school. Journal of
Social Sciences, 33(1), 21-30. https://doi.org/10.1080/09718923.2012.11893083
Omotayo, B. K. (2014). Teachers' characteristics and students' performance level in senior
secondary school financial accounting. Journal of Empirical Studies, 1(2), 48-53.
Pereira, L., & Sithole, B. M. (2019). Towards constructivist learning and teaching in
accounting education. Journal of Emerging Trends in Educational Research and Policy
Studies, 10(1), 1–9.
Pereira, L., & Sithole, B. M. (2020). Learner-centered pedagogy in accounting:
Understanding its meaning from a Bernsteinian perspective. African Educational
Research Journal, 8(1), 20-30. https://doi.org/10.30918/AERJ.81.20.002
Qhosola, M. R. (2015). Accounting teacher preparation: A critical accounting perspective.
Council for the Development of Social Science Research in Africa, 13(1&2), 213-228.
Quan-Baffour, K. P., & Arko-Achemfuor, A. (2009). Bridging the content knowledge gap
of teachers: An agenda to improve business studies teaching in South African
countryside schools. Problems of Education in the 21st Century, 12, 108-116.
Retrieved from http://hdl.handle.net/10500/14405
Salami, I. A., & Okeke, C. I. O. (2017). Transformation and decolonization of mathematics
education for sustainable development: A case study of its learning trend in
Nigeria. Perspectives in Education, 35(2), 45-59.
https://doi.org/10.18820/2519593X/pie. v35i2.4
Kwarteng, J. T. (2016). Concerns of accounting teachers in implementing Ghana’s 2007
education reform: Revisited. International Online Journal of Education and Teaching
(IOJET), 3(3). 212-216. Retrieved from
http://iojet.org/index.php/IOJET/article/view/137/136
Seehawer, M. (2018). Decolonising research in a sub-Saharan African context: Exploring
ubuntu as a foundation for research methodology, ethics and agenda. International
Journal of Social Research Methodology, 21(4), 453–466.
https://doi.org/10.1080/13645579.2018.1432404
South Africa, Department of Basic Education. (2011). Curriculum and assessment policy
statement: Accounting. Further Education and Training Phase, Grades 10-12. Pretoria:
Government Printer.
Shepherd, D. L. (2013). The impact of teacher subject knowledge on learner performance
in South Africa: a within-pupil across-subject approach. International Workshop on
Applied Economics of Education (pp. 1–32). Cantanzaro, Italy.
Vorster, J. A., & Quinn, L. 2017. The ‘decolonial turn’: What does it mean for academic
staff development? Education as Change, 21(1), 31-49.
https://doi.org/10.17159/1947-9417/2017/853
Ward, P., He, Y., Wang, X., & Li, W. (2018). Chinese secondary physical education
teachers’ depth of specialized content knowledge in soccer. Journal of Teaching in
Physical Education, 37 (1), 101–112. https://doi.org/10.1123/jtpe.2017-0092.
Zipin, L., Fataar, A., & Brennan, M. (2015). Can social realism do social justice? Debating
the warrants for curriculum knowledge selection. Education as Change, 19(2), 9–36.
https://doi.org/10.1080/16823206.2015.1085610

PUBLISHER International Journal of Learning, Teaching and
Society for Research and Knowledge Management Educational Research
Port Louis
Republic of Mauritius
www.ijlter.org The International Journal of Learning, Teaching
and Educational Research is an open-access
Chief Editor journal which has been established for the dis-
Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the
Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER
welcomes research articles from academics, ed-
Editorial Board
ucators, teachers, trainers and other practition-
Dr. Fatima Zohra Belkhir-Benmostefa
ers on all aspects of education to publish high
Dr. Giorgio Poletti
quality peer-reviewed papers. Papers for publi-
Dr. Chi Man Tsui
cation in the International Journal of Learning,
Dr. Fitri Suraya Mohamad
Teaching and Educational Research are selected
Dr. Hernando Lintag Berna
through precise peer-review to ensure quality,
Dr. Charanjit Kaur Swaran Singh
originality, appropriateness, significance and
Dr. Abu Bakar
readability. Authors are solicited to contribute
Dr. Eglantina Hysa
to this journal by submitting articles that illus-
Dr. Mo'en Salman Alnasraween
trate research results, projects, original surveys
Dr. Hermayawati Hermayawati
and case studies that describe significant ad-
Dr. Selma Kara
vances in the fields of education, training, e-
Dr. Michael B. Cahapay
learning, etc. Authors are invited to submit pa-
Dr. Bunmi Isaiah Omodan
pers to this journal through the ONLINE submis-
Dr. Vassiliki Pliogou
sion system. Submissions must be original and
Dr. Meera Subramanian
should not have been published previously or
Dr. Muhammad Kristiawan
be under consideration for publication while
Dr. Wahyu Widada
being evaluated by IJLTER.
Dr. Som Pal Baliyan
Dr. Reem Khalid Abu-Shawish
Dr. Froilan Delute Mobo
Dr. Mohamed Ali Elkot
Dr. Anabelie Villa Valdez
Mr. Teody Lester Verdeflor Panela

Vol 19 No 8 August 2020

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 19 No 8 August 2020

Uploaded by

Copyright:

Available Formats

IJLTER.

Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Aims and Objectives

The main objective of this journal is to provide a platform for educators,

Indexing and Abstracting

The International Journal of Learning, Teaching and Educational

We are very happy to publish this issue of the International Journal of

The International Journal of Learning, Teaching and Educational

We are grateful to the editor-in-chief, members of the Editorial Board

The Editorial Board is composed of renowned people from across the

We will endeavour to ensure the reputation and quality of this journal

Editors of the August 2020 Issue

Lecture-simulation-combined Education Improve Nursing Undergraduates' Knowledge and Attitude for

Influence of Demotivators on Acceptance of Technology: Challenges of Expatriate School Teachers while

Conceptual Framework of Evaluation Model on 4 C'S-Based Learning Supervision ............................................... 173

Authentic Videos in Teaching English to Engineering Students at Universities ........................................................ 350

International Journal of Learning, Teaching and Educational Research

Training Professional Humanities’ Teachers:

Abstract. In the 21st century, generic methods and synergetic learning

Keywords: Generic studies; Interdisciplinary; Pedagogical attitudes;

©2020 The author and IJLTER.ORG. All rights reserved.

We are now witnessing an ever-developing trend of research and workspace that

©2020 The author and IJLTER.ORG. All rights reserved.

2. Personal success vs. group achievement: fostering generic learning

The generic and synergetic approach became an essential part of many

©2020 The author and IJLTER.ORG. All rights reserved.

professionalization streams have not been pursued obtaining a diploma, status,

©2020 The author and IJLTER.ORG. All rights reserved.

Remaining for a lengthy period in a specific place to secure a higher position is no

©2020 The author and IJLTER.ORG. All rights reserved.

philharmonic work to guarantee the best well-suited product (Brophy, 2006;

However, in a combined project, be it learning, research, or practical, it is

©2020 The author and IJLTER.ORG. All rights reserved.

3. Going back to disciplinary learning: Methodology and study

Nonetheless, a rational teacher is an intellectual who deals with the continuous

Compared to previous methods, this method comes closer to liberal perceptions

©2020 The author and IJLTER.ORG. All rights reserved.

classrooms, activity areas, and offices, concepts such as dialogue, openness,

This research is based on a primarily qualitative study, and on a constructive

4. Methods and results

©2020 The author and IJLTER.ORG. All rights reserved.

Table 1: Students’ attitudes to generic training in the humanities, 2017

©2020 The author and IJLTER.ORG. All rights reserved.

Specialized Generic training Generic learning Sciences and The humanities

Figure 1: Students’ attitudes to generic training in the humanities, 2017

©2020 The author and IJLTER.ORG. All rights reserved.

Table 2: Students’ attitudes to generic training in the humanities, 2020

Yes No Not sure

There is a There is an Disciplinary Separate Separate

The conclusions mentioned above, do not underestimate the value of generic,

©2020 The author and IJLTER.ORG. All rights reserved.

©2020 The author and IJLTER.ORG. All rights reserved.

©2020 The author and IJLTER.ORG. All rights reserved.

©2020 The author and IJLTER.ORG. All rights reserved.

Hattie, J. (2016). The applicability of Visible Learning to higher education. Scholarship of

©2020 The author and IJLTER.ORG. All rights reserved.

International Journal of Learning, Teaching and Educational Research

The Effects of Mobile Learning on Listening

Abdullah Al-Shamsi, Abdo Mohamed Al-Mekhlafi*,

Keywords: Mobile learning; listening comprehension skill; attitudes;

*Corresponding author: Abdo Mohamed Al-Mekhlafi; Email: raymoh123321@gmail.com

©2020 The authors and IJLTER.ORG. All rights reserved.