Vol 19 No 10 October 2020

IJLTER.
ORG
International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN: 1694-2493
e-ISSN: 1694-2116
Vol.19 No.10
International Journal of Learning, Teaching and Educational Research
(IJLTER)
Vol. 19, No. 10 (October 2020)
Print version: 1694-2493
Online version: 1694-2116
IJLTER
International Journal of Learning, Teaching and Educational Research (IJLTER)
Vol. 19, No. 10
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically those of translation, reprinting, re-use of illustrations,
broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.
Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Research is a peer-reviewed open-access journal which has been
established for the dissemination of state-of-the-art knowledge in the
fields of learning, teaching and educational research.
Aims and Objectives
The main objective of this journal is to provide a platform for educators,

teachers, trainers, academicians, scientists and researchers from over the
world to present the results of their research activities in the following
fields: innovative methodologies in learning, teaching and assessment;
multimedia in digital learning; e-learning; m-learning; e-education;
knowledge management; infrastructure support for online learning;
virtual learning environments; open education; ICT and education;
digital classrooms; blended learning; social networks and education; e-
tutoring: learning management systems; educational portals, classroom
management issues, educational case studies, etc.
Indexing and Abstracting

Research is indexed in Scopus since 2018. The Journal is also indexed in
Google Scholar and CNKI. All articles published in IJLTER are assigned
a unique DOI number.
Foreword
We are very happy to publish this issue of the International Journal of

Learning, Teaching and Educational Research.

Research is a peer-reviewed open-access journal committed to
publishing high-quality articles in the field of education. Submissions
may include full-length articles, case studies and innovative solutions to
problems faced by students, educators and directors of educational
organisations. To learn more about this journal, please visit the website
http://www.ijlter.org.
We are grateful to the editor-in-chief, members of the Editorial Board

and the reviewers for accepting only high quality articles in this issue.
We seize this opportunity to thank them for their great collaboration.
The Editorial Board is composed of renowned people from across the

world. Each paper is reviewed by at least two blind reviewers.
We will endeavour to ensure the reputation and quality of this journal

with this issue.
Editors of the October 2020 Issue

VOLUME 19 NUMBER 10 October 2020
Table of Contents
Using Systematic Synthetic Phonics to Accelerate Rural Indigenous Children’s Acquisition of Early Literacy
Skills.......................................................................................................................................................................................... 1
Jia Rong Yap and Mellisa Lee Lee Chin
Active Learning in Economic Subject: A Case Study at Secondary School ................................................................... 19

Ramlee Ismail, Marinah Awang, Seow Yea Pyng and Muhammad Ridhuan Bos Abdullah
Inclusion of the FuzzyILS Method in MOODLE for Creating Effective Courses ......................................................... 32
Antonio Silva Sprock
Supporting Natural Science Pre-Service Teachers during Work-Integrated Learning: A Case of a Lesson Study
Approach ............................................................................................................................................................................... 60
Wiets Botes, Boitumelo Moreeng and Moeketsi Mosia
Effect of Differentiated Instruction on the Achievement and Development of Critical Thinking Skills among
Sixth-Grade Science Students .............................................................................................................................................. 77
Mohammad Salih Al-Shehri
Rethinking Privilege in Teaching English in Japanese Higher Education .................................................................. 100

Khatereh Hosseininasab
The Power Sources and Influences of Secondary School Principals in Eastern Ethiopia .......................................... 115
Birhanu Sintayehu
Exploring Educators’ Challenges of Online Learning in Covid-19 at a Rural School, South Africa ........................ 134
Kananga Robert Mukuna and Peter J. O. Aloka
Managing Continuing Education via Distance Learning and Face-to-Face Courses for Human Resource
Development in the Mekong Delta of Vietnam .............................................................................................................. 150
Phuong-Tam Pham, Tran-Binh Duong, Thi-Thuy-Trang Phan, Thai-Huu Nguyen, Minh-Thanh Nguyen, Trinh Le Thi
Tuyet, Nguyen Duong Hoang, Duong Hoang Yen and Tien-Trung Nguyen
Sociocultural Adaptation and Program Management Strategies for International Doctoral Students of the
“Confucius China Studies Program” ............................................................................................................................... 172
Fan Yang
Autonomous English Language Learning Beyond the Classroom: Indonesian Tertiary Students’ Practices and
Constraints ........................................................................................................................................................................... 194
Daflizar
Computer Coding and Choreography: Contrasting Experiences of Learning About Collaboration in Engineering
and Creative Arts ................................................................................................................................................................ 214
Nicholas Rowe, Rose Martin and Nasser Giacaman
Instructional Leadership and Students Academic Performance: Mediating Effects of Teacher’s Organizational
Commitment........................................................................................................................................................................ 233
Adeel Ahmed Khan, Soaib Bin Asimiran, Suhaida Abdul Kadir, Siti Noormi Alias, Batool Atta, Bukar Ali Bularafa and
Masood Ur Rehman
The Impact of Inclusion Setting on the Academic Performance, Social Interaction and Self-Esteem of Deaf and
Hard of Hearing Students: Systematic Review and Meta-Analysis ............................................................................. 248
Sulaiman M. Alshutwi, Aznan Che Ahmad and Lay Wah Lee
Teacher Support for Eliciting Students Mathematical Thinking: Problem Posing, Asking Questions, and Song . 265
Ary Woro Kurniasih, Isti Hidayah and Mohammad Asikin
Move to Online Learning during COVID-19 Lockdown: Pre-Service Teachers’ Experiences in Ghana ................. 286
Ugorji I. Ogbonnaya, Florence C. Awoniyi and Mogalatjane E. Matabane
Current Methods for Assessing the Level of Foreign Language Proficiency of University Students ..................... 304
Nataliia S. Ivasiv, Mariya S. Kozolup, Olena V. Oleniuk, Nataliia V. Rubel and Nataliya Y. Skiba
Teaching through Experiential Learning Cycle to Enhance Student Engagement in Principles of Accounting .... 323
Rohaila Yusof, Khoo Yin Yin, Norlia Mat Norwani, Zuraidah Ismail, Anis Suriati Ahmad and Salniza Salleh
The Value of Feedback in Primary Schools: Students’ Perceptions of the Practice .................................................... 338
Abatihun Alehegn Sewagegn and Askalemariam Adamu Dessie
Digital Collaboration in Teaching and Learning Activities: The Reflexivity Study on Educational Digital
Empowerment ..................................................................................................................................................................... 355
Irwansyah and Sofiatul Hardiah
“This is why students feel lost when they go into teaching practice”: English Language Teachers’ Views on their
Initial Teacher Education ................................................................................................................................................... 371
Sue Garton
1

Vol. 19, No. 10, pp. 1-18, October 2020
https://doi.org/10.26803/ijlter.19.10.1
Using Systematic Synthetic Phonics to Accelerate

Rural Indigenous Children’s Acquisition of Early
Literacy Skills
Jia Rong Yap

University of Malaya, Jalan Universiti, 50603 Kuala Lumpur, Malaysia
https://orcid.org/0000-0002-8352-682X
Mellisa Lee Lee Chin

University of Malaya, Jalan Universiti, 50603 Kuala Lumpur, Malaysia
https://orcid.org/0000-0002-5049-2205
Abstract. Studies focusing on the strategy of phonics in Malaysia have

highlighted the insufficiency and ineffectiveness of SBELC phonics
training received by teachers, resulting in confusion among them as to
what really constitutes effective use of the phonics strategy. On the other
hand, systematic synthetic phonics (SSP) has been proven beneficial in
accelerating the performance of children in their early literacy. However,
few studies have been conducted on English language learners as the
majority of those research was focused on native speakers of the English
language. Against this background, this article presents a description of a
systematic way of teaching phonics that could inform teachers on how
the strategy can be optimally utilised to accelerate the performance of
students who are possibly at risk of being left behind. It then reports an
investigation that compared the efficacy of SSP against SBELC phonics in
accelerating the acquisition of early literacy skills with a group of
indigenous children residing in the rural parts of Sarawak, Malaysia. Five
instruments; (1) productive letter-sound test, (2) free-sound isolation test,
(3) reading test, (4) spelling test, and (5) oral-reading fluency test were
administered to measure phonemic awareness, decoding, reading, and
spelling ability. Data were collected from the pretest and the posttest. The
results demonstrate that both groups recorded significant improvement
in reading and spelling, but children in the experimental group (SSP)
outperformed the control group (SBELC phonics) significantly. Following
this, SSP should be implemented in classrooms to help accelerate
children’s early reading fluency and spelling ability.
Keywords: early literacy; English language learners; indigenous children;

phonemic awareness; systematic synthetic phonics
©2020 The authors and IJLTER.ORG. All rights reserved.

2
1. Introduction
Throughout the course of the reformation of English Language Education (ELE)
in Malaysia, various pedagogical approaches have been employed by the
Malaysian Ministry of Education (MOE) to ensure the competent acquisition of
the language among Malaysians (Hazita, 2016). One significant initiative is the
introduction of the Communicative Language Teaching method in the 1982’s
Primary School Integrated Curriculum (KBSR) to promote second language
vocabulary acquisition. This method remains beneficial in the development of
communicative competence (Chin, Karunakaran & Yap, 2019). Presently, the
CEFR-aligned Standards-Based English Language Curriculum [CEFR-aligned
SBELC] (MOE, 2017) continues to map out “pedagogical approaches [that are]
built on the foundations of communicative competences” (pp. 1 – 2). To achieve
this, the CEFR-aligned SBELC recommends the principle of going “back to basics”
and states that “it is essential for teachers to begin with basic literacy skills in order
to build a strong foundation of language skills” (p. 6). Based on this premise, the
MOE’s move to incorporate phonics as a strategy for English teaching and
learning is arguably a step in the right direction in providing a solid foundation
for students’ subsequent successful acquisition of the English language. Indeed,
phonics as a useful strategy for early literacy has been widely acknowledged by
both international (e.g., Ehri, 2020; Wyse & Goswami, 2008) and local researchers
(e.g., Su & Hawkins, 2013; Zulkifli & Melor, 2019) alike. First introduced in 2011
and as stipulated in the then newly-revamped Standards-Based English Language
Curriculum (SBELC), “the Years 1 and 2 learning standards address basic literacy
using the strategies of phonics to develop phonemic awareness in pupils to enable
them to become independent readers by the end of Year 2” (MOE, 2011, pp. 8-9).
This strategy is carried over into the CEFR-aligned SBELC, with two dedicated
documents now prepared by the MOE to guide teachers with classroom phonics
teaching and learning practices.
However, despite the Malaysian government’s substantial investment in revising

the English language curriculum and providing continuous professional
development courses to teachers, several key challenges remain to be addressed.
Fundamentally, the implementation of CEFR-aligned SBELC left much to be
desired because teachers lack a full understanding of the suggested teaching
methods, and have limited knowledge of the curriculum altogether due to the
inadequacy of training (Sidhu, Kaur & Chi, 2018). Next, studies focusing on the
strategy of phonics in Malaysia (e.g., Nadiah Yan, Napisah & Mariyatunnitha,
2014; Rabindra, Nooreiny & Hamidah, 2016) have similarly highlighted the
inconsistency, insufficiency, and ineffectiveness of the SBELC phonics training
received by teachers, resulting in misconceptions and confusion among teachers
as to what really constitutes effective use of the phonics strategy.
In this regard, systematic synthetic phonics (SSP) could be the answer to the
abovementioned issues. Educational groups in Anglophone countries such as the
United States of America’s National Institute of Child Health and Development,
the United Kingdom’s Department for Education and Skills (through
recommendations of The Rose Review, 2006), New Zealand Ministry of
Education’s Literacy Experts Group, and Australian National Inquiry into the

3
Teaching of Literacy have acknowledged the centrality of SSP in accelerating the

acquisition of early literacy (Bowey, 2011; Jolliffe, Waugh & Gill, 2019). As
suggested in the term itself, the superiority of SSP lies in its systematicity. It begins
with developing learners’ phonemic awareness through the letter-sound training
(both productive and receptive), followed by the five-phase phonics training, and
the after-phase blending and segmenting practices. Additionally, SSP includes
pseudowords to ensure children apply the phonics strategy in reading and
spelling. In comparison, the SBELC phonics conducts the letter-sound training
and phonics training concurrently. It uses only real words, with occasional
blending and segmenting activities. Unsurprisingly, findings from the present
study have shown the experimental group (SSP) outperforming the control group
(SBLEC phonics) in their early literacy skills.
Against this background, the purpose of this article is two-fold. First, it advocates
for and presents a detailed description of a systematic way of teaching phonics
that could inform teachers on how the strategy can be optimally utilised to
accelerate the performance of students who may be at risk of being left behind (or
are still preliterate at Primary 1/Primary 2). Second, as a means of supporting the
effectiveness of SSP, it reports an investigation that compared the SSP programme
with SBELC phonics in imparting early literacy skills among young learners in the
rural setting. In the study, early literacy was defined as reading fluency and
writing in the form of spelling ability, whereby children’s performances were
measurable for documentation purposes (Purewal, 2008).
2. Literature Review
2.1 Phonics for Early Literacy
Fundamentally, phonics is a goal of enabling learners to associate sounds to the
prints and subsequently to transfer this skill into reading or spelling. It is also an
umbrella term that constitutes an organised set of rules about vowels, consonant-
blends and syllables, the key to which is to recover the sounds from the prints
(Griffith & Olson, 1992). It reflects Rose’s (2006) Simple View of Reading that
posits reading as a two-process skill; (i) the automatic word recognition skills, and
(ii) the ability to tap into prior knowledge and experience to gain comprehension.
The fundamental step in achieving word recognition is decoding, whereby a child
can associate the sounds (phonemes) represented by a letter or a combination of
letters (graphemes), and to identify the complete word (Rose, 2006). Rose (2006)
further emphasises that decoding is the precursor to comprehension and as such,
children need to first acquire the decoding skills in their beginning reading before
they are to progress to the task of comprehension.
Having said that, for successful teaching of reading and spelling through phonics,
the development of learners’ foundation in phonemic awareness should take
precedence. According to Cunningham (1988, as cited in Griffith & Olson, 1992),
phonemic awareness is the understanding that the sounds of a spoken language
work together to make words. Specifically, phonemic awareness does not sound
out words, but its skill enables children to use grapheme-phoneme relationships
to read and spell words by understanding the structure of the spoken language.
Ukrainetz et al. (2000) propose that this can be achieved by carefully choosing the

4
type of phonics instructions, defined as teaching practices that are designed to

help students acquire knowledge of the relationships between graphemes and
phonemes, and the ability to do blending (Armbruster, Lehr & Osborn, 2001). An
essential component of phonics, blending is the process of putting individual
phonemes together to read a complete word and it requires phonemic awareness
(Griffith & Olson, 1992). Beck and Beck (2013) further recommend scaffolding
blending whereby this sequential process of learners sounding each phoneme,
remembering the sequence, and blending the segments be developed. This
scaffolding blending process was integrated into this study as part of the
systematic synthetic phonics programme, which is discussed in further detail in
the subsequent section.
2.2 Systematic Synthetic Phonics

The term ‘Systematic Synthetic Phonics’ engaged in this study is composed of two
major concepts; (i) systematic phonics instruction, and (ii) synthetic phonics.
Exactly a score years ago, the US National Reading Panel (2000) released a 449-
page report which reviewed more than 100000 research studies on reading and
has recommended systematic phonics instruction for reading. Correspondingly,
Mesmer and Griffith (2005) explain that a systematic phonics programme
encompasses three elements; (i) a curriculum with a specific, sequential set of
phonics elements, (ii) instruction that is direct, precise and unambiguous, and (iii)
opportunities for learners to use phonics to read words. As for synthetic phonics,
this approach begins by teaching learners the identification of phonemes that are
represented by graphemes in a word, before putting them together to form a
complete word (de Graaff, Bosman, Hasselman & Verhoeven, 2009). It shares the
principles in the bottom-up processing of reading which views the ability to
decode efficiently and to recognise words automatically as vital skills. De Graaff
et al. (2009) suggest that once learners grasped these basic grapheme-phonemes
correspondences (GPCs), they can decode a number of words in English without
much difficulty and hence expand their reading vocabulary.
2.3 Systematic Synthetic Phonics Programme

The Systematic Synthetic Phonics (SSP) programme in this study embraces all the
elements of systematic phonics presented by Mesmer and Griffith (2005) and is
inspired by de Graaff et al.’s (2009) computer-assisted model which has been
modified into a human model. This SSP programme contains two parts. Firstly,
the letter-sound training introduces the phonemes and their represented
graphemes and is organised into two sections: (i) the receptive and (ii) the
productive. In the receptive way of training, the teacher says aloud a phoneme
twice and then places four graphemes cards (1 target phoneme, 3 distractors)
before their learners. The learners listen to the phoneme uttered and select its
corresponding grapheme out of the four cards. In the productive way of training,
the learners see the grapheme cards first and point at the corresponding
graphemes as the teacher produces the phonemes orally. Once the learners have
undergone the letter-sound training and successfully mastered all the phonemes
and their corresponding graphemes, they advance into the phonics training.
In this part of the programme (phonics training), learners are required to practise
reading, blending, and segmenting randomly presented words and pseudowords.

5
Pseudowords are a pronounceable combination of graphemes that have the

characteristics of a known real word but are not real words according to common
English dictionaries (Cardenas, 2009, in de Graff et al., 2009). For instance, the
phoneme ai/eɪ/ may form words such as ‘sail’ and ‘bail’, or pseudowords like
‘dail’ and ‘phail’. Their integration is unique to this SSP programme, as using both
words and pseudowords will ensure learners acquire the intended phonics
knowledge for reading, and the syllabic patterns for spelling (Harris & Hodges,
1995). This phonics training is planned to be carried out in stages, with a
predetermined number of target GPCs in each stage. Nevertheless, despite the
emphasis on adhering to the scope and sequence of introducing the GPCs, the
teacher holds the autonomy in deciding the number of GPCs to begin with in the
first stage, and the addition of new GPCs in the subsequent stages until all 44
phonemes are covered. The decision can be made depending on their learners’
capability and progress.
Another important feature of this SSP programme is that each stage comprises
five phases. In Phase 1, the graphemes at the beginning and the end of the
word/pseudoword [(pseudo)word] are given. In Phase 2, only the grapheme at
the end is given. In Phase 3, the grapheme at the beginning is given. In Phase 4,
no graphemes are given and in Phase 5, a complete CVC (pseudo)word is given.
Specifically, in Phase 5, learners have to select the corresponding (pseudo)word
spoken by the teacher out of the four presented word-cards (1 target word, 3
distractors). The construction of 15 words in the first four phases and the
synthesising of 10 words in Phase 5 entitle the learners to proceed to an extended
blending and segmenting practice. In this after-phase activity, the teacher will
demonstrate smooth blending (the sounding of phonemes without pausing) and
smooth segmenting (the automatic association of a phoneme to its grapheme) as
a part of the skills training. For the next two sessions, learners practise blending
to form complete (pseudo)words and segmenting them for spelling. When all the
five phases within a stage are completed and the learners are able to blend and
segment 10 (pseudo)words, they progress to the next stage. A summary of the
phases and an overview of the SSP programme are presented in Table 1 and
Figure 1, respectively.
Table 1: Summarised details of phases in a stage in SSP

Sample Item
Phase Description Example
(CVC word)
Graphemes at the beginning and the m__d
1
end are given f__m
maid jail train
__ __ d
2 snail float foam Grapheme at the end is given
__ __ m
goat toast ties
m__ __
3 lies pies dies Grapheme at the beginning is given
f__ __
sheep green cheek
___ ___ ___
4 wheel torch sport No graphemes are given/presented
___ ___ ___
fork form
A complete CVC (pseudo)word is 1. maid**
given. Learners select the 2. foam
*words in italic are
5 corresponding (pseudo)word spoken 3. form
used as examples
by the teacher out of the 4 wordcards 4. green
given (1 target word, 3 distractors) **target word

6
Figure 1. An overview of the Systematic Synthetic Phonics programme
To date, despite the growing body of literature that supports the benefits of
systematic synthetic phonics, little studies have been conducted on children who
are English language learners as the majority of those research was focused on
native speakers of the English language (McGeown, Johnston & Medford, 2012;
Watts & Gardner, 2012; Wyse & Goswami, 2008; Yap, 2014). Therefore, as outlined
in the purpose of this article, the next section presents a quantitative randomised
comparison experimental study that investigated the effects of SSP and SBELC
phonics on reading fluency and spelling ability with a group of indigenous
children (Iban) residing in the rural parts of Sarawak. These children were likely
to be at a higher risk of falling behind their city peers in early literacy if their ability
to read in the English language was not addressed in time (UNICEF, 2008). The
hypothesis and research questions are as follows:
The indigenous children who undergo SSP training will attain higher levels of
reading fluency and spelling ability than the children who receive SBELC phonics
training.
1. What is the relative effect of SSP as compared to the SBELC phonics on
children’s early reading fluency?
2. What is the relative effect of SSP as compared to the SBELC phonics on
children’s early spelling ability?
3. Method
The experimental study, which was quantitative in nature, took place in a real-life
natural setting of an educational organisation. It intended to prove the hypothesis
by determining whether or not the independent variable (the type of phonics

7
training) caused an effect on the dependent variable (the children’s reading

fluency and spelling ability). It followed the features of a true experimental study
with the inclusion of three key components – (i) pre-posttest design, (ii) a
treatment (or experimental) group and a control group, and (iii) random
assignment of study participants (Carpenter et al., 1989).
3.1 Participants
The participants in the present study consisted of 32 Primary 2 schoolchildren, in
which they were equally and randomly assigned into either the experimental or
the control group. They were from three neighbouring national schools located in
the rural parts of Bintulu, Sarawak. This study had engaged a non-probability
sampling method in the recruitment of participants, as they were the researchers’
existing students and students of English teachers known to the researchers. Table
2 shows the participants’ mean age, socioeconomic status, and level of
proficiency from the SBELC school-based assessment.
Table 2: Participants’ background

Mean age 92.6 months (SD = 3.5 months)
Socioeconomic status Good Average Hardcore Poor
15 8 9
Level of proficiency Band 3 Band 2 Band 1
from SBELC assessment 7 20 5
The children were a homogenous group from the indigenous tribe of ‘Iban or Sea
Dayak’. Before primary education, all 32 participants had received a year of
kindergarten education and mastered all the 26 letter-names in the English
alphabet. However, formal learning and immersion into English language only
began in Primary 1. As SBELC phonics began in Primary 1, they had learned and
mastered 30 GPCs of 21 consonants, five short vowels, and four digraphs. This
conclusion was made based on the results of the achievement test where all 32 of
them received perfect scores, conducted at the beginning of 2013. The objectives
and nature of the experiment were explained to the participants’ parents prior to
obtaining their consent. They also met the following inclusion criteria: (a)
indigenous children from the rural parts in Sarawak, (b) learning English as a
foreign language, (c) undergoing SBELC phonics for reading, and (d), the ability
to attend phonics training for 30 minutes a day.
3.2 Phonics Training Procedure

The experiment consisted of two types of training: the SSP and the SBELC
phonics. Both phonics-training programmes contained 40 sessions of 30-minute
each that were executed over a period of eight weeks. The training duration and
session were planned in conformity with the SBELC phonics scheme-of-work. The
participants had 60 minutes of English lesson daily from Mondays to Fridays and
learnt approximately nine GPCs in eight weeks. The researcher purchased
commercially available Jolly Phonics products from the authorised distributor in
Malaysia and conducted the SSP training with the experimental group. This study
recruited the help of one phonics-instruction trained teacher to act as the SBELC
phonics trainer and also as the inter-rater (Teacher X). Teacher X carried out

8
SBELC Phonics training with the control group using the materials in the form of
textbook and teachers’ guidebook provided by the MOE. Prior to the actual
experimentation, the researcher and Teacher X (the trainers) simulated the
training procedures in SSP and SBELC Phonics twice to ensure a uniform
administration of the phonics training.
3.3 Training Scope and Sequence

Scope refers to the content of the phonics instruction and the range of GPCs
covered, while sequence is the order for teaching the GPCs. Both the experimental
and control groups were given the same 11 long vowel and diphthong sounds
(phonemes) represented by 16 graphemes. Thus, both groups have 16 GPCs
(ai/eɪ/, oa/əʊ/, ie/aɪ/, ee/iː/, or/ɔː/, oo/ʊ/, oo/uː/, oi/ɔɪ/, ou/aʊ/, er/əː/,
ar/ɑː/, ay/eɪ/, ow/əʊ/, igh/aɪ/, ea/iː/, and ue/uː/). The IPA symbols were not
introduced to the participants to avoid possible confusion.
3.3.1 Systematic Synthetic Phonics (SSP) Training

Following the procedure as presented in Figure 1, the SSP training began with the
letter-sound training in which one GPC was taught in each session and altogether
16 sessions were allocated for this. The phonics training comprised 24 sessions
and required the participants to practise reading, blending, and segmenting
randomly presented (pseudo)words in five stages. At Stage A, children practised
with randomly presented (pseudo)words with the five GPCs of ai/eɪ/, oa/əʊ/,
ie/aɪ/, ee/iː/, and or/ɔː/. Each participant was given two attempts to listen to
the (pseudo)words given by the researcher and fill in the blanks with the
grapheme-cards provided to form the complete CVC (pseudo)words. Upon the
second erroneous attempt, the correct answer was given. Participants jotted down
the correctly formed words into their personal logbooks as a record of their
individual progress. This allowed them to proceed at an individual pace. The
participants went through the five phases in each stage (see Table 1). Three new
GPCs were added in Stage B (oo/ʊ/, oo/uː/, oi/ɔɪ/), Stage C (ou/aʊ/, er/əː/,
ar/ɑː/), Stage D (ay/eɪ/, ow/əʊ/, igh/aɪ/) and lastly, two in Stage E (ea/iː/,
ue/uː/). When all the five stages have been completed, children repeated the five
phases in Stage E until all 24 sessions were fulfilled.
3.3.2 SBELC Phonics Training

The letter-sound training and phonics training ran concurrently in SBELC phonics
training. Teacher X extracted the phonics components, the accompanying word
list and reading texts from the SBELC Year 2 English textbook and followed the
phonics instructions and activities stipulated in it. The SBELC phonics training
procedure was repetitive in nature, beginning with the introduction to and
practices of sounding out the target phonemes. The children were to associate a
phoneme to its corresponding grapheme by choosing the correct letter card. Then,
they were instructed to listen to a list of words presented to them by Teacher X
and to orally identify the vowel sound in those words. For example, the vowel
sound in ‘broach’ is oa/əʊ/. After that, they were expected to know how to blend
and segment by using the list of words provided in the textbook. The phonics
training of every unit ended with a reading text. The text integrated some of the
target GPCs and encompassed CVC, CV and VC words. The reading texts also
contained two- and three-syllable words that required Teacher X to demonstrate

9
using the whole-word approach. In each unit, two new phonemes were
introduced, with no reference or revision of the past phoneme learnt.
3.4 Instruments
The trainers attended two training sessions a fortnight before the pretest in April
to ensure a uniform administration of the tests. The tests took place in the evening
at the school’s library, after the day’s schooling session has concluded. The whole
process was digitally recorded for all five tests, to allow for an after-test review
and cross-examination between the trainers. Also, the Malaysian English
curriculum uses Standard British English as a reference and model for teaching
the language, as well as for spelling and pronunciation for standardisation (MOE,
2011). As such, the judgment of the pronunciation of phonemes cross-referred to
the phonemic chart from the British Council website. The judgment of the
pronunciation of words was cross-referred with oxforddictionaries.com.
Nonetheless, following studies by Wang and Koda (2005), all acceptable
pronunciations were scored correct. For example, the word ‘sail’ pronounced as
/seɪl/ and /sɛl/ were both acceptable.
Pretest and Posttest. The participants were tested twice; before the experiment
commenced in May (pretest) and after the experiment in August (posttest). Five
tests measuring (a) productive letter-sound knowledge, (b) phonemic awareness,
(c) reading ability, (d) segmenting/spelling ability, and (e) sentence-level reading
ability were administered to each child individually for a maximum of 30 minutes
each. Tests (a) to (d) and their scoring criteria were adapted from de Graaff et al.
(2009). Test (e) and its scoring criteria were adapted from Eun (2012). The
adaptations were necessary as the content needed to correspond to the phonemes
introduced in this study. Each of the instruments is elaborated below.
Productive Letter-Sound Test (PLST). This test measured the participants’

knowledge of the GPCs. They were given letter cards containing the sixteen GPCs
presented during the letter-sound training and asked to produce the phonemes.
The trainers gave a short demonstration (using the GPCs ur/ɜː/, ng/ŋ/) and the
children practised with two non-tested GPCs (a/æ/, ch/ʧ/) before the actual
testing commenced. This test carried a minimum score of 0 and a maximum score
of 16, with 1 point being given for each successful sound-production.
Free Sound-Isolation Test (FSIT). This test was conducted to test the participants’
phonemic awareness. They were presented with a list of 12 consonant-vowel (CV)
and 36 consonant-vowel-consonant (CVC) words (see Table 3). These words were
selected from the SBELC Year 2 English textbook, and they included the vowel
sounds presented in the experiment. The children were asked to segment the
words on the word chart into their individual sounds or to identify the phonemes
present in a word. For example, the word ‘pail’ has three phonemes /p/eɪ/l/.
Those children who have achieved phonemic awareness would be able to identify
and say /p/,/eɪ/ and /l/. The trainers gave a short demonstration and children
practised orally with two non-tested words before the actual test began. This test
carried a minimum score of 0 and a maximum score of 132, with 1 point being
awarded for each successful sound- production.

10
Table 3: Free Sound-Isolation Test

Demonstrated word: pail
Practiced words: coat, fork
CV words CVC words
bow loud float stern
lie farm sheep cart
flow maid hook light
tray dream boil jail
die train herd foam
day mouth room cheek
grow form night shook
true moon cream coin
glue cloud boat nerd
pie sharp green sport
play torch look fight
sue peak join stool
Reading Test (RT). A total of 3 CV, 13 CVC words, and 3 CV, 13 CVC pseudowords
were administered to gauge the children’s blending skills (see Table 4). The final
list was derived from a combination of (pseudo)words formed from the 21
consonants, 14 digraphs acquired in Primary 1, and the 16 vowel sounds
presented during the training. The items were both in accordance with the 5 stages
of SSP training and SBELC Phonics training. To elaborate, the vowel sounds from
Stage A formed 10 items, Stages B to D formed six items each and lastly, Stage E
formed four items. The children were presented with the list of 32 (pseudo)words
and were required to read each (pseudo)word aloud. In the event of a child
mispronouncing a word, they were instructed to engage their blending skills.
However, if they still could not read the word after two additional tries, they
would proceed to the next word. This test carried a minimum score of 0 and a
maximum score of 32, with 1 point awarded for each successful (pseudo)word
produced.
Table 4: Summary of words formed from stages A – E

Stage Phoneme Word Pseudoword Number of Item
A ai /eɪ/ said bain 10
oa /əʊ/ gloat coam
ie /aɪ/ lie wie
ee /iː/ steep cheel
or /ɔː/ stork chorm
B oo /ʊ/ crook pook 6
oo /uː/ droop flop
oi /ɔɪ/ coil moin
C ou /aʊ/ stout boust 6
er /əː/ perch wern
ar /ɑː/ chart spart
D ay/eɪ/ dray glay 6
ow /əʊ/ grow drow
igh /aɪ/ flight spight
E ea /iː/ speak pleak 4
ue /uː/ glue crue

11
Segmenting Skill/ Spelling Test (ST). The 32 items presented during the reading test
were reemployed to determine children’s spelling ability. The children were
asked to write the sounds they heard in a (pseudo)word, in sequential order. This
test carried a maximum score of 32, with 1 point awarded for each (pseudo)word
spelt correctly.
Oral-Reading Fluency Test (ORFT). This test was administered to determine the
participants’ reading fluency, defined as the ability to read a piece of text
automatically and accurately with expressions. However, prosody was not
included in the test as studies by Jiang, Sawaki and Sabatini (2012) and Lems
(2003, in Eun, 2012) have reported on the difficulty to achieve an acceptable
reliability given the subjective nature of deciding desirable prosody. The text was
adopted from Jolly Readers Level 2, published by Jolly Learning Limited. It
featured words that were phonetically decodable, and could be sounded out with
the 21 consonants, 14 digraphs acquired in Primary 1, and the 16 vowel sounds
presented during training in the current study. However, unlike the Reading Test
(RT), ORFT assessed participants’ ability to read at the sentence level by counting
the number of words the children read in a minute.
ORFT was conducted in this manner. The trainers and the children each had a
copy of the same reading text. The children were instructed to begin reading aloud
and while they read, the trainers noted any errors the children made by circling
the mispronounced words in their copy. Once the minute on the stopwatch held
by trainers was up, they marked in their sheet the children’s progress at the 60 th
second and let them finish reading the text. The trainers then totalled the number
of words read within 60 seconds and subtracted them with errors made by the
children. For the purpose of this study, only errors made on the trained vowel
sounds were considered. For example, if ‘Child A’ read 65 words in a minute but
made a total of 6 errors (2 untrained-vowel words, 4 trained-vowel words), their
reading rate would be 61 words correct per minute. The children’s oral reading
fluency rate was compared against the benchmark adapted from Johns and
Berglund (2009), which states that the average second grade or primary 2
students’ mean words targets is 50 correct words per minute in February, 70 in
June, and 90 in October.
3.5 Data Analysis

Data for this study were analysed using IBM Statistical Packages for Social
Sciences (SPSS) version 21. The findings are presented in two parts. The first part
consists of the analysis of the pretest for both the experimental (SSP) and control
groups (SBELC Phonics) using independent samples t-test. This was conducted in
order to establish equality among both groups’ early literacy levels before the
intervention. Levene’s test for equality of variance was applied. Next, the
hypothesis and research questions were addressed through the analyses of
paired-samples t-test for each outcome variable. A paired-samples t-test was used
to compare the means of the pretest and posttest scores obtained from the
experimental group and control group, in order to determine the effectiveness of
the phonics training by looking at the significant difference between the two
scores.

12
4. Findings and Discussion

4.1 Analyses of Pretest
The results of the pretest aimed at establishing the assumption of equality of
variance are presented in Tables 5 and 6. The null hypothesis to be tested (Ho: µE
= µC) states that the PLST, FSIT, RT, ST and ORFT pretest mean scores of the
experimental group are equal to the pretest mean scores of the control group.
Conversely, the alternative hypothesis (H1: µE ≠ µC) states that the pretests PLST,
FSIT, RT, ST and ORFT mean scores of the experimental group are not equal to
the pretests mean scores of the control group. The significance level alpha is
specified at .05.
Table 5: Descriptive statistics

Groups N Mean Std. Deviation Std. Error
Mean
PLST Experimental 16 5.688 1.195 0.299
Control 16 5.750 1.390 0.348
FSIT Experimental 16 96.625 7.013 1.753
Control 16 96.938 6.547 1.637
RT Experimental 16 10.375 2.446 0.612
Control 16 10.750 2.206 0.552
ST Experimental 16 7.500 2.129 0.532
Control 16 8.125 2.306 0.576
ORFT Experimental 16 34.500 5.808 1.452
Control 16 34.438 6.491 1.623
Table 6: Independent samples t-test

Levene’s
Test for t-test for Equality of Means
Equality of
Variances
95% CI of
Sig. (2- Mean Std. Error
F Sig. t df Difference
tailed) Diff. Diff.
Lower Upper
PLST .104 .750 -.136 30 .892 -.063 .458 -.999 .874
FSIT .062 .805 -.130 30 .897 -.313 2.399 -5.211 4.586
RT .239 .628 -.455 30 .652 -.375 .823 -2.057 1.307
ST .085 .772 -.797 30 .432 -.625 .785 -2.227 .977
ORFT .092 .763 .029 30 .977 .063 2.177 -4.385 4.510
As shown in Table 6, since all the significant value was greater than alpha at .05
level of significance, there was no sufficient evidence to reject the null hypothesis.
It can be concluded that there is no significant difference between experimental
and control groups’ pretest scores in PLST, FSIT, RT, ST and ORFT. Results from
the Levene’s test also showed that the equality of variances is assumed. Therefore,
participants in both groups had similar levels of reading fluency and spelling
ability and so were deemed comparable prior to the intervention.
4.2 The Relative Effect of SSP and SBELC Phonics Training

To find out if there was a difference between the posttest scores of PLST, FSIT, RT,
ST and ORFT assessments of the SSP group and SBELC phonics group, an analysis

13
of paired-samples t-test was computed. This was to analyse the mean scores of the
pretest and the posttest of the experimental and control groups. The significance
level is specified at .05 (alpha, α = .05). Results are presented in Tables 7 and 8 (for
the experimental group), and Tables 9 and 10 (for the control group). To address
the hypothesis that the children who undergo the SSP training would
demonstrate a better improvement in their reading fluency and spelling ability
than the children of SBELC phonics, a comparison was made by looking at the
higher Partial Eta Squared value of the two groups. The null hypothesis to be
tested (Ho: µ1 = µ2 or µ1 - µ2 = 0) states that the PLST, FSIT, RT, ST and ORFT
mean scores of the pretest are equal to the mean scores of the posttest. Conversely,
the alternative hypothesis (H1: µ1≠ µ2 or µ1 - µ2 ≠ 0) states that the PLST, FSIT,
RT, ST and ORFT mean scores of the pretest are not equal to the mean scores of
the posttest.
Table 7. Paired samples descriptive statistics for the experimental group

N Mean Std. Deviation Std. Error
Mean
PLST Pretest 16 5.688 1.195 .299
Posttest 16 13.876 1.857 .464
FSIT Pretest 16 96.625 7.013 1.753
Posttest 16 118.750 9.842 2.461
RT Pretest 16 10.375 2.446 .612
Posttest 16 24.875 3.096 .774
ST Pretest 16 7.500 2.129 .532
Posttest 16 19.250 3.493 .873
ORFT Pretest 16 34.500 5.808 1.452
Posttest 16 44.375 6.956 1.739
Table 8. Paired samples t-test for the experimental group

Partial
Paired Sig. (2- 95% CI of
t df ETA
Differences tailed) Difference
Squared
Mean SD Lower Upper
PLST (PT-PST) -8.188 1.109 -29.54 15 .000 .880 -8.778 -7.597
FSIT (PT-PST) -22.125 3.557 -24.88 15 .000 .641 -24.020 -20.230
RT (PT-PST)) -14.500 1.633 -35.52 15 .000 .878 -15.370 -13.630
ST ((PT-PST) -11.750 2.266 -20.74 15 .000 .815 -12.957 -10.543
ORFT (PT-PST) -9.875 2.825 -13.98 15 .000 .388 -11.381 -8.369
Note. PT – Pretest, PST - Posttest
On average, based on the descriptive statistics shown in Table 7, it seems that the
experimental group performed better in the posttest. Since all mean differences
are negative (see Table 8), the posttest results are better than the pretest results.
The results suggest that there is sufficient evidence to reject the null hypothesis as
all the significant value was smaller than alpha at .05 level of significance. Thus,
it can be concluded that SSP had a significant effect on the children’s reading
fluency and spelling ability.

14
Table 9. Paired samples descriptive statistics for the control group

N Mean Std. Deviation Std. Error
Mean
PLST Pretest 16 5.750 1.390 .348
Posttest 16 10.188 1.940 .449
FSIT Pretest 16 96.938 6.550 1.637
Posttest 16 108.563 9.252 2.313
RT Pretest 16 10.750 2.206 .552
Posttest 16 20.313 3.005 .751
ST Pretest 16 8.125 2.306 .576
Posttest 16 14.063 2.670 .668
ORFT Pretest 16 34.438 6.491 1.623
Posttest 16 39.938 7.316 1.829
Table 10. Paired samples t-test for the control group

Paired Partial 95% CI of
Sig. (2-
Differences t df ETA Difference
tailed)
Mean SD Squared Lower Upper
PLST (PT-PST) -4.438 1.504 -11.80 15 .000 .648 -5.239 -3.636
FSIT (PT-PST) -11.625 4.745 -9.80 15 .000 .359 -14.154 -9.096
RT (PT-PST)) -9.563 2.309 -16.57 15 .000 .778 -10.793 -8.332
ST ((PT-PST) -5.938 2.462 -9.65 15 .000 .602 -7.250 -4.625
ORFT (PT-PST) -5.500 1.713 -12.85 15 .000 .144 -6.413 -4.587
Note. PT – Pretest, PST – Posttest
Overall, based on the descriptive statistics shown in Table 9, participants in the

control group appears to perform better in the posttest as compared to the pretest.
From the results of the paired samples t-test (Table 10), since all mean differences
are negative, the posttest results are better than the pretest results. Since all the
significant value was smaller than alpha at .05 level of significance, there was
sufficient evidence to reject the null hypothesis. It can be concluded that SBELC
phonics had a significant effect on children’s reading fluency and spelling ability.
As can be seen, the mean differences between the pretest and posttest for all five
assessments show a significant increase in the reading and spelling performances
for both experimental (see Table 8) and control (see Table 10) groups at .05 level
of significance. However, as seen in the Partial Eta Squared values, the
experimental group gained significantly higher in all the five assessments (PLST
= .880, FSIT = .641, RT = .878, ST = .815, ORFT = .388) compared to the control
group (PLST = .648, FSIT = .359, RT = .778, ST = .602, ORFT = .144). This confirms
the hypothesis that children who undergo SSP will attain higher levels of reading
fluency and spelling ability than those who receive SBELC phonics.
4.3 Using Systematic Synthetic Phonics to Accelerate the Acquisition of Early

Literacy Skills
Findings from this study have shown that synthetic phonics, whether systematic
(SSP programme) or unsystematic (SBELC phonics), helps children to develop
their decoding skills which apply in reading regular or phonetically decodable
words. Children from the experimental and control groups recorded significant
growth in their decoding ability (assessed through the RT, ST and ORFT). This

15
indicates that both approaches were beneficial in building their blending and
segmenting skills (two components of synthetic phonics) that had contributed to
their improvement in beginning reading. Nonetheless, the experimental group
had higher levels of attainment as compared to the control group in productive
letter-sound knowledge, phonemic awareness, reading at world level, and
spelling, while achieving a similar level in passage reading with the control group.
As aforementioned, phonics training only comes after the letter-sound training in
the SSP programme. To elaborate, what this essentially means is that the superior
performance of the SSP group could be attributed to the following strategies. The
reading-supporting strategies were presented in the productive letter-sound
training (from grapheme to phoneme), Phase 5 (from written word to oral form)
and the after-phase blending session of the phonics training. The spelling-
supporting strategies involved the receptive letter-sound training (from phoneme
to grapheme), Phases 1 through 5 (from oral form to written word) and the after-
phase segmenting activities of the phonics training. The success also lay in the
implementation of phases and stages. Children were only allowed to progress to
the next phase after completing the current phase, and were only promoted to the
next stage after mastering the 5 phases, blending and segmenting sessions in each
stage.
To encapsulate, SSP could be the answer for teachers of learners who may be at
risk of being left behind and those from a disadvantaged background such as the
indigenous groups and/or rural schools (Johnson & Tweedie, 2010) to gain similar
improvements in their learners’ early literacy in English language. The empirical
evidence gathered in the current study also serves to confirm the success of similar
projects using systematic phonics for beginning reading (e.g., Hawkins & Su,
2013; Zulkifli & Melor, 2019). However, in addition to reporting the effectiveness
of the SSP programme, this article has also thoroughly described the principles
and step-by-step procedure of how teachers can carry out the systematic synthetic
phonics training in classrooms with their learners. This corresponds with findings
from Rabindra et al.’s (2016) study in which teachers are calling for “a specific
training session on phonics” as information from courses is often “in a diluted and
watered form” (p. 14). This too possibly answers Warid’s (2015) calls for more
guidance and support for teachers of English Language in indigenous rural
schools.
5. Conclusion
This study has examined the application of the phonics method in improving
young children’s early literacy. As discussed previously, reading consists of two
distinct components: (i) word-recognition and (ii) comprehension. Phonics
instruction supports the development of children’s decoding ability that enhances
their word-recognition and thus improves their overall early literacy. In this
regard, the highly systematic strategies prescribed in SSP can provide a
methodological sequence of introducing the synthetic phonics skills and letter-
sound training. Children in the present study had felt a sense of achievement
when they used the SSP strategies and successfully read storybooks
independently (Jolly Readers Level 1 and Level 2 had been given for reading after

16
the training). This was achieved despite the children’s language-disadvantaged

background.
Based on the findings, the followings recommendations are offered for further
investigations. Firstly, as an improvement, future studies can be repeated for a
longer period of time, possibly for the entire school year, and begin phonics
training of all the 44 phonemes at the beginning of the school term. Doing so may
provide a better idea of whether a complete SSP program helps rural children
develop early reading fluency and spelling ability, and if the intervention helps in
their overall acquisition of literacy skills in the English language. Another
consideration for future research is to increase the sample size for the study,
possibly by extending the intervention to other rural schools. Doing so will enable
the researcher(s) to collect and analyse more data across more settings and
enhance the generalisability of SSP in developing early literacy. Finally, this study
focused on word recognition only and has yet to study the effects of SSP on
reading comprehension. It is therefore recommended future research to explore
this area by incorporating the assessments on complete reading processes; both
word recognition and comprehension.
6. Acknowledgments
The authors offer their sincerest appreciation to Kuang Ching Hei for her
encouragement, and the blind reviewers for their assistance and constructive
feedback that have led to the publication of this paper.
7. References
Armbruster, B. B., Lehr, F., & Osborn, J. (2001). Put reading first: The research building blocks
for teaching children to read kindergarten through grade 3. Washington, D.C., WA:
National Institute for Literacy. Retrieved from http://www.nifl.gov
Beck, I. L., & Beck, M. E. (2013). Making sense of phonics: The hows and whys (2nd ed.). New
York, NY: Guilford Press.
Bowey, J. A. (2011). Need for systematic synthetic phonics teaching within the early
reading curriculum. Australian Psychology, 41(2), 79–84.
doi:10.1080/00050060600610334
Carpenter, T. P., Fennema, E., Peterson, P. L., Chiang, C. P., & Loef, M. (1989). Using
knowledge of children’s mathematics thinking in classroom teaching: An
experimental study. American Educational Research Journal, 26(4), 499-531.
doi:10.3102/00028312026004499
Chin, M. L. L., Karunakaran, K., & Yap, J. R. (2019). The role of negotiated interaction in
L2 vocabulary acquisition among primary ESL learners. 3L: The Southeast Asian
Journal of English Language Studies, 25(2), 1-21. doi:10.17576/3L-2019-2502-01
De Graaff, S., Bosman, A. M. T., Hasselman, F., & Verhoeven, L. (2009). Benefits of
systematic phonics instruction. Scientific Studies of Reading, 13(4), 318-333.
doi:10.1080/10888430903001308
Ehri, L. C. (2020). The science of learning to read words: A case for systematic phonics
instruction. Reading Research Quarterly, 55(S1), 45-60. doi:10.1002/rrq.334
Eun, H. J. (2012). Oral reading fluency in a second language reading. Reading in a Foreign
Language, 24(2), 186-208. Retrieved from http://nflrc.hawaii.edu/rfl

17
Griffith, P. L., & Olson, M. W. (1992). Phonemic awareness helps beginning readers break
the code. The Reading Teacher, 45(7), 516-523. Retrieved from
https://www.jstor.org/stable/20200912
Harris, T. L., & Hodges, R. E. (1995). The literacy dictionary: The vocabulary of reading and
writing. Newark, DE: International Reading Association. Retrieved from
https://eric.ed.gov/?id=ED385820
Hazita, A. (2016). Implementation and challenges of English language education reform
in Malaysian primary schools. 3L: The Southeast Asian Journal of English Language
Studies. 22(3), 650-678. doi:10.17576/3L-2016-2203-05
Jiang, X., Sawaki, Y., & Sabatini, J. (2012). Word reading efficiency and oral reading fluency
in ESL reading comprehension. Reading Psychology, 33, 323–349.
doi:10.1080/02702711.2010.526051
Johns, J., & Berglund, R. L. (2009). Fluency: Strategies and assessments (3rd ed.). Dubuque, IA:
Kendall Hunt Publishing.
Johnson, R. C., & Tweedie, M. G. (2010). Could phonemic awareness instruction be (part
of) the answer for young EFL learners? A report on the early literacy project in
Malaysia. TESOL Quarterly, 44(4), 822-829. doi:10.5054/tq.2010.238131
Jolliffe, W., Waugh, D., & Gill, A. (2019). Teaching systematic synthetic phonics in primary
schools (3rd ed.). London, England: Sage Publications.
Malaysia Ministry of Education. (2011). Primary Standard-Based Curriculum Dokumen
Standard Kurikulum Sekolah Rendah Bahasa Inggeris SK Tahun Satu & Dua. Putrajaya,
Malaysia: Bahagian Pembangunan Kurikulum.
Malaysia Ministry of Education. (2017). KSSR: Bahasa Inggeris Dokumen Standard Kurikulum
dan Pentaksiran Tahun 2. Putrajaya, Malaysia: Bahagian Pembangunan Kurikulum.
McGeown, S. P., Johnston, R. S., & Medford, E. (2012). Reading instruction affects the
cognitive skills supporting early reading development. Learning and Individual
Differences, 22(3), 360-364. doi:10.1016/j.lindif.2012.01
Mesmer, H. A. E., & Griffith, P. L. (2005). Everybody's selling it: But just what is explicit,
systematic phonics instruction?. The Reading Teacher, 59(4), 366-376.
doi:10.1598/RT.59.4.6
Nadiah Yan, A., Napisah, K., & Mariyatunnitha, S. (2014). Implementing the teaching of
phonics in Malaysian primary schools. Asian Journal of English Language and
Pedagogy, 2, 95-111. Retrieved from https://ejournal.upsi.edu.my/index.php/
AJELP/article/view/1103
National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the
scientific research literature on reading and its implications for reading instruction.
Reports of the subgroups. Washington, D.C., WA: National Institute of Child Health
and Human Development.
Purewal, S. (2008). Synthetic phonics and the literacy development of second language young
learners. A literature review of literacy ideologies, policies, and research. (Master’s
dissertation). University of Leeds, Leeds, England.
Rabindra, D. P., Nooreiny, M., & Hamidah, Y. (2016). Implementing phonics in Malaysia.
International Journal of English Language Teaching and Linguistics, 1(1), 1-18.
Rose, J. (2006). Independent review of the teaching of early reading: Final report. Retrieved from
www.standards.dfes.gov.uk/rosereview
Sidhu, G. K., Kaur, S., & Chi, L. J. (2018). CEFR-aligned school-based assessment in the
Malaysian primary ESL classroom. Indonesian Journal of Applied Linguistics, 8, 452-
463. doi:10.17509/ijal.v8i2.13311
Su, S. C. S., & Hawkins, J. (2013). THRASS phonics: A case study of Thomas as an emerging
reader in English. The English Teacher, 42(1), 52-73. Retrieved from
https://journals.melta.org. my/index.php/tet/article/view/231

18
Ukrainetz, T. A., Cooney, M. H., Dyer, S. K., Kysar, A. J., & Harris, T. J. (2000). An
investigation into teaching phonemic awareness through shared reading and
writing. Early Childhood Research Quarterly, 15(3), 331–355. doi:10.1016/S0885-
2006(00)00070-3
UNICEF. (2008). Education is a human right. Literacy and education in Malaysia: Key actions.
Retrieved from http://www. unicef.org/malaysia/index.html
Wang, M., & Koda, K. (2005). Commonalities and differences in word identification skills
among learners of English as a second language. Language Learning, 55, 71–98.
doi:10.1111/j.1467-9922.2007.00416.x
Warid, M. (2015). The English language curriculum in Malaysian indigenous primary
classrooms: The reality and the ideal. 3L: The Southeast Asian Journal of English
Language Studies. 21(3), 1-12. Retrieved from
http://ejournals.ukm.my/3l/article/view/8580
Watts, Z., & Gardner, P. (2012). Is systematic synthetic phonics enough? Examining the
benefit of intensive teaching of high frequency words (HFW) in a year one class.
Education 3-13: International Journal of Primary, Elementary and Early Years
Education, 1-10. doi:10.1080/03004279.2012.710105
Wyse, D., & Goswami, U. (2008). Synthetic phonics and the teaching of reading. British
Educational Research Journal, 34(6), 691-710. doi:10.1080/01411920802268912
Yap, J. R. (2014). Using systematic synthetic phonics as an approach for early literacy. The case of
rural indigenous children (Master’s thesis). University of Malaya, Kuala Lumpur,
Malaysia). Retrieved from http://studentsrepo.um.edu.my/5425/
Zulkifli, A., & Melor, M. Y. (2019). The effectiveness of using Jolly Phonics blending
phonemes to Year 3 English Language classroom. International Journal of
Humanities, Philosophy and Language, 2(8), 150-162. doi:10.35631/ijhpl.280011

19

Vol. 19, No. 10, pp. 19-31, October 2020
Active Learning in Economic Subject:

A Case Study at Secondary School
Ramlee Ismail*, Marinah Awang and Seow Yea Pyng

Universiti Pendidikan Sultan Idris, Perak, Malaysia
https://orcid.org/0000-0002-0511-0343
https://orcid.org/0000-0001-7734-6366
https://orcid.org/0000-0002-7720-4875
Muhammad Ridhuan Bos Abdullah

Universiti Utara Malaysia, Malaysia
https://orcid.org/0000-0003-2018-8605
Abstract. This study aims to investigate the impact of active learning in

the economics subject in Perak secondary school on students’
achievement, motivation, interest, and social interaction. This study
applies a quasi-experimental research design, which involves the control
and the treatment groups with thirty and thirty-four participants,
respectively. The active learning materials for the treatment group were
based on active learning methods provided by the Ministry of Education
with support from structured lesson plans namely: simulations,
discussions, brainstorming, case studies, and visits in the school’s
compound. A questionnaire is also used to measure students’
motivation, interest and social interaction before and after the
intervention. The results showed that students who had experienced
active learning activities score higher than their counterparts
significantly for topic 1 and 2. Besides, findings show that they were
motivated and interested in learning economics through active learning
compared to the traditional approach. However, the researchers find
that the social interaction element is not as significant as the others. It
has been concluded that the active learning method attracts student’s
interest and motivation in the economic subject and subsequently
improves their achievement. Students also will get benefit from the
varieties of teaching method with a focus of learning outcomes to enrich
student-learning activities.
Keywords: active learning; economics education; experimental study;

teaching economic
*
Corresponding author: Ramlee Ismail; Email: ramlee@fpe.upsi.edu.my

20
1. Introduction
The subject of Economics has long been taught in Malaysian secondary schools
under the Secondary Schools Integrated Curriculum since 1991. This subject
aims to provide the basic knowledge to students to enable them to understand
its principles in the modern world. The fundamentals of economics for
secondary schools will assist students in making rational economic decisions in
their daily lives. It helps also to train them to be more critical and creative in
their thinking (Ministry of Education, 2015). Add to this, Generic skills such as
interpersonal skills, managing and problem solving would be emphasised
through this subject (Ministry of Education, Malaysia, 2005). Typically, the
economic subject is more likely to use traditional methods such as lectures (chalk
and talk), note-taking and it is teacher-centered. Benzing and Christ (1997) and
Allgood, Walstad and Siegfried (2013) emphasised that economics courses
feature more chalk and talks than other courses. However, the survey done by
Watt and Beckers (2008) had pointed out that some of the changes in the
teaching method have slowed down, especially in the use of technologies among
young economist. In this vein, the notion of active learning pedagogies,
including peer learning, flipped classroom, problem-based learning, cooperative
learning, and blended learning, has shaped the teaching learning process.
Accordingly, many studies revealed that the active learning method is
successfully increasing the students’ understanding of economics contents
(Tatsumi, 2012; Johnson & Meder, 2019). However, few studies investigate the
effect of active learning in secondary or high school.
In the secondary school contexts, learning economics is associated with the fact
of memorising activities to prepare for the national examination, compared to
student-centered learning process. Consequently, students are less likely to be
motivated and are reluctant to its learning. In comparison with another subject
such as the Principle of Accounts or Business, the academic achievement in
Economics subject is still behind and unsatisfactory. For example, the average
numbers of students who passed this subject in the national examination
(Malaysian Education Certificate) was 63%, from 2000 to 2010. In contrast, those
who passed the Principle of Accounts and Business have scored 69.9% and
76.2% in the same period (Ministry of Education, Malaysia, 2010). Nevertheless,
for the last five years, the results have increased by 5%to give 75.7% in 2014 and
80.20% in 2015.
A lower achievement in the economic subject is to certain extent related to

students’ poor academic background in calculation, the negative attitudes
towards the subject , unattractive teaching methods, and the teaching load
(Becker & Watts, 1996, 2001a, 2001b; Watts & Becker, 2008; Backhouse, 2012).
Many educationists believe that students’ attitude and motivation towards this
subject should cooperate with an attractive teaching method. However, the
likelihood of economic teachers using other methods of teaching is lower than
teachers of other subjects (Becker, 1998; Watts & Schaur, 2011). As a reaction, the
Ministry of Education in Malaysia (MoE) promotes and encourages active
learning because it betters students' performance. Additionally, the teaching
learning process that involves active learning in a classroom is more enjoyable

21
and exciting (Becker, 1997; Zheng, 2017; Schlehuber, 2017). Therefore, the MoE
has introduced a module of Active Learning Practices in Economics since 2005 to
help teachers involve in active learning in the classroom (Ministry of Education,
Malaysia, 2005). This module covers all topics in the syllabus and teaching
materials to support all activities. Yet, it has been observed that there have been
no recent developments in active-learning module in the secondary school
economic subject. Therefore, the present paper aims to determine whether active
learning in the Economics Module could improve secondary school students’
interest, motivation, social interaction, and their achievement.
Simply put, active learning is a method of teaching and learning that involves
direct participation of students in the learning process (Ministry of Education,
Malaysia, 2005). Students not only learn to do something, but they also think
about the actions and decisions taken in those activities (Bonwell & Eison, 1991;
Roach, 2014). According to Silberman (1996) and McLaughin et al. (2013),
learning is not about a plain absorption of information into the students’ minds,
but it also needs the involvement of their minds and actions. The outcomes of
active learning will be permanent because they are engaged in tasks that involve
higher cognitive thinking skills such as analysis, synthesis, and evaluation
(Bonwell & Eison, 1991; McLaughin, 2013). Thus, students can solve problems
that require higher cognitive levels as well as strengthening the skills to think
critically and creatively. The benefits of active learning documented with the
economics education literature have always been highlighted as one of the
teaching method principals (Carlson & Skaggs, 2000; Ginsburg, 2009; Jensen &
Owen, 2003; Maier & Keenan, 1994; Manning & Riordan, 2000; Watts & Schaur,
2011, Moon, Wold & Francom, 2017; Zheng, 2017; Bryan & Jett, 2018). Therefore,
active learning in economics education is mandatory (Becker, 1997; Becker &
Watts, 2001a; Hansen, 2001; Salemi, 2002) to be used in the classroom. This
method of teaching requires the active involvement of students to achieve
sustainable learning outcomes in the economics subject (Cross, 1987; MOE, 2005;
Siegfried et al.; 1991; Whiting, 2006). Some empirical studies on active learning
find that students who are involved in teaching and learning using this
approach performed better than in the traditional approach (Gratton-Lavoie &
Stanley, 2009). Moreover, students’ interaction and collaboration in small groups
do not only increase, but also will contribute to greater subject materials
(Yamarik, 2007; Bryan & Jett, 2018).
With regard to the above said, the traditional learning process carried out by
teachers in the classroom does not highlight the concept of independent
learning. Rather, in the teacher-centered learning process, students are given less
opportunity to apply or develop their cognitive and affective skills. They only
receive the information from their teachers passively and are required to act on
what is instructed by the teachers (Prince, 2004). This philosophy is old-
fashioned and does not align with the actual role students need to play to
survive in the teaching learning process. In this respect, active learning activities
such as group discussions, problem-solving, simulations, games and case studies
provide students with the opportunity to express and support their ideas as well

22
as to consider the thoughts and the opinions of others (Meyers & Jones, 1993;
McCarty, Ford & Ludes, 2018). With this in mind, students can exchange their
ideas and interact freely with their classmates. Additionally, active learning
activities can attract students' interest and create a fun, lively and cheerful
classroom atmosphere (Salemi, 2002).
In fact, active learning is not only useful in enhancing students' understanding,

but it also increases students’ achievement (Carlson & Velenchik, 2006; Bartlett,
2006; Buckles & Hoyt, 2006; Yamarik, 2007; Filio et al., 2013; Calimaries & Sauer,
2015; Cavigllia-Harris, 2016; Rita et al., 2016). Carlson and Velenchik (2006)
demonstrate that the technique of discussions in the economics class could
develop students' analytical thinking skills. Students can apply the economic
theory using the information and data provided by their instructor. Meanwhile,
Bartlett (2006) finds that the cooperative learning technique in economics
education is effective in improving students' academic performances in
examinations. Active learning activities provide opportunities for students to
acquire higher thinking skills when interpreting economic concepts. In this
sense, Rupp (2014) comes across the fact that elementary school students have
significant improvements in their understanding of fundamental economic
concepts. This situation sharply contrasts with the traditional learning situation
where students are merely asked to understand the precise concept from the
teacher's explanation (Salemi, 2002).
In the same line of thought, many scholars also realized that active learning can
improve students’ interest towards the economics subject (Brokaw & Merz, 2004;
Dixit, 2006; Rotgans & Schmidt, 2011; Strow & Strow, 2006). Brokaw & Merz
(2004) show that active learning could trigger students' interest in the critical
concepts of the economy. Games technique, for example, can improve student's
understanding of economic theories that are too abstract for them to
comprehend (Dixit, 2006). Furthermore, role-play can also enhance student's
interest in the subject, as well as prevents boredom and sleepiness in the
classroom. In parallel, teachers should provide guidance, coaching, and
motivation that are necessary for students who have difficulties with active
learning activities in class (Buckles & Hoyt, 2006). Also, rewards that are given
by teachers to those who did well in a particular activity either in verbal forms,
such as praises and encouragement or in material forms such as chocolates or
small gifts, will make students more motivated to actively participate in group
activities (Slavin, 1990). The latter do not only foster intrapersonal and
interpersonal skills among the students, but also inculcate ethical values such as
respecting the opinion of others, collaborating and, to be fair and equitable in
emphasizing logical facts. In doing so, learner's autonomy and collaboration will
increase (Becker, 1997; Bonwell & Eison, 1991; Johnson, Johnson & Smith, 1998;
Salemi, 2010; Toh Wah Seng; 2008; Bailey et al., 2013; Bergmann et al., 2013), and
their interaction improves academic achievement and interest (Brooks &
Kandler, 2002). Which in return give more opportunities and flexibilities to
identify their learning style, interests, and abilities (Fuller et al., 2015 & Mazur et
al., 2015).

23
Though the significant endeavour done by the MoE and the outstanding
materials for effective outcomes, some economics teachers have almost forgotten
the existence of this module as most of them prefer the traditional method of
instruction which is ‘chalk and talk,'. In this context, Hansen at al. (2002)
contended that economics teachers or instructors have been slow to adopt the
new teaching methods. The common reasons for using the traditional approach
in the economics subject includes the large class size, lack of materials, and the
many topics covered in the syllabus. Goffe and Kauper (2014) suggest that the
predominant reason why teachers prefer to use the lecturing method in the
economics subject is the ability to control the delivery and coverage of content.
In the Malaysian context, teachers complained that they are reluctant with non-
related teaching activities at school such as too much clerical work, data
entering, and preparing the students for various competitions at school, district,
state and national levels. A survey has been carried out on the teacher’s
workload in Malaysia and findings show that the average number of hours is 57
hours per week, but some of them have workload up to 76 hours per week.
Unfortunately, the proportions of time spent in preparing teaching activities
were low (MoE, 2013). Indeed, when the educational system is streaming into
so-called science and art, economics students then fall into the ‘second category’
wherein the likelihood of passivity and anonymity exists and therefore becomes
a barrier of active teaching and learning ( Hoyt at al., 2010; Roach, 2014). The
quality of learning also depends on learners' abilities to steer and control their
learning processes (Niemi, 2002) and past performance (Denny, 2014). The
outcome of students’ achievement is measured by “how many got A’s or how
many per cent passed the economics subject” which is in fact not an encouraging
statement teachers appreciate to spend more time on effective classroom
management, solid planning teaching materials, and activities. However, these
are not the ultimate reasons why the active teaching and learning module by
MoE should be left behind. Hence, the researcher’s rationale is to study the
effectiveness of the current teaching method, using the materials provided by
the MoE, to improve students’ interest, motivation, social interaction and
achievement in the Economics subject at secondary schools.
3. Methodology
The conceptual framework for this study adopted the three-phase learning
model introduced by Biggs (1978). The first phase of the survey is a pre-study,
which refers to the pre-review factors of active learning namely: student’s
achievement, interest, motivation and social interaction of students in an
Economic subject. The second phase involves the process of teaching and
learning (T&L), which is active learning that was carried out in the Form Four
economics classes. In hope to meet the intended results, discussions,
simulations, brainstorming, case studies and visits within the school’s
compound are the five active learning activities the researchers have used in this
research paper. These learning activities are indeed extracted from the “Best
Active Learning Practices in Fundamental Economics” as provided by MoE. The
materials and tools for these learning activities will further enhance the
effectiveness of student's learning in the final phase. Figure 1 shows the three
phases.

24
Pre-review of the effectiveness Teaching & learning process -

Active learning Output
• Academic achievement • Discussions • Academic

• Interest • Simulations achievement
• Motivation • Brainstorming • Interest
• Social interaction • Case studies • Motivation
• Visit school compound • Social
interaction
Figure 1. Active Learning Process in Teaching and Learning of Fundamental
Economics
This research is a quasi-experimental design which is used to test whether there

are any significant effects of active learning and the traditional methods on the
academic achievement of students in the Form Four FE subject. Also, the
researchers used a questionnaire to obtain students’ feedback on the
effectiveness of active learning methods in their economics class. The
questionnaire is administered to sixty-three participants and it consists of two
sections. The first section covers information on the personal background of the
respondents such as gender, parent’s educational level, and others. Whereas,
section two is meant to collect data on the respondents’ interest, motivation, and
social interaction using the five Likert scale. The questionnaire is adapted from
Motivated Strategies for Learning Questionnaire – MSLQ research tool, which was
developed by Pintrich and DeGroot (1990). This MSLQ survey tool has been
widely used to measure the level of motivation in students' learning.
Meanwhile, social interaction instruments have been modified from the
Inventory of Parent and Peer Attachment (IPPA) survey by Armsden and
Greenberg (1987).
Preliminary test (pretest) and performance tests (posttest) are used to measure
students’ basic knowledge, and evaluate their academic achievement,
respectively. These tests are developed by the researchers in which they are
based on the Malaysian Certificate Education Examination and certified by an
expert from the same state. Indeed, the preliminary test is used as a covariate to
streamline the fundamental difference between treatment and control groups.
The present study takes place in Perak secondary school where the researchers
explicitly introduced the studied criteria and the materials to be used in both
groups (treatment group, N=34 and Control group, N=30). The investigators
have used random sampling method to group the participants based on their
registration number, but those with odd registration number are reallocated to
the treatment group, including those with registration number who are in the
control group. The control class is used to eliminate the variation effect of the
student's knowledge and economics background. The same teacher teaches both
classes on the same topics but with different methods. The experiment lasts for
four weeks, which covers Unit 5 with the Topic: Market. The teaching load is "2
+ 1" per week, which is 80 and 40 minutes for each session. The materials and
lesson plans used were Active Learning Module from page 148 to 167; covering

25
11 teaching materials with four main activities namely: brainstorming, role play,
simulation and group work. The control group uses a traditional method.
Sample (64 students)
Quasi-experimental study and survey
Pre-test
Treatment group (34 students) Control Group (30 students)
Pre survey
Traditional learning
Active learning
Achievement Test Achievement Test
Post survey
Data analysis
Summary/discussion/recommendations
Source: Modified from Shadish, Cook, and Campbell (2002)

Figure 2: The Intervention vs the Traditional Method
The researchers have also provided plans to be used in daily lessons according
to topics, reference materials, papers, and other group activities to help instruct
others entirely. Also, guidance and answers for teachers are also provided to
ensure that active learning activities are carried out smoothly in the economics
class. A summary of the activities for both methods is shown in Table 1.
Table 1: Comparison of Active and Traditional Learning Methods

Active Learning Traditional Learning
Students were introduced to study The teacher introduced the lesson topic
topics through an impressive set of through a brief question and answer
Induction.
Teaching activities using active learning The teacher presented the lesson
materials. content by using the lecture method.
Students complete the individual and Students listen to while taking notes
groups’ learning materials
The conclusion made by students or The teacher made a summary at the
teachers at the closing session closing session
Students do the activities in the form of The teacher gave a few questions for
active learning homework

26
4. Results
Descriptive statistics were used to describe the demographical factors of our
sample. Table 2 depicts some students’ background for both groups. The male
sample was slightly larger than the female sample in the treatment group
compared to the control group. Table 2 summarises the collected data on the
respondents’ background information.
Table 2: Background Profile of the Study’s Respondents (n=64)

Treatment Group Control Group
Item n (%) n (%)
Gender
Male 19 (55.9) 13 (43.3)
Female 15 (44.1) 17 (56.7)
Parents’ education
Primary school 7 (20.6) 7 (23.3)
Secondary school 25 (73.5) 23 (76.7)
Tertiary education 2 (5.9) -
Parents’ / Guardian’s Employment
Government Employees 11 (32.4) 8 (26.7)
Private Sector Employees 6 (17.6) 7 (23.3)
Self-employed 13 (38.2) 11(36.7)
Retirees 4 (11.8) 4 (13.3)
Total Monthly Household Income
Less than RM 1000 21 (61.8) 20 (66.7)
RM 1000 – RM 1500 8 (23.5) 7 (23.3)
RM 1501 – RM 2000 4 (11.8) -
More than RM 2000 1 (2.9) 3 (10.0)
The differential in the mean score of active learning activities in the classroom is
tested by the Paired Sample T-Test and analysis of covariance (ANCOVA) for the
achievement test. The difference between the mean scores for the treatment
group before and after involvement in the active learning process is compared
and determined whether there were any significant differences between the two
data sets. The Paired Sample t-test is used to test whether there is any significant
increase in interest, motivation and social interaction in the treatment group
after they have experienced the activities that were carried out using the active
learning method. While, the ANCOVA technique is used to eliminate the
existence of economics knowledge of the two groups that were involved in the
current attempt (Shadish, Cook & Campbell, 2002). A covariate uses the latest
monthly assessment for both groups as the pre-test. Column five in Table 3
depicts students’ achievement in the first and second topics.
Table 3: Analysis of Covariance

Experiment Topic Group N Mean Test F p
Performance
Active 1 Control 30 53.00
learning Treatment 33 61.52 73.181 0.000
2 Control 30 56.00
Treatment 33 67.42 92.659 0.000

27
As displayed, the mean score for the treatment group was higher than the
control group in both topics. The findings show that the academic achievement
of students who used active learning activities was significantly higher than
their counterpart with the F=92.66.
Table 4 reflects the results of students’ experience in active learning. The mean
difference between pre and post-survey for "Interest, Motivation, and Social
Interaction" are shown in column three. The positive value indicates that the
average score for post-test is higher than the pre-test.
Table 4: Paired Sample t-test

Variables Mean difference after and df t value Sig. (two tails)
before treatment
Interest 1.817 33 14.240 0.000
Motivation 1.425 33 10.572 0.000
Social Interaction 0.225 33 1.688 0.101
The significant or the non-significant of the mean difference in these activities

depends on the t value. Therefore, the t value for “Interest and Motivation” is, t
(33, p = 0.000) = 14.24 and t (33, p = 0.00) = 10.57 respectively, which was smaller
than 0.05 which indicates that both variables are significant. That is to say, the
active learning activities conducted in the classroom enabled students to show
more interest and motivation in learning the economics subject. However, as
displayed, there is no significant difference in the mean score of “Social
Interaction” before and after learning activities.
5. Discussion and Conclusions

This study concludes that the treatment group with the notion of active learning
method performs better than those following the traditional learning process in
the control group. In this regard, some studies also showed that active learning
had improved students’ academic achievement, (Budd, 2004; Johnson &
Johnson, 1994; Johnson, Johnson & Smith, 1998; Meyers & Jones, 1993; Roche,
2014 and Slavin, 1995). However, Malek, Hall and Hodget (2014) found that
there is no statistically significant improvement when the traditional teaching
methods were tested with the alternative teaching method. The findings of the
current paper dictate that students become more motivated in learning
economics using the active teaching and learning method. These findings are
also found by other researchers namely, Bartlett (2006), Becker (1997), Bonwell
and Eison (1991), Brokaw and Merz (2004), Carlson and Velenchik (2006), Dixit
(2006), Hazlett (2006) and Salemi (2002).Students are fond of the active learning
activities conducted in class mainly in simulations, group discussions, case
studies and visits within the school’s compound. They approach these practices
as being attractive and fun which help them to understand the concepts of
economics better. More than that, active learning methods can also enhance
students' motivation in the process of learning economics for Form Four.
Although the application of motivational research to the economics subject is
scarce, there is some evidence that motivation is an additional factor to
successful output in economics among students (Arnold & Straten, 2012),

28
because the teaching aids and materials in active learning are rewarding and fun
for students (Salemi, 2002). Nevertheless, results of the current endeavour do not
show the mean difference before and after experiment on the aspect of social
interaction. That is to say, it appears that there is no conclusive evidence that the
active learning method is effective in improving the social interaction of
students. Probably, changes in a class setting should be included in preparing
class activities. Because, active learning classroom (ACL) is a common setting
and arrangements for enhance effective learning process (Baepler & Walker,
2014; Metzger, 2015), the latter will contribute to make a significant impact on
social interaction with new team members and foster a closer relationship with
new friends. As far as economics teachers are concerned, they should not solely
rely on traditional learning methods, as a reason, to complete the syllabus given
for a large number of students in a class. The active teaching materials provided
by educational department should be frequently used, diversified and blended
with latest teaching and learning devices to motivate students. Indeed, further
empirical studies should explore promising alternatives to enable learners
understand the significant role social interaction plays and what pedagogies to
develop for successful integration.
References
Allgood, S., Walstad, W. B., & Siegfried, J. J. (2015). Research on teaching economics to
undergraduates. Journal of Economic Literature, 53(2), 285-325.
Armsden, G. C., & Greenberg, M. T. (1987). The inventory of parent and peer
attachment: relationships to well-being in adolescence. Journal of Youth and
Adolescence, 16(5), 427-454.
Arnold, I. J. M., & Straten, J. T. (2012). Motivation and math skills as determinants of
first-year performance in economics. The Journal of Economic Education, 43(1), 33–
47.
Baepler, P., & Walker, J. D. (2014). Active learning classrooms and educational alliances:
Changing relationships to improve learning. New Directions for Teaching and
Learning, 2014(137), 27–40. doi:10.1002/tl.20083
Bartlett, R. L. (2006). The evolution of cooperative learning and economics instruction. In
W. E. Becker, M. Watts & S. R. Becker (Eds.), Teaching Economics: more alternatives
to chalk and talk (pp. 39-58). Cheltenham: Edward Elgar Publishing Limited.
Becker, W. E. (1997). Teaching economics to undergraduates. Journal of Economic
Literature, 35(9), 1347-1373.
Becker, W. E. (1998). Engaging student in quantitative analysis with the academic and
popular press. In W. E. Becker & M. Watts (Eds.), Teaching Economics: more
alternatives to chalk and talk (pp. 241-267). Cheltenham, UK: Edward Elgar
Publishing Limited.
Becker, W. E., & Watts, M. (1996). Chalk and talk: A national survey on teaching
undergraduate economics. American Economic Review, 86(2), 448–453.
Becker, W. E., and M. Watts. (2001a). Teaching economics at the start of the 21st century:
Still chalk-and-talk. American Economic Review, 91(2), 446–51.
Becker, W. E., & M. Watts. (2001b). Teaching methods in U.S. undergraduate economics
courses. Journal of Economic Education, 32, 269–279.
Benzing, C., & Christ, P. (1997). A survey of teaching methods among economics faculty.
The Journal of Economic Education, 28(2), 182-188.
Biggs, J. B. (1978). Individual and group differences in study processes. British Journal of
Educational Psychology, 48, 266-279.

29
Bonwell, C. C., & Eison, J. A. (1991). Active Learning: Creating Excitement in the Classroom.
ASHE-ERIC Higher Education Report No. 1. Washington, DC: The George
Washington University, School of Education and Human Development.
Brokaw, A. J., & Merz, T. E. (2004). Active Learning with Monty Hall in a Game Theory
Class. Journal of Economic Education, 35(3), 259-268.
Bryan, N., & Jett, C. C. (2018). “Playing school”: Creating possibilities to inspire future
Black male teachers through culturally relevant play. Journal for Multicultural
Education, 12(2), 99–110. https://doi.org/10.1108/JME-04-2017-0024
Brooks, T. B., & Khandker, A. W. (2002). A Collaborative Learning Lab: Does The Form
Matter? Contemporary Economic Policy, 20(3), 330-338.
Buckles, S., & Hoyt, G. M. (2006). Using Active Learning Techniques in Large Lecture
Classes. In W.E. Becker, M. Watts & S.R. Becker (Eds.), Teaching economics: More
alternatives to chalk and talk (pp. 75-88). Cheltenham, UK: Edward Elgar
Publishing.
Budd, J. W. (2004). Mind maps as classroom exercises. Journal of Economics Education,
35(1), 35-46.
Carlson, J. L., & Skaggs, N.T. (2000). Learning by trial and error: A case for moot courts.
Journal of Economic Education, 31(2), 145-155.
Carlson, J. A., & Velenchik, A. (2006). Using the Case Method in the Economics
Classroom. In W.E. Becker, M. Watts & S.R. Becker (Eds.), Teaching economics:
More alternatives to chalk and talk (pp. 59-74). Cheltenham, UK: Edward Elgar
Publishing.
Cross, P. K. (1987). Teaching for learning. AAHE Bulletin, 39(8), 2-7.
Denny, E. (2014). Factors influencing the performance of non-economics majors in an
introductory economics course. International Review of Economics Education, 17, 1-
16.
Dixit, A. (2006). Restoring Fun to Game Theory. In W.E. Becker, M. Watts &S.R. Becker
(Eds.), Teaching Economics: more alternatives to chalk and talk (pp. 1-20).
Cheltenham, UK: Edward Elgar Publishing.
Goffe, W. L., & Kauper, D. (2014). A Survey of Principles Instructors: Why Lecture
Prevails. The Journal of Economic Education, 45(4), 360-375.
doi:10.1080/00220485.2014.946547
Ginsburg, M. (2009). Active learning pedagogies as a reform initiative: Synthesis of case studies.
Washington, DC: Academy for Educational Development.
Hansen, W. L. (2001). Expected Proficiencies for undergraduate economics majors.
Hazlett, D. (2006). Using Classroom Experiments to Teach Economics. In W.E. Becker, M.
Watts & S.R. Becker (Eds.), Teaching economics: More alternatives to chalk and talk
(pp. 21-38). Cheltenham, UK: Edwar Elgar Publishing.
Hoyt, G., Kassis, M., Vera, D., & Imazeki, J. (2010). Making cooperative learning effective
for economics. In W. B. Walstad & M. K. Salemi (Eds.), Teaching innovations in
economics: Strategies and applications for interactive instruction (pp. 65–94).
Jensen, E. J., & Owen, A. L. (2003). Appealing to good students in introductory
economics. Journal of Economic Education, 34(4), 299-325.
Johnson, D. W., & Johnson, R. T. (1994). Learning together and alone: Cooperative,
competitive and individualistic learning. Boston: Allyn &Bacon.
Johnson, D. W., Johnson, R. T., & Smith, K. A. (1998). Cooperative learning returns to
college: What evidence is there that it works?, Change, 30, 26-35.
Johnson, M., & Meder, M. (2019). A Meta-Analysis of Technology Interventions in
Collegiate Economics Classes. Journal of Economic Education, 1-27.
doi:10.2139/ssrn.3501227

30
Maier, M. H., & Keenan, D. (1994). Teaching Tool: Cooperative learning in economics.
Economic Inquiry, 32(2), 358-361.
Malek, N. P., Hall, J. P., & Hodges, C. (2014). A review and analysis of the effectiveness
of alternative teaching methods on student learning economics. Perspectives on
Economic Education Research, 9(1), 75-85.
Manning, L. A., & Riordan, C. A. (2000). Using groupware software to support
collaborative learning in economics. Journal of Economic Education, 31(3), 244-252.
Meyers, C., & Jones, T.B. (1993). Promoting active learning: Strategies for the college
classroom. San Francisco: Jossey-Bass.
Metzger, K. J. (2015). Collaborative teaching practices in undergraduate active learning
classrooms: A report of faculty team teaching models and student reflections
from two biology courses. Bioscene, 41(1), 3–9.
Moon, A. L., Wold, M. C., & Francom, G. M. (2017). Enhancing reading comprehension
with student-centered iPad applications. G. M. Tech Trends, 61(2), 187–194.
Ministry of Education (MoE). (2005). The best practice of active learning in Fundamental
Economics (Amalan Terbaik Pembelajaran Aktif Dalam Ekonomi Asas). Kuala
Lumpur: Pusat PerkembanganKurikulum.
Ministry of Education (MoE). (2010). AnalisisKeputusan Peperiksaan SPM 2000-2009. Kuala
Lumpur: Lembaga Peperiksaan Malaysia.
Ministry of Education (MoE). (2013). Malaysia Education Blueprint 2013 – 2015. Kuala
Lumpur: Ministry of Education (MoE).
Ministry of Education (MoE). (2015). Kurikulum Standard Sekolah Menengah: Ekonomi.
Kuala Lumpur: Ministry of Education (MoE).
Niemi, H. (2002). Active learning – a cultural change needed in teacher education and
schools. Teaching and Teacher Education, 18(7), 763–780.
Pintrich, R. R., & DeGroot, E. V. (1990). Motivational and self-regulated learning
components of classroom academic performance. Journal of Educational
Psychology, 82(1), 33-40.
Prince, M. (2004). Does Active Learning Work? A Review of the Research. Journal of
Engineering Education, 93(3), 223-231.
Roach, T. (2014). Student perceptions toward flipped learning: New methods to increase
interaction and active learning in economics. International Review of Economics
Education, 17, 74-84.
Roche, K. (2014). An active-learning exercise on learning negotiation as a way to mitigate
the gender wage gap for introductory microeconomics. International Review of
Economics Education,1, 32-42.
Rotgans, J. I., & Schmidt, H. G. (2011). The role of teachers in facilitating situational
interest in an active-learning classroom. Teaching and Teacher Education, 27(1), 37-
42.
Rupp, N. G. (2014). Teaching economics with a bag of chocolate: A classroom experiment
for elementary school students. International Review of Economics Education, 16,
122-128.
Salemi, M. K. (2002). An Illustrated Case for Active Learning. Southern Economic Journal,
68(3), 721-731.
Salemi, M. K. (2010). Developing teacher expertise for economists through a workshop
experience. In M. K. Salemi, & W. B. Walstad (Eds.), Teaching innovations in
economics: Strategies and applications for interactive instruction (pp. 1-24).
Schlehuber, L. (2017). The future of student life: Learning. On the Horizon, 25(3), 169–172.
doi:10.1108/OTH-05-2017-0023.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-
Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.

31
Siegfried, J. J., Bartlett, R., Hansen, W. L., Kelley, A. C., McClosky, D. N., & Tietenberg, T.
H. (1991). Thestatus and prospects of the economics major. Journal of Economic
Education, 22(3), 195-224.
Silberman, M. L. (1996). Active Learning: 101 strategies to teach any subject. London: Allyn
and Bacon.
Slavin, R. E. (1990). Research on cooperative learning: Consensus and controversy.
Educational Leadership, 47(4), 52-54.
Slavin, R. E. (1995). Cooperative learning: Theory, research, and practice. Massachusetts:
Allyn & Bacon.
Strow, B. K., & Strow, C. W. (2006). A Rent-Seeking Experiment for the Classroom.
Yamarik, S. (2007). Does Cooperative Learning Improve Student Learning Outcomes?
Journal of Economic Education, 38(3), 259 – 277.
Watts, M., & Schauer, G. (2011). Teaching and assessment methods in undergraduate
economics: A fourth national quinquennial survey. Journal of Economic Education,
42,294–309.
Whiting, C. (2006). Data-based Active Learning in the Principles of Macroeconomics
course: A Mock FOMC Meeting. Journal Economics of Education, 37(2), 171-177.
Zheng, X. (2017). Research on the student-centered learning in mass media reading
course. Theory and Practice in Language Studies, 7(3), 227–233.
doi:10.17507/tpls.0703.09

32

Vol. 19, No. 10, pp. 32-59, October 2020
Inclusion of the FuzzyILS Method in MOODLE

for Creating Effective Courses
Antonio Silva Sprock

Universidad Central de Venezuela, Caracas, Venezuela
https://orcid.org/0000-0002-9911-4774
Abstract. The work exposes the analysis of the effectiveness of courses

in MOODLE when the course is developed with activities according to
the learning styles of the students which have been determined using
the FuzzyILS Method. The investigator has adapted the FuzzyILS
method and designed the FuzzyILS questionnaire based on former
attempts and the researcher’s observations to calculate the degrees of
belonging of the fuzzy sets, and evaluate the activities of MOODLE,
according to the learning styles of students. The investigator has used
the experimental research with forty students from the Central
University of Venezuela grouped into two groups, namely the
experimental group and the control group with twenty students each.
The results show evidence of improvement in learning and reflect
greater participation in the proposed activities when the FuzzyILS
Method and MOODLE techniques are used with the learning styles of
students.
Keywords: FuzzyILS Method; Learning Styles; Fuzzy Sets; MOODLE
1. Introduction
Information and Communication Technologies (ICT) continue to advance in
education and makes it possible to extend teaching and learning environments.
Its enrichment permits the development of collaborative and interactive virtual
environments, with the use of different didactic materials, such as presentations,
multimedia elements, intelligent tutors, among others. The use of ICT has also
allowed the creation of new educational models, related to distance education,
such as E-Learning, Blended-Learning, and Mobile-Learning (Dorfsmani, 2012).
Distance education is advancing, not only as an alternative in 2020, but also it
becomes the only option in the face of the COVID-19 pandemic. That is to say,
the new situation leads to new challenges to take place: to navigate the
pandemic by maintaining classes without overloading the different actors
mainly in finding ways to establish a pedagogical link remotely, and be attentive
to all that is happening around this context (Artopoulos, 2020). Many tools
favour the virtual environments in e-learning, part of these are the Learning
©2020 The author and IJLTER.ORG. All rights reserved.

33
Management Platforms (Learning Management Systems or simply LMS), for

example, MOODLE.
On the other hand, students, regardless of the learning environment, they

structure their strategy for learning depending on their learning style, they use
in a conscious, controlled, and intentional way, procedures (set of steps,
operations, or skills) to learn and solve problems (Díaz-Barriga & Hernández,
2010). Its effectiveness depends to a great extent on the instructional strategy
used by the teacher (Ossandón & Castillo, 2006). Therefore, learning styles
represent a determining factor of the teaching and learning process (Paredes,
2008). Felder & Silverman (1988) for example, argue that students with a strong
preference for a learning style may have difficulties in the process if the teaching
environment is not adapted to their learning style. In this concern, students learn
and perform better if the learning environment includes activities in which they
participate and apply knowledge, rather than being passive in the process and
simply listening (Prince, 2013; Freeman, Eddya, McDonougha, Smithb,
Okoroafora, Jordta & Pat Wenderotha, 2014), bearing in mind the fact that the
effectiveness of a technique is related to the expected result when performing
tests that measure the level of knowledge of the student about the knowledge
imparted (Lozano, Suescún, Vallejo, Mazo & Correa, 2020).
This study aims to analyze the effectiveness of a course MOODLE, using the
techniques appropriate to the students learning styles being determined by the
FuzzyILS Method, which broke with the dichotomy of the Index of Learning
Styles Questionnaire (ILS) of Felder and Soloman (2007).
This section is meant to review the relevant concepts and elements related to this
study. First, it begins with the MOODLE issue, the learning styles concept, the
correlation among MOODLE and learning styles, and the FuzzyILS Method.
2.1 MOODLE
The modular object-oriented dynamic learning environment (MOODLE) is a
learning management platform (Learning Management Systems or simply LMS)
where teachers can build courses using templates for contents developments,
among them forums, chat, quizzes, glossaries, workshops, surveys, and exercises
type multiple-choice, true/false, and one-word answers. The services provided
generally include access control, educational content development,
communication tools, and administration of student groups. The LMS has
numerous advantages in the field of distance education, especially in creating a
student-teacher link which translates into educational and professional success
(Díaz, 2009; Boneu, 2007). The LMS favours communication and interaction
between the students of the subject and the teacher who dictates it. These tools
overcome the barrier of shyness and favored the interaction and bonding among
students. Such a fact generates collaborative work to occur (Silva & Vicari, 2016).

34
For this, MOODLE offers multiple activities and tools (Moodle, 2020):
• Chat: Allows participants to have a real-time synchronous discussion
• Choice: A teacher asks a question and specifies a choice of multiple responses
• Forum: Allows participants to have asynchronous discussions
• Glossary: Enables participants to create and maintain a list of definitions, like
a dictionary
• Lesson: For delivering content in flexible ways
• Survey: For gathering data from students to help teachers learn about their
class and reflect on their teaching
• Wiki: A collection of web pages that anyone can add to or edit
• Workshop: Enables peer assessment.
These functionalities support flexible and active learning under the

constructivist approach (Cabero-Almenara, Arancibia & Del Prete, 2019), and the
teachers have much freedom concerning the format of the content they can
create in MOODLE. They have access to a wider variety than when using
traditional handouts (Szirmai, 2020); however they do not assist the teacher in
the construction of the courses as it is often a challenge for a teacher to
successfully structure courses and activities in the LMS (Silva, Ponce & Meneses,
2013). This happens because they must prepare the contents and the
instructional activities and direct the learning. This process is efficient only when
teachers consider the characteristics of the students and the previously
established instructional objectives. These students’ characteristics include their
profiles, and MOODLE is pedagogically neutral for it does not consider student
profiles to structure the courses (Almeida, Romero & Arce, 2017). Then this is a
deficiency of MOODLE because the instructional techniques must have different
degrees of adequacy and effectiveness in the teaching and learning process. This
effectiveness calls forstudents’ learning styles (Silva, Ponce & Meneses, 2013).
2.2. The Learning Styles

There are multiple conceptions of learning styles (Silva, 2018), among them the
one proposed by James Keefe (1987), who defines them as "cognitive, affective,
physiological traits, preferences for the use of the senses, environment, culture,
behavior, comfort, development and personality, which serve as relatively stable
indicators, of how people perceive, interrelate and respond to their learning
environments and their methods or strategies in their way of learning" (Keefe,
1987). The importance of knowing learning styles is helpful mainly in adapting
the way of teaching to each student's style, improving education, and making
education more effective (Silva, 2018). The current research paper is based on the
Felder and Silverman Learning Styles Model (FSLSM) (Felder & Silverman,
1988), since it is considered to be the most used in technologies and sciences
students.
2.3. MOODLE and Learning Style

Regarding MOODLE, two ways in incorporating learning styles in the LMS have
been proposed. Firstly, some authors added learning styles questionnaire to
MOODLE, such as the CHAEA questionnaire being associated with the Kolb
Model (Torres, Molina & Álvarez, 2010; Lavigne, Díaz, McAnally & Organista,
2013; Rubio, 2009; Pacheco, Miranda & Alonso, 2020); others (Peña, Mendoza &

35
González, 2015; Puello, Fernández & Cabarcas, 2014; Graf & Kinshuk, 2007; Graf,
Kinshuk & Liu, 2008; Mohd, Yahya, Ismail, Jalil & Noor, 2019) associate it with
the ILS (Felder & Soloman, 2007), while some others assimilate it with the Myers
and Briggs model (Bignetti & Jasbinschek, 2011).
The second way concerned its association with FSLSM (Felder & Silverman,
1988) which has been studied by Despotović-Zrakić, Marković, Bogdanović,
Barać and Krčo (2012) and more recently with Karagiannis and Satratzemi (2018)
study. In fact, Rubio (2009) incorporated a learning styles module based on the
CHAEA questionnaire in MOODLE, which was called LSTest. Subsequently,
Puello, Fernandez and Cabarcas (2014) modified the LSTest and incorporated
the Index Learning styles ILS (Felder & Soloman, 2007) to this LSTest. Both
attempts serve as the basis for this proposal. In addition to these works, the
study by Despotović, Marković, Bogdanović, Barać and Krčo (2012) shows the
relationship between MOODLE activities and tools with the learning styles of
FSLSM (Felder & Silverman, 1988), where they assigned "Yes", "No" or "Shortly"
as indexes to display the adequacy of the MOODLE activities with the learning
styles. Table 1 shows this relation.
Table 1: The relationship of MOODLE activities with the learning styles

(Despotović et al., 2012).
MOODLE FSLSM Dimensions
Activities Sens. Intui. Visual Verb. Sec. Glob. Act. Refl.
Forum Yes Yes No Yes Yes Yes Yes Yes
Chat Yes No No Yes Yes No Yes No
Glossary Yes No No Yes Yes No Yes Yes
Workshop Yes Yes Yes Yes Yes Yes Yes Yes
Survey Yes Yes Yes Yes No Yes No Yes
Choice Yes No Yes Yes Yes Shortly Yes Yes
Lesson Yes Shortly Yes Yes Yes Shortly Yes Yes
Wiki No Yes No Yes Yes No Yes Shortly
2.4. The FuzzyILS Method

The FuzzyILS Method (Silva, 2020) is a system to evaluate the learning styles
using a fuzzy questionnaire (Appendix), based on the FSLMS (Felder &
Silverman, 1988) and as an alternative fuzzy to ILS (Felder & Soloman, 2007).
The FSLSM (Felder & Silverman, 1988) considers four dimensions of learning
styles, wherein each dimension is dichotomous having two extremes, these are:
Active/Reflective, Sensing/Intuitive, Visual/Verbal, and Sequential/Global.
After, Felder and Soloman (2007) developed the ILS, related to FSLSM, to
determine the learning styles. The ILS has forty-four questions, while each
dimension of the FSLSM covers eleven questions. Each one has two possible
answers, in which each answer corresponds with one of the extremes of the
dimension. For example, in the Visual/Verbal "I prefer to read" meets the Visual
extreme and "I prefer to hear" for Verbal extreme.
With the ILS, Silva, Ponce & Villalpando (2013, 2014, 2014a), created the
MeLOTS method for the development of Learning Objects based on
instructional techniques and learning styles, where they relate thirty-one

36
instructional techniques to the dimensions of the FSLSM. This method is

evaluated with students and their opinions as key foundations for the
development of the FuzzyILS Method. In this respect, students indicated that
some answers or options of the questions should not be exclusive. They said that
the dichotomy cannot exist in some questions and that they wish that some of
their answers were intermediate (Silva, Ponce & Villalpando, 2015). The same
conception is also found by Cataldi, Figueroa, Méndez, Lage, Vigliecca and
Kraus (2006).
Taking the abovesaid into consideration, Silva (2020) shows a way to break this
dichotomy. The solution included the notion of intermediate answers what have
different degrees of belonging to the sets associated with the four dimensions,
wherein these degrees are calculated with fuzzy logic (Zadeh, 1965). In this
sense, the author proposed the fuzzy sets as an extension of the ILS dichotomous
sets which have been further developed and evaluated. Thus, unlike the original
ILS put forward by Felder and Soloman (2007), each question has five options.
The following is an example that is used to display how the options are
organised, established, and how the values are distributed for each answer.
Question 1, associated with the Active/Reflective dimension, “I understand

something better after:”
a. Always practice
b. Almost always practice, and rarely reflect on them
c. Sometimes practice, and sometimes reflect on them
d. Almost always reflect on them, and rarely practice
e. Always reflect on them
Each option is value with fuzzy sets shown in Figure 1. The values of each
answer using these fuzzy sets are displayed in Table 2.
Figure 1: Fuzzy sets used to establish the values of the FuzzyILS Questionnaire
answers
Table 2 shows the different values possible for the first answer in the FuzzyILS
Questionnaire.

37
Table 2: Values each answer using the fuzzy sets for the first question in the FuzzyILS
Questionnaire
Answer Active Reflective
a) Always practice 1 0
b) Almost always practice, and rarely reflect on them 0.75 0.25
c) Sometimes practice, and sometimes reflect on them 0.50 0.50
d) Almost always reflect on them, and rarely practice 0.25 0.75
e) Always reflect on them 0 1
With the value of each question, FuzzyILS Method adds all values for each
dimension. Table 3 shows the possible values for all questions in the
Active/Reflective dimension.
Table 3. Possible values for each question for the Active/Reflective dimension.
Questions Active Reflective
1 0.75 0.25
5 0.5 0.5
9 0.75 0.25
13 0.25 0.75
17 0.25 0.75
21 0.5 0.5
25 0.75 0.25
29 0.25 0.75
33 0.75 0.25
37 0.5 0.5
41 0.25 0.75
Total 5.5 5.5
After calculating the total value of each dimension, manipulating the degree of
belonging of each dimension is shown in the following figure.
Figure 2: Fuzzy sets to evaluate the dimensions of the FSLSM in the FuzzyILS Method

38
These fuzzy sets make Equation 1 for displaying the sets (Weak, Moderate, and
Strong) and the formulas for each interval in which X reflects the total value of
the dimension.
(1)
With the degree of belonging, the learning styles are clearly obtained. For
example, the value in table 3 (Active = 5.5 and Reflective = 5.5), corresponds
with the second interval in Equation 1, and the degree of belonging is 0.56 for
Active and reflective, and both values are in the Moderate set. Integrating the
FuzzyILS Method with MOODLE to associate the LMS activities to the learning
styles is the main concern in this research work. In any course development
platform, one can integrate it as a move to relate instructional techniques with
students’ learning styles (Silva, Ponce & Villalpando, 2014) in hope to offer
support to the teachers for effective and appropriate courses.
3. Methodology
This section discusses the objective of the study, the incorporation of the
FuzzyILS Method in MOODLE, the sample of the study, and the
experimentation with this sample.
3.1. Objective
As indicated at the beginning, the main objective of the study is to analyze the
effectiveness of a MOODLE course taking into consideration students' learning
styles being determined by the FuzzyILS Method. To validate its effectiveness,
the researcher evaluates students’ participation in the MOODLE activities by
considering grade they obtain in a content evaluation test. The idea that if a
student participates actively in the activities, he will learn better, as indicated by
other authors (Prince, 2013; Freeman et al., 2014) is the standing conception the
researcher used. On the other hand, the investigator based the evaluation of the
qualification on the fact that the effectiveness of a technique is related to the
expected result when performing tests that measure the student's level of
knowledge about the knowledge imparted (Lozano, et al., 2020). First, the
FuzzyILS Method in MOODLE is integrated, then the questionnaire is added
along with the fuzzy sets, values of degree of belonging, and the evaluation of
MOODLE activities associated with values of learning styles. All these steps
combine to validate the MOODLE course.
3.2. FuzzyILS Method in MOODLE

The incorporation phase of the FuzzyILS Method in MOODLE takes some stages
that are summarised as follows.

39
3.2.1. LSTest adaptation

As indicated, previous works were analysed to incorporate the FuzzyILS
Method into MOODLE. In the first instance, the learning styles module (LSTest)
was downloaded for MOODLE (Rubio, 2009), from the site
http://innova.cicei.com/course/view.php?id=24, which consist of the
programs, tables of the Database, and files to be incorporated to the LMS.
Subsequently, the elements of the Module were modified, considering other
work (Puello, Fernández & Cabarcas, 2014). Figure 3 shows the components of
the LSTest Module, where modifications made by Puello and colleagues are
reflected in light gray and the investigator’s modifications are displayed in dark
gray.
Figure 3: LSTest module (adaptation of Puello, Fernández & Cabarcas, 2014)
Puello, Fernández and Cabarcas (2014) did modifications which are described
below:
1. In the test folder -> examples, they added the ILS.
2. They modified the install.php and install.xml files to add the changes
and optimize the module installation with the ILS.
3. They added and modified several functions in the localib.php file, the file
where the working logic of the Test and the reports of the Module.
4. In the save.php file, they incorporated functions to store the results of the
ILS.
5. They modified the report.php file to present the results of the ILS on a
worksheet per student per course.
6. They also modified the stylestatistic.php, teststatistic.php, and
userstatistic.php files to present the results´ tables and the statistics
according to ILS.

40
3.2.2. Inclusion of FuzzyILS Questionnaire

The FuzzyILS Questionnaire is added in the test folder, wherein Figure 4 shows
the Test interface of MOODLE.
Figure 4: FuzzyILS Questionnaire interface
3.2.3. FuzzyILS Questionnaire in Portuguese.

The researcher incorporated the Portuguese into the MOODLE being developed
in the lang folder -> pt. Figure 5 shows the Test interface in Portuguese.
Figure 5: The interface of the FuzzyILS Questionnaire in Portuguese
3.2.4. Fuzzy Sets to Establish the Values of the FuzzyILS Answers

Actually, some functions are inserted and the localib.php file is modified to
incorporate the fuzzy sets as shown in Figure 1. These modifications were used
to modify the Questionnaire logic and Module reports.

41
3.2.5. Degrees of Belonging to Each Dimension

As an endeavour, the grad_pert.php file is included to develop the logic of
evaluation of the degrees of membership of the dimensions, according to
Equation 1 and Figure 2so that students’ learning styles could be get.
3.2.6. Evaluation of MOODLE Activities

The file eval_activ.php is a fixed, where the researcher incorporates the
adaptation of Table 1 and change the “yes” value by numeric valor 2, the “no”
value by numeric value 0, and the “shorter” value by 1. Table 4 shows the
adaption made.
Table 4: Valuation of MOODLE in each learning style

Activities Sens. Intui. Visual Verb. Sec. Glob. Act. Refl.
Forum 2 2 0 2 2 2 2 2
Chat 2 0 0 2 2 0 2 0
Glossary 2 0 0 2 2 0 2 2
Workshop 2 2 2 2 2 2 2 2
Survey 2 2 2 2 0 2 0 2
Choice 2 0 2 2 2 1 2 2
Lesson 2 1 2 2 2 1 2 2
Wiki 0 2 0 2 2 0 2 1
(Adaptation of Despotović et al., 2012)
With the valuation shown in Table 4 and the evaluation of the learning styles of
each student, then Equation 2 is formulated. For each technique, we need to
multiply the value with each student’s learning style, where:
(2)
1. ti is each MOODLE technique

2. σ is the dimension value in the technique i (Table 4)
3. µ is the evaluation of dimension in the students (described in 3.2.5 and
associated with Figure 2)
For example, the values of the dimensions for a student are as follows:
1. Sensing = 0.69; Intuitive = 0.44

2. Visual = 1; Verbal = 0
3. Sequential = 0.94; Global = 0.19
4. Active = 0.56; Reflective = 0.56
Thus, with Equation 2, it is possible to evaluate the Moodle activities. For

example, the evaluation of Forum activity is shown in Table 5.
Table 5: Evaluation of the Forum activity in each FSLSM dimensions.

Activities Sens. Intui. Visual Verb. Seq. Glob. Act. Refl. Tot.
Forum 2*0.69 2*0.44 0*1.00 2*0.00 2*0.94 2*0.19 2*0.56 2*0.56 8.07

42
3.3. The sample

A sample of forty students of an introductory computer course are used in this
research work by dividing them into two groups (experimental and control
group). Both groups took a mini Unified Modeling Language (UML) course
structured in MOODLE. The experimental group used the MOODLE with
FuzzyILS Method whereas the control group took the original MOODLE
without FuzzyILS Method. The intention is to validate that a MOODLE course
structured with techniques associated with the learning styles of the students is
an effective course, and the students get better results than a course with
techniques not associated with Learning styles.
3.4. The evaluation of learning styles of the experimental group

Table 6 shows the result of the responses of the twenty students of the
experimental group.
Table 6: Evaluation of learning styles dimensions using the FuzzyILS Questionnaire

Students Sens. Int. Visual Verb. Seq. Glob. Act. Refl.
1 6.5 4.5 10 1 8.5 2.5 5.5 5.5
2 2.5 8.5 8.5 2.5 10 1 4 7
3 9.5 1.5 9 2 5.5 5.5 2.5 8.5
4 3 8 9 2 6 5 3 8
5 3 8 3.5 7.5 8.5 2.5 3 8
6 7 4 6.5 4.5 6 5 7.5 3.5
7 3 8 2.5 8.5 6 5 6.5 4.5
8 10 1 8 3 8.5 2.5 8 3
9 9 2 4 7 6.5 4.5 2.5 8.5
10 3.5 7.5 4 7 4.5 6.5 6.5 4.5
11 5 6 3 8 4.5 6.5 3 8
12 9.5 1.5 6 5 8 3 3.5 7.5
13 9 2 3 8 9.5 1.5 4 7
14 9.5 1.5 9.5 1.5 2.5 8.5 5.5 5.5
15 3.5 7.5 9.5 1.5 7 4 6.5 4.5
16 7 4 5 6 3 8 9.5 1.5
17 5.5 5.5 3.5 7.5 5 6 2.5 8.5
18 4.5 6.5 5 6 4.5 6.5 3.5 7.5
19 10 1 5 6 9.5 1.5 10 1
20 5.5 5.5 6.5 4.5 5.5 5.5 10 1
After applying the FuzzyILS Questionnaire, it is high time to evaluate the

dimensions with the fuzzy sets to know each degree of belonging of each
dimension using Equation 1. Table 7 displays the evaluation of the learning style
of the experimental groups.

43
Table 7: Evaluation of learning styles using FuzzyILS sets

Students Sens. Int. Visual Verb. Seq. Glob. Act. Refl.
1 0.69 0.44 1.00 0.00 0.94 0.19 0.56 0.56
2 0.19 0.94 0.94 0.19 1.00 0.00 0.38 0.75
3 1.00 0.06 1.00 0.13 0.56 0.56 0.19 0.94
4 0.25 0.88 1.00 0.13 0.63 0.50 0.25 0.88
5 0.25 0.88 0.31 0.81 0.94 0.19 0.25 0.88
6 0.75 0.38 0.69 0.44 0.63 0.50 0.81 0.31
7 0.25 0.88 0.19 0.94 0.63 0.50 0.69 0.44
8 1.00 0.00 0.88 0.25 0.94 0.19 0.88 0.25
9 1.00 0.13 0.38 0.75 0.69 0.44 0.19 0.94
10 0.31 0.81 0.38 0.75 0.44 0.69 0.69 0.44
11 0.50 0.63 0.25 0.88 0.44 0.69 0.25 0.88
12 1.00 0.06 0.63 0.50 0.88 0.25 0.31 0.81
13 1.00 0.13 0.25 0.88 1.00 0.06 0.38 0.75
14 1.00 0.06 1.00 0.06 0.19 0.94 0.56 0.56
15 0.31 0.81 1.00 0.06 0.75 0.38 0.69 0.44
16 0.75 0.38 0.50 0.63 0.25 0.88 1.00 0.06
17 0.56 0.56 0.31 0.81 0.50 0.63 0.19 0.94
18 0.44 0.69 0.50 0.63 0.44 0.69 0.31 0.81
19 1.00 0.00 0.50 0.63 1.00 0.06 1.00 0.00
20 0.56 0.56 0.69 0.44 0.56 0.56 1.00 0.00
3.5. Evaluation of MOODLE Activities

After evaluating learning styles, valuing the MOODLE activities with Equation 2
is necessary Table 8 shows the evaluation of the MOODLE activities in the FSLM
dimensions for the first student. The MOODLE activities for all students are
evaluated and their values are summarised in Table 9.
Table 8: Evaluation of the MOODLE activities in the FSLSM dimensions for the first
student
Activities Sens. Intui. Visual Verb. Seq. Glob. Act. Refl. Tot.
Forum 2*0.69 2*0.44 0*1.00 2*0.00 2*0.94 2*0.19 2*0.56 2*0.56 6.76
Chat 2*0.69 0*0.44 0*1.00 2*0.00 2*0.94 0*0.19 2*0.56 0*0.56 4.38
Glossary 2*0.69 0*0.44 0*1.00 2*0.00 2*0.94 0*0.19 2*0.56 0*0.56 4.38
Workshop 2*0.69 2*0.44 2*1.00 2*0.00 2*0.94 2*0.19 2*0.56 2*0.56 8.76
Survey 2*0.69 2*0.44 2*1.00 2*0.00 0*0.94 2*0.19 2*0.56 2*0.56 6.88
Choice 2*0.69 0*0.44 2*1.00 2*0.00 2*0.94 1*0.19 2*0.56 0*0.56 6.57
Lesson 2*0.53 1*0.59 2*0.31 2*0.81 2*0.31 1*0.81 2*0.53 1*0.59 6.16
Wiki 0*0.53 2*0.59 0*0.31 2*0.81 2*0.31 0*0.81 0*0.53 2*0.59 4.60

44
Table 9: Evaluation of the MOODLE activities in the experimental group

Est. Forum Chat Glossary Workshop Survey Choice Lesson Wiki
1 6.76 4.38 4.38 8.76 6.88 6.57 6.16 4.60
2 6.88 3.50 3.50 8.75 6.13 6.88 6.81 5.75
3 6.88 3.75 4.88 8.88 6.50 8.19 8.25 2.81
4 7.00 2.50 3.50 9.00 6.50 6.75 7.50 4.63
5 8.38 4.50 4.88 9.00 7.88 7.06 7.19 6.63
6 7.63 5.25 6.25 9.00 6.00 7.75 8.00 4.81
7 8.63 5.00 6.00 9.00 7.25 6.75 7.50 6.69
8 7.00 6.13 6.50 8.75 5.25 8.56 7.81 4.38
9 8.25 5.25 6.13 9.00 7.88 8.31 8.19 4.44
10 8.25 4.38 5.75 9.00 6.88 6.69 7.75 5.81
11 8.50 4.13 5.50 9.00 8.00 7.06 7.94 5.25
12 7.63 5.38 5.88 8.88 7.00 8.50 7.94 4.31
13 8.38 6.50 6.63 8.88 7.63 8.56 7.75 5.50
14 6.75 3.63 5.50 8.75 5.63 7.69 8.50 2.31
15 6.88 3.63 4.38 8.88 5.50 6.88 7.31 5.06
16 7.88 5.25 7.00 8.88 5.88 7.25 8.25 4.56
17 8.38 4.13 5.38 9.00 8.00 7.25 7.94 5.06
18 8.00 3.63 5.00 9.00 7.38 6.94 7.88 4.94
19 7.38 7.25 7.38 8.38 5.38 8.31 7.38 5.25
20 7.38 5.13 6.25 8.75 5.38 7.06 7.63 5.13
Figure 6 shows the three best activities for the experimental group.
Figure 6: Evaluation of MOODLE activities according to the learning styles of the

experimental group
After the evaluation, there are many coincidences in the most appropriate
MOODLE activities. Figure 7 displays the intersections and numbers of students
in these intersections.

45
Figure 7: The intersection of MOODLE activities, evaluated in the experimental group
For the twenty students of the experimental group, the best MOODLE activities,
according to their learning styles, are: Lesson, Workshop, Choice and Survey; for
eight students are: Workshop, Choice, and Lesson; for seven Forum, Workshop
and Lesson; for three students Forum, Workshop and Choice and finally for two
students are: Forum, Workshop, and Survey.
3.6. Present test and the evaluation of participation

A face-to-face testis designed, along with analyzing students’ participation in
MOODLE activities. Table 10 reflects the obtained data.
Table 10: Participation of the students in the MOODLE activities and grade of the
present test of the experimental group
Participation in MOODLE Activities
Est. Forum Workshop Choice Lesson Participation Mean Grade
1 85% 100% 100% 100% 96% 19
2 60% 95% 100% 85% 85% 19
3 50% 100% 100% 90% 85% 19
4 25% 75% 100% 100% 75% 18
5 10% 0% 0% 50% 15% 8
6 25% 80% 100% 100% 76% 17
7 33% 100% 100% 75% 77% 18
8 50% 60% 100% 100% 78% 18
9 33% 100% 100% 100% 83% 19
10 50% 85% 0% 100% 59% 19
11 75% 100% 0% 75% 63% 18
12 0% 25% 0% 50% 19% 10
13 100% 80% 100% 100% 95% 19
14 100% 100% 100% 100% 100% 18
15 25% 100% 100% 75% 75% 19
16 50% 95% 0%% 100% 82% 17
17 75% 100% 100% 100% 94% 19
18 75% 80% 100% 100% 89% 18
19 75% 100% 100% 100% 94% 18
20 60% 75% 100% 75% 78% 16
Mean 52.8% 82.5% 75% 88.75% 75% 17.3
The MOODLE activity that most participants participate in is the Lesson, while
the Forum is the lea stone. The mean of participation is 75%. The mean of the
grade is 17.3, where the scale is 0-20. The results are shown in Table 11.

46
Table 11: Participation of students in MOODLE activities & grade of the control group
Participation in MOODLE Activities
Est. Forum Workshop Choice Lesson Activities Mean Grade
1 100% 100% 100% 75% 94% 15
2 0% 0% 0% 80% 20% 12
3 50% 85% 0% 75% 53% 18
4 0% 100% 0% 80% 45% 16
5 10% 80% 100% 70% 65% 14
6 25% 100% 0% 75% 50% 13
7 0% 75% 0% 50% 31% 15
8 20% 80% 100% 50% 63% 18
9 50% 80% 100% 100% 83% 18
10 75% 25% 100% 75% 69% 17
11 15% 75% 100% 100% 73% 20
12 0% 25% 100% 80% 51% 18
13 0% 75% 0% 50% 31% 13
14 0% 80% 0% 50% 33% 16
15 100% 50% 0% 100% 63% 17
16 75% 75% 100% 100% 88% 15
17 100% 75% 100% 100% 94% 17
18 50% 100% 100% 75% 81% 15
19 100% 50% 100% 80% 83% 18
20 25% 25% 100% 60% 53% 17
Mean 39.75% 68% 60% 76% 68% 16.1
Similar to the experimental group, in the control group, the MOODLE activity
with most participation was the Lesson, and the least was the Forum. The mean
of participation was 68%. The mean of the grade was 16.1, where the scale was 0-
20. Figure 8 shows the participation comparative of the two groups. Figure 9
shows the grade of the face-to-face test of the two groups.
Mean participation of the 4

principals MOODLE activities
100% 88.75%
90% 82.50%
76.00%
% of participation
80% 75.00%
68.00%
70% 60.00%
60% 52.80% Experimenta
50% 39.75% l group
40%
Control
30%
group
20%
10%
0%
Forum Workshop Choice Lesson
MOODLE activities
Figure 8: ComparingParticipants’ Participantion in Both Groups

47
Participation in all MOODLE activities was greater in the experimental group

than in the control group. Also, the mean grade of the face-to-face test was
greater in the experimental group than the control group, showed in Figure 9.
Figure 9: Grade of the face-to-face test of the two groups
4. Results
In the experimental group, the MOODLE activity with most participation was
the Lesson, and with the least was the Forum. The participation mean was 75%,
and the mean of the grade was 17.3/20. The correlation between the
participation of students and their grades, results in a strong positive correlation
of the variables (r=0.86), and the value of the coefficient of determination is (r2) =
0.75. This coefficient is the common variability between both variables. Figure 10
shows and explains the correlation.
Figure 10: Correlation between the students’ participation and their grade in the
experimental group
Similar to the experimental group, in the control group, the MOODLE activity
with most participation was the Lesson, and the least was the Forum. The
participation mean was 68%. The mean of the grade was 16.1/20. The correlation

48
between the student's participation and their grades was r = 0.41, which
indicates a weak positive correlation of the variables. The value of the coefficient
of determination (r2) = 0.17. This coefficient is the common variability between
both variables. Figure 11 summarises these data.
Figure 11: Correlation between the students’ participation and their grade in the
control group
5. Conclusion
In this research paper, the FuzzyILS Method is incorporated into MOODLE as a
starting point to the previous proposals of other researchers (Borga, 2009; Puello,
Fernandez & Cabarcas, 2014), who added the Learning styles Test into the LMS.
The MOODLE Database is modified unlike to the previous existing proposals
wherein new programs are created to assess degrees of membership in the fuzzy
sets and to evaluate MOODLE activities with the FuzzyILS Method. In the
experimentation, the Glossary, Chat, and Wiki activities were poorly evaluated
in all cases in the sense that they were not selected among the first four
MOODLE activities. Prince (2013), and Freeman and his colleagues (Freeman et
al., 2014) affirm that students learn and perform better if the learning
environment includes activities in which they participate and apply knowledge,
rather than being passive in the process and simply listening. The results of the
experimental group show evidence that when MOODLE is used, students use
appropriate techniques learning styles, participate more in the activities, and
obtain better grades. As far as the latter is concerned, students achieve good
grades owing to their understanding of the content delivered and the
participation they perform (Prince, 2013; Freeman et al., 2014). A strong positive
correlation and a great coefficient of determination are met in the experimental
group. Meanwhile, weak positive correlations along with students’ grades are
less with the control group. Hence, it is safe to mention that the intervention
brought interesting results and the hybridization process the present paper
suggests is workable for effective course and meeting urgent needs. Further
attempts need to cover large population in different contexts to collect evidences
to elaborate references on this issue.

49
6. References
Almeida, G., Romero, W., & Arce, A. (2017). MOODLE una contribución para elevar la
calidad del proceso de enseñanza-aprendizaje [MOODLE a contribution to raise
the quality of the teaching-learning process]. Retrieved from
http://www.eumed.net/rev/atlante/2017/10/Moodle-ensenanza-
aprendizaje.html
Artopoulos, A. (2020). COVID-19: ¿Qué hicieron los países para continuar con la
educación a distancia? [COVID-19: What did countries do to continue distance
education?]. Revista Latinoamericana de Educación Comparada, 11(17), 1-11.
Retrieved from http://www.saece.com.ar/relec/revistas/17/doc2.pdf
Bignetti, J., & Jasbinschek, C. (2011, August 30-September 02). Estilos de aprendizagem e
interfaces adaptativas em ambientes virtuais baseados na plataforma MOODLE
[Learning styles and adaptive interfaces in virtual environments based on the
MOODLE platform]. Conference proceedings of the 17º congresso internacional
de educação a distância. Manaus. Amazonas, Brasil. Retrieved from
http://www.abed.org.br/congresso2011/cd/250.pdf
Boneu, J. M. (2007). Plataformas abiertas de e-learning para el soporte de contenidos
educativos abiertos [Open e-learning platforms to support open educational
content]. Revista de Universidad y Sociedad del Conocimiento (RUSC), 4(1), 36-46.
http://dx.doi.org/10.7238/rusc.v4i1.298
Cabero-Almenara, J., Arancibia, M., & del Prete, A. (2019). Technical and Didactic
Knowledge of the Moodle LMS in Higher Education. Beyond Functional Use.
Journal of New Approaches in Educational Research (NAER Journal), 8(1), 25-33.
https://doi.org/10.7821/naer.2019.1.327
Cataldi, Z., Figueroa, N., Méndez, P., Lage, F., Vigliecca, M., & Kraus, G. (2006).
Herramienta automatizada para la determinación de los estilos en ingresantes cursos de
programaciónbásica de aprendizaje [Automated tool for determining styles in incoming
basic learning programming courses]. Conference proceedings of the XII Congreso
Argentino en Ciencias de la Computación. Retrieved from
http://sedici.unlp.edu.ar/handle/10915/22529
Despotović-Zrakić, M., Marković. A., Bogdanović, Z., Barać, D., & Krčo, S. (2012).
Providing adaptivity in MOODLE LMS Courses. Educational Technology &
Society, 15(1), 326-338. Retrieved from
www.jstor.org/stable/jeductechsoci.15.1.326
Díaz, S. (2009). Introducción a las Plataformas Virtuales en la Enseñanza [Introduction to
Virtual Platforms in Teaching]. Temas para la educación, 2, 1-7. Retrieved from
https://www.feandalucia.ccoo.es/docuipdf.aspx?d=4920&s=
Díaz-Barriga, F., & Hernández, G. (2010). Teaching strategies for meaningful learning (3rd
ed.). McGraw-HILL: México.
Dorfsmani, M. (2012). La profesión docente en contextos de cambio: El docente global en
la Sociedad de la Información [The teaching profession in contexts of change:
The global teacher in the Information Society]. RED – Docencia Universitaria en la
Sociedad del Conocimiento - Universidad de Murcia,6, 1-23. Retrieved from
http://www.um.es/ead/reddusc/6/marcelo_dusc6.pdf
Felder, R., & Silverman, L. (1988). Learning and Teaching Styles in Engineering
Education, Engr. Education, 78(7), 674-681. Retrieved from
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/LS-
1988.pdf.
Felder, R., & Soloman, B. (2007). Index of Learning styles [Blog post]. Retrieved from
http://www.engr.ncsu.edu/learningstyles/ilsweb.html.

50
Freeman, S., Eddya, S., McDonougha, M., Smithb, M., Okoroafora, N., Jordta, H., & Pat
Wenderotha, M. (2014). Active learning increases student performance in
science, engineering, and mathematics. Proceedings of the National Academy of
Sciences, 111(23), 8410 -8415. DOI: 10.1073/pnas.1319030111
Graf, S., & Kinshuk, K. (2007). Providing Adaptive Courses in Learning Management
Systems with Respect to Learning styles. In T. Bastiaens & S. Carliner (Eds.),
Proceedings of the World Conference on E-Learning in Corporate, Government,
Healthcare, and Higher Education. Quebec City, Canada. Retrieved from
https://www.learntechlib.org/primary/p/26739/
Graf, S., Kinshuk, K., & Liu, T. (2008). Identifying Learning Styles in Learning Management
Systems by Using Indications from Students´ Behaviour. Paper presented at the 2008
eighth IEEE international conference on advanced learning technologies.
https://doi.org/10.1109/ICALT.2008.84
Karagiannis, I., & Satratzemi, M. (2018). An adaptive mechanism for Moodle based on
automatic detection of learning styles. Education and Information Technologies, 23,
1331–1357. https://doi.org/10.1007/s10639-017-9663-5
Keefe, J. W. (1987). Learning Style Theory and Practice. National Association of Secondary
School Principals. Reston, VA, USA.
Lavigne, G., Díaz, K., McAnally, L., & Organista, J. (2013). Navegar y aprender: una
aproximación a las relaciones entre estilos de aprendizaje y la navegación en
MOODLE [Navigating and learning: an approach to the relationships between
learning styles and navigation in MOODLE]. Universities and Knowledge Society
Journal, 10(2), 81-97. http://dx.doi.org/10.7238/rusc.v10i2.1402
Lozano, S., Suescún, E., Vallejo, P., Mazo, R., & Correa D. (2020). Comparando dos
estrategias de aprendizaje activo para enseñar Scrum en un curso introductorio
de ingeniería de software [Comparing two active learning strategies for teaching
Scrum in an introductory software engineering course]. Ingeniare, Revista Chilena
de Ingeniería, 28(1), 83-94. http://dx.doi.org/10.4067/S0718-33052020000100083
Mohd, F., Yahya, W. F. F., Ismail, S., Jalil, M. A., & Noor, N. M. M. (2019). An
Architecture of Decision Support System for Visual-Auditory-Kinesthetic (VAK)
Learning styles Detection Through Behavioral Modelling. International Journal of
Innovation in Enterprise System, 3(2), 24-30.
http://dx.doi.org/10.25124/ijies.v3i02.37
Moodle. (2020). Retrieved from https://docs.Moodle.org/38/en/Activities
Ossandón, Y., & Castillo, P. (2006). Propuesta para el Diseño de Objetos de Aprendizaje
[Proposal for the Design of Learning Objects]. Revista Facultad de Ingeniería.
Universidad del Tarapacá, 14(1), 36-48. Retrieved from
http://www.scielo.cl/scielo.php?pid=S0718-
13372006000100005&script=sci_arttext
Pacheco, J., Miranda, Y., & Alonso, R. (2020, March 16-20). Adaptación de un aula virtual en
Moodle a los estilos de aprendizaje [Adaptation of a virtual classroom in Moodle to
learning styles]. Paper presented at the XVIII Congreso Internacional de
Informática en la Educación INFOREDU, La Habana, Cuba. Retrieved from
http://www.informaticahabana.cu/sites/default/files/ponencia-
2020/EDU077.pdf
Paredes, P. (2008). A Proposal for Incorporating Learning styles into User Models in Adaptive
Teaching Systems (Doctoral dissertation, the Universidad Autónoma de Madrid.
Departamento de Ingeniería Informática, Madrid, España). Retrieved from
http://arantxa.ii.uam.es/~pparedes/tesis.pdf
Peña Estrella, N. F., Mendoza Moreno, M. A., & González Serrano, C. (2015). Diagnóstico
de estilos de aprendizaje para favorecer la personalización de materiales

51
educativos mediante redes bayesianas: servicios web para MOODLE [Diagnosis

of learning styles to favor the personalization of educational materials through
Bayesian networks: web services for MOODLE]. Revista Virtual Universidad
Católica del Norte,46, 4-20. Retrieved from
https://revistavirtual.ucn.edu.co/index.php/RevistaUCN/article/view/694
Prince, M. (2013). Does active learning work? A review of the research. Journal of
engineering education,93(3), 223-231. https://doi.org/10.1002/j.2168-
9830.2004.tb00809.x
Puello, P., Fernández, D., & Cabarcas, A. (2014). Herramienta para la detección de estilos
de aprendizaje en estudiantes utilizando la plataforma MOODLE [Tool for
detecting learning styles in students using the MOODLE platform]. Formación
Universitaria, 7(4), 15-24. http://dx.doi.org/10.4067/S0718-50062014000400003
Rubio, R. B. (2009). Nuevo módulo de test de estilos de aprendizaje [New learning styles
test module]. MOODLE Forum. Retrieved from
https://Moodle.org/mod/forum/discuss.php?d=140052
Szirmai, M. (2020). Moodle: The Ubiquitous Teacher. Electronic Journal of Foreign
Language Teaching, 17(1), 190–204. https://doi.org/10.1007/978-1-4020-9732-4_12
Silva, S. A., Ponce, G. J., & Meneses, H. J. (2013). Gestionando el Conocimiento en
MOODLE [Managing Knowledge in MOODLE]. Proceedings of the VIII
Conferencia Latino americana de Objetos de Aprendizaje y Tecnologías para el
Aprendizaje (LACLO2013) (pp. 115-126). Valdivia, Chile.
Silva, S. A., Ponce, G. J., & Villalpando, C. M. (2013). Hacia un Método Recomendador de
Técnicas Instruccionales para el Desarrollo de Objetos de Aprendizaje [Towards
a Recommending Method of Instructional Techniques for the Development of
Learning Objects]. Proceedings of the VIII Conferencia Latinoamericana de
Objetos de Aprendizaje y Tecnologías para el Aprendizaje (LACLO2013) (pp.
127-138). Valdivia, Chile.
Silva, S. A., Ponce, G. J., & Villalpando, C. M. (2014). Development Model of Learning
Objects Based on the Instructional Techniques Recommendation. International
Journal of Learning. Teaching and Educational Research, 4(1), 27-35. Retrieved from
http://www.ijlter.org/index.php/ijlter/article/view/42/pdf
Silva, S. A., Ponce, G. J., & Villalpando, C. M. (2014a). Sistema Recomendador de
Técnicas Instruccionales, Basado en Objetivos Pedagógicos – ReTIBO
[Recommend System of Instructional Techniques, Based on Pedagogical
Objectives - ReTIBO]. Revistavenezolana de educación (EDUCERE), 60, 281-287.
Retrieved from
http://www.saber.ula.ve/bitstream/123456789/38892/1/articulo9.pdf
Silva, S. A., & Ponce, G. J. (2015). Evaluación del Método para el Desarrollo de Objetos
de Aprendizaje. basado en Estilos de Aprendizaje: MeLOTS [Evaluation of the
Method for the Development of Learning Objects. Based on Learning styles:
MeLOTS]. Revista Tecnológica ESPOL – RTE. Ecuador, 28(5), 39-53. Retrieved from
http://rte.espol.edu.ec/index.php/tecnologica/article/view/424/290
Silva, S. A., & Vicari, R. (2016). New Moodle Blocks for Knowledge Management. In F.
Koch, A. Koster & T. Primo (Eds), Social Computing in Digital Education.
SOCIALEDU 2015. Communications in Computer and Information Science (pp. 104-
123). Springer, Cham. https://doi.org/10.1007/978-3-319-39672-9_8
Silva, S. A. (2018). Conceptualización de los Modelos de Estilos de Aprendizaje
[Conceptualization of Learning Style Models]. Journal of Learning styles, 11(21), 1-
33. Retrieved from http://revistaestilosdeaprendizaje.com/article/view/1088
Silva, S. A. (2020). Towards a Fuzzy Questionnaire of Felder and Soloman for
determining learning styles without dichotomic in the answers. Journal of

52
Learning styles, 13(15), 146-166. Retrieved from

http://revistaestilosdeaprendizaje.com/article/view/1853
Torres, S., Molina, B., & Álvarez, D. (2010). Integración de los Estilos de Aprendizaje en
Moodle. para el Proyecto Guadalinfo [Integration of Learning styles in Moodle. for the
Guadalinfo Project]. Proceedings of the I Congreso de Calidad de la Formación
Virtual CAFVIR (pp. 413-418). Alcalá de Henares, España.
Zadeh, L. A. (1965). Fuzzy Sets. Information and control, 8, 338-353.
Appendix
FuzzyILS Questionnaire
Question 1, associated with the Active/Reflective dimension, “I understand something
better after:”
a. Always practice
b. Almost always practice, and rarely reflect on them
c. Sometimes practice, and sometimes reflect on them
d. Almost always reflect on them, and rarely practice
e. Always reflect on them
Question 2, associated with the Sensing/Intuitive dimension, "I consider myself:"
a. Always realistic
b. Almost always realistic, and rarely innovative
c. Sometimes realistic, and sometimes innovative
d. Almost always innovative, and rarely realistic
e. Always innovative
Question 3, associated with the Visual/Verbal dimension, "When I think about what I
did yesterday, I am more likely to do it based on:"
a. Always an image
b. Almost always an image, and rarely words
c. Sometimes an image, and sometimes words
d. Almost always words, and rarely an image
e. Always words
Question 4, associated with the Sequential/Global dimension, "I tend to:"
a. Always understand the details of a topic, but not see its complete structure
b. Almost always understand the details of a topic, but not see its complete
structure, and rarely understand the complete structure, but not see the details
c. Sometimes understanding the details of a topic, but not seeing its complete
structure, and other times understanding the complete structure, but not seeing
the details
d. Almost always understand the complete structure, but not see the details, and
rarely understand the details of a topic, but not see its complete structure
e. Always understand the complete structure, but not see the details.
Question 5, associated with the Active/Reflective dimension, "When I am learning
something new, it helps me:"
a. Always talk about it
b. Almost always talk about it, and rarely think about it
c. Sometimes talk about it, and sometimes think about it
d. Almost always think about it, and rarely talk about it
e. Always think about it
Question 6, associated with the Sensing/Intuitive dimension, "If I were a teacher, I
would prefer to give a course:"
a. Whenever it deals with facts and real-life situations
b. Almost always when dealing with real events and situations in life, and rarely
dealing with ideas and theories

53
c. Sometimes that deals with facts and real situations of life, and other times that
deals with ideas and theories
d. Almost whenever he deals with ideas and theories, and rarely deals with facts
and real-life situations
e. Whenever dealing with ideas and theories
Question 7, associated with the Visual/Verbal dimension, "I prefer to obtain new
information:"
a. Always of images, diagrams, graphs or maps
b. Almost always of images, diagrams, graphs, or maps, and rarely of written
instructions or verbal information
c. Sometimes of images, diagrams, graphs, or maps, and sometimes of written
instructions or verbal information
d. Almost always written instructions or verbal information, and rare images,
diagrams, graphs, or maps
e. Always written instructions or verbal information
Question 8, associated with the Sequential/Global dimension, "Once I understand:"
a. All parties, I always understand the total
b. All parties, I almost always understand the total, and when I understand the
total of something, I rarely understand how their parts fit together
c. All parties, sometimes I understand the total, and when I understand the total of
something, sometimes I understand how their parts fit together
d. The total of something, I almost always understand how its parts fit together,
and when I understand all the parts, I rarely understand the total
e. The total of something, I always understand how its parts fit together
Question 9, associated with the Active/Reflective dimension, "In a study group that
works with a difficult material, it is more likely that:"
a. Always participate and contribute ideas
b. Almost always participate and contribute ideas, and rarely do not participate
and just listen
c. Sometimes participate and contribute ideas, and sometimes do not participate
and just listen
d. Almost always do not participate and just listen, and rarely participate and
contribute ideas
e. Never participate and just listen
Question 10, associated with the Sensing/Intuitive dimension, "It's easier for me:"
a. Always learn facts
b. Almost always learn facts, and rarely learn concepts
c. Sometimes learn facts, and sometimes learn concepts
d. Almost always learn concepts, and rarely learn facts
e. Always learn concepts
Question 11, associated with the Visual/Verbal dimension, "In a book with many images
and graphics it is more likely that:"
a. Always check images and graphics carefully
b. I almost always carefully check the images and the graphics and rarely focus on
the written text
c. Sometimes I carefully review the images and the graphics and other times I
focus on the written text
d. I almost always concentrate on the written text, and rarely check the images and
graphics carefully
e. Always concentrate on the written text
Question 12, associated with the Sequential/Global dimension, "When I solve math
problems:"
a. I always work on solutions one step at a time

54
b. I almost always work on solutions one step at a time and rarely know what the
solutions are, but then I have difficulty imagining the steps to reach them
c. Sometimes I work on the solutions one step at a time and other times I know
what the solutions are, but then I have difficulty imagining the steps to reach
them
d. I almost always know what the solutions are, but then I have difficulty
imagining the steps to reach them and I rarely work on the solutions one step at
a time
e. I always know what the solutions are, but then I have difficulty imagining the
steps to reach them
Question 13, associated with the Active/Reflective dimension, “In the classes, I have
attended:”
a. I have usually come to know how many of the students are
b. Many times I have come to know how many of the students are
c. Sometimes I have come to know how many of the students are
d. I have rarely come to know how many of the students are
e. I have rarely come to know how many students are
Question 14, associated with the Sensing/Intuitive dimension, “When I read non-fiction
topics, I prefer:”
a. Always something that teaches me new facts or tells me how to do something
b. Almost always something that teaches me new facts or tells me how to do
something and rarely something that gives me new ideas to think about
c. Sometimes something that teaches me new facts or tells me how to do
something and sometimes something that gives me new ideas to think about
d. Almost always something that gives me new ideas to think about, and rarely
something that teaches me new facts or tells me how to do something
e. Always something that gives me new ideas to think about
Question 15, associated with the Visual/Verbal dimension, “I like teachers who:”
a. Always use many schemes on the board
b. They almost always use many schemes on the board, and rarely take much time
to explain
c. Sometimes they use many schemes on the board, and sometimes they take a
long time to explain
d. They almost always take a long time to explain, and rarely use many schemes on
the board
e. They always take a long time to explain
Question 16, associated with the Sequential/Global dimension, “When I am analyzing a
story or a novel:”
a. I always think about the incidents and try to accommodate them to configure the
issues
b. I almost always think about the incidents and try to accommodate them to
configure the issues, and rarely do I realize what the issues are when I finish
reading, and then I have to go back and find the incidents that show them
c. Sometimes I think about the incidents and try to accommodate them to
configure the topics, and other times I realize what the issues are when I finish
reading and then I have to go back and find the incidents that show them
d. I almost always realize what the issues are when I finish reading and then I have
to go back and find the incidents that demonstrate them, and I rarely think
about the incidents and try to accommodate them to configure the topics
e. I realize what the issues are when I finish reading and then I have to go back and
find the incidents that show them
Question 17, associated with the Active/Reflective dimension, “When I start solving a
task problem, it is more likely that:”

55
a. Always start working on your solution immediately

b. Almost always start working on your solution immediately, and rarely try first
to fully understand the problem
c. Sometimes start working on your solution immediately, and other times try first
to fully understand the problem
d. Almost always try first to fully understand the problem, and rarely start
working on your solution immediately
e. Always try to fully understand the problem first
Question 18, associated with the Sensing/Intuitive dimension, “I prefer the idea of:”
a. Always certainty
b. Almost always a certainty, and rarely the theory
c. Sometimes certainty, and sometimes the theory
d. Almost always the theory, and rarely certainty
e. Always the theory
Question 19, associated with the Visual/Verbal dimension, "I remember better:"
a. Always what I see
b. Almost always what I see and rarely what I hear
c. Sometimes what I see and sometimes what I hear
d. Almost always what I hear and rarely what I see
e. Always what I hear
Question 20, associated with the Sequential/Global dimension, "It is more important to
me than a teacher:"
a. Always expose the material in clear sequential steps
b. I almost always expose the material in clear sequential steps, and rarely give me
an overview and relate the material to other topics
c. Sometimes exposing the material in clear sequential steps, and sometimes give
me an overview and relate the material to other topics
d. Almost always give me an overview and relate the material to other topics, and
rarely expose the material in clear sequential steps
e. Always give me an overview and relate the material to other topics
Question 21, associated with the Active/Reflective dimension, “I prefer to study:”
a. Always in a study group
b. Almost always in a study group, and rarely alone
c. Sometimes in a study group, and sometimes only
d. Almost always alone, and rarely in a study group
e. Always alone
Question 22, associated with the Sensing/Intuitive dimension, "I consider myself:"
a. Always careful in the details of my work
b. Almost always careful in the details of my work and rarely creative in the way I
do my work
c. Sometimes careful in the details of my work and sometimes creative in the way I
do my work
d. Almost always creative in the way I do my work and rarely careful in the details
of my work
e. Creative in the way I do my job
Question 23, associated with the Visual/Verbal dimension, “When someone gives me
directions to new places, I prefer:”
a. Always a map
b. Almost always a map and rarely written instructions
c. Sometimes a map and sometimes written instructions
d. Almost times written instructions, and rarely a map
e. Always written instructions
Question 24, associated with the Sequential/Global dimension, “I learn:”

56
a. Always at a constant pace. If I study hard I get what I want

b. Almost always at a constant pace. If I study hard I get what I want, and rarely in
the beginning and pauses. I get confused and suddenly I understand
c. Sometimes at a constant pace. If I study hard I get what I want, and other times
in the beginning and pauses. I get confused and suddenly I understand
d. Almost always in the beginning and pauses. I get confused and suddenly I
understand, and rarely at a constant pace. If I study hard I get what I want
e. Always at the beginning and pauses. I get confused and suddenly I understand
Question 25, associated with the Active/Reflective dimension, “I prefer first:”
a. Always do something and see what happens
b. I almost always do something and see what happens, and rarely think about
how I am going to do something
c. Sometimes do something and see what happens, and sometimes think about
how I am going to do something
d. I almost always think about how I am going to do something, and rarely do
something and see what happens
e. Always think about how I'm going to do something
Question 26, associated with the Sensing/Intuitive dimension, "When I read for fun, I
like writers who:"
a. They always say clearly what they want to imply
b. They almost always say clearly what they want to imply, and rarely say things
creatively and interestingly
c. Sometimes they say clearly what they want to imply, and other times they say
things creatively and interestingly
d. They almost always say things creatively and interestingly, and rarely say
clearly what they want to imply
e. They always say things creatively and interestingly
Question 27, associated with the Visual/Verbal dimension, "When I see a scheme or
outline in class, I am more likely to remember:"
a. Always the image
b. Almost always the image and rarely what the professor said about it
c. Sometimes the image and sometimes what the professor said about it
d. Almost always what the professor said about her, and rarely the image
e. Always what the teacher said about her
Question 28, associated with the Sequential/Global dimension, “When I face an
information body:”
a. I always concentrate on the details and lose sight of the total
b. I almost always concentrate on the details and lose sight of the total, and rarely
try to understand the whole before going to the details
c. Sometimes I focus on the details and lose sight of the total, and other times I try
to understand the whole before going to the details
d. I almost always try to understand the whole before going to the details, and
rarely do I focus on the details and lose sight of the total of it
e. I always try to understand the whole before going to the details
Question 29, associated with the Active/Reflective dimension, "I remember more easily:"
a. Always something I have done
b. Almost always something I've done, and rarely something I've thought about a
lot
c. Sometimes something I've done, and sometimes something I've thought about a
lot
d. Almost always something I've thought about a lot, and rarely something I've
done
e. Always something I've thought about a lot

57
Question 30, associated with the Sensing/Intuitive dimension, “When I have to do a job,
I prefer:”
a. Always master a way of doing it
b. Almost always master a way of doing it, and rarely try new ways to do it
c. Sometimes master a way of doing it, and sometimes try new ways to do it
d. Almost always try new ways to do it, and rarely master a way to do it
e. Always try new ways to do it
Question 31, associated with the Visual/Verbal dimension, “When someone teaches me
data, I prefer:”
a. Always graphics
b. Almost always graphics, and rarely summaries with text
c. Sometimes graphics, and sometimes summaries with text
d. Almost always summaries with text, and rarely graphics
e. Always summaries with text
Question 32, associated with the Sequential/Global dimension, “When I write a paper, I
am more likely to do it (think or write):”
a. Always from the beginning and advance
b. Almost always from the beginning and advance, and rarely in different parts
and then order them
c. Sometimes from the beginning and advance, and sometimes in different parts
and then order them
d. Almost always in different parts and then order them, and rarely from the
beginning and advance
e. Always in different parts and then order them
Question 33, associated with the Active/Reflective dimension, “When I have to work on
a group project, I first want to do:”
a. Always a “brainstorm” where everyone contributes ideas
b. Almost always a "brainstorm" where everyone contributes ideas, and rarely the
"brainstorm" personally and then join the group to compare ideas
c. Sometimes a "brainstorm" where everyone contributes ideas, and sometimes the
"brainstorm" personally and then join the group to compare ideas
d. Almost always the "brainstorm" personally and then join the group to compare
ideas, and rarely a "brainstorm" where everyone contributes ideas
e. Always "brainstorm" personally and then join the group to compare the ideas
Question 34, associated with the Sensing/Intuitive dimension, "I consider:"
a. Whenever it is better to praise to call someone Sensing
b. Almost always it is better to praise calling someone Sensing, and rarely is it
better to call it imaginative
c. Sometimes it is better to praise to call someone Sensing, and other times to call it
imaginative
d. Almost always it is better to praise calling someone imaginative, and rarely
calling him Sensing
e. Whenever it is better to praise to call someone imaginative
Question 35, associated with the Visual/Verbal dimension, "When I meet people at a
party, I am more likely to remember:"
a. Always their appearance
b. Almost always their appearance, and rarely what they say about themselves
c. Sometimes what their appearance looks like, and sometimes what they say
about themselves
d. Almost always what they say about themselves, and rarely how is their
appearance
e. Always what they say about themselves

58
Question 36, associated with the Sequential/Global dimension, “When I am learning a

subject, I prefer:”
a. Always stay focused on that topic, learning as much as I can from him
b. I almost always stay focused on that topic, learning as much as I can from it, and
rarely make connections between that topic and related topics
c. Sometimes I stay focused on that topic, learning as much as I can from it, and
other times make connections between that topic and related topics
d. I almost always make connections between that topic and related topics, and
rarely stay focused on that topic, learning as much as I can from it
e. Always make connections between that topic and related topics
Question 37, associated with the Active/Reflective dimension, “I consider myself:”
a. Always open
b. Almost always open, and rarely reserved
c. Sometimes open, and sometimes reserved
d. Almost always reserved, and rarely open
e. Always reserved
Question 38, associated with the Sensing/Intuitive dimension, “I prefer courses that give
more importance to:”
a. Always concrete material (facts, data)
b. Almost always concrete material (facts, data), and rarely abstract material
(concepts, theories)
c. Sometimes concrete material (facts, data), and sometimes abstract material
(concepts, theories)
d. Almost always abstract material (concepts, theories), and rarely concrete
material (facts, data)
e. Always abstract material (concepts, theories)
Question 39, associated with the Visual/Verbal dimension, "To have fun, I prefer:"
a. Always watch television
b. Almost always watch television, and rarely read a book
c. Sometimes watching television, and sometimes reading a book
d. Almost always read a book, and rarely watch television
e. Always read a book
Question 40, associated with the Sequential/Global dimension, “Some teachers start
their classes by outlining what they will teach. Those sketches are: ”
a. Always something useful for me
b. Almost always something useful, and rarely very useful for me
c. Sometimes something useful, and sometimes very useful for me
d. Almost always very useful, and rarely useful for me
e. Always very useful for me
Question 41, associated with the Active/Reflective dimension, “The idea of doing a
group task with only one qualification for all:”
a. It always seems good to me
b. It almost always seems good to me, and rarely
c. Sometimes it seems good to me, and sometimes it doesn't
d. It almost never seems good to me and rarely does it
e. It never seems good to me
Question 42, associated with the Sensing/Intuitive dimension, "When I do large
calculations:"
a. I always tend to repeat all my steps and check carefully
b. I almost always tend to repeat all my steps and carefully review my work, and
rarely do I get tired of doing their review and I have to make an effort to do it
c. Sometimes I tend to repeat all my steps and carefully review my work, and other
times I get tired of doing its review and I have to make an effort to do it

59
d. I almost always tend to repeat all my steps and carefully review my work, and
rarely do I tend to repeat all my steps and carefully review my work
e. I always get tired of doing his review and I have to make an effort to do it
Question 43, associated with the Visual/Verbal dimension, "I tend to remember places
where I have been:"
a. Always easily and quite accurately
b. Almost always easily and quite accurately, and rarely with difficulty and
without much detail
c. Sometimes easily and quite accurately, and sometimes with difficulty and
without much detail
d. Almost always easily and fairly accurately, and rarely easily and fairly
accurately
e. Always with difficulty and without much detail
Question 44, associated with the Sequential/Global dimension, “When I solve problems
in a group, it is more likely that I:”
a. Always think of the steps to solve problems
b. Almost always think about the steps for solving problems, and rarely think
about the possible consequences or applications of the solution in a wide range
of fields
c. Sometimes think about the steps to solve the problems, and sometimes think
about the possible consequences or applications of the solution in a wide range
of fields
d. Almost always think about the possible consequences or applications of the
solution in a wide range of fields, and rarely think about the steps to solve the
problems
e. Always think about the possible consequences or applications of the solution in
a wide range of fields

60

Vol. 19, No. 10, pp. 60-76, October 2020
Supporting Natural Science Pre-Service Teachers

during Work-Integrated Learning: A Case of a
Lesson Study Approach
Wiets Botes, Boitumelo Moreeng and Moeketsi Mosia

Sol Plaatje University, Kimberley, South Africa
https://orcid.org/0000-0002-3024-8969
https://orcid.org/0000-0002-3644-3140
https://orcid.org/0000-0002-7189-0018
Abstract. This paper is concerned with the issue associated with final-
year Natural Science pre-service teachers’ lesson design abilities. This
paper reports how the implementation of a lesson study approach served
as an intervention to support the lesson preparation abilities of pre-
service teachers during a work-integrated learning (WIL) process. The
case study generated rich qualitative data from multiple sources such as
e-mail communications, WhatsApp responses, video capturing, and
focus group team discussions. Couched within the critical emancipatory
theory, the study found that the planning and preparation of innovative
and inclusive Natural Science lessons are inseparably associated with the
collaborative practices and interaction that existed among the team of pre-
service teachers. The accommodative and engaging nature of the lesson
study sessions empowered pre-service teachers to share their diverse
subjective experiences and contextual ideas, which were pivotal to the
generation of knowledge and design of Natural Science lessons.
Consequently, the lesson study approach not only affected the pre-service
teachers’ lesson design abilities but also better positioned them for the
WIL experience.
Keywords: Lesson study approach; critical emancipatory theory; work-

integrated learning; Natural Science education; pre-service teacher
development
1. Introduction and background to the study

The work-integrated learning experience (WIL) is an experiential and practical
learning process, whereby pre-service teachers partake in an authentic teaching
process that enables them to learn in and from their own teaching practice
(Johnston, 2011). This approach allows pre-service teachers to gain a hands-on
experience of the teaching profession (Barends & Nel, 2017). During this period,

61
pre-service teachers are placed at schools and teacher educators are tasked to
evaluate and assess the quality of their lesson presentations (DHET, 2015).
Serving as a link between academic theory and the practice of teaching (De Beer,
2017), the WIL process provides a sense of whether pre-service teachers acquired
the deemed teacher competencies required for the profession (Tran &
Soejatminah, 2016). The rationale for WIL is thus to empower pre-service teachers
to engage in a real-world teaching context with the hope to conscientise them
about the practice of teaching (Jovanovic, Fane & Andrew, 2018). This is done to
instil confidence in their ability to transfer their educational knowledge developed
as part of their initial teacher education into the practice of teaching, hence
promoting workplace readiness (Matoti, Junqueira & Odora, 2011).
While some consider the WIL process a testing ground for pre-service teachers to
demonstrate their teacher qualities (Ngibe, Pylman, Mammen & Adu, 2019),
others label the WIL process as a gruelling experience (De Beer, 2017). In the
context of teacher education, researchers often refer to the WIL process as the
“elephant in the room” because of various issues that manifest from the WIL
process (Pham, Bao, Saito & Chowdhury, 2018). The literature is scattered with
issues associated with the WIL process, that predominantly stems from the pre-
service teacher voice (Choy & Delahaye, 2011; Ngibe et al., 2019). To be specific, a
study by Matoti, Junqueira and Odora (2011) reported how pre-service teachers
were subject to emotional stress, frustration and anxiety because of issues
surrounding lesson preparation for the authentic teaching experience. It was this
problem that sparked the researchers’ interest in this study.
This study concerns itself with the issue associated with Natural Science pre-
service teachers’ competence in terms of Natural Science lesson preparation.
Couched within the critical emancipatory theory, the aim of the study is to
describe how a lesson study approach supported the lesson preparation abilities
of pre-service teachers during a work-integrated learning experience. To achieve
this aim, the study follows a qualitative case study research approach to respond
to a couple of pertinent research questions. These include trying to understand
“whether the Natural Science pre-service teachers are experiencing issues
pertaining to lesson preparation?” Secondly, “how can a lesson study approach
serve to support the Natural Science pre-service teachers’ development of Natural
Science lessons for an authentic teaching experience?” The response to these
research questions will not only be key in honing their lesson preparation skills
but would ultimately put them in a good stead for their professional teaching
career.
1.1 Envisioning a lesson study approach in a school of education

The lesson study approach has its roots in Japanese education from the mid-19th
century (Pham, Bao, Saito & Chowdhury, 2018) but has since been adopted in
other parts of the world to promote pre-service teacher development (Bjuland &
Mosvold, 2015; Fernandez & Zilliox, 2011). The lesson study approach is regarded
as an intervention that enables teachers to formulate teaching goals collectively,

62
prepare lessons and reflect back on past teaching experiences to improve future
teaching practices (Larkin, 2017; Sims & Walsh, 2009).
In the context of Natural Science teacher education, the ‘lesson preparation’

would serve as a means for Natural Science pre-service teachers to brainstorm and
share lesson ideas that would enable them to set the tone for the lesson
structuring. The next step involved in the lesson study process deals with the
‘actual planning’ of the Natural Science lesson with a focus to make the lesson
innovative and inclusive of nature to cater for the diverse learning needs of
learners.
The third and final aspect involved in the lesson study approach includes ‘lesson
reflection’ (Chizhik et al., 2017). Lesson reflection deals with how the pre-service
teacher enacted the Natural Science lesson during a WIL process. At the core of
the lesson reflection process lies the opportunity for pre-service teachers to share
their “rich” subjective experiences of “what worked” and “what did not work” in
terms of their Natural Science lesson presentations (Runesson, Martin & Clerc-
Georgy, 2015). Akerson et al. (2017) argue that this process allows individuals to
learn from each other’s teaching practices and experiences. The
interconnectedness of these three aspects is vital to the effectiveness of the lesson
study approach.
2. Aligning critical emancipatory theory with the lesson study approach

Osanloo and Grant (2014) suggest that the selection of a theoretical framework for
a study requires one to first confront your own individualised philosophical
orientations about reality. Lysaght (2011) takes it a step further by arguing that
the researcher’s choice of a theoretical framework is not arbitrary but reflects
personal beliefs and understanding about the ontological and epistemological
nature of knowledge within a study. This aligns well with the view of Shaull
(1974), who decades ago suggested that the theoretical and philosophical
orientations are inseparable from one another. Therefore, in an attempt to select a
theoretical framework for this study, we first had to confront our own, personal
beliefs and understanding about the nature of knowledge and reality as it existed
within this study.
Drawing on the views of Osanloo and Grant (2014) and Lysaght (2011), the
selection of a theoretical framework for this study rests on the following personal
beliefs and understandings. First of all, we share an awareness of the pre-service
teachers’ challenge associated with Natural Science lesson preparation and
acknowledge that this challenge inputs negatively on their WIL experience.
Secondly, we are of the view that the lesson study approach could serve as a driver
to overcome this particular challenge that they are confronted with. Thirdly, upon
implementation of the lesson study approach, we are confident that the pre-
service teachers themselves are equipped with the knowledge to support each
other in the preparation of Natural Science lessons because of their diverse
teaching experiences stemming from diverse teaching contexts. Finally, we are
confident that their ability to support each other in preparing Natural Science

63
lessons collectively would better position them in their quest to teach within an
authentic teaching space.
In light of these assumptions, which are linked to aspects of criticality, analyticity

and education, critical emancipatory theory fits well as a theoretical framework
for the study. Critical emancipatory theory, which emanates from critical theory
(Creswell & Creswell, 2017) is concerned with research that emancipates the
disempowered, cultivate hope and stimulate lifelong learning opportunities for
individuals (Gordon, 1986; Shor, 1988). Drawing on seminal work of scholars such
as Paulo Freire, Ira Shor and Cynthia Brown critical emancipatory theory serves
as a paradigmatic framework that enables teachers to identify issues related to
educational practice and seek practical solutions to overcome the issues identified
(Freire, 1972; Shor, 1993; Brown, 2001).
Nested in critical emancipatory theory, this paper describes how a lesson study
approach supported the lesson preparation abilities of pre-service teachers during
a work-integrated learning (WIL) process. This sharing of subjective experiences
allows for the formulation of Natural Science lesson ideas, which, according to
Fetterman (2015) aligns well with the concept of practical empowerment. Instead
of following a conventional approach characterised by teacher educators
assuming the role as the sole providers of absolute knowledge, we opt to turn the
conventional approach upside down thus allowing for a stimulating way of
information sharing that is innovative and developing in nature. Our roles in this
study are thus driven towards facilitating their efforts in preparing lessons. This
approach is also referred to a practice of transformative empowerment
(Fetterman, 2001; Fetterman, 2009; Zimmerman, 2000).
The drive towards practical and transformative empowerment unlocks the

concept of self-determination. Fetterman (2015) defines self-determination as the
ability of an individual to demonstrate a range of capabilities. In this study, self-
determination among the pre-service teachers surfaced in terms of their ability to
identify and express the need for support in lesson preparation, partake in a plan
of action to address the issue associated with lesson preparation, identify
resources needed for the structuring of Natural Science lessons, implement the
prepared lessons in an authentic teaching experience, and finally, re-examine the
effectiveness of the lesson study approach with the hope to better the initiative for
future implementation.
3. Research approach and methodology

A series of lesson study sessions took place throughout the duration of the study.
To document this systematic approach towards the planning of lessons and
reflection on lesson presentations, a purposeful sampling qualitative case study
research approach was followed.
Permission for this study was granted by the University of the Free State (Ethical
clearance nr UFS-HSD2018/0073) and the study formed part of the researcher’s
PhD project. Thorough protocols were followed to ensure that the research
participants from now onwards referred to as the research team gave consent and

64
was comfortable in participating in this study. As the principal researcher, I

assured the research team confidentiality throughout the duration of the study.
The team members’ actual names were withheld and their faces were blurred
from photographs. The team members were also allowed to withdraw from the
study at any point of time. All team members signed a consent form and all of the
signed copies are safely stored.
The research site was situated in a School of Education at a South African

University. The School of Education offers a Bachelor of Education degree with
multiple areas of specialisation in intermediate phase (IP), senior phase (SNR) and
further education and training phase (FET) teaching. In this study, the research
team comprised eight 3rd-year undergraduate students specialising in the
teaching of Natural Science.
This study relied on multiple data generation sources such as spontaneous free-
response e-mail communications, photo-voice methodology and a focus group
team discussion to generate and capture empirical data. Critical discourse
analysis was used to analyse the qualitative data since it allows for the
deconstruction and meaning-making of data in an interpretive, descriptive and
exploratory manner (Creswell & Creswell, 2017; Guba & Lincoln, 1988). This
approach to analysing data allowed us to be receptive to the subjective,
individualised and contextualised experiences of the research team.
To be specific, the use of spontaneous free-response e-mail communications shed

light on the first research question of the study, which was to understand the types
of challenges Natural Science pre-service teachers encounter regarding lesson
preparation. A response to this research question was crucial for justifying their
need for lesson study support. Qualitative data that stemmed from the
spontaneous free-response e-mail communications were digitally captured,
thematically analysed to determine patterns from the team’s responses and then
transcribed.
Moreover, the use of photo-voice methodology assisted in clarifying research

question two where the focus was to determine how a lesson study approach can
serve to support the Natural Science pre-service teachers’ development of Natural
Science lessons? The use of photo-voice methodology and a focus group team
discussion enabled the researcher to have an-depth look at the team’s lesson
planning encounters. Striking features of the lesson planning sessions were
visually captured, thematically analysed and transcribed. At the same time, verbal
responses from the focus group team discussion were audio-recorded, transcribed
and thematically analysed through an open-and-axial coding method. This
approach allowed for the identification and description of patterns, trends and
consistencies that existed within the data.
Following the data analysis process, the qualitative data were given back to the
research team to evaluate whether the transcribed data were a true reflection of
their subjective, individualised and contextualised experiences thus complying to
the aspects of credibility, reliability and validity (Guba & Lincoln, 1988).

65
4. Results
The research questions in this study yielded rich qualitative data. Table 1 outlines
the two research questions with the corresponding findings in terms of themes
and sub-themes.
Table 1. General overview of findings to the research questions

Research questions Data collection Themes Sub-themes
technique
Whether the Natural Realising the - Student anxiety,
Science pre-service Spontaneous need for lesson stress, hopelessness
teachers are free-response e- study support and frustration
experiencing issues mail - Unfavourable
pertaining lesson communication learning
preparation? environments
How can a lesson Photo-voice Designing well- - Team-discussions,
study approach serve methodology structured collaborative
to support the Natural Natural Science brainstorming and
Science pre-service lessons as a sharing of lesson
teachers’ development collective ideas
of natural science - The detailed
lessons for a WIL structuring of Natural
experience? Science lessons
Focus group Lesson - Quality Natural
discussion reflections Science learning
experience
Spontaneous - Quality Natural
free-response e- Science teaching
mail experience
communication
4.1 Theme: Realising the need for a lesson study support

The first step towards realising the aim of the study, which was to describe “how
a lesson study approach supported the lesson preparation abilities of pre-service
teachers during a work-integrated learning experience”, was to determine
whether the pre-service teachers did, in fact, experience an issue associated with
lesson preparation? A response to this question is crucial since it justifies the need
to initiate a lesson study approach.
With this research question in mind, a series of spontaneous free-response e-mail

communications from the team were collected and analysed, which resulted in
one overarching theme titled “Realising the need for a lesson study support”. This
theme was supported by two sub-themes, namely “Student anxiety, stress,
hopelessness and frustration” and “Unfavourable learning environments”.
4.1.1 Sub-theme: Student anxiety, stress, hopelessness and frustration

Snippets of spontaneous free-response e-mail communication gave voice to the
research team’s feelings of anxiety, stress, hopelessness and frustration associated
with lesson preparation, as witnessed in Figures 1 and 2.

66
Figure 1. Electronic feedback provided by a pre-service teacher.
A closer look at both these responses provided draws one’s attention to feelings
of hopelessness and frustration with lesson preparation, which in this case is an
outcome of the school visitation process. This assumption can be confirmed with
the phrases “I don’t know where to start”, “before you come”, “lecturers are
coming” and “I’ve been looking for you the whole week”.
4.1.2 Sub-theme: Unfavourable learning environments

In addition, another team member expressed concerns regarding the current
status of her teaching during a school visitation opportunity. To be specific, the
team member voiced her frustration of the “unfavourable learning environment”,
which overshadowed her quality of teaching at a certain school. This can be
witnessed in the e-mail communication below:
An analysis of this response delivers multiple meanings. In the first instance, one
finds the member’s wish to establish a more engaged and meaningful learning
experience for the learners. This can be supported by the phrase “I just want to
engage them more”. Besides boosting learner engagement in the classroom
setting, the team member also wishes to better her classroom management for a
sustained learning experience. This can be confirmed when the pre-service teacher
mentioned that “the class don’t take her serious” and “they just sit there”. All
three electronic responses detailed the severity of the challenge associated with
lesson preparation.
4.2 Theme: Designing well-structured NS lessons as a collective

With the challenge associated with lesson preparation confirmed, the next step
was to address the second research question of the study, that deals with “How a
lesson study approach can serve to support the Natural Science pre-service

67
teachers’ development of Natural Science lessons?”. This time around the use of
photo-voice methodology and a focus group team discussion were key in
determining striking features associated with the lesson study approach.
To be specific, an analysis of a series of photographs taken throughout the lesson

study encounters, pointed to 1 broad theme: “Designing well-structured NS
lessons as a collective”. This particular theme was supported by two sub-themes:
“Team-discussions, collaborative brainstorming and sharing of lesson ideas” and
“The detailed structuring of a NS lesson”.
4.2.1 Sub-theme: Team-discussions, collaborative brainstorming and sharing of lesson

ideas
During the first lesson study session, the team of pre-service teachers got to work
and planned their future NS lessons as a collective. A closer look at figure 4
provides evidence of the discussions, note-making and brainstorming that took
place.
Figure 4. Pre-service teachers preparing NS lessons as a collective.
A glance at these photographs reveals how the team of pre-service teachers

planned their lessons holistically by sharing details about the structure of future
Natural Science lessons. They did this in a group format and the team agreed to
brainstorm and share their teaching ideas with each other. Part of the lesson
planning included their prediction of how the teaching process should unfold. By
the look of things, one gets the sense that the team could share ideas on how to
make their teaching innovative and inclusive.
4.2.2 Sub-theme: The detailed structuring of a NS lesson

Besides the sharing of NS lesson ideas, a closer look at figure 5 suggests that the
pair of pre-service teachers described finer details surrounding the objective of
the Natural Science lesson when they noted on the whiteboard that the lesson
should be “simple, straightforward with one or two action verbs”.

68
Figure 5. Pre-service teachers focusing on the structural design of a NS lesson.
After that, they focused on the structure of the introduction by suggesting that it
be “2 min” long and “catchy”. The pair of pre-service teachers furthermore shared
details about the main activity of the lesson when they suggested it be no more
than “10 to 15 min long”.
Besides discussing the time-span of the main section of the lesson, they could also
brainstorm ideas that might affect teaching and learning dynamics of the lesson.
This included a discussion that dealt with aspects such as teaching strategies,
teaching aids, learner activities and practical demonstrations. Finally, the pair
could also elaborate on a potential conclusion for the lesson when they noted that
the conclusion should be “5 min long” and should be focussed on “wrapping up”
the lesson.
Coupled with the structuring of the Natural Science lesson outline, a pre-service
teacher could unpack finer details of her lesson, as found in figure 6 below.
Figure 6. A pre-service teacher focusing on the structural design of a NS lesson.
An analysis of this photograph details how the pre-service teacher structured a

lesson that dealt with the topic “factors impacting on the human environment”.
In her planning she could unpack the objective of her lesson (for example, learners
should be able to draw a mind-map based on the human factors that influence the

69
environment), consider factors that affect the teaching and learning context (for
example, time and language of instruction), refer to the topics that should be
included in her lesson (for example, human factors leading to desertification,
deforestation and water pollution), consider the teaching strategies that she
wishes to implement in her lesson (for example, engage learners in a jig-saw
activity within pre-determined groups) and, finally, identify potential assessment
approaches (for example, using questioning, self-assessment and learning logs to
test learners’ understanding).
4.3 Theme: Lesson reflections

Following the planning of NS lessons, the team engaged in a focus group
discussion to share their lesson reflections with their peers. An analysis of the
verbal responses from the focus group discussion pointed to two sub-themes “An
improved NS learning experience” and “An improved NS teaching experience”,
which seemed to correspond with the theme “lesson reflections”.
4.3.1 Sub-theme: An improved NS learning experience

Given the opportunity to prepare lessons as a collective, a team member provided
a personal account of her teaching experience during the WIL process:
“Okay, so I had to do a lesson on the topics sorting and sieving of materials

with the kids. [Didi] [a member of the research team] then helped me and
we designed a lesson where I use a funnel with different materials such as
small rocks, sand and different soil types to show the learners that we can
sort and sieve things. What was nice about this lesson was that I could
start the lesson by asking the grade 5’s how to sort the small rocks, fine
sand and marbles from each other. They then had to identify different
ways to sort it. Then I did a practical demonstration where we sorted the
materials using a funnel. I threw the marbles in then the small rocks and
then the sand and the sand went through. We could then see that the sand
went down into the beaker while the rocks and marbles were stuck in the
funnel. This really helped them to understand the concept of sieving. Also,
it was really hands-on. Oh of course this would not have been possible
without you [Didi]”.
(Team member 3; focus-group team discussion)
This particular response confirms the influence of the lesson study intervention
on both the lesson preparation and teaching experience of a pre-service teacher.
In a nutshell, based on the response provided, the assumption is that the lesson
that the pre-service teacher presented provoked the learners’ curiosity of the topic
“sieving of materials” through acts of self-discovery. The pre-service teacher’s
lesson objective was focused on allowing grade 5 learners to gain a better
understanding of the sieving process by engaging them in a practical
demonstration.
To be specific, the pre-service teacher prepared a lesson that required learners to

observe how certain materials such as “small rocks, marbles and sand” can be
sieved by using a funnel. Aspects of self-discovery surfaced when the pre-service
teacher asked the learners the question “how can we sort the small rocks, fine sand

70
and marbles from each other”. This open-ended question was then complemented
by the pre-service teacher practically demonstrating how “sand went down into
the beaker while the rocks and marbles remained stuck”. As part of her lesson
reflection, the pre-service teacher also admitted that the lesson was a “hands-on”
experience.
Another pre-service teacher provided a more spontaneous verbal response of her

experience of her lesson presentation, when she mentioned that:
“Their vibe was amazing it worked everybody wanted to give [an] answer
to the introduction and everybody wanted participated in the problem
solving”.
(Team member 1; focus-group team discussion)
This response confirms the enthusiasm and excitement of the pre-service teacher
of the lesson she presented as part of the WIL process. The phrase “their vibe was
amazing” suggests that the learning environment allowed learners to be excited
and motivated to learn. When the pre-service teacher mentioned that “they all
wanted to participate” confirms the learners’ eagerness to engage in the learning
process which, in this case, required them to “solve problems”.
4.3.2 Sub-theme: An improved NS teaching experience

Besides the verbal feedback provided, another team member spontaneously
reported how the lesson study intervention impacted his quality of teaching, as
witnessed in an e-mail response below:
The phrases “I could not explain the feeling I had, how impresses I was” “was
really helpful” and “fruitful help” confirm the effect of the lesson study approach
on the Natural Science pre-service teachers’ teaching practice.
5. Discussion of findings
From a teacher education point of view, the expectation is that pre-service
teachers develop the deemed teaching knowledge, skills and values within a
specific period while enrolled in an initial teacher education programme.
Unfortunately, as Hadi (2019) point out, not all pre-service teachers develop these
expected qualities at the same tempo. Learning to teach, through the eyes of Parfitt
(2020) is a developmental process, where some pre-service teachers develop
teaching knowledge, skills and values sooner than others. As a result, pre-service
teachers often find themselves in a situation where they need additional time and
support to develop the required teacher attributes, which in the context of this

71
study include 3rd-year pre-service teachers’ skill to effectively prepare Natural

Science lessons for an authentic teaching experience.
In light of this, Janssen, Knoef and Lazonder (2019) suggest that it is essential for
teacher education programmes to mindfully prepare and support pre-service
teachers on their journey to becoming professionally skilled for the teaching
profession. Mansfield and Beltman (2019) are of the view, that pre-service teacher
support is essential for entering teachers to cope with the expected teaching
qualities and values portrayed in national educational policy documents. In
response to the preceding argument, which shed light on the expected teacher
qualities that pre-service teachers should demonstrate as part of their teacher
education, this study acknowledged the need to better support the professional
development of Natural Science pre-service teachers in their 3rd-year of
undergraduate study. This paper describes how a lesson study approach
supported the lesson preparation abilities of pre-service teachers during a work-
integrated learning (WIL) process.
In the first instance, this paper recognises the need for a lesson study approach in
a School of Education through acknowledging the emotional turmoil that the 3rd-
year pre-service teachers endured that were associated with lesson preparation,
particularly during school visitation intervals. This response aligns well with the
view of Fetterman (2017) when he cautions that in doing critical emancipatory
research, one should be critically conscious of the contextual factors that influence
the wellbeing of individuals.
The participants’ expressions of anxiety, stress, hopelessness and frustration with

lesson preparation signalled their call for support and justified the need for the
lesson study approach. Darling-Hammond (2017) reminds us that the WIL
process is characterised by teacher educators being tasked to observe and evaluate
the pre-service teachers’ teaching practice. During this process, the pre-service
teachers are expected to follow a school curriculum that requires them to teach
specific subjects and perform teacher's duties similar to that of established
teachers with vast teaching experiences. More often than not, the pressures of
meeting the teaching expectations may have a negative effect on the pre-service
teacher’s self-esteem, motivation and confidence (Beltman, 2015). This can be
substantiated with the participants’ responses provided such as “I don’t know
where to start”, “before you come” and “lecturers are coming”, “I just want to
engage them more”, “the class don’t take me serious” and “they just sit there”.
Secondly, given the issue associated with lesson preparation, this paper describes
how a lesson study approach practically empowered the research participants to
develop Natural Science lessons within a group format with their fellow peers as
a collective. The use of photo-voice methodology and a focus-group team
discussion were helpful in detailing the key features associated with the lesson
study approach. Evidence suggests that aspects of collective brainstorming and
ideas sharing assisted the team of pre-service teacher to collaboratively structure
Natural Science lessons. Moreover, the team could also focus on structuring
specific components of the lesson plan that includes, the lesson objective (i.e.

72
“lesson to be straightforward with one or two action verbs”), introduction (i.e.

“catchy introduction that is 2min long”), main activity (i.e. “learner activities and
practical demonstrations”) and conclusion (i.e. “wrap up of lesson that is 5min
long”) of the lesson in detail.
In addition, the participants could also describe their teaching approaches (i.e. “I
want them to engage more”) that they wish to follow during these planned
lessons. This particular focus on the various aspects that affect teaching and
learning aligns well with the work of Larkin (2017), when it is mentioned that the
lesson planning process entails a meticulous collection of thoughts and resources
concerning what needs to be taught. Lesson planning, as a key feature of the
lesson study approach, allows pre-service teachers to proactively decide on the
aspects that would be needed to establish a favourable teaching and learning
environment aimed at accommodating diverse learner needs (Sims & Walsh,
2009).
Finally, as part of the lesson reflection interval of the lesson study approach, the
participants’ verbal responses from a focus group team discussion suggested that
the lesson study approach resulted in an improved Natural Science learning
experience as one team member indicated that: “Their [learners] vibe was
amazing it worked everybody wanted to give an answer to the introduction and
everybody wanted to participate in the problem solving” (Team member 1). Given
this particular response provided, an expectation of quality Natural Science
teaching includes teachers being equipped with the knowledge to develop tasks
that would expose learners to acts of self-discovery of scientific knowledge.
Molefe, Stears and Hobden (2016) suggest that the aspect of self-discovery can be
sparked by engaging and exposing learners to a “hands-on” practical
investigation in the Natural Science classroom. Flores (2015) adds, that this
approach to teaching allows for a deeper understanding of Natural Science subject
matter and the development of scientific reasoning skills.
Furthermore, it was pointed out that the lesson study approach allowed for an
improved teaching experience. This assumption can be supported by a participant
who mentioned that: “After the lesson presentation, I could not explain the feeling
I had, how impresses I was” (Team member 3). A closer look at this particular
response suggests that the lesson study approach impacted the pre-service
teacher’s self-efficacy and confidence. Given this response, one could assume that
the pre-service teacher has undergone an element of transformative
empowerment (Fetterman, 2015) since the lesson study approach enabled him to
experience the liberating feeling of self-determination in presenting a lesson with
confidence. These reflections on lesson presentations provided correspond with
the view of Chizhik et al. (2017) when it is suggested that lesson reflection allows
teachers to share their “rich” subjective experience of what “worked” and “what
did not work” in terms of their teaching practice. This approach, in turn, enables
teachers to learn from each other’s teaching experience.

73
6. Conclusion
With the focus on the challenge associated with Natural Science lesson
preparation, this study reported on how the implementation of a lesson study
approach served as an intervention to support the lesson preparation abilities of
pre-service teachers during a WIL process. Couched within critical emancipatory
research, implementing the lesson study approach allowed the pre-service
teachers to enjoy a significant number of benefits. Although the steps of the lesson
study intervention were meticulously followed, the accommodative and engaging
nature of the lesson study approach allowed for the existence of the Natural
Science pre-service teachers’ multiple realities that relate to the WIL process. The
study enabled the Natural Science pre-service teachers to enjoy a sense of
empowerment towards the sharing of diverse contextual experiences and ideas
pivotal to the development of Natural Science lessons for the WIL process.
In this study, the generation of knowledge was inseparably associated with the
collaborative practices and interaction that existed between the team of pre-
service teachers. Key to this free-flowing of information sharing were the values
of respect and trust that existed within the research team (Stebleton, Soria &
Huesman, 2014). The lesson study approach allowed for an engaging space
characterised by values of trust, respect, support and empathy for each other. The
collaborative practices that formed part of the team dynamics challenged the
conventional belief that teacher educators are the ones in power who poses
absolute knowledge. Instead, recognition was given to the subjective experiences
that the pre-service teachers’ voice offered. This approach allowed the team to be
critically conscious of the contextual factors that impact their teaching experience
during the WIL process.
Drawing on Freire’s (1972) understanding of collaborative practices, the

preparation of Natural Science lessons in this study were not carried out by one
for another but rather by one with each other. This approach symbolises the
collective role that the team of pre-service teachers played in the structuring of
lessons. Ultimately, the activities carried out promoted the principle of
emancipation since the lesson study approach served as a means to better position
the pre-service teachers for the WIL experience.
7. References
Akerson, V. L., Pongsanon, K., Rogers, M. A. P., Carter, I., & Galindo, E. (2017). Exploring
the use of lesson study to develop elementary pre-service teachers’ pedagogical
content knowledge for teaching nature of science. International Journal of Science
and Mathematics Education, 15(2), 293-312. doi.org/10.1007/s10763-015-9690-x
Barends, Z., & Nel, C. (2017). Work-integrated learning within the reading literacy
component of foundation phase teacher preparation programmes. South African
Journal of Childhood Education, 7(1), 1-13. doi.org/10.4102/sajce.v7i1.435
Beltman, S. (2015). Teacher professional resilience: Thriving not just surviving. In N.
Weatherby-Fell (Ed.), Learning to teach in the secondary school. Melbourne,
Australia: Cambridge University Press.
Bjuland, R., & Mosvold, R. (2015). Lesson study in teacher education: Learning from a
challenging case. Teaching and teacher education, 52(1), 83-90.
doi.org/10.1016/j.tate.2015.09.005

74
Brown, C. (2001). Our side? Critical theory and international relations. Critical theory and
world politics.
Chizhik, E. W., Chizhik, A. W., Close, C., & Gallego, M. (2017). SMILE (shared mentoring
in instructional learning environments) effectiveness of a lesson-study approach
to student-teaching supervision on a teacher-education performance
assessment.’ Teacher Education Quarterly, 44(2), 27-47. doi.org/10.3390/90010517
Choy, S., & Delahaye, B. (2011). Partnerships between universities and workplaces: some
challenges for work-integrated learning. Studies in Continuing Education, 33(2),
157-172. doi.org/10.1080/0158037X.2010.546079
Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed
methods approaches. Sage publications.
Darling-Hammond, L. (2017). Teacher education around the world: What can we learn
from international practice? European Journal of Teacher Education, 40(3), 291-309.
doi.org/10.1080/02619768.2017.1315399
De Beer, J. (2017). ‘The elephant in the room: the work-integrated learning of student
teachers, and the influence of teacher educator scholarship on their professional
development.’ Africa Education Review, 42(3), 1-12.
doi.org/10.1080/18146627.2016.1224593
Department of Higher Education and Training. (2015). Policy on the Minimum Requirements
for Teacher Education Qualifications, as revised 2014. Pretoria: Government Gazette.
Fernandez, M. L., & Zilliox, J. (2011). Investigating approaches to lesson study in
prospective mathematics teacher education. In Lesson study research and practice in
mathematics education. Springer, Dordrecht.
Fetterman, D. M. (2001). Foundations of empowerment evaluation. Thousand Oaks, CA: Sage.
Fetterman, D. M. (2009). Empowerment evaluation at the Stanford University School of
Medicine: Using a critical friend to improve the clerkship experience. Ensaio:
Avaliaç ãoe Polıticas Publicas em Educacao, 17(63), 197–204.
doi.org/10.1590/S0104-40362009000200002
Fetterman, D. M. (2015). Empowerment evaluation and action research: A convergence of values,
principles, and purpose. In H. Bradbury (Ed.), The SAGE Handbook of Action
Research. Thousand Oaks, CA: Sage.
Fetterman, D. (2017). Transformative empowerment evaluation and Freirean pedagogy:
Alignment with an emancipatory tradition. New Directions for Evaluation, 2017
23(155), 111-126. doi.org/10.3390/laws7010010
Flores, I. M. (2015). Developing pre-service teachers' self-efficacy through field-based
science teaching practice with elementary students. Research in Higher Education
Journal, 27(1), 1-19. doi.org/10.1080/13611267.2016.1222812
Freire, P. (1972). Education for critical consciousness. New York: Seabury.
Gordon, B. M. (1986). The use of emancipatory pedagogy in teacher education. The Journal
of Educational Thought (JET)/Revue De La Pensée Éducative, 2(4) 59-66.
doi.org/10.18297/etd/3160
Guba, E. G., & Lincoln, Y. S. (1988). Do inquiry paradigms imply inquiry
methodologies? Qualitative approaches to evaluation in education, 1(1), 89-115.
doi.org/10.1590/S0104-403620090232
Hadi, A. (2019). Exploring Preparation of Pre-Service Teachers' English Proficiency and
Pedagogy: Stories from an EFL Teacher Education Program. The Qualitative
Report, 24(8), 1946-1966. doi.org/10.3390/laws7010010
Janssen, N., Knoef, M., & Lazonder, A.W. (2019). Technological and pedagogical support
for pre-service teachers’ lesson planning. Technology, Pedagogy and
Education, 28(1), 115-128. doi.org/10.1080/1475939X.2019.1569554

75
Johnston, J. (2011). Interrogating the goals of work-integrated learning: Neoliberal

agendas and critical pedagogy. Asia-Pacific Journal of Cooperative Education, 12(3),
175-182. doi.org/10.1590/S0104-40362009000200002
Jovanovic, J., Fane, J., & Andrew, Y. (2018). Giving institutional voice to work-integrated
learning in academic workloads. International Journal of Work-Integrated Learning,
19(2), 93-109. doi.org/10.1007/s10763-015-1523-x
Larkin, D. (2017). Planning for the elicitation of students’ ideas: A lesson study approach
with pre-service science teachers. Journal of Science Teacher Education, 28(5), 425-
443. doi.org/10.1080/1046560X.2017.1352410
Lysaght, Z. (2011). Epistemological and paradigmatic ecumenism in “Pasteur’s
Quadrant:” Tales from doctoral research. Official Conference Proceedings of the
Third Asian Conference on Education in Osaka, Japan.
Mansfield, C., & Beltman, S. (2019). Promoting resilience for teachers: pre-service and in-
service professional learning. The Australian Educational Researcher, 46(1), 583–588.
doi.org/10.1007/s13384-019-00347-x
Matoti, S. N., Junqueira, K. E., & Odora, R. J. (2011). A comparative study of pre-service
teachers' self-efficacy beliefs before and after work-integrated learning. South
African Journal of Higher Education, 25(6), 1140-1154.
doi.org/10.1080/18146627.2013.855423
Molefe, L., Stears, M., & Hobden, S. (2016). Exploring student teachers’ views of science
process skills in their initial teacher education programmes. South African Journal
of Education, 36(3), 1-12. doi.org/10.1108/IJLLS-10-2014-0039
Ngibe, N.C.P., Pylman, J., Mammen, K.J. & Adu, E.O. (2019). ‘Turning Pre-service School
Experience Challenges into Strengths.’ J Hum Ecol, 66(1-3), 33-44.
doi.org/10.1080/18146627.2013.853558.
Osanloo, A., & Grant, C. (2014). Understanding, selecting, and integrating a theoretical
framework in dissertation research: Creating the blueprint for your
“house”. Administrative issues journal: connecting education, practice, and
research, 4(2), 12-26. doi.org/10.5929/2014.4.2.9
Parfitt, A. (2020). Can the concept of the protean career help us to understand millennial
pre-service teacher retention challenges? A study of two pre-service teachers’
career pathways in England. Journal of Education for Teaching, 4(2), 1-12.
doi.org/10.1080/02607476.2020.1733403
Pham, T., Bao, D., Saito, E., & Chowdhury, R. (2018). Employability of international
students: Strategies to enhance their experience on work-integrated learning
(WIL) programs. Journal of Teaching and Learning for Graduate Employability, 9(1),
62-83. doi.org/10.1080/03307476.2020.122303
Runesson, U., Martin, D., & Clerc-Georgy, A. (2015). Use of theoretical concepts in lesson
study: An example from teacher training. International Journal for Lesson and
Learning Studies, 4(3), 261-273. doi.org/10.1108/IJLLS-10-2014-0039
Shaull, R. (1974). Foreword. In P. Freire (Ed.), Pedagogy of the oppressed. New York, NY:
Seabury Press.
Shor, I. (1988). Working hands and critical minds: A Paulo Freire model for job training.
The Journal of Education, 170(2), 102-121. doi.org/10.1177/002205748817000206
Shor, I. (1993). ‘Education is politics: Paulo Freire’s critical pedagogy.’ Paulo Freire: A
critical encounter.
Sims, L., & Walsh, D. (2009). Lesson study with pre-service teachers: Lessons from
lessons. Teaching and teacher education, 25(5), 724-733.
doi.org/10.1016/j.tate.2008.10.005
Stebleton, M. J., Soria, K. M., & Huesman, R. L. (2014). First‐generation students' sense of
belonging, mental health, and use of counselling services at public research

76
universities. Journal of College Counselling, 17(1), 6-20.

doi.org/10.1080/01425692.2015.1044070
Tran, L. T., & Soejatminah, S. (2016). Get the foot in the door’: international students’
perceptions of work-integrated learning. British Journal of Educational
Studies, 64(3), 337-355. doi.org/10.1080/00071005.2015.1128526
Zimmerman, M. (2000). Empowerment theory. In J. Rappaport & E. Seidman (Eds.), Handbook
of community psychology. New York, NY: Kluwer Academic.
doi.org/10.1080/02673843.2016.1209120

77

Vol. 19, No. 10, pp. 77-99, October 2020
Effect of Differentiated Instruction on the

Achievement and Development of Critical
Thinking Skills among Sixth-Grade Science
Students
Mohammad Salih Al-Shehri

Faculty of Education, King Khalid University, Abha, Saudi Arabia
https://orcid.org/0000-0001-8098-6153
Abstract. The objective of this study was to explore the effect of using a
differentiated instruction method on the achievement and development
of critical thinking skills among sixth-grade students in Abha, Saudi
Arabia. The study used the experimental method through the
application of the instructional program on a sample of 50 students, who
were chosen purposefully from one school in Abha city. The sample was
divided into an experimental group (n = 25), which received instructions
using differentiated instruction, and a control group (n = 25), which
received instructions using conventional methods. Two tests were
developed, an academic achievement test and a critical thinking test, to
measure the performance of students before and after using the
instructional program. Findings showed an improvement in the
academic performance of the experimental group, which were taught
using the instructional program based on differentiated instruction.
Participants were able to increase their level of critical thinking skills in
science. The study recommends using this instructional strategy in
different school subjects other than science. The study also recommends
training teachers on its implementation in classrooms.
Keywords: differentiated instruction; academic performance; critical

thinking; science subject
1. Introduction
Each classroom has students at different levels in terms of previous experiences,
background knowledge, learning methods, needs, tendencies, and interests.
Using one method in teaching without responding to the differences between
students may not bring out the desired results. This may result in neglecting
essential segments of the classroom and in failure to meet students’ needs and
diversifying the outputs. Low achievers may not find support and help to
overcome the educational process and address their deficiencies. At the same

78
time, outstanding students will not find new materials to enrich their skills and
experiences or even to develop them (Al-Tuwairqi, 2013).
Increasing students' interaction with their peers must go beyond the traditional
methods that create a negative role from students. Instead, educators should
work on creating a better learning environment by utilizing modern methods
that increase interaction and students' ability to think, and involve them in
obtaining knowledge, making conclusions, and developing positive
relationships between students themselves. Such a procedure should also
stimulate their potentials in being more creative. The big role relies on the
teacher, as the teacher is one of the sources of the students' motivation. Students
have different levels and types of intelligence, learning styles, interests,
tendencies, aptitudes, and cultures, and this puts the educational process before
a significant challenge to meet their different needs (Abu Jadu & Nawfal, 2010).
There is a need to understand students' learning styles, which is increasingly

important in light of the call for group learning within the heterogeneous
classroom. The teacher's responsibility here is to ensure that students absorb and
engage in learning while identifying their preferred methods. The teacher
should help and encourage students to learn while considering their preferred
learning style (Al-Rashud & Nawfal, 2017). Based on these needs, the concept of
differentiated instruction emerged in the educational field.
1.1. Differentiated instruction

Differentiated instruction aims at utilizing the abilities and skills of every
student in the learning process. Additionally, differentiation techniques can be
achieved through various activities when teachers use it as a philosophy in the
classroom. Teachers are supposed to benefit from effective practices in response
to the diverse needs of students (Subban, 2007). This practical definition of
differentiated instruction reflects the sociocultural theory of Vygotsky that lies in
the social and interactive teacher-student relationship. Diversified education also
provides an effective way to address disparity and differences between students.
Differentiated instruction is the recognition of students' different backgrounds,
aptitude levels, languages, interests, and learning profiles (Subban, 2007).
In the process of using differentiated teaching, the teacher provides specific

ways to teach each student according to their particular needs through
employing different teaching strategies flexibly. Because differentiated
instruction is drawn from theories of multiple intelligences and cooperative
learning, it provides an appropriate educational environment through its diverse
methods, instructions, and activities. Thus, students achieve the required goals
with the method, tools, and activities that suit them (Fadel, 2018; Hakami & Al-
Amarin, 2015). This approach urges the teacher to exert the maximum of their
practices and teaching methods to meet students’ needs to achieve the goals of
learning (Muhammad, 2013). This type of education forms a philosophy or a
method of thinking about teaching and learning through flexibility in setting
learning objectives, presenting scientific content, providing a wide range of
teaching strategies, and planning educational activities that are centered around
the student (Hassanein, 2016). To achieve the main objectives of this method,

79
teachers need to take into account students' knowledge, preferences, and

capabilities, and how they are organized for learning (Ismajli & Imami-Morina,
2018).
1.2. Principles of differentiated instruction

Differentiated instruction stems from Vygotsky's social and cultural philosophy
and his most important views. One of his basic ideas is that learners differ from
each other in terms of characteristics, tendencies, home environment, previous
knowledge, and learning styles and that each student has a mentality different
from his classmates, as they learn in different ways and at different times. In
addition, this theory provides methods, procedures, and activities that enable
each student to achieve the required goals using the appropriate method and
tools, and provides a suitable learning environment for all learners (Al-Jamal,
2019). This kind of education also believes that feelings, emotions, and passions
affect the learning process as well (Al-Tuwairqi, 2013).
The main pillars of differentiated instruction are the teacher and the student,
where the teacher is the coordinator and facilitator of the teaching process and
the student is the recipient of the educational process. The teacher has a clear
idea of the subject matter, and takes into account, appreciates, and builds upon
the differences between students. The teacher also modifies content, processes,
and product in response to the students’ aptitudes, tendencies, and attitudes,
and according to the appropriate teaching style. One of the principles of
differentiated instruction is the positive and active participation of all students
in an environment characterized by mutual respect, where the student and
teacher collaborate in learning and work together flexibly (Tomlinson, 2005).
1.3. Objectives and importance of differentiated instruction

Differentiated instruction seeks to achieve several goals, including the provision
of flexible approaches in content, processes, and outputs (Heacox, 2012). It also
seeks to develop educational activities based on critical topics, concepts,
processes, and essential skills, as well as on multiple methods of presenting the
learning process. This kind of education takes into account students’ readiness
levels, their teaching needs and interests, and the details of the learning process.
It provides opportunities for students to work according to different teaching
methods and agrees with curriculum standards and requirements for each
student. In general, differentiated instruction aims at creating classrooms that
include the responding student and the teacher-facilitator of the learning process
(Abu Qabytah, 2013).
There are several justifications to apply differentiated instruction in education. It

helps the teacher to see students according to the differences and variations
among them. High-achieving students have needs, demands, and interests in the
same way as students who face learning difficulties. Therefore, differentiated
instruction focuses on using modern teaching techniques through the
diversification of teaching methods to help all students of different levels to
achieve maximum levels of excellence in the light of their abilities (Fadel, 2018;
Hakami & Al-Amari, 2015). Eventually, students' self-confidence increases, and

80
the level of motivation for students to complete educational tasks successfully is

more enhanced (Abdulqader, 2019).
1.4. Critical Thinking

Critical thinking is one of the topics that educators are interested in because of
its importance in empowering students with essential skills during the
educational process. Part of this interest in critical thinking is evident in the
tendency of educators of all scientific positions to adopt strategies for teaching
and learning critical thinking skills (Abu Jadu & Nawfal, 2010). The definitions
of critical thinking have varied due to the different theoretical perspectives of
researchers. John Dewey made the first attempts to define critical thinking,
where he argued that it is part of the contemplative thinking of the individual's
ability to be active and persistent. He also believed that it is cautious thinking
that deals with studying and analyzing beliefs and what is expected of
knowledge based on the real ground, supported by the ability to deduce. On the
other hand, Sternberg linked critical thinking with the ability to solve problems,
as he defined it as mental processes and strategies that the individual uses to
solve problems. Tishman, Parkins, and Sternberg believed that critical thinking
directs individuals' minds to produce innovative results, decisions, and
solutions to various issues (Al-Shamlati, 2015).
Abu Jadu and Nawfal (2010) defined critical thinking as reflective, inferential,
self-evaluative thinking that includes a set of interconnected cognitive strategies
and processes such as interpretation, analysis, evaluation, and making
conclusions to examine opinions, beliefs, evidence, concepts, and claims. They
argued that these processes are relied upon when issuing a judgment, solving a
problem, or making a decision, taking into account the views of others. Al-Jaber
(2016) defined it as a group of mental practices that lies in the process of
interaction between the individual and the different educational situations.
Critical thinking is of great importance in education for both teachers and

students. It improves teachers' ability to teach and produce valuable and
responsible practical achievements. It also helps teachers to create activities that
allow their students to practice these skills in the classroom. Conversely, it helps
in developing students' ability to be objective and adhere to clarity and accuracy.
Critical thinking also improves students’ achievement in various school subjects
by allowing them to practice a wide range of thinking skills. It urges students to
observe and control their thinking, which helps them in making important
decisions (Abu Jadu & Nawfal, 2010). Using critical thinking skills encourages
teachers in creating situations to elevate students’ interest by focusing on open
questions, encouraging classroom discussions, respecting the opinion of
colleagues, listening to others' points of view, and providing the opportunity for
all students to express their opinions (Al-yarbou, 2017). Critical thinking
includes several central skills that need to be worked on. Peter Facione (2007)
classified them into six skills: interpretation, analysis, evaluation, deduction,
explanation, and finally, self-organization. Saadeh (2011) believed critical
thinking skills to include induction, deduction, comparison, classification,
evaluation, priority setting, and unification of the links between relationship,

81
cause and effect, analysis, succession, recognition of views, and analysis of

arguments.
1.5. Previous studies

Several studies have been conducted on differentiated instruction. Magableh
and Abdullah (2020) investigated the impact of using differentiated instruction
strategies in improving the academic achievements of students in the English
language. The study was applied to 60 students from the eighth grade, who
were selected randomly from two schools in Jordan. The sample was divided
into two groups. The control group (n = 30) learned English using the traditional
method, whereas the experimental group (n = 30) learned using the
differentiated instruction methods. The study tested participants’ performance
before and after applying the experiment. The results revealed that
differentiated instruction had a great impact on the performance of the
experimental group in learning English language skills.
Dack (2019) collected data from 250 teachers working in the education field on
the role of differentiated instruction. The study collected data through desk
review, assignments, observations, and interviews. The interviews consisted of
ten questions and the observations were done in the classrooms. The results
revealed that a better understanding of the role of differentiated learning helps
teachers and students to achieve better outcomes from the learning process.
Melese and Tinoca (2019) explored the instructor’s knowledge, attitudes, and
practice of differentiated instruction at Bahir Dar University. The study used the
qualitative method. The sample consisted of faculty members. Date were
collected through interviews and focus group discussions. The results showed
that the attitudes of faculty members were positive towards differentiated
instruction. The results also revealed that faculty members have a medium level
of knowledge on differentiated learning and they do not practice it properly.
Al-Rashud and Nawfal (2017) investigated the effectiveness of a differentiated

instruction-based training program in improving the academic achievement of
science, self-concept, and parallel thinking among third-grade intermediate
students. The instructional program was developed to include the dimensions of
differentiated education, which is to modify the content according to the
differentiation of students and to integrate parallel thinking skills. The study
used the achievement test, the self-concept scale, and the parallel thinking scale
to test a sample consisting of 59 students. The results revealed the existence of
differences due to the application of the training program in favor of the
experimental group on the achievement test, the self-concept scale, and the
parallel thinking scale.
Muhammad (2017) identified the effect of a proposed unit in science based on

differentiated education to teach scientific concepts and scientific sense to
second-grade students. The study relied on descriptive, analytical, and quasi-
experimental approaches. The study sample consisted of 44 students. The results
of the study indicated that teaching the unit using differentiated education
developed students' awareness of relationships, knowledge, and scientific

82
concepts. Students received organized knowledge while their learning styles,

abilities, and interests were kept in consideration.
Al-Rashidi (2015) identified the effectiveness of differentiated education in

improving levels of primary stage deaf students’ motivation towards learning
science. The study used a quasi-experimental method. The study sample (20
students) was divided into one experimental and one control group. The results
of the study indicated that teaching according to differentiated instruction
improved the level of motivation towards science learning among deaf students,
as well as the existing differences between the level of the deaf students in the
experimental group on the scale of motivation towards learning science as a
whole.
Hakami and Al-Amarin (2015) investigated the degree of applying differentiated

education for science teachers among their students during classroom teaching
in Syria. The researchers built a questionnaire consisting of 30 items on the
practice of science teachers for differentiated education, and it was applied to
125 science teachers. The results revealed that the level of practicing
differentiated education by science teachers was very low. The results also
showed no differences because of the gender variable. As for the experience
variable, there were significant differences in favor of less-experienced teachers
as opposed to those with higher educational qualifications.
Al-Mahdawi (2014) identified the impact of differentiated teaching on academic

achievement in the biology course of secondary students. The study sample
consisted of 45 students. The results of the study showed that the students who
received training through differentiated teaching performed better than the
students in the control group in the achievement test.
Abu Qabytah (2013) explored the impact of using differentiated education on

acquiring scientific concepts and increasing tenth-grade students’ motivation
towards learning science. The study was applied to a sample of 136 tenth-grade
students in two schools, chosen purposefully. The results found differences in
the scores of students in the test of acquiring scientific concepts and test of
motivation towards science due to the teaching strategy. Female students
achieved better scores than male students in both tests.
Kinsberg (2012) examined the effects of differentiated education in teaching

geology for sixth graders through three lessons within one unit of study. The
study used the survey method, interviews, and teacher observations as tools to
collect data. The results showed that there was an impact on student
achievement, participation, and attitudes. Detailed instructions improved
achievement for low-achieving students, whereas the results for other students
were mixed. Students had more negative attitudes. After applying differentiated
instruction, this experiment yielded mixed effects on student and teacher
engagement.
1.6. The problem of the study

There is an urgent need to accomplish the objectives of any designed curriculum
and the principles of equal educational opportunities among students in the

83
education process. Students' motivation to learn must also be increased, while

taking into account students' needs, preferences, previous experiences, and their
differences in skills, interests, abilities, and intelligence capabilities. We find in
the real application that science teachers mostly focus on middle-level students,
without taking into account the high and low achievers. Different studies and
research (Al-Qahtani, 2013; Al-Shahrani, 2016; Al-yarbou, 2017) have raised the
importance of differentiated instruction. Therefore, science teachers must adapt
teaching methods based on their knowledge of students’ abilities and mental
capabilities and their levels of growth and achievement. They should also pay
attention to students' scientific, economic, and social backgrounds, as well as
possess knowledge of their attitudes, tendencies, and values. Science teachers
rely heavily on the teacher's guide and curriculum, which leads to lower quality
in teaching and fewer outcomes.
Teaching methods vary according to students' academic levels and abilities.

Therefore, choosing methods should suit student needs that stimulate their
motivations towards learning and which are in a manner consistent with the
nature of the material presented to them. The science subject can be presented in
more than one advanced way to achieve the desired goals. The researcher
noticed, during his visit to schools in Aseer educational region, as a supervisor
of field education specializing in elementary education, the frequent complaints
of many science teachers about the difficulty of teaching developed science
curricula. Most of the teachers use conventional methods without activating the
student’s role due to their lack of familiarity with modern techniques.
Therefore, the study raised the following question:

• What is the effect of teaching science using differentiated instruction on
the achievement and development of critical thinking skills among sixth-
grade students?
This main question has two sub-questions:

1- What is the effect of teaching science using differentiated instruction in
developing the academic achievement of sixth-grade students?
2- What is the effect of teaching science using differentiated instruction in
developing the critical thinking skills of sixth-grade students?
The importance of the current study lies in the fact that it aims to improve
students’ performance in their academic achievement and to develop their
critical thinking skills in science. The study raises and directs attention to the
importance of addressing modern topics related to science education. It also
raises the issue of the importance of preparing teachers and developing their
teaching methods, in addition to developing the teaching performance of
teachers through the use of modern techniques.
2. Methodology
The study followed the quasi-experimental approach. The study sample was
divided into two groups: a control group, which studied the unit on space using
the conventional method, and an experimental group, which studied the space
unit using differentiated instruction. The variable of the study was an

84
independent variable, which was represented in the method of teaching and has
two levels: differentiated instruction and the conventional method. The
dependent variable was represented in the participants' responses to the
academic achievement test and the critical thinking test.
2.1. Sampling
The study population included sixth-grade students who studied science in
Abha during the 2019/2020 academic year. The sample included 50 male
students from the sixth grade who were studying science at Habib bin Zaid
Elementary School in Abha. The school was chosen purposefully due to the
cooperation of the principal and its teachers and the availability of the necessary
facilities to conduct the study. As for the two study groups, one experimental
and one control, they were divided randomly into two equal groups.
2.2. Instrumentation
The following instruments were used to answer the questions of the study:
Frist: the academic achievement test

This test was designed to measure students' achievement in science before and
after the experiment according to Bloom’s levels for academic achievement. It
was built according to the following procedures:
1. Space unit content analysis and formulation of behavioral goals spread
across Bloom's cognitive levels.
2. Preparing a table of specifications for the test, taking into account the relative
weights of the content dimensions and the levels of behavioral goals.
3. Writing the test paragraphs of the multiple-choice type. Their final number
were 20 paragraphs.
4. The content analysis, behavioral objectives, and test items were presented to
the judges, where they were asked to review the content analysis, test items,
alternatives provided for each paragraph, and linguistic accuracy.
5. In light of the opinions of the judges, the test was finalized and applied to an
exploratory sample outside the study sample to ensure its reliability and to
calculate the discrimination and difficulty coefficients.
Validity of the achievement test

The validity of the achievement test was tested by presenting it to a group of
eight judges who are experts and specialists in science teaching methods. The
judges provided comments on the correctness of alternatives and the accuracy of
the language. In light of the judges' comments and suggestions, some
adjustments were made to the test.
The discrimination coefficients were calculated using the following equation:
Eqn 1:
Discrimination coefficient
Number of correct answers for the upper group − number of correct answers for the lower group
=
Number of students in one group

85
The discriminations coefficients for the test items ranged between 0.30 and 0.81,
which are acceptable rates.
The difficulty coefficients were calculated using the following equation:
Eqn 2:
Difficulty coefficients = Number of errors in the questions

Number of participants
The difficulty coefficients for the test items ranged between 0.28 and 0.77, which
are acceptable rates.
Reliability of the achievement test

The reliability of the test was ensured using the test re-test method. The test first
was distributed to a sample consisting of 20 students other than the study
sample, and was then re-applied after two weeks. The stability coefficient was
calculated and was 0.89, which is suitable for the study, and thus the test was
put in its final form (see Appendix 1).
Second: the critical thinking test

This test was designed to measure the extent to which sixth-grade students
acquire critical thinking skills in science before and after experimenting with five
skills: interpretation, deduction, comparison, classification, and evaluation of
arguments. It was developed based on the following steps:
1. Identifying the test objective to measure the critical thinking skills of
sixth-grade students in its five dimensions. The test items were
distributed on these dimensions.
2. The test items were as follows:
▪ Explanation skill, consisting of three questions, including
12 paragraphs.
▪ Deduction skill, consisting of three questions, including
12 paragraphs.
▪ Comparison skill, consisting of three questions, including
12 paragraphs.
▪ Classification skill, consisting of three questions, including
4 paragraphs.
▪ Evaluating arguments, consisting of three questions, including
8 paragraphs.
3. The number of test items were 48 divided into 15 questions (see
Appendix 2).
Validity of the critical thinking test

The critical thinking test was presented to eight judges who are experts in
curriculum design and teaching methods to consider the suitability of the test.
They checked the test in terms of the correctness of the language and suitability
for the study. In light of their comments and suggestions, some modifications
were made.

86
Reliability of the critical thinking test

The test re-test method was used to check the reliability of the critical thinking
test. An exploratory sample (n = 20) from outside the study sample was tested,
who received the test again after two weeks. The reliability coefficient of the test
was 0.89 (see Appendix 2).
2.3. The educational material

The educational material focused on the space unit, which was prepared to suit
the nature of differentiated instruction to be a guide for science teachers to use
during preparation and application. The following are the steps for preparing
the educational material:
1. Analyzing the space unit content and setting behavioral goals for each
lesson.
2. Developing a daily preparation plan for each lesson according to the
differentiated instruction (think-pair-share/brainstorming/cooperative
learning).
3. Presenting the preparation plans to a group of eight judges to express their
observations and opinions. Amendments were made from deletion and
addition according to their observations.
4. The material was ready for implementation in the classroom on the
experimental group, and was then presented to the teachers participating in
the experiment, who trained to implement it.
2.4. Procedures
The following steps were applied to implement the study:
1. Reviewing the literature and previous studies related to recent trends in
science education, in particular the use of differentiated instruction that is
suitable for the age characteristics of sixth-grade students.
2. Reformulating the lessons of the space unit according to differentiated
instruction and presenting them to a group of judges and to make
adjustments according to their opinions and observations.
3. Preparing two study tools (achievement test and critical thinking test) in the
subject of science.
4. Choosing the school of the study – Habib bin Zaid Elementary School in
Abha – in a purposeful manner. The experimental and control study groups
were divided randomly. The experimental group received instructions using
differentiated learning, whereas the control group received instructions
using the conventional method.
5. Identifying the time for conducting the study. It took three weeks to execute
the study.
6. Applying the pre-achievement and pre-critical thinking tests before
conducting the study to the two study groups to assess the level of sixth-
grade students in critical thinking and achievement in science.
7. Teaching the reformulated space unit using differentiated instruction to the
experimental group, and teaching the control group the same unit using the
conventional method.
8. Applying the post-achievement and post-critical thinking tests to the two
study groups to assess the level of sixth-grade students in critical thinking
and achievement in science.

87
9. Correcting the two tests, recording their results, and conducting appropriate
statistical analyses using SPSS to extract data for discussion.
10. Discussing the results and coming up with recommendations.
2.5. Data Analysis:

To answer the study questions, statistical analysis was done by extracting
standard deviations and mean scores of participants' responses to the
achievement test and the critical thinking test. The t-test was used to answer the
study’s questions related to the detection of any statistical differences between
the two groups according to their scores in the post-achievement and the post-
critical thinking tests.
3. Results and discussion

3.1. Results before applying the study
To ensure levels of the experimental and control groups were equal, the
achievement test and the critical thinking test were applied to the two study
groups before experimentation. Table 1 shows the extracted mean scores and
standard deviations of the responses to the academic achievement test.
Table 1. t-test results for the responses of the two study groups to the pre-achievement test
Variable Group No. Mean St. dev t-value Sig.
score
Pre-test Control 25 14.52 3.5356 0.824 Not
Experimental 25 15.04 2.1213 statistically
Total 50 significant
Table 1 shows that there were no significant differences in the overall mean
scores for both the control (M = 14.52, SD = 3.5356) as well as the experimental
group (M = 15.04, SD = 2.1213). After conducting the t-test, the t-value was 0.824,
which is not statistically significant (at α = 0.05). Therefore, there was an absence
of statistical differences between the two groups, which means that the two
groups learned on approximately the same level.
Table 2. t-test results for the responses of the two study groups to the pre-critical thinking test
Variable Group No. Mean St. dev t-value Sig.
score
Pre-test Control 25 14.88 3.6778 1.1157 Not statistically
Experimental 25 13.68 3.9234 significant
Total 50
Table 2 shows that there were no differences in the overall mean scores for both
the control (M = 14.88, SD = 3.6778) as well as the experimental group (M =
13.68, SD = 3.9234). The t-value at 1.1157 is not statistically significant (at α =
0.05). This indicates that there are no statistically significant differences in the
scores of the two groups, which means that the two groups learned on
approximately the same level.

88
3.2. Results of the first question

The first research question was: What is the effect of teaching science using
differentiated instruction in developing the academic achievement of sixth-grade
students?
The mean scores and standard deviations of the participants' responses to the
post-achievement test were extracted as shown in Table 3.
Table 3. t-test results for the responses of the two study groups to the post-
achievement test
Teaching method No. Mean St. dev t-value Sig.
score
Experimental 25 17.04 1.4854 5.0104 Significant
(differentiated (at α = 0.05)
instruction)
Control 25 14.16 2.461
(conventional
method)
Total 50
Table 3 shows that the calculated t-value was 5.0104, which is statistically
significant (at α = 0.05). This shows the existence of statistical differences
between the mean score of the control and experimental groups. The
experimental group outperformed the control group after having studied science
using differentiated instruction. The mean score of the control group was 14.16,
with a standard deviation of 2.461. This indicates the usefulness of using
differentiated instruction in teaching science to the sixth grade. The reason for
the better achievement of the experimental group could be attributed to the
nature of differentiated instruction and its appropriateness to the science subject.
The unit from the science curriculum was from newly developed material
focusing on contemplating space and linking it to cosmic phenomena that
students see in reality. Some cosmic phenomena require deduction, knowledge,
and application in reality. Using this teaching method provided participants the
desire to think, expand their knowledge, conclude, and link the information
contained therein. In addition, the diversity of the differentiated instruction
patterns was to facilitate the process of understanding information, assimilating
it, and then recalling and remembering it by all participants. This was
demonstrated through participants’ interaction and responses during the
experiment, which facilitated the acquisition and retention of scientific materials
and experiences and their transfer to similar new educational situations. They
also developed their self-dependence in answering and correcting themselves
when wrong and enriching their information when incomplete.
3.3. Results of the second question

The second research question was: What is the effect of teaching science using
differentiated instruction in developing the critical thinking skills of sixth-grade
students?

89
The mean scores and standard deviations of the participants' responses to the
post-critical thinking test were extracted as shown in Table 4.
Table 4. t-test results for the responses of the two study groups to the post-critical
thinking test
Teaching method No. Mean St. dev t-value Sig.
score
Experimental 25 29.68 9.37959 5.65022 Significant (at
(differentiated α = 0.05)
instruction)
Control 25 24.64 10.5154
(conventional
method)
Total 50
Table 4 shows that the t-value was 5.65022, which indicates the existence of
statistically significant differences (at α = 0.05) in the mean scores of the control
and experimental groups. The scores of the experimental group that studied
using differentiated instruction were better. The mean score of the control group
that studied in the conventional method was 24.64, with a standard deviation of
10.5154, which the experimental group performed more superiorly in the post-
test.
This may be attributed to the great compatibility between differentiated

education styles used (brainstorming, cooperative learning, and think-pair-
share) and between critical thinking and its relationship with the science subject
of space. This enabled the participants to practice various mental skills while
applying the study, raising different discussions and expressing opinions,
organizing ideas, and allowing them to meditate and think. It also provided
participants the chance to link ideas and topics with each other, which helps in
developing critical thinking skills. Using this strategy enabled all participants to
delve into the sciences because this strategy is built on respect for students'
minds and potentials and encourages them to present their ideas and opinions,
listen to their colleagues, and discuss with them, thereby enriching them with
multiple critical thinking skills.
4. Conclusion
The effect of using differentiated instruction in teaching science to sixth-grade
students in Abha, Saudi Arabia was investigated in this study. The researcher
developed training material based on the differentiation strategy. The
participating students received the training and were tested afterward. The
results of the study revealed an improvement in the academic level of the
experimental group after being taught using the differentiation strategy. This
indicates the effectiveness of this strategy and its appropriateness to be used in
classrooms, not only in science classes, but also in other subjects.
4.1. Recommendations
The study recommends extending the application of differentiated instruction to
include the rest of the branches of science curricula in other educational levels.

90
Teachers of science need to receive training on how to use differentiated

instruction in teaching because of the positive impact on academic achievement
and the development of critical thinking skills. It is important also to identify the
effectiveness of using differentiated teaching in other learning outcomes such as
creative thinking, reflective thinking, and attitudes towards the school subjects.
The study also recommends applying differentiated instruction to a larger
sample or another sample of middle and high school students, or a sample of
female students, or a sample from other educational areas.
4.2. Limitations of the study

Thematic limitations: the study was limited to the Space Unit from the textbook
of the sixth grade, edition 2019. The study focused on measuring the impact of
differentiated instruction strategy on academic achievement according to
Bloom’s levels. The critical thinking test was limited to the following skills:
interpretation, deduction, comparison, classification, and evaluation of
arguments. Human limitations: The study was conducted on a sample of sixth-
grade male students from Habib bin Zaid Elementary School in Abha. Time
limitations: The study was conducted in the second semester of the 2019/2020
academic year.
5. References
Abdulqader, M. H. (2019). The effect of using the differentiated education approach on
primary school students' acquisition of some linguistic structures and the
development of their language performance skills. Sultan Qaboos University
Journal of Educational and Psychological Studies, 13(2), 337-367.
Abu Jadu, S. M., & Nawfal, M. B. (2010). Teaching thinking: Theory and practice (1st ed.).
Amman: Dar Al-Masirah.
Abu Qabytah, S. M. (2013). The effect of using the differentiated instruction strategy on the
acquisition of scientific concepts and motivation towards science learning among tenth-
grade students (Unpublished MA thesis). Al-Quds University, Palestine.
Retrieved from: https://dspace.alquds.edu/handle/20.500.12213/4498
Al-Jaber, L. (2016). Critical thinking skills in the activities included in the chemistry
textbook for secondary school students: An analytical study. Journal of
Educational and Psychological Sciences, Qassim University, 9(2), 421-458. Retrieved
from: http://search.shamaa.org/FullRecord?ID=121146
Al-Jamal, L. (2019). Effectiveness of an enrichment unit based on differentiated education and
teaching my beautiful language to develop critical thinking and life skills among sixth-
grade female students (Unpublished MA thesis). King Khalid University, Abha,
Saudi Arabia.
Al-Mahdawi, F. (2014). The effect of differentiated teaching strategy on developing biology
course achievement among twelfth-grade students (Unpublished master's thesis).
College of Education, Umm Al-Qura University: Makkah, Saudi Arabia.
Al-Qahtani, M. M. (2013). The effect of using the septuple learning course in science teaching
on developing critical thinking skills among sixth-grade students (Unpublished
master's thesis). King Khalid University, Abha.
Al-Rashidi, K. M. (2015). The effectiveness of differentiated education in improving the
level of motivation towards learning science for deaf students in the primary
stage. Journal of the College of Education – Al-Azhar University – Egypt, 163(1), 1-52.
Retrieved from: http://search.mandumah.com/Record/704962

91
Al-Rashud, J. S., & Nawfal, M. B. (2017). The effectiveness of using a training program
based on the theory of differentiated instruction in academic achievement in
science, self-concept, and parallel thinking among third-grade intermediate
students. Educational Sciences Studies, 44(4), 249-269.
Al-Shahrani, R. M. (2016). The impact of science teaching using the reciprocal teaching strategy
on achievement and development of critical thinking skills among second-grade
intermediate students (Unpublished master’s thesis). King Khalid University,
Abha.
Al-Shamlati, O. A. (2015). The effect of teaching interpretation using probing questions
on achievement and development of critical thinking skills among intermediate
second grade students. Journal of Human and Administrative Sciences, Majmaah
University, 7, 64-95.
Al-Tuwairqi, H. M. (2013). Differentiated teaching and its effect on motivation, thinking, and
academic achievement (1st ed). Khwarazm Scientific, Jeddah Publishers, and
Libraries. Jeddah, Saudi Arabia.
Al-yarbou, F. F. (2017). The effect of science teaching using probing questions on developing
understanding and critical thinking skills among third-grade middle school students
(Unpublished master's thesis). King Khalid University, Abha.
Dack, H. (2019). Understanding teacher candidate misconceptions and concerns about
differentiated instruction. The Teacher Educator, 54(1), 22-45.
https://doi.org/10.1080/08878730.2018.1485802
Fadel, S. H. (2018). The effectiveness of differentiated education strategy on fourth grade
literary achievement in history. Journal of the College of Basic Education for
Educational and Human Sciences / University of Babylon, 38, 840-850.
Facione, P. A. (2007). Critical thinking: What it is and why it counts. Millbrae, CA:
California Academic Press.
Hakami, S., & Al-Amarin, Y. (2015). The degree of practicing differentiated teaching by
science teachers on their students during classroom instruction in the basic
education stage in the province of Sweida. Al-Baath University Journal, 73(11),
137-159.
Hassanein, A. A. (2016). The effectiveness of teaching science using differentiated
instruction in developing achievement and creativity skills such as critical
thinking and communication among fourth-grade female students. Arab Studies
in Education, Psychology Egypt, 69, 208-359.
Heacox, D. (2012). Differentiating instruction in the regular classroom: How to reach and teach
all learners (Updated anniversary edition). Free Spirit Publishing. Minneapolis.
Ismajli, H., & Imami-Morina, I. (2018). Differentiated instruction: Understanding and
applying interactive strategies to meet the needs of all the students. International
Journal of Instruction, 11(3), 207-218.
Kinsberg, B. R. (2012). The effects of differentiating instruction in a mixed-ability middle school
science class (Unpublished master’s dissertation). Montana State University,
Montana. Retrieved from:
https://scholarworks.montana.edu/xmlui/handle/1/1641
Magableh, I. S. I., & Abdullah, A. (2020). On the effectiveness of differentiated
instruction in the enhancement of Jordanian students' overall achievement.
International Journal of Instruction, 13(2), 533-548.
https://eric.ed.gov/?id=EJ1249110
Melese, S., & Tinoca, L. (2019). Instructors’ knowledge, attitude, and practice of
differentiated instruction: The case of the college of education and behavioral
sciences, Bahir Dar University, Amhara region, Ethiopia. Cogent Education, 6(1),
1642294. https://doi.org/10.1080/2331186X.2019.1642294

92
Muhammad, E. H. (2013). The effectiveness of the differentiated education strategy in the

academic achievement of fourth grade primary students in literature and texts
(Unpublished master’s thesis). Faculty of Education for Humanities, Diyala
University, Iraq.
Muhammad, K. A. (2017). A proposed unit in science based on differentiated education
to provide scientific concepts and a scientific sense for second-grade students.
Scientific Education Journal - Egypt, 20(1), 1-49.
Saadeh, J. (2011). Teaching thinking skills with hundreds of practical examples (5th ed.).
Amman: Dar Al Shorouk.
Subban, P. (2007). Differentiated instruction: A research basis. International Education
Journal, 2006, 7(7), 935-947.
Tomlinson, C. A. (2005). Grading and differentiation: Paradox or good practice? Theory
into Practice, 44(3), 262-269.

93
Appendix 1
The academic achievement test
1. The apparent motion of the sun is caused by:

A. Earths circulation around its orbit
B. The succession of the seasons
C. Earth's rotation around the sun
D. Earth's axis.
2. The longitude that shows the change in dates is called:
A. Baseline latitude
B. International dateline
C. Equator
D. Standard time zone
3. The rotation of the earth around itself results in:
A. The succession of night and day
B. Solar eclipse
C. Lunar eclipse
D. Phases of the moon
4. The craters on the surface of the moon, which are in the form of deep
plates, are called:
A. Lunar seas
B. Moon phase
C. Nozzles
D. The sun and the stars
5. The tide is a phenomenon that arises due to the force of attraction
between:
A. Earth and the Moon
B. The Sun and Earth
C. The Sun and the stars
D. The ocean and land
6. Earth’s rotation around the sun is:
A. The annual Earth cycle
B. The earth’s monthly cycle
C. Tides
D. Gravity
7. The main reason for the occurrence of the four seasons is:
A. The angle of inclination of the Earth's axis changes as it orbits around
the sun.
B. The angle of inclination of the Earth's axis changes as it orbits around
the moon
C. The distance of Earth from the sun changed during its orbit around
the sun
D. Earth's circulation around its orbit.
8. Which of the following is not a galaxy form:
A. Spiral
B. Irregular
C. Elliptical
D. Square

94
9. One of the following happens to the universe from the moment of the Big
Bang until today:
A. Overheating
B. Contraction
C. Expansion
D. Explosion
10. The color.............indicates a greater surface temperature of the star:
A. red
B. yellow
C. Bluish white
D. Orange
11. The Milky Way galaxy is:
A. Primitive galaxy
B. An irregular galaxy
C. Elliptical galaxy
D. Spiral galaxy.
12. Astronomers call the small rocky bodies that collide with the surface of
the moon as:
A. Meteors
B. Moons
C. Meteorites
D. Comets
13. Which of the following planets is closer to the size of the Earth:
A. Mercury
B. Mars
C. Venus
D. Jupiter
14. The suitable unit for measuring distances between stars:
A. meter
B. kilometer
C. mile
D. light year
15. A planet that can have rings is:
A. Mercury
B. Saturn
C. Neptune
D. Venus
16. What separates the inner and outer planets of the solar system is:
A. Asteroids belt
B. Meteors and meteorites belt
C. Stars
D. An Atmosphere
17. If it is seven o'clock in the evening in Riyadh, then the time in Marrakesh
is:
A. 10 pm
B. 4 pm
C. 7 pm
D. 6 pm

95
18. Sundial is:

A. A simple tool to tell time using shadow length and direction
B. A tool for measuring the distance between Earth and the sun
C. A tool for measuring the length of the tides
D. Standard timing instrument
19. The width of the standard time zone is:
A. 15 degrees between the meridians of Earth
B. 15 degrees between latitudes on Earth
C. 24 degrees between the meridians of Earth
D. 20 degrees between latitudes of Earth
20. The "large group of stars that are bound together by gravity" is called:
A. Nebula
B. galaxy
C. comet
D. Meteorite

96
Appendix 2
The critical thinking skills test
Interpretation skill
- Place (√) in the box you consider the right answer:

- Some phrases may have more than one result, and you must specify them
all.
1. If you travel west of the International Date Line, what happens?

No. Interpretation Consequential Not
Consequential
1 Delay in time
2 Advance in time
3 Time stays as it is
2. Earth is the only planet suitable for life.

Consequential
1 For the presence of water and air
2 There is a life that we do not know about on
other planets
3 For its distance from the sun
4 Because of gravity
3. There are no crater edges on the moon's surface:

Consequential
1 Because of the ongoing erosion processes
2 There is no life on the moon
3 There is no water or air on the surface of the
moon
4 Space objects collide with the moon
5 Spaceships land on the surface of the moon

97
Deduction skill
- This test includes phrases followed by several conclusions that may be

correct or wrong. You are required to read the phrase well and then read the
conclusions resulting from it, and then put (√) in the box representing your
answer.
1- A lunar eclipse occurs when:
No. Suggested conclusions Right Wrong

1 Earth passes in the moon’s shadow
2 Earth is located between the sun and the moon
3 The sun is between Earth and the moon
4 The sun passes in the shadow of the moon
2- Tide happens because of:

1 Gravity between Earth and the moon
2 Gravity between the sun and the moon
3 solar eclipse
4 Lunar eclipse
3- What happens because of the moon's rotation around the earth?

1 The tides
2 Moon phases
3 A succession of the four seasons
4 a succession of night and day
Comparison of skill
- The following phrases and concepts have a commonality in a certain

meaning. Identify what is unique about each concept and write it down in
the space provided.
1. Concept one: Earth

Concept two: the Moon
1 What is common between the two

concepts?
2 What is unique in the first concept?
3 What is unique in the second concept?

98
2. Concept one: Meteor

Concept two: Comet

concepts?
3. Concept one: Annual Earth Cycle

Concept two: Daily Earth Cycle

concepts?
4. Concept one: Spiral galaxy

Concept two: irregular galaxy

concepts?
Classification skill
This test includes lists of vocabulary, each list has one characteristic except for
one vocabulary between that does not belong to the group. You are required to
read the vocabulary lists, and then identify the different vocabulary and write it
in the space provided.
1. Which of the following is different from the others?

List Different vocabulary
Sun - Mercury - Venus - Mars
2.
Phases of the moon - lunar eclipse - craters - solar
eclipse
3.
Asteroid - comet - meteor - light year
4.
Reflecting astronomical telescope - sundial -
refracting astronomical telescope

99
Evaluation of arguments:
The following are a set of issues related to the science course. Each case
was placed in a special table and followed by four arguments, some of
them are strong and some of them are weak. Place (√) in front of the
argument that represents your answer.
1. We cannot live without stars:
No. Evaluation of argument Strong Weak

1 They help in setting directions
2 They decorate the sky
3 They light the universe
4 They help in identifying seasons
2. We cannot live on Mercury because:
No. Evaluation of argument Strong Weak

1 Very high temperature
2 Very low temperature
3 No gravity
4 No water on the planet

100

Vol. 19, No. 10, pp. 100-114, October 2020
Rethinking Privilege in Teaching English in

Japanese Higher Education
Khatereh Hosseininasab
Graduate School of Human Sciences, Osaka University, Osaka, Japan
https://orcid.org/0000-0001-7460-135X
Abstract. This article addresses the issue of native-speakerism in teaching

English in the context of Japanese higher education and the privilege
arising from it. Previous research has shown that native speakers are
often regarded as highly skilled and qualified teachers in teaching their
mother tongue. This has often led to the marginalization of teachers who
speak the language they teach as an additional language. In the case of
Japan, however, there is doubt about the existence of such a privilege for
native-speaker teachers and some studies have shown that native
speakers of English do not receive preferential treatment in this context
as they are often perceived as replaceable and temporary. The present
study aimed to further explore this issue by focusing on the varieties of
English Japanese universities expect their teachers to speak. In so doing,
the study has investigated hiring policies of Japanese universities with
reference to the job advertisements they post on a designated portal. The
results of the qualitative thematic analysis indicated that the majority of
the advertisements demanded prospective candidates to be native
speakers of English, which also meant that this subgroup of teachers has
a privilege in landing academic jobs by token of the variety of English
they speak. The article suggests that the critical pedagogical approach of
teaching English as an international language (TEIL) can mitigate such
privilege by raising awareness towards the validity and appropriateness
of different varieties of English spoken in the world.
Keywords: critical pedagogy; higher education; identity; privilege;

teaching English as an international language
1. Introduction: Teaching English in Japanese higher education

Ever since English was accepted as the most frequently used language for
international communications, it has become a symbol for globalization around
the world (Seargeant, 2011). The consequences of this for education systems have
been clear in the sense that they are expected to enable students to speak English
as this will benefit not only the students but also their nations in the long run. In
other words, the rationale for the current emphasis on the development of English
language skills is that English is the language of science and technology and those
who do not speak it will fail to be competitive in the global job market. That is

101
why teaching English as a foreign language (TEFL) has attracted a lot of attention
in the education systems of countries in which English is not spoken as the first
language. Japan is not an exception in this regard. Pretty similar to many other
countries, in Japan also, English is considered essential for participation in global
economy (Crystal, 2003). However, there seems to be a mismatch between what
is expected and what has actually happened in Japan, as the criticism is often
heard that Japanese university graduates do not acquire sufficient English
proficiency to meet the needs of the business sector (Aspinall, 2006).
This dissatisfaction with the English proficiency of Japanese youth triggered a

change in the course of studies by the Japan Ministry of Education, Sport, Science
and Technology (MEXT) in 2003. The last revisions in 2003, also known as the
Action Plan, emphasize the development of spoken communication skills in
students through increasing secondary school teachers’ communicative and
pedagogical skills as well as boosting students’ motivation and offering
alternative methods of assessment for the English component in university
entrance examinations (MEXT, 2003, as cited in Nagatomo, 2012). The extent to
which such new concerns of MEXT have been successfully translated to the reality
of language teaching in Japan is a legitimate question; however, another question
to ask is why the ministry has remained indifferent to how English is taught at
universities in Japan. Nagatomo (2012) argued that except demanding a change
in university entrance exams and an increase in the number of university classes
to be taught in English, the Action Plan does not address English education at
universities. It makes no mention of the need to increase university teachers’
communication and pedagogical skills and neither does it mention the preferred
methodology and desired outcome of English education at universities.
The Action Plan, some may argue, is not by definition supposed to address
tertiary education. This argument is, of course, fair enough, as the section in
charge of the plan is primarily concerned with secondary education in Japan.
Nevertheless, it is still a legitimate question to ask MEXT why they have not had
similar concerns for post-secondary English education. The failure to instigate
reform in English education at tertiary level may partly explain the current
dissatisfaction with the proficiency level of Japanese graduates. The failure has
become even more imminent after the introduction of the concept of “global
jinzai” (global human resource) into the discourse of human resource
development in tertiary education. The proposed definition for global jinzai
introduces a range of competencies expected to be developed in graduates, of
which one of the components is the development of foreign language (usually
English) communication skills. According to the objectives stated for this
particular component, students are expected to become proficient in English
communication while still being able to appreciate the Japanese culture and its
values. The burden of realizing these objectives is at least in part on the shoulders
of English teachers in universities. In the absence of systematic pre-service and
in-service teacher-training programs at tertiary level, however, how have
universities coped with the twofold goal of making their students fluent in
English while assuring that they deeply understand and appreciate Japanese
cultural values? One way to address this issue has been prioritizing Japanese
English teachers over native speakers of English, or the so-called Japanization of

102
English teaching (Hashimoto, 2000). What Japanization of English has involved is

the tendency of many higher education institutions to employ Japanese teachers
who teach the language the Japanese way instead of using approaches such as
communicative language teaching, which is believed to be a Western approach
and even a colonizing force (Phan, 2008). Now, how can such a tendency explain
privilege in language teaching in Japan?
Such a policy, one may argue, does not promote native-speakerism as it does not
privilege native-speaker teachers of English in Japan. Holliday (2005) defined
native-speakerism as an ideology in the language teaching profession
“characterized by the belief that ‘native-speaker’ teachers represent a ‘Western
culture’ from which spring the ideals both of the English language and of English
language teaching (ELT) methodology.” Considering the aforementioned idea of
the Japanization of language teaching, it looks like native-speakerism is not
relevant in the Japanese context. Such an argument is also consistent with what
some studies have had to offer. Stewart (2005), for instance, believed that such a
view has led to the deprofessionalization of native-speaker English teachers, who
are often regarded as temporary and replaceable (Nagatomo, 2012). As a result,
native-speaker teachers have been reported to face discrimination in hiring
practices and are offered short-term and non-renewable contracts (Hayes, 2011).
This paper does not intend to negate such arguments. It argues, however, that
there are more subtle practices in the field of language teaching in Japan which
may privilege certain speakers of English and marginalize others as language
teachers. The argument is grounded in the fact that although there are many
varieties of English (native and non-native) spoken in the world, conventional
ELT practices do not consider them all equally valid and hence privilege speakers
of certain varieties of English and not others.
2. The ideological choice

Teaching English as a foreign language in the age of globalization has to reflect
the new definitions for what it takes to be able to communicate internationally.
Not a long time ago it was an unquestionable fact that the purpose of ELT was to
enable learners to speak and use English like native English speakers (NESs) do
for the obvious reason that this gave the learners the ability to communicate with
NESs with ease (McKay, 2002). ELT, thus, has traditionally favored standard
American and British varieties of English. The inherent flaw in this argument,
however, is the assumption that all individuals who learn English as a foreign or
a second language are doing so in order to communicate with NESs. In other
words, what this viewpoint misses is that in a world that is constantly becoming
globalized, the vast majority of such learners may need to speak English to
communicate with other non-native English speakers (NNESs) rather than NESs
(Kirkpatrick, 2006). This very issue, along with findings of recent studies
regarding the changing status of the English language, has raised a lot of criticism
against the traditional approaches to ELT (Marlina, 2014; Matsuda, 2006; McKay,
2012).
Moreover, traditional ELT approaches have also raised ideological and political
concerns. As learners are constantly exposed to certain varieties of English
(mainly American and British), they end up believing in the supremacy of those

103
varieties over the others (Kirkpatrick, 2006). Along the same lines, there are
scholars who believe that the traditional approaches to ELT have promulgated a
form of linguistic imperialism (Phillipson, 2009) through which Western powers
have maintained their colonial positions even in the present time. The recognition
of new varieties of English as equally valid ones has been a reaction to traditional
ELT in this regard. As Saraceni (2015) concluded, such recognition has created the
belief among language teaching scholars that
“[l]earning English need not to be seen as a strenuous journey whose
ultimate destination is the achievement of ‘native-like’ status or a
linguistic ‘visa’ into a special ‘inner circle’. Learning English means,
above all, making it easier to take part, actively and critically, in the
practices and discourses that (re)present, (re)construct and (re-)shape the
global and local worlds we live in.” (p. 187)
In response to the criticism leveled against traditional ELT, other approaches have
emerged in the field that are more critical by nature and more sensitive to the
varieties of English spoken in the world. One such approach is English as an
international language (EIL).
The proponents of EIL argue that as in a global world there is a much higher
possibility for NNESs to communicate with other NNESs in English, language
teaching has to embrace the diversity of the Englishes used in the four corners of
the world. Emphasis must be placed here on equipping language learners with
the skills to negotiate for meaning with other speakers of English who come from
different linguistic and cultural backgrounds. Those approaches to language
teaching that emphasize an acceptance of different varieties of English thus cry
for providing students with more exposure to the diversity of the Englishes
spoken around the world (Rose & Montakantiwong, 2018). In such a context,
multilingualism and mutual intelligibility become the ultimate goal in
communication instead of native-like competency (McKay, 2012). The emphasis
of EIL on accepting different varieties of English and considering them as equally
valid leads to giving voice to speakers of those varieties which were once
considered inferior to native-speaker norms. Therefore, it can be regarded as a
critical pedagogical approach to teaching and learning the English language
(Pennycook, 2017). It strives to give voice not only to the speakers of such varieties
of English, but also to the teachers who teach English but do not have the privilege
of speaking it as their mother tongue.
Sharifian (2009) conceptualized EIL as a paradigm for thinking as well as research

and practice in language teaching. Such a paradigm can serve as an analytical tool
for educators and researchers to critically revisit the way they conceptualize
English and the way they teach English, particularly with reference to the way its
use has expanded globally in recent years (Marlina, 2013). EIL is therefore
primarily concerned with the way English is used in international settings and by
speakers coming from diverse linguistic and cultural backgrounds (Hino, 2019).
One important issue to bear in mind here is that EIL is not a single variety of
English. It is rather a framework that considers English “with its pluralized forms
[…] a language of international and therefore intercultural communication”
(Sharifian, 2009, p. 2). It also underlines the importance of linguistic negotiation

104
for achieving mutual intelligibility by speakers of English who bring with

themselves many different varieties of the language into the interaction. It is
against this backdrop that EIL does not prioritize native-speaker teachers of
English over other teachers because of the variety of English they speak. However,
this theoretical potential of EIL for tackling the native-speaker privilege in ELT
has not always translated into practice (Matsuda, 2017). Theory and practice in
language teaching have not always been convergent in language teaching, and for
various reasons this mismatch has always been present in different areas within
language teaching, including teacher education, assessment, and materials
development (Tomlinson & Masuhara, 2018). Back now to the main concern of the
present study, the question is whether or not ELT in Japanese higher education is
concerned about native-speaker privilege and has the intention to remedy it by
resorting to EIL or other similar approaches.
3. Methodology and data collection

The present paper reports on a preliminary stage of a larger study investigating
the issue of native-speaker privilege in teaching English in Japanese higher
education. In this preliminary stage, the main goal has been to explore the
prevalent policies of universities in Japan in hiring English language teachers. The
question of whether or not privilege is a relevant concern in teaching English in
Japanese higher education institutions can be addressed with a number of
different approaches. Surveys and interviews targeting language teachers from
different backgrounds and cohorts as well as program directors can be among
them. The researcher is aware of the need to collect data from all stakeholders
involved in teaching English in Japanese universities and intends to do that in the
next phases of the study. The present paper, however, is delimited to the
exploration of English-teacher hiring policies by doing document analysis on job
advertisements posted by Japanese universities on an online portal hosted by
Japan Technology and Science Agency, which serves as the main space for looking
for academic jobs and positions in Japan.
For this purpose, all language-teaching job advertisements posted by Japanese

universities on the portal mentioned above (accessible at https://jrecin.jst.go.jp)
in the timespan of early October to early November were documented and
analyzed by the researcher. The researcher focused on the qualifications the job
advertisements required the applicants to possess as well as other factors,
including the type of university (national, public, or private), department
(specialized in language teaching or otherwise), job type (part-time or full-time),
and employment status (tenured or non-tenured). Twenty-four job
advertisements for language-teaching positions were documented and analyzed.
Thematic analysis, which is one of the most frequently used techniques in the
analysis of qualitative data (Creswell & Poth, 2018), was employed to explore the
required qualifications in the job advertisements. The whole dataset was first
coded and then the relationships between codes were explored. As a result, a
number of themes emerged that indicated the tendencies of Japanese universities
in hiring English teachers.
The data also underwent descriptive statistical analysis to determine the

frequency of each of the emerged themes. Furthermore, the researcher used

105
cross-tabulation to test possible relationships between the identified variables.

Finally, a chi-square test was performed to determine whether the observed
relationships between variables were statistically significant.
4. Results and discussion

The first stage in the documentation of the whole dataset led to the identification
of different types of variables that were referred to in the job advertisements.
These variables and their types are introduced in Table 1.
Table 1: Types of variables identified

Type Variables
Institutional University type, department
Job-related Job type, employment status
Qualifications Education, professional skills, interpersonal skills,

nationality, Japanese language skills, English language
skills, teaching experience, residence status, aspiration
As Table 1 indicates, the 13 variables identified in the job advertisements posted

on the portal could be categorized into three major types, namely institutional,
job-related and qualifications. There were also other categories in the
advertisements, including documents to be submitted or the method of
submission, but as these issues were not relevant to the issue of privilege, they
were excluded from the list of variables. From this list, institutional and job-
related variables were included in all job advertisements in the dataset. However,
the variables in the qualifications category were present in some advertisements
and absent in others. That is to say, the qualifications required differed from one
job advertisement to another. The variables included in Table 1, therefore, contain
all variables identified in the whole dataset.
To estimate the priorities of Japanese universities in hiring English teachers, the

frequency of occurrence of all variables in the qualifications category was
calculated. However, for the sake of space, Tables 2 and 3 present the calculated
frequencies and percentages only for the two variables that are more directly
related to the issue of privilege with regard to the variety of English spoken by
the teachers. These two variables are nationality and English language skills.
Whereas some of the job advertisements did not mention looking for any
particular nationality, others stated that they were exclusively hiring teachers
either from countries in which English is spoken as the first language (NESs) or
from Japan. Moreover, in terms of required English proficiency, whereas some of
the advertisements did not refer to any required proficiency level for the
prospective applicants, others either required sufficient proficiency or native-like
proficiency. Both of these variables (nationality and English language skills) were
relevant to the theme of this research, which is the privilege arising from the
variety of English spoken by teachers. Table 2 summarizes the calculated
frequencies for three representations of the nationality variable.

106
Table 2: Frequency of variables related to nationality of prospective applicants

Nationality Frequency Percentage Cumulative
percentage
Not mentioned 9 37.5 37.5
Native English speaker 13 54.2 91.7
Native Japanese speaker 2 8.3 100
Total 24 100
As the figures in Table 2 indicate, more than half of the job advertisements (54.2%)
required prospective applicants to be native speakers of English. There was no
indication in the advertisements as of what being a native speaker of English
means. For instance, it was not clear whether a teacher born in a country in which
English is not the first language but raised in an English-speaking country would
be regarded as a native speaker. Despite this, 13 universities (54.2%) sought only
native speakers for their English-teaching positions. In addition, there were two
universities (8.3%) that looked for Japanese applicants exclusively, whereas the
remaining nine universities (37.5%) did not refer to nationality as a criterion for
employment. Table 3 shows the frequency of three different representations of the
English language skills variable.
Table 3: Frequency of variables related to English language skills of prospective

applicants
English language skills Frequency Percentage Cumulative
percentage
Not mentioned 11 45.8 45.8
Sufficient proficiency 7 29.2 75
Native-like proficiency 6 25 100
Total 24 100
As Table 3 suggests, 11 advertisements (45.8%) took for granted that an applicant

for an English-teaching job must be proficient in the language. This can probably
be explained by the fact that these universities were among those that looked for
native-speaker teachers. From the remaining universities, seven (29.2%) looked
for applicants with English skills sufficient for teaching the language, and the other
six (25%) required applicants with native-like proficiency in English. The
advertisements in this latter group, however, did not specify what exactly they
meant by native-like proficiency. What is interesting here is that besides the 13
universities which exclusively sought to hire native-speaker teachers (Table 2), six
more universities had the normative requirement that applicants must be able to
speak English similar to native speakers of English. This reference to native-
speaker norms English use lies at the center of the argument for the presence of
privilege with reference to the variety of English spoken by teachers in Japan and
will be discussed in the subsequent sections of this paper.

107
In line with the goal of the study and after the descriptive analysis of the data, the
researcher sought to find relationships between the qualification variables and the
institutional and job-related variables. The data were cross-tabulated and a
chi-square test was performed to evaluate the significance of potential
relationships among the variables. The results indicated that there was no
significant relationship between any of the qualification variables and the
institutional and job-related variables. This means, for example, that the type of
university or the department did not influence the tendency to hire native-speaker
teachers or otherwise. There was only one exception to this, however. The analysis
indicated that there was a significant relationship between the nationality variable
and the employment status variable. Tables 4 and 5 respectively summarize the
results of the cross-tabulation and chi-square test for these two variables.
Table 4: Cross-tabulation of nationality and employment status

Employment status
Non-tenured Tenured Total
Nationality Not mentioned Count 5 4 9
Expected count 7.1 1.9 9.0
% within nationality 55.6% 44.4% 100.0%
Native English Count 13 0 13
% within nationality 100.0% 0% 100.0%
Native Japanese Count 1 1 2
Expected count 1.6 .4 2.0
Total Count 19 5 24
Table 4 summarizes the distribution of the three representations of the nationality

variable in non-tenured and tenured English-teaching positions. Of particular
interest to the topic of this research is the fact that all of the universities that looked
at hiring native English teachers had non-tenured positions available. On the
other hand, 44.9% of the universities that did not mention a particular nationality
as an employment requirement offered tenured positions to successful
candidates. Table 5 shows the results of the chi-square test to explore whether this
observed relationship is statistically significant.
Table 5: Chi-square tests to explore the relationship between nationality and

employment status
Value df Asymp. Sig. (2-sided)
Pearson chi-square 7.495a 2 .024
Likelihood ratio 9.426 2 .009
Linear-by-linear association 1.542 1 .214
N of valid cases 24

108
Value df Asymp. Sig. (2-sided)

Pearson chi-square 7.495a 2 .024
Likelihood ratio 9.426 2 .009
Linear-by-linear association 1.542 1 .214
a. Four cells (66.7%) have an expected count of less than 5. The minimum
expected count is .42.
To interpret the results of the chi-square test, a null hypothesis was first
developed. According to the null hypothesis, there is no significant relationship
between the nationality and employment status variables. As can be seen in
Table 5, however, the p value in the calculated figures is less than 0.05 (p = 0.024),
and therefore the null hypothesis is rejected. In other words, universities that
preferred to hire NESs offered the applicants only non-tenured positions and this
relationship was statistically significant.
What can be inferred from these results is twofold. In the first place, the results
indicate that hiring policies of Japanese universities grant privilege to native
speakers of English. Teachers in this subgroup have more job opportunities to
choose from compared to their colleagues who are either Japanese or
non-Japanese and non-native speakers of English. This seems to be in line with
what previous research on native-speakerism has had to offer (Shiroza, 2020).
Furthermore, the data also implies that in the job advertisements documented in
this study, English teachers are regarded as either native speakers or Japanese
applicants who are obviously non-native speakers of English. In other words, a
third possible category, namely teachers who are neither native speakers of
English nor Japanese, seems to have been excluded from the advertisements as
well as previous research.
The second issue inferred from the results is that privilege cannot be regarded as
an absolute concept. Based on the results, although there appears to be privilege
for native-speaker teachers with reference to the nationality variable in the hiring
policies, employment status of teachers as referred to by universities proves the
contrary. Native-speaker teachers were mostly offered non-tenured and
limited-term contracts, which left them in a disadvantaged position compared to
their Japanese colleagues. This finding corroborates what previous research has
found about discrimination in the workplace against different groups of foreign
university teachers (Hayes, 2011; Nagatomo, 2012; Nagatomo, Brown, & Cook,
2020). The problem seems to be clear now: privilege exists in the field of language
teaching in Japanese higher education. The following two sections deal with the
ways this problem can be further studied and addressed.
5. The relevance of studying teacher identity to challenge privilege

Understanding policies behind language teaching practices is indeed key to
exploring privilege in language teaching. Of equal importance, however, is
making sense of who language teachers are and how they construct or perceive
this privilege (Varghese, Morgan, Johnston, & Johnson, 2005). Therefore,
understanding “the professional, cultural, political, and individual identities
which [language teachers] claim or which are assigned to them” (Varghese et al.,

109
2005, p. 22) is the very first step in studying privilege in the language classroom.
Language teacher identities are discursively constructed by the self as well as the
discourses in the society and through the teachers’ interactions with their
students, pedagogy, curriculum, and assessment (Glodjo, 2017).
Duff and Uchida (1997) studied language teacher identity in tertiary education in
Japan and found that although their participants considered themselves teachers
of the English language and not the culture, they were actually teaching culture
in line with their sociocultural identification. The study found that “language
teachers are very much involved in the transmission of culture, and each selection
of videos, newspaper clippings, seating plans, activities, and so on has social,
cultural, and educational significance” (Duff & Uchida, 1997, p. 476). What is
more, as teaching the elements of culture is embedded in the teaching of the
subject matter in the classroom, it positions students in a particular way either
intentionally or unintentionally. The very fact that teachers’ intentional or
unintentional pedagogical choices and preferences can influence student
positioning and possibly identity formations opens up the issue of privilege in the
language classroom.
A number of scholars (Block, 2014; Vandrick, 2014; Yazan & Rudolph, 2018) have
addressed the issue of identity and privilege with reference to Bourdieu (1986)
notion of social reproduction which results from “the cultural and symbolic
capital in a particular field and habitus” (Glodjo, 2017). It is assumed in these
studies that the dominant group that possesses the social and cultural capital
defines normality in the habitus. That is to say, the values, beliefs, and lifestyle of
the dominant group become the norm in a habitus, and this normality is
reproduced constantly by the dominant group. As it is reproduced constantly, it
becomes legitimate to the extent that it can no longer be easily discarded or even
criticized. Similarly, in the social context of the language classroom, the social
reproduction of the values of the dominant group creates a “hidden curriculum”
which shows “the tacit ways in which knowledge and behavior get constructed,
outside the usual course materials and formally scheduled lessons” (McLaren,
2009, p. 75). As a result, those outside the dominant group are considered as other
and are silenced to the extent that they are expected to follow the standards of the
dominant group. In other words, individuals outside the dominant group are
conditioned to be subordinate to those who possess the social and cultural capital.
They are conditioned to “behave in certain ways, make certain interpretations
about their place in society and the way the world functions, and adjust their
expectations of their future opportunities accordingly” (Sanders & Mahalingam,
2012, p. 114).
As discussed earlier, the role of English is constantly changing in today’s

globalized world. While not a long time ago it was considered as an
unquestionable fact that learners of English have to imitate and emulate NESs in
order to learn the language, recent critical approaches to language education,
including EIL, have shown that native-like proficiency is neither a requirement
nor a desired attribute for individuals who seek to communicate in English
internationally. The number of NNESs who need to interact with other NNESs is
currently increasing and this entails raising awareness of the English spoken by

110
NNESs rather than narrowly focusing on how NESs use the language and see the
world. This, however, is not consistent with the realities of the language
classroom. Evidence for this claim can be found in the textbooks and audiovisual
language-learning materials published by well-known publishers, as well as
international tests of English such as TOEFL and IELTS. Many published
language-learning materials still primarily manifest the culture and lifestyle of a
dominant group in Anglophone countries (Kazemi, Asadi & Davatgari, 2017).
Text and audiovisual supplements used in these materials almost exclusively
uphold standard American, British, and Australian varieties of English as valid,
normal, and standard and that must to be followed by English learners. The same
is true about international tests of English. It is rare, if not impossible, for such
tests to include Indian, Malaysian, South African, or even Arabic varieties of
English. Such varieties of English seem to have been totally neglected and left
behind. However, in the real world, a Japanese learner of English may need to
interact with speakers of English from all these countries (and more) and hence
has to be prepared for using and understanding English in such contexts.
For various historical reasons, the countries in which English is spoken as the first
language hold economic, political, military, and even cultural dominance in many
parts of the world. This has led to a state in which they are looked up to by many
nations as having high levels of social and cultural capital. As they are wealthy
nations, they have been controlling media as well, and this has led to the
reproduction of the status associated with them. This may explain why, despite
the fact that English is an international language in the world now, it is taken for
granted that the native-speaking countries have the ownership of English. Such a
discourse obviously operates in favor of a system that seeks to create outsiders
(speakers of non-native varieties of English) that are subordinate to particular
ingroups (NESs). One important question seems to have remained unanswered
yet: how can the language classroom be liberated from such biased classifications?
One way to address this issue is to resort to critical pedagogy.
Giroux (2009), who is believed to be the founding father of critical pedagogy

alongside Freire, argued that a prerequisite to such a liberation is the presence of
teachers who can critically reflect on their positionality, identity and privilege. He
calls such a teacher a “transformative intellectual […] who exercises forms of
intellectual and pedagogical practice […] arguing that schooling represents both
a struggle for meaning and a struggle over power relations” (Giroux, 2009, p. 439).
This has clear implications for teacher-education research showing the need for
training teachers who can critically reflect on the way their identity is constructed,
and value differences between their students and themselves as resources rather
than deficits (Glodjo, 2017).
6. A solution: EIL as critical pedagogy

Central to the idea of critical pedagogy (Freire, 1999) is transforming the
conception of knowledge as apolitical and neutral in education. It aims at
empowering the students by teaching them how to challenge and defy
domination and dominating discourses of power in society. From a linguistic
point of view, EIL shares this concern as well. It strives to give voice to speakers
of non-native varieties of English. Its point of departure is the contention that all

111
varieties of English are equally valid, and English should be taught without an
undue reference to how NESs use it. Also, it emphasizes the fact that not all people
who wish to learn English have the intention to learn and adopt the Anglo-
American culture, lifestyle, and worldview. They may well have instrumental
motivation to learn the language as a tool for communication with other speakers
of the language around the world. Pennycook (2017) thus argued that “[n]either
the version of language produced by the discourse of ElL, nor the discourse of ElL
itself can be accepted as neutral, for both language and discourse always imply a
politics” (p. 301). The politics EIL stands against are that of superiority and
dominance of certain varieties of English and their speakers (including the
teachers who speak them and teach them) over the others. Broadly defined,
critical pedagogy is a desire for bringing about social change through education,
and so is the case with EIL. Schools, and the language classroom by extension,
therefore cannot be regarded as a space for transmitting a neutral body of
knowledge to the students or language learners. Akin to critical pedagogy that
takes schools as areas for constant social, political, and cultural struggle, EIL
rejects the traditional approaches of language teaching that deal with the
language-learning process merely as a psycholinguistic one isolated from its
social and cultural context.
Not reckoning the principles of critical pedagogy, Giroux (1998) argued that we
need to oppose the notion of curriculum knowledge as sacred and impeccable and
explore the many reasons why certain types of culture and knowledge are given
precedence in educational settings. From an EIL perspective, for instance, the
question would be why certain accents of English from particular varieties, values
and cultures associated with them and their sociolinguistic norms are considered
“standard” while others are not. In such an environment, teachers need to be what
Geroux called “transformative intellectuals”. A similar statement was made by
EIL proponents (Hino, 2019; McKay, 2012; Renandya, 2012), who argued that
teachers in this changing sociolinguistic landscape of English need to promote
multiculturalism in the language classroom and decentralize the native-speaker
norms governing language use. That is what makes an exploration of the teachers’
perceptions and classroom practice relevant and significant.
7. Conclusion
This paper addressed the issue of privilege and how it may exist in teaching
English at higher education institutions in subtle and less noticeable ways.
Particularly in the case of Japan, despite research showing that native-speaker
teachers of English in Japanese universities do not enjoy any privilege and are
even faced with discrimination, the results of this study showed the contrary. It
was found that privilege in hiring policies of Japanese universities lies in the
variety of English teachers speak The native-speaker varieties of English
showcased in language-teaching materials used in Japanese higher education
institutions as well as elsewhere result in the belief that the desired varieties of
English to be learned by learners are those spoken by NESs. As a result, Japanese
universities are more interested in hiring native-speaker teachers. Whereas
traditional ELT approaches have by and large remained insensitive to this issue,
EIL and other similar approaches have addressed it. While privilege may well
exist in other subtle ways that future research is encouraged to explore, this study

112
stayed focused on privilege arising from native-speaker-oriented hiring policies.

It suggested the adoption of EIL principles to teach English in higher education as
it both offers a more realistic outlook towards the way English is used in today’s
globalized world and challenges the privilege it produces for speakers of certain
varieties of English. In this sense, EIL can be regarded as a critical pedagogical
approach to teaching English as it strives, in part, to bring about equity and
inclusiveness in the profession.
8. References
Aspinall, R. W. (2006). Using the paradigm of “small cultures” to explain policy failure in
the case of foreign language education in Japan. Japan Forum, 18(2), 255–274.
doi:10.1080/09555800600731197
Block, D. (2014). Social class in applied linguistics. New York: Routledge.
doi:10.4324/9781315871141
Bourdieu, P. (1986). The forms of capital. In D. W. O’shea, & J. G. Richardson (Eds.),
Handbook of theory of research for sociology of education (pp. 241–258). New York:
Greenwood Press. doi:10.2307/2069964
Creswell, J. W., & Poth, C. N. (2018). Qualitative inquiry & research design: Choosing among
five approaches. Los Angeles: Sage.
Crystal, D. (2003). English as a global language. Cambridge: Cambridge University Press.
Duff, P. A., & Uchida, Y. (1997). The negotiation of teachers’ sociocultural identities and
practices in postsecondary EFL classrooms. TESOL Quarterly, 31(3), 451–486.
doi:10.2307/3587834
Freire, P. (1999). Pedagogy of the oppressed. New York: Continuum.
Giroux, H. A. (1998). Education in unsettling times: Public intellectuals and the promise
of cultural studies. In D. Carlson, & M. Apple (Eds.), Power/knowledge/pedagogy, the
meaning of democratic education in unsettling times (pp. 40–60). CO: West view Press.
doi:10.4324/9780429498060-1
Giroux, H. A. (2009). Teacher education and democratic schooling. In A. Darder,
M. P. Baltodano, & R. D. Torres (Eds.), The critical pedagogy reader (pp. 438–459).
New York: Routledge.
Glodjo, T. (2017). Deconstructing social class identity and teacher privilege in the second
language classroom. TESOL Journal, 8(2), 342–366. doi:10.1002/tesj.273
Hashimoto, K. (2000). ‘Internationalisation’ is ‘Japanisation’: Japan’s foreign language
education and national identity. Journal of Intercultural Studies, 21(1), 39–51.
doi.org/10.1080/07256860050000786
Hayes, B. E. (2011). Hiring criteria for Japanese university English-teaching faculty. In
T. Skutnabb-Kangas (Ed.), The native speaker English teacher: From exclusion to
inclusion. Linguistic diversity and language rights series (pp. 270–287). Abingdon:
Multilingual Matters. doi:10.21832/9781847698704-013
Hino, N. (2019). EIL education for the expanding circle: A Japanese model. Oxon: Routledge.
Holliday, A. (2005). The struggle to teach English as an international language. Oxford: Oxford
University Press.
Kazemi, S., Asadi, A. N., & Davatgari, A. H. (2017). Manifestations of globalization and
linguistic imperialism in English language teaching and materials preparation:
Ideology in the international ELT textbooks. Research in English Language Pedagogy,
5(2), 223–246.
Kirkpatrick, A. (2006). Which model of English: Native-speaker, nativized, or lingua
franca. In R. Rubdy, & M. Saraceni (Eds.), English in the world: Global rules, global
roles (pp. 71–83). London: Continuum. doi:10.1093/applin/amn015

113
Marlina, R. (2013). Teaching English as an international language: Voices from an Australian

university classroom (Unpublished doctoral dissertation). Monash University,
Melbourne, Australia.
Marlina, R. (2014). The pedagogy of English as an international language: More reflections
and dialogues. In R. Marlina, & R. A. Giri (Eds.), The pedagogy of English as an
international language: Perspectives from scholars, teachers, and students (pp. 1–19).
Cham: Springer. doi:10.1017/s0266078419000488
Matsuda, A. (2006). Negotiating ELT assumptions in EIL classroom. In J. Edge (Ed.),
(Re)locating TESOL in an age of empire (pp. 158–170). Basingstoke: Palgrave
MacMillan. doi:10.1057/9780230502239
Matsuda, A. (2017). Preparing teachers to teach English as an international language. Bristol:
Multilingual Matters. doi:10.21832/9781783097036
McKay, S. L. (2002). Teaching English as an international language: Rethinking goals and
approaches. Oxford, England: Oxford University Press.
McKay, S. L. (2012). Teaching materials for English as an international language. In
A. Matsuda (Ed.), Teaching English as an international language: Principles and
practices (pp. 70–83). Bristol: Multilingual Matters. doi:10.21832/9781847697042
McLaren, P. (2009). Critical pedagogy: A look at the major concepts. In A. Darder,
M. P. Baltodano, & R. D. Torres (Eds.), The critical pedagogy reader (pp. 61–83). New
York, NY: Routledge.
Nagatomo, D. H. (2012). Exploring Japanese university English teachers professional identity.
Bristol: Channel View Publications. doi: 10.21832/9781847696489
Nagatomo, D. H., Brown, K., & Cook, M. (2020). Foreign female English teachers in Japanese
higher education: Narratives from our quarter. Tokyo: Independent Publication.
doi:10.47908/11
Pennycook, A. (2017). The cultural politics of English as an international language. Milton:
Taylor & Francis. doi:10.4324/9781315225593
Phan, L. H. (2008). Teaching English as an international language: Identity, resistance and
negotiation. Clevedon: Multilingual Matters. doi:10.21832/9781847690500
Phillipson, R. (2009). Linguistic imperialism. Oxford: Oxford University Press.
doi:10.4324/9780203857175
Renandya, W. A. (2012). Teacher roles in EIL. The European Journal of Applied Linguistics
and TEFL, 1(2), 65–80.
Rose, H., & Montakantiwong, A. (2018). A tale of two teachers: A duo ethnography of the
realistic and idealistic successes and failures of teaching English as an
international language. RELC Journal, 49(1), 88–101.
doi:10.1177/0033688217746206
Sanders, M. R., & Mahalingam, R. (2012). Under the radar: The role of invisible discourse
in understanding class-based privilege. Journal of Social Issues, 68(1), 112–127.
doi:10.1111/j.1540-4560.2011.01739.x
Saraceni, M. (2015). World Englishes: A critical analysis. New York: Bloomsbury Publishing.
doi:10.1515/jelf-2016-0009
Seargeant, P. (2011). English in Japan in the era of globalization. Basingstoke: Palgrave
Macmillan. doi:10.1057/9780230306196
Sharifian, F. (2009). English as an international language: Perspectives and pedagogical issues.
Clevedon: Multilingual Matters. doi:10.21832/9781847691231-004
Shiroza, S. (2020). Changing language, continuing discourse: A shift toward ELF and
persistent native-speakerism in Japan’s ELT policy. In R. A. Giri, A. Sharma, &
J. D'Angelo (Eds.), Functional variations in English. Multilingual education. Springer,
Cham. doi.org/10.1007/978-3-030-52225-4_18

114
Stewart, A. (2005). Teaching positions: A study of identity in English language teachers in

Japanese higher education (Unpublished PhD thesis). University of London,
London, England.
Tomlinson, B., & Masuhara, H. (2018). The complete guide to the theory and practice of materials
development for language learning. Oxford: Wiley Blackwell.
Vandrick, S. (2014). The role of social class in English language education. Journal of
Language, Identity and Education, 13(2), 85–91. doi:10.1080/15348458.2014.901819
Varghese, M., Morgan, B., Johnston, B., & Johnson, K. A. (2005). Theorizing language
teacher identity: Three perspectives and beyond. Journal of Language, Identity &
Education, 4(1), 21–44. doi:10.1080/15348458.2014.901819
Yazan, B., & Rudolph, N. (2018). Criticality, teacher identity, and (in)equity in English language
teaching. Issues and implications. Cham: Springer International Publishing.
doi:10.1007/978-3-319-72920-6

115

Vol. 19, No. 10, pp. 115-133, October 2020
The Power Sources and Influences of Secondary

School Principals in Eastern Ethiopia
Birhanu Sintayehu
Haramaya University, Haramaya, Ethiopia
https://orcid.org/0000-0002-7935-9264
Abstract. This study aimed to critically examine the power sources and
influences of school principals in secondary schools of Eastern Ethiopia.
A descriptive survey research design was employed to carry out the
study. The participants of the study were 145 teachers, 78 principals,
and 41 supervisors who were selected by using stratified and random
sampling techniques. The researcher adopted descriptive and inferential
statistics to make sound interpretations of data. The results revealed that
school principals were mostly used expert, legitimate, and reward
sources of power. Likewise, school principals have predominantly
exercised a positional basis of power rather than personal power. There
was a significant statistical difference in power sources of school
principals regarding positions, gender, and service years. The findings
also showed that school principals dominantly practiced proactive
influencing tactics. Moreover, findings indicated that subordinates
carried out school principals' compliance to obtain a prize or avoid
punishment by applied reactive influence tactics. The study further
discovered that subordinates were inclined to resist school principals'
influence. This study suggests secondary principals should rethink how
power is managed and deployed to make sound influence over
subordinates to assure quality education. Hence, the results of the study
may serve as a springboard to improve secondary school leadership and
equip novice teachers to bring them a principalship position. Plus, this
study may provide a clear picture for policymakers, scholars, and
government officials to support and retain principals for long-term
school improvement, and it may also a theoretical benefit for future
research on the area of study.
Keywords: Eastern Ethiopia; influences; leadership; power sources;

secondary school; school principals
1. Introduction
Power and influences are the most important areas of interest within the field of
educational leadership. Leadership is an individual's ability to influence,
motivate, and allow others to contribute to an organization's effectiveness and
success (Leithwood, Harris & Hopkins, 2020; Northouse, 2013). To do so, a

116
leader uses power as a means of influence to achieve institutional objectives

(Alsobaie, 2015). Leadership is an art of utilizing power to influence others
(Aslanargun, 2011). Hence, power and leadership are different sides of the same
coin. So, power is a leader’s capacity to inspire, direct, lead, and motivate others.
In brief, power is the ability of a leader to influence subordinates to achieve
institutional aims (Nelson & Quick, 2013). To influence others leader has to be
grip power from true sources. People often get confused on interrelated
terminologies i.e. leadership-power-influence. In short, Aslanargun (2011)
described the relationship between influence, authority, and hierarchy as
follows:
Power is the potential of “A” to influence “B” despite resistance.
Influence is the result of “A’s” changeable act over “B’s” behavior. An
authority is bound to position in an organization based on power and
legitimacy. Control is compliance behavior that “A” manages over “B”.
It is the success of the influence process. Hierarchy refers to the formal
structure of an organization and emphasizes the structure, roles, and
division of labor (p. 7).
In brief, power represents the ability to induce someone to do something

whereas influence is the exercise of that power. Power is the potential to
influence, while the influence is power in action. “Authority is the leader’s
power to influence others in a specific way, and an important basis for the
influence of under formal organization” (Yukl, 2010: p.185). Although power
and influence are interrelated in complex ways, they can be separate
constructions (Lunenburg, 2012).
Where does this power come from? What gives a person or group influence over
others? Power can come from many sources. Based on the concept of leader
power, Yukl (2010) pointed out that “the potential influence derived from a
leader's position in the organization is the power of the position, and includes
legitimate, rewarding information, coercive and ecological, while the
characteristic of those in a leadership position is personal power, and includes
expert and referent power (p. 185)”.
There are two sources of power which namely positional power and personal
power (Yuki, 2010). The sources of power that derived from a leader's position
are positional power which includes legitimate, rewarding information, coercive
and ecological, and while the character of those in a leadership position is
personal power, and it includes expert and referent power (Northouse, 2013).
Positional power specified that reinforcement and punishment depended on the
formal roles of the agent in the organization. Personal power mostly relies on
one’s charisma, experience, and information based on the characters of the
agents (Krause, 2004). Details of each power base and sources are presented in
Table 1.
Table 1: Different types of power

Position power Personal power
Legitimate sources of power Expert sources of power
Information sources of power Referent sources of power

117
Reward sources of power

Ecological sources of power
Coercive sources of power
Source: Yukl (2010, p. 155)
Pertaining to power sources, an effective and efficient leader mostly used

personal power than positional power (Northouse, 2013). However, "it remains
important and interacts in a complex way with the personal power to determine
the influence of a leader over his followers” (Weinstein, Raczynski & Peña, 2020:
p. 72). Personal sources of power are more closely related to employees' job
satisfaction, organizational commitment, and employee job performance than
organizational power sources. Furthermore, true leaders are likely to use power
subtly and carefully that minimizes differences in status and avoids threats to
the target person's self-esteem (Northouse, 2013).
School principals applied different power sources in different circumstances

with different situational factors. Most studies of power sources have found that
legitimate, rewarding, and coercive powers were incompatible and that
correlations with criteria are generally negative or insignificant rather than
positive (Goshu & Woldeamanuel, 2019). Lunenburg (2012) argues that
legitimate power could lead to disappointment and resistance if it is not
supported by expertise or used excessively. As a result, when a leader frequently
relies on legitimacy power, the possible outcome of subordinate is resistance. It
has also been found that the powers of experts and referents have a substantial
influence on all interactive groups. The formal power only influences
subordinates, and that there is little or no influence of the reward and coercive
power on external groups in developed organizations (Singh, 2009).
As pertains to school principals' influencing process, tactics, and outcomes of

their influence over subordinates, and to be effective as a leader, it is essential to
guide followers to comply with given assignments. School principals influence
not only subordinates but also students (Dhuey & Smith, 2018). To influence
others, leaders use four very fundamental influencing tactics that may differ in
terms of purpose. These are impression management tactics, political tactics,
proactive tactics, and reactive (or resistance) tactics (Yukl, 2010). Any tactic can
fail if it is not used skillfully and ethically or if it is not appropriate for the
purpose and situation of influence.
1.1 Literature Review

Sources of Power
Most scholars identified that there are two broad types of power; positional
power and personal power (Yukl, 2010). An influence derived from the position
of a leader in an organization is positional power that consists of the legitimate
power, the power of reward, and the power of coercion, while the characteristics
of a person who holds a position in leadership are personal power sources that
comprise of information and ecological power (Yukl, 2010). Positional power
specified that reinforcement and punishment depended on the formal roles of
the agent in an institution. On the other hand, personal power typically relies on
one’s charisma, experience, and information which is directly related to the

118
characters of the agents. Details of each power base and sources are presented as
follows.
Legitimate Power
Most scholars agree on this source of power as the power which derives from
the position itself and formal authority overwork. This source of power is
awarded by written or verbal contracts from higher officials, and it describes the
responsibilities of the manager, and is based on this position sanctioned,
authorized, and exalted (Singh, 2009). The amount of legitimate power is
determined through the extent of the power the person possesses. So,
experienced leaders are inclined to have more power than novice managers and
vice versa (Northouse, 2013).
Reward Power
Reward power is the manager's ability to award or hold back prize like money,
privilege, endorsement, or status in itself carries official power (Singh, 2009).
Likewise, this power mainly arises from grasping over tangible benefits, and if
what a leader presenting as a prize is of no value to an individual, it is likely not
to influence behavior (Northouse, 2013).
Coercive Power
This is the opposite of reward power. This source of power is not only managing
prizes but dealing with penalties. So, the coercive power of a leader over his
follower is based on the right over the penalty, which differs significantly
between different types of organizations (Northouse, 2013). Subordinate mostly
believe that he/she can be deprived of something if he/she does not comply
(Singh, 2009). When leaders using this power in an institution could not
implement sanctions such as arming and shooting, however, it can indirectly
bring cancellation of the privilege of workers such as additional payments, a day
off (Yukl, 2010). In brief, the manager executes coercive power either by taking
away the most important thing from his/her suburbanites or by providing
harmful things that may hurt him/her.
Referent Power
The power of referral arises from the aspiration of someone to please an agent to
whom they have strong feelings of adore, respect, and faithfulness (Cogaltay,
Yalcin & Karadag, 2016). These charismatic leaders positively influence others
since they are respected and admired by their followers. Likewise, the power of
reference allows a leader a shared identity, the personification of the personality,
the cult of the hero, the shared culture, or idolatry are some of the sources of
power (Singh, 2009). Therefore, the advantage of having a reference power is
that it can induce people to do things that may not translate into a tangible
reward; the reward comes from the relationship with the charismatic person.
Expert Power
Many scholars agree that this source of power arises from a person's know-how,
knowledge, or talent. Perception of people on an expert as he/she has superior
knowledge, skill, and experience on a specific issue than they possess
(Aslanargun, 2011). If a problem is fixed enduringly or others train to fix it on
their own, the agent's experience is no longer valuable. Most challenging in any

119
organization is substituting experienced personal and expert power that he/she

possesses. Experience is a basis authority as long as people depend on the
agent's pieces of advice and services.
Information power
This source of power implies figuring out vital information, and then
organizing, analyzing, managing, dissemination to others by using his/her
position, and communication skill, and network. Information is sources of
power in the 21st century, having this power double advantage to the leader to
lead a dynamic world (Tran, 2020).
Ecological power
This power derived from moral, Christian values and Confucian values by the
principle of doing good for others as what you would like them to do to you
”(Singh, 2009; Trinh, Pham, Cao, Nguyen, Nghiem & Tran, 2019). It is once able
to control physical surroundings, technology, assessing, and scanning the work
environment, and sometimes it is known as situational engineering or ecological
control.
Investigating the school principals’ power sources and influences is a continuing

concern within the field of educational administration in Ethiopian secondary
schools, yet there is insufficient research that includes their perspective. School
principals can play a major role in engaging the school-community and other
shareholders to participate in improving school (Mitchell, 2015). Despite the
importance of school leadership, there remains a paucity of evidence on the
school principals of Ethiopia. The main concern in Ethiopian schools is quantity
rather than quality (Semela, 2011), and school leaders face more pressure to
increase and maintain enrollment numbers than keeping quality education.
Among the countless problems for the deteriorating education quality in
Ethiopian secondary is school principals’ related problems. The enrolment rate
of students from lower primary to higher education is accelerating, but the
quality issue is under questing (Goshu & Woldeamanuel, 2019). Most scholars
agreed on fundamental causes for deteriorating education quality is school
leadership-related problems.
One of the greatest challenges to the school leadership is a fatalistic picture for
school principals. School principals lack the autonomy of administering funds,
mobilizing resources, and determining important issues related to teacher
recruitment and training (Mitchell, 2015). Utmost challenges for this may be that
school principals are not aware of their sources of power and influencing
process. In the long Ethiopian education history, there were no formal
qualification criteria to lead schools, and school leaders simply elected
appointed by government officials (Tekleselassie, 2005). Due to the reasons, the
government should take first place for the failure of the quality of schools,
school leadership, or principal ship (Goshu & Woldeamanuel, 2019). Another
main challenge faced by many schools is the school principals’ improperly
excising power and authority over subordinates (Goshu & Woldeamanuel,
2019). Further, the school management has a strong impact on the job
satisfaction of subordinates especially teachers (Cogaltay et al., 2016). Overall,

120
these studies highlight the need to examine the school principals’ power sources,
and the influence of school principals in Eastern Ethiopian secondary schools.
This study aims to fill the following research gaps in the context of Eastern
Ethiopia secondary schools. First, there has been no detailed investigation of the
mentioned concern, as stated by Mitchell (2012:12: p. 3), “the role of principals in
educational leadership particularly proper application of power and influences
is a vast field of study, yet little has been written from the perspective of
Ethiopia”. The impact of educational expansion on the quality of secondary
education is understudied, particularly for school leadership. For the past 20
years, the Ethiopian education system has been growing fast, and despite this
expansion, however, it created a slew of new problems, such as funding
shortages, and a deterioration of quality (Mitchell, 2015). Consequently,
concerns for the quality of education under questioned because of rapid
expansions of schools Minister of Education (MoE, 2008). Thus, this study
provides new insights into power sources and influences of school principals
and its implication on education quality.
Finally, to design a framework for the study, it is essential to differentiate leader

power sources, influencing tactics, processes, and outcomes. There are three
main influencing tactics that any leaders can possess to lead institutions like
political, proactive, and reactive tactics. Furthermore, engagement, compliance,
and resistance are the three main outcomes of influences (Yukl 2010). The
following figure 1 indicates the framework of the study.
Principal’s Power
Leader influence
behavior (Influences Influence Influences outcome
tactics) processes
Source: Adapted from Yukl (2010)

Figure 1: Conceptual framework of school principals’ power sources, and influencing
tactics, process, and outcome
1.2 Basic Research Questions

In light of all the above, the aim of this study critically examines the power
sources and influences of school principals in secondary schools of Eastern
Ethiopia. Hence, this study answers the following basic research questions.
1. What sources of power do school principals mostly use?
2. What kinds of tactics do school principals apply to influence subordinates?
3. Why do subordinates comply for school principals?
4. What outcomes are observed among subordinates for the influences of their
school principals?

121
2. Methodology
Research Design
This study used a descriptive survey research design. The basic ground behind
applying this design is to collect data to answer questions about people’s
opinions, beliefs, attitudes, behaviors, and demographic composition on the
current study (Mills & Gay, 2016). Additionally, this design is appropriate to
collect and analyze data with a limited timeline and budget. The researcher also
used a quantitative approach for a research strategy that requires collecting
numerical data, quantification of collected data, and analysis of data that were
selected from school teachers, principals, and supervisors (Creswell, 2012). The
participants of this study were made up of teachers, principals, and supervisors
who have been serving in different secondary schools in Eastern Ethiopia.
Population and Sample / Study Group/Participants

This study was conducted in Eastern Ethiopia. It comprised of two zones of
Oromia National Regional State (East and West Hararghe), one Administrative
City (Dire Dawa), and two districts in Harari National Regional State. The study
population (N=1,015) was composed of secondary school teachers (n=541),
principals (n=301) and supervisors (n=173). The participating secondary school
teachers had a Postgraduate Diploma in Teaching (PGDT) of the 2018/19
academic year batch. Participants were selected via stratified and simple random
sampling techniques. The basic reason behind employing the stratified sampling
technique was that study groups had different categories (teachers, principals,
and supervisors), and it was important to take into account such diverse strata.
To determine sample size from study population, the researchers applied

Yamane (1967) n=N÷1+NE2; which N=total population; n=sample size, E=error
rate /margin of error (.05) provides a simplified formula to calculate sample
sizes and guess: 95% confidence level P=.05. Thus, sample size determination
was calculated as follows:
𝑁 = 1015⁄1 + NE2
𝑁 = 1015⁄(1 + 361 ∗ 0.052)
𝑁 = 1015⁄3.5375
𝑁 = 287
After determining the sample size from the three groups of the respondents, the
researcher used stratified proportional sampling techniques to select
respondents from each category.
Table 2: Stratified Sampling Frame

Strata Population size Sample size Sampling technique/s
Teachers 541 153 stratified and random
Principals 301 85 stratified and random
Supervisors 173 49 stratified and random
Total 1,015 287

122
The primary data were gathered via questionnaires (Appendix) from teachers,
principals, and supervisors who worked the 2018/19 academic year of
secondary schools of Eastern Ethiopia. The first part of the questionnaires was
adapted from Hersey and Natemeyer (1979) entitled ‘power perception profile-
perception of others’; it includes 21 pairs of reasons frequently reported when
asked why subordinates obey the orders of their school principals. The second
part of the questionnaire was prepared by the researcher that aimed to measure
school principals’ influencing process, tactics, and outcome of the leaders’
influence on their subordinates’ performance.
The instruments were converted to local languages, such as Afan Oromoo and
Amharic to reduce language barriers. To check the reliability of the items, a pilot
test was conducted in Bate and Haramaya town secondary schools of Haramaya
town. To check the validity of the instrument, area experts were consulted, and
their comments were incorporated.
Data Analysis
The researcher applied descriptive statistics (frequency, percentage, mean score,
standard deviation) to describe the nature of the data and the characteristics of
the sample in a meaningful manner. Also, inferential statistics (independent t-
test and one-way ANOVA) were adopted to compare significant differences that
may exist between and within groups of teachers, principals, and supervisors in
their perception of principals’ power sources about their position, sex, service
years, and academic qualification.
Ethical Considerations
The researcher had given full attention to the moral and ethical issues. Due
consideration has been given to ethical concerns of the participants to ensuring
and informing consent and developing confidentiality, maintaining anonymity,
and other related ethical issues (Cohen, Manion & Morrison, 2007). The
researcher had communicated and assured the participants on their responses
will be used only for academic purposes and will remain confidential.
3. Results
The demographic data of respondents in this study can be described based on
roles and gender. The questionnaire was initially distributed to (n=153)
secondary school teachers, (n=85) school principals, and (n=49) supervisors with
a total of (n =287). Of the total number of the questionnaire distributed to the
groups, 264(92%) i.e. (n=145) teachers, (n=78) principals, and (n=41) supervisors
were appropriately filled and returned. Even though the questionnaire retrieval
rate is not at 100%, results could not have been affected by this retrieval rate and
returned data has statistically sound to precede analysis (Cohen et al. 2007).
Based on gender, it was found that about 221 males and 43 females have
participated in this study.
The sources of power school principals mostly use

Frequency counts and percentages were used to identify the power sources that
school principals mostly applied, as perceived by teachers, principals, and
supervisors. The summary of results presented in Table 3.

123
Table 3: School Principals’ Power Bases and Sources

Categories
Total Summary of
power basis
Power basis Sources power Teachers Principals Supervisors
N % N % N % N % N %
Legitimate
16 6.1 20 7.6 8 3 44 16.7
Power
Information
8 3 5 1.9 7 2.7 20 7.6
Positional Power
power Reward Power 14 5.3 16 6.1 3 1.1 33 12.4 155 58.7
Ecological
15 5.7 11 4.2 2 .8 28 10.6
Power
Coercive
27 10.2 2 .8 1 .4 30 11.4
Power
Personal Expert Power 51 19.3 19 7.2 15 5.7 85 32.2
power 109 41.3
Referent Power 14 5.3 5 1.9 5 1.9 24 9.1
Total 145 55 78 29.5 41 15.5 264 100 264 100
Looking at the detailed power sources of school principals reported in Table 3,

most participants perceived that school principals apply expert power (32.2%),
legitimate power (16.7%), and reward power (12.4%) most of the time. On the
other hand, ecological power (10.6%), referent power (9.1%), and information
power (7.6%) are sources of power that school principals slightly used to
influence their subordinates. The last column in Table 3 summarizes the seven
power sources that school principals are executing and further categorizes them
into two power bases. Accordingly, school principals are exercising positional
power (58.7%) and personal power (41.3%). To elaborate more about sources of
power that school principals mostly possessed, the following figure summarizes
in detail.
Figure 2: Summary of Power Sources that School Principals’ Mostly Used

124
The above bar graph described the power sources which school leaders mostly
applied, as perceived by teachers, principals, and supervisors. From the graph, it
can be seen that expert, legitimate, and reward power are the most utilized
power sources by school principals. On the other hand, information, ecological,
and referent power sources were reported as principals’ least utilized power
sources.
Testing Variables on Perceptional Differences in Principals’ Sources of Power

One-way ANOVA tests were conducted to uncover if there were statistically
significant differences amongteachers, supervisors, and principals. The results of
these tests are shown in Table 4 below.
Table 4: One-Way ANOVA Test on Perceptual Differences in Principals’ Sources of

Power by Position
Descriptive ANOVA Summary
Groups N Mean SD SV SS Df MS F Sig.
Supervisors 41 3.90 .55 Between Groups 3323.71 2 161.85 33.08 .00
Principals 78 3.73 .51 Within Groups 13111.25 260 50.24
Teachers 145 3.15 .73 Total 16434.96 263
Total 264 3.47 .72
As Table 4 shows, there was a significant statistical difference in the perception

of principals sources of power between the three groups: F (2, 263) = 33.08,
p<0.001. Despite the statistically significant results, the real difference in mean
scores between groups was quite slight (Group 1: M = 3.90, Group 2: M = 3.73
and Group 3: M = 3.15). To find out exactly where the differences between the
groups occurred, post hoc comparisons was applied by using the Tukey HSD
test indicated that the mean score for group 2 (M = 3.73, SD = 0.51 ) was
significantly different from that of group 3 (M = 3.15, SD = 0.73). Group 1 (M =
3.90, SD = 0.55) did not differ significantly from group 2 or 3. Tukey HSD test
which indicated that the mean score for supervisors was significantly higher
than that of principals and teachers. This implies that supervisors perceive
principals’ sources of power differently than teachers and principals.
An independent sample t-test was employed to compare whether there was a

statistically significant difference in the sources of power of school principals
between male and female participants. The results are presented in Table 5.
Table 5: Gender Difference in Sources of Power of School Principals

t-test for Equality of Mean
Sex N Mean SD F Sig. T Df Sig. (2-tailed) MD

Male 221 3.53 .70 .011 .916 3.89 262 .000 4.893
Female 43 3.09 .71 3.85 58.28 .000 4.893
There was a significant statistical difference in scores for males (M = 3.53, SD =

.70) and females (M = 3.09, SD = .71); t (287) = 3.89, p < .001). The degree of the
differences in the means (mean difference = -1.03, 95% CI: 2.42 to 7.36) was very
small (η2= .005). This clearly indicates that there is a difference in perceptions

125
between males and females on exercising power sources of school principals.

Female principals are more excises personal sources of power than positional
than male school principals while leading schools.
One-way between-groups analysis of variance was conducted to compare the

mean scores of three groups that were categorized based on academic
qualification with sources of power, and One-way among groups ANOVA test
results were summarized and presented in Table 6.
Table 6: One-Way ANOVA Test on Perceptual Differences of Power Sources by

Academic Qualification
Items N Mean SD SV SS df MS F Sig.
Diploma 2 3.45 .13 Between Groups 10.403 2 5.20 .083 .92
Degree 261 3.43 .72 Within Groups 16424.55 261 62.93
Master 1 3.72 - Total 16434.96 263
Total 264 3.44 .72
Table 6 displays the one-way ANOVA results comparing participants’ reported

perceptions of principals’ sources of power by using the mean scores of three
groups categorized by participants’ academic backgrounds. Participants are
categorized into three groups according to their level of academic qualifications
(Group 1: Diploma; Group 2: Degree; Group3: Master’s). There was no
statistically significant difference in perceptions of school principals’ power
between the three groups: F (2, 263) = .83, p = .92. Moreover, the descriptive
results of each group assured that almost there was no mean difference between
groups; G1 (M=3.45, SD=.13); G 2 (M=3.43., SD=.72); G 3 (M=3.72, SD=-). The
result indicates that level of academic qualifications has does not affect
exercising different sources of power of school principals.
One-way between-groups ANOVA with posthoc tests administered to see the

difference among respondents that grouped according to years of service and
perception of principals’ use of power, and the test results were summarized
and presented in Table 7.
Table 7: One-Way ANOVA Test on Perceptual Differences of Power Sources by

Service Years
Items N Mean SD SV SS Df MS F Sig.
<5 Years 174 3.34 .70 Between Groups 606.64 4 151.66 2.48 .04
6- 10 Years 64 3.64 .74 Within Groups 15828.32 259 61.11
11-15 Years 21 3.57 .52 Total 16434.96 263
16-20 Years 3 3.21 .51
>21 Years 2 3.77 .32
Total 264 3.44 .72

126
Table 7 displays the results of the one-way between-groups analysis of variance

were conducted to explore the impact of years of service in study group
perception of the use of power by principals. Participants were divided into five
groups according to their years of service (Group 1: <5 years; Group 2: 6-10
years; Group 3: 11-15 years, Group 4: 16-20 years; and Group 5 :> 21 years). The
result revealed that there was a significant statistical difference in the perception
of the use of power scores by principals for the year of service of the five groups:
F (4, 259) = 2.48, p = 0.04Despite obtaining statistical significance, the actual
difference in mean scores between groups was medium (3.34, 3.64, 3.57, 3.21 and
3.77, respectively). To find out exactly where the differences between the groups
occurred, post hoc comparisons was employed using the Tukey HSD test which
indicated that the mean score for group 1 (M = 3.34, SD = 0.70 ) was significantly
different from that of group 2 (M = 3.64, SD = 0.74). Group 3 (M = 3.57, SD =
0.52), group 4 (M = 3.21, SD = 0.51) and group 5 (M = 3.77, SD = 0.32) did not
differ significantly from group 1 or 2.
Influencing tactics, process and outcomes of school principals

The respondents were asked about school principals’ influencing tactics, process,
and outcome, and they were requested to choose one among the given
alternatives. The results are presented in Table 8.
Table 8: School Principals’ Influencing Tactics, Process and Outcome

Types of Items Categories Total %
influences Teachers Principals Supervisors
N % N % N % N %
School Reactive influence
27 9.4 0 0.0 1 .3 28 9.8
Principals’ tactics
Influencing Proactive influence
68 23.7 42 14.6 32 11.1 142 49.5
tactics tactics
Political tactics 36 12.5 13 4.5 6 2.1 55 19.2
Impression
22 7.7 30 10.5 10 3.5 62 21.6
management tactics
Total 153 53.3 85 29.6 49 17.1 287 100
School Reactive influence
76 26.5 31 10.8 13 4.5 120 41.8
Principals’ tactics
Influencing Internalization 41 14.3 42 14.6 30 10.5 113 39.4
Processes Instrumental
36 12.5 12 4.2 6 2.1 54 18.8
compliance
Total 153 53.3 85 29.6 49 17.1 287 100
School Commitment 23 8 15 5.2 5 1.7 43 15.0
Principals’ Compliance 44 15.3 21 7.3 12 4.2 77 26.8
Influence Resistance 86 30.0 49 17.1 32 11.1 167 58.2
outcomes
Total 153 53.3 85 29.6 49 17.1 287 100
As indicated in Table 8, school principals are influencing their subordinates by

using reactive influence tactics (9.8%), political tactics (19.2%), impression
management tactics (21.6%), and proactive influence tactics (49.5%). These
findings clearly indicate that most of the respondents noted their school
principals applied influencing tactics, specifically proactive inflecting tactics.

127
Pertaining to school principals’ influencing processes, principals were liable to

rely upon instrumental compliance (18.8%), internalization (39.4%), and reactive
influence tactics (41.8%). Therefore, most of the respondents stated that school
principals are predominantly using reactive influencing processes for
persuading school subordinates.
Relating to the influence outcome of school principals on the job performance

and satisfaction of their subordinates, the summary results of the respondents
reported that commitment (15.0%), compliance (26.8%), and resistance (58.2%).
Thus, statistically, data clearly indicated that subordinates are mostly resisting
their school principals' influence. If the subordinates are dissatisfied by their
respective leaders it is too hard to bring change and reform in an institution
(Brezicha, Ikoma, Park & LeTendre, 2020).
As seen in the above Table 8, school principals are mostly applying proactive
influencing tactics to influence subordinates. Likewise, reactive influencing
tactics are a reason for subordinates complying with principals’ requested
actions. Furthermore, most of the time subordinates showed resistance towards
their school principals’ influence.
4. Discussions
The major aim of this study was to examine school principals’ power sources,
and influencing tactics, processes, and outcomes in secondary schools of Eastern
Ethiopia. Four research questions guided the study.
The first key finding was that expert, legitimate, and reward powers are the
power sources which school principals mostly applied to influence their
subordinates. As Aslanargun (2011) argues, expert power is strongly accredited
in an organization that leaders and followers trust one another. When school
leaders appropriately applied the above-mentioned sources powers, the leader’s
behaviors could simply be internalized, leading to compliance. Likewise, if
subordinates are internally motivated, they no longer need to be controlled as is
the case with reward and coercion. Further, Singh (2009) conducted a study on
'Organizational Power in Perspective': results depicted that excessive use of
legitimate and expert power, slows down motivation and creativity of workers,
while the lack of power delay decision-making. Likewise, Mitchell (2012)
suggests the most powerful means of influencing others is having a positive
relationship.
Ironically, school principals slightly used referent, information, and ecological

power sources to influence their followers. If school principals rarely apply the
power of referent, subordinates cannot evoke a sense of trust, loyalty,
fulfillment, and responsibility towards subordinates, as well as enthusiasm
towards a leader (Aslanargun, 2011). Also, school principals are exercising
positional power (58.7%) and personal power (41.3%). Likewise, principals
mostly applied coercive influence of power over the teachers (Özaslan, 2018).
School principals are steadiness applying a positional and personal basis of
power. However, they are slightly exercising the positional power basis than
personal sources of power. This finding directly contradicts the personal sources

128
of power is paramount important for employee job satisfaction, organizational

commitment, and job performance than positional sources of power (Lunenburg,
2012; Weinstein et al., 2020). Likewise, even though school leaders are expected
to apply shared leadership to influence their followers, their level of application
is not this much (Goksoy, 2016). Based on the result and discussions, it is
possible to conclude that secondary school principals in Eastern Ethiopia mostly
practice positional power. In particular from power sources perspectives they
are highly exercising expert, legitimate, and reward power.
Another finding was that there was a statistically significant difference amongst
teachers, principals, and superiors in their perception of power sources of school
principals. The disparity in mean scores between principals and teachers was
quite small. In addition to the gender variable, there was a significant difference
in scores of males and females, and the degree of the differences in the means
was very small. This clearly indicates that gender has a significant effect on male
and female principals to apply different sources of power. Regarding to
academic qualification, there was no statistically significant mean differences in
perception of the power of sources of school principals for the three-level
(Diploma, Degree, and Master’s) qualifications. Moreover, there was a
statistically significant difference in the perception of principals’ use of power
scores for the five groups’ service years. The actual difference occurred in mean
scores of service years between (<5 Years) and (6- 10 Years) were a small effect
size. Based on the result and discussions it is briefed that there was a significant
difference in power sources of school principals concerning positions, gender,
and service years; however, there was no statistically significant difference in
academic qualification.
The second key finding was that, even though school principals are using four
basic influencing tactics namely impression management, political, proactive,
and reactive influence tactics; they are predominantly applying proactive
influence tactics. A certain situation may dictate the leaders what influencing
tactics they should apply based on a given circumstance (Theron, 2020; Yukl,
2010). School administrators ought to recognize and give values to how they are
exercising their power and influencing others (Stravakou, Lozgka &
Melissopoulos, 2018). Based on the result and discussions it is concluded that
school principals are mostly using proactive influence tactics than the rest types
of tactics. Thus, school principals are using reasonable influence and factual
evidence to make the request feasible (Aslanargun, 2011).
The third important finding was influencing the processes of school principals
are instrumental compliance, internalization, and reactive influence. Most of the
respondents, however, reported that subordinates carried out compliance and
principals’ requested action due to reactive influence, which suggests they obey
to obtain a reward or avoid a penalty. Ironically, the result is reversed with
stakeholders of the secondary are overloaded with resistance forces, while
principals' experiences of change or leadership are more optimistic with fewer
resistance forces (Van Wyk, Van der Westhuizen & Van Vuuren, 2014). School
stakeholders resist while principals’ experiences of change or leadership are
more optimistic with fewer resisting forces

129
The fourth major finding was that the influence of the school principals brings
three types of influence outcomes on subordinates’ performance: commitment,
compliance, and resistance. Most of the respondents, however, reported that
subordinates mostly resisting their school principals' influence (Twalh,
Alsolami, Cheng, & Islam, 2016). Further, the best way to lead employees as a
manager is to become a storyteller meaning that leaders should always focus on
creating user-friendly and interactive environments, rather than tense
environments, in which the storyteller (manager) always engages and puts the
audience at ease (Yan, 2020).
5. Conclusions
Based on the prior results and discussions, the researcher drew the following
conclusions:
It is possible to conclude that secondary school principals in Eastern Ethiopia are
mostly practicing positional power. In particularly from power sources
perspectives they are highly exercising an expert, legitimate, and reward power.
Thus, it is recommended that school principals should balance both sources of
power, and then apply personal sources of power to ensure employee job
satisfaction, organizational commitment, and job performance. There was a
statistically significant disparity in power sources of school principals
concerning positions, gender, and service years; however, there was no
statistically significant difference in academic qualification. Also, higher
education officials and policymakers at the MoE, Minister of Sciences and
Higher Education (MoSHE), Regional Education Bureau, Zonal Education
Bureau, and district education offices should take in to account staff seniority,
gender, and service years when they assign and appoint principals (Tingle,
Corrales, & Peters, 2019).
Most of the time, an organization does not succeed because attention had not
given to how leaders’ power is managed and deployed (Twalh et al., 2016).
School principals are mostly using proactive tactics than the rest two. Therefore,
school principals should create a conducive work environment and positive
school culture so that subordinates carry out requested action by recognizing
compliance intrinsically, allowing them to maintain a relationship and favor
with their leader. Teachers’ perceptions of school principals influenced school
culture and affected the teacher’s work (Britton, 2018). Subordinates are obeyed
by the school principals simply to reacting to order and instruction which they
have given. Thus, it is possible to conclude subordinates are obeying school
principals intended to obtain a tangible reward or avoiding punishment. It is of
paramount importance that all school shareholders should friendly work
together for the realization of intuitional goals (Kolleck, 2019; Eyal & Yarm,
2018). Lastly, resistance influence outcome is mainly observed among
subordinates for the influences of the power of their school principals. Thus,
subordinates are mostly resisting their school principals' influence. Principals
then need with teachers to maximize their potential and efficiently utilize their
capacity (Tang, Chen, Knippenberg, &Yu, 2020; Heffernan,2018). School
principals suggested motivating, encouraging, and inspiring subordinates to
overcome unnecessary resistance and confrontation while managing schools.

130
The Implication for Future Research

The study implies that there are only a few studies carried out in Ethiopia in
general and Eastern regions in particular that are related to the power sources
and influences of secondary school principals. Thus, based on the results, it is
clear that how school principals’ power is managed and deployed to make
sound influence over subordinates to assure quality education. Also, it is
evidence-based that the result of the study serves not only as a means of
improving school leadership and teaching but also in rising and maximizing
junior staff to shoulder responsibility in administrative areas for the future and
to enhance their problem-solving skills. This study would add new knowledge
on the power of and sources of influences of secondary school principals. The
findings of the study have theoretical, practical, and policy-related benefits for
the improvement of secondary school leadership.
However, this study has its own limitations, firstly, due to the resource
constraints, the research addresses only some selected secondary schools of
eastern Ethiopia. As a result, the research finding may not be generalized for all
secondary school principals of Ethiopia. Secondly, since the study secured only
quantitative data from the respondents it has methodological limitations too.
Future research is needed to gain a more general understanding of the power
sources and influences of school principals in the same areas in depth (Yan,
2020).
6. References
Alsobaie, M. F. (2015). Power and Authority in Adult Education. Journal of Education and
Practice, 6(15), 155–159. https://doi.org/10.2304/power.2010.2.1.63
Aslanargun, E. (2011). The power sources that principals handle in school
administration. Eğitim ve İnsani Bilimler Dergisi: Teori ve Uygulama, 2(3), 3–28.
Retrieved from https://dergipark.org.tr/tr/download/article-file/210634
Brezicha, K. F., Ikoma, S., Park, H., & LeTendre, G. K. (2020). The ownership perception
gap: Exploring teacher job satisfaction and its relationship to teachers’ and
principals’ perception of decision-making opportunities. International Journal of
Leadership in Education, 23(4), 428–456.
https://doi.org/10.1080/13603124.2018.1562098
Britton, E. M. (2018). Influence of School Principals on Teachers’ Perceptions of School Culture.
Doctoral Dissertation, Walden University. Retrieved from
https://scholarworks.waldenu.edu/dissertations.https://scholarworks.walden
u.edu/cgi/viewcontent.cgi?article=6448&context=dissertations
Cogaltay, N., Yalcin, M., & Karadag, E. (2016). Educational leadership and job
satisfaction of teachers: a meta-analysis study on the studies published between
2000 and 2016 in Turkey. Eurasian Journal of Educational Research, 62, 255-282.
http://dx.doi.org/ 10.14689/ejer.2016.62.13
Cohen, L., Manion L., & Morrison K. (2007). Research Methods in Education (6th ed.). New
York: Routledge. Retrieved from
https://www.tandfonline.com/doi/abs/10.1111/j.14678527.2007.00388_4.x
Creswell, J. W. (2012). Education research; planning, conducting, evaluating, quantitative and
qualitative research. United states of America, Pearson Education, inc. Retrieved
from https://www.amazon.co.uk/Educational-Research-Conducting-
Quantitative-Qualitative/dp/0134519361

131
Dhuey, E., & Smith, J. (2018). How school principals influence student learning. Empirical
Economics, 54(2), 851–882. https://doi.org/10.1007/s00181-017-1259-9
Eyal, O., & Yarm, M. (2018), Schools in cross-sector alliances: what do schools seek in
partnerships?, Educational Administration Quarterly, 54(4), 648-688.
https://doi.org/10.1177/0013161x18765268
Goksoy, S. (2016). Analysis of the relationship between shared leadership and
distributed leadership. Eurasian Journal of Educational Research, 65, 295-312
http://dx.doi.org/10.14689/ejer.2016.65.17
Goshu, B. S., & Woldeamanuel, M. M. (2019). Education Quality Challenges in Ethiopian
Secondary Schools. Journal of Education, Society and Behavioural Science, 31(2), 1-
15. https://doi.org/10.9734/jesbs/2019/v31i230147
Heffernan, A. (2018). Power and the ‘autonomous’ principal: Autonomy, teacher
development, and school leaders’ work. Journal of Educational Administration and
History, 50(4), 379–396. https://doi.org/10.1080/00220620.2018.1518318
Hersey, P., & Natemeyer, W. E. (1979). Power perception profile. Escondido, CA: Center for
Leadership Studies.
Krause, D. E. (2004). Influence-based leadership as a determinant of the inclination to
innovate and of innovation-related behaviors: An empirical investigation. The
leadership quarterly, 15(1), 79-102.
https://doi.org/10.1016/J.LEAQUA.2003.12.006
Kolleck, N. (2019). The power of third sector organizations in public education. Journal of
Educational Administration, 57(4), 411-425. https://doi.org/10.1108/jea-08-2018-
0142
Leithwood, K., Harris, A., & Hopkins, D. (2020). Seven strong claims about successful
school leadership revisited. School Leadership & Management, 40(1), 5–22.
https://doi.org/10.1080/13632434.2019.1596077
Lunenburg, F. C. (2012). Power and leadership: An influence process. International
Journal of Management, Business, and Administration, 15(1), 1–9. Retrieved from
https://www.coursehero.com/file/26929224/Power-and-Leadership-An-
Influence-Process-pdf/
th
Mills, G. E., & Gay, L. R. (2016). Educational research (11 eds.). Upper Saddle River, NJ:
Pearson. Retrieved from
https://journals.sfu.ca/jalt/index.php/jalt//view/41(Accessed on April May
19, 2019).
Mitchell, R. (2012). Practitioner reflections from a cluster unit in a college of teacher education
in northern Ethiopia. MA assignment, University of Leeds, UK. Retrieved from
https://www.educ.cam.ac.uk/people/staff/mitchell/
Mitchell, R. (2015). The implications of school improvement and school effectiveness
research for primary school principals in Ethiopia. Educational Review, 67(3), 328–
342. Retrieved from https://www.academia.edu/7108725/The_implications
_of_school_improvement_and_school_effectiveness_research_for_primary_scho
ol_principals_in_Ethiopia
Minster of Education (MoE). (2008). General Education Quality Improvement Package
(GEQIP), Addis Ababa. Retrieved from
http://documents1.worldbank.org/curated/en/ 258681468023106655
/pdf/451400PAD0P1061LY10IDA1R20081031311.pdf
Nelson, D. L., & Quick, J. C. (2013). Organizational behavior: Science, the real world, and you
engage learning (4th ed.). Boston: Pearson education. Inc. Retrieved from
http://go-pdf.online/nelson-and-quick-2013-organizational-behavior-isbn.pdf

132
th
Northouse, P. G. (2013). Leadership: Theory and practice (8 eds.). SAGE Publications.
Retrieved from
https://studydaddy.com/attachment/146554/Peter_G._Northouse_Leadership
__Theory_and_Practiz-lib.org.pdf(Accessed on April 26, 2019).
Özaslan, G. (2018). Principals’ conceptions of their current power basis revealed through
phenomenography. Journal of Educational Administration, 56(2), 220-235.
https://doi.org/ 10.1108/ JEA-10-2016-0120
Semela, T. (2011). Breakneck expansion and quality assurance in Ethiopian higher
education: Ideological rationales and economic impediments. Higher Education
Policy, 24(3), 399–425. https://doi.org/10.1057/hep.2011.11
Singh, A. (2009). Organizational power in perspective. Leadership and Management in
Engineering, 9(4), 165–176. https://doi.org/10.1061/(asce)lm.1943-5630.0000018
Stravakou, P. A., Lozgka, E. C., & Melissopoulos, S. (2018). The influence of values on
educational administration: The School Principals’ perspective. International
Journal of Education and Research, 6(4), 147–160. Retrieved from
https://www.ijern.com/journal/2018/April-2018/12.pdf
Tang, G., Chen, Y., van Knippenberg, D., & Yu, B. (2020). Antecedents and consequences
of empowering leadership: Leader power distance, leader perception of team
capability, and team innovation. Journal of Organizational Behavior, 41(6), 551–566.
https://doi.org/10.1002/job.2449
Tekleselassie, A. A. (2005). Teachers’ career ladder policy in Ethiopia: An opportunity for
professional growth or “a stick disguised as a carrot?” International Journal of
Educational Development, 25(6), 618–636.
https://doi.org/10.1016/j.ijedudev.2004.11.022
Tingle, E., Corrales, A., & Peters, M. L. (2019). Leadership development programs:
Investing in school principals. Educational Studies, 45(1), 1–16.
https://doi.org/10.1080/03055698.2017.1382332
Theron, S. (2020). Power and influence in post-secession South Sudan: A leadership
perspective on nation-building. African Security Review, 29(1), 1-24.
https://doi.org/10.1080/10246029.2020.1748672
Tran, H. (2020). Revolutionizing school HR strategies and practices to reflect talent
centered education leadership. Leadership and Policy in Schools, 1-15.
https://doi.org/10.1080/15700763.2020.1757725
Trinh, T. P. T., Pham, T. V., Cao, H. T., Nguyen, T.-T., Nghiem, T. T., & Tran, T. (2019).
The Profile of Professional Standards for Secondary School Principals in
Vietnam. International Journal of Education and Practice, 7(4), 310–323.
https://doi.org/10.18488/journal.61.2019.74.310.323
Twalh, A. A. I., Alsolami, H. A., Cheng, K. T. G., & Islam, G. M. (2016). Power and
Influence: CEO Power and the Use of Tactical Influence. Journal of Management
and Strategy, 7(2), 46–52. https://doi.org/10.5430/jms.v7n2p46
Van Wyk, A., Van der Westhuizen, P. C., & Van Vuuren, H. (2014). Resistance to change in
schools: Perceptions of principals and teachers in a South African province. Problems
and Perspectives in Management, 12(4), 457-656. Retrieved from
https://www.researchgate.net
/publication/286661673_Resistance_to_change_in_schools_Perceptions_of_prin
cipals_and_teachers_in_a_South_African_province.
Weinstein, J., Raczynski, D., & Peña, J. (2020). Relational trust and positional power
between school principals and teachers in Chile: A study of primary schools.
Educational Management Administration & Leadership, 48(1), 64–81.
https://doi.org/10.1177/1741143218792912
Yamane, T. (1967). Elementary sampling theory. New York: Hamper. Retrieved from

133
https://www.tandfonline.com/doi/abs/10.1080/01621459.1968.11009297
Yan, R. (2020). The influence of working conditions on principal turnover in K-12 public
schools. Educational Administration Quarterly, 56(1), 89–122.
https://doi.org/10.1177/0013161x19840391
Yukl, G. (2010). Leadership in organizations (7th ed.). Upper Saddler River. New Jersey:
Person. Retrieved from https://www.pearson.com/us/higher-
education/product/Yukl-Leadership-in Organizations-7th-
Edition/9780132424318.html

Appendix
HARAMAYA UNIVERSITY
COLLEGE OF EDUCATION AND BEHAVIORAL SCIENCES
DEPARTMENT OF EDUCATIONAL PLANNING AND MANAGEMENT
Questionnaire to be filled by: Secondary School Teachers, Principals and Supervisors
Dear Respondents, the purpose of this questionnaire is to conduct a research entitled “The Power
Sources and Influences of Secondary School Principals in Eastern Ethiopia”. The responses you provide will
have a paramount importance for the successful accomplishment of this study. Thus, you are kindly
requested to give your genuine response. Your responses will be used only for academic purpose and
will remain confidential.
Direction:
 Please read each item carefully and record your genuine opinion based on your exposure and
knowledge in the area of ethical leadership.
 Please do not consult others while responding the items.
 Please answer all questions in their order.
 It is not necessary to write your name on the questionnaire.
 Please return the completed questionnaire to the designated person/supervisor.
Thank you in advance for your kind cooperation!
Part I. General /Personal Information

1.1. Name of the school that you are working for__________________________________
1.2. Your current job position___________________________________________________
1.3. Sex: Male ❑ Female ❑
1.4. Age: <20 ❑ 21-25 ❑ 26-30 ❑ 31-35 ❑>36-40 ❑> 41 ❑
1.5. Total work experience in leadership position years: < 5 ❑ 6-10 ❑ 11-15 ❑ 16-20 ❑ >21 ❑
1.6. Educational qualification:
Diploma ❑ BA/BSC Degree ❑ MA/MSC Degree ❑ PhD ❑
1.7. Field of specialization: Educational Leadership and Management ❑Educational/school
Leadership and Management Others ❑
Instruction below, you will find 21 pairs of reasons frequently given by people when asked why
they obey the leaders of a school leader. Read each of these pairs of answers and allocate3-
pointsbetweenthe two alternative reasons in each pair. Base your point allocation on your
judgment of each alternative's relative importance (0=Not important, 1= less important, 2
important and 3=Very Important). This is in reference to your perception of why your
subordinates follows and obey you. Remember, for each pair only allocate a total number of
three points .i.e. A+B should be =3
A I can “punish” those who don’t co-operate with me.
1 They realize that I am in contact with very important and influential
B
people.
C They respect my knowledge, capacity, judgment, and experience.
2 They have access to information that is valuable to others.
D
E My position in the organization confers me the authority to manage its

3 work appreciate
They activities. me and what to please me.
F
G I can reward and support those who co-operate with me

4 I can “punish” those who do not co-operate with me.
A
B They realize that I am in contact with very important and influential
5 people.
They respect my knowledge, capacity, judgment and experience.
C
D I have access to information that is valuable to others.

6 My position in the organization confers me the authority to manage its
E
work activities.
F They appreciate me and what to please me.
7 I can reward and support those who co-operate with me.
G
A I can “punish” those who don’t co-operate with me.

8 They respect my knowledge, capacity, judgment and experience.
C
B They realize that I am in contact with very important and influential people.
9 I have access to information that is valuable to others.
D
C They respect my knowledge, capacity, judgment and experience.

E
work activities.
11 A I can “punish” those who don’t co-operate with me.

12 work activities.
They realize that I am in contact with very important and influential
B
people.
13 They respect my knowledge, capacity, judgment, and experience.
C
G I can reward and support those who co-operate with me.

14 They realize that I am in contact with very important and influential
B
people.
A I can “punish” those who do not co-operate with me.
E
work activities.
B They realize that I am in contact with very important and influential people.
16 They appreciate me and what to please me.
F
C They respect my knowledge, capacity, judgment, and experience.

17 I can reward and support those who co-operate with me.
G

18 They appreciate me and what to please me.
F

19 Iwork.
can reward and support those who co-operate with me.
G
20 I can “punish” those who do not co-operate with me.
A
G I can reward and support those who co-operate with me.

21 I have access to information that is valuable to others.
D
TOTALS:
A B C D E F G Sum
= 63
Key: Please read each statement, and show your position by putting a tick mark ( ) under
1=Never, 2=Rarely, 3=Sometimes, 4=Usually, 5=Always
No A. Items prepared to assess influences Tactics of principals and supervisors Responses
adopted from Yukl (2010) 1 2 3 4 5
No A. Items prepared to assess Proactive Influence Tactics of principals and
supervisors
1. use logical arguments and factual evidence to make request feasible
2. explain how the person could benefit from carrying out a requested task.
3. arouse the target person’s emotions to gain commitment for a request
4. encourage the target to suggest improvements
5. offer an incentive and promise to provide an appropriate reward who carry
out a difficult request.
6. provide relevant resources and assistance if the subordinates will carry out a
request
7. ask the person to do a favor for you as a friend
8. use praise before or during an influence by expressing person’s ability while
carry out request.
9. establish the legitimacy of a request by referring to rules, policies & contracts
10. use demands and frequent checking that the staff must do what I asked to do
11. seek the aid of others to persuade the subordinates to do something
Direction: Read the following questions carefully and choose the best answer that describes your
influencing tactics and process in your school?
1. What kind of Influencing Tactics you are applying to influences your immediate fellow?
A. Provide praise or offering unconditional help for your subordinate self-promotion.
B. Deception, manipulation and influencing decision makers to promote your interests.
C. Uses logical arguments and factual evidence to make request feasible
D. have switched roles with subordinates and subordinates resist and show unwanted behavior
2. What do you think that your subordinates carry out your requested action?
A. Obtaining a tangible reward or avoiding a punishment.
B. By recognizing compliance is intrinsically desirable & committed to support their leaders.
C. To gain approval, maintain a relationship , need for acceptance and favor from the leader
134

Vol. 19, No. 10, pp. 134-149, October 2020
Exploring Educators’ Challenges of Online Learning

in Covid-19 at a Rural School, South Africa
Kananga Robert Mukuna

Psychology of Education, University of Free State, South Africa
https://orcid.org/0000-0002-1787-4543
Peter J. O. Aloka
Wits School of Education, University of the Witwatersrand, South Africa
https://orcid.org/0000-0002-4298-9211
Abstract. This study explored the perceived challenges of online learning

encounter by rural educators in response to COVID-19 pandemic at a
selected rural school in South Africa. Within the qualitative approach, a
multiple case study was used as a research design. A sample size of six
participants (N=6, two males and four females) was drawn from a rural
school in a Thabo Mofutsanyana District in the Free State Province.
Purposive and convenience sampling techniques were used to select
participants in this study. The participants had teaching experience
ranging from 2 to 7 years. Data were collected through telephone
interviews and analysed using thematic analysis. The finding suggested
that the Protection Motivation Theory was an insightful framework in
responding to the challenges emanating from COVID-19 induced online
teaching and learning. Thus, the study revealed that there are various
challenges to online learning. These include poor parental involvement in
children’s homework, incomplete work and poor performance,
insufficient personal protective equipment, poor access to network access,
and lack of learning devices. Subsequently, we recommend that
stakeholders such as the Department of Education, Department of Basic
Education, and School Governing Bodies, among others, should ensure
that parents are sensitized about the need for learners to manage the
resources at their disposal. Furthermore, the provision of adequate
resources such as; Internet connectivity and Information and
Communication Technologies learning devices is imperative.
Keywords: Online learning; COVID-19; Protection Motivation Theory;

Rural school; Educators’ challenges

135
1. Introduction
Over the last 20 years, learning facilitated online through network technologies
has emerged across South African Higher Education Institutions (Ravjee, 2007).
However, online learning refers to learning taken partially or entirely through the
Internet (Gilbert, 2015; Sharma, Bhardwaj & Bhardwaj, 2017; MS’Sakshi & Dhull,
2018). The use of online learning involves many technologies that allow
knowledge delivery over computer networks in an educational context (Dorin,
2011; MS’Sakshi & Dhull, 2018). However, online learning is an Information and
Communication Technologies (ICT) enhanced practice in institutions which
include but is not limited to email provision, online journals, and networked
libraries to developing creative software solutions for information management
tasks in teaching, researching, and administrative systems.
The Department of Basic Education (DBE) responsible for primary school

education acknowledges the importance of blended learning in South African
schools (Green, 2011). Furthermore, the national Department of Education (2006)
published its White Paper on E-Education in 2004 and called together a ‘think-
tank’ that is based on an overview of research and delivery needs related to the
‘roll-out’ of e-Learning in schools. Thus, it is essential to mention that South
African rural areas mostly lack the social and economic viability needed to sustain
technological improvement (Cristobal-Fransi et al., 2020). This is because
governments find it more challenging to supply quality education services in rural
areas. Various factors weaken the quality of learning and teaching in South
African rural areas (Du Plessis & Mestry, 2019). It has been realised that online
learning is seen as a tool to address trajectories during the times of crisis, not only
for survival (by communicating essential information on the fight against COVID-
19) but also for teaching and learning by members of deprived communities, such
as those of rural schools (Dube, 2020). In lieu of this, Du Plessis and Mestry (2019)
recommended, there is a need to put strategies in place to improve teachers'
working conditions and the teaching at rural schools to improve learner
achievement across the rural areas of South Africa.
The Corona Virus Disease, otherwise known as COVID-19 pandemic, has affected
all the nations globally. While some countries have gone past the peak, South
Africa continues to experience the effects of this novel global pandemic. Various
measures have been devised to mitigate the virus's spread, such as social
distancing and the wearing of facemasks, among many other strategies
(Scherbina, 2020). Effective implementation of these measures requires the
cancellation of public events, educational and commercial activities, and limiting
individuals’ freedom to non-essential movement to encourage people to remain
at home. However, the above-measures' effectiveness is critically based on the
compliance of human behaviours in society (Delaney et al., 2013). A plethora of
factors that include awareness of the measure, the severity of penalties, and the
cost of isolation could affect these measures (Briscese et al., 2020). Among other
spheres of life, COVID-19 has reconfigured the teaching and learning process,
resulting in online learning as the dominant modus operandi.

136
Prior to the COVID-19 pandemic, scholars had predicted that online learning
styles would grow extensively, coupled with a sharp increase in demand for such
programs (Beyer, 2013; Ali & Smith, 2015). In a way, online learning can be
considered as a form of distance education where learners are physically or
geographically separated from their educators. Some scholars have argued that in
online learning, learners should be prepared to develop independent learning
capability and motivate themselves to learn and engage in learning activities that
make them persistent and successful (Zuhairi, Karthikeyan & Priyadarshana, 2019).
Online educational institutions are faced with challenges related to
transformation, which entails the digitalisation of learning and teaching.
Furthermore, the transition to digitalise the schools should consider the learning
needs and the technological support required for learning (Zuhairi, Karthikeyan
& Priyadarshana, 2019). However, an assumption exists that the use of online
learning is beneficial to learners because it is convenient for them within the
comfort of their home. That is, its advantages are vast, and they have been
significant to students’ development (Lim, 2004; Shimada, 2017; Guspatni, 2018).
According to Dorin (2011), online learning facilitates transformative learning,

which involves critical reflection, reflective discourses, and action. Such an
approach encourages learners’ engagement with academia. It also ensures
flexibility, which entails free learning conditions, self-paced studying
convenience, and a comfortable learning environment (Lim, 2004; MS’Sakshi &
Dhull, 2018). Furthermore, several scholars have identified the various
advantages of online learning that include easy accessibility, cost-effectiveness,
personalised learning, development of cognitive abilities, prompting computer
skills, developing self-pacing, and accessing globalisation (Bakia et al., 2012;
Norman, 2017). Thus, online learning allows individuals to depend more on
technology than human connection (Amichai-Hamburger & Ben-Artzi, 2003;
Ranaeiy, Taghavi & Goodarzi, 2016). However, research has suggested that online
learning is suitable for educational productivity (Bakia et al., 2012). To maximise
the benefits of online learning, there is a need to harness the requisite resources,
which are carefully managed. In the same vein, there is a need to inculcate such
critical values as intensive self-discipline and self-direction among learners
(Hickey, 2015). Online learning has changed the very traditional way of learning
characterised by loneliness (Pappano, 2001). While online learning, as an
alternative to traditional forms of learning, is associated with many advantages,
some challenges are context-specific. For example, learners in rural settings bear
the burden of online learning. The lack of adequate resources may explain that
learners in rural settings are uniquely disadvantaged by online learning.
Furthermore, online learning does not effectively facilitate the affective domain in
the teaching and learning process offered by face-to-face contact (Ali & Smith,
2015). Deducing from the above gaps, this paper explored the usefulness of the
Protection Motivation Theory (PMT) to understanding the perceived challenges
faced by educators in a rural setting in South Africa in the context of COVID-19
induced online learning.

137
2. Theoretical framework
Several models have been developed to respond to life-threatening situations,
such as the outbreak of pandemics. These are useful for understanding individual
precautionary behaviour to limit diseases' contamination (Jiang et al., 2009;
Williams et al., 2015). Others have reviewed how awareness mechanisms might
assist in preventing the outbreak of diseases (Redd, 2012; Ling, Kothe & Mullan,
2019). Some scholars have also considered the PMT as an intervention framework
for various pandemics (Plotnikoff et al., 2010). Therefore, this study adopted the
Protection Motivation Theory (PMT) developed by Ronald Rogers in 1975 to
comprehend educators’ responses to the challenges emanating from the COVID-
19 pandemic. PMT is a psychological model developed to understand better the
individual’s motivational responses to potential threats related to their health and
safety (Rogers, 1975). This theory suggests that individual and environmental
factors may affect or contribute to the development of protective behaviours,
which should be mediated by individual cognitive processes (Clubb & Hinkle,
2015). It has generally been used to understand protective behaviours that people
may adopt to reduce the perceived threat of pandemic diseases in various fields
such as health, education, and psychology (Baghianimoghadam et al., 2014;
Williams et al., 2015). Most of the studies on health psychology have focused on
research that are mostly scared of the pandemic. However, the present paper
demonstrated protective measures in response to COVID-19. PMT's relevance to
online teaching and learning in the context of COVID-19 is reflected in
considerable alignment. That is, the consistent implementation of protective
measures is necessary to protect vulnerable populations in unaffected regions.
Hence, it averts unnecessary alarm and harm during outbreaks of emerging

infections (Jiang et al., 2009). Therefore, we argue that PMT is critical because it
motivates using individual protective equipment and actions to normalise human
behaviour and enhance safety. This theory is useful in exploring the perceived
challenges to online learning encountered by educators in rural settings.
3. Literature review
Several studies have focused on conceptualising social isolation feelings. Social
isolation is a lack of meaningful human contact with other persons (Hortulanus et
al., 2006). Priego and Peralta (2013) highlighted that lack of face-to-face contact
with others could cause social isolation. Studies have revealed that the face-to-
face connection's quality is helpful because it allows individuals to express their
emotions towards peers (Holmes & Reid, 2017). Currently, individuals struggle to
connect with others due to COVID-19 related restrictions. For example, the need
to maintain social distancing compels people to limit discretional movement. In
online learning, communicating with fellow learners and educators is
challenging, especially when all interactions occur remotely. It is essential to
consider that the era of COVID-19 may contribute to increased feelings of anxiety.
It is relevant to differentiate the cause of social isolation and its development.
Some scholars revealed that social isolation could contribute to a high withdrawal
rate during online learning compared to the traditional face-to-face approach to
learning (Ali & Smith, 2015; Schaeffer & Konetes, 2010). Studies have suggested
that social isolation could be an obstacle to academic performance and

138
achievement (Bester & Budhal, 2001; Tingstad, 2019). It may lead to increased
anxiety, depression, and feelings of sadness (Ali & Smith, 2015; Matthews et al.,
2016).
Despite that other communication modes such as phone and email can be used
for interactions where both audio and video messages can be exchanged, the real
facial expressions and body movement cannot be detected during a phone call
(Conrad & Donaldson, 2012; Lehman & Conceição, 2010). However, social
distancing and social isolation are important strategies that an individual may
adopt to mitigate the risk of acquiring or spreading COVID-19. Therefore,
individual rights to free choice and to accomplish routine activities are limited to
decreasing the risk of spread and contamination. Research has shown that
engaging in protective measures can significantly decrease the possible infection
(Clubb & Hinkle, 2015). Considering the nexus between social isolation and online
learning, it is important to link social isolation to online learning and teaching in
schools. Social distancing is an action taken to minimise contact with other
individuals. It worth noting that social distancing measures comprise one
category of Non-Pharmaceutical Countermeasures (NPCs) aimed at reducing
disease transmission, which translates to reduced pressure on health services
(Anderson et al., 2020; European Centre for Disease Prevention and Control
[ECDPC], 2020). There is a lack of knowledge to explain behavioural responses to
the pandemic's cognitive of contamination risk. Social distancing might reduce
social contacts, responding to the pandemic (Williams et al., 2015).
Literature review on challenges of education delivery about pandemics like

COVID-19 exists. For example, Wenham, Smith & Morgan (2020) reported that the
closure of beyond might have a differential effect on women, who provide most
of the informal care within families, with the consequence of limiting their work
and economic opportunities. Similarly, McKibbin and Fernando (2020) found out
that the pandemic could significantly impact the short run's global economy.
Daniel (2020) adds that the COVID-19 pandemic is a huge challenge to education
systems and suggested that the normal classroom subjects teaching should put
COVID-19 in a global and historical context. Di Pietro et al. (2020) suggested that
COVID-19 influence negatively both cognitive and non-cognitive skills
acquisition and may have significant long-term consequences in addition to the
short-term ones. Besides, Dube (2020) reported that online learning alternative in
the context of COVID-19 excludes many rural learners from teaching and learning
in South African primary schools. Due to a lack of resources to connect to the
Internet, the learning management system, and low-tech software. Furthermore,
Ebrahim et al. (2020) argue that the state of lockdown halted learning in South
African schools, and learners and educators in rural areas are helpless on
approaching online learning during the COVID-19 lockdown. Finally, Ngogi
(2020) concludes that blended learning practised in South African Schools is
commendable but suggested that rural schools would be disadvantaged because
of several challenges.
The self-efficacy associated with PMT components was a strong predictor of the
intention to stay at home in a pandemic using social distancing behaviours in a

139
hypothetical pandemic. Similarly, Williams et al. (2015) found out that both PMT
and social distancing behaviour were responded to simulated factors in reducing
the infectious disease epidemics. However, in rural locations of South Africa, the
inability of learners to socialise may have affected their online learning capacities.
Studies have shown that rural schools lack Internet connectivity, hence, learners
in such environments have little or no access to the relevant tools required for
online learning tools during the era of COVID-19 (Omodan, 2020; Dube, 2020).
From the literature review conducted, little exists on the challenges of online
learning in South African schools. Therefore, the present study explored the
perceived challenges encountered by educators at a selected rural school in South
Africa in meeting their mandate during the era of the COVID-19 pandemic.
4. Current study
This study explored the PMT's usefulness to understand educators' challenges of
online learning in COVID-19 at a selected rural school in South Africa.
5. Methodology
5.1 Research design and participants
The study adopted a phenomenology research design. This design is suitable for
research that seeks to describe the essence of a phenomenon by exploring it from
the perspective of those who have experienced it (Teherani et al., 2015). The goal
of phenomenology is to describe the meaning of experience in terms of what was
experienced and how it was experienced (Teherani et al., 2015). This design was
chosen because it helped the researchers to explore the individual challenges
experienced by educators from a selected rural school.
The population consisted of all educators from the rural schools in the Thabo
Mofutsanyana District, Free State province, South Africa. The participants were
selected using purposive and convenient sampling techniques due to accessible
factors to the population's specific characteristics in exploring challenges
educators encountered during the COVID-19 period at the rural primary school.
These sampling techniques were important to select the participants who were
available to participate in the study during the national lockdown level 3 in South
Africa. During this phase, schools were closed, and participants were encouraged
to work remotely from home. Thus, the participants had been contacted to
participate in the study. Six educators (N=6, two males and four females)
constituted the sample size. All participants had at least two to seven years of
teaching experience. They were responsible for Life Skills and Life Orientation
subjects that empower learners to achieve their full physical, intellectual,
personal, emotional, and social potential. Their experiences were considered
phenomenal to enable juxtaposition of both pre-COVID and COVID-19 related
experiences.
5.2 Instruments
Data collection was made through qualitative telephone interviews. This
technique was adopted because it provides a balanced distribution of power
between interviewer and interviewee (Vogl, 2013). This method was applicable to
the study because it aligned with the social distancing, as it was a strategy that

140
meant currently to reduce the spread of COVID-19. This technique was

reasonable during COVID19 because it was not feasible for the researchers
to travel to individual educators’ schools or homes to conduct the
interviews. This study used open-ended questions related to the educators’
challenges during the COVID-19 at a rural school. The interviews included
parental involvement in children’s homework, network access, and learning
devices. All participants felt comfortable participating in this study, and the
interviews were naturally and friendly taken in English to facilitate the
data collection process. Participants were asked more than once to ensure
data credibility. The trustworthiness was made and followed up.
5.3 Procedure for data collection

The Research and Ethics Committee of the University of Free State approved the
study. The participants were informed of their right to voluntary participation
and the right to withdraw from the study at any stage of the research process
without expecting any negative consequences. All the interviews were conducted
telephonically using English as the medium of communication. Each interview
session was audio-recorded. All the participants voluntarily provided their
consent to be audio-recorded. The individual interviews with the educators
took approximately thirty to forty-five minutes per participant. The study
was conducted at a rural school in the Thabo Mofutsanyana district in Free State
province, targeted Life Skills (LS)/Life Orientation (LO) educators, and used
telephone interviews. The educators were interviewed during the day. They
selected times that were convenient for them.
5.4 Data analysis

The thematic analysis approach was adopted for data analysis. This method seeks
to identify categories and to report patterns (themes) that may emerge within
data. It minimally organises and describes the data set in (rich) detail. However,
thematic analysis often goes further than this and interprets various aspects of the
research topic (Boyatzis, 1998). First, the interview data were transcribed, and
interviewee responses were coded as participants: Participant 1, Participant 2,
Participant 3, Participant 4, Participant 5, and Participant 6 in line with their
responses to each question. Researchers familiarised themselves with data by
transcribing, reading, re-reading, and taking notes. We coded the data into
potential themes and gathered all relevant data to each potential theme to
measure the coding quality. We reviewed themes generated from a thematic
“map” of the analysis and checked the themes' relevancy from coded extracts and
entire data. We named emerging themes refined from theme's specifics and the
overall story; the analysis tells and generated clear definitions and names for each
theme. We finally produced integrated results, which assist the coherence of the
write-up report.
6. Results
The study explored the perceived challenges to online learning in the COVID-19
pandemic at a selected rural school in South Africa. The results suggested that as
an approach, PMT is insightful for understanding intention in online learning and

141
allowing the implementation of social distancing behaviours among rural school

educators to reduce the spread of COVID-19. Several themes emerged, indicating
that various challenges affect the efficacy of COVID-19 induced online learning
and teaching. These include poor parental involvement in children’s homework,
submission of poor quality work, and incomplete tasks, insufficient personal
protective equipment, poor network access, and lack of Information
Communication Technology (ICT) devices to access the Internet.
Poor parental involvement in children’s homework

The results indicate that many factors influenced poor parental involvement in
children’s homework in the COVID-19 induced lockdown at the selected rural
school. Language barriers could be a factor of poor parental involvement in
children’s homework. Most participants indicated that language barriers
inhibited some parents from assisting their children with homework during the
COVID-19 induced lockdown. This challenge mostly affected those that were
always dependable on the educators and friends’ assistance their children needed
help. For example, learners that reside in rural areas with their grandparents or
other family members during this time had to endure the most of online learning.
One participant reported that:
“We have found that many parents do not want to help their
children. They claim that it is the school’s responsibility to help
learners. Hence, they are not supporting their children, the school
or the educator” (Participant 2).
The lack of time could influence the poor parental involvement in children’s
homework in lockdown at the rural school from the interview excerpt. However,
most parents designated as essential workers, spending a lot of time at work, were
not available to assist their children with homework. While most of them would
be available only during the weekend, many be catching up with their work. As a
result, they could not contribute meaningfully to their children’s education.
“Parents do not have time as they are working during the lockdown,
complained of lacking time” (Participant 3).
From the interview excerpt, the lack of technological gadgets could explain the
poor parental involvement in children’s homework, the COVID-19 induced
lockdown at the rural school. Participants mentioned that the parents are not
much involved because they lacked the technological access, among other socio-
economic barriers. As a result, education changed dramatically, with the
distinctive rise of online learning, and the study materials were required.
Furthermore, online submissions required internet connection, and parents were
unable to check their children’s work. Participants indicated that it was difficult
to join and communicate with the learners’ parents or guardians during the
COVID-19 induced lockdown.
“Study materials are available on the Department of Education’s
website and also on other Internet platforms such as a worksheet,
cloud, and other such platforms. Unfortunately, most parents are
unable to help their children with school work” (Participant 1).

142
Regarding those mentioned above, the extract from one participant illustrates the
following:
“Subjects like Mathematics and Technology are the types of subjects
where learners need the most intervention. Unfortunately, these
topics suffer the most. Those schools that have been sharing work
with learners cannot carry out assessments and observation, which
is vital when educators want to continue with lessons. Feedback is
impossible as the platforms are not interactive” (Participant 6).
From the above interview excerpts, Participant 1 confirms that online learning
tools are available at the learners’ disposal. Still, they were not effectively utilised
because of illiteracy among the parents, resulting in them failing to assist their
children. This scenario is peculiar to schools in rural settings. Participant 6 did
not only link the matter to parents but that some of the important subjects were
not effectively discharged because of the technicalities involved. Therefore, poor
parental involvement was not solely a result of parents' reluctance to assist their
children. Lack of Internet connectivity was also a major challenge.
Poor quality and incomplete work submissions

The participants' responses suggested that educators experienced poor work
submissions because of using online teaching and learning as a method of content
delivery during the COVID-19 induced lockdown. They further mentioned that
parents were unable to assist their children in delivering quality work.
“Even under normal circumstances, most children struggle with
their schoolwork because they live with old people who cannot help
them. To them, the only place they should learn is at school; hence,
home is only space where they must relax” (Participant 4).
From the interview excerpt, participants indicated that learners were produced
and submitted incomplete or unfished works while others could not even attempt
the tasks given to them. Participants indicated that they received learners’ work
of poor quality-suggesting that they would have rushed through the work to
complete it. Furthermore, they complained that learners were not following
instructions when doing tasks, and some were writing illegibly.
“We also receive work that is of a very poor quality, where you can
easily see that the learner has rushed the work to complete it. They
also do not follow the instructions sent to them. Furthermore, they
write illegibly. We also have some learners who have not submitted
any work over 4 weeks after the due dates” (Participant 2).
From the interview excerpt, the result of poor quality work submissions could be
regarded as socio-economic barriers. Most of the parents who live in rural
communities are uneducated, unemployed, and faced serious challenges with
limited infrastructure available for their children at home, making it difficult for
teaching and learning to occur effectively. The below statement corroborates the
above;
“The ability to go outside and play with their friends is an easier
option for them. I have spoken to some parents, and all of them say

143
that everything in the townships is normal, and children are playing

around outside and should be at school”(Participant 6).
Therefore, participants believed that perhaps some parents were helping or even
doing the work for their children. The participants demonstrated that learners
produced a high quality of work and performed better than they did in class. It is
important to note that some parents inspired the high standard of work their
children produced and did not accept any poor work. They mentioned that
‘online’ assessment should be continued to be a language of the educator, and the
learners could still complete essays and other written tasks.
During the COVID-19 induced lockdown, we have also realised that some
learners depended on others' help. For example, not doing their work but wait for
others to do it for them. From the above analysis, it is clear that the outbreak of
COVID-19 precipitated the ‘new normal’, which hindered learners’ productivity
as confirmed by the participants that learners produced poor quality work.
Insufficient personal equipment and poor network access

The participants’ transcripts indicated that educators lacked sufficient personal
equipment, such as personal computers and electronic devices, laptops, and
smartphones. This equipment has become a learning device with extraordinary
potential for both classrooms and outdoor learning.
“Out of 48 learners, 28 parents have cell phones, which are
smartphones, but they are not familiar with how to use them, and
the rest have cell phones that can be only be used for phone calls
(Participant 5).
“The lack of devices makes it almost impossible to teach and learn

during this time since we are not able to meet face-to-face. The lack
of ICT devices means that children will miss out. In the context of
COVID-19, working without these devices means that both
educators and learners will not make any significant progress. Most
learners cannot afford ICT devices, and that means they
automatically become academically excluded. Not being able to get
an education because you lack facilities constitutes a violation of
one`s right to education.” (Participant 4).
From the above interview excerpts, it is clear that most of the participants lacked
the relevant ICT devices, and some did not know how to use them to enhance
online learning.
Participants also indicated that most parents and guardians were unfamiliar with
ICT gadgets due to several factors such as the cost of purchasing mobile phones
that are compatible with Internet connectivity, lack of electricity, low network
coverage, and expensive maintenance requirements for data, Wi-Fi. The
participants reported that the lack of these devices affected online teaching and
learning because even when trying to make means, it becomes a challenge to
parents due to data costs. Regarding the school environment, these are the main
challenges that hinder learning and for it to continue taking place during the

144
COVID-19 pandemic. Participants mentioned that the lack of ICT devices made
them frustrate both educators and learners.
“Network limitations and Internet access have had a big impact on
learning. Not all learners have easy access to the work sent to them.
This results in their failure to keep abreast with the syllabus”
(Participant 2).
The response from Participant 2 demonstrated that poor network access was a
challenge that affected online teaching and learning during the COVID-19
induced lockdown. They recognised that most learners hailed from informal
settlements and townships. In these settings, Internet access is considered a luxury
compared to other things such as food and shelter. Most residents in informal
settlements had no access to the Internet. Lack of internet connectivity made it
difficult for learners to access study materials. Furthermore, participants
mentioned that network-related limitations affected many learners, who
struggled to submit their work before stipulated due dates. This, in our argument,
can cause anxiety in the learners as they are scared that they will not get the marks
they have worked for, and educators battle to mark and control work as they are
constantly trying to gain access to learners' work.
Lack of learning devices

Since the outbreak of the COVID-19 pandemic has redirected the teaching and
learning spaces in the schools, adequate provision of online teaching-learning
material are imminent. In this regard, there is a need for personal electronic and
Internet devices for learners to cope and meet up with their urban counterparts.
Failure to access the online learning devices has many effects, such as withdrawal
and feeling isolated due to the lack of needful material to support learning. The
revelations from educators showed that the reverse is the case. The below
statement contained the statements;
“Unfortunately, learners’ parents do have some access to devices;
however, they will not be prepared to share it with their children,
and they will rather wait for schools to provide the necessary
equipment” (Participant 5).
“Some learners have to share devices with siblings and parents,
often resulting in their work falling behind or causing friction in the
households” (Participant 6).
The transcripts from participants’ responses suggested the lack of school

provisions’ devices such as laptops and smartphones affect online learning and
teaching during the COVID-19. The participants recognised that some parents
had these devices. Participants highlighted that learners who do not have access
to devices could fall behind, as they do not have access to their work, causing
them to wait for long periods. Another concern has also been that not all devices
can access the various platforms used for online learning. This includes the
devices' inability to access Microsoft Software, such as teams and forms and
WhatsApp to receive work. From the above, Dube (2020) argued that one of the
challenges learners face during COVID-19 is a lack of teaching and learning
resources such as the Internet of things, gadgets, tools that could enhance their
productivity towards their academic performances.

145
7. Discussion
The study sought to explore the perceived challenges of online learning during
the COVID-19 pandemic. It focused on a selected rural school in South Africa. The
findings revealed that the challenges to online learning in the context of COVID-
19 were poor parental involvement in children’s homework, poor quality and
incomplete class-work submissions, insufficient personal equipment, poor
network access, and lack of learning devices. In terms of poor parental
involvement in children’s homework, the study found out parents were not
adequately involved and available to assist their children. According to the
analysis, this was linked to poor socio-economic backgrounds, rural
characteristics, and illiteracy through argument within. It also emerged that some
parents were busy doing their work and running errands; hence, they were unable
to attend to their children’s schoolwork during the COVID-19 lockdown. The
above finding contradicts Đurišic and Bunijevac (2017), who argued that parents'
involvement in children’s education is significant to their academic performance.
Similarly, Lemmer (2007) argued that some parents cannot read and write and can
only communicate in their mother tongue, which presents a challenge when they
are expected to assist their children at home. On the other hand, Lau, Li and Rao
(2011) found that parents’ involvement in children’s education yielded positive
outcomes in primary and high schools. These findings may have originated
because the learners in lower education grades, such as primary and
secondary/high schools, are still underage and needed to be guided both at home
and in schools. However, this study confirms that the study area is faced with
poor parental involvement in children’s schoolwork, which is not healthy for the
academic development and productivity of the learners, especially in the context
of COVID-19 lockdown.
The study revealed that the selected schools' learners were submitting poor
quality work to their educators during COVID-19-induced lockdown. In some
instances, the learners were not serious about their homework. It was reported
that learners could submit incomplete tasks to their educators, which was also
linked to rural characteristics, including the poor socio-economic status of parents
who were unable to provide the necessary resources for their children to facilitate
online learning. Similarly, Omodan (2020) found out that the outbreak of COVID-
19 increased rural dwellers' vulnerability because learners living in rural locations
have the relevant infrastructure, such as the Internet, required to sustain the new
normal. Furthermore, Dube (2020) also found out that, although the government
of South Africa tried to provide various academic relief materials for learners to
cope with online learning. Therefore, learners in rural communities lack access to
Internet gadgets such as computers, phones, and other equipment to enhance
their academic progress during COVID-19 (Dube, 2020). This finding is in tandem
with the World Bank's (2020) observation that online learning disproportionately
benefits the advantaged students ahead of their disadvantaged counterparts. In
this context, learners in the selected location are regarded as disadvantaged. This
is not only peculiar to secondary schools. For example, Cristobal-Fransi et al.
(2020) confirmed that rural universities had limited educational resources to
sustain their students, especially those living in remote areas. Generally, rural

146
locations suffer unequal distribution of academic resources (Grydehøj, 2016;

Omodan, 2020).
The study also found that learners lacked ICT-related gadgets to work from home
during COVID-19 lockdown. On the one hand, this was linked to parents' inability
to share their devices with their children, and on the other hand, it was linked to
schools' inability to provide devices for their learners. Learners living in rural
locations, either by omission or commission, have little or no online learning
access. Montrieux et al. (2015) argued that policy and curriculum planners should
consider providing technical and pedagogical support to both learners and
educators to enable them to cope with the demands of 21st-century teaching and
learning. In line with this, we also argue that there is a need for increased personal
protective equipment to redress the challenge faced by rural schools.
8. Conclusion and recommendations

The study concludes that online learning was initially considered as a major
problem for learners and affected learners’ completion of their academic
programmes. Further, it also affected their academic performance because of
several limitations related to the subjects offered. The recent COVID-19 pandemic
increased the need for online teaching and learning and changed people's
perceptions. It is important to understand the gravity of social isolation and online
learning before we approach online education. This was unpacked with a
qualitative approach employed by this study. From the above-reported results,
we conclude that learners' challenges in rural communities during the COVID-19
pandemic included poor parental involvement in children’s homeworking, poor
quality and unfinished academic works, insufficient personal protective
equipment, and poor network access, and lack of learning devices. Upon the
exposition of the challenges, we recommend that all stakeholders such as the
Department of Education, Department of Basic Education, and SGBs, among
others, to ensure that parents are sensitised about online learning to ensure that
they take charge of their children when they are working at home, should make a
concerted effort. Secondly, the learners must be encouraged by both parents and
the educators to always manage the resources at their disposal to ensure that their
work is done perfectly and submitted on time. Besides, the government should,
as a matter of importance, rescue the learners living in rural communities by
providing adequate resources in the form of internet access and ICT devices.
9. References
Ali, A., & Smith, D. (2015). Comparing social isolation effects on students attrition in
online versus face-to-face courses in computer literacy. Issues in Informing Science
and Information Technology, 12, 11-20.
Amichai-Hamburger, Y., & Ben-Artzi, E. (2003). Loneliness and Internet use. Computers in
human behavior, 19(1), 71-80.
Anderson, R., Heesterbeek, H., Klinkenberg, D., & Hollingswort, T. (2020). How will
country-based mitigation measures influence the course of the COVID-19
epidemic? Lancet, 395(10228), 931-934.
Baghianimoghadam, M. H., Sharifpour, Z., Lotfizadeh Dehkordi, M., Nadgarzadah, A., &
Hashemi, A. S. (2014). The role of protection motivation theory in predicted of

147
nutritional behavior in prevention cancers in mothers in Yazd city, Iran. Progress

in Nutrition, 16(3), 197-203.
Bakia, M., Shear, L., Toyama, Y., & Lasseter, A. (2012). Understanding the Implications of
Online Learning for Educational Productivity. Office of Educational Technology, US
Department of Education.
Bester, G., & Budhal, R. S. (2001). Social isolation: a learning obstacle in the primary school.
South African journal of education, 21(4), 330-335.
Briscese, G., Lacetera, N., Macis, M., & Tonin, M. (2020). Compliance with covid-19 social-
distancing measures in Italy: the role of expectations and duration (No. w26916).
National Bureau of Economic Research.
Clubb, A. C., & Hinkle, J. C. (2015). Protection motivation theory as a theoretical
framework for understanding the use of protective measures. Criminal justice
studies, 28(3), 336-355.
Conrad, R. M., & Donaldson, J. A. (2012). Continuing to engage the online learner: More
activities and resources for creative instruction (Vol. 35). John Wiley & Sons.
Cristobal-Fransi, E., Montegut-Salla, Y., Ferrer-Rosell, B., & Daries, N. (2020). Rural
cooperatives in the digital age: An analysis of the Internet presence and degree of
maturity of agri-food cooperatives' e-commerce. Journal of Rural Studies, 74, 55–66.
https://doi.org/10.1016/j.jrurstud.2019.11.011
Daniel, S. J. (2020). Education and the COVID-19 pandemic. Prospects, 1-6.
https://doi.org/10.1007/s11125-020-09464-3
Delaney, L., Kleczkowski, A., Maharaj, S., Rasmussen, S., & Williams, L. (2013). Reflections
on a virtual experiment addressing human behavior during epidemics.
Proceedings of the 2013 Summer Computer Simulation Conference, 33, 1-8.
Department of Education. (2006). White Paper on e-Education Transforming Learning and
Teaching through Information and Communication Technologies (ICTs). Pretoria:
Government Printers.
Di Pietro, G., Biagi, F., Costa, P., Karpiński, Z., & Mazza, J, (2020). The likely impact of
COVID-19 on education: Reflections based on the existing literature and international
datasets. Publications Office of the European Union, Luxembourg.
https://doi.org/10.2760/126686, JRC121071
Dorin, H. (2011). E-learning tools for teaching and learning. Journal Plus Education, 7(2),
110-115.
Du Plessis, P., & Mestry, R. (2019). Teachers for rural schools – a challenge for South Africa.
South African Journal of Education, 39(1), 1-9.
Dube, B. (2020). Rural Online Learning in the Context of COVID-19 in South Africa:
Evoking an Inclusive Education Approach. Multidisciplinary Journal of Educational
Research, 10(2), 135-157. http://dx.doi.org/10.447/remie.2020.5607
Đurišic, M., & Bunijevac, M. (2017). Parental involvement as an important factor for
successful education. CEPS Journal, 7(3), 137-152.
Ebrahim, S. H., Ahmed, Q. A., Gozzer, E., Schlagenhauf, P., & Memish, Z. A. (2020). Covid-
19 and community mitigation strategies in a pandemic. British Medical Journal, 2,
368-378.
European Centre for Disease Prevention and Control [ECDPC]. (2020). Guidelines for the
use of non-pharmaceutical measures to delay and mitigate the impact of 2019-nCoV.
Stockholm: ECDC.
Gilbert, B. (2015). Online learning revealing the benefits and challenges. Education Masters. St.
John Fisher College.
Green, W. (2011). Foundation Phase teacher provision by publishing her education
institutions in South Africa. South African Journal of Childhood Education, 1, 1-10.

148
Grydehøj, A. (2016). Navigating the binaries of island independence and dependence in

Greenland: Decolonisation, political culture, and strategic services. Political
Geography 55, 102-112. https://doi.org/10.1016/j.polgeo.2016.09.001
Guspatni, M. (2018). Students’ activities in, perceptions of and expectations for e-learning:
A case in Indonesia. Knowledge Management & E-Learning: An International Journal,
10(1), 97-112.
Hickey, R. (2015). Disadvantages to consider about online education. Retrieved from
https://www.petersons.com/blog/5-disadvantages-to-consider-about-online-
education/ Retrieved May 2020
Holmes, C. M., & Reid, C. (2017). A comparison study of on-campus and online learning
outcomes for a research methods course. The Journal of Counselor Preparation and
Supervision, 9(2), 15.
Hortulanus, R., Machielse M., & Meeuwesen, L. (2006). Social isolation in modern society.
New York, NY: Routledge.
Jiang, X., Elam, G., Yuen, C., Voeten, H., de Zwart, O., Veldhuijzen, I., & Brug, J. (2009).
The perceived threat of SARS and its impact on precautionary actions and adverse
consequences: a qualitative study among Chinese communities in the United
Kingdom and the Netherlands. International Journal of Behavioral Medicine, 16(1),
58-67.
Lau, E. Y. H., Li, H., & Rao, H. (2011). Parental involvement and children's readiness for
school in China. Educational Research, 53(1), 95-113.
Lehman, R. M., & Conceição, S. C. (2010). Creating a sense of presence in online teaching: How
to" be there" for distance learners (Vol. 18). John Wiley & Sons.
Lemmer, E. M. (2007). Parent involvement in teacher education in South Africa:
International Journal about parents in education, 1(0), 218-229.
Lim, D. H. (2004). The Effect of Flexible Learning Schedule on Online Learners' Learning,
Application, and Instructional Perception.Presented at Academy of Human Resource
Development International Conference (AHRD) 2004. Retrieved from
https://www.learntechlib.org/p/59833/.
Ling, M., Kothe, E. J., & Mullan, B. A. (2019). Predicting intention to receive a seasonal
influenza vaccination using Protection Motivation Theory. Social Science &
Medicine, 233, 87-92.
Matthews, T., Danese, A., Wertz, J., Odgers, C. L., Ambler, A., Moffitt, T. E., & Arseneault,
L. (2016). Social isolation, loneliness and depression in young adulthood: a
behavioural genetic analysis. Social psychiatry and psychiatric epidemiology, 51(3),
339-348.
McKibbin, W. J., & Fernando, R. (2020). The Global Macroeconomic Impacts of COVID19:
Seven Scenarios. Centre for Applied Macroeconomic Analysis. Working Paper
19/2020. Australian National University.
http://dx.doi.org/10.2139/ssrn.3547729
Montrieux, H., Vanderlinde, R., Schellens, T., & De Marez, L. (2015) Teaching and
Learning with Mobile Technology: A Qualitative Explorative Study about the
Introduction of Tablet Devices in Secondary Education. PLoS ONE, 10(12),
0144008. https://doi.org/10.1371/journal.pone.0144008
MS’Sakshi, P., & Dhull, I. (2018). Collaborative and Cooperative Learning: An Overview.
Journal of Education, 34, 21-30.
Ngogi, E. M. (2020). The Impact of COVID-19 Pandemic on Education: Navigating Forward the
Pedagogy of Blended Learning. Department of Education KwaZulu-Natal, South
Africa.
Norman, S. (2017, August 03). 5 Advantages of Online Learning: Education without Leaving
Home. Retrieved August 08, 2017, from https://elearningindustry.com/5-
advantages-of-online-learning-education-without-leaving-home

149
Omodan, B. I. (2020). The Vindication of Decoloniality and the Reality of COVID-19 as an

Emergency of Unknown in Rural Universities. International Journal of Sociology of
Education. 1-26. https://doi.org/10.17583/rise.2020.5495
Pappano, L. (2001). The connection gap: Why Americans feel so alone. New Brunswick, NJ:
Rutgers University Press.
Plotnikoff, R. C., Lippke, S., Trinh, L., Courneya, K. S., Birkett, N., & Sigal, R. J. (2010).
Protection motivation theory and the prediction of physical activity among adults
with type 1 or type 2 diabetes in a large population sample. British Journal of Health
Psychology, 15(3), 643–661. https://doi.org/10.1348/135910709X478826
Priego, R. G., & Peralta, A. G. (2013). Engagement factors and motivation in e-learning and
blendedlearning projects. In Proceedings of the First International Conference on
Technological Ecosystem for Enhancing Multiculturality (pp. 453-460).
Ranaeiy, S., Taghavi, M. R., & Goodarzi, M. A. (2016). The effect of loneliness on social
networking sites use and its related behaviors. Global journal of health science, 8(8),
162-171.
Ravjee, N. (2007). The politics of e-learning in South African higher education. International
Journal of Education and Development using Information and Communication
Technology (IJEDICT), 3(4), 27-41.
Redd, B. R. (2012). Using the protection motivation theory to examine the effects of obesity fear
arousal on the physical activity of young adult female college students. Wayne State
University Dissertations.
Rogers, R.W. (1975). Cognitive and psychological processes in fear appeals and attitude change:
A revised theory of protection motivation. Social psychophysiology: A sourcebook,
153-176.
Schaeffer, C. E., & Konetes, G. D. (2010). Impact of learner engagement on attrition rates
and student success in online learning. International Journal of Instructional
Technology & Distance Learning, 7(5), 3-9.
Scherbina, A. D. (2020). Determining the Optimal Duration of the COVID-19 Suppression
Policy: A Cost-Benefit Analysis. http://doi.org/10.2139/ssrn.3562053
Sharma, P., Bhardwaj, K., & Bhardwaj, D. (2017). An Easy, Smart & Intelligent Way to
Learn: “Online Education”. International Journal of Advanced Research in Computer
Science, 8(9), 735-737.
Shimada, S. (2017). An effective method of collecting practical knowledge by presentation
of videos and related words. Knowledge Management & E-Learning: An International
Journal, 9(4), 468-483.
Teherani, A., Martimianakis, T., Stenfors-Hayes, T., Wadhwa, A., & Varpio, L. (2015).
Choosing a qualitative research approach. J Grad Med Educ, 7, 669–670.
Tingstad, E. B. T., (2019). Social Withdraws and Academic Achievements: Longitudinal Relations
through Ages 6, 8, and 10 (Master's thesis). Norwegian University of Science and
Technology (NTNU).
Vogl, S. (2013). Telephone versus Face-to-Face Interviews: Mode Effect on Semi-
Structured Interviews with Children. Sociological Methodology, 43(1), 133-177.
Wenham, C., Smith, J., & Morgan, R. (2020). COVID-19: The gendered impacts of the
outbreak. Lancet 395, 846–848.
Williams, L., Rasmussen, S., Kleczkowski, A., Maharaj, S., & Cairns, N. (2015). Protection
motivation theory and social distancing behaviour in response to a simulated
infectious disease epidemic. Psychology, health & medicine, 20(7), 832-837.
Zuhairi, A., Karthikeyan, N., & Priyadarshana, S. T. (2019). Supporting students to succeed
in open and distance learning in the Open University of Sri Lanka and Universitas
Terbuka Indonesia. Asian Association of Open Universities Journal, 15(1), 13-35.
https://doi.org/10.1108/AAOUJ-09-2019-0038

150

Vol. 19, No. 10, pp. 150-171, October 2020
Managing Continuing Education via Distance

Learning and Face-to-Face Courses for Human
Resource Development in the Mekong Delta of
Vietnam
Phuong-Tam Pham
Can Tho University, Can Tho, Vietnam
http://orcid.org/0000-0002-1745-9067
Tran-Binh Duong
Saigon University, Ho Chi Minh City, Vietnam
http://orcid.org/0000-0002-7062-8341
Thi-Thuy-Trang Phan
Can Tho Vocational College, Can Tho, Vietnam
http://orcid.org/0000-0002-9120-1566
Thai-Huu Nguyen & Minh-Thanh Nguyen

Can Tho University, Can Tho, Vietnam
http://orcid.org/0000-0002-7839-3735
http://orcid.org/0000-0001-6128-0171
Trinh Le Thi Tuyet & Nguyen Duong Hoang

Dong Thap University, Dong Thap, Vietnam
http://orcid.org/0000-0003-3970-9773
http://orcid.org/0000-0003-3869-8371
Duong Hoang Yen

VNU University of Education, Hanoi, Vietnam
http://orcid.org/0000-0001-6685-4103
Tien-Trung Nguyen
Duy Tan University, Vietnam
Vietnam Journal of Education, Hanoi, Vietnam
http://orcid.org/0000-0002-3320-8962
Abstract. Continuing education has increased in importance and

popularity in the Mekong Delta of Vietnam as a means of ensuring a
sustained and qualified workforce for the region. This research was
conducted to investigate the input and output of continuing education

151
through distance learning and face-to-face courses at selected institutions

in the Mekong Delta. Institutional strategies and capacity in managing
continuing education via two-course types were examined. The two main
methods employed were demographic data analyses and quantitative
analyses with a questionnaire of 70 Likert items distributed to 100 teachers
and 20 administrators. The findings revealed an increase in the input
quantity and quality of CE and the small gap between distance learning
and face-to-face output. Regulatory management, finance, and promotion
were the contents identified as areas that need improvement in managing
continuing education; yet the unbalanced capacities of different sectors,
favoring business and management, remained.
Keywords: continuing education; distance learning; face-to-face training;

human resource development
1. Introduction
Continuing education (CE) has been recognized as an effort to implement life-
long learning for sustainable development. Despite being a non-formal mode in
the education system, CE is becoming increasingly popular as it supports rather
than competes with the mainstream mode in solving the issue of enhancing the
quality and quantity of human resources (Alamgir, 1999). More importantly, it
applies to tertiary education and yields the potentials for producing desirable
outcomes of qualified human resources (Lege, Bonner, Frazier, & Pascucci, 2020).
In many inter-governmental initiatives (Council of Europe, 2003; UNESCO
International Institute for Educational Planning (IIEP), 2006; UNICEF, 1993), CE
is seen to contribute to changing educational and development landscape
worldwide. Dated back in the 1970s and the 1980s, a massive outburst of interest
regarding non-formal education in the world emerged (King, 1982; Rogers, 2004).
In the 1990s, the Massachusetts Institute of Technology (MIT) set up the first major
initiative of CE, but until 2002, 50 free courses were made available through the
CE initiative. One year later, MIT officially launched its CE course with 500 classes
(Berti, 2018). This proves a successful model of CE and has widened the
educational landscape as being accessible globally. Wickline (2013) estimates that
around 263 million will be eligible for higher education in 2015 and this will
require building 3 new campuses for 30,000 students every week for the next 15
years. Without CE, this number is unachievable in the face of the increasing
demand for learners (Berti, 2018). For developing countries, however, the
development of CE has been deemed slow and in need of better contextualization
to make it more effective (Association for Development of Education in Africa
(ADEA), 2012; Khurshid, 2014; Ochukut & Omwansa, 2016; Situma, 2015). The
two main means of conducting CE are distance learning (or online learning) (DL)
and face-to-face learning (F2F). The widespread of CE is noticeably fostered with
the advent of the Internet. Particularly, educational technology has advanced to
an unprecedented extent and radically changed the ways lessons are delivered,
which is via virtual CE classes. Distance learning, therefore, emerged and has
made irreplaceable contributions to the implementation of CE (Allen & Seaman,
2011; Shelton & Saltsman, 2005; Urueta & Ogi, 2020). Due to the convenience and
efficiency of online courses, it is estimated that “education will be cyberised” in
the future (Sener, 2012, p. 157).

152
Recognizing the benefits of CE and especially distance learning in training human

resources, Vietnam has adopted CE in parallel with mainstream schooling, using
both face-to-face and online mediums to cater to its increasing demand for higher
quality human resources. The Higher Education Law 08/2012/QH13 demands
tertiary institutions and trans-institutional programs to supply learners with CE
courses that are suitable for learners’ needs. Those programs aim at training
human resources, increasing intellectual levels, and cultivating talented people. T.
Pham (2018) finds that CE in Vietnam has provided a significantly large number
of employees trained for a variety of disciplines at the tertiary level, accounting
for approximately 3,000 graduates per institution per year. Recently, the
Vietnamese Government has removed the classification between formal
education and continuing education printed on diplomas. This has been regarded
as the government’s formal recognition of CE, which can encourage and promote
equality in education for learners nationwide. Despite a facilitative regulatory
environment for the development of CE, administrators and educational
institutions in Vietnam still encounter challenges in implementing this form of
training including managing budgets, infrastructure, personnel, and materials for
CE courses. Research into these aspects of implementing CE has not received
compatible interest and investment.
The current research was conducted aiming at providing a better insight into the
administration and management of CE in the higher education sector in Vietnam.
Firstly, through examining relevant policy and institutional documents, the
research provides an overview of the educational input and output of CE via DL
and F2F courses at a university in the Mekong Delta. Following that, the research
investigates the current management of CE implementation at the specified
university. The findings help the authors find out the potential for developing CE
at the university under its current management scheme. For that purpose, this
study aims to find answers to the following three research questions:
1. What are the differences in the input and output of CE through DL and F2F
courses?
2. What are the institutional strategies in managing CE courses?
3. How effective is the current management of CE at the specified institution in
catering to the human resource needs of the Mekong Delta region?
2.1. Continuing Education in human resource training
It is widely recognized that the development of human resources is the key to a
country’s socio-economic development and education lies at the core of the
process (Gadekar, 2020). Enhancing educational quality and widening access to
education enable the development of human resources and, thus, foster the
development of society and economy. This lays the ground for continued support
for CE alongside mainstream education.
Compared with mainstream education, CE is a relatively new phenomenon and

is considered a non-formal variation of education and training. This type of
education emerged following the promotion of life-long learning (Faure, 1972)
and has been regarded as a social policy for lifelong learning for both children and

153
adults (T. Pham, 2018). CE contributes to a learning society by granting learners

with wider access to education either full time or part-time. It plays four major
roles in the general education system, namely replacing, continuing,
supplementing, and completing (Hoppers, 2006; UNESCO, 2002; IIEP, 2006). Once
connected successfully to mainstream education, CE can create a breakthrough in
education and make lifelong learning feasible (UNESCO, 2002). Nonetheless, CE
features several drawbacks including small-scale, short-term programs with
limited funding. These disadvantages limit the impact and sustainability as well
as negatively affect the quality and effectiveness of CE. It is recognized that as CE
is widely perceived as inferior to formal education, CE programs may not always
be aligned with broader national education and development policies or demands
from the world of work in many countries (Yasunaga, 2014). On that recognition,
UNESCO IIEP (2006) requires equal stimulation to be provided for CE. Hoppers
(2006) also suggests implementing CE in parallel with the mainstream model in
an organized way to best serve learners from different backgrounds. In the
increasingly interconnected world with cutting-edge technological advances, CE
should become the focus in development policies of the international community
due to its capacity to satisfy multiple learning needs, particularly those who lack
access to formal education or are unable to complete a full cycle of basic education
(Yasunaga, 2014). Since CE can improve social cohesion and create responsible
future citizens, it can contribute to building up extracurricular knowledge and
skills for learners so that they can “learn to be” and “learn to live together” (Delors
et al., 1996, p. 7). Individuals and societies in the world are putting higher
expectations on the impact of CE on economic productivity, which can, in turn,
lead to higher social productivity and economic growth (Yasunaga, 2014).
CE has been delivered mainly through two mediums, brick-and-mortar (or face-
to-face) classes, and online (or distance learning) classes. There has been a debate
about the quality of these CE delivery modes (Bernard et al., 2004; Jahng, Krug, &
Zhang, 2007; Magagula & Ngwenya, 2004; McGuire & Castle, 2010). Several
researchers (Hong, 2002; Kleinman & Entin, 2002; Phipps & Merisotis, 1999; Rovai,
2002) suggest that there is no significant difference in the quality between DL and
F2F. Both are considered as being equally effective in delivering content and
methodology at tertiary institutions. Bernard et al. (2004) also share similar
findings when investigating students’ achievement, attitudes, and retention rates
of these two-course types. In contrast, several other studies support the outcomes
of CE courses conducted through DL over those taught F2F. For example, when
reviewing 86 studies from 1990 to 2002 that compare students’ final grades in
distance education with those in classroom-based education, Shachar and
Neumann (2003) found that students enrolled in distance learning achieved
higher grades than those in face-to-face education. Such improvement in students’
performance is, according to the researchers, attributed to technological advances
that allow better interaction among students and between students and teachers.
While F2F has remained more preferable than DL courses, DL is gaining greater
popularity in the tertiary education sector. Among advocates of F2F learning,
Bernard et al. (2004) and Shachar and Neumann (2003) contend that online
education is incapable of transferring personal knowledge as it ignores the

154
interaction between teachers and students, and therefore fails to fulfill the
function of socialization. Similarly, in the same ground, Berg and Seeber (2016)
further argue that online courses better serve as a promotion and awareness-
raising tool rather than a proper means of educating learners in intellectual terms.
However, the ample benefits that DL brings to learners have been widely
recognized. For one thing, content and instruction delivered through DL are
renowned for being highly flexible and accessible to learners (McGuire & Castle,
2010). This creates an advantage for learners who have difficulty in accessing
formal education due to geographical or time barriers. In this way, DL is capable
of widening access to further education to a wider learning community and
enhancing the educational level. Other benefits of DL are in terms of cost-
effectiveness and its ability to provide up-to-date content materials, stimulate self-
paced learning, and facilitate customized courses (Magagula & Ngwenya, 2004;
Puthe, 2008).
As a developing country, Vietnam is in great need of a qualified labor force that

can help the country achieve its goals of sustainable development. CE has been
implemented in Vietnam with the recognition of its potential in quickly
expanding the quantity while ensuring the quality of education to less
advantaged students (Vietnamese Government, 2011). Incorporating CE in
tertiary education and recognizing its role has been considered a timely and
effective resolution to fulfill the requirement of enhancing human resources for
the country’s social and economic development (Dang & Nguyen, 2004). Several
universities all over Vietnam have claimed success with CE in helping thousands
of undergraduate and graduate students to receive a quality education. In 2017-
2018, the number of students enrolled in CE courses was reported to account for
over 20% of the total enrollments in tertiary education institutions (289,939). The
figure for the academic year 2018-2019 was lower but nontrivially stood at 13%
(180,494) (Ministry of Education and Training (MOET), 2019b). The statistics
indicate the popularity of CE as well as its contribution to the human resource
enhancement for Vietnam. In a word, CE has been playing an indispensable role
in raising the educational level of human resources in the education and training
landscape of Vietnam, which highlights the significance of investigating the extent
to which institutions can be better supported in their implementation of CE.
2.2. Human resource needs of the Mekong Delta

The Mekong Delta is located in the South of Vietnam and consists of 13 provinces.
Despite playing a vital role in the economy of the South, the region’s education
and training at the tertiary level are often a point of criticism for being unable to
meet the region’s needs for human sources. As reported in the workforce and
employment survey conducted in 2016 by the General Statistics Office of Vietnam,
as of 2016, the percentage of trained workforce of Vietnam was 20.9% whereas
that of the Mekong Delta was 12.2%. The region was ranked the lowest among the
surveyed regions (Statistics Directorate, 2016). The fact that the majority of the
human resource in the Mekong Delta (87.8%) are unqualified for the region’s
demands for the labor force makes it a concern for policymakers and institutions
alike (Figure 1).

155
35
28.9
30
25 20.9
20
13.6 12.2
15
10
5
0
Whole country Red River Delta Central Mekong Delta
Highlands
Figure 1: Proportion of trained workforce by region in 2016 (Statistics Directorate,

2016)
As of 2015, the Mekong Delta region has 17 universities and 26 colleges (Ministry
of Education and Training (MOET), 2019a). These institutions host around 192,213
students, accounting for 8.13% of the total student population of the country.
Reflecting on the current issue of the higher education sector in Vietnam, the
Mekong Delta is experiencing difficulties in meeting the training needs of
students who have graduated from high school and are ready for higher
education. Due to the limited training capacity of regional universities and
colleges, only 22.2% of high-school graduates have been offered places in training
programs by local colleges and universities. The establishment of new universities
in the region still has not been able to accommodate demands (Ministry of
Education and Training (MOET), 2019b).
Another issue facing training institutions in the Mekong Delta concerns the
training of a qualified labor force that can meet the region’s needs. Almost all the
sectors in the region have reported a lack of qualified human resources (Hong,
2015). It has been estimated that, by 2025, the Mekong Delta will need up to
500,000 trained laborers to fully satisfy the requirement of trained human
resources in all sectors, especially in the fields of aquaculture, technology,
engineering, business, and agriculture (Ministry of Education and Training
(MOET), 2019b). What this means for the region is that over 300,000 graduates (or
three-fifths of the total figure) will have to be trained to fill in the gap in the next
5 years. For healthcare, currently, only 60% of health centers have active nurses
while the region aims for 90% coverage. Similarly, the tourism sector presently
has only 100,000 qualified laborers while the targeted figure for 2020 is 236,000.
With the currently low training capacity of universities and colleges in the
Mekong Delta, MOET (2019b) contends that the region will remain short of a
qualified human resource in the long run and accordingly fails to meet the
increasing investments being poured into the region. According to a report by the
Vietnam Chamber of Commerce and Industry (VCCI), as of June 2019, the
Mekong Delta has attracted 1,609 foreign direct investments with the registered
capital totaling 22.3 billion US dollars (Ho, 2019). This entails an increase in
employment from companies and entrepreneurs for a qualified human resource
that can acquire not only a compatible insight into the theoretical bases but also a
mastery of related skills.

156
To provide a sufficient and sustained source of qualified labor for the region,
higher education institutions in the Mekong Delta needs to consider mobilizing
resources to launch courses more efficiently. CE should be included in
institutional initiatives for that purpose. Presently, out of 43 institutions in the
region, only three have implemented CE for a total of 18 disciplines (P. Pham,
2017). At these three institutions, seven disciplines that have not taken enrolments
for online CE training are crop science, industrial management, information
technology, veterinary medicine, tourism, literature, and environmental and
resource management. CE in the Mekong Delta region needs both urgent and on-
going improvement and completion in the management schemes since the gap in
figures of graduates who need training is even higher than the current available.
This task is more daunting when the needs for employment are increasing for
qualified graduates with the mushrooming of foreign projects and companies.
Markedly, education in the Mekong Delta region requires more investment in
terms of appropriate policies and finance made by the authorities and educational
managers from tertiary educational institutions.
2.3. The desired outcomes of continuing education in the Mekong Delta

In the context of numerous shortages in terms of funds, cutting-edge programs,
materials, infrastructures, and legal systems that continuing education in the
Mekong Delta is faced with, CE still undertakes the role of a reliable provider of
human resources for society. The graduated laborers are, thus, expected to be
sufficiently qualified to contribute to the social and economic development of the
Mekong Delta region (Duong, 2015; P.T. Pham, 2017; Vietnamese Government,
2011). In other words, students graduating from CE courses should be recognized
and efficiently employed in society. That does not mean any students holding a
diploma can meet the requirements. Indeed, those students must not only be fully
equipped with good theoretical knowledge but also with sound practical skills
related to their majors to be effective workers in their future jobs. On top of that,
a professional human source that can adapt to and keep pace with the
international level and advancement is highly expected to be produced by
continuing educational institutions (Duong, 2015; Vietnamese Government, 2011).
To assess the outcomes of continuing education, Griscti and Jacono (2006) and
Barriball, While and Norman (1992) find little evidence about tools to measure the
effectiveness or frequency of CE. However, several criteria for assessment have
been put forward (Brennan, 1997; Duong, 2015; Romi & Schmida, 2009). The major
ones include enrolment, completion rates, output performance, access to
disadvantaged, cost recovery, the motivation of students, acceptance of the
program by the society, and social and economic benefits to the country (Welsh &
Dey, 2002).
To achieve the desired outcomes, CE management must always be prioritized.

Edirisingha (1999) highlights the combination of managing in the input, the
output, and the influential elements of management that are core to the
development of CE. Rumble (1997) suggests the four main contents as the
influential factors to the implementation of CE, including material, logistical,
learning process, and regulatory management. Material condition is the
management of the design, publication, and delivery of course books. The

157
logistical condition involves management in personnel, funding, and promotion.

The learning process is managed through media, tutoring, and assessment.
Regulatory management is focused on institutional structure, management
structure, and collaboration with other institutions.
Edirisingha (1999) also emphasizes the importance of the management combining

the conditions above with the input and output of the whole CE system to bring
about the most desirable outcomes. The Mekong Delta is a developing region of
Vietnam, which requires a large amount of workforce who can live up to society’s
and entrepreneurs’ expectations. Thus, managing a CE system that can attract
higher-level input and encourage the activeness of each condition to produce the
desired output is both a must and a challenge to administrators of education in
the Mekong Delta. For the purpose of sustainable development of the whole
country, the achievements that the Mekong Delta has achieved and will achieve
in CE education will play a pivotal role in boosting the social and economic chain
of the whole country to develop.
3. Research methodology
3.1. Research instrument
This study employed a mixed methodology with document analysis and a
quantitative research method to find answers to the three research questions. As
far as the first research question is concerned, what are the differences in the input
and output of continuing education through DL and F2F courses, demographic
information based on the statistical data recorded by a university in the Mekong
Delta was analyzed. The five indicators of the effectiveness of an educational
program introduced by Welsh and Dey (2002) were utilized in the research. They
include admitted enrolment (AE), average admission score (AAS), graduation (G),
the average final score (AFS), and employment after graduation (EAG). These five
aspects were investigated in two modes, face-to-face (F2F) and distance (DL) in
eleven majors of the university which had enrolment for CE in both F2F and DL
courses, namely law, veterinary medicine, accountancy, TESOL, resources and
environment management, business management, construction engineering,
vegetation protection, food technology, seafood processing management, and
information technology. Drawing on those preliminary results, a questionnaire
for 100 teachers and 20 administrators from the university was employed. The
teachers and administrators selected were those having experiences in teaching
and managing both DL and F2F courses in CE at the university.
The questionnaire included 75 Likert items, which served to explore the priority
in management that each content received and the current situation of CE. A score
of 1 represented the least significant condition (the lowest level of agreement or
satisfaction) and a score of 5 represented the most significant one (the highest level
of agreement or satisfaction). Three more questions asking about explanations for
their answers were also added. The core factors surveyed were based on those
suggested by Rumble (1997), which were synthesized into four major conditions
that need supervision to ensure the implementation of CE. They were material,
logistical, learning process, and regulatory conditions each of which contained
three elements that are coded in Table 1.

158
Table 1. Coding of management contents and sub-contents

Content Sub-content Details of each content Code
Instruction Coursebook design M1
Material (M) Production Coursebook publication M2
Distribution Coursebooks’ delivery M3
Personnel Teacher, officers, and administrators L1
Logistical (L) Finance Funding L2
Marketing Promotion (upgrade the image) L3
Media Tools used for delivering lessons LP1
Learning
Tutoring Teacher-student interactions LP2
process (LP)
Assessment Grade point average (GPA) based LP3
Planning Governance, administrative structure, R1
institutional status
Regulatory (R) Managing Management structure R2
Decision Collaboration with other institutions, R3
making nationally and internationally
According to Rumble (1997), the four contents illustrated in Table 1 are

fundamental to construct and manage comprehensive and efficient online and
face-to-face programs, the lack of any of which would lead to failure in
implementations of the programs. Building these four contents is to facilitate
learners and teachers with materials, infrastructure, methodology, and
curriculum to implement course classes. To add, managing the implementation of
these four contents is also core to successful educational management (Nolan,
Owens, & Nolan, 1995). To this end, the current study took these four contents for
both qualitative and quantitative research.
3.2. Sampling
The university surveyed was one of the biggest in the Mekong Delta. It was the
first accredited educational institution to adopt CE in the region. And so far, the
university has gained prestige for its quality and quantity of CE in the region.
Regarding the document analysis, the statistics were collected from the training
department of the university for the year 2019. A total of 1642 students admitted
to CE courses and 1531 students were qualified for graduation in 2019. One
hundred teachers surveyed from eleven faculties of the university were invited to
participate in the survey. Besides, 20 administrators who, at the time of the survey,
were serving as from the principal to associate deans of these faculties were
involved in the survey.
3.3. Data analysis

The data collected from the Training Department were entered into a spreadsheet
to provide an overview of the input and output of CE through DL and F2F
courses.
Additionally, responses from the survey were coded and entered in SPSS Version
20 and checked for reliability using the Cronbach's α reliability estimate
(α=Nρ/[1+ρ(N-1)]). A high-reliability coefficient was achieved, with the
Cronbach’s α estimates ranging from 0.722 to 0.911 (≥ 0.7) (Table 2). The corrected

159
item-total correlation estimates were also higher than 0.3, showing good
correlations between the variables. The other two indices that improve the
reliability of variables, namely composite reliability (CR) and average variance
extract (AVE), were also examined. The results reveal high reliability of variables
with CR ranging from 0.788 to 0.858 (≥ 0.5) and AVE from 0.523 to 0.636 (≥ 0.5).
Table 2. Reliability estimates

Corrected
Cronbach's
Factors Observed variables item-total CR AVE
Alpha
correlation
M1_DL, M2_DL,
M_DL 0.722 - 0.728 > 0.3 0.788 - 0.794 0.523 – 0.534
M3_DL
M1_F2F, M2_F2F,
M_F2F 0.727 - 0.731 > 0.3 0.804 - 0.811 0.551 – 0.558
M3_F2F
L1_DL, L2_DL,
L_DL 0.883 - 0.886 > 0.3 0.829 - 0.837 0.609 – 0.618
L3_DL
L1_F2F, L2_F2F,
L_F2F 0.879 - 0.892 > 0.3 0.854 - 0.858 0.614 – 0.636
L3_F2F
LP1_DL, LP2_DL,
LP_DL 0.731 - 0.737 > 0.3 0.796 - 0.805 0.527 – 0.558
LP3_DL
LP1_F2F, LP2_F2F,
LP_F2F 0.765 - 0.786 > 0.3 0.816 - 0.823 0.578 – 0.593
LP3_F2F
R1_DL, R2_DL,
R_DL 8.893 - 0.902 > 0.3 0.823 - 0.842 0.568 - 0.589
R3_DL
R1_F2F, R2_F2F,
R_F2F 0.894 - 0.911 > 0.3 0.844 - 0.851 0.583 – 0.604
R2_F2F
M1_DL: M1 in distance learning courses, M1_F2F: M1 in face-to-face courses
L1_DL: L1 in distance learning courses, L1_F2F: L1 in face-to-face courses
LP1_DL: M1 in distance learning courses, LP1_F2F: M1 in face-to-face courses
R1_DL: M1 in distance learning courses, R1_F2F: M1 in face-to-face courses
Following a reliability check, an Exploratory Factor Analysis (EFA) was

conducted to determine the correlation between their factor loadings. Kaiser-
Meyer-Olkin (KMO) Test = 0.736 (satisfying 0.55 ≤ KMO ≤ 1), Sig Barlett’s Test =
0.000 (<0.05) (Table 3) showed good correlations between the observed variables.
Table 3. KMO and Barlett’s Test

Kaiser-Meyer-Olkin Measure of Sampling Adequacy .736
Approx. Chi-Square 3528.693
Bartlett's Test of Sphericity Df 372
Sig. .000
The Rotated Component Matrix (Table 4) showed that the 10 observable variables
loaded on 4 factors and there were good correlations between the variables.

160
Table 4. Rotated Component Matrix

Component
1 2 3 4
M1 .702
M2 .691
M3 .684
L1 .688
L2 .663
L3 .634
LP1 .651
LP2 .617
LP3 .603
R1 .615
R2 .592
R3 .589
The F-ratio in the ANOVA table (Table 5) tests whether the overall model is a
good fit for the data. The table shows that the independent variables statistically
significantly predict the dependent variable, F = 62.720, p = .000 < .0005.
Table 5. ANOVA
Sum of Mean
Model Df F Sig.
Squares Square
Regression 81.356 6 13.559 62.720 .000b
1 Residual 84.313 390 .216
Total 165.668 396
4. Findings and discussion

4.1. The input and output of continuing education via F2F and DL courses
In this research, the input of CE via DL and F2F was investigated based on
admitted enrolment and average admission scores in 11 majors of the university.
The document analyses reveal that though the number of students admitted to
F2F classes was noticeably greater than that of DL classes, the students admitted
to 2 types of classes had similar high-school academic records. The finding
confirms what was concluded in the research by P. Pham (2017) in which even
though the DL courses do not attract as many students as F2F, there is an
insignificant difference in the quality of students enrolling F2F and DL courses of
CE.
Preferences for CE were also markedly different in each sector of college.

Specifically, the most preferable sector belonged to law with 61.2% and 52.4% in
F2F and DL respectively. Yet, the least preferable belonged to information-library
with 1.2% in F2F and 1.5% in DL. This outcome shows a noticeable shift in choices
of disciplines which was mainly economics (at 30%) in 2011 (MOET, 2010). The
differences in choices of major also reflect an imbalance in employment structure
in the society. This leads to redundancy in law graduation, yet shortages in the

161
library. Such imbalance in choices of academic majors is affected by the lack of

career-orientation for high school students (Duong, 2015) or the trendy preference
to work for managerial and office work in developing countries (Madhi &
Barrientos, 2003). Such shortage leads to perspective or trendy rather than
practical and necessary choices for future careers (Khurshid, 2014). When
examining the data synthesized in the previous years (from 2016 to 2019), a
reserve trend of enrolment is observed in both types of courses in CE. Specifically,
from 2016 to 2019, there was a downward trend of enrolment in both F2F (from
2093 to 986) and DL (from 954 to 300) (See Table 6). However, in 2019, enrolment
in F2F and DL classes experienced a remarkable rise to 1098 and 544 respectively.
Such a reverse trend confirmed what Duong (2015) had found about the
increasing popularity of CE in the Mekong Delta. Therefore, a foreseeable future
of a bigger contribution of CE to human resource training can be revealed.
Table 6. Students admitted and enrolled in CE courses at the university (from 2016 to 2018)
Course type Year Admitted Enrolled
2016 2,577 2,093
2017 3,415 2,477
F2F
2018 1,235 986
2019 1,098
2016 966 954
2017 634 588
DL
2018 343 300
2019 544
4.2. Comparing the output of F2F and DL education

Three contents concerning the output of CE which were investigated in this
research include graduation, average final score, and employment after
graduation. Data analyses showed significant differences in the figures of
graduated as well as employed students after graduation between F2F and DL
courses. To be more specific, the percentage of graduate students from F2F
courses was nearly twice as large as that from DL courses (68.8% to 31.2%
respectively). Similarly, the employment rate after graduation of F2F courses was
around two-thirds of the total figure (68.9%). This finding is a confirmation that
F2F courses dominated DL ones with a significantly higher figure of graduation
and employment after graduation (Jahng et al., 2007). This is understandable due
to the big difference in enrollment between the two-course types that can entail
the corresponding difference in the output in terms of graduation, employment,
and major preference. On another note, despite its lower figure in both graduation
and employment, DL courses experienced slightly higher final average scores.
Besides, the highest limit of Grade Point Average (GPA) in DL courses was higher
than that in F2F courses. Specifically, 2 cases of graduation in law_DL were
qualified as the final good cumulative GPA (>3.2). Simultaneously, no final good
cumulative GPA was found in any surveyed F2F courses. These finding illustrates
the quality of students in DL courses which has been improved and is potential to
surpass that in F2F courses. Also, the ratio of graduation (output) to enrolment
(input) in F2F courses saw no significant difference to that in DL courses. A similar
level of difference between the two-course types was found in the ratios of

162
employment after graduation to graduation. Detailed figures are shown in Table

7.
Table 7. Ratios of input and output of DL and F2F in CE

Course type Ratio E/G Ratio EAG/G
Qualified Unqualified Employed Unemployed
F2F 93.2% 6.8% 67.9% 32.1%
DL 85.5% 14.5% 67.3% 32.7%
Overall, the outcome reveals the high level of course completion as well as the
capacity to provide the trained labor force for the Mekong Delta region through
continuing education. The level also confirms the high competitiveness of
distance learning to face-to-face learning in quality and expected results. In other
words, DL, though attracting about half as many enrollments as F2F, proved to be
valuable and of equally crucial contribution to fulfilling the employment to the
F2F. However, the comparison between the numbers of human resource required
for the whole Mekong delta region in 2025 (500,000 qualified laborers per year)
with the current available graduated students from the university through CE
(1489 students), it can be seen that CE in the Mekong Delta still has a long way to
reach its targets in catering to the increasing requirements of the human resource
of the whole region.
4.3. Managing the implications of CE in the Mekong Delta

4.3.1. Managing CE through DL and F2F courses
In this regard, twelve elements of four core critical conditions were examined.
Administrators were asked on the extent to which they agree on each element
received priority in the management of the university during the implementation
of CE through DL and F2F. The mean scores of answers reveal medium priority
with little significant difference between the two mediums. Specifically,
regulatory management in terms of managing, decision making, and planning
received the highest appreciation respectively, whereas tutoring the lowest in
both DL and F2F courses. The other contents had their mean scores of more than
2.7, which shows a middle level of appreciation. This finding is echoed by many
studies on the issues of non-formal education management. Edirisingha (1999),
Khurshid (2014), Duong (2015), and P. Pham (2017) came to the similar conclusion
that regulatory management receives the highest priority as it plays the most
crucial role in continuing education. That is, if the framework of laws and policies
for non-formal education is invested initially, it will effectively integrate CE into
the broader national education system (Nolan et al., 1995; T.D. Pham, 2018).
Additionally, logistical management in terms of personnel, finance, and
marketing received the second-highest priority in CE management in both DL and
F2F courses (see Table 8). Finance and marketing are agreed to contribute to the
wider accessibility and feasibility of CE while personnel to the quality and
preference (Edirisingha, 1999; Rumble, 1997).

163
Table 8. The priority of each condition received in management of DL and F2F in

continuing education
Priority in Standard Priority in Standard
management DL deviation management F2F deviation
M1 3.12 0.67 3.17 0.63
M2 2.83 0.71 3.01 0.67
M3 2.72 0.73 2.89 0.75
L1 3.51 0.58 3.6 0.54
L2 3.45 0.62 3.4 0.59
L3 3.56 0.57 3.61 0.55
LP1 2.92 0.76 2.95 0.71
LP2 2.56 0.88 2.75 0.85
LP3 3.17 0.69 3.21 0.61
R1 3.48 0.59 3.54 0.57
R2 3.62 0.53 3.77 0.50
R3 3.6 0.54 3.76 0.52
4.3.2. The current situation of non-educational management via DL and F2F courses
The twelve elements were brought into the questionnaire to investigate the
current situation of managing non-educational operation via the two education
mediums in the university. The results disclose a rather low satisfaction among
150 subjects in both course types (Table 9).
Table 9. Effectiveness of management of each condition in implementing CE

Mean Standard Mean Standard
(DL) deviation (F2F) deviation
M1 3.16 0.78 3.21 0.74
M2 3.31 0.73 3.37 0.67
M3 3.28 0.74 3.33 0.69
L1 3.42 0.68 3.59 0.63
L2 2.52 0.85 2.64 0.83
L3 2.89 0.80 2.51 0.89
LP1 2.73 0.83 2.96 0.79
LP2 3.58 0.64 3.31 0.70
LP3 3.12 0.77 3.18 0.76
R1 3.21 0.80 3.35 0.68
R2 3.5 0.66 3.66 0.62
R3 3.33 0.71 3.58 0.65
On the one hand, when asked about the extent to which current management in
each regulatory element is effective to the development of continuing education?”
the respondents show a higher mean score in F2F than in DL courses. In other
terms, while regulatory in F2F courses was rated the best-managed group, it was
the second best-managed group in DL classes. In other words, the importance of
regulatory was best realized by managers in F2F courses and second-best in DL
courses. This is also concluded in several surveys that recognize the efforts of the

164
managing and authorizing board in building an effective framework of policies

and strategies for the development of CE. Regulatory management was even
highlighted as urgent due to the increasing demand for a qualified human source
in the region (Duong, 2015; Edirisingha, 1999; Rumble, 1997).
On the other hand, despite the significant role that the logistical group played in
the management of CE, this group received incompatible low ratings in finance
and marketing. In detail, more than 95% of surveyed subjects agreed upon the
limited funding and marketing that both DL and F2F courses received. This
finding was previously confirmed by UNESCO (2008) which emphasizes the
extremely limited budget and promotion for CE. Such limitations in budget and
marketing create a huge obstacle for tertiary institutions to guarantee long-term
and equitable provision to people in the region, and thus negatively affect
fulfilling the need for qualified human resources (MOET, 2010; P.T Pham, 2017;
Vietnamese Government, 2011). Under that recognition, UNESCO (2014)
recommended educational institutions involve the contribution of NGOs to
supply funds for bettering the quality of CE. Exceptionally, personnel is one
element of this group that received compatible management to its pivotal role.
The content was rated the third most considered of all in DL and F2F courses.
However, this result is a reverse of UNESCO (2014) stating that only about 25%
of development plans included teacher training in CE. Again, UNESCO (2014)
brought in NGOs as a reliable source of providing teacher and manager training
and related services to enhance personnel quality.
Materials and learning received medium satisfaction in management.

Particularly, DL and F2F tutoring management was one of the best whereas media
was one of the worst. Such a lack of media investment is explained to be caused
by the lack and the uneven distribution in the budget for media (Duong, 2015; P.T.
Pham, 2014, 2017). Shortages in media have particularly a direct impact on the
implementation of DL classes, and thus needs more consideration from the
authorizing bodies.
4.3.3. The capacity of training qualified human resources catering to the needs of the
society.
For this concern, five focuses namely seafood, technology, engineering, business,
and agriculture on the employment structure of the Mekong Delta region were
investigated. When asked the question “to what extent do you think that the needs
of human resources majoring in each sector are high?” the answers revealed
significant differences in their assessment for the requirement of human resources
in each sector. Specifically, the sector that needs to provide the trained human
resource the most was technology whereas the least was business. This is echoed
by the conclusion of the Ministry of Education and Training (MOET) (2019b)
which highlights the higher needs of technical rather than managerial and
economic major laborers. The finding also matches with the current situation in
the society where there is an uneven distribution of graduated students in
different majors, favoring economics and management (P.T. Pham, 2017).
When asked for their opinion about the extent to which they think that CE at the
university can contribute to the completion of the educational system in

165
generating 500,000 tertiary-level laborers per year until 2025, responses reveal
differences among the investigated majors. In detail, majors belonging to business
received the highest rate of agreement in satisfying the human resource need
(M=3.07). Simultaneously, those belonging to engineering and technology
received the lowest rate (M=2.53) (Table 10).
Table 10. Correlation between labor needs and capacity to provide required laborers
Seafood Technology Engineering Business Agriculture
Needs of
4.0 4.63 4.21 3.82 3.95
qualified labors
Capacity to
provide the 2.61 2.55 2.51 3.0 2.75
needed labors
The table reveals that the biggest mean difference between the need for laborers
in the society and the capacity that CE can train for the society belongs to
technology (MD = 2.08) whereas business the smallest (MD = 0.82). Referring back
to the analysis of the input and output, this trend can be explained to result from
the unevenly distributed choices of majors among students. That means, more
students enrolled in business majors than technology and engineering in both DL
and F2F courses and therefore the number of graduate students in business majors
was far higher than that in technology and engineering. More importantly, in the
assessments of respondents, CE at the university did not have very high capacity
in providing sufficient labors catering to the needs of society in all surveyed
sectors (only from 2.53 to 3.07). The result poses challenges for administrators and
teachers to make major shifts in CE to achieve the goal of 500,000 labors by 2025.
It was recommended in several documents that CE management in both DL and
F2F in Vietnam should focus on addressing the urgent problem in capital
shortages and regulatory incompletion (P.T. Pham, 2017; T.D. Pham, 2018). One
of the biggest efforts to be recognized is an amendment in Educational Law in
higher education in July 2019, removing the distinction in the diplomas of formal
and non-formal education (P.T. Pham, 2018). Such a move is, agreed by 82% of
respondents, considered important in triggering the development of non-formal
education if it is well-managed. However, it is believed to be impeding and even
reversing the development of the educational system in general if management in
non-formal education is ignored or loosened due to shortages in facilities,
resources, and management capacity. In the context of a developing country, DL
and F2F non-formal education cannot be prioritized over the formal ones
(Alamgir, 1999; Duong, 2015; Mnyanyi & Mbwette, 2009). In fact, non-formal
education received the least portion of the total budget for education from the
Vietnamese Government with no more than 1.8% from 2008 to 2014 (Dinh, 2017).
At the same time, the need for quality human resources in the Mekong Delta is
increasingly high in terms of quantity and quality. Upgrading the quality and
quantity of non-education management is, as such, becoming more urgent than
before. Also, the analysis of output in this research shows that even though the
mean graduation score only ranged from 2.0 to 3.2, the rate of employment of the
graduated students was quite high. When asked about the confidence in the
success in building an effective CE, meeting the needs for human resources of
500,000 qualified laborers in 2025, 45.7% of respondents were confident whereas

166
only 18.5% were totally and very confident. The rest of 31.8% were rather
confident in the capacity of the university management in building a qualified
non-formal education system. The data reveal rather high level of confidence in
the success of the system.
In summary, although the fundamental role of F2F and DL continuing education

in providing a timely and qualified human resource to the region is highly
recognized among teachers and administrators, the reality is still having a long
way far from the expected results. However, a majority of respondents are still
confident in the success of the CE system in contributing to providing the required
number of qualified human resources for the Mekong Delta region in 2025.
5. Concluding remarks
Continuing education (CE) has been highlighted as a solution to educational
development and socio-economic sustainability. For the Mekong Delta region of
Vietnam, to achieve the goal of training 500,000 qualified labors by 2025, CE, in
both DL and F2F training modes, plays an indispensable role. The current research
reveals an increasing enrolment trend in CE-based courses in the Mekong Delta,
with both DL and F2F actively contributing to the thriving of the region’s CE. DL
courses, in particular, have managed to narrow the gap in enrolment and
employment rates with F2F training courses and confirm their competitive
quality. This study recognizes the effort of administrators and teachers in
managing the implication of CE and delivering lessons via both DL and F2F
modes. However, it points out that many CE courses still unevenly focus their
intention on areas of training that do not require substantial investment in training
facilities, for example, business and management disciplines, while failing to
adequately invest in technological fields where the region is in strong need of
qualified human resources. Among the four contents related to managing CE
courses, namely the material, logistical, learning process, and regulatory aspects,
the issues of regulatory management and logistical management are identified by
key stakeholders to be at the top priorities. However, improvements in these two
aspects are challenging due to a notable lack of financial resources and insufficient
promotion of related CE courses. This substantially affects the capacity of the
region’s education system in general and the region’s CE in particular. Despite the
difficulties and challenges, the study confirms the confidence among stakeholders
in the success of CE in developing a sustainable education system that caters to
the increasing demand for a qualified and high-quality workforce of the region.
6. Limitations and Implications

This study was conducted with several unavoidable limitations, which can be
fruitful for further academic implications. Firstly, the scope of the study was
limited to one university in the region with 130 subjects. This can be widened to
more universities and involve more administrators from higher positions to have
a broader and more reliable overview of the situation. Besides, the analysis of the
output of the study can gain further investigation into higher academic
advancement or professional promotion after graduation rather than only
employment. Also, the study provides an overview of four contents in general,
which can open to further research into each content or sub-content.

167
For managerial implications, this study mainly worked on management and thus
it can contribute to the improvement of the management of F2F and DL
continuing education by both the government and by institutional managers. To
start with, it is a good sign that the national policies have re-positioned the role of
non-formal education within the national education systems by recognizing no
difference in its diploma with formal education. Nevertheless, it is important to
explore more about the relevance of CE to formal education to make better
coordination and supplementation between the two modes and thus strengthen
the development of the tertiary education system in the region. Furthermore, the
quality of the output remains controversial since there is no standard and fixed
tools of measurement for this regard. As such, the administrators need to invest
more in building an effective measurement for output quality to better assess the
quality of CE in tertiary institutions. To add, the biggest challenge is mainly
related to inadequate financial support, which hinders the implications of CE,
especially DL courses. Therefore, managers need to encourage more coordinated
partnerships from NGOs and the private sectors, donors, and communities. In the
context of economic difficulties and uneven distribution of the Government
budget, the assistance of non-government and private organizations will be
valuable. Since the Mekong Delta region is having its general economy stimulated
and improved by foreign bodies, such financial assistance in education is
mobilizable if educational administrators have an encouraging and opening
strategy in fostering the investment of the external stakeholders. Regulatory
management is believed to be improved the most among the investigated
contents. Nonetheless, more efforts are required for further enhancement of
national legal frameworks and full implementation of legislation, particularly at
regional and institutional levels. This can be done by developing further databases
concerning non-formal education and integrating monitoring systems for
effective policy management for CE. Another step to be taken is enhancing the
capacities of institutions and education personnel with training programs under
the investment and guidance of the Government and NGOs. This should be
carried out in parallel with designating clear objectives and reliable measures in
realizing a vision in the curriculum, teaching and learning materials, and
assessment in F2F and DL courses.
For educational implications, this research makes a significant contribution to the

fundamental shift in the operation of educational virtues from preferable brick-
and-mortar classrooms to online classrooms under the impact of the global Covid-
19 pandemic. The pandemic hitting almost every country, including Vietnam, has
frozen many sectors, including education. Without the implementation of virtual
classrooms, the effects of the downturn of education and thus of many economies
would be immeasurable. However, according to Dinh and Nguyen (2020), the
effectiveness of operating online classes of all educational levels, especially of
higher education remains questionable. This research can therefore open up
several issues to a more in-depth investigation, including management of hybrid
education, facilitation of technological platform, materials, and curriculums. By
confirming the importance and the reasonability of continuing education by
applying for online courses, the current study sets a steppingstone to a stronger
development of online courses in the future in developing countries like Vietnam.

168
Acknowledgment:
Funding: No funding
Compliance with Ethical Standards:
The authors assure that this research paper was done in compliance with Ethical
Standards.
Conflict of Interest:
The authors declare that they have no conflict of interest in this research.
7. References
Alamgir, A. (1999). Non-formal education programmes in bangladesh: A first step for developing
countries. Paper presented at the International Conference of Commonwealth of
Learning, Brunei, 01-05 March.
Allen, I., & Seaman, J. (2011). Going the distance: Online education in the United States in 2011.
United States: Babson Survey Research Group and Quahog Research Group.
Association for Development of Education in Africa (ADEA). (2012). Strategic orientation
framework for non-formal education in a holistic, integrated and diversified vision of
lifelong education. Burkina Faso: ADEA Working Group on Non-Formal
Education.
Barriball, L., While, A., & Norman, I. (1992). Continuing professional education for nurses:
A review of literature. Journal of Advanced Nursing, 17(9), 1129–1140.
Berg, M., & Seeber, B. (2016). The slow professor: Challenging the culture of speed in the
academy. Toronto: University of Toronto Press.
Bernard, R., Lou, Y., Abrami, P., Wozney, L., Borokhovski, E., Wallet, P., . . . Fiset, M.
(2004). How does distance education compare to classroom instruction? A meta-
analysis of the empirical literature. Review of Educational Research, 74(3), 379-493.
Berti, M. (2018). Open educational resources in higher education. Issues and Trends in
Learning Technologies, 6(1), 4-15.
Brennan, B. (1997). Reconceptualizing non-formal education. International Journal of
Lifelong Education, 16(3), 185-200.
Council of Europe. (2003). Non-formal education. Brussels: Council of Europe.
Dang, Q., & Nguyen, D. (2004). Vietnamese education towards the future: Problems and
solutions. Hanoi: National Politics Publishing House.
Delors, J., Mufti, I., Amagi, I., Carneiro, R., Chung, F., Geremek, B., . . . Nanzhao, Z. (1996).
Learning: The treasure within. Paris, France: UNESCO.
Dinh, L., & Nguyen, T. (2020). Pandemic, social distancing, and social work education:
Students’ satisfaction with online education in Vietnam. Social Work Education,
32(5), 1-10.
Dinh, T. (2017). Governmental investment in education and training: Situation and
recommendations. Financial Magazine. Retrieved from
http://tapchitaichinh.vn/nghien-cuu-trao-doi/dau-tu-cua-nha-nuoc-cho-giao-
duc-dao-tao-thuc-trang-va-mot-so-de-xuat-130918.html. (accessed on October 29,
2017)
Duong, D. (2015). Human resource training in Mekong Delta: Situation and solutions.
Development and Integration, 21(31), 78-81.
Edirisingha, P. (1999). Open and distance learning for basic and non-formal education in
developing countries. Brunei: Pan-Commonwealth Forum on Open Learning.
Faure, E. (1972). Learning to be: The world of education today and tomorrow. Paris: UNESCO.
Gadekar, D. (2020). Level of human resources development: A conceptual and review
exposition. International Journal for Research in Applied Science & Engineering
Technology, 8(3), 687-691.

169
Griscti, O., & Jacono, J. (2006). Effectiveness of continuing education programmes in

nursing: Literature review. Journal of Advanced Nursing, 55(4), 449-456.
Ho, P. (2019, October 11). Solutions to balancing the labor market in the Mekong Delta.
Soctrang. Retrieved from http://baosoctrang.org.vn/doi-song-xa-hoi/giai-phap-
can-bang-thi-truong-lao-dong-dang-thieu-hut-tai-dbscl-31427.html (accessed on
May 24, 2020)
Hong, H. (2015, November 16). The human resource for the Mekong Delta region. Baclieu.
Retrieved from http://baobaclieu.vn/chinh-tri/nguon-nhan-luc-cho-dong-
bang-song-cuu-long-33810.html (accessed on May 24, 2020)
Hong, K. (2002). Relationships between students' and instructional variables with
satisfaction and learning from a web-based course. Internet and Higher Education,
5(3), 267-281.
Hoppers, W. (2006). Non-formal education and basic education reform: A conceptual review.
Paris: UNESCO International Institute for Educational Planning.
Jahng, N., Krug, D., & Zhang, Z. (2007). Student achievement in online distance education
compared to face-to-face education. European Journal of Open, Distance and E-
Learning, 10(1), 1-12.
Khurshid, Z. (2014). Continuing education for human resource development in the
Arabian Gulf region. International Journal of Continuing Engineering Education and
Life-Long Learning, 7(1), 1-12.
King, K. (1982). Formal, non-formal and informal learning: Some north-south contrasts.
International Review of Educational Research, 28(2), 177-187.
Kleinman, J., & Entin, E. (2002). Comparison of in-class and distance-learning students'
performance and attitudes in an introductory computer science course. The Journal
of Computing in Small Colleges, 17(6), 206-219.
Lege, R., Bonner, E., Frazier, E., & Pascucci, L. (2020). Pedagogical considerations for
successful implementation of virtual reality in the language classroom. In M. Kruk
& m. Peterson (Eds.), New technological applications for foreign and second language
learning and teaching (pp. 24-46). Pennsylvania, USA: IGI Global.
Madhi, S., & Barrientos, A. (2003). Saudisation and employment in Saudi Arabia. Career
Development International, 8(2), 74-82.
Magagula, C., & Ngwenya, A. (2004). A comparative analysis of the academic
performance of distance and on-campus learners. Turkish Online Journal of Distance
Education-TOJDE, 5(4), 1-11.
McGuire, C., & Castle, S. (2010). An analysis of student self-assessment of online, blended,
and face-to-face learning environments: Implications for sustainable education
delivery. International Education Studies, 3(3), 36-40.
Ministry of Education and Training (MOET) (2010). Report on planning the network of tertiary
institutions in the Mekong Delta region from 2011 to 2020. Hanoi: Ministry of
Education.
Ministry of Education and Training (MOET). (2019a). General educational statistics. Hanoi,
Vietnam: Ministry of Education and Training. Retrieved from
http://csdl.moet.gov.vn/.
Ministry of Education and Training (MOET). (2019b, November 18). The labor market in
Hochiminh city and other key economic regions in the South of Vietnam.
Educational Magazine. Retrieved from
https://tapchigiaoduc.moet.gov.vn/vi/dao-tao-viec-lam/dao-tao-viec-lam/thi-
truong-lao-dong-cua-thanh-pho-ho-chi-minh-va-cac-vung-kinh-te-trong-diem-
phia-nam-90.html (assessed on May 24, 2020)
Mnyanyi, C., & Mbwette, T. (2009). Open and Distance Learning in Developing Countries: The
Past, the Present, and the Future. Dares Salaam: Open University of Tanzania.

170
Nolan, M., Owens, R., & Nolan, J. (1995). Continuing professional education: Identifying
the characteristics of an effective system. Journal of Advanced Nursing, 21(1), 551-
560.
Ochukut, S., & Omwansa, T. (2016). Implementation of open education resources in developing
countries. Paper presented at the Nairobi Innovation Week, Nairobi, 01-05 August.
Pham, P. T. (2014). Distance training management catering to tertiary-leveled human
resource: A review of the literature. Journal of Educational Sciences, Special issue,
October 2014, 64-66.
Pham, P. T. (2017). Management of continuing education catering to well-qualified human
resources in Mekong Delta. Hanoi: Vietnam National University Press.
Pham, T. D. (2018). Continuing education in a learning society. Hanoi: Ministry of Education
and Training. Retrieved from https://moet.gov.vn/giaoducquocdan/giao-duc-
thuong-xuyen/Pages/default.aspx?ItemID=5441 (accessed on May 15, 2018)
Phipps, R., & Merisotis, J. (1999). What’s the difference? A review of contemporary research on
the effectiveness of distance learning in higher education. Washington, DC: The
Institute for Higher Education Policy.
Puthe, M. (2008). E-learning concepts and literature review. In M. Fuad & M. Salleh (Eds.),
E-learning issues in Malaysia higher education (pp. 1-22). Malaysia: Teknologi
University.
Rogers, A. (2004). Non-formal education: Flexible schooling or participatory education?
Hongkong: The University of Hongkong.
Romi, S., & Schmida, M. (2009). Non-formal education: A major educational force in the
postmodern era. Cambridge Journal of Education, 39(2), 257-273.
Rovai, A. (2002). A preliminary look at the structural differences of higher education
classroom communities in traditional and ALN courses. Journal of Asynchronous
Learning Networks, 6(1), 41-56.
Rumble, G. (1997). The costs and economics of open and distance learning. London: Kogan
Page.
Sener, J. (2012). The seven futures of American education: Improving learning and teaching in a
screen captured world. North Charleston: Create Space.
Shachar, M., & Neumann, Y. (2003). Differences between traditional and distance
education academic performances: A meta-analytic approach. The International
Review of Research in Open and Distributed Learning, 4(2), 1-20.
Shelton, K., & Saltsman, G. (2005). An administrator’s guide to online education. Greenwich,
CT: Information Age Publishing.
Situma, D. (2015). Open and distance learning and information and communication
technologies-implications for formal and non-formal education: A Kenyan case.
Journal of Learning for Development, 2(1), 1-14.
Statistics Directorate. (2016). Report on labour and employment in Vietnam in 2016. Paris:
Statistics Publishing House.
UNESCO. (2002). Innovations in non-formal education: A review of selected initiatives from the
Asia-Pacific region. Bangkok: UNESCO.
UNESCO. (2008). EFA global monitoring report: Overcoming inequality – why governance
matters. Paris: UNESCO.
UNESCO. (2014). EFA global monitoring report - teaching and learning: Achieving quality for
all. Paris: UNESCO.
UNESCO International Institute for Educational Planning (IIEP). (2006). Guide for planning
education in emergencies and reconstruction: Non-formal education. Paris: UNESCO-
IIEP.

171
UNICEF. (1993). Reaching the Unreached: Non-Formal Approaches and Universal Primary
Education. New York: UNICEF.
Urueta, S., & Ogi, T. (2020). School media education during the covid–19 pandemic:
Limitations and new opportunities. Media Education, 60(3), 367-374.
Vietnamese Government. (2011). Strategies for developing the human resource of Vietnam from
2011 to 2020. Hanoi: Vietnamese Government. Retrieved from
https://www.vietnamlawmagazine.vn/decision-no-579-qd-ttg-of-april-19-2011-
approving-the-strategy-on-development-of-vietnamese-human-resources-
during-2011-2020-4810.html
Welsh, J., & Dey, S. (2002). Quality measurement and quality assurance in higher
education. Quality Assurance in Education, 10(1), 17-25.
Wickline, H. (2013). Open educational resources: Breaking the lockbox on education. California,
USA: William and Flora Hewlett Foundation.
Yasunaga, M. (2014). Non-formal education as a means to meet learning needs of out-of-school
children and adolescents. Paris: UNICEF and UNESCO for Statistics.

172

Vol. 19, No. 10, pp. 172-193, October 2020
Sociocultural Adaptation and Program

Management Strategies for International
Doctoral Students of the “Confucius China
Studies Program”
Yang Fan
Beijing Language and Culture University, Beijing, China
https://orcid.org/0000-0002-2856-5957
Abstract. The purpose of this study was to understand the sociocultural

adaptation status for international doctoral students of the “Confucius
China Studies Program” (CCSP), as well as obtain suggestions from those
students to the management of the program. This qualitative study
applied an instrumental case study at a research site university in Beijing.
Through distributing questionnaires to international students who
participated in the program, as well as conducting one-to-one interviews
with seven international students and two program management
teachers, this study found that the sociocultural adaptation status of the
international students of CCSP was generally high. The sociocultural
adaptation status for doctoral students of the CCSP was analyzed from
five aspects including interpersonal communication, academic
performance, personal interest and social participation, environmental
adaptation, and language proficiency. The interviewed international
students proposed three suggestions to the management of the CCSP,
including holding exchange and sharing activities for international
doctoral students of the CCSP, compiling CCSP manual for international
students of the program, and the establishment of a student writing
center. This study also provides implications to administrators of the
CCSP, international doctoral students, and policy-makers of this issue.
Keywords: “Confucius China Studies Program”; sociocultural

adaptation; international doctoral students; program management
1. Introduction
Since China's acceded to the World Trade Organization in 2001, the number of
international students admitted to China has been steadily on the increase. China
has become the third largest study abroad destination country in the world and
the largest in Asia (Ministry of Education of the People’s Republic of China,
2018a).

173
In recent years, the Chinese government has been paying close attention to the
development of international student education in China. It has vigorously
implemented various policies to recruit international students from other
countries to come to study in China, such as the Chinese Government Scholarship,
“One Belt and One Road” Scholarship, and the Confucius Institute Scholarship.
According to the statistics from the Ministry of Education (MOE) in China, a total
of 492,185 international students of various types from 196 countries and regions
studied at 1,004 higher educational institutions in 31 provinces (including
autonomous regions and municipalities) across China in 2018, with an increase of
3,013, or 0.62%, over 2017 (Ministry of Education of the People’s Republic of China,
2019).
The Ministry of Education's Study in China Plan, introduced in October 2010, had
proposed to build a system of international students studying in China services
that is commensurate with China's global status and education scale (Ministry of
Education of the People’s Republic of China, 2010). In September 2018, the
Ministry of Education in China issued a talent cultivation objective on the Quality
Standards for International Students in Higher Education (Trial), which states that
international students in China should have the awareness, knowledge, and skills
to adapt to cultural diversity (Ministry of Education of the People’s Republic of
China, 2018b). The purpose of this study was to understand the current
sociocultural adaptation status of international doctoral students from the
“Confucius China Studies Program” (CCSP), and obtain suggestions from
international students of CCSP at the program management level.
2.Literature Review
2.1 Sociocultural adaptation theory
Intercultural communication theorist Collen Ward (1996) classifies intercultural
adaptation competence into two dimensions: psychological adaptation and
sociocultural adaptation. Psychological adaptation is based on emotional
responses, pointing to psychological health and life satisfaction in a cross-cultural
environment, measured in terms of psychological health (Ward, 1996).
Sociocultural adaptation is the adjustment of the sojourner to the local society,
sufficient contact with local members, interaction, and negotiation with the local
environment through the acquisition of culturally appropriate skills, based on the
measurement of the difficulties experienced by the sojourner in the local society
(Ward, 1996). The current academic classification of intercultural adaptation
competence mainly follows Ward and Kennedy's view (Ward & Kennedy, 1992).
Research indicates that the factors that influence sojourners' adjustment can be
classified into four categories: 1) social elements of the native culture: including
social, economic, and cultural aspects; 2) local cultural factors: including cross-
cultural elements in social and economics; 3) individual characteristics: such as
personality, previous sojourn experience, cross-cultural training experience,
language ability, etc.; 4) individual and local cultural environment: includes
factors such as length of cross-cultural contact, amount of contact with local
groups, social support, and some other factors (Ward & Kennedy, 1992).

174
2.2 Sociocultural adaptation of international students in China

Research shows that the causes of a series of problems among international
students in colleges and universities are closely related to cross-cultural
adjustment (Chen, 2004). Lei and Gan (2004) from at Peking University
investigated the cross-cultural adjustment status of 96 international students in
Beijing and concluded that international students in China are a particular group
of people who have more psychological problems and need to be targeted for
primary prevention. Ding and Zhou’s (2019) research found that the main reasons
for the emergence of cross-cultural adjustment problems among international
students in China include language barriers, lifestyle differences, social needs,
and cultural differences.
The Chinese language level of international students is an essential factor that

affects their sociocultural adaptation. A study on 68 international students in
China found that interpersonal adaptation of international students with
intermediate Chinese ability was significantly better than that of international
students with primary Chinese ability (Li, 2009). There was no significant
difference in interpersonal adaptation between international students with
intermediate and advanced Chinese levels (Li, 2009). To a great extent, language
difficulties hinder international students' learning process as well as social and
cultural communication, make it difficult for them to participate in various
academic activities at the university, also significantly affect their interpersonal
adaptation situation.
The country of origin for international students is also an essential factor that
affects their cultural adaptation. Ward, Bochner, and Furnham (2001) analyzed
the relationship between cultural distance and cultural adaptation. Based on the
results of their study, the cultural distance between European and American
countries and China is considerable, so the level of cultural adaptation should be
relatively low; the cultural distance between Japan and South Korea and China is
small, so the level of cultural adaptation should be relatively high. However, a
study conducted in nearly ten universities with 704 international students of all
levels concluded that European and American students were more comfortable
with Chinese people’s attitudes toward them (Chen, Zhu & Che, 2006). In contrast,
students in Korea and Southeast Asian countries were sometimes uncomfortable
with Chinese people’s attitudes towards them (Chen, Zhu & Che, 2006). The study
shows that cultural distance is not the only factor that influences international
students' adaptability, but also many other variables come into play. For example,
because Chinese students are comparatively motivated to learn English, many
Chinese students want to make friends with international students from countries
where English is the native language. As a result, students from European and
American English-speaking countries received relatively more social support in
China compared to international students from other countries (Chen, Zhu & Che,
2006). In terms of language adaptation, the level of adaptation of students from
Korea and Japan is significantly higher than that of students from Europe,
America, and other Southeast Asian countries, which is closely related to the
social factors of international students’ home cultures (Chen, Zhu & Che, 2006).

175
Other factors that affect the cultural adaptation of international students to study
in China include the expectations of sojourners and the purpose of study in China.
Students who come to China for cultural analysis and research have a deep love
for China and a strong interest in Chinese culture. On the other hand, those who
come to China for study and employment in China are optimistic about China's
rapid development, economy, and numerous employment opportunities, and
wish to work and live in China in the future. Those different motivations may
have other effects on international students' adjustment during their time in China.
Research suggests that international students' excessive expectations of the host
country would increase their difficulties in intercultural adjustment. A better
understanding of the host country could help international students to prepare
for the problems they might face, thus forming the right expectations and
reducing their problems in intercultural adjustment (Li, 2009). Also, other factors
that influence international students' intercultural adaptability include subjective
factors such as their characters, knowledge of the culture of the destination
country, and their values (Zu, 2015).
Much research on intercultural adaptation has been done in the field, but much of
the existing research is based on international students study abroad in western
countries. Philip Altbach (1991), an internationally renowned expert in
comparative education states that the vast majority of the existing literature deals
with the problems of adaptation and return of third-world students in
industrialized countries, which are only a part of the total international student
research. Due to cultural and national conditions differences between China and
the west, western research results and literature cannot adequately explain the
problem of international students' adjustment in China.
3. Confucius China Studies Program (CCSP)

The Confucius Institute Headquarters established the CCSP to help young
scholars from all over the world understand China and Chinese culture, promote
the study of sinology, enhance the sustainable development of the Confucius
Institute, and contribute to the friendly relations between China and the peoples
of other countries. CCSP is a high-end international exchange program launched
by the Confucius Institute Headquarters in 2013, which aims to provide high-
quality training resources in China for overseas students, scholars, and elites from
all walks of life who are studying China. The CCSP specializes in the humanities
and social sciences which consists of six programs: the Joint Research PhD
Fellowship, the PhD in China Fellowship, Understanding China Fellowship,
Young Leaders Fellowship, Publication Grant, and the International Conference
Grant (Confucius Institute Headquarters, 2020).
There are two types of doctoral student cultivation fellowships under the
“Confucius China Studies Program”: the Joint Research PhD Fellowship and the
PhD in China Fellowship. The target population of this study is international
students currently enrolled in the Joint Research PhD Fellowship and the PhD in
China Fellowship. CCSP provides financial support for non-Chinese master’s
degree holders to study their doctoral degrees in China for three to four years.
Applicants are required to reach at least level 5 in the Chinese Proficiency Test

176
(HSK). Still, in the actual admission process, those with excellent Chinese
proficiency are given priority under the same conditions. The significant subsidies
of the program include living allowance and accommodation, research expenses,
round-trip international travel expenses, tuition fees, training fees for Chinese
universities, medical and accident insurance in China, and expenses for group
activities and cultural experiences. The CCSP Joint Research PhD Fellowship for
PhD students registered at foreign universities to study, conduct research, and
write doctoral dissertations in China. According to the official website of the
Confucius Institute Headquarters, applicants for the Joint Research PhD
Fellowship is required to have a Chinese Proficiency Test (HSK) score of Level 3,
but applicants with excellent Chinese proficiency are given priority. For the Joint
Research PhD Fellowship, the main subsidized expenses are the same as students
who participate in the PhD in China Fellowship for the period of studying in
China for the doctoral degree (Confucius Institute Headquarters, 2020).
According to the Confucius Institute Headquarters’ official website, there are 16

high-level universities in China participating in the PhD in China Fellowship by
2020, of which five universities are in Beijing, including Peking University,
Renmin University of China, Beijing Normal University, Beijing Foreign Studies
University, and Beijing Language and Culture University (Confucius Institute
Headquarters, 2020).
The Ministry of Education (MOE) in China stressed the need to further implement
the CCSP and expand the enrollment of outstanding young people from various
countries to study doctoral degrees in China and to jointly cultivate doctoral
students (Ministry of Education of the People’s Republic of China, 2017). Research
on international students studying for postgraduate degrees in Beijing under the
CCSP would benefit their sociocultural adaptation among international students
in China, as well as to cultivate high-level overseas talents for international
Chinese language education. This study hopes to improve the management and
serviceability of colleges and universities for international doctoral students in
China. This study might also provide implications to policy-makers of the CCSP,
thus promoting this international student community to play a more significant
role in the international arena in the future.
4. Research Methods
4.1 Research participants
The research site was a university in Beijing, research participants were seven
current doctoral international students who participated in the CCSP and two
international student management teachers for this program. The seven
participant students include five students who got the Joint Research PhD
Fellowship and two students who received the PhD in China Fellowship. Among
the seven students, there were four males and three females. The seven students
came from seven different countries in Europe, Asia, and Africa. The majors they
studied include International Chinese Language Education, Linguistics and
Applied Linguistics, Foreign Linguistics and Applied Linguistics, and Literary
Studies. Among the seven students, three students are between the ages of 26 to
29 years old, two students are between the ages of 36 to 39 years old, and two

177
students are between the ages of 40-45 years old.
4.2 Research questions

Through conducting questionnaires and individual interviews with the nine
research participants, this study focused on the following two research questions:
1. What are the sociocultural adaptation status for the currently enrolled CCSP
international doctoral students during their study in China?
2. What are some suggestions from international doctoral students to the CCSP at
the program management level?
4.3 Research methods and procedures

A qualitative study was used since qualitative methods typically produce a
wealth of detailed information about a smaller number of people and cases to
increase the depth of understanding of the cases and situations studied (Patton,
2002). This research applied an instrumental case study research design to
understand the sociocultural adaptation status for international doctoral students
of the CCSP (Merriam, 1990; Yin, 2003). An instrumental case study is applied
when the intent of the case study is to understand a specific issue, problem, or
concern and a case selected to best understand the problem (Stake, 1995). Through
organizing and analyzing the information in the collected questionnaire, the
researcher plans to understand the personal information, social and cultural
adaptation status, and needs of international students at the doctoral level.
Through semi-structured individual interviews, the researcher plans to gain
further insight into the current situation and needs of international doctoral
students’ sociocultural adaptation status, as well as to understand the suggestions
of international students on the program management level of the CCSP (Creswell,
2013).
Students enrolled in the CCSP is a relatively small population at the research site
university, with a total of around 18 students enrolled in the program in the 2020
spring semester (including graduates in the class of 2020). Therefore, this study
used a purposeful sampling strategy to select the intended participants of the
study. The participants must be students who currently enrolled in the CCSP
during the time of research. They must have been spent at least one semester at
the research site university. The participants were recruited by contacting the
program leader at the research site. After getting permission from the program
leader, the researcher contacted ten international doctoral students of the CCSP
through WeChat (a popular social networking software), and introduced the
purpose of the study, research methods, confidentiality, and anonymity. Among
the ten students, a total of seven international doctoral students agreed to
participate in this study. The researcher sent the informed consent letter to all
participants through WeChat and finalized nine research participants.
The researcher distributed an electronic version of the “Sociocultural Adaptation

Questionnaire for CCSP International Students” to the international doctoral
students. The questionnaire consists of two sections: the first section is the basic
personal information of the participants such as their nationality, years of stay in
China, and major; the second section of the questionnaire was adapted from Ward
and Kennedy's (1999) version of the Sociocultural Adaptation Scale by Wilson

178
(2013) to assess students’ sociocultural adaptation status. The modified version

highlights the academic performance dimension. It assesses the sociocultural
adaptation status from five dimensions: interpersonal communication, academic
performance, personal interest and social participation, environmental adaptation,
and language proficiency (Wilson, 2013). To better understand the environmental
adaptation dimension, the question “accommodate to food in the host country”
was added to a total of 22 items (see Appendix 1). Scores are calculated by
averaging the individual item scores, where higher scores represent greater
competency in a new cultural environment (see Table 1). The internal consistency
reliability of the adapted scale was 0.92, and the test result was greater than 0.7,
indicating that the reliability of the scale was high. All question options were in
Likert-scale format, with five levels ranging from “completely inadaptable” to
“very adaptable”. After collecting the information of the participants, the
researcher applied descriptive analysis to analyze the sociocultural adaptation of
the participants to measure their sociocultural adaptation status.
After collecting the “Sociocultural Adaptation Questionnaire for CCSP

International Students” from seven currently enrolled CCSP doctoral students,
the researcher conducted one-to-one semi-structured interviews with the seven
students and two CCSP student management teachers. Each interview was
scheduled for 45 minutes, while the actual interview length ranged from
approximately 35 minutes to 1 hour and 23 minutes under the interviewees’
permission. All interviews were conducted in Chinese and were audio-recorded
throughout. Since this study was conducted from June 2020 to August 2020 during
the COVID-19 pandemic, all student interviews were conducted online via voice
communication. The semi-structured interview protocols included open-ended
questions for the "Confucius China Studies Program" International Doctoral
Students and International Student Management Teachers. The sample interview
question was, “please describe your adaptation in the aspect of interpersonal
communication during your study of the program in China." The interview
protocol designed for international doctoral students of the CCSP focused on five
dimensions of the sociocultural adaptation of CCSP international doctoral
students (see Appendix 2).
The interviews for program administrators were conducted through face-to-face

communication. The interview protocol for administrators of the CCSP program
was divided into four sections: personal information, professional quality,
sociocultural adaptation status of students in the CCSP, as well as questions
related to the CCSP (see Appendix 2). The researcher transcribed all interviews
verbatim in Chinese. The nine interviews lasted about nine hours in total, and the
length of the interview transcripts is about 94,000 words.
The researcher applied content analysis and thematic analysis to draw

preliminary themes and conclusions of the study (Maxwell, 2013). After
transcribing the interviews, the researcher repeatedly read through the transcripts
several times and used NVivo 12.0 software for categorical aggregation and
hierarchical coding (Chen, 2000). The questionnaires and transcripts were
analyzed in Chinese and a large number of quotes were translated into English

179
and included in the article. Combining information from the official websites of
the research site and the introduction of the CCSP, the research questions were
answered through questionnaires, individual interviews, and text analysis to
generate themes. The researcher intends to ensure the trustworthiness of the study
through rich and thick descriptions of the themes.
5. Findings
5.1 The sociocultural adaptation status for CCSP international doctoral students
Through analyzing the collected data from the Revised Sociocultural Adaptation
Scale (Table 1), the results of the seven participants’ sociocultural adaptation
ability were relatively competent. Among the seven participants, one student’s
sociocultural adaptation status is in the scale of “extremely competent” (5) during
his study and life in the CCSP in China, the sociocultural adaptation status for
four students are between the scales of “competent” and “extremely competent”
(4-5), one student is “competent” (4) of sociocultural adaptation while study and
live in China, and one student perceives his sociocultural adaptation status as
“moderate” to “competent” (3-4).
Table 1: Results of the sociocultural adaptation scale

Communication
Years of Living
Environmental
Interpersonal
Sociocultural
Participation
Performance
Interest and
Adaptation
Adaptation
Proficiency
Academic
Language
Personal
in China
Number
Gender
Status
Major
Social
1 PCF Female Literary 4.14 4.75 4.25 5 5 4.55 12 years

Studies
2 PCF Female International 5 5 4.75 4.6 5 4.77 7 years
Chinese
Language
Education
3 PCF Male International 5 5 5 5 5 5 6 years
Chinese
Language
Education
4 PCF Female Linguistics 4.14 4.75 4.5 4.4 4 4.36 4 months
and Applied
Linguistics
5 JRPF Male International 3.71 4.25 2.75 4.8 5 4 6 months
Chinese
Language
Education
6 JRPF Male Foreign 3.86 4 4.25 4.8 3.5 4.14 2 years
Linguistics
and Applied
Linguistics
7 PCF Male Linguistics 3.57 3.75 3.75 3.8 4.5 3.77 8.5 years
and Applied
Linguistics
Joint Research PhD Fellowship = JRPF
PhD in China Fellowship = PCF
Combining interviews with international student administrators of the CCSP, the

180
study found that the sociocultural adaptation level of international students in the
CCSP is generally high. The first reason is that many international students in the
program have completed their master’s and even bachelor’s degrees in China.
Therefore, they already possess some sociocultural adaptation competence during
their doctoral level of study. Among the seven international students who
participated in the study, four students have lived in China for six years or more,
accounting for 57.1% of the total number of students surveyed. Secondly, as a
high-quality study abroad program from the Confucius Institute Headquarters,
the international students admitted to the CCSP are elder than undergraduate
international students, five of the seven students interviewed have about one to
eight years of work experience, and five of them are married and have children.
Those students have a mature and experienced way of coping with sociocultural
adaptation. They can transfer the social interactions they have experienced in their
home country to the country of study. During the interviews, two international
student management teachers commented on the characteristics of the program
students as having “high language and research proficiency, professional
competence, all-around development, and leadership skills.” In terms of
sociocultural adaptability, they are able to use the perspective of “cultural
relativism” to be aware of different standards of judgment and perceptions, and
consciously change their reference standards in dealing with culture (Bennett,
1993). For example, international student 1 mentioned in the interview:
“Many international students come to China with the concepts of ‘the self
and the other’, and measure a country and its culture from his or her
standard. I am not saying this is wrong, but this standard is not
conducive to living in a foreign country for a long time. I feel that since I
came to this country, I understand that this country has its own culture
and some of its characteristics, and I accept these characteristics. But
don't compare, ‘oh, how China is like this, in my country…’ You should
look at a country's culture with a tolerant attitude, just understand and
accept it.”
Interpersonal communication
In terms of interpersonal communication, two of the seven international students
evaluated their interpersonal communication adaptation in China as fully
adapted. Another two international students felt that their level of adaptation in
interpersonal communication was between adapted to very adapted, the rest of
the three students was between average to adapted. During the interview, one
student from Japan shared his challenges in adjusting to the mono-chronic culture
while studying in China. He mentioned a typical case of being invited the night
before a conference to give an oral presentation and another specific case of
completing a 10,000 words research article within three weeks. The cases
exemplify the discomfort in terms of time perspective and culture with the
frequent changes in plans and schedules that characterize the monochronic
culture in China (Hall & Hall, 1990). According to Hofstede's Cultural Dimensions
Theory, Japan as a distinctive culture with a high degree of uncertainty avoidance
is characterized by a low tolerance for uncertainty and an expectation of clear
regulations and planning systems (Hofstede, 2001).
In terms of interpersonal interactions, the international students in the program

181
were able to maturely cope with the interactions between classmates and teachers
in their daily lives. The international students in the CCSP doctoral program
spoke highly of the educational management teachers at their research sites,
which demonstrated the students’ ability of empathy, for example, international
student 2 stated:
“Me and the supermarket aunties, the security guard at our school, we're
all good friends. I respect them a lot, I respect their labor. I think what I
like most about Chinese people is those most ordinary people, it's their
labor that makes China so glorious.”
International student 5 noticed the lack of lunch breaks for staff members during
the busy time and hopes to reduce the burden on some of the staff. Several other
program students from different countries thanked the teachers work in all kinds
of university departments to manage international students, with international
student 1 stating:
“I would like to express my special thanks to the teachers in the Division
of Confucius Institute Development, they have been helping us with all
kinds of problems. Whatever big and small businesses we will go to them,
academically related and unrelated. The teachers are especially patient to
give us answers every time.”
This study found that individual international students have some difficulties in
adapting to the different customs, personal habits, and rest schedules patterns of
different countries when it comes to roommates for on-campus accommodation.
Indicating that while adjusting to Chinese society and culture, international
students also need to adjust to the social and cultural habits and customs of
different countries among the international student groups, especially when it
comes to personality characteristics and living habits.
Academic performance
The doctoral dissertation topics of the CCSP include eight aspects related to China:
1) research on the Confucius Institute; 2) research on language, literature, and art,
which involves sino-foreign studies or sino-foreign comparative studies; 3)
research on history and philosophy, which involves sino-foreign comparative
studies; 4) research on sociology, which involves Chinese minority studies,
women’s literature or women’s status, Chinese society, Chinese rural areas, etc.;
5) political science and law studies; 6) economics studies; 7) education studies, and
8) international relations and cross-cultural studies. Among the above eight areas,
international students often choose doctoral dissertation topics focusing on the
second type of language, literature and art studies, and type four education
studies, especially studies on language policy, overseas sinology, and localization
of Chinese language education.
In terms of learning motivation, the international students in the CCSP are mainly
intrinsically motivated, supplemented by extrinsic motivation, with a
combination of instrumental motivation. Among the research participants, four
students were intrinsically motivated. They were interested in their research
directions, wanted to better understand China through the doctoral program,
facilitate research data collection in China, and had the experience of studying

182
abroad in China. Two international students were extrinsically motivated. They

were attracted by the authority of the CCSP and the generous fellowship support
provided by the program. One student was inspired by a combination of intrinsic
and instrumental motivation, on the one hand, she would like to continually
challenge and improve herself; on the other hand, she wants to have a better job
position after finishing the program and back to her home country. The seven
interviewed students learned about the program in several different ways,
including recommendations from Chinese professors who came to study in
Beijing, suggestions from mentors at foreign institutions or local Confucius
Institute directors, and recommendations from their advisors while finishing their
master's degree. The strong intrinsic motivation of international students is a good
facilitator of their learning. For example, international student 2 is deeply
interested in Chinese culture, and mentioned in the interview that her favorite
course “Chinese Studies” is specifically designed for international students. The
course introduces Chinese massage, Chinese medicine, tea ceremony, ancient
poetry, and philosophy about Chinese culture, which satisfies the students’
understanding and interest in China.
Of all the interviewed students, international student 1 and international student

3 believed that their biggest academic problem was self-discipline, including strict
adherence to a prescribed schedule and time management. The student indicated
that the low credit requirements of the program also impacted students’ self-
discipline. International student 6 noted that participation in reading groups has
a positive effect on promoting peer interaction and academic achievement.
Studies have shown that field-independent learners are mostly analytical, prefer
independent learning, and focus on abstract and objective factors. In contrast,
field-dependent learners prefer to learn with others, focus on context and
relationships, and expect guidance from teachers and peers (Brown, 2006).
Brown’s (2006) study suggested that field-independent and field-dependent
learning styles were related to the sociocultural characteristics of international
students. International student 7’s elaboration of learning styles in the interviews
is in line with Brown's findings that students from low-context cultures, highly
industrialized countries, and individualistic cultures are inclined to have field-
independent learning styles (Brown, 2006). However, field-independent or field-
dependent learning styles are not absolute, and the following expressions reflect
the shift from field-independent to the co-existing of field-independent and field-
dependent learning styles of international student 7:
“One of the big differences between undergraduate and master's study
with a doctoral is that doctoral study centered on an individual... I
sometimes write my dissertation behind closed doors, probably because I
want to avoid socializing to make my dissertation better or to supervise
my writing process. However, social interaction can sometimes be
beneficial for my dissertation and research. So how to strike that balance,
I think, is also a challenge during the doctoral program.”
Personal interest and social participation

Combining the results of the interviews and the survey, five of the seven
interviewed international students perceive that their personal interest and social

183
participation had been accommodated and satisfied during their study in China.
In terms of personal interest, thanks to the program’s fellowship in the forms of
living and research stipends and the convenient campus facilities at the research
site, all international students generally felt that their personal interests in travel,
sports, reading, calligraphy, music, and learning a new language other than
Chinese were satisfied. The social involvement of international students showed
a rich and varied character.
In terms of student activities on campus, international students in the CCSP have

participated in the annual school gala, spoken at academic conferences as
representatives of international students, hosted school events, and participated
in the Association of International Student Union (AIU). In terms of social
participation in off-campus activities, the Confucius Institute headquarters
(current name Center for Language Education and Cooperation) has created lots
of opportunities for students to participate in a variety of social activities and
symposiums. During the interview, the international students mentioned that
they have been invited by events at Chinese embassies, interviewed by China
Central Television and People’s Daily newspaper, selected as delegates in the
International Chinese Language Education Conference, joined the open day ceremony
from Confucius Institute Headquarters, enjoyed the cultural experiences (Chinese
Studies Tours), participated in student exchanges between China and other
countries, traveled throughout China, honored as international student
representatives in the 70th National Day parade, presented in the Doctoral Student
Forum of the CCSP, and so on. Living in China also allowed the international
students in the program to gain a better understanding of the Chinese language,
culture, customs, and experiences of Chinese people’s lives, such as finding an
apartment, applying to the Internet, exercising at the gym, and shopping online.
Of the two international students who self-rated their involvement in social

activities as low, one student chose to focus more on her doctoral dissertation and
thus deliberately avoided or reduced socializing. The other student was less
involved in on-campus and social activities due to his living off-campus and
brought his children while studying in China, combined with the influence of a
comparatively elder age. This indicates that age, family situation, length of study
in China, and academic pressures of international students in the program had
some degree of influence on their personal interests and social involvement.
Environmental adaptation
In terms of environmental adaptation, whether students live on or off-campus is
an essential factor in influencing their sociocultural adaptation. As a result of the
COVID-19 pandemic in the spring semester of 2020, the research site university
waived the accommodation fees for international students who did not live on
campus during that semester, which was practical assistance for international
students. In terms of accommodation, six students chose to live on-campus during
their stay in China because of the convenience of campus facilities, and one
student decided to live off-campus for family reasons. One student thought that
giving stipends as a package and self-pay accommodation fees is not as
convenient as the Chinese Government Scholarship. The Chinese Government

184
Scholarship, which is another competitive scholarship compared with the CCSP,

directly waives students’ on-campus accommodation fees. CCSP, on the other
hand, provides students with a certain amount of stipend and students still need
to pay for accommodations themselves, which adds a certain financial burden to
the students’ lives. However, two students in favor of this payment method since
they were not required to live on campus and that they could bring their children
and families to live in China with them, so they prefer this flexible way of
accommodation. In terms of adapting to living in the city, one student suggested
that the amount of stipend should take into account the price level of the city. The
compensation provided by the CCSP would be sufficient for living in a small city
in China, whereas in a large city like Beijing the compensation would be tight.
Five students, on the other hand, felt that the living and academic stipends
sponsored by the CCSP were generally sufficient to cover their life in Beijing.
Language proficiency
In the interviews, all interviewed students indicated that there were no significant
obstacles in communicating with their Chinese teachers and classmates in Chinese,
understanding the course content, and meeting the course credit requirements. In
terms of Chinese listening, speaking, reading, and writing skills, one student said
that it was difficult to fully understand the content when someone speaks fast in
Chinese. In addition, three students indicated that the biggest obstacle in language
proficiency was Chinese writing, especially dissertation writing. One advisor also
required the student in the CCSP to publish one or two articles in Chinese. Some
students said that their professors had commented that their writing was simple
and lack of logic. They regarded the primary cause of the situation as a lack of
specific training in Chinese writing. This was also agreed by the international
student administration teachers, who were interviewed and believed that the
obstacles in Chinese writing were influenced by the large difference between oral
and written Chinese. It was more difficult for international students to write in
Chinese. Besides, two students also believed that their academic research ability
needs to be improved.
5.2 Suggestions for management of CCSP

CCSP experience sharing events for international doctoral students
During the interviews, three of the international students suggested that the
research site should provide opportunities for doctoral students to meet and
know each other by organizing CCSP freshman orientations, CCSP experience
sharing sessions, and scheduled seminars. Those events could invite current or
graduated students of the CCSP to share their experiences in research methods,
dissertation writing, study methods, and so on. Through senior students sharing
experiences, freshman or sophomore doctoral students could gain learning
experiences and form a professional learning community. International Student 7
expressed:
“I think the CCSP doctoral program could have a certain kind of
experience sharing event. It could be in the form of a seminar or report
session. I think one of the things I'm more concerned about is at the
research level, including what problems my colleagues encountered, and
what kind of process they experienced while conducting the research. I am

185
not indicating the clean process written on the final dissertation, but the
one that is quite rugged, the real process.”
One student suggested that the experience sharing session could be conducted in
small groups to provide opportunities for interactions among the presenter and
the students for the best effectiveness.
International student 6 believes that dinner or orientation could be held when the
CCSP students entered the program. Therefore, new students could get to know
the program management teachers as soon as possible and close the distances
among each other. This could also benefit the new students to adapt to their study
and life in China. Several interviewed students also suggested that after the fifth
or tenth anniversary of the international students’ graduation, the research site
could invite the international students to share their experiences, introduce their
latest research, maintain contact with the university, and at the same time better
publicize the CCSP. After the international students return to their home countries,
it is important to strengthen the ties with the alumni of the program, promote the
development and the output of the CCSP through various means.
Compiling a manual for international students of the CCSP

Many of the interviewed students said that they most often contacted the two
teachers at the Division of Confucius Institute Development at the research
institution. Teachers of the Division of Confucius Institute Development handled
a series of matters including new student registration, program extensions,
graduation, suspensions, end-of-term supervisors’ evaluations, annual
summaries, training plans, and a series of documents required by the program.
Those teachers are also responsible for distributing living and academic stipends,
airline ticket reimbursement, accommodation arrangements, and invitation letters
for students and their families to obtain visas. The Division of Confucius Institute
Development is also responsible for informing the students about specific
information from the Confucius Institute Headquarters through WeChat.
When teachers in the Division of Confucius Institute Development were unable

to answer the students’ questions, they would contact other departments.
Departments such as the International Student Administration Office, students’
graduate schools, and the Accommodation Center could help resolve problems
related to new student registration, visas, course selection, and so on. During the
interviews, two interviewees mentioned that they often encountered the problem
of not knowing which teacher in which department to contact for specific
questions and through what means. International student 4 and international
student 6 suggested making a detailed manual for international students of the
CCSP based on the existing manual for all international students from the
International Student Administration Office. The manual for CCSP could explain
in more detail the procedures involved in the program, such as orientation, class
selection, accommodation, etc., so that students would know which specific
department to contact, whom to contact, and by what means. On the other hand,
the manual could reduce the workload and pressure of international student staff,
avoid repeated inquiries from different students on the same issue, and improve
working efficiency.

186
Establishing a student writing center

In response to the academic writing difficulties raised by a number of
international students, international student 7 noted that the current methods of
learning academic writing include self-study, advisor’s guidance, and help from
colleagues. However, he mentioned that seeking support from Chinese colleagues
caused some trouble and added workload for them. He wished to get more
professional and efficient help with writing. International student 7 suggested
that the university could create a student writing center at the research site
institution. The writing center would be staffed with professional teachers or
graduate students to tutor international students in Chinese academic writing. In
addition, currently enrolled Chinese students and university faculty members
could also get assistance from the Writing Center. Possible services the center
could provide include text proofreading, editing, essay formatting, workshops
about writing skills, as well as bilingual writing tutoring in both Chinese and
English. Regular lectures, seminars, salons on academic writing, and books on
writing skills could also facilitate students’ learning. The Writing Center could set
up a support group for essay writing, where monthly meetings can be held to
discuss the progress of their reading and dissertation writing, as well as the
problems they may face during their writing process. Through the creation of a
student writing center, the Chinese writing skills of international students, not
only for students in the CCSP but also the international student group in the
research site in general, could be effectively improved.
6. Conclusions
This study focuses on understanding the sociocultural adaptation of international
students in the CCSP through questionnaires and one-on-one interviews, as well
as their suggestions on program management. All research aims have been
accomplished through this study. It is found that the social and cultural
adaptability of international students in the CCSP is generally high. This study
provides implications to policy-makers and the research site university to hold
exchange and sharing activities for international doctoral students of the CCSP,
compile CCSP manual for international students of the program, and establish a
student writing center at the research site university. This study also has several
limitations, the first limitation is the small sample size of the participants. Because
of the relatively small population of students enrolled in the CCSP in the research
site university, it was difficult to recruit a large number of students to participate
in this study. The second limitation is that because of the COVID-19 pandemic, all
student interviews were conducted online via WeChat voice communication. The
third limitation of the study is that if possible, a triangulation of the research
methods such as observation and document analysis could be used to enhance the
trustworthiness of the study. As a high-level international student program in
China, understanding the social and cultural adaptability of international doctoral
students of the CCSP during their study in China, obtaining suggestions and
demands of the international students in program management could help
promote the standardization and long-term effectiveness of international student
management and contribute to the development of international student affairs in
Chinese universities.

187
Funding: This research program is supported by the Science Foundation of

Beijing Language and Culture University (supported by “the Fundamental
Research Funds for the Central Universities”) (Approval number: 19YBB21).
7. References
Altbach, P. G. (1991). Impact and adjustment: Foreign students in comparative
perspective. Higher Education, 4, 305-323. Retrieved from
https://link.springer.com/article/10.1007/BF00132723
Bennett, M. J. (1993). Towards a developmental model of intercultural sensitivity. In R. M.
Raige (Ed.), Education for the intercultural experience. Yarmouth, ME: Intercultural
Press.
Brown, H. D. (2006). Principles of language learning and teaching (5th ed.). White Plains, New
York: Pearson Longman.
Chen, H., Zhu, M., & Che, H. S. (2006). Zai Beijing gaoxiao de waiguo liuxuesheng shiying
yinsu yanjiu [A study of adaptation factors for foreign students in Beijing
universities]. Qingnian yanjiu [Journal of Youth Studies], 27-36.
Chen, X. M. (2000). Zhi de yanjiu fangfa yu shehui kexue yanjiu [Qualitative research methods
and social science research]. Beijing: Educational Science Press.
Chen, X. M. (2004). Sojourners and “Foreigners”: A study on Chinese students’ intercultural
interpersonal relationships in the United States. Educational Science Press.
Confucius Institute Headquarters. (2020). Confucius China Studies Program. Retrieved from
http://ccsp.chinese.cn/index.html
Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches
(3rd ed.). Thousand Oaks, CA: Sage.
Ding, S. Y., & Zhou, Z. C. (2019). Kuawenhua shijiaoxia yidaiyilu yanxian guojia
liuxuesheng guanli tanxi [An analysis of the management of international
students study abroad in China from countries along the "Belt and Road" from a
cross-cultural perspective]. Nanyang ligong xueyuan xuebao [Journal of Nanyang
Institute of Technology], 3, 80-84.
Hall, E. T., & Hall, M. R. (1990). Understanding cultural differences: Germans, French and
Americans. Yarmouth, ME: Intercultural Press.
Hofstede, G. (2001). Culture’s consequences (2nd ed.). Beverly Hills, CA: Sage.
Lei, L. Y., & Gan, Y. Q. (2004). Laihua liuxuesheng de kuawenhua shiying qingkuang
diaocha [Survey on the sociocultural adaptation status for international students
study abroad in China]. Zhongguo xinli weisheng zazhi [Chinese Mental Health
Journal], 10, 729.
Li, P. (2009). Liuxuesheng kuawenhua shiying xianzhuang yu guanli duice yanjiu
[Research on the current situation of cross-cultural adaptation and management
strategies of international students in China]. Zhejiang shehui kexue [Zhejiang Social
Science], 5, 114-119.
Merriam, S. B. (1990). Case study research in education: A qualitative approach. San Fransico,
CA: Jossey-Bass Publishers.
Maxwell, J. A. (2013). Qualitative research design: An interactive approach (3rd ed.). Thousand
Oaks, CA: Sage.
Ministry of Education of the People’s Republic of China. (2010). Study abroad in China
program. Retrieved from
http://www.moe.gov.cn/srcsite/A20/moe_850/201009/t20100921_108815.htm
l
Ministry of Education of the People’s Republic of China. (2017). Letter of response to the
proposal No. 0580 (Education No. 059) of the fifth session of the twelfth National

188
Committee of the CPPCC. Retrieved from

http://www.moe.gov.cn/jyb_xxgk/xxgk_jyta/jyta_gjhb/201803/t20180302_32
8506.html
Ministry of Education of the People’s Republic of China. (2018a). National statistical bulletin
on the development of education in 2017. Retrieved from
http://www.moe.gov.cn/jyb_sjzl/sjzl_fztjgb/201807/t20180719_343508.html
Ministry of Education of the People’s Republic of China. (2018b). The Ministry of
Education's circular on the issuance of “Higher education for international students in
China quality specification (trial)” notice. Retrieved from
http://www.moe.gov.cn/srcsite/A20/moe_850/201810/t20181012_351302.htm
l
Ministry of Education of the People’s Republic of China. (2019). Statistics on international
students studying in China in 2018. Retrieved from
http://www.moe.gov.cn/jyb_xwfb/gzdt_gzdt/s5987/201904/t20190412_37769
2.html
Oxford, R. L. (1996). Language learning strategies around the world: Cross-cultural perspectives.
Manoa, HI: University of Hawaii Press.
Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks,
CA: Sage.
Stake, R. (1995). The art of case study research. Thousand Oaks, CA: Sage.
Ward, C. (1996). Acculturation. In D. Landis & R. Bhagat (Eds.), Manual of intercultural
training (pp. 124-147). Thousand Oaks, CA: Sage.
Ward, C., & Kennedy, A. (1992). Locus of control, mood disturbance, and social difficulty
during cross-cultural transitions. International Journal of Intercultural Relations,
16(2), 175-194. doi:10.1016/0147-1767(92)90017-0
Ward, C., & Kennedy, A. (1999). The measurement of sociocultural adaptation.
International Journal of Intercultural Relations, 4, 659-677. doi:10.1016/S0147-
1767(99)00014-0
Ward, C., Bochner, S., & Furnham, A. (2001). The psychology of culture shock. New York:
Routledge.
Wilson, J. (2013). Exploring the past, present and future of cultural competency research: The
revision and expansion of the sociocultural adaptation construct [Unpublished Doctoral
dissertation]. Victoria University of Wellington.
Yin, R. K. (2003). Case study research: Design and method (3rd ed.). Thousand Oaks, CA: Sage.
Zu, X. M. (2015). Kuawenhua jiaoji [Intercultural communication]. Beijing: Foreign
Language Teaching and Research Press.

189
Appendix
Questionnaire on the Sociocultural Adaptation Status of International Doctoral

Students of the “Confucius China Studies Program”
Dear student,
Thank you so much for agreeing to participate in this study. I am conducting a

survey to understand the sociocultural adaptation of international doctoral
students in China under the Confucius China Studies Program. The
questionnaire will be anonymous and all information will be kept confidential.
The obtained data will only be used for this study. Please read each question
carefully and select the option that best fits your situation.
Your completion of the questionnaire will be very helpful to my research, thank

you for your participation and support!
Section 1: Basic Personal Information
1. First three letters of your name: _________

2. Gender: Male Female
3. Nationality: ________
4. Age: _________
5. Major of study in the CCSP: _____________
6. How long have you been living in China so far: _________
7. Which type of CCSP are you in:
Joint Research PhD Fellowship PhD in China Fellowship
8. HSK level: ______________
9. Years of study in the doctoral program: ____________
10. Native language: ________
11. Besides your native language, how many languages can you speak and what
are the languages: _______________________
12. Before joining the CCSP, do you have any work experience? If so, what was
your job and how many years have you been working?
________________________________________________
13. Before joining the CCSP, do you have any experience in studying and
working in overseas countries? If so, which country have you been living, and
for how many years?
_________________________________________________________
14. Before joining the CCSP, what is your major(s) during your undergraduate
and master’s degree of study: __________________________
15. Have you ever suffered from economic difficulties during your study of the
CCSP? What is your primary income during your study at the CCSP?
_________________________________________________
16. Have you got married during your study at the CCSP? Do you have any
children?
_________________________________________________

190
Section 2: Sociocultural Adaptation Scale
Thank you for answering the basic information questions in Part I. In Part II, the
questions will focus on understanding your intercultural adaptation while
studying in China. There are five level options for each question and you will be
asked to select the one that best fits your situation. The options on the left are not
at all appropriate for your situation and the options on the right best fits your
situation. To ensure the accuracy and validity of the data, please answer the
questions based on your real situation.
Thank you for your cooperation and understanding!
SCAS-R
Revised Sociocultural Adaptation Scale
Living in a different culture often involves learning new skills and behaviors.
Thinking about life in [country], please rate your competence at each of the
following behaviors (1 = Not at all competent; 5 = Extremely competent).
1. Building and maintaining relationships. 1 2 3 4 5
2. Managing my academic/work responsibilities. 1 2 3 4 5
3. Interacting at social events. 1 2 3 4 5
4. Maintaining my hobbies and interests. 1 2 3 4 5

5. Adapting to the noise level in my neighborhood. 1 2 3 4 5
6. Accurately interpreting and responding to other
1 2 3 4 5
people’s gestures and facial expressions.
7. Working effectively with other students/work
1 2 3 4 5
colleagues.
8. Obtaining community services I require. 1 2 3 4 5
9. Adapting to the population density. 1 2 3 4 5

10. Understanding and speaking [host language]. 1 2 3 4 5
11. Varying the rate of my speaking in a culturally

1 2 3 4 5
appropriate manner.
12. Gaining feedback from other students/work
1 2 3 4 5
colleagues to help improve my performance.
13. Accurately interpreting and responding to other
1 2 3 4 5
people's emotions.
14. Attending or participating in community
1 2 3 4 5
activities.
15. Finding my way around. 1 2 3 4 5
16. Interacting with members of the opposite sex. 1 2 3 4 5

191
17. Expressing my ideas to other students/work

1 2 3 4 5
colleagues in a culturally appropriate manner.
18. Dealing with the bureaucracy. 1 2 3 4 5
19. Adapting to the pace of life. 1 2 3 4 5
20. Reading and writing [host language]. 1 2 3 4 5

21. Changing my behavior to suit social norms,
rules, attitudes, beliefs, and customs. 1 2 3 4 5
22. Accommodating to food in the host country.
1 2 3 4 5
SCAS-R Scoring
SCAS-R items are based on a Likert scale of 1 (= Not at all competent) to 5 (=
Extremely competent). Scores are calculated by averaging the individual item
scores, where higher scores represent greater competency (skills or behaviors) in
a new cultural environment.
Scores for each of the SCAS-R subscales may also be calculated:
Interpersonal Communication: Items 1, 3, 6, 11, 13, 16, 21

Academic Performance: Items 2, 7, 12, 17
Personal Interest and Social Participation: Items 4, 8, 14, 18
Environmental Adaptation: Items 5, 9, 15, 19, 22
Language Proficiency: Items 10 and 20
Appendix 2
Interview Protocol
("Confucius China Studies Program" International Doctoral Students)
Interview questions:
1. Please tell us why you joined the “Confucius China Studies Program”
(hereinafter referred to as “the program”).
2. Please describe the training model and graduation requirements (e.g. credits,
dissertation, other requirements, etc.) of the program.
Intercultural adaptation:
3.What do you think was the biggest challenge you encountered during the
program?
4.What do you think is the most important thing you have achieved during the
program?
5. Please describe your adaptation in the aspect of interpersonal communication
during your study of the program in China.
6. Please describe your adaptation of academic performance during your study of
the program in China.

192
7. Please describe in what ways the needs of your personal interest and social
participation have been accommodated and met during your participation in the
program in China.
8. How would you evaluate the level of your adaption to the local environment
during your study and life in the program?
9. How would you evaluate your Chinese language proficiency while
participating in the program?
Program Management strategies for the CCSP:

10. How do you think the Confucius Institute headquarters has helped you in
your study and life during your participation in the program?
11. How do you think the international student affairs administrators at your
university have helped you in your study and life during the program?
12. If possible, what advice do you have for the university's administration and
services to better serve future international students participating in the program?
13. If possible, what suggestions do you have for the policies of international
students coming to study in China for the betterment of the program in the future?
14. If possible, what advice would you give to future international students
participating in the program to help them better adapt to their study and life in
China?
15.What are some questions that I hadn't thought of, but that you think I should
know about or should be explored.
Thank you for your support of this interview and research!
Interview Protocol
(International Student Management Teachers)
I. Basic personal information

1. Your field of study: ___________
2. Number of years you have worked in international student management:
_________
3. Did you study or work overseas before working in the field of international
student management: ________
4. If you have overseas experience, the country in which you previously studied
or worked overseas is: ________ and the number of years you have worked or
studied overseas is: ________
II. Interview questions

1. Please tell a little bit about the main tasks of your work in the "Confucius China
Studies Program".
The professionalism of international student managers:

2. Did you have any professional training or work experience before engaging in
the management of international students, if so, can you tell us about it?
3. What are the main administrative challenges for international doctoral students
in the CCSP?
4. Have you ever felt overwhelmed in managing international students? Were
there times when you felt fulfilled with this work?

193
Intercultural adaptation of doctoral students:

5. The sociocultural adaptation dimension of intercultural adaptation consists of
five aspects: interpersonal communication, academic performance, personal
interest and social participation, environmental adaptation, and language
adaptation. In your work, have you encountered any problems with any of the
above dimensions of intercultural adjustment among the doctoral students of the
CCSP? If so, can you tell us about them?
6. Has your department taken any measures or organized any activities to
improve the intercultural adaptation of international doctoral students under the
CCSP during their study in China, and if so, what measures and activities have
been taken and organized?
7. What do you think are the differences and characteristics of the international
doctoral students in the CCSP compared to other international students?
8.What do you think are the differences between international students at the
doctoral level and other undergraduate or non-degree international students in
terms of learning and adjustment during their study in China?
9. In your opinion, what measures can universities take in the future to improve
the intercultural adaptability of international doctoral students in the CCSP?
The “Confucius China Studies Program”:

10. Do you have any suggestions for the development of the CCSP at the policy
and management level?
11. Do you have any advice for international students who are participating in the
CCSP to improve their adjustment to study and life in China?
12. Are there any issues that I haven't thought of, but you think I should know
about or discuss?
Thank you for your support of this interview and research!

194

Vol. 19, No. 10, pp. 194-213, October 2020
Autonomous English Language Learning

Beyond the Classroom: Indonesian Tertiary
Students’ Practices and Constraints
Daflizar
State Islamic Institute of Kerinci, Indonesia
https://orcid.org/0000-0003-3625-5315
Abstract. In response to the interest in learner autonomy in recent years,

educational research has been increasingly paying attention to students’
out-of-class autonomous learning activities. This study aims to (1)
describe the extent to which Indonesian tertiary students engaged in
autonomous English language learning outside the class, (2) explore
their perceived constraints in practicing autonomous learning, and (3)
examine whether there are any significant differences in the
autonomous learning activities between female and male students and
between the English major students and non-English major students.
Employing the explanatory mixed-method design, a total of 402 first-
year students completed a questionnaire, and 30 of whom were
interviewed. The questionnaire data were analyzed using descriptive
statistics and non-parametric tests, and the interview data were
analyzed using thematic analysis. The results showed that the students
did engage in several out-of-class English learning activities, however
many of the activities were more receptive than productive. The
interviews echoed the questionnaire results, and the students claimed
that they were not autonomous in their learning due to several
constraints. The results also revealed that there is no significant
difference in the level of practice of autonomous out-of-class activities
based on gender but a significant difference was found concerning
majors of study. Practical implications for the Indonesian context are put
forward.
Keywords: learner autonomy; out-of-class autonomous learning;

perceived constraints; gender; majors of study
1. Introduction
Learner autonomy, which is often defined as “the ability to take charge of one’s
own learning” (Holec, 1981, p. 3), is increasingly regarded as an imperative in
foreign language learning. An escalating interest in learner-centered approaches
to language teaching, coupled with recent advancements in technology-based
approaches, makes clear the point that learner autonomy is an essential element

195
in foreign language learning. In response to these trends, educational research is

increasingly paying attention to students’ out-of-class autonomous learning
activities (Benson 2011).
Out-of-class language learning is often used to describe “non-prescribed

activities that students carry out independently to broaden their knowledge of a
subject” (Benson, 2011, p. 9). Unlike classroom discourse that tends to be
structured and hierarchical involving simple rituals and routines as well as
display language, discourse beyond the classroom is somewhat unstructured
and its contexts of use generate multifaceted rituals and routines resulting in the
authentic language (Nunan, 2014). Previous studies in a variety of contexts
suggest that students’ active engagement in language learning beyond the
classroom leads to the improvement in learners’ proficiency, confidence,
motivation, along with intercultural awareness, and compensates limitations of
classroom-based learning (Benson 2011; Nunan, 2014; Ushioda 2001; Yorozu,
2001).
In any investigation into out-of-class English language learning, context is of

essential consideration (Hyland, 2004). This is because learners of English take
part in particular local contexts with particular practices that create English
learning opportunities (Norton & Toohey, 2001, p. 311). In other words, a
different learning context may provide learners with different learning
experiences, including the amount of exposure to the target language. In the
context of English as a foreign language (EFL) like Indonesia, classroom teaching
and learning may be the only venue where students have contact with English.
Once the students leave the classroom, they are immersed in their first language
environment, in which opportunities to use English in real settings are limited.
The challenges of English language teaching (ELT) in Indonesia have been well
documented. Although many efforts have been made to improve ELT quality,
including the introduction of different curricula, there still has been widespread
dissatisfaction with the English achievements of Indonesian students
(Dardjowidjojo, 2000; Madya, 2002; Marcellino, 2008; Nuh, 2013). Several studies
argued that the continuing problems do not only stem from the curriculum but
also from several other factors, such as limited time allocation for English
instruction, students’ lack of opportunity to use English out of the classroom,
and lack of teacher’s encouragement for students’ participation in the classroom
activities (Dardjowidjojo, 2000; Musthafa, 2001; Nur, 2004; Weda, 2018). It is
widely accepted that in the Indonesian context, the amount of exposure to the
target language, which is the most fundamental prerequisite of language
acquisition (Rowland, 2014), is very limited. The total number of hours allocated
for English instruction for the entire six-year period of secondary schools (junior
high school and senior high school) is 720 hours (Mustafa, 2018) with no more
than four contact hours per week (Yulia, 2014). To compensate for this limited
number of hours, therefore, students need to extend their engagement in English
language-related activities outside the classroom for more language exposure.

196
However, while learner autonomy requires students’ acceptance of

responsibility to take control of their learning (Benson, 2001), several studies on
learner autonomy in the Indonesian context showed that Indonesian students’
learning behavior is otherwise. In a study of secondary school students, Cirocki,
Anam and Retnaningdyah (2019) found that many students were not familiar
with the concept of learner autonomy. Based on their dependency on teachers,
only 1.4% of the students appeared to be autonomous learners. The results also
showed that the students had fairly low motivation to learn English and were
not ready to act as autonomous learners due to a lack of typical competencies. At
the university level, Hermagustiana and Anggriyani (2019) found that most
students perceived that their teachers had a dominant role in controlling
teaching and learning, which eventually led the students to become less
autonomous both inside and outside the classroom. In an investigation of
teachers’ beliefs, Saraswati (2019) found that more than 80% of the teachers
agreed that they were responsible for the teaching and learning process and less
than 60% thought that their students’ are autonomous learners. These results
suggest that learner autonomy is an essential measure to pursue in the
Indonesian context.
It is suggested in the literature, however, that the development of learner

autonomy is a gradual and intricate process (Benson, 2011; Blidi, 2017; Little,
2007). Learner autonomy is the product of an interactive process in which
teachers gradually expand the scope of their learners’ autonomy by gradually
allowing them to take more control over their learning (Little, 2007, p. 26). Its
intricacy derives from several factors, including culture, learner’s beliefs,
attitudes, motivation, and personality (Chen & Li, 2014). This suggests that the
development of learner autonomy will likely be different from culture to culture
and will depend on students’ readiness to exercise autonomous learning, which
may be reflected in their behaviors and beliefs. Thus, before any interventions
aiming to promote learner autonomy are implemented, exploring students’
practice of autonomous learning outside the class and perceived constraints they
have in performing their learning is an important step to take. A better
understanding of students’ learning beyond the classroom can assist in the
implementation of learner autonomy as a goal and make guidance given by
teachers to learners more effective (Pearson, 2004). Based on this framework, the
current research attempted to answer the following questions:
1. To what extent did Indonesian tertiary students engage in autonomous
English language learning activities outside the class?
2. What perceived constraints did the students have in their autonomous
learning outside the class?
3. Are there any statistically significant differences in the autonomous English
language learning activities outside the class between females and males and
between the students who are English majors and the students who are non-
English majors?

197
Learner Autonomy
Learner autonomy has been defined in many ways, indicating that it is a
multidimensional concept whose meaning can take many different forms and be
viewed from many different standpoints (Benson, 2001; Smith, 2008). The term
‘learner autonomy’ was first coined by Holec (1981), and he defined it as “the
ability to take charge of one’s own learning” (p. 3). Since then, many definitions
have arisen but researchers have not agreed on one straightforward definition
that sufficiently describes the concept. Interestingly, learner autonomy has also
been described in terms of levels and versions. Littlewood (1999), for example,
proposed a two-level category of autonomy, i.e. ‘proactive’ and ‘reactive’
autonomy. The former refers to circumstances where learners are able to take
charge – plan, monitor, and evaluate – of their own learning. The latter, the
second level of autonomy, is “the kind of autonomy which does not create its
own directions but, once a direction has been initiated, enables learners to
organize their resources autonomously in order to reach their goal” (p. 75).
Benson (1997) identified three versions of autonomy: technical, psychological,
and political. Technical autonomy is defined as the act of learning a language
outside the context of an educational institution and without the involvement of
a teacher. In the psychological version, autonomy refers to an ability that allows
learners to take more responsibility for their learning. The political version is
related to control over the process and content of learning. The main concern in
this version is “how to achieve the structural conditions that will allow learners
to control both their own individual learning and the institutional context within
which it takes place (Benson, 1997, p. 19).
Previous Studies on Out-of-class Learning

Over the last few decades, an increasing number of studies into students’ out-of-
class English language learning have been conducted. One earlier study was
conducted by Chan, Spratt, and Humphreys (2002) with a group of tertiary
students at the Hong Kong Polytechnic University. The results showed that the
major out-of-class activities the learners engaged in were related to
communication and entertainment, such as watching movies and television in
English and using the internet. In a similar context, Hyland (2004) found that
students spent considerable time on receptive activities such as listening and
reading, rather than speaking and writing. In a study of 324 students, Wu (2012)
found that the most frequently practiced activities are watching films and
television, reading, and listening to English songs, music, and radio channels.
In a study involving 121 students at an English Language Institute in Saudi

Arabia, Tamer (2013) found that watching English movies and listening to
English songs were the top most frequently practiced activities among the
students, followed by reading English signboards, watching TV in English, and
using the internet in English. In the Turkish context, Inozu, Sahinkarakas and
Yumru (2010) found that students most frequently practiced their English doing
internet activities, such as e-mailing or chatting, listening to music, watching TV
programs and movies, and reading books or magazines. In a similar context,

198
Orhon (2018) found that learners were mostly engaged in listening to songs in
English, followed by watching TV programs, videos, or movies in English.
Maros and Saad (2016) conducted a study in the Malaysian context. The results
showed that the participants preferred to learn English through watching
television programs or movies and make use of tools mainly technology-
affiliated in advancing themselves in the language. In a similar context, Hashim,
Yunus, and Hashim (2018) found that listening to talk shows on the radio,
watching TV shows, or seeing movies in the English language were the highest
rated activities the students engaged in for improving their listening and
speaking skills.
In the Indonesian context, very few studies on the issue of out-of-class English
language learning have been published (e.g. Ardi, 2013; Muthalib, Ys & Mustafa,
2019; Sutiono, Saukah, Suharmanto & Oka, 2017) and each had a different focus
from the present study in some ways. Ardi (2013) investigated the autonomous
behaviors and out-of-class English language learning activities of 192 first-year
university students. Muthalib et al. (2019) looked at the methods for language
exposure outside the language classroom context among 50 first-year university
students who were considered successful language learners as measured by
their TOEFL scores. Sutiono et al. (2017) explored the out-of-class activities
employed by six successful and three unsuccessful students of the English
major. The present study is of significance as it attempted to fill the gaps in the
projects mentioned above. Besides investigating students’ autonomous learning
beyond the classroom, it explored students’ perceived constraints in practicing
autonomous learning, the matters that have been little or not explored in
previous research. Also, this study examined whether there are any statistically
significant differences in the autonomous learning activities outside the class
regarding gender and major of study. In terms of methodology, this study
employed a mixed-methods approach and involved a larger number of
participants. The participants were EFL students from four different institutions
of higher education spreading over 20 majors of study.
3. Research Methodology
Design
To collect the data of this research, the explanatory mixed-methods design,
which comprises a quantitative phase, followed by a qualitative phase, was
employed. The explanatory design requires two different reciprocal phases
beginning with the collection and analysis of quantitative data, followed by the
collection and analysis of qualitative (Creswell & Plano Clark, 2011). The overall
purpose of this design is to be able to use the qualitative data collected in the
second phase to help explain initial quantitative results.
Participants
The quantitative phase of the research involved 402 first-year students from four
higher educational institutions in Jambi province, Indonesia; two institutions of
Islamic studies, an institution of administrative studies, and an institution of
economics studies. The participating students consist of 192 males and 210

199
females that spread over 20 major fields of study, 52 of whom were doing
English major. The participants were about 18 to 20 years of age and were with
different English proficiency levels. In the qualitative phase, 30 of the students
were selected for interviews. To select the sample in the quantitative phase, both
stratified sampling and purposive sampling were used. To obtain a
representative sample of the whole population in terms of gender, the students
from each of the majors were divided into two groups: male and female. After
that, the males and females in each of the majors were randomly selected
according to their proportion. The purposive sampling was used to include all
the students of the English major in the sample to address one of the research
purposes i.e. to examine if there were differences between the students of the
English major and those of non-English majors in their autonomous English
language learning activities outside the class. In the second phase of the study,
30 participants were selected from those who indicated their availability to take
part in an interview in the consent forms. The selected participants had varied
autonomous English language learning practices outside the class.
Research Instruments
A questionnaire and interviews were used to collect the data. In the quantitative
phase, a questionnaire adapted from Chan et al. (2002) was used. The adapted
questionnaire consists of 22 items that explore students’ autonomous English
language learning activities outside the class. The participants rated their
answers on a four-point scale ranging from ‘never’ to ‘often’. In the qualitative
phase, semi-structured interviews were conducted with 30 purposefully selected
participants to explore the perceived constraints they had in performing
autonomous English learning outside the class.
Data Collection Procedure

Before deciding whether to participate in the study, all the participants were
provided with an information form, which explains the purposes and benefits of
the study. The students who agreed to participate in the study returned the
signed consent forms. Before the administration of the questionnaire, the
participants were requested to complete the questionnaire as honestly as they
could. The students who were chosen for the interviews were contacted to make
arrangements for the interviews. Before the interviews, the interviewees were
also requested to be sincere and honest in answering the interview questions. To
avoid miscommunication due to the low English proficiency level of the
participants, the interviews were conducted in Bahasa Indonesia. Lopez, Figueroa,
Connor and Maliski (2008) suggested that researchers’ interviews should be
conducted in the participants’ preferred language so that they will have a clearer
understanding of the issues under investigation. The interview was conducted
in a quiet classroom to help maximize its quality. Each interview was planned to
last about 15 minutes but some of the interviews lasted longer and each
interview was audio-recorded as all the participants had consented to this.
Data Analysis
The data obtained through the questionnaires were first organized into a
suitable form for its analysis. Since the questionnaire items were closed-ended
questions, steps proposed by Dörnyei (2010) were followed to process the

200
information. These steps include data check and cleaning, data manipulation,
reduction of the number of variables, measurement of data reliability and
validity, and statistical analyses. The data were then analyzed using descriptive
statistics and non-parametric tests with the help of SPSS. The data obtained
through the interviews were analyzed using a thematic analysis following the
steps proposed by Braun and Clarke (2006). Before the analysis was conducted,
the audio-recorded interviews were transcribed verbatim. After all the interview
data were transcribed, the transcripts were rechecked to verify that there were
no obvious mistakes made during the transcription process. Then, the analysis
was conducted in the source language (Bahasa Indonesia). Pseudonyms were used
in reporting the interview results to keep the anonymity of the participants.
4. Findings
Autonomous English Language Learning Activities outside the Class
Table 1 shows the percentages of the students’ responses to items probing their
autonomous English learning activities outside the class. As shown in the table,
9 out of 22 activities appeared to be frequently (‘often’ or ‘sometimes’) practiced
by more than half of the students. Meanwhile, 13 activities were infrequently
(‘never’ and ‘rarely’) practiced.
Table 1: Students’ autonomous English language learning activities outside the class
Often Sometimes Rarely Never
No. Activities
(%) (%) (%) (%)
1. Reading grammar books on your own 11.19 54.73 29.10 4.98
2. Doing exercises which are not compulsory 11.44 36.07 36.82 15.67
3. Noting down new words and their meanings 37.81 33.08 21.14 7.96
4. Reading English notices around you 20.65 36.57 30.35 12.44
5. Reading newspapers in English 6.72 20.40 37.06 35.82
6. Sending e-mails in English 8.96 18.66 26.87 45.52
7. Reading books or magazines in English 12.44 28.61 40.55 18.41
8. Watching English TV programs 43.28 33.58 20.15 2.99
9. Listening to English radio 7.21 17.66 32.59 42.54
10. Listening to English songs 63.43 25.12 8.46 2.99
11. Talking to foreigners in English 4.98 13.43 31.34 50.25
12. Practicing using English with friends 19.65 38.06 32.84 9.45
13. Doing English self-study in a group 13.93 31.09 39.30 15.67
14. Watching English movies 61.69 26.37 9.20 2.74
15. Writing a diary in English 6.97 15.42 26.37 51.24
16. Using the internet in English 29.60 32.84 27.11 10.45
17. Doing revision not required by the teacher 6.47 26.12 35.57 31.84
18. Collecting texts in English (e.g. articles, 12.69 23.38 35.07 28.86
brochures, labels, etc.)
19. Going to see the teacher about your work 9.95 26.37 39.80 23.88
20. Attending meetings in English 4.73 11.69 28.36 55.22
21. Watching videos/DVDs/VCDs 45.77 28.86 18.91 6.47
22. Reading English news online 15.67 27.61 35.32 21.39

201
The most frequently (often or sometimes) practiced activities are listed below in
the order of frequency:
- Listening to English songs (88.55% ‘often’ or ‘sometimes’)
- Watching English movies (88.06% ‘often’ or ‘sometimes’)
- Watching English TV programs (76.86% ‘often’ or ‘sometimes’)
- Watching videos/DVDs/VCDs (74.63% ‘often’ or ‘sometimes’)
- Noting down new words and their meanings (70.89% ‘often’ or
‘sometimes’)
- Reading grammar books on your own (65.92% ‘often’ or ‘sometimes’)
- Using the internet in English (62.44% ‘often’ or ‘sometimes’)
- Practicing using English with friends (57.71% ‘often’ or ‘sometimes’)
- Reading English notices around them (57.22% ‘often’ or ‘sometimes’)
The students indicated that ‘never’ or ‘rarely’ engaged in the following activities:
- Attending meetings in English (83.58% ‘never’ or ‘rarely’)
- Talking to foreigners in English (81.59% ‘never’ or ‘rarely’)
- Writing a diary in English (77.61% ‘never’ or ‘rarely’)
- Listening to English radio (75.13% ‘never’ or ‘rarely’)
- Reading newspapers in English (72.88% ‘never’ or ‘rarely’)
- Sending e-mails in English (72.39% ‘never’ or ‘rarely’)
- Doing revision not required by the teacher (67.41% ‘never’ or ‘rarely’)
- Collecting texts in English (e.g. articles, brochures, labels, etc.) (63.93%
‘never’ or ‘rarely’)
- Going to see the teacher about your work (63.68% ‘never’ or ‘rarely’)
- Reading books or magazines in English (58.96% ‘never’ or ‘rarely’)
- Reading English news online (56.71% ‘never’ or ‘rarely’)
- Doing English self-study in a group (54.97% ‘never’ or ‘rarely’)
- Doing exercises which are not compulsory (52.49% ‘never’ or ‘rarely’)
The results in the interview confirm those obtained in the questionnaire, which
indicated listening to music was the highest-rated activity. When asked whether
the activity was done more to learn or just for fun, some of the students said that
they did it for both, others said it was more for fun, and the rest claimed that
they did it more for learning. Kartika and Ahmad, for example, said that they
listen to music more for pleasure than a learning purpose. Kartika stated, ‘Saya
sering mendengarkan musik tapi hanya untuk hiburan saja. Saya hanya mendengarkan
dan menikmati lagunya, tidak pernah menterjemahkan liriknya’ (I often listen to
English songs but just for fun. I just listen and enjoy the songs, never translate
the lyrics). A similar sentiment was that of Ahmad. He said, ‘Saya cuma
mendengarkan lagu bahasa Inggris sekedar untuk hiburan saja’ (I just listen to English
songs for fun only). Shinta, on the other hand, said that she listened to music
more for learning than for fun. She commented, ‘Saya biasanya menterjemahkan
lirik lagu yang saya dengarkan. Sering mendengarkan musik akan memudahkan saya
menghafal liriknya’ (I usually translate the lyrics of the song I listen to. Listening
to music frequently will allow me remembering the lyrics easier).
Perceived Constraints in Performing Autonomous Learning outside the Class

Semi-structured interviews were conducted with 30 selected students to ask
about the constraints they had in performing autonomous learning outside the

202
class. First, the students were asked whether they considered themselves
autonomous learners. A vast majority of the interviewed students admitted that
they were not autonomous learners. When asked the reason for not exercising
autonomous learning, the students gave various answers. Andika, for example,
commented that he did not know how to learn autonomously. He said, ‘Saya
tidak tahu bagaimana belajar mandiri itu dilakukan’ (I don’t know how autonomous
learning should be done). Riana shared similar reasoning but added that she
needed to learn how to learn autonomously from others. She commented, ‘Saya
masih belum mampu untuk belajar secara mandiri, saya masih ingin belajar dari orang
yang lebih tahu daripada saya’ (I can’t learn autonomously, I still need to learn
from those who are more able than me).
Sintia, Rinjani, and Amanda shared another different reason, stating that English
is hard to learn. As Sintia said: ‘Ya jujur saja bagi saya Bahasa Inggris itu sangat
sulit dipelajari. Tulisan dan bacaannya beda, tidak seperti Bahasa Indonesia’ (Frankly
speaking, for me, English is very hard to learn. Unlike Bahasa Indonesia, English
words are pronounced differently compared to how they are written). A similar
opinion was given by Rinjani, saying, ‘Saya kurang minat belajar bahasa Inggris.
Saya tidak suka. Sudah pernah mencoba belajar seperti menghafal kosa kata, tapi sangat
sulit, terutama pengucapannya’ (I am not interested in learning English. I just don’t
like English. I have tried to learn, like memorizing vocabulary, but I find it very
difficult, especially pronunciation). Meanwhile, Amanda related this difficulty
with English grammar. She commented, ‘Salah satu alasan kenapa saya tidak belajar
mandiri itu karena bahasa Inggris sangat sulit, terutama tenses-nya’ (One reason why
I do not learn autonomously is that English is very difficult, especially its
tenses).
Eko and Bambang remarked that they did not engage in autonomous English
learning activities because they have limited time after class. Eko explained,
‘Waktu sata sangat terbatas karena saya harus bekerja sepulang kuliah’ (I have very
limited time because I have to work after class). For Bambang, much of his time
after class was spent on some off-campus social activities. He said, ‘…saya sangat
sibuk diluar. Saya aktif dalam kegiatan-kegiatan sosial diluar kampus’ (…I am quite
busy outside. I am active in the activities of social organizations outside the
campus). For Melani, most of her time after class was used for helping her
parents doing housework. However, she tried to learn English if she had spare
time. She said, ‘Meskipun saya sangat sibuk melakukan pekerjaan di rumah, kadang-
kadang saya belajar bahasa Inggris bila saya punya waktu senggang disela-sela
kesibukan’ (Although I am very busy at home doing housework, I sometimes
study English if I have spare time in my busy days).
Another reason given by the students for not practicing autonomous learning
outside the class was that learning resources were limited. This point was made
by Budi who commented, ‘Sumber belajar seperti buku sangat terbatas’ (Learning
materials, such as books, are very limited)’. On this same matter, Eva remarked:
‘…buku-bukunya terbatas… Dan juga, saya sulit mencari teman yang mau berdiskusi,
belajar bersama, dan lain-lain’ (…the availability of books is limited… Also, it is
very hard for me to find a friend who wants to discuss, study together, and so
on).

203
Interestingly, out of 30 interviewed students, only few who thought that they
were autonomous learners. Putri, for example, commented, ‘Diluar kelas insya
Allah saya dapat belajar bahasa Inggris sendiri. Saya merasa bahwa belajar dengan
arahan dari orang lain sangat sulit. Namun bagi saya belajar sendiri itu lebih mudah’
(Outside the class, God willing I can learn English by myself. I feel learning
through a guide is difficult. I found that learning English on my own is easier).
Zaskia also believed that she had already undertaken autonomous learning. She
said that although she did not have a class on campus, she often came and
attended other courses. ‘Saya sangat menyukai bahasa Inggris… Maksud saya,
bersama teman, saya sering mengikuti kelas lain di jurusan Bahasa Inggris. Kami hanya
duduk dan mengikuti pelajarannya. Menyenangkan dan kita dapat memperoleh
pengetahuan baru karena kelasnya diajarkan oleh dosen yang berbeda’ (I really like
English… I mean, with a friend of mine, I often attend other courses in the
English major. We just sit and follow the lesson. I find it enjoyable and we can
get new knowledge because the courses are taught by different teachers). These
comments indicate that, although the majority of the students did not engage in
autonomous learning after class, a small minority were determined to learn and
find learning opportunities.
Autonomous English Learning Activities outside the Class Concerning

Gender and Majors of Study
Two different Mann-Whitney U tests were conducted to examine if females
(n=210) and males (n=192) and the students doing an English major and the
students doing majors other than English differed in autonomous English
learning activities outside the class (See Table 2 and Table 3).
Table 2: Mann-Whitney U test results of the differences in students’ perceptions of

their autonomous English learning activities outside the class regarding gender
(N=402)
Activities outside the class

Mann-Whitney U 18412.500
Female mean rank 209.82
Male mean rank 192.40
z-score -1.502
p-value .133
Table 3: Mann-Whitney U test results of the students’ autonomous English learning

activities outside the class depending on major of study (N=402)
Activities outside the class

Mann-Whitney U 6222.500
English major mean rank 256.84
Non-English major mean
193.28
rank
z-score -3.682
p-value .000

204
As can be seen in Table 2, there is no significant difference in the level of practice

of autonomous out-of-class activities between female students (Mean rank =
209.82, n = 210) and male students (Mean rank = 192.40, n = 192), U =
18412.500, z = -1.502 (corrected for ties), p>.05, two-tailed. Meanwhile, as shown
in Table 3, the level of practice of out-of-class activities of the English major
students (Mean rank = 256.84, n = 52) is significantly higher than that of the non-
English major students (Mean rank = 193.28, n = 350), U = 6222.500, z = -3.682
(corrected for ties), p< .05, two tailed.
5. Discussion
The first research question focused on the extent to which Indonesian tertiary
students engage in autonomous language learning activities outside the class. As
shown in the previous section, there were 9 out of the 22 activities appeared to
be more frequently (‘often’ or ‘sometimes’) practiced by more than half of the
students. In contrast, 13 activities that infrequently (‘never’ and ‘rarely’)
practiced by more than half of the students.
It is interesting to note that the four activities that were ‘often’ practiced involve
the use of technology. These results confirm the results obtained by Ardi (2013)
in a study conducted in the Jakarta context, Indonesia, in which most of the
widely practiced activities were also related to the use of technology. In terms of
the type of activities, these results are consistent with the results achieved in
several studies conducted in different contexts (e.g. Chan et al., 2002; Koçak,
2003; Pearson, 2004; Tamer, 2013), which have generally highlighted some
similar out-of-class activities, and most of the activities involved receptive rather
than productive activities.
It appears that the advancements of technology have provided the students with
access to a variety of English programs and facilitated their engagement in
language learning without the presence of a teacher. This was confirmed in the
interviews where the students took advantage of the ease of access to technology
for language skill practice and this was not only for entertainment purposes. The
benefits of using technology for language learning have been endorsed in the
literature. Kessler (2009) stated that technology provides opportunities for
students to use language in authentic contexts. “Such activities encourage
students to strive for autonomy in the target language” (p. 79). Quite evidently,
the use of technology for out-of-class language learning is a means of enhancing
students’ exposure to the target language by providing opportunities for
language practice in different contexts (Lai, Yeung & Hu, 2015). Furthermore,
“Technology has the potential to not only provide access to resources for
learning in a superficial sense but also to offer increased affordances for
autonomous learning” (Reinders & White, 2011, p. 1). However, previous
research has suggested that students lack a good understanding of how
available technologies can be used effectively for language learning purposes
(Lai et al., 2015). Therefore, it may be necessary for teachers to consider
providing students with support on how to make the most of such resources for
effective English learning.

205
The results of this study also showed that there were 13 activities in which more
than half of the students said that they ‘never’ or ‘rarely’ practiced. One reason
why some of the activities were infrequently practiced could be due to the
limited access to the resources in students’ living environment. For instance, the
students are unlikely to find foreigners in their daily life with whom they can
practice their English. They also have limited access to resources such as English
newspapers or radio programs. Of note here is that the present study was
conducted in two regencies in one of the many provinces in Indonesia where
access to the resources such as English radio, English newspapers, and
foreigners are scarce. The low frequency of participation in other activities such
as attending meetings, writing a diary, sending e-mails, and reading books or
magazines in English may indicate that these learning activities are also not a
common part of students’ learning experiences in this context.
Although the questionnaire results indicated that the students did engage in
some out-of-class activities, the majority of the students in the interviews
admitted that they were not autonomous. There was apparent conformity
among them regarding the reasons why they did not practice autonomous
learning. Lack of capacity to learn autonomously was one of the most mentioned
reasons, and there was apparent agreement that they needed guidance from the
teacher. Some of the students also commented that for them, English is hard to
learn. This perceived difficulty of English influenced their attitude towards
English and made them not interested to learn it. For several students, time
shortage outside the class was regarded as another constraint to engaging in
autonomous learning. Although an enormous amount of time is available after
class, some students were using it for other activities, including work
commitments, social organization, and doing housework. This suggests that the
limited time is more due to the students’ environmental factors than the
workload they carry in their educational institutions. It may be interesting to
note that although some of the students suggested that they had the intention to
study after class, they had to help their parents do housework. This is very
common in the Indonesian context, where children, especially women, are
supposed to help their parents with in-house related work.
Some of the interviewees also mentioned that learning resources for English
learning were limited. In this respect, the students mentioned that there were
very few hard copy books available and a lack of interest from friends in
studying together. In this regard, researchers have suggested that learning
resources are indeed an essential element in developing learner autonomy. Zhao
and Chen (2014) stated that materials play a pivotal role in developing and
breeding learner autonomy, predominantly because they motivate learners for
their English study. Dickinson (1987) believed that for learner autonomy to be
exercised, materials need to be made available in a site where learners can have
access to them.
The third question asked if there are any statistically significant differences in
the autonomous English learning activities outside between females and males
and between the students who are English majors and the students who are non-

206
English majors. The results of the analysis in the previous section showed that
there is no significant difference in the level of practice of autonomous out-of-
class activities between female students and male students. These results
confirm the results obtained in Koçak’s (2003) study done in a different cultural
context. However, the results of the current research contradict those achieved
by Varol and Yilmaz (2010), which showed that there were significant
differences between female and male students in their autonomous learning
activities outside the class in favor of females.
Regarding the majors of study, the results showed that the level of practice of
out-of-class activities of the English major students is significantly higher than
that of the non-English major students. These suggest that the students of the
English major accept more responsibilities and engage more in autonomous
English learning activities outside the class than the other group does. One
possible explanation is that, unlike the students of non-English majors, the
students of English majors are expected to have stronger motivation to learn
English as they have chosen the career as English language teachers. English
major students were students enrolled in a teaching degree which was preparing
them to be English teachers. Thus, they assume greater responsibilities in their
English learning compared to those of non-English majors. In this respect, it may
be important to consider Locke and Latham's goal-setting theory (Locke &
Latham, 1994) which states that human activity is stimulated by purpose, and
for action to happen, “goals have to be set and pursued by choice” (Dörnyei,
1998, p. 120). The goals, according to Dörnyei, are regarded as “the 'engine' to
fire the action and provide the direction in which to act” (p. 120). In other words,
while for non-English majors, the subject of English was a compulsory part of
their degree, for English major, it was a personal choice or drive. This personal
interest in the language might have fuelled their engagement in out of class
learning activities and their adoption of taking more responsibilities towards
their learning.
6. Conclusion and Limitations

The purpose of the present study was three folds: (1) to describe the extent to
which Indonesian students engaged in autonomous language learning beyond
the classroom, (2) to explore the students’ perceived constraints in performing
their autonomous learning, and (3) to examine whether there are any statistically
significant differences in the autonomous English learning activities outside
between females and males and between the students who are English major
and the students who are non-English majors. The results showed that the
students did engage in several out-of-class English learning activities; however
many of the activities were more receptive than productive. In the interviews,
the students claimed that they were not autonomous learners and offered
several reasons for not exercising autonomous learning. The results also
revealed that there is no significant difference in the level of practice of
autonomous out-of-class activities between female students and male students,
but there is a significant difference in the level of practice of autonomous out-of-
class activities between English major students and non-English major students.
All these findings formed the basis for offering recommendations for the

207
teaching and learning context. Since the teacher plays a vital role in the effort to
promote learner autonomy, focusing on what the teacher can do to help students
develop their autonomy is imperative.
The results highlighted that most of the activities the students frequently
engaged in involved the use of technology. Therefore, teachers are expected to
provide students with support on how to make the most of technology for
effective English learning. The role of teachers could be in forms of
encouragement, recommendations on which resources to use, tips on how to use
the resources, advice on metacognitive and cognitive strategies, using
technology in the classroom, and assigning homework involving the use of
technological resources. Drawing the students’ attention to the benefits of
technology would be a step toward increasing their motivation in language
learning and discovering a range of online resources that would likely trigger
their excitement and interest, which in turn would lead to autonomous learning.
Also, since social interaction is an integral part of the development of learner
autonomy, teachers should encourage more social interaction and collaboration
among students either inside or outside the classroom or both.
The results also showed that the students’ perceived constraints in performing
autonomous learning are both extrinsic and intrinsic. While some of the
problems such as the shortage of time and learning resources are conditional on
external factors, students’ lack of capacity to learn autonomously and difficulties
experienced in learning English can be addressed in the language learning
classroom. Providing the students with training in autonomous learning and
motivating them are critical initial steps that the teacher could take to support
the students.
In the institutions of higher education, especially in the English teaching

programs, training on learner autonomy should be given to pre-service teachers.
They should be provided with the skills to promote learner autonomy and be
given the first-hand experience of learner autonomy in their training. Rigorous
training not only would enhance their autonomous ability for their learning as
pre-service teachers but also improve their professional development giving
them knowledge and skills they could apply in their teaching in the future.
Additionally, since the development of learner autonomy is dependent upon the
development of teacher autonomy, teachers should be autonomous themselves
before they can give training to their students. Thus, the teachers who have
inadequate knowledge of strategy training should update their knowledge and
skills either through self-study or professional development programs.
As with any research, it is essential to acknowledge that the present study has
some limitations. The first limitation is that the data were collected through
student reporting, i.e. questionnaires and interviews. Using other types of
instruments, such as observations and learner diaries, could give more detailed
information about the students’ autonomous English learning realities beyond
the classroom. Secondly, the data were collected only from four out of many
institutions of higher education in Indonesia and focused only on one

208
geographical area. The inclusion of more higher education institutions from

different regions could increase the generalizability of the findings.
Recommendations for Further Research

The present study provides a better understanding of Indonesian tertiary EFL
students’ out-of-class autonomous English learning and their perceived
constraints in performing their English learning beyond the classroom.
Moreover, it has identified some potential lines of inquiry that future research
should explore. Future research should investigate English teachers’ beliefs
about learner autonomy, which will provide information on teacher readiness to
promote learner autonomy. Future research could usefully explore the
relationships between autonomous learning and other variables such as
geographical areas, proficiency levels, socio-economic background, and
personality traits to give a better picture of the factors that potentially affect
learner autonomy. Future research could also extend its scope to younger
students, such as those of junior and senior high school, to give a better
understanding of students’ perspectives on autonomous learning. More
information on this topic could assist efforts to promote learner autonomy,
implement appropriate tasks and strategies to promote it as early as possible.
7. References
Ardi, P. (2013, April 25-28). Investigating tertiary students' autonomy in out-of-class EFL
learning. Paper presented at the Asian Conference on Language Learning 2013,
Osaka, Japan. Retrieved from http://25qt511nswfi49iayd31ch80-
wpengine.netdna-ssl.com/wp-
content/uploads/papers/acll2013/ACLL2013_0282.pdf
Benson, P. (1997). The philosophy and politics of learner autonomy, In P. Benson and P.
Voller (eds), Autonomy and Independence in Language Learning. London:
Longman, 1997.
Benson, P. (2001). Teaching and researching autonomy in language learning. Harlow,
England: Longman.
Benson, P. (2011). Language learning and teaching beyond the classroom: An
introduction to the field. In P. Benson & H. Reinders (Eds.), Beyond the language
classroom (pp. 7-16). Basingstoke, England: Palgrave Macmillan.
Blidi, S. (2017). Collaborative learner autonomy: A mode of learner autonomy development.
Singapore: Springer.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative
Research in Psychology, 3(2), 77-101. doi:10.1191/1478088706qp063oa
Chan, V., Spratt, M., & Humphreys, G. (2002). Autonomous language learning: Hong
Kong tertiary students' attitudes and behaviours. Evaluation & Research in
Education, 16(1), 1-18.
Chen, H., & Li, N. (2014). The analysis of factors affecting autonomy of English-major
postgraduates. In S. Narayanasamy (Ed.), 2014 International Conference on
Social Science and Management (pp. 250-253). Lancaster, PA: DEStech
Publications.
Cirocki, A., Anam, S., & Retnanigdyah, P. (2019). Readiness for autonomy in English
language learning: The case of Indonesian high school students. Iranian Journal of
Language Teaching Research, 7(2), 1-18.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods
research (2nd ed.). Thousand Oaks, CA: Sage.

209
Dardjowidjojo, S. (2000). English teaching in Indonesia. English Australia Journal, 18(1),

21-30.
Dickinson, L. (1987). Self-instruction in language learning. Cambridge, England:
Cambridge University Press.
Dörnyei, Z. (1998). Motivation in second and foreign language learning. Language
Teaching, 31, 117-135. doi:10.1017/S026144480001315X
Dörnyei, Z. (2010). Questionnaires in second language research: Construction, administration,
and processing (2nd ed.). New York, NY: Routledge.
Hashim, H. U., Yunus, M. M., & Hashim, H. (2018). Language learning strategies used
by adult learners of Teaching English as a Second Language (TESL). TESOL
International Journal, 13(4), 39-48.
Hermagustiana, I., & Anggriyani, D. (2019). Language Learner Autonomy: The Beliefs of
English Language Students. IJEE (Indonesian Journal of English Education), 6(2),
133-142. doi:10.15408/ijee.v6i2.15467
Holec, H. (1981). Autonomy and foreign language learning. Oxford, England: Pergamon
Press.
Hyland, F. (2004). Learning autonomously: Contextualizing out-of-class English
language learning. Language Awareness, 13(3), 180-202.
Inozu, J., Sahinkarakas, S., & Yumru, H. (2010). The nature of language learning
experiences beyond the classroom and its learning outcomes. US-China Foreign
Language, 8(1), 14-21. Retrieved from
https://www.academia.edu/574045/The_nature_of_language_learning_experie
nces_beyond_the_classroom_and_its_learning_outcomes
Kessler, G. (2009). Student-initiated attention to form in wiki-based collaborative writing.
Language Learning & Technology, 13(1), 79-95. Retrieved from
http://llt.msu.edu/vol13num1/kessler.pdf
Koçak, A. (2003). A study on learners’ readiness for autonomous learning of English as a foreign
language (Master’s thesis, Middle East Technical University, Ankara, Turkey).
Retrieved from https://etd.lib.metu.edu.tr/upload/1217728/index.pdf
Lai, C., Yeung, Y., & Hu, J. (2015). University student and teacher perceptions of teacher
roles in promoting autonomous language learning with technology outside the
classroom. Computer Assisted Language Learning, 1-21.
Little, D. (2007). Language learner autonomy: Some fundamental considerations
revisited. Innovation in Language Learning and Teaching, 1(1), 14-29.
Littlewood, W. (1999). Defining and developing autonomy in East Asian contexts.
Applied Linguists, 20(1), 71-94.
Locke, E. A., & Latham, G. P. (1994). Goal setting theory. In H.F. O’Neil, Jr. & M.
Drillings (Eds.), Motivation: Theory and research (pp. 13-29). Hillsdale, NJ:
Lawrence Erlbaum Associates.
Lopez, G. I., Figueroa, M., Connor, S. E., & Maliski, S. L. (2008). Translation barriers in
conducting qualitative research with Spanish speakers. Qualitative Health
Research, 18(12), 1729-1737. doi: 10.1177/1049732308325857
Madya, S. (2002). Developing standards for EFL in Indonesia as part of the EFL teaching
reform. TEFLIN Journal, 13, 142-51.
Marcellino, M. (2008). English language teaching in Indonesia: A continuous challenge in
education and cultural diversity. TEFLIN Journal, 19(1), 57-69.
Maros, M., & Saad, N. S. M. (2016). The out-of-class language learning strategies of
international students in Malaysia. International Journal of Asian Social Science,
6(8), 478-486. Retrieved from http://www.aessweb.com/pdf-files/ijass-2016-
6(8)-478-486.pdf

210
Mustafa, F. (2018). How much do high schools contribute to improving students’ English
proficiency? Seeking alumni’s perception in Indonesia. Asian EFL Journal, 20(2),
49-61.
Muthalib, K. A, Ys, S. B., & Mustafa, F. (2019). Why are you different? Investigating
reasons of success by high achieving EFL students. The Asian EFL Journal, 21(2.4),
166-182. Retrieved from
https://www.researchgate.net/publication/332195934_Why_are_you_different
_Investigating_reasons_of_success_by_high_achieving_EFL_students
Norton, B., & Toohey, K. (2001). Changing perspectives on good language learners.
TESOL Quarterly, 35(2), 309-22.
Nuh, M. (2013). Pengembangan Kurikulum 2013. Ministerial presentation at Universitas
Pendidikan Indonesia, Bandung, Indonesia, 16 March 2013.
Nunan, D. (2014). Beyond the classroom: A case of out-of-class language learning. In C.
K. Thomson (Ed.), National Symposium on Japanese Language Education Proceedings
(pp. 28-37). Chippendale, NSW: The Japan Foundation, Sydney.
Orhon, Y. (2018). An investigation of out-of-class language activities of tertiary-level EFL
learners. Education Reform Journal, 3(1), 1-14. Retrieved from
https://files.eric.ed.gov/fulltext/ED586036.pdf
Pearson, N. (2004). The idiosyncrasies of out-of-class language learning: A study of
mainland Chinese students studying English at tertiary level in New Zealand. In
H. Reinders, H. Andersson, M. Hobbs & J. Jones-Parry (Eds.), Supporting
independent learning in the 21st century. Proceedings of the Inaugural Conference
on the Independent Learning Association, Melbourne, Australia. 13–14
September 2003.
Reinders, H., & White, C. (2011). Special issue commentary: Learner autonomy and new
learning environments. Language Learning & Technology, 15(3), 1-3.
Rowland, C. (2014). Understanding child language acquisition. Oxon, England: Routledge.
Saraswati, S. (2019, April). Learner autonomy: Investigating Indonesian English teachers’
beliefs. Paper presented at the First Bogor English Student and Teacher
Conference, Bogor, Indonesia. Retrieved from http://pkm.uika-
bogor.ac.id/index.php/best/article/download/534/481
Smith, R. (2008). The history of learner autonomy. In L. Dam (Ed.), 9th Nordic conference
on developing learner autonomy in language learning and teaching: Status and ways
ahead after twenty years, 2006. Copenhagen: CVU. Retrieved from
http://autonomyhistories.pbworks.com/f/The+History+of+Learner+Autonom
y.pdf
Sutiono, C., Saukah, A., Suharmanto, & Oka, M. D. D. (2017). Out-of-class activities
employed by successful and unsuccessful English department students. Jurnal
Pendidikan Humaniora, 5(4), 175-188.
Tamer, O. (2013). A Dissertation on students’ readiness for autonomous learning of
English as a foreign language. Master’s Dissertation, The University of
Sunderland, Sunderland, UK. Retrieved from
http://www.awej.org/images/Theseanddissertation/OmerTamer/
109thesis.pdf
Ushioda, E. (2001). Language learning at university: exploring the role of motivational
thinking. In Z. Dörnyei & R. Schmidt (Eds.) Motivation and second language
acquisition (Technical Report-23) (pp. 93-125). Honolulu, Hawaii: University of
Hawaii, Second Language Teaching and Curriculum Center.
Varol, B., & Yilmaz, S. (2010). Similarities and differences between female and male
learners: Inside and outside class autonomous language learning activities.
Procedia Social and Behavioral Sciences, 3, 237-244. doi:10.1016/j.sbspro.2010.07.038

211
Wu, M. M. (2012). Beliefs and out-of-class language learning of Chinese-speaking ESL

learners in Hong Kong. New Horizons in Education, 60(1), 35-52. Retrieved from
http://eric.ed.gov/?id=EJ974077
Yorozu, M. (2001). Interaction with native speakers of Japanese: What learners say in
Japanese. Studies, 21(2), 199-213.
Yulia, Y. (2014). An evaluation of English language teaching programs in Indonesian junior
high schools in the Yogyakarta province (Doctoral thesis, RMIT University,
Melbourne, Australia). Retrieved from
https://researchbank.rmit.edu.au/eserv/rmit:160956/Yulia.pdf
Zhao, X., & Chen, W. (2014). Correlation between learning motivation and learner
autonomy for non-English majors. World Transactions on Engineering and
Technology Education, 12(3), 374-379.

212
APPENDICES
Appendix A
Students’ Questionnaire
Student Name:
Student Identification Number:
Major:
Sex (Please tick):  Male  Female
Please tick the appropriate box.

In this last academic year, how often have you:
often Sometimes rarely never
1. read grammar books on your own?    
2. done exercises which are not    
compulsory?
3. noted down new words and their    
meanings?
4. read English notices around you?    
5. read newspapers in English?    
6. sent e-mails in English?    
7. read books or magazines in English?    
8. watched English TV programs?    
9. listened to English radio?    
10. listened to English songs?    
11. talked to foreigners in English?    
12. practiced using English with friends?    
13. done English self-study in a group?    

14. watched English movies?    
15. written a diary in English?    
16. used the internet in English?    
17. done revision not required by the    
teacher?
18. collected texts in English (e.g. articles,    
brochures, labels, etc.)?
19. gone to see the teacher about your    
work?
20. attended meetings in English?    
21. watched videos/DVDs/VCDs?    
22. read English news online?    

213
Appendix B
Interview Questions
1. To what extent do you consider yourself an autonomous learner? Explain

your answer.
2. What constraints do you face in practicing autonomous learning outside
the class?

214

Vol. 19, No. 10, pp. 214-232, October 2020
Computer Coding and Choreography:

Contrasting Experiences of Learning About
Collaboration in Engineering and Creative Arts
Nicholas Rowe
University of Auckland, Auckland, New Zealand
https://orcid.org/0000-0001-7304-1047
Rose Martin
Norwegian University of Science and Technology, Trondheim, Norway
https://orcid.org/0000-0003-1732-2666
Nasser Giacaman
University of Auckland, Auckland, New Zealand
https://orcid.org/0000-0001-6885-1571
Abstract. This article argues that how collaboration is taught can have a
significant impact on the ways in which collaboration is experienced,
understood and valued. In doing so, the study draws attention to
performing arts studio-pedagogies, and their potential relevance to
enhancing creativity within science, technology, engineering, and
mathematics (STEM) education. Through a mixed-methods study of
teachers’ and students’ experiences of group work, this article compares
two disciplines that maintain distinct discourses on teaching
collaboration: Software design and choreography. The quantitative data
reveals that despite significant demographic differences, students from
the two disciplines maintain a common enthusiasm for group learning.
There are significant distinctions however, on student perceptions of the
teaching and learning of collaboration, their learning achievements about
group work, and the relevance of group work in their discipline.
Qualitative commentaries from students and teachers extend the
arguments across both the distinctions and the similarities, emphasizing
the impact of particular teaching practices and establishing standpoints
for further research into the pedagogy of collaboration in higher
education.
Keywords: collaboration; choreography; group-work; pedagogy;

software design

215
1. Introduction
The ability to effectively collaborate is a key professional competence within the
knowledge economies of the 21st century. As a result, ‘collaboration’ has become
ubiquitous as a graduate attribute across tertiary education programmes
(Burbach, Matkin, Gambrell & Harding, 2010). While often referred to as a ‘soft
skill’, an ability to collaborate can be as valuable to employers as the ‘hard-skill’
comprehension of domain-specific knowledge, as group collaboration can
develop more creative ideas than individuals working in isolation (McMahon,
Ruggeri, Kammer & Katsikopoulos, 2016). Collaboration allows professionals to
contribute to creative strategies and ideas, extend a sense of purpose, agency and
motivation to others in the workplace, and thereby enhance the prospects of a
collective venture (Hung, Durcikova, Lai & Lin, 2011). How students experience
learning about collaboration can therefore have a significant impact on their
professional future.
The ability to collaborate has often been assumed by educators to be an innate

behavior or characteristic, which can simply be enhanced by practice (Hesse, Care,
Buder, Sassenburg & Griffin, 2015). Evidencing this practice within the
curriculum has generally involved the assessment of small-group tasks, in which
students collaborate to solve problems and/or design solutions (Griffin & Care,
2015). In many contexts, the assignment of a collaborative task is where the
teaching of collaboration starts and ends, following the educational proposition
that “the best way to learn about working together is by doing it” (Horder, 1995,
p.158). Collaboration is a complex skill and disposition however, which can
actually be taught and learnt through exposure to diverse approaches, guidance
into deeper reflection and relevant application within specific disciplinary
contexts (Hesse et al, 2015). Within this article we argue that students can
experience significantly different teaching approaches to collaborative group-
work assignments, which can impact how they subsequently value those learning
experiences, and how they perceive the relevance of collaboration within their
discipline.
To deeply understand the ways that teaching through small group tasks can
impact on students’ experiences of learning about collaboration, we engaged in a
multi-phase cross-faculty study. This study investigated teaching practices within
two programmes in our institution that approach collaborative coursework in
very distinct ways: Software design and choreography. While previous research
has investigated the significance of integrating arts learning within engineering
education to enhance creativity, our cross-disciplinary comparative analysis seeks
to go a step further: to enhance understandings of how and why specific creative
arts studio-pedagogies (rather than practices) might be transferred across
disciplines, to advance the teaching of creativity and collaboration within STEM
subjects in tertiary education.
The pedagogic distinctions between choreography and software design presented

here extend beyond the particular practices of individual teachers, and can be seen
as rooted within the educational discourses of these two disciplines, and the wider
educational discourses of engineering and creative arts (Rowe, 2019). Our article

216
therefore begins with a consideration of how collaboration has been theorized

within educational literature, and its relevance to the domains of dance and
software design. This leads into a review of our mixed-methods approach to this
study.
We then explore the themes that emerged from our research into students and
teachers of software design and choreography. Interviews with teachers and
classroom observations revealed that teachers from both groups value
collaborative coursework as relevant to the students’ professional futures, yet
their approaches to teaching group work are markedly different. Our student data
reveals that while students in both groups hold generally similar attitudes
towards group projects in class, there is a marked difference in how the two
student groups feel that they are taught about collaboration, what they feel that
they learn about working in groups, and ultimately how relevant they think group
work is to their discipline. These findings support our argument that how
collaboration is taught can have a significant impact on how students learn and
value group tasks within coursework. This suggests the need for further research
into how tertiary educators teach collaboration, and the potential transference of
pedagogic strategies from creative arts to engineering.
2. Collaboration in work and education: software design and

choreographic arts
In professional contexts, collaboration requires individuals to contribute (and
accommodate) divergent perspectives within the development of innovative
products and services. This distinguishes collaboration from more pervasive,
industrialized models of professional collective endeavor, like cooperation.
Through cooperation, individuals provide (often highly skilled) labour to an
enterprise, without necessarily engaging in processes of collective deliberation on
the nature of that enterprise (Mattessich & Monsey, 1992). The collective
deliberation that distinguishes collaboration from cooperation can require
individuals to engage in promotive relationships (Johnson & Johnson, 2009) that
maintain a sense of symmetry (Dillenbourg, 1999) and value pluralism (Owen,
2015) when making collaborative decisions. Participating in such creative
decision-making can be self-actualizing for individuals within an enterprise
(Csikszentmihalyi, 1996). Research has further shown that the more motivated
that collaborators are to support the success of the collective enterprise (rather
than motivated by just an economic exchange for their contributions), the more
valuable their contributions tend to be towards the creative outcomes (Hung et al,
2011). Developing a collaborative disposition can therefore be seen as an
important professional competence, which can require learning processes that
enhance both collaborative skills and collaborative motivations.
Within education, collaboration is often associated with small-group learning, in

which students interact on creative tasks related to the subject being studied.
Small-group learning is not new in education: such peer-to-peer activity has been
a central tenet of progressive education systems for the last half century (Rodgers,
1982). It is important to note however, that such peer-learning relationships were
not introduced into formal education in order to enhance students’ abilities to

217
collaborate; small-group work was introduced with the goal of advancing

individual student achievement (Hennessy & Murphy, 1999). The need for
students to learn to collaborate (rather than through collaboration) came later, in
response to industry demands that graduates have an enhanced capacity to
effectively innovate in teams (Boden & Nedeva, 2010). The distinction between
learning to and learning through collaboration has not been pervasively
recognized within much of the literature on collaborative learning in higher
education (Griffin & Care, 2015). This can lead tertiary educators to suppose that
simply engaging in small-group tasks is sufficient to learn about collaboration,
sustaining the belief that “the sheer amount of interaction [. . .] is a powerful
predictor of learning” (Cohen, 1994, p.16). Transitioning into a collaborative
mindset can be a complex process, requiring students to adapt their social
relationships, conceptual understandings and personal inclinations. This can
demand deliberate pedagogic actions that constructively align the graduate
attribute of collaboration with curricula design, teaching activities, learning
resources and assessment strategies (Biggs, 1996).
So how and why is collaboration relevant to professional contexts and tertiary

courses in software design and choreography? The teaching approaches to
collaboration in dance and engineering will inevitably vary across classrooms and
institutions, and our research does not seek to establish firm pedagogic categories
within software design and choreography. It is possible to note, however, that the
teaching practices that are described within this study are not random
illustrations of personal pedagogic preferences but fit within the predominant
discourses associated with teaching collaboration within these disciplines, which
we summarize below.
Software design involves the development of digital programs that provide a

service to the users of digital devices. While writing code is an integral part of
software design, software engineers also engage in non-coding tasks such as
gathering requirements from clients, brainstorming concepts, executing designs,
testing new programs and documenting processes. Due to the increasing
complexity of modern software, development requires the participation of
multiple individuals working together and consulting each other (Whitehead,
2007). The concept of collaboration is therefore not new to the profession of
software engineering, as the process of partnering in code writing and engaging
stakeholders in the design process has long been encouraged (Beck, 2000). Project
management systems such as Agile have evolved from software design
(Cockburn, 2006), promoting the use of collaboration within other project
management contexts, as a means of enhancing productivity, innovation and
social capital within the knowledge economy workplace (Barksdale &
McCrickard, 2012; Dawson & Jones, 2003).
The valuing of collaboration within the profession of software engineering has led
to a valuing of collaboration within tertiary education programmes that teach
software design. This has involved the integration of small-group tasks within the
assessments of software engineering degrees, to emulate scenarios that students
will encounter in professional life. Incorporating these group tasks into the

218
software design curriculum has increasingly involved the use of online

communication systems, which identify the quantity and quality of each
collaborator’s contribution (Soller, Martinez, Jermann & Muehlenbrock, 2005),
allowing teachers to monitor and measure free riding (Jermann & Dillenbourg,
2008). While a growth in digital tools have sought to support parity in assessment,
less scholarly attention has been given to the deliberate teaching of collaboration
in these classrooms (Hesse et al, 2015).
The discipline of choreography involves the design of dance movement for

presentation by dancers (Lavender & Predock-Linell, 2001), within dance contexts
that range from professional performances to more social/participatory events
(Nahachewsky, 1995). While the physical interaction of dance inevitably requires
social engagement, the choreographic process does not inherently involve
collaboration, and the creative leadership of choreography can range from
authoritarian to egalitarian (Butterworth, 2004). Within the 21st century however,
professional choreographers are increasingly required to engage in collaborative
processes with other choreographer/performers from diverse genres, cultural
backgrounds and physical capabilities, and artists from other disciplines, during
diverse phases of the creative production process (Foster-Sproull, 2017). With the
expansion of dance as a professional practice beyond concert hall contexts, the
development of choreographic competencies has come to include education,
social development and entrepreneurial applications of dance (Lehikoinen, 2018).
In tertiary degree programmes, choreography is taught as both a specific subject

and as an integral component of general courses on performance, exploring how
to improvise to generate or interpret movement (Rowe & Zeitner-Smith, 2011) and
pedagogy, exploring how to facilitate the creative practices of others (Lavender,
2009). Choreography is generally taught through experiential learning in studio
settings, where students review the dances and creative processes of others and
actively create their own dances (Van Dyke, 2005). This can involve collaborative
practices with other students to explore ideas, and develop group choreographic
work for assessment. Such learning generally involves a scaffolding of
collaborative processes, so that students are actively collaborating with each other
on small, non-assessed tasks in lessons before moving to assessed group work
(Buck & Rowe, 2015). These teaching practices have sought to emphasize the
importance of collaboration amongst students, as a means of engaging in social
animation (Thapalia, 1996) with diverse groups, cultures and body types (Amans,
2017).
Both software design and dance value collaboration as a graduate attribute in

alignment with wider trends in 21st century creative economies, yet our research
suggests that how tertiary educators approach the teaching of this graduate
attribute can be markedly different. Within the classrooms participating in our
study, collaborative innovation in software design is generally valued as a means
of enhancing productivity (Giacaman & Sinnen, 2018), whereas collaborative
innovation in choreography is generally valued as a means of enhancing self-
actualization, pluralism and inclusion (Buck & Snook, 2018). This opens questions
as to how these differing mandates may play out in the classroom. What do these

219
two cohorts of students feel about group-work in general? How do they

experience the teaching of collaboration? How do they feel that small-group
learning has helped them develop relevant, professional skills?
3. Methodology
To address these questions, our mixed-methods research engaged a grounded
theory approach, with an iterative, hermeneutical literature review into the
teaching of collaboration in higher education. This occurred in tandem with a
qualitative exploration (Denzin & Lincoln, 1998) of the teaching of collaboration
in Dance Studies and Software Engineering within our institute, including
classroom observations (Dingwall, 1997) and semi-structured interviews (Weiss,
1994) with 4 teachers in each discipline. This revealed significant differences in the
approach to teaching collaboration within our institution, which led us to posit
that how students are taught collaboration can impact on their experience of
learning about collaboration.
To investigate student perceptions of collaboration in coursework, we

subsequently developed an online survey, which included five closed-ended
demographic questions, 20 statements seeking agreement or disagreement along
a 5-point Likert Scale, and four open-ended questions seeking qualitative
commentary. The survey sought responses from students in the first four years of
degree study in Dance Studies and Software Engineering at the University of
Auckland, and data was gathered from approximately one-third of the students
enrolled in each of the disciplines (111 participants). The research was undertaken
under the auspices of the University of Auckland Human Participant Ethics
Committee. To maintain participant anonymity and mitigate coercion,
postgraduate research assistants were employed to interview the teachers,
administer the survey and collate the data through Qualtrics (Rowe, Martin, Buck
& Mabingo, 2020) for an application of the process and outcomes).
Within this article, we draw on the quantitative and qualitative data from the
interviews and survey to compare Software Engineering and Dance Studies
students’ perspectives on their experiences of learning about collaboration at
university. We have selected 15 of the questions most relevant to collaboration
and group-tasks and clustered these into four thematic sections based on A)
attitudes towards collaboration, B) the pedagogy of collaboration, C) perceived
learning about collaboration, and D) relevance of collaboration to discipline. The
quantitative results are presented in bar charts at the beginning of each section. A
5-point Likert Scale (5=Strongly Agree, 4=Agree, 3=Neutral, 2=Disagree and
1=Strongly Disagree) is used to map the range of responses to each question,
allowing for a two-tailed t-test to be performed comparing the two independent
samples. The resulting t-value is a standardized score representing the difference
between the two groups, while the p-value is the probability these results
occurred purely by chance. Through this process we sought to reveal contrasts
and similarities between students in Software Engineering and Dance Studies,
with the null hypothesis being that there are no differences in perspective between
the two groups. We extend our quantitative analysis of these points with a

220
qualitative analysis (Creswell, 2016) that draws on relevant quotations from the
teacher interviews and the open-ended qualitative commentary of the students.
4. Discussion
It is worth noting that there were significant demographic distinctions between
the two groups of participants. In the Choreography cohort, 78% identified as
female and 38% as ethnically Māori/Pasifika. In contrast, in the Software
Engineering cohort, only 23% identified as female and 3% as ethnically
Māori/Pasifika. As the following analysis reveals, these demographic and
disciplinary distinctions do not appear to transform into differences in attitude
towards collaboration. As evidenced in the first section, students in Dance Studies
and Software Engineering share a common feeling towards learning and working
in groups and the null hypothesis is retained. This supports our proposition that
demographic factors such as gender, cultural background or even chosen
discipline are not a significant factor in determining student attitudes towards
engaging in group work in class.
The null hypothesis is subsequently rejected, and greater distinctions emerge

between these two groups within the following sections however, which
examines how collaboration has been taught in each discipline, what students feel
they have learnt through group work, and how students subsequently value
group-work in their studies and profession. This supports our proposition that
students’ sense of learning about collaboration and their valuing of collaboration
as a professional skill can be impacted by how they experience learning about
collaboration within their discipline.
What does learning and working together feel like?

The only null hypothesis we could confidently reject in figure A was A1 (t= 2.3443,
p= 0.0209). This evidenced that dance studies students were less likely to dread
group learning activities than their software engineering peers. Both groups
showed little difference in opinion in relation to their preference of not working
on coursework alone (A2 t = 1.0293, p = 0.3056); to view group work as conducive
to a positive in-class environment (A4 t = 1.4056 p = 0.1627); and to view group
interactions positively (A5 t = 1.0247 p = 0.3078). Despite this, both groups’
responses were comparably negative for prompt A3 (t = -0.4068, p = 0.685). In this
regard, there was no statistically significant difference in both groups’ opinion in
viewing workloads as unevenly distributed in class.

221
Figure A: Learing and working together
This corresponded with the comments from teachers in both disciplines, and how
they perceived student attitudes towards group work. As a software design
teacher reflected, “I'd say in general they're quite keen and excited to be working
together.” Within our student commentary, both groups expressed the enjoyment
experienced working in groups, with numerous Software Engineering students
expressing a desire for “more of them!”. Their comments emphasized how
socialization can increase the enjoyment of a course, through the idea of having
“fun with friends” and the suggested equation “social aspect = more fun”.
While the socialized learning was enjoyed by both groups, half of the Software
Engineering students did not refuse the proposition that group learning and
projects gave rise to dread. This prompts questions as to what is it about group
learning that these students dread (especially given their more positive
expressions towards socialized learning activities), and how might this dread be
extended or mitigated by pedagogic actions. As both groups identified the uneven
distribution of workloads in group tasks, this may be a starting point for
understanding student apprehensions towards group work. A choreography
student expressed concerns over “people who slack off”, and a software design
student emphasized the need to “ensure people pull their weight”. This concern over
‘free riders’ (Fehr & Gachter, 2002) was a sentiment shared by students in both
groups, although amongst software design students there were comments
seeking punitive actions to “enforce participation” and “keep people more accountable
for their contributions”. A desire for external monitoring of contribution aligns with
the broader discourse in software design education literature, and the
development of extensive software programs for engineering courses that
measure and quantify an individual’s communications and contributions to
group projects (Jermann & Dillenbourg, 2008).
Activating people to effectively work together can be considered central to the

theme of teaching (and qualifying) students to collaborate. How teachers
rationalize student non-engagement in collaboration can, however, vary. A
software design teacher associated this ‘free-riding’ with student capability,
explaining how “Weak students could surf through the course with group work, because

222
sometimes the strong ones pull them through and many students will not feel comfortable
to complain.” By contrast, the choreography teachers tended to attribute student
non-contribution to the complexities of engaging in collaborative endeavour,
identifying how “all the social stuff combined with the creative stuff can cause a bit of
confusion and discomfort.”
This perceptual difference might be associated with notably distinct pedagogic

approaches. From our interviews with teachers and observations of classes in both
disciplines, two factors might be considered here: when group work takes place,
and how it is scaffolded.
Firstly, within the software design courses, the small group tasks took place
entirely outside scheduled lesson time. As a software teacher explained,
“…it's not like group work where we expect them to do the work in front
of us. … these assignments take a long amount of time and it's kind of
implied that they'll be doing that as their homework.”
This contrasted with the choreography course, in which small group tasks
occurred extensively during class time. As a choreography teacher described,
“most of what we do is small-group-based learning.” While experiential learning is an
established convention within the studio-based learning of dance, the progression
to more active learning environments is becoming increasingly promoted in
software engineering (Chen, Wang, Kinshuk & Chen, 2014). The idea that more
class-time should be dedicated to group tasks was a recurrent theme within the
comments offered by Software Engineering students, who sought, “more facilitated
time allocated for collaboration” “Less lectures, more meetings” “More in-course
opportunities for group work”. This aligns with research that affirms student interest
in more collaborative activities within class time (Freeman, Eddy, McDonough,
Smith, Okoroafor, Jordt & Wenderoth, 2014).
Secondly, within the choreography courses, these in-class tasks allowed students
to progress from smaller, unassessed group tasks to more complex, assessed
group tasks. A choreography teacher described how “…we are trying to just scaffold
them as much as we can, so they can build up to that.” Another choreography teacher
rationalized the importance of non-assessed group tasks, explaining that “the
whole premise behind that incremental development is to build confidence” in
collaboration. By contrast, within software design, group tasks were only
introduced as assessment items, usually halfway through the semester. This was
rationalized by a software design teacher as logistical issue within the time frame
of the institutional learning, explaining that “There’s no time to do group activities
that are not leading to a grade.” When an assessed group-activity is suddenly
introduced to an otherwise private learning experience however, students can feel
underprepared for both the social and cognitive challenges presented by the task
(Hesse et al, 2015). As a software student suggested,
“Start earlier in the degree. Start in the classroom, with smaller exercises
that aren't worth grades. Do it often. Build up a culture of working
together. Extend that out to assessments as soon as you can, trying to get
students to work together as often as it makes sense.”

223
From the two programmes studied within our research, the scaffolding of group
tasks and the incorporation of group activities into class time represent
significantly contrasting pedagogic strategies. Do these distinctions explain why
software students expressed a greater sense of dread towards group work? As the
following section reveals, there were also significant distinctions in how
collaboration was actually taught.
How was collaboration taught?

Figure B shows the results for prompts on the teaching of collaboration.
Statistically significant differences were evidenced in all figure B prompt results,
leading to our rejection of each null hypothesis—these results illustrating a
marked difference in positive responses in favor of choreography students.
Prompt B1 confirms choreography students were significantly more likely to
agree that teacher enthusiasm motivated their collaboration (t= 4.8248, p<0.0001).
This distinction is extended within B2 as choreography students more positively
identified the role of teachers in facilitating collaboration (t= 4.8248, p<0.0001).
Choreography students were also significantly more likely to agree that teachers
outlined effective theories and practices for collaboration (B3, t = 7.1950, p<0.0001)
and that course outline and materials outlined effective theories and practices for
collaboration (B4, t = 5.0956, p<0.0001). Relative to the software design student
group, the high t-values for both B3 and B4 indicate both a meaningful and
pronounced difference between the groups in their responses. These four points
suggest that students within choreography perceived a more deliberate teaching
of collaboration than software design, which supported our observations and
interviews with teachers.
Figure B: teaching collaboration
As teacher’s attitudes and behaviors towards a subject can significantly impact

student attitudes to the subject (Ađalsteinsdóttir, 2004), how the teacher expresses
enthusiasm for collaboration may spread to students. This might involve an
enthusiasm for ongoing processes of collaboration, that create, in the words of a
Dance Studies student, “a focused environment that urges us to keep going, where we
might've thought finished”. Another Dance Studies student expressed the
importance of the teacher’s energy in this process,

224
“I think group work can be improved by whoever facilitates the space and
allows the group work to happen. This can be done by the energy they set
from the get go.”
The concept of “allowing” the group work to happen is an interesting insight, in

that it acknowledges that this pedagogical attitude can involve a giving-of-
permission, rather than an establishing of boundaries and rules. In the context of
socializing classrooms, such a giving of permission can be a necessary step to
allow students to move away from a culture of privatized learning. This can
require, as one software design student suggested, activating socialization in the
classroom, through “more encouragement to meet new people, work with them”.
Central to this socialization is the way that groups are formed and how differences
are valued within the groups. A choreography teacher explains how and why she
purposefully constructs the groups based on differences:
“I try and really mix them up all the time because they will always want
to work with their friend… they're forced to figure it out and they don't
necessarily figure out their social stuff by talking about it but they figure
it out through the creative stuff, so just by doing the task.”
By contrast, in software design, a teacher explains how “They get told ‘get into pairs’
and they decide who they want to work with.” This was rationalised as a means of
avoiding student contentions over who they are placed with, but also a belief in
the value of similarities within small groups, as a software design teacher
rationalized, “The groups that take the task seriously often have likeminded individuals
together.” This familiarity lead to issues monitoring free-riders however, as a
software design teacher reflected, “They'll tend to buddy up with their friends and
sometimes it's very difficult for them to dob in their mates.”
From our observations of classes and interviews with teachers, the software
design student groups tended to be composed of more ethnically and culturally
homogenous groups than the choreography student groups. The value of teacher-
guided selection of heterogeneous groups is promoted by a student comment
from software design,
“Make sure the people working together are always changing. We need to
work with a diverse range of people, including differing backgrounds and
skill levels.”
Student comments from both groups emphasized a preference for the teacher’s
selection of groups, an idea supported by previous research (Hansen, 2006).
The teachers also presented very contrasting approaches to the management of

problems within the groups. As a choreography teacher explains,
“I'm always looking for social stuff, I'm always looking for the two in the
corner who are standing there kind of staring into space not looking at
each other with not much action going on and I'll go over and I'll try and
figure out what's happening, whether they just don't have the right
language or they don't know where to start work or they don't like each

225
other or what's going on for those groups that might be a bit

dysfunctional.”
This proactive engagement with student interactions was reiterated by all of the
dance teachers we interviewed. As a software design teachers suggests, this was
not perceived as a convention within software design teaching “I think it's fair to
say that I don't really follow the groups closely unless there is like, really serious
concerns”. This is connected to the practice of the group work happening
predominantly outside the classroom, as a software design teacher explains,
“it can be quite hard to sort of for us to up front know who is
communicating and who is not… in terms of keeping an eye on them all
I don't think a lot of that happens…. I would say most of the time where
there are issues, they tend to be brought to our attention late.”
That teachers might provide more active guidance in collaboration was repeatedly
expressed by software design students, who sought “more facilitation from lecturers
and guidance towards group work”. A Software Engineering student further explains
why such assistance is necessary at a tertiary level,
“I think, especially early on in our degrees, it’s worthwhile having staff
supervise group work. The level of professionalism required at a tertiary
level isn’t necessarily a skill built up in schools. There needs to be some
supervision when building a culture of collaboration, so it doesn’t spiral
away from being useful.”
The ways in which collaborative work was theorized was also distinct within the
course designs and teaching practices. Within software design, the theorization of
group tasks tended to focus on the management of the production process,
through reference to differing theories of project management such as Agile and
Waterfall (Balaji & Sundararajan Murugaiyan, 2014). A software teacher
explained, “They have a project briefing, but as a group they have to develop a plan how
they want to implement the task at a hand, I don’t tell them how to do it.” Another
software teacher expressed, “I didn’t really think about any differences about how
groups might work together.”
Within choreography, the theorization of group work tended to focus more on

creative deliberation, through reference to theories on choreographic decision-
making processes (Butterworth, 2004). In alignment with this, the choreography
teachers explained the importance of deliberately focusing on interpersonal skills.
As one teacher describes,
“the collaborative skills are things that you can learn to do, and if we can
break them down into some real life actions, not just thinking about them
hypothetically, we can know what the strategies are.”
These differing approaches to the theoretical support for group work appears to
have significantly impacted on how students perceived their learning about
collaboration. Software design students expressed numerous comments seeking
further readings and curricula resources to enhance their understandings of
collaborative interactions, including:
Structured guides on group work and interacting with others.

226
Education resources on how to better interact with partners such as examples, success
stories.
To develop shared understandings and expectations of effective collaboration

within the class, the software design students also emphasized their desire for the
teacher to provide:
More teaching on how to program in groups.
Better discussion for tactics on how to have an effective group.
More guidance towards group work so everyone is on the same page.
The responses from choreography students expressed a stronger sense that

guidance is currently provided, which aligned with our classroom observations
and interviews with teachers. The ways that the teacher takes responsibility for
both facilitating and educating about collaboration therefore appears to be
important and worthy of further consideration. These perceived differences in
teaching subsequently leads to significant differences regarding the students’
sense of learning achievement from group tasks.
What was learnt about group work through these tasks?

Figure C provides additional evidence for a difference of opinion between the
dance and software engineering students. The null hypothesis for C1 is rejected
(t= 4.7710, p < 0.0001), suggesting that choreography students are significantly
more likely to agree that group work has helped with their people skills. Similar
results were evident for prompts C2 and C3, rejecting the null hypotheses for both.
Choreography students were significantly more likely to agree that their
knowledge of how to participate in small-group collaborative tasks had expanded
(t= 4.8532, p < 0.00001) and that group work helped them understand course
content (t= 4.7545, p < 0.00001). All three prompts from figure C show that
choreography students held more positive learning outcomes regarding
collaboration than students in software design.
Figure C: Learning about group work
Extending upon these quantitative results, a choreography student further

commented that group work helped in “understanding how others work”. While the
value of observing others in a group was acknowledged as an effective means of
developing greater social sensitivity, some students also noted the skills gained
by responding to others in a group. A choreography student explained, “I enjoy
giving constructive feedback as well as developing materials as a group”. Such comments
reflect how the students gained a deeper sense of social regulation and

227
perspective taking (Oztop, Katsikopoulos & Gummerum, 2018), as suggested by

another choreography student, this involved a balance of “learning when to follow
and contribute”.
By contrast, the comments of the software design students reflected a belief in the
benefits of group tasks in terms of a division-of-labour:
Delegation of tasks for maximum competency.
Being able to specialize on particular tasks.
Workload is distributed, each person can focus on simpler aspects.
These comments suggest that group work processes were cooperative, with
individuals working separately and in parallel, rather than collaboratively. As the
students explain, this division of labour minimized effort and maximized
productivity.
The speed things can be completed.
Having to do less work.
The only comment from a choreography student relating to the scale and effort of
group-work challenged this assumption, suggesting “just because there is a bigger
group working on it doesn't mean it takes less time.”
This distinction in students’ perceptions of the learning value of group tasks may
relate to the teachers’ understandings of what is being evaluated through group
tasks. For the software design teachers interviewed, the function of group tasks
was predominantly framed in terms of the productivity, as one described how
“We tell them that the grade is about what at the end they produce.” At the same time,
software teachers identified problems that emerged from this focus on
productivity. A teacher recalled a “dysfunctional” group that was “kind of a
dictatorship” in which one individual did all the work and marginalized the other
group members. He describes how the final product “….was good in terms of
software implementation, but was very bad in terms of teamwork and learning about the
development process, which is an essential part of this course.” Another software
teacher noted how this focus on productivity meant that some students “not only
did not participate, but even did not have a good understanding of what the others had
one.”
By contrast, choreography teachers explained how the collaborative process was a

specific and important item within assessment. As a teacher describes,
“that's an objective within the assessment rubric…the criteria was based
on how they were discussing their idea, their level of organisation, their
ability to critically reflect upon the process and how that went.”
Through understanding what the students perceive that they are actually learning
regarding group work, and what teachers perceive that they are actually teaching,
further distinctions between the two programmes emerged regarding the
pedagogic approach to collaboration. These distinctions underpin the contrasting
survey results, and ultimately relate to how students perceive the relevance of
small-group tasks.

228
How relevant is learning and working together?

Figure D illustrates that while students from both groups think that small-group
task-based learning is relevant, students from dance were more likely to present
positive responses. We were unable to reject the null hypothesis for prompt D2 (t
= 1.1457, p = 0.2545) with this showing that both groups were similarly likely to
agree on the necessity of group learning to their respective disciplines. Stronger
contrasts emerged in the responses to D1 (t= 3.1696, p=0.002) and D3 (t= 3.3914,
p=0.001), both illustrating that choreography students were more likely to
perceive group tasks as good practice for professional life and necessary to their
learning. While both groups leaned towards a positive perspective, these p-values
evidence a difference of opinion between the groups, allowing us to reject the null
hypothesis in each case.
Figure D: Relevance of learning and working together
Despite reporting that group work did less to enhance their professional
interpersonal skills, software design students nevertheless emphasized the
significance of group course work as “very helpful for future industry work” and
“industry group tools”, emphasizing the belief that, “especially for Software
Engineering, group work and ability to work in a team are essential”.
Both groups presented comments that emphasized the instrumental function of

cooperative learning, as a means of enhancing knowledge in the discipline
(Nuthall, 1997). One dance studies student expressed, there is “more analysis of
things, talking about a concept or task” and another suggested “[i]t makes learning
more fun and easier. It also helps you get a better understanding of the tasks”. That
students can make valuable contributions to knowledge acquisition was
emphasized by a Dance Studies student, who wrote, “learning off each other is what
I value because your peers are your greatest teachers.”
These sentiments were shared by Software Engineering students, who valued

“learning new things from partner” and “learning new solutions to problems from other
team members and being able to teach things to other team members that they haven't seen
before.” This was seen as particularly relevant to enhancing comprehension, “I like
how people can build ideas off of each other… in terms of coding, we are able to help each
other if one person misses out on a certain aspect of the problem.”
One point for consideration from these comments might be that the comments by
Dance Studies students predominantly rationalized the importance of this group

229
learning in terms of how such tasks allowed them to gain more diverse
perspectives, rather than ‘right’ answers. As a choreography student expressed,
“Everyone’s ideas are different but still relevant to the task given. I also
like how people agree with what you say and can incorporate that within
other ideas that are said.”
While pluralism was also valued by the Software Engineering cohort, the students
emphasized how diversity can enhance the final product for assessment, rather
than how the diversity can more generally enrich students’ perspectives. A
software student quantified this as “The ability to use other's ideas. Double creativity.”
How students value group work might in this way be associated with how
teachers assess group work, with a focus on the product (rather than process)
leading to a valuing of group-work in terms of productivity, rather than pluralism
and innovation.
These distinctions are of significance to institutions that claim that (through

group-work) students are expanding not only their abilities, but also their
dispositions, to collaborate. So long as the value of group-work remains focused
on the efficacy of the productivity, an individual’s motivation to engage in group
work can remain limited to a sense of economic exchange (Hung et al, 2011).
5. Conclusion
We acknowledge that the research presented in the previous pages does not
comprehensively represent the diversity and complexity of pedagogical
approaches to Dance Studies and Software Engineering (in our university or in
tertiary education in general). This study is limited in both scope and scale,
conducted in a particular institutional context and within a limited timeframe.
Others who may engage in similar studies might discover very different insights.
We also acknowledge the logistical contrasts between teaching large student
cohorts within lecture halls and teaching smaller student cohorts in studios.
However, while the disciplines, cohorts and educational logistics present distinct
differences and limitations, we follow Biggs (1996) in arguing for a constructive
alignment of curriculum and pedagogic practices; the way we teach should be
guided by what are the required graduate outcomes, not by the traditions and
logistics of the institute.
Our mixed method approach therefore provides quantitative and qualitative data
that reveals that student interest in collaboration is not necessarily guided by their
disciplinary interest or demographic background. Such a methodological
approach has limitations in what can be explored with the data that is discovered
and the scope of one scholarly article, and future studies might dive further into
the qualitative or quantitative aspects of what this research opens as starting
points. Nevertheless, the data reveals that students who experience a more
deliberate teaching of collaboration, experience a greater sense of learning about
group work, and subsequently value an ability to work well in groups as a
relevant professional skill. This research therefore establishes a platform for
further evidenced-based research into how the teaching of collaboration may lead
to a shift in the experience of small-group learning tasks. Such research might
involve further analyses into how pedagogic practices can support or diminish

230
collaboration within groups, and how effective pedagogical approaches to

collaboration might be transferred across disciplines. Ultimately, this may
provide an effective higher education response to the demands of industry;
allowing graduates to enter the workplace with a strong capacity, and inclination,
to collaborate on creative endeavors.
6. References
Ađalsteinsdóttir, K. (2004). Teachers’ behaviour and practices in the
classroom. Scandinavian Journal of Educational Research, 48(1), 95-113.
https://doi.org/10.1080/0031383032000149869
Amans, D. (2017). An introduction to community dance practice. London, UK: Macmillan
International Higher Education.
Balaji, S., & Sundararajan Murugaiyan, M. (2012). Waterfall vs. V-Model vs. Agile: A
comparative study on SDLC. International Journal of Information Technology and
Business Management, 2(1), 26-30.
Barksdale, J. T., & McCrickard, D. S. (2012). Software product innovation in agile usability
teams: an analytical framework of social capital, network governance, and
usability knowledge management. International Journal of Agile and Extreme
Software Development, 1(1), 52-77. https://doi.org/10.1504/IJAESD.2012.048302
Beck, K. (2000). Extreme Programming Explained: Embrace Change. Boston, USA: Addison-
Wesley Longman.
Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education,
32(3), 347-364.
Boden, R., & Nedeva, M. (2010). Employing discourse: universities and graduate
‘employability’. Journal of Education Policy, 25(1), 37-54.
https://doi.org/10.1080/02680930903349489
Buck, R., & Rowe, N. (2015). “Threshold concepts, academic identity and arts curricula
design: Dr Who and Indiana Jones traverse learning landscapes for dance”. In M.
Fleming, L. Bresler & J. O Toole (Eds.), The Routledge International Handbook of the
Arts and Education (pp.170-184). London, UK: Routledge.
Buck, R., & Snook, B. (2018). Undisciplining arts education: Towards arts
integration. Journal of Artistic and Creative Education, 12(1), 21-40.
Burbach, M. E., Matkin, G. S., Gambrell, K. M., & Harding, H. E. (2010). The impact of
preparing faculty in the effective use of student teams. College Student Journal,
44(3), 752-761.
Butterworth, J. (2004). Teaching choreography in higher education: A process continuum
model. Research in Dance Education, 5(1), 45–67.
https://doi.org/10.1080/1464789042000190870
Chen, C., Wang, Y., Kinshuk, & Chen, N. (2014). Is FLIP enough? Or should we use the
FLIPPED model instead?. Computers & Education, 79(2014), 16-27.
https://doi.org/10.1016/j.compedu.2014.07.004
Cockburn, A. (2006). Agile software development: The cooperative game. London, UK: Pearson
Education.
Cohen, E. G. (1994). Restructuring the classroom: Conditions for productive small groups.
Review of Educational Research, 64, 1–35.
https://doi.org/10.3102%2F00346543064001001
Creswell, J. (2016). Reflections on the MMIRA the future of mixed methods task force
report. Journal of Mixed Methods Research, 10(3), 215–219.
https://doi.org/10.1177/1558689816650298

231
Csikszentmihalyi, M. (1996). Creativity: Flow and the psychology of discovery and invention.
New York, USA: Harper Collins.
Dawson, M. J., & Jones, M. L. (2003). Human change management: Herding cats. London, UK:
Price Waterhouse Cooper.
Denzin, N. K., & Lincoln, Y. S. (1998). Strategies of qualitative inquiry. Thousand Oaks, USA:
Sage.
Dillenbourg, P. (1999). Collaborative Learning: Cognitive and Computational Approaches.
Bingley, USA: Elsevier.
Dingwall, R. (1997). Accounts, interviews and observations. Context and method in
qualitative research, 51-65.
Fehr, E., & Gachter, S. (2002). Altruistic punishment in humans. Nature, 415(10 January
2002), 137-140.
Foster-Sproull, S. (2017). Embodied economies: Locating the ‘prosumer-dancer’ within dnacers’
experiences of choreographic practice (Unpublished Master’s of Dance Studies thesis).
University of Auckland, New Zealand.
Freeman, S., Eddy, S., McDonough, M., Smith, M., Okoroafor, N., Jordt, H., & Wenderoth,
M. (2014). Active learning increases student performance in science, engineering,
and mathematics. Proceedings of the National Academy of Sciences of the United States
of America, 111(23), 8410-8415. https://doi.org/10.1073/pnas.1319030111
Giacaman, N., & Sinnen, O. (2018). Preparing the software engineer for a modern multi-
core world. Journal of Parallel and Distributed Computing, 118(1), 247-263.
https://doi.org/10.1016/j.jpdc.2018.02.028
Griffin, P., & Care, E. (2015). Assessment and teaching of 21st century skills: Methods and
approach. New York, USA: Springer.
Hansen, R. (2006). Benefits and problems with student teams: Suggestions for improving
team projects. Journal of Education for Business, 82(1), 11-19.
https://doi.org/10.3200/JOEB.82.1.11-19
Hennessy, S., & Murphy, P. (1999). The potential for collaborative problem solving in
design and technology. International Journal of Technology and Design Education,
9(1), 1–36.
Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable
collaborative problem solving skills. In E. Care, P. Griffin, & M. Wilson
(Eds.), Assessment and teaching of 21st century skills (pp. 37-56). Dordrecht:
Springer.
Horder, J. (1995). Inter-professional education for primary health and community care:
Present state and future needs. In K. Soothill, L. Mackay & C. Webb (Eds.), Inter-
professional Relations in Health Care. London, UK: Edward Arnold.
Hung, S. Y., Durcikova, A., Lai, H. M., & Lin, W. M. (2011). The influence of intrinsic and
extrinsic motivation on individuals' knowledge sharing behavior. International
Journal of Human-Computer Studies, 69(6), 415-427.
https://doi.org/10.1016/j.ijhcs.2011.02.004
Jermann, P., & Dillenbourg, P. (2008). Group mirrors to support interaction regulation in
collaborative problem solving. Computers and Education, 51, 279–296.
Johnson, D., & Johnson, R. (2009). An educational psychology success story: Social
interdependence theory and cooperative learning. Educational Researcher, 38(5),
365-379. https://doi.org/10.3102/0013189X09339057
Johnson, R. B., & Onwuegbuzie, A. (2004). Mixed methods research: A research paradigm
whose time has come. Educational Researcher, 33(7), 14–26.
https://doi.org/10.3102/0013189X033007014

232
Lavender, L. (2009). Dialogical practices in teaching choreography. Dance Chronicle, 32(3),

377-411. https://doi.org/10.1080/01472520903276735
Lavender, L., & Predock-Linnell, J. (2001). From improvisation to choreography: The
critical bridge. Research in Dance Education, 2(2), 195-209.
https://doi.org/10.1080/14647890120100809
Lehikoinen, K. (2018). Setting the context: Expanding professionalism in the arts - A
paradigm shift. In B. Revelli & S. B Florander (Eds.), Careers in the Arts: Visions for
the Future. (pp. 16-30). Amsterdam, The Netherlands: ELIA.
Mattessich, P., & Monsey, B. (1992). Collaboration: what makes it work. A review of research
literature on factors influencing successful collaboration. St. Paul, USA: Amherst H.
Wilder Foundation.
McMahon, K., Ruggeri, A., Kammer, J. E., & Katsikopoulos, K. V. (2016). Beyond idea
generation: The power of groups in developing ideas. Creativity Research Journal,
28(3), 247-257. https://doi.org/10.1080/10400419.2016.1195637
Nahachewsky, A. (1995). Participatory and presentational dance as ethnochoreological
categories. Dance Research Journal, 27(1), 1-15. https://doi.org/10.2307/1478426
Nuthall, G. (1997). Understanding student thinking and learning in the classroom. In B.
Biddle, T. Good & I. Goodson (Eds.), The International Handbook of Teachers and
Teaching (pp. 681-768). Dortrecht, Germany: Kluwer.
Owen, D. (2015). Collaborative decision making. Decision Analysis, 12(1), 29-45.
https://doi.org/10.1287/deca.2014.0307
Rodgers, D. T. (1982). In search of progressivism. Reviews in American History, 10(4), 113–
132. https://doi.org/10.2307/2701822
Rowe, N. (2019). The great neoliberal hijack of collaboration: a critical history of group-
based learning in tertiary education. Higher Education Research & Development, 1-
14. https://doi.org/10.1080/07294360.2019.1693518
Rowe, N., Martin, R., Buck, R., & Mabingo, A. (2020). Teaching collaborative dexterity in
higher education: Threshold concepts for educators. Higher Education Research and
Development. https://doi.org/10.1080/07294360.2020.1833843
Rowe, N. & Zeitner-Smith, D. (2011). Teaching creative dexterity to dancers: critical
reflections on conservatory dance education in the UK, Denmark and New
Zealand. Research in Dance Education, 12(1), 42-51.
https://doi.org/10.1080/14647893.2011.556716
Soller, A., Martinez, A., Jermann, P., & Muehlenbrock, M. (2005). From mirroring to
guiding: A review of state of the art technology for supporting collaborative
learning. International Journal of Artificial Intelligence in Education, 15(4), 261-290.
https://telearn.archives-ouvertes.fr/hal-00197378/
Thapalia, C. F. (1996). Animation and leadership. In J. Servaes, T. L Jaconson, & S. A. White
(Eds.), Participatory Communication for Social Change. (pp. 150-161). New Delhi,
India: Sage.
Van Dyke, J. (2005). Teaching choreography: Beginning with craft. Journal of Dance
Education, 5(4), 116-124. https://doi.org/10.1080/15290824.2005.10387300
Weiss, R. S. (1994). Learning from strangers: The art and method of qualitative interview studies.
New York, USA: The Free Press.
Whitehead, J. (2007). Collaboration in software engineering: A roadmap. In 2007 Future of
Software Engineering. (pp. 214–225). IEEE Computer Society.
https://doi.org/10.1109/FOSE.2007.4

233

Vol. 19, No. 10, pp. 233-247, October 2020
Instructional Leadership and Students Academic

Performance: Mediating Effects of Teacher’s
Organizational Commitment
Adeel Ahmed Khan, Soaib Bin Asimiran,

Suhaida Abdul Kadir and Siti Noormi Alias
UPM Malaysia
https://orcid.org/0000-0001-6145-5760
https://orcid.org/0000-0002-2981-9509
https://orcid.org/0000-0002-2761-6265
https://orcid.org/0000-0002-4324-182X
Batool Atta
UAJK Muzaffarabad
https://orcid.org/0000-0002-0601-3983
Bukar Ali Bularafa

CABS Nigeria
https://orcid.org/0000-0002-7068-8600
Masood Ur Rehman
https://orcid.org/0000-0002-1713-7244
Abstract. This study investigates the mediating effects of teacher

organizational commitment on the relationship between principal's
instructional leadership and students' academic performance. Principals'
effective leadership performance is assumed to be responsible for
students' academic performance. However, many findings from the
literature indicate no significant direct effect between instructional
leadership and students' academic performance. Therefore, the current
study hypothesized that principals' instructional leadership indirectly
enhances students' academic performances through teacher
organizational commitment. Data obtained from 440 primary school
teachers from the state of Azad Jammu & Kashmir of Pakistan and pre-
existing school academic performance data obtained from each school
participated. The data were analyzed using structural equation modeling,
employing AMOS. The study's findings indicate that there is an indirect
positive effect of principals' instructional leadership and students'
academic performance through teacher's organizational commitment.
Hence, schools with higher instructional leadership levels had better
teachers, commitment to school, teaching work, teaching occupation,
workgroup, and higher student's academic performances. Therefore,

234
increasing the instructional leadership practices in schools through

teacher organizational commitment contributes to Pakistan's overall
student academic performances. Therefore, it is recommended to use
instructional leadership components in the curriculum that effectively
train new principals.
Keywords: Instructional leadership; Primary school; Principal; Students’

academic performance; Teacher organizational commitment
1. Introduction
Educationists, mentors, and investigators have long been concerned about finding
factors contributing to learners' performance. In order to produce a
knowledgeable nation, the better factor to be confronted is the student's academic
performances and different factors that affect a student's academic performances
(Wahlstrom, Thomas, Leithwood, & Anderson, 2010). State-funded schools
always confronted with challenges of improving student academic performances
in Pakistan (Hayat, Nisar, Sajjad, & Abbas, 2018). The documented investigation
about the role of factors established back in the seventeenth century. Researchers
indicated that along with classroom instruction, an essential element in students'
academic performance, leadership contributes better than any other factor
(Leithwood, Louis, Anderson & Wahlstrom, 2004). Several researchers have
echoed this idea stating the importance of leadership, particularly instructional
leadership in an educational institution (Louis, Murphy & Smylie, 2016).
Excellence is determined predominantly on the way schools are managed more

than on the significant number of existing resources. Proficiency in schools and
expansion of teaching and learning is primarily influenced by the quality of
leadership delivered by the head teacher (Louis et al., 2016). Principals are
supposed to take responsibility for setting targets, guaranteeing coordination and
team spirit, managing the organization established on shared values, creating
opportunities, demonstrating practices, and supervising (McLeskey, Billingsley &
Waldron, 2016). Researchers concluded that one of the leadership styles that has
garnered interest in the principal job and its different dimensions to improve
students' academic performance is instructional leadership (Hallinger,
Hosseingholizadeh, Hashemi & Kouhsari, 2018). Instructional leadership
investigators agree that teaching and achievement are the core of instructional
leadership (Boyce & Bowers, 2018).
Alongside principals, teachers are the prevailing variable correlational to enhance

students' academic performances. The ever-salient process of transforming
teaching practice and especially in low performing schools are correlational to
students' academic performances (Hines, Moore, Mayes, Harris, Vega, Robinson
& Jackson, 2017). Educators have to undergo the various opinions of those
assessing whether they can positively influence students and their academic
progress. Conversely, what may be more critical is how teachers evaluate
themselves regarding their talent to affect student academic performance.
Likewise, achieving school objectives depends on a considerable part of the
enhanced comprehension of the sources, nature, and increase of a teacher's
organizational commitment (Tentama & Pranungsari, 2016). Awareness of

235
teachers' level of commitment is vital because it reveals their understanding of

how captivating and meaningful their work experiences (Zhang & Jing, 2016).
Empirical studies have shown a strong correlation among instructional
leadership, teachers, and students' academic performances (Bush, 2017; Paletta,
Alivernin & Manganelli 2017). Many studies indicated the inconsistency of these
variables in size and direction. Thus, it is essential to observe the relationship
between principal instructional leadership, teacher's organizational commitment,
and students' performances.
In this study the researcher has re-analyzed formerly reported databases to test
numerous models associating instructional leadership with student academic
performance. Hence, the researchers focused on the mediating effects of teacher
organizational commitment as a mediating variable that has not been earlier
perceived as a source of indirect effect on the relationship between instructional
leadership and students' academic performance. The researchers investigated by
assembling what was observed as a model and then established numerous
variants of it. The researchers anticipated that instructional leadership impacts
student academic performance by enhancing the organization's capacity in terms
of a teacher by their commitment to the organization's goals. In the current study,
the researchers expanded the model to inspect the indirect effects of instructional
leadership on student academic performance, using earlier research on
instructional leadership and school improvement to hypothesis routes structured
into the model.
Researchers have long been concerned in determining factors contributing
commendably for the excellence of learner's performance. There are some internal
factors in schools and also some external that affect students' academic
performance. Studies of effective schools have concluded that school leaders and
teachers affect student's academic performances more than other factors
(Gannouni & Ramboarison-Lalao, 2018). Theoretically, a trustworthy and robust
combination of school leaders and teachers can increase student academic
performance. From the last many years' number of different innovative
benchmarks and milestones in studying educational leadership have been made,
but one of the most prominent with a high rate of empirical studies focused on it
is instructional leadership (Hallinger et al., 2018). Difficulties of restructuring
educational institutes have been noted as reasons for supporting Instructional
leadership in schools (Kulophas & Hallinger, 2019). Instructional leadership
progressed substantially across the past number of years in Western countries and
evident higher students' academic performances (Hallinger et al., 2018;
Leithwood, Patten & Jantzi, 2010).
Empirical investigations of instructional leadership have likewise started to

appear in the emerging societies of Asia as well (Hallinger et al., 2018).
Instructional leadership is a multifaceted progression, which varies across
settings, based on individual style, school background, and components. Based
on local empirical research, most of the principals in Pakistan had insufficient time
to provide innovation to the school due to their administrative duties and day to
day matters (Adeel, Soaib, Suhaida & Ramli (2020); Nasreen & Odhiambo, 2018).

236
Academic work has progressed in determining leadership impacts on student

academic performances, but the literature is distributed on leadership's direct or
indirect effects on student academic achievements. The recent research indicated
that correlation among principals' direct leadership and student academic
performances is little (DiPaola & Hoy, 2015). Research emphasizes indirect
leadership impacts and such focus is heavily mediated by external variables like
the dynamism of relationship and connections between leaders and the
subordinates, teacher's organizational commitment, and other factors (Sebastian
& Allensworth, 2012). One such influential aspect that is highly effective is the
teacher's organizational commitment. Ross and Gray (2006) verified a model
postulating that principals can enhance student performances indirectly through
teacher commitment and beliefs. It is associated with many encouraging
outcomes, comprising negligible absenteeism, work engagement, and greater job
contentment (Hallinger et al., 2018). Research has revealed that teacher
commitment has a positive impact on student academic performance (Xiao &
Wilkins, 2015).
Improving the visibility of classroom implementation through teachers

undoubtedly associated with such benefits as better-quality instruction, enhanced
teacher commitment, and developed teacher attitudes toward professional
development (Hallinger et al., 2018). Many researchers have recognized the same
inferences for principals who used to be in classrooms and form instructional
competence through comprehensive feedback (Thoonen et al., 2011). However,
this method needs the principal in numerous classrooms maximum times, which
rapidly converts into an incontrollable assignment. Therefore, a core issue for
leadership is the enacting of potent leadership mediators that offer modest
guidance to practising leaders, so they would be more focused on their efforts that
will affect classroom practices and student learning to foster student's academic
performance.
3. Model
Instructional leadership was anticipated for this study for the reason that it is
harmonious with broadly based tendencies of teacher authorization, various
stakeholders’ contributions to school improvements. Also, substantial evidence
exists that revealed teacher’s organizational commitment has a positive impact on
student academic performance (Xiao & Wilkins, 2015). Improving the visibility of
classroom implementation through teachers undoubtedly associated with such
benefits as better-quality instruction, enhanced teacher organizational
commitment and developed teacher attitudes toward professional development
(Hallinger et al., 2018; Azodi, 2006).

237
Figure 1. Hypothesized model linking Instructional leadership to student academic

performance through teacher organizational commitment.
The model estimates that instructional leadership will affect teachers'

organizational commitment delineated here as a commitment to school, teaching
work, teaching occupation and workgroup. Zikhali and Perumal (2016)
discovered from their empirical statistics that principal's instructional leadership
practices had an encouraging influence on employees' commitment level in
schools. A study conducted by (Ail, Taib, Jaafar, Salleh & Omar, 2015) using
quantitative approaches explored instructional leadership and teachers'
organizational commitment in Pahang Malaysia. The results displayed a
significant relationship between the two variables. Concerning the implications of
the study, the principal must practice his aptitude in instructional leadership
expertise to nurture teachers' organizational commitment.
In Figure 1 above, four dimensions of organizational commitment were observed
thus; commitment to school, commitment to teaching work, commitment to
teaching occupation and commitment to the workgroup. The Figure further,
recommends that all the dimensions of instructional leadership will contribute to
each of the dimensions.
3.1 Path from organizational commitment to student’s academic performance

It is anticipated that learner is a vital component of the educational process.
Conferring to Tentama and Pranungsari, (2016) effective teachers require a high
level of organizational commitment. It ultimately advocates that a high level of
student accomplishment needs devoted teachers who contribute commendably.
There is no uncertainty that the high level of student attainments is intensely

238
associated with a high level of organizational commitment. A study conducted by

Avalos, (2011) found a positive relationship between teacher organizational
commitment and students’ academic performances. Also, in a study piloted by
Joffres and Haughey, (2001) recommended that student performances, mainly in
mathematics, are associated with specific components of communal schools.
Students’ performances in school appeared to be correlated to teacher
commitment, cooperation, and concentration on students.
4. Method
In its basic framework, this study is quantitative research with a descriptive
correlational research design. The researchers requested all primary teachers in
the Muzaffarabad Division (three districts) of Azad Jammu and Kashmir Pakistan
to participate. Schools were engaged if at least five teachers’ responses were
received from a school. Keeping in view the criteria we received responses of at
least five teachers from (N = 79 schools; 440 teachers) out of total 217 schools and
1301 total teachers. To fulfill ethical considerations a consent letter was given to
every participant to read and understand his/her right before participating in the
study.
4.1 Sampling
For the present study cluster and simple random sampling was used to select an
appropriate sample for the study. Campbell and Stanley (2015) identified that
random selection of between 4 to 10 respondents in institute sufficiently
represents the frame of mind and insights of the whole school. Accordingly, the
researcher tried to get at least 5 from each school. To ensure the adequacy of the
sample, a sample is chosen by noting the utmost quoted rules of thumb contained
by multivariate analysis and the needs of data analysis using Structural Equation
Modelling (SEM) with Analysis of Moment Structure (AMOS).
4.2 Instruments
The instruments for the study was adopted from prior studies (Celep, 2000;
Hallinger & Murphy, 1985; Hallinger, 2013). Instructional leadership involved 50
items in determining teacher observations that their principal leads by defining
the school mission, managing the instructional program and developing the
learning climate of the organization and motivates its members to adjust to the
demands of a competitive environment. Teacher commitment to an organization
comprised of four variables: commitment to school, commitment to teaching
work, commitment to teaching occupation and commitment to the workgroup.
All dimensions consisted of five (05) items, each a total of 20 items to measure the
overall organizational commitment of the teachers. Likert Scale with a 5-point
scale (from 1 strongly Disagree to 5 Strongly Agree) were used to measure
respondent’s responses. The adequacy of the variables was tested with
confirmatory factor analysis. Teachers completed the survey in November 2019.
In this study student’s academic performance characterized by the overall
performance in each subject taken by the grade 5th students in a standardized
examination across the state and culminated in a Grade Point Average (GPA). The
GPA score was taken into account student’s academic performance in course

239
work and examinations. The example method of calculating the student’s

academic performances is as shown below:
GPA = Sum of (number of grade *grade point) /Total number of the student
taking the subjects. The minimum and maximum GPA scores are 1.0 (Grade A)
and 5.0 (Grade E) respectively. This GPA score infers that the lower the score, the
better the students had performed academically.
Table 1 describes the variables. All were reliable (alphas ranged from .84 to .93).
Student academic performances correlated with all variables in the model.
Table 1: Results of Individual Reliability Test of the Constructs in the Instruments

Variables Variable Items Cronbach’s Cronbach’s
Alpha(Pilot Alpha(Main
Study) Study)
FSG 5 0.934 0.911
CSG 5 0.702 0.867
SEI 5 0.867 0.846
Principal CTC 5 0.913 0.927
Instructional
MSP 5 0.874 0.932
leadership
PIT 5 0.902 0.894
PPD 5 0.907 0.913
MHV 5 0.892 0.925
PIFT 5 0.898 0.935
PIFL 5 0.926 0.933
________
Techer’s CTS 5 0.802 0.866
Organizational CTW 5 0.842 0.876
Commitment CTO 5 0.894 0.885
CTWG 5 0.899 0.895
The study has been subjected for normality test, the skew of all variables and items
were laid between ±3 while the kurtosis was laid at ±7 (Byrne, 2013). Such reading
can be interpreted as the data set of all items having normal distribution and thus,
can be considered as well-modeled. Specifically, the skew and kurtosis readings
can be seen within the range of -.299 to -.208 and -1.534 to -1.433 respectively.
5. Results
The process of determining appropriate methods in analyzing data was based on
the hypothesis of the study. Statistical Package for Social Sciences (IBM SPSS)
version 23 and the Analysis of Moment Structures (AMOS) version 23-based SEM
was utilized to analyze the data obtained in this study. Furthermore, the study of
discriminant validity and convergent validity through CFA and AMOS was used
to validate the model of measurement. Conducting path analysis, the AMOS
software was used again to examine the research hypotheses and construct the
structural model. The SPSS was used to detect univariate outliers and conduct the
frequency analysis (i.e., sample profile), descriptive analysis and internal
reliability/Cronbach Alpha.

240
The path statistics displayed in Figure 2 are standardized regression weights.

Figure 2 offers support for the indirect effects model of principal assistance to
student's academic performance. The figure indicates that principals who
implement instructional leadership styles contribute to teachers' organizational.
There was an indirect effect of instructional leadership on student's academic
performance. The path coefficients and the results of examining hypothesized
mediating effects are presented in Table 3.
Figure 2. Base model linking instructional leadership to student academic

performance through teacher organizational commitment
Table 2: Summaries of the Goodness-of-fit Indices of the Overall Measurement Model

CMIN DF CMIN/DF RMSEA AGFI GFI CFI NFI TLI
23.849 18 1.325 .035 .957 .978 .998 .991 .997
Each of the criteria for the goodness of fit was met. (Table 2 displays the
summaries of the goodness of fit statistics for the model, for the exploration and
validation samples).
Table 3: Results of Examining Mediation Effect of (TOC) on the Relationship between

(PIL) and (SAP)
95% Confidence
BC CI
Model /Hypothesized Path Beta P LB UB
Direct Model
PIL → SAP .550 .000
Full Mediation
PIL → SAP .103 .097
Std. Indirect Effect (SIE) .447 .000 .035 .560
The result showed that there is a significant relationship between Principal

Instructional Leadership (PIL) and Student Academic Performance (SAP) in the

241
absence of Teacher Organizational Commitment (TOC), by way of the

standardized total effect of .550 and P-value of 0.000. Therefore, the total effect of
Principal Instructional Leadership (PIL) as IV on Student Academic Performance
(SAP) as DV without the inclusion of Teacher Organizational Commitment (TOC)
as M was statistically significant.
This relation remains insignificant after the inclusion of Teacher Organizational

Commitment (TOC) into the model, with a standardized direct effect of 0.103 and
a P-value of 0.097. Hence, the direct effect of Principal Instructional Leadership
(PIL) as IV on Student Academic Performance (SAP) as DV with the inclusion of
Teacher Organizational Commitment (TOC) as M was statistically not significant.
Further, the results revealed that Principal Instructional Leadership (PIL) had a
full mediation effect on Student Academic Performance (SAP) through Teacher
Organizational Commitment (TOC) with the standardized indirect effect of 0.447
and P-value of 0.000 with LB .035 and UB is .560 and degree of mediation of
teacher organizational commitment was full mediation.
6. Discussion
The results of the mediation analysis indicated that Teacher Organizational
Commitment mediated the effects of Principal Instructional Leadership on
Student’s Academic Performance, i.e. full mediation. The results indicated that
principal instructional leadership had a significant indirect positive effect on
student’s academic performance through teacher organizational commitment
with standardized indirect effect (SIE) of 0.447 and P-value of 0.000. These
findings remained consistent with previous research that has explored direct
effects of leadership on student learning outcomes has stated weak effects, while
research that has incorporated mediating variables has reported significant effects
(Leithwood & Jantzi, 1999). Principals in this data analysis observed by their
teachers as a practitioner of instructional leadership and enhance teamwork and
commitment among teachers that improve Teacher’s organizational commitment
to generate new idea and knowledge, in turn, promotes student’s academic
performance.
Principals generally have potent effects on school processes than on student’s

academic performance, and statistically significant assistances to academic
performances dependent on indirect effects have been demonstrated. These
findings advocate that principals should exhibit instructional leadership practices
as this expertise help to develop unity of vision and mission through the
commitment and teamwork amongst teachers. Instructional leadership practices
can bring teachers out of isolation and lead towards a commitment to teaching
occupation for improvement. Principals can improve cooperation, production,
and commitment by encouraging strong organizational commitment that
emphasis improving the school environment and ultimately, student’s academic
performances. These findings are consistent with the findings of past studies.
Empirical studies have discovered that school principal’s work as instructional
leaders has been associated indirectly with student’s academic performances
(Alig-Mielcarek, 2003; Hallinger, 2003). Alig-Mielcarek (2003) proposed that
performing the dimensions of instructional leadership offers good foundations for

242
creating a climate that presses for academic consistency. On the other hand,
principals will also require to offer resource support, through monitoring and
informal discussions, that teachers have resources and classroom materials to
teach curriculum efficiently.
7. Implications
7.1 Theoretical Implications
The focus area of this study, student’s academic performance, is one of the
educational development agendas of societies, regions, and nations as a whole.
Therefore, the main contribution of this study concerns empirical literature by
modifying and testing a framework that improves our knowledge, which
indicates that instructional leadership and teacher organizational commitment are
indeed antecedents to students’ academic performances in the context of Pakistan.
This study fills the existing gap in much western context literature on instructional
leadership, teacher organizational commitment and students’ academic
performances, and it has proven that teacher organizational commitment is a
valid mediator in this relationship. Hence, no doubt this study extends the
Hallinger and Murphy, (1985) instructional leadership model and Celep (2000)
model of organizational commitment.
The findings of the current study also underscore the importance of respondent’s
demographic background (such as teachers experience, years of service with the
current principal), also finding indicates that these demographic factors
influenced the perceptions of instructional leadership, teacher organizational
commitment and students’ academic performances, which then tried to explain
the Pakistani level of response to the theories and models.
7.2 Practical Implications

As contained in educational reform there is an amplified emphasis on holding
schools responsible for all student performances at all levels. Principals and
teachers are under huge pressure to nurture student’s performance scores that
indicate school academic outcomes. Nevertheless, findings of this study have
provided evidence base information that may be useful to assist practitioners and
administrators to develop and manage policy directions regarding human
resource management in the institutions. Also to regulate policies and practices
that would have a positive impact on instructional leadership, teacher
organizational commitment and students’ academic performances such that
principals, teachers, policymakers and stakeholders can make transformations to
meet these challenges. By displaying behaviors consistent with instructional
leadership through developing a positive organizational commitment will
increase student academic performances.
Accountability is now filtering down to individual schools and classroom levels.

Leading a school to meet these demanding standards is becoming imperative, not
an exception. Leaders in schools need tangible activities they can implement to
nurture a climate for accomplishments to flourish. This study offers principals
with numerous suggestions for implementing instructional leadership behaviors
and building a strong commitment that can establish a vision of academic success

243
for all students. Principals may start, in association with staff, students and
community, to cultivate and communicate common goals (Murphy, 1990).
Practitioners may offer mastery experiences for teachers by working together with
them to develop school goals and curriculum; by ensuring the accessibility of
instructional resources and by giving ample preparation time with associates.
Instructional leaders may sort out vicarious experiences for their resources by
displaying instructional strategies for educators who have had challenges
enhancing student academic performance. Principals can also provide mediated
experiences by targeting teachers with low levels of commitment to learn and
observe an exemplary model of classroom instruction devised by the lead teachers
and administrators. The present study has successfully developed a sound
operational model and it is hoped to address the problem of students’ academic
performances by using a new model on the relationship between instructional
leadership, teacher organizational commitment, and students’ academic
performances. Principal working towards improving instructional leadership
skills in managing instructional programs might improve their organizational
commitment and could, in turn, possibly improve student’s academic
performances (Blasé & Blasé, 2000).
8. Limitations
The study is not an exception like other research has some limitations the first
involved the data collection method. Methodologically, restraint to quantitative
research infers that relationships among leaders and other factors can be
documented but cannot be completely understood.
The second limitation of the study researcher concentrated solely on primary
schools situated in a single state of Pakistan. Because of reduced size and
complication, primary schools are generally considered as a more encouraging
environment for instructional leadership than other school levels (Bellibas, Bulut,
Hallinger & Wang, 2016). Thus, it is possible that to some extent, different patterns
might be established in other schools levels or primary schools sited somewhere
else in Pakistan. The third limitation is to quantify Students' Academic
Performances (standardized examination across the whole state of grade V). To
determine comprehensive academic performances, it would be required to
administer several tests during a student's academic career. The fourth limitation
is the instruments that were used in the study are from the western country, which
is already established. The researcher had minimal time to adopt and adapt a few
tools and develop a new instrument that will suit with Pakistani context.
9. Conclusion
The purpose of the article is to investigate the mediating effect of teacher
organizational commitment on the relationship between principal’s instructional
leadership and students’ academic performance. The article made a valid
hypothesis – Principal’s instructional leadership enhances students’ academic
performance indirectly through teacher organizational commitment. For the
study, cluster and simple random sampling were used to select the samples. The
sample data from 440 primary school teachers have been collected. The data has
been analyzed through structural equation modelling, employing AMOS.

244
The findings of the study appear very useful to the academic world. It is proved
that there is an indirect positive effect of principal’s instructional leadership (IL)
on students’ academic performance (SAP) through teachers’ organizational
commitment (TOC). It is also proved that teacher organizational commitment is
a valid mediator in the relationship between instructional leadership and
students’ academic performance. The implication is that with higher levels of
instructional leadership have better teachers commitment to schools, pedagogy
works, and higher students’ academic performance. Hence investing in
instructional leadership (IL) practices in schools through teacher organizational
commitment makes a vital contribution to overall student performances.
The study makes some practical insights as there is pressure on schools from
parents and society to enhance the student performances. The study also makes
a scope for further research using different measurements of academic
performances. The instructional leadership components can be probed.
10. Recommendations
The current study promoted research about principal leadership by the
contribution of an effective and reliable path that affects a student’s academic
performance. The student’s academic performance model hypothesized in this
study was supported by structural equation modelling. Through the exercise of
monitoring the teaching and learning process, principals may work with teachers
to categorize professional advancement needs. This study indicated that
principals who promote professional development impacted academic
performance. Base on the findings and their interpretations, the current study
recommends the following for future research:
1) Future research should be conducted using different measurements of
academic performances.
2) Future study should do a case study using mixed-methods that gathers insight
from the principals on how they perceive their instructional leadership behavior
effects on student’s academic performance.
3) Future research should examine the use of instructional leadership components
in the curriculum that effectively train new principals.
4) Future studies should be conducted to replicate the current study using
different instruments for instructional leadership and teacher organizational
commitment.
11. References
Avalos, B. (2011). Teacher professional development in teaching and teacher education
over ten years. Teaching and Teacher Education, 27(1), 10–20.
https://doi.org/10.1016/j.tate.2010.08.007
Azodi, D. S. (2006). Principal leadership, trust, and teacher efficacy (Doctoral dissertation, Sam
Houston State University) Dissertations and Theses database. (UMI No. 3250639).
Alig-Mielcarek, J. M. (2003). A model of school success: Instructional leadership, academic
press, and student achievement (Doctoral dissertation, The Ohio State University).
Ail, N. M. B. M., Taib, M. R. bin, Jaafar, H. bt, Salleh, W. A. R. bt M., & Omar, M. N. bin.
(2015). Principals’ Instructional leadership and teachers’ commitment in three
Mara junior science colleges (Mjsc) in Pahang, Malaysia. Procedia - Social and
Behavioral Sciences, 191(2010), 1848–1853.
https://doi.org/10.1016/j.sbspro.2015.04.512

245
Adeel, A. K., Soaib, A., Suhaida, A. K., & Ramli, B. (2020). Principals' instructional
leadership practices in Pakistan elementary schools: Perceptions and
implications. Universal Journal of Educational Research, 8(1A), 16-23.
https://doi.org/10.13189/ujer.2020.081303.
Bellibas, M. S., Bulut, O., Hallinger, P., & Wang, W. C. (2016). Developing a validated
instructional leadership profile of Turkish primary school principals. International
Journal of Educational Research, 75, 115–133.
https://doi.org/10.1016/j.ijer.2015.10.002
Byrne, B. M. (2013). Structural Equation Modeling With AMOS. Structural Equation Modeling
With AMOS. https://doi.org/10.4324/9781410600219
Boyce, J., & Bowers, A. J. (2018). Toward an evolving conceptualization of instructional
leadership as leadership for learning: Meta-narrative review of 109 quantitative
studies across 25 years. Journal of Educational Administration, 56(2), 1–20.
https://doi.org/10.1108/JEA-06-2016-0064
Bush, T. (2017). School leadership and student outcomes in Asia: What makes the
difference? Educational Management Administration and Leadership, 45(2), 193–195.
https://doi.org/10.1177%2F1741143216686723
Blasé, J., & Blasé, J. (2000). Effective instructional leadership. Journal of Educational
Administration, 38(2), 130–141. https://doi.org/10.1108/09578230010320082
Celep. (2000). Teachers’ Organizational commitment in educational organizations. 2000-
00-00. National FORUM of Teacher Education Journal, 10E(3), 1999–2000. Retrieved
from http://www.nationalforum.com/17celep.htm.
DiPaola, M., & Hoy, W. K. (Eds.). (2015). Leadership and school quality. IAP.
Gannouni, K., & Ramboarison-Lalao, L. (2018). Leadership and students’ academic
success: Mediating effects of self-efficacy and self-determination. International
Journal of Leadership in Education, 21(1), 66-79.
https://doi.org/10.1080/13603124.2015.1123300
Hallinger, P. (2013). Measurement properties of the principal instructional management
rating scale: Technical Report Ver, 4.5, 114.
https://doi.org/10.1177/0013161X12468149
Hallinger, P., Adams, D., Harris, A., & Suzette Jones, M. (2018). Review of conceptual
models and methodologies in research on principal instructional leadership in
Malaysia. Journal of Educational Administration, 56(1), 104–126.
https://doi.org/10.1108/jea-03-2017-0025
Hallinger, P., Hosseingholizadeh, R., Hashemi, N., & Kouhsari, M. (2018). Do beliefs make
a difference? Exploring how principal self-efficacy and instructional leadership
impact teacher efficacy and commitment in Iran. Educational Management
Administration and Leadership, 46(5), 800–819.
https://doi.org/10.1177/1741143217700283
Hallinger, P. (2003). Leading educational change: Reflections on the practice of
instructional and transformational leadership. Cambridge Journal of Education,
33(3), 329–351. https://doi.org/10.1080/0305764032000122005
Hallinger, P., & Murphy, J. (1985). Assessing the Instructional Management Behavior of
Principals. The Elementary School Journal, 86(2), 217–247.
https://doi.org/10.1086/461445
Hines, E. M., Moore, J. L., Mayes, R. D., Harris, P. C., Vega, D., Robinson, D. V., Jackson,
C. E. (2017). Making Student Achievement a Priority. Urban Education,
004208591668576. https://doi.org/10.1177/0042085916685761
Hayat, A., Nisar ul Haq, M., Muhammad Sajjad, H., Abbas, Z., & Raza, S. (2018).
Assessment of Students’ Academic Performance in Government Schools of
Pakistan. European Online Journal of Natural and Social Sciences: Proceedings, 7(3).

246
Joffres, C., & Haughey, M. (2001). Elementary teachers ’ commitment declines :

antecedents , processes , and outcomes. The Qualitative Repart, 6(1), 1–22.
Kulophas, D., & Hallinger, P. (2019). Leading when the mouth and heart are in unison: a
case study of authentic school leadership in Thailand. International Journal of
Leadership in Education, 1-12. https://doi.org/10.1080/13603124.2019.1591519
Leithwood, K., Patten, S., & Jantzi, D. (2010). Testing a conception of how school
leadership influences student learning. Educational administration quarterly, 46(5),
671-706. https://doi.org/10.1177/0013161X10377347
Leithwood, K. A., Louis, S. K., Anderson, S., & Wahlstrom, K. (2004). How leadership
influences student learning: A review of research from the learning for leadership
project. The Wallace Foundation, 1–90. https://doi.org/10.1007/978-90-481-2660-6
Leithwood, K., & Jantzi, D. (1999). The relative effects of principal and teacher sources of
leadership on student engagement with school. Educational administration
quarterly, 35(5), 679-706. https://doi.org/10.1177%2F0013161X99355002
Louis, K. S., Murphy, J., & Smylie, M. (2016). Caring leadership in schools: Findings from
exploratory analyses. Educational Administration Quarterly, 52(2), 310–348.
https://doi.org/10.1177/0013161X15627678
Murphy, J. (1990). Principal instructional leadership. Advances in educational administration:
Changing perspectives on the school, 1(Part B), 163-200.
McLeskey, J., Billingsley, B., & Waldron, N. L. (2016). Principal leadership for effective
inclusive schools. Advances in Special Education, 32, 55–74.
https://doi.org/10.1108/S0270-401320160000032005
Nasreen, A., & Odhiambo, G. (2018). The Continuous Professional Development of School
Principals: Current Practices in Pakistan. Bulletin of Education and Research, 40(1),
245-266.
Paletta, A., Alivernin, F., & Manganelli, S. (2017). Leadership for learning : The
relationships between school context ,. International Journal of Educational
Management31, 31(2), 98–117. https://doi.org/10.1108/IJEM-11-2015-0152
Ross, J. A., & Gray, P. (2006). School leadership and student achievement: The mediating
effects of teacher beliefs. Canadian Journal of Education/Revue canadienne de
l'éducation, 798-822. https://doi.org/10.2307/20054196
Sebastian, J., & Allensworth, E. (2012). The influence of principal leadership on classroom
instruction and student learning: A study of mediated pathways to
learning. Educational administration quarterly, 48(4), 626-663.
https://doi.org/10.1177%2F0013161X11436273
Tentama, F., & Pranungsari, D. (2016). The Roles of teachers’ work motivation and
teachers’ job satisfaction in the organizational commitment in extraordinary
schools. International Journal of Evaluation and Research in Education (IJERE), 5(1),
39. https://doi.org/10.11591/ijere.v5i1.4520
Thoonen, E. E., Sleegers, P. J., Oort, F. J., Peetsma, T. T., & Geijsel, F. P. (2011). How to
improve teaching practices: The role of teacher motivation, organizational factors,
and leadership practices. Educational administration quarterly, 47(3), 496-536. DOI:
10.1177/0013161X11400185
Wahlstrom, K. L., Thomas, E., Leithwood, K., & Anderson, S. E. (2010). Learning from
Leadership : Investigating the Links to Improved Student Learning. Vol. 42. New
York, NY: Wallace Foundation.
Xiao, J., & Wilkins, S. (2015). The effects of lecturer commitment on student perceptions of
teaching quality and student satisfaction in Chinese higher education. Journal of
Higher Education Policy and Management, 37(1), 98–110.
https://doi.org/10.1080/1360080X.2014.992092
Zikhali, J., & Perumal, J. (2016). Leading in disadvantaged Zimbabwean school contexts:

247
Female school heads’ experiences of emotional labour. Educational Management

Administration & Leadership, 44(3), 347-362.
Zhang, L. F., & Jing, L. Z. (2016). Organisational commitments and teaching styles among
academics in mainland China. Educational Psychology, 36(3),
http://dx.doi.org/10.1080/01443410.2014.955462

248

Vol. 19, No. 10, pp. 248-264, October 2020
The Impact of Inclusion Setting on the Academic

Performance, Social Interaction and Self-Esteem
of Deaf and Hard of Hearing Students:
Systematic Review and Meta-Analysis
Sulaiman M. Alshutwi, Aznan Che Ahmad and Lay Wah Lee

School of Educational Studies, Universiti Sains Malaysia, Malaysia
https://orcid.org/0000-0003-2604-4821
https://orcid.org/0000-0003-1691-925X
https://orcid.org/0000-0002-0684-7902
Abstract. This research investigates the impact of inclusion setting on

academic performance, social interaction, and deaf and hard-of-hearing
(D/HH) students’ self-esteem. As opposed to the traditional nature of
conducting a literature review, meta-analysis has the advantage of
offering a critical evaluation of several previous studies with a common
theme and combining their results. This has the potential of increasing
statistical power which improves the generalizability of findings to a
wider population. Thus, this study provides important conclusions that
are likely to influence future research and decisions made by
policymakers regarding the impact of inclusion setting on academic
performance, social interaction and self-esteem of D/HH students.
Guidelines of PRISMA (2009) was adopted for searching relevant studies
by using keywords. The websites of World Library World Cat, Cochrane
Library, Web of Science, Biomed Central, Psyc INFO, Global Health,
MEDLINE, EMBASE, PubMed, and Google Scholar were searched for
relevant research conducted from January 1st, 1990 to February 28th,
2020. Therefore, the present study is a systematic review of the studies
followed by a meta-analysis of proportions carried out using Med Calc
online software. The results of the present findings have demonstrated
that inclusion setting improves academic performance, social interaction,
and self-esteem of D/HH students. Based on our findings, we
recommend that D/HH children should have access to a ‘good inclusion
setting’. Therefore, policymakers and practitioners ought to advocate the
need for developing special strategies for such children.
Keywords: Inclusion setting; Academic Performance; Social Interaction;

Self-Esteem; Deaf and Hard of Hearing

249
1. Introduction
1.1 Background of the Study
More than 5% of the world’s population suffers from D/HH (432 million adults
and 34 million children). By 2050 there will be more than 900 million hearing-
impaired (HI) people. That is to say, one out of every ten people shall suffer
hearing impairment (WHO, 2020). The condition of hearing loss in adults
whereby hearing loss exceeds 30-40 decibels (dB) is referred to as disabling
hearing loss. ‘Deaf’ people have a hearing impairment and customarily
communicate with the aid of sign language (WHO, 2020). Experts opined that the
impact of hearing loss on children’s educational achievements and development
could be reduced if the disorder is detected early and timely intervention is
applied. For instance, Callow-Heusser (2011) reported that early detection as well
as management of hearing loss improves language and educational outcomes for
D/HH children. Evidence in literature identifies various effects of hearing loss.
For example, functional, social, emotional, and economic are a few among the
effects (National Research Council, 2004). A major effect of hearing impairments
is evident in the ability of individuals while communicating with others. The
spoken language development in children with unwanted hearing loss is often
delayed. There are several detrimental effects concerning the inability of
individuals to communicate causing a significant impact on their daily life such
as experiencing feelings of loneliness.
The estimated value of about $ 750 billion is believed to be the annual global cost
of hearing loss according to the WHO (2020) report. These include costs of
educational support, productivity reductions, social and health care (without the
cost of hearing aids). D/HH children tend to have less self-esteem because of
differences in the audience regarding social maturity, physical appearance, and
communication skills. Since successful communication skills are not always
coupled with excellent living standards, professionals, and clinicians working
with D/HH children need recognition, counseling, and external counselling for
children in this population. As a result, it is critical to understand the components
which contribute to self-esteem to improve communication skills (Warner-Czyz,
Loy, Evans, Wetsel & Tobey, 2015). D/HH children tend to display improved
performance in literacy skills, ability to speak, and language development with
appropriate intervention at an early stage. However, despite the aforementioned
successes, several pupils continue to display disturbances in emotional, cognitive,
and behavioural control, executive functioning, and self-regulation. Even though
developmental learning is a complex process, it becomes necessary for
educational settings to make available services that promote skill development in
multiple dimensions, via ongoing assessment and revision. A model that involves
articulated curriculum approaches for early intervention and special education
programs can be considered and adopted in educating D/HH students (Mellon,
Ouellette, Greer & Gates-Ulanet, 2009).
The adoption of higher education policies in many developing countries is

opening up higher education for D/HH students. However, available data
indicates that the participation of D/HH students in higher education is low, and
research indicates that allocation support is insufficient for those receiving
benefits (Bell, Carl & Swart, 2016). There is considerable research investigating

250
social skills training and its effectiveness. However, there is a paucity of

information about the impact of such training on deaf adolescents (Naeini,
Arshadi, Hatamizadeh & Bakhshi, 2013). Further, evidence in the study by
Jaiyeola and Adeyemo (2018) demonstrates that D/HH students attending
general education classes face difficulties in social skills and relationships and
those consistent expectations of social outcomes include students' classroom
communication and participation in extracurricular activities.
1.2 Rationale of the Study

Inclusive learning is a range of strategies that help students’ learning, but some
specific strategies can be used to teach a group that includes D/HH students.
Research suggests that deaf children may exhibit social difficulties in comparison
with their listening contemporaries. Therefore, there is a need for a review of
factors that influence these social interactions to design possible interventions
(Batten, Oakes & Alexander, 2014). A limited number of studies (e.g. Batten et al.,
2014) have investigated the phenomenon, but they fall short to investigate the
critical role of ‘inclusion setting’ on academic performance, social engagement,
and students' self-esteem in D/HH. The present study makes use of systematic
review and meta-analysis to unify those that have been done before. The outcome
is likely to contribute to theory, practice, and policy.
1.3 Objectives of the Study

The following research questions guided the conduct of this study:
• What are the key themes in the selected studies involving inclusion setting as
the independent variable and academic performance, social interaction, and
self-esteem of deaf and hard-of-hearing students as the dependent variables?
• What is the effect size of the selected studies?
2. Methodology
2.1 Data Sources
Following the guidelines of PRISMA (2009), a comprehensive search of various
studies around the world was conducted using five keyword search terms:
"inclusion setting", "academic performance”, “social engagement”, “self-esteem”
and “hearing loss". The websites that were searched for systematic reviews
included World Library WorldCat, Cochrane Library, Web of Science,
BiomedCentral, PsycINFO, Global Health, MEDLINE, EMBASE, PubMed, and
Google scholar. Search dates were for studies carried out from January 1st, 1990 to
February 28th, 2020. This was followed by a meta-analysis using MedCalc online
software.
2.2 Inclusion criteria

The study was limited to include settings D/HH for students at any age only on
their academic performance, social engagement, and self-esteem. This includes
January 1, 1990, to February 28, 2020 (> more than 30 years). The major websites
selected for systematic review and meta-analysis are Google, PubMed, Embase,
Medline, Global Health, Psych Info, Biomed Central, Web of Science, Cochrane
Library, World Library, and World Cat.

251
2.3 Exclusion criteria

Studies that did not involve deaf or hard-of-hearing students were excluded, as
well as those without the remaining keywords. Any study published outside the
period January 1st 1990 and February 28th, 2020 did not qualify for inclusion.
2.4 Systematic Review Strategy

The Preferred Reporting Item for Systematic Review and Meta-Analysis
(PRISMA) was used for the systematic review. PRISMA consists of a checklist and
a diagram of a set of evidence-based topics that enables authors to conduct
systematic reviews and meta-analyses (PRISMA, 2009). Figure 1 is a flow diagram
showing the systematic review strategy following the PRISMA (2009) Guidelines.
Figure 1: Systematic Review Strategy based on PRISMA (2009) Guidelines
2.5 Meta-Analysis Strategy

Arsenic square root transformation was used to calculate summary proportions
weighted under fixed and random effects models according to the procedure
described by Negeri, Shaikh & Beyene (2018). The equation is embedded in
MedCalc software. Approaches suggested by Campbell, Braspenning,
Hutchinson and Marshall (2003) as well as Moher et al. (2015) were adopted in
this study. The systematic review involved five steps; finding good evidence by
visiting relevant sites, doing a search using keywords, digging deeper, reading

252
the forest and funnel plots, and applying the evidence to address the research
questions.
The forest plot served two purposes; [i] to determine the effect size and [ii] to
assess the level of difference (or heterogeneity) among the different publications.
The level of asymmetry in the funnel plot was used to establish the degree of
publication bias. A study conducted by Sterne and Egger (2001) evaluated several
possible measures for the vertical axis in funnel plots, found that the use of
standard error was likely to be the best choice as it is easy to interpret. For this
reason, the funnel plot was used to assess publication bias with standard error in
the vertical axis and proportion in the horizontal axis.
3. Study Findings
The results of this study are presented in two sections: [i] systematic review and
[ii] meta-analysis. These are presented below.
3.1 Systematic Review

Numerous studies have highlighted key elements for the success of inclusion
settings on students such as (a) availability of the interdisciplinary intervention
services in areas such as motor and cognitive development, psychosocial
functioning, communication and language (Mason, Williams & Cranmer, 2009), (b)
the academic environment needs to motivate students to examine their hearing
impairment in order to support them on the learning process. (Bell et al., 2016),
attending regular schools, and having a well-spoken language high test score
(Marschark, Shaver, Nagle & Newman, 2015), (c) the total perceived ability score
of adolescents with deafness and their ability to learn social skills (Soleimanieh,
Arshadi, Hatamizadeh & Soleimanieh, 2013), (d) The use of strategies to support
communication, language, education and social/emotional development for their
involvement (Borders, Bock, Giese, Gardiner-Walsh & Probst, 2018).
The findings of a study by Theunissen, Netten, Briaire, Soede, Kouwenberg &

Frijns (2014) revealed that hearing-impaired children participating in mainstream
education had better language skills (t = 3.17, p < .001), higher intelligence scores
(t = 2.22, p < .001) and improved communication skills (t = −22.69, p <.001) as
compared with children participating in special education. Like Chia-Fen (2013),
there are a few authors who have highlighted issues on inclusion. Their findings
suggest that D/HH students experience less family stress despite low educational
difficulties or a good GPA and family. Furthermore, contextually, the findings
reveal the importance of relationships for assessing academic achievement and
social competence between Taiwan’s D/HH college students. The findings by
Antia, Jones, Luckner, Kreimeyer & Reed (2011) also revealed major positive
effects of inclusion setting on disabled students and their teachers.
Some studies have highlighted new models of inclusion teaching for D/HH
students. For instance, Kathleen and McCain (2005) emphasize that D/HH
students can achieve sustained academic progress over the years to include
D/HH students. Some studies have asserted that D/HH children are associated
with equal levels of attention, while children with specific hearing have
emphasized domains of social participation and role function and engagement

253
(Constantinescu-Sharpe, Phillips, Davis, Dornan & Hogan, 2017). This study also
found that the majority of child-friendly social inclusion outcomes were for those
with normal hearing and good hearing. From the Master’s Thesis study by Allison
(2015) also appears that - teaching teachers about the opportunities for
development in all students when disabled students are integrated at the high
school level, is further required.
A review of the study by Dalton (2011) also highlighted the fact that how teachers
can support autonomy, competence, and relatedness for students with MMHL. A
study by Rachel (2012) revealed that inclusion classrooms generally promote
desired behaviours and social skills and ESE inclusion practices are effective.
Liversidge (2003) found that many factors can significantly help the students in
their ability to decide to enroll and stay in a mainstream university such as level
of commitment to attaining a college degree, past experience from the
mainstream, ability to self-advocate, support systems and development of study
skills. Davis (2004) observed that there is a need for a multi-method approach to
teaching pupils with special educational needs. King and Ryan (2019) suggested
that in a suitable physical and cultural environment, improvement in social
interaction with their peers is possible.
The scientometric analysis is utilized to objectively map the scientific knowledge

area while the critical review aims for the identification of the research themes as
well as the resultant challenges based on the result of scientometric. Table 1 is a
scientometric analysis of the eligible studies. The studies were published between
2005 and 2019. They included three surveys, two theses, one cohort pilot study,
one longitudinal study, one qualitative study, two cross-sectional studies, two
exploratory studies, one retrospective study, one quasi-experimental study, one
evaluation case study, and one secondary analysis. The theme synthesis for the
studies is presented in Table 2.
Table 1: Scientometric Analysis of Eligible Studies in SR and MA (n=16).

Country of
S/ N Author(s) Year Type of Study
Study
1.
Ameye, et al. Nigeria 2015 A cohort Pilot study
Ottawa,
2. Eriks-Brophy, A., & Whittingham, J. 2013 Survey
Canada
3. Mapolisa & Tshabalala Zimbabwe 2013 Qualitative study
4. Alasim, K. N. Saudi Arabia 2018 Exploratory Study
5. Marschark et al. . US 2015 Secondary Analysis
6. Soleimanieh et al. Iran 2013 Quasi-experimental
7. Jaiyeola, M. T., & Adeyemo, A. A. Nigeria 2018 Cross-sectional study
8. Chia-fen, L. Taiwan 2013 Thesis
9. Antia et al. UK 2011 Longitudinal study
10. Prakash, S. S. India 2012 Survey
11. Schmidt, M., & Čagran, B. Slovenia 2008 Evaluation case study
12. Kathleen, G., & McCain, S. A. US 2005 Cross-sectional study
Retrospective,
13. Theunissen et al. Netherlands 2014
multicenter study

254
Alegre de la Rosa, O. M. A., & Exploratory factor

14. Spain 2019
Angulo, L. M. V. analysis
15. Constantinescu-Sharpe et al. Australia 2017 Survey
16. Rachel, E. US 2012 Thesis
Table 2: Theme synthesis in the systematic review

S/N Author(s) Study Key findings Implications of the Key themes
methodology studies synthesized from the
studies
1. Ameye, et al. A cohort pilot • A significant percentage Economic and Minor impact of
(2015). study involving of the students had a educational policies inclusion
deaf students in limitation in achievement. need sensitization of deaf children
special education • More than 33% of the
secondary schools students were denied
(n=50) privileges as a result of being
deaf.
• A significant percentage
of the students were not
satisfied with the placement.
2. Eriks-Brophy, 63 classroom The teachers had favorable For inclusion to Minor impact of
A. & teachers involved attitudes, knowledge and succeed, teacher inclusion
Whittingha, J. in the study to teaching skills for students education programs of children
(2013). determine whether with hearing loss should: with hearing loss
teachers had the • Emphasize more
right attitudes, on the needs of
knowledge, and learners with hearing
teaching skills impairment
• Provide requisite
support for both
students and
teachers.
3. Mapolisa & Qualitative Inquiry The study found that: Regular teachers • Inclusion is
Tshabalala of teachers’ • Hearing impaired should: important for hearing
(2013). experience in children in ordinary schools • Undergo in- impairment children
handling hearing had various problems. service training • The provision of
impairment among • Majority of the regular • Be provided with equipment for effective
children teachers were lacking proper equipment for teaching of children
n=50 teachers adequate resources and the effective teaching with hearing
using purposive necessary expertise. children with hearing impairment is a
sampling. impairment necessity.
• Minor impact of
inclusion setting
4. Alasim, K. N. The study was to The study found that DHH There is a need for• Minor impact of
(2018) identify strategies students face barriers to specific strategies to inclusion setting
that facilitate participation and interaction facilitate the • Inclusion improves the
participation and in the general education participation of participation of Deaf
interaction of Deaf classroom. d/Deaf and hard of and hard-of-hearing
and hard of hearing students students
hearing students.
5. Marschark et al. The study involved Findings indicated that Teachers need to • Instructional
(2015). analysis of data teachers can appropriately develop quality quality impacts
from the National target their instruction by instructional positively on the
Longitudinal recognizing the diverse approaches that take performance of
Transition Study–2 strengths and needs of into account the children with hearing
(NLTS2), funded hearing impaired students diverse needs of loss
by the U.S hearing impaired • Major impact of
(n=11,000) students inclusion setting

255
6. Soleimanieh et This was a pre-post There was significant Improvement in the • Social skills among
al. (2013). quasi-experimental improvement in total adolescents’ sense of adolescent students
design involving perceived competence scores competence and with deafness impacts
female students of adolescents with deafness emotional wellbeing positively on their
with deafness due to intervention (P < through learning emotional and
(n=69) 0.001) in addition to three social skills. communication
domains of socio-emotional competence
competence (P = 0.003), • Major impact of
school adjustment (P = 0.018) inclusion setting
and communication
competence (P < 0.001).
7. Jaiyeola, M. T. The study The study found that the Factors influencing • Poor quality of life
& Adeyemo, investigates the majority of deaf and hard of the quality of life for for Deaf and hard-of-
A.A. (2018). quality of life of hearing students (57.8%) had DHH students hearing students may
DHH students in poor quality of life. should be attended affect their academic
Ibadan, Nigeria to. achievement.
using a cross- • Major impact of
sectional study. inclusion setting
(n=110)
8. Chia-fen, L. The thesis The study found that In the university level Academic achievement
(2013). objective was to students with hearing loss have to develop a and social aspect of
identify the factors experienced less family stress helpful program and DHH students’
influencing the and had fewer academic an encouraging remains obscure in
academic and difficulties leading to better environment for Taiwan.
social adjustment GPAs. Als, the family DHH undergraduate
of college students relationship was altogether students in Taiwan.
with hearing loss related with academic
in Taiwan achievement.
9. Antia et al. This was a 5-year The study found that some The impact of • Greater focus on
(2011). longitudinal study students placed in inclusive inclusion on the characteristics of
that examined the settings did not realize important outcomes effective instruction in
social skills and positive outcomes. needs more research inclusion settings is a
problem attention. necessity.
behaviours of • Major impact of
D/HH students inclusion setting
10. Prakash, S.S. This was a survey • Most teachers were for Interventional needs • Major positive
(2012) the inclusion of students to bring out teachers’ impact of inclusion of
with disabilities. more positive children with hearing
• There were significant attitudes. loss.
differences in teachers’ • Teachers’ attitudes
attitudes towards inclusion towards inclusion
across their qualification, varies across
management level, level of qualification,
teaching, gender and experience, gender,
teaching experience. level of teaching and
management level.
11. Schmidt, M. & This was an • Integrated students with Students with • Hearing
Čagran, B. evaluation case hearing impairment in hearing impairment impairment impacts
(2008). study that regular classrooms need adequate negatively on self-
investigated the- performed below their support concept.
self-concept of hearing counterparts in • Inclusion of
students from 7th academics and social self- children with
grade in regular concept hearing loss impacts
primary schools. • There was a noticeable positively on their
advantage for the students self-concept
from the class with • Major impact of
integrated learners over the inclusion setting
other class serving as
control group

256
12. Kathleen, G. & The study involved • Findings indicated that Co-enrolment is a • Positive impact of
McCain, S. A. communication differences did exist in possible model of inclusion of children
(2005). participation, academic achievement inclusion for D/HH with hearing loss
academic across disability type. students. • Major impact of
achievement, and • Students with D/HH inclusion setting
social behavior of 5 differed significantly from
D/HH students, 5 their hearing peers in the
D/HH students variables under study.
with additional
disabilities
(D/HH-D), and 18
nondisabled,
hearing peers in a
co-enrolled, Grade
3-4-5 combination
classroom
(n=28)
13. Theunissen et This was a Hearing-impaired children Children with • Enhancement of
al. (2014). retrospective, experienced lower levels of hearing impairment self-esteem is
multicenter study self-esteem than those with need adequate important for
(n=252) no hearing impairment. support.to improve children with
their self-esteem. hearing impairment.
• Major impact of
inclusion setting
14. Alegre de la This was a study The result indicates four The students’ Major impact of
Rosa, O. M. A.& which made use of factors of students’ attitudes attitudes towards IE inclusion setting
Angulo, L. M. exploratory factor towards IE: Family was significant. The
V. (2019). analysis to collaboration and utilize of scale (ICAQ) valid to
determine the technology, inclusion of the measure the attitudes
reliability and students in the centre, of students with
relevance of ICAQ communication with medical impairments towards
professionals, and evaluation the inclusive setting.
of the supporting technology
15. Constantinescu- This is a study that The majority of the children The use of LSL • Approach to
Sharpe et al. involves parents of with hearing loss are as well approach in the teaching needs to be
(2017). children with equated in terms of outcomes teaching of children considered by
hearing with those with typical with hearing loss teachers when
impairment hearing in terms of may be beneficial. teaching children
between the age of education, interaction with with hearing loss.
4–5 years who society, and fulfilment of • Major impact of
learnt through LSL social roles’ aspects of social inclusion setting
approach inclusion.
(n = 78).
16. Rachel, E. This was a doctoral Findings indicated that ESE inclusion • Positive impact of
(2012). thesis inclusion classrooms: practices are inclusion of children
• Do not hurt a student’s effective. with hearing loss
self-esteem. • Major impact of
• Promote desired inclusion setting
behaviours and social
skills.
Two themes emerged from the systematic review; studies in which inclusion
setting had a minor impact on deaf children and those in which inclusion setting
had a major impact on deaf children. The two themes are discussed below.

257
Theme 1: Minor Impact on Deaf Children (n = 4)

Following the conduct of a systematic review of literature, four studies were
found in which inclusion setting had a minor impact on academic performance,
social participation, and students' self-assessment. These were studied by Ameye,
Adeyemo, Eziyi, Amusa, Ogunniyi and Otoghile (2015); Eriks-Brophy and
Whittingham (2013); Mapolisa and Tshabalala (2013) and Alasim (2018). These
studies suggest that teacher education programs should be tailored to suit deaf
children. Students with hearing impairment should be taught in a way that
mitigates hindrances in general education classrooms. Further, these studies
emphasize the use of appropriate pedagogical skills and the availability of
resources for teachers so that they can manage hearing impaired children. Thus,
there must be specific strategies for deaf and hard-of-hearing students to ensure
their participation in classroom settings.
Theme 2: Major Impact on Deaf and Hard-of-Hearing Children (n = 12)

Figure 2 presents the studies which displayed a major positive impact of inclusion
setting on deaf and hard-of-hearing children. Nine of the studies (75%) showed a
positive impact of over 30%.
Figure 2: Studies with major impact of inclusion setting on hearing-impaired children
3.2 Findings for Meta-Analysis

Table 3 contains summary statistics of studies included in the meta-analysis. A
total of 12 studies which were found to be eligible for meta-analysis had a mean
sample size of 539.7 (95% CI= -402 to 1481.5), Median value of 55.0 variance of
4974537.50 (SD=2230.3), SE Mean=455.2, Coff. of Skewness=4.8 (p<0.0001) and
Coff. of Kurtosis=23.8 (p<0.0001).

258
Table 3: Summary Statistics of Studies Included in meta-analysis.

Statistic Value
Sample size 24
Lowest value 14.0
Highest value 11000.0
Arithmetic mean 539.7
95% CI for the Arithmetic mean -402.0 to 1481.5
Median 55.0
95% CI for the median 37.4 to 83.5
Variance 4974537.5
Standard deviation 2230.3
Relative standard deviation 4.1322 (413.2%)
Standard error of the mean 455.2
Coefficient of Skewness 4.8 (p<0.0001)
Coefficient of Kurtosis 23.8 (p<0.0001)
D'Agostino-Pearson test Reject Normality
for Normal distribution (p<0.0001)
Figure 3 is a forest plot for the 12 studies. All the studies except that by Marschark
et al. (2015) had similar effect sizes. Incidentally, the study by Marschark et al.
(2015) was the most reliable in the meta-analysis yet it was the same study that
exhibited the least impact of inclusion setting among the 12 studies that were in
the major impact category. The least reliable was that Kathleen and McCain
(2005).
Figure 3: Forest Plot for the impact of inclusion setting on D/HH students (n=12)
The relationship between effect sizes in the studies and their statistical power was
assessed by examining the funnel plot (Fig. 4). The plot displays a huge
asymmetry caused by eleven studies. This suggests the possible presence of
publication bias which means that the outcome of the studies influenced the

259
decision of whether to publish them or not. Such bias may be reduced by having
journals publish high-quality studies on inclusion setting regardless of novelty or
unexciting results, and by publishing protocols or full-study data sets.
Figure 4: Funnel plot for the effect of inclusion setting on D/HH Students
The proportion for the total fixed effect was 6.6% and that of the total random
effect was 44.3% (See Table 4). The combined value for I2 was 99% suggesting that
there existed a high degree of heterogeneity across the studies included in the
analysis (p<0.0001).
Table 4: Effect of inclusion setting on D/HH students

Study Sample size Proportion 95% CI Weight (%)
(%) Fixed Random
Ameye et al. (2015). 50 56.0 41.2 to 70.0 0.4 8.2
Eriks-Brophy &
63 52.3 39.4 to 65.1 0.5 8.3
Whittingham (2013).
Marschark et al.
11000 4.5 4.1 to 4.9 91.7 8.5
(2015).
Soleimanieh et al.
69 40.5 28.9 to 53.0 0.5 8.3
(2013)
Jaiyeola & Adeyemo
110 53.6 43.8 to 63.1 0.9 8.4
(2018)
Chia-fen, L. (2013) 132 46.2 37.5 to 55.0 1.1 8.4
Mapolisa &
50 40.0 26.4 to 54.8 0.4 8.2
Tshabalala (2013)
Prakash, S. S. (2012) 100 51.0 40.8 to 61.1 0.8 8.3
Maida, S. (2013) 42 50.0 34.1 to 65.8 0.3 8.2
Kathleen, G. &
28 50.0 30.6 to 69.3 0.2 8.0
McCain, S. A. (2005)
Theunissen et al.
252 50.0 43.6 to 56.3 2.1 8.4
(2014)
Constantinescu-
78 50.0 38.4 to 61.5 0.6 8.35
Sharpe, C. G. (2017)
Total (fixed effects) 11974 6.6 6.2 to 7.1 100.0 100.0
Total (random
11974 44.3 23.4 to 66.3 100.0 100.0
effects)

260
4. Discussion
Scientometric analysis in this research involved 16 studies. The studies were
conducted in diverse countries using diverse research designs. When all these
studies were further analysed through a systematic review, two themes emerged;
which are the studies with minor and major impact.
Numerous studies (Ameye et al., 2015; Rachel, 2012) have systematically

highlighted the key to the success of inclusion settings on D/HH students. For
instance, the role of trans-disciplinary intervention services (Mellon et al. 2009),
facilitating staff's understanding of 'identity' (Bell et al., 2016), attending regular
secondary schools, and the role of good spoken language (Marschark et al., 2015)
in their ability and emotional well-being and studies on improving intervention
(Solaimanih et al., 2013; Ameye et al., (2015). However, contrary to studies by
Mapolisa and Tshabalala (2013) who conclude that there are no specific strategies
in inclusion settings for deaf and hearing students. In addition, some studies also
emphasize factors that encourage D/HH students to have uncontrolled attention
before a lecture (Warner-Czyz et al., 2015). Previous studies have suggested that
attitudes of teachers toward disabled students in inclusive classrooms settings are
highly significant and do influence these students with respect to the reporting of
low loneliness, maintaining their self-esteem, and acceptance of peers (Luftig,
1985; Campbell, Dodson & Bost, 1985). The study by Pavri and Luftig (2000) also
indicated the urgent need for the preparation programs to be able to provide and
address social support for students with disabilities in the general classroom.
Therefore, it appears certain strategies are needed to facilitate social work for
students in all educational settings. Therefore, it is imperative for students with
learning disabilities to receive active and intentional coaching that could result in
acceptance in learning social behaviour. Mellon et al. (2009) found that
educational programs need to provide children with social-emotional learning,
innovations for auditory perception, access to the full range of basic skills essential
for social and academic achievements. Another study by Warner-Czyz et al.
(2015) concluded that children with hearing disabilities were significantly more
positive in global self-esteem ratings compared to hearing individuals
(t = 2.38, p = .02). Similarly, they found significant positive correlations between
self-esteem ratings with affiliation (r = .42, p = .002). Consequently, it indicates an
urgent need for professionals and clinicians working with children suffering from
hearing loss to recognize components responsible for self-esteem. A study by
Pujari and Annapurna (2015) in Indian primary schools on mental retardation
cases also came with a similar conclusion of the need for an adequate support
system for the success of inclusive education in different kinds of patients.
The findings of our systematic review are in line with previous research. For
example, a systematic review by Warner-Czyz. et al. (2015) reported that children
with hearing loss rated global self-esteem significantly more positively compared
to hearing peers and a significant positive relationship was established between
self-esteem ratings with both affiliation (r=.42, p =.002) and attention (r =.45, p
=.001). Whereas, a significant negative relationship was established between
global self-esteem and depressive mood in the present study (r= .60, p =.001).

261
It was widely reported that self-esteem is highly significant for psychosocial

functioning. Hypothetically, hearing-impaired (HI) children have lower levels of
self-esteem, due to the frequent experience of lower language and communication
skills (Theunissen, et al., 2014). The special education services legal and scientific
basis was based on the positive outcomes for students with disabilities when
exposed to inclusive versus segregated education (National Council on Disability,
2018). Experienced teachers do not have challenges to structure lessons in
advance, however, planning in teaching inclusion classes is vital and considered
a valuable procedure. To be specific, the procedure of how to figure out the best
strategies to engage and accommodate students' challenges, physical needs,
interests, developmental levels, and gifts that can help teachers in achieving
realistic inclusion for all students (Willis, 2007)
Surprisingly, a few findings of our systematic review have displayed

contradictory results as compared to prior studies by Ameye et al. (2015) and
Alasim (2018). A systematic review of literature by Yu-Han, Potměšil and Brenda
(2014) identified two broad themes – including process and interactions - with
peers and intervention programs. Studies have indicated that D/HH children are
liable to encounter difficulties in managing, initiating, and communicating with
hearing-peers in an inclusive environment. For the social participation of DHH
children, co-enrollment and social skills training programs were reported to be
effective interconnections. The communication and social skills of D-HH children,
children with general hearing reactions, and the influence of encyclopaedias are
described as important features as far as social interactions are a concern. A study
by Batten et al. (2014) also reveals that a wide range of factors are associated with
social relationships between deaf and hearing children. A study by Antia et al.
(2011) also reported that deaf and hearing children differed in social competence
and that deaf children heard less on social and socially re-trained behaviour than
hearing children. All these studies show the variability of the data in the literature,
which is evident in our study. We can, therefore, conclude from our meta-analysis
that overall the impact of inclusion setting is significant (6.6% to 44.3% in the fixed
and random effect models, respectively).
5. Conclusion
The combined proportion impact of ‘inclusion setting’ ranged from 6.6 percent
(fixed effect) to 44.3 percent (random Effect) as per calculated effect sizes in meta-
analysis. Data also displayed statistical significance [p <0.0001, Q=1153.8, 95% CI
for I2=98.8-99.9]. Thus, we can conclude from the findings of the present study
that the inclusion setting improves academic performance, social interaction, and
self-esteem of D/HH students. Furthermore, based on the outcomes of the present
study, we recommend that there is a need to handle D/HH students not only
appropriately but also understand the fact that it is difficult for many such
children to develop a language unless they have access to a ‘good inclusion
setting’. Therefore, practitioners and policymakers must advocate the need for
devising special strategies for such children. Consequently, the inclusion of D-HH
children in mainstream classrooms needs to focus more on extensive research in
‘inclusive education’ in the future for better communication and social interaction
of D/HH students.

262
6. References
Alasim, K. N. (2018). Participation and Interaction of Deaf and Hard-of-Hearing Students
in Inclusion Classroom. International Journal of Special Education, 33(2), 1-14.
Alegre de la Rosa, O. M. A., & Angulo, L. M. V. (2019). Attitudes of Children with Hearing
Loss towards Public Inclusive Education. Education Sciences, 9(3), 244.
https://doi:10.3390/educsci9030244
Allison, E. (2015). The Effects of Inclusion Classrooms on Students with and Without
Developmental Disabilities: Teachers’ Perspectives on the Social, Emotional, and
Behavioral Development of All Students in Inclusion Classrooms. Graduate School
of Professional Psychology: Doctoral Papers and Masters Projects, 31.
Ameye, S. A., Adeyemo, A., Eziyi, J., Amusa, Y., Ogunniyi, G., & Otoghile, B. (2015). The
psychosocial consequences of severe deafness and the modifying effect on the
socioeconomic position in a special education institution setting: A pilot study.
Journal of Indian Speech Language Hearing Association, 29, 8-11.
Antia, S. D., Jones, P., Luckner, J., Kreimeyer, K. H., & Reed, S. (2011). Social Outcomes of
Students Who are Deaf and Hard of Hearing in General Education
Classrooms. Exceptional Children, 77(4), 489–504.
Batten, G., Oakes, P. M., & Alexander, T. (2014). Factors associated with social interactions
between deaf children and their hearing peers: a systematic literature
review. Journal of deaf studies and deaf education, 19(3), 285–302.
https://doi.org/10.1093/deafed/ent052
Bell, D., Carl, A., & Swart, E. (2016). Students with hearing impairment at a South African
university: Self-identity and disclosure. African Journal of Disability, 5(1), 229.
https://doi.org/10.4102/ajod.v5i1.229
Borders, C., Bock, S., Giese, K., Gardiner-Walsh, S., & Probst, K. (2018). Interventions for
Students Who Are Deaf/Hard of Hearing. In F. Obiakor & J. Bakken
(Eds.) Viewpoints on Interventions for Learners with Disabilities (Advances in Special
Education) Vol. 33, pp. 75-105, Emerald Publishing
Limited. https://doi.org/10.1108/S0270-401320180000033005
Callow-Heusser, C. A. (2011). The effects of early identification and intervention on language
outcomes of children born with hearing loss. [Dissertation]. Utah State University.
Campbell, N. J., Dodson, J. E., & Bost, J. M. (1985). Educator perceptions of behavior
problems of mainstreamed students. Exceptional Children, 51,298- 303.
Campbell, S. M., Braspenning, J., Hutchinson, A., & Marshall, M. N. (2003). Research
methods used in developing and applying quality indicators in primary care. BMJ
(Clinical research ed.), 326(7393), 816–819.
https://doi.org/10.1136/bmj.326.7393.816
Chia-fen, L. (2013). Academic and social adjustment among deaf and hard of hearing college
students in Taiwan (Dissertation). University of Kansas. Retrieved from
https://goo.gl/yUT7vd
Constantinescu-Sharpe, G., Phillips, R. L., Davis, A., Dornan, D., & Hogan, A. (2017).
Social inclusion for children with hearing loss in listening and spoken Language
early intervention: an exploratory study, BMC Pediatr, 17(1), 74.
https://doi.org/10.1186/s12887-017-0823-y
Dalton, C. (2011). Social-emotional Challenges Experienced by Students Who Function
with Mild and Moderate Hearing Loss in Educational Settings. Exceptionality
Education International, 21, 28-45. Retrieved from
https://ir.lib.uwo.ca/eei/vol21/iss1/4
Davis, P., & Florian, L. (2004). Teaching Strategies and Approaches for Pupils with Special
Educational Needs: A Scoping Study. Report submitted to the universities of

263
Manchester and Cambridge. Retrieved from

https://dera.ioe.ac.uk/6059/1/RR516.pdf
Eriks-Brophy, A., & Whittingham, J. (2013). Teachers’ Perceptions of the Inclusion of
Children with Hearing Loss in General Education Settings. American Annals of the
Deaf, 158(1), 63-97.
Jaiyeola, M. T., & Adeyemo, A. A. (2018). Quality of life of deaf and hard of hearing
students in Ibadan metropolis, Nigeria. PloS one, 13(1).
https://doi.org/10.1371/journal.pone.0190130
Kathleen, G., & McCain, S. A. (2005). Academic and Social Status of Hearing, Deaf, and
Hard of Hearing Students Participating in a Co-enrolled Classroom.
Communication Disorders Quarterly, 27(1), 20-32.
King, N., & Ryan, B. J. (2019). The Impact of Social Inclusion on the Social Development of
Students with a General Learning Difficulty in Postprimary Education in Ireland.
Education Research International, 2019, 7. https://doi.org/10.1155/2019/6327567
Liversidge, A. G. (2003). Academic And Social Integration Of Deaf And Hard-Of-Hearing
Students (PhD Thesis). Carnegie Research-I University Liversidge.
Luftig, R. L. (1985). The reality of children's loneliness. Paper presented at the Annual
Meeting of the American Educational Research Association, Chicago.
Mapolisa, T., & Tshabalala, T. (2013). The impact of inclusion of children with hearing
impairment into regular schools: A case study of Dakamela primary school in
Zimbabwe, International Journal of Asian Social Science, 3(7), 1500-1510.
Marschark, M., Shaver, D. M., Nagle, K. M., & Newman, L. A. (2015). Predicting the
Academic Achievement of Deaf and Hard-of-Hearing Students from Individual,
Household, Communication, and Educational Factors. Exceptional Children; 81(3),
350-369.
Mason, G., Williams, G., & Cranmer, S. (2009). Employability skills initiatives in higher
education: what effects do they have on graduate labour market outcomes?,
Education Economics, 17(1), 1-30, https://doi.org/10.1080/09645290802028315
Mellon, N. K., Ouellette, M., Greer, T., & Gates-Ulanet, P. (2009). Achieving developmental
synchrony in young children with hearing loss. Trends in amplification, 13(4), 223–
240. https://doi.org/10.1177/1084713809356701
National Council on Disability. (2018). The Segregation of Students with Disabilities.
https://ncd.gov/sites/default/files/NCD_Segregation-SWD_508.pdf
National Research Council. (2004). Impact of Hearing Loss on Daily Life and the
Workplace. In R. A. Dobie & S. Van Hemel (eds.) Hearing Loss: Determining
Eligibility for Social Security Benefits. Washington (DC): National Academies Press
(US). Retrieved from https://www.ncbi.nlm.nih.gov/books/NBK207836/
Negeri, Z. F., Shaikh M., & Beyene J. (2018). Bivariate random‐effects meta‐analysis
models for diagnostic test accuracy studies using arcsine‐based transformations.
Biom J., 60(4), 827‐844.
Pavri, S., & Luftig, R. (2000). The Social Face of Inclusive Education: Are Students with
Learning Disabilities Really Included in the Classroom? Journal of Preventing
School Failure: Alternative Education for Children and Youth, 45(1), 8-14,
https://doi.org/10.1080/10459880109599808
Prakash, S. S. (2012). Inclusion of Children with Hearing Impairment in Schools: A Survey
on Teachers’ Attitudes. Disability’, CBR and Inclusive Development Journal, 23(3), 90-
111.
Preferred Reporting Items For Systematic Review & Meta-Analysis (PRISMA). (2009).
Retrieved from http://www.prisma-statement.org/

264
Pujari, J., & Annapurna, M. (2015). A study on available support systems in inclusive
setting for the students with mental retardation. Indian Journal of Cerebral Palsy, 1,
35-41.
Rachel, E. (2012). Inclusion Classrooms as it Relates to Self-Esteem, Behavior, and Social Skills
(PhD Thesis) Trampler Southeastern University – Lakeland.
Soleimanieh, N. T., Keshavarzi, A. F., Hatamizadeh, N., & Bakhshi, E. (2013). The effect of
social skills training on perceived competence of female adolescents with
deafness. Iranian Red Crescent Medical Journal, 15(12):e5426.
https://doi.org/10.5812/ircmj.5426
Sterne, J. A., & Egger, M. (2001). Funnel Plots for Detecting Bias in Meta-Analysis:
Guidelines on Choice of Axis. Journal of Clinical Epidemiology, 54(10), 1046-1055.
https://doi.org/10.1016/s0895-4356(01)00377-8.
Theunissen, S. C., Rieffe, C., Netten, A. P., Briaire, J. J., Soede, W., Kouwenberg, M., &
Frijns, J. H. (2014). Self-esteem in hearing-impaired children: the influence of
communication, education, and audiological characteristics. PloS one, 9(4),
e94521. https://doi.org/10.1371/journal.pone.0094521
Warner-Czyz, A. D., Loy, B. A., Evans, C., Wetsel, A., & Tobey, E. A. (2015). Self-esteem in
children and adolescents with hearing loss. Trends in hearing, 19,
2331216515572615. https://doi.org/10.1177/2331216515572615
WHO. (2020). Deafness and hearing loss 2020. Retrieved from
https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-
loss
Willis, J. (2007). Brain-friendly strategies for the inclusion classroom: Insights from a neurologist
and classroom teacher. Alexandria, VA: Association for Supervision and
Curriculum Development.
Yu-Han, X., Potměšil, M., & Brenda, P. (2014). Theory/Review Children Who Are Deaf or
Hard of Hearing in Inclusive Educational Settings: A Literature Review on
Interactions with Peers. Journal of Deaf Studies and Deaf Education. 19, 4, 423-437.
https://doi.org/10.1093/deafed/enu017

265

Vol. 19, No. 10, pp. 265-285, October 2020
Teacher Support for Eliciting Students

Mathematical Thinking: Problem Posing, Asking
Questions, and Song
Ary Woro Kurniasih, Isti Hidayah and Mohammad Asikin

Department of Mathematics, Faculty of Mathematics and Natural Sciences,
Semarang State University, Semarang, Indonesia
https://orcid.org/0000-0003-1674-4226
https://orcid.org/0000-0001-8462-3376
https://orcid.org/0000-0003-1121-3629
Abstract. Mathematical thinking and reasoning are at the core of

mathematics learning, strategies to facilitate their development are global
issues for various countries. This qualitative study aims to describe the
fifth-grade elementary school teacher’s stimulus form, to expand
students’ thinking ability in mathematics learning and obtain a stimulus
formulation of the fifth-grade elementary school teacher in developing
students’ mathematical thinking ability. Data were obtained by using
open-ended questionnaire methods and passive participation
observation. The study results showed that the stimulus form of the fifth-
grade elementary school teacher in expanding students’ thinking skill in
learning the mathematics comprises problem posing, asking guiding
questions, facilitating technology (learning
videos/PowerPoint/Plickers), and song. The recommendation of fifth-
grade elementary school teacher stimulus for developing students’
mathematical thinking ability was asking the probing question, playing,
and games. Furthermore, the results of the second year’s research were
widely disseminated to teachers. Primary school teachers who are part of
the teacher working group also need to improve their professionalism,
especially concerning developing stimulus thinking and applying it in
learning. Therefore, the follow-up as the management of learning in the
form of INNOMATTS training.
Keywords: guiding question; mathematical thinking; problem posing;

song; students thinking
1. Introduction
The enforcement of education in Indonesia has followed its 2013 curriculum since
the academic year 2013/2014. Because mathematical thinking and reasoning are
at the core of mathematics learning, strategies to facilitate their development are
global issues for various countries and organizations, including the Australian

266
Curriculum and Assessment Reporting Authority (Harrington, 2008) and the

National Council of Teachers of Mathematics (NCTM) (2017a). In Indonesia, the
country has followed its 2013 curriculum since the academic year 2013/2014. This
curriculum is based on Regulation of Ministry of Education and Culture Republic
of Indonesia No. 21 of 2016 (Ministry of Education and Culture Republic of
Indonesia, 2016), which concerns elementary and secondary education content
standards. It identifies three graduate competencies: attitude, knowledge, and
skill. These three competencies are further broken down into four dimensions, or
core competencies: spiritual, social, knowledge, and skill. Regulation of Ministry
of Education and Culture Republic of Indonesia No. 37 of 2018 (Ministry of
Education and Culture Republic of Indonesia, 2018) states that for the fifth-grade
elementary school student, the core competency of knowledge means the ability
of the students to comprehend knowledge both of factual and conceptual by
inspecting and questioning based on curiosity about himself, God’s creatures and
their activities, and the objects found at daily life. Students’ questioning activity
is, therefore, the way in which they will attain knowledge competency. According
to Tofade, Elsner, and Haines (2013), a question is a learning tool that can
stimulate critical thinking. Questions asked must be analytical, essential, and
creative so that the questioning activity conducted by the students has a role in
encouraging high thinking competency (Samo, Darhim, & Kartasasmita, 2017).
Such activity is thus seen as critical to developing a student’s ability to think—
and thereby learn—in the mathematical context.
The Programme for International Student Assessment (PISA) is a worldwide

program by the Organisation for Economic Co-operation and Development
(OECD). In terms of Indonesian students’ mathematics performance, PISA scores
in 2003, 2006, 2009, 2012, 2015, and 2018 were 360, 391, 371, 375, 386, and 379,
respectively (OECD, 2019). Overall, the performance trajectory in mathematics
performance is hump-shaped, i.e., it is more negative over more recent years
(OECD, 2019). The 2015 percentage distribution of 15-year-old students on the
PISA mathematics literacy scale is presented in Table 1, and that of 2018 is
presented in Table 2.
Table 1: The distribution of 15-year-old Indonesian students recorded in the PISA

mathematics literacy scale, based on the proficiency level and education system, in
2015
Below Level 1 Level 2 Level 3 Level 4 Level 5 Level 6
Education Level 1
system % SE % SE % SE % SE % SE % SE % SE
Indonesia 37.9 1.68 30.7 1.12 19.6 1.01 8.4 0.72 2.7 0.38 0.6 0.17 ‡ †
† Not applicable.
‡ Reporting standards not met due to coefficient of variation over 50 percent.
Based on the OECD (2019), there are six proficiencies in mathematics on

the PISA scale (from Level 1, the lowest, to Level 6, the highest). Based on
Table 1, 68.6% of students in Indonesia attained Level 1 (i.e., they were
proficient below Level 1 and at Level 1). Based on Table 2, 71.9% of students
in Indonesia attained Level 1 (i.e., they were proficient below Level 1 and

267
at Level 1). Based on Table 1, 31.4% of students in Indonesia attained Level

2 or higher (i.e., they were proficient at Levels 2, 3, 4, 5, or 6). Based on Table
2, only 28.1% of Indonesia's students attained Level 2 or higher. According
to OECD (2019), students who scored below Level 2 are considered “low-
achieving students”; thus, most Indonesian students fall into this category.
Based on the OECD (2014), there are six proficiencies in mathematics on the PISA
scale (from Level 1, the lowest, to Level 6, the highest). Based on Table 1, 68.6% of
students in Indonesia attained Level 1 (i.e., they were proficient below Level 1 and
at Level 1). Based on Table 2, 71.9% of students in Indonesia attained Level 1 (i.e.,
they were proficient below Level 1 and at Level 1). Based on Table 1, 31.4% of
students in Indonesia attained Level 2 or higher (i.e., they were proficient at Levels
2, 3, 4, 5, or 6). Based on Table 2, only 28.1% of Indonesia's students attained Level
2 or higher. According to OECD (2019), students who scored below Level 2 are
considered “low-achieving students”; thus, most Indonesian students fall into this
category.
Table 2: Indonesian student performance in mathematics (OECD, 2019)

All students
Below Level 1 Level 2 Level 3 Level 4 Level 5 Level 6
Level 1
Education system % SE % SE % SE % SE % SE % SE % SE
Indonesia 40.6 (1.6) 31.3 (1.2) 18.6 (1.0) 6.8 (0.7) 2.3 (0.5) 0.4 (0.2) 0.0 (0.0)
The description of proficiency mathematics performance at Level 5 are: (a)

students can develop and work with models for building complex berth,
identifying constraints and specifying laying claims; (b) students can select,
compare, and evaluate appropriate problem-solving strategies for transaction
with complex job related to these models; (c) students at this grade can work
strategically using broad, well-developed thinking and reasoning science,
appropriate linked internal representation, symbolic and formal characterization,
and brainstorm about these situations; (d) students have begun to develop the
ability to reflect on their work and to communicate determination and rendering
in written form; and (e) students master a high level of conceptual understanding
and mathematical reasoning (OECD, 2019). Based on Tables 1 and 2, Indonesian
students' performance at Level 5 was only 0.6% and 0.4% for 2015 and 2018,
respectively; this means that they did not yet have maths skills at Level 5.
The description of proficiency mathematics performance at Level 6 are: (a)

students can conceptualize, generalize, and use information based on their
investigations and moulding of coordination compound job situations and can
use their cognition in relatively non-standard contexts; (b) students can linkup
different information sources and representations together and move flexibly
among them; and (c) students at this level are capable of advanced mathematical
thinking and reasoning. Based on Tables 1 and 2, Indonesian students'
performance at Level 6 was 0% for both 2015 and 2018; this means that their math
skills at this level need to be explored and developed.

268
To accomplish this goal, mathematical thinking ability must be nurtured at the

earliest stages, from basic to secondary education. Mathematical thinking
increases the capacities of making estimations and rounding error, measure and
constructing, collecting and handling information, representing and interpreting
data, recognizing and representing relationships mathematically, using algorithm
and relationship, solving trouble, and making decisions among students. When
mathematical thinking is at a high grade, students manifest a positive position
toward mathematics, encouraging them to approach—and solve—all
mathematics problems.
Many researchers in maths education assert the importance of such thinking in

assessing the quality of mathematics learning (Cai & Jiang 2017). Focusing on
students’ mathematical thinking remains a powerful mechanism for bringing
pedagogy, maths, and student understanding together. High-quality
mathematics instruction includes three aspects: the teacher's role, classroom
discourse, and mathematical tasks (Munter, 2014). Concerning the first aspect,
there are three teachers’ roles: (a) teacher should engage with educatee in
mathematical argument; (b) they should play a proactive role in supporting and
scaffolding students’ talk by utilizing students’ explanation and questions as
lesson content, choosing appropriate moment to share essential information such
as conventions and alternative methods or articulating important ideas in
students’ methods; and (c) teachers should ensure that “the responsibility for
determining the validity of ideas resides with the classroom community” and not
solely with the teacher or the textbook (Munter, 2014).
Moreover, one of the professional skills that mathematics teachers should

develop is professional attention. The professional noticing pr professional
attention demands that instructor attend to students’ thought, interpret
their thinking, and decide how to respond based on their own assertions.
Noticing student thinking is an important part of the teacher’s skill set and
in particular, professional noticing about mathematical thinking has
garnered the attention of many researchers (Lee, 2018). By adopting this
approach, teachers facilitate the learning that in turn leads to the
development of mathematical thinking. As they are expected to provide
erudition programs from kindergarten through grade XII, teachers or
instructors can enable students to recognize reasoning and proofs as basic
aspects of mathematics; then further make and investigate mathematical
conjectures; also develop and evaluate mathematical arguments and
proofs; and choose and use various types of reasoning and verification
methods (NCTM, 2017b). Due to the PISA data, NCTM statement, and
statements of the experts above, this study conducted on fifth-grade elementary
school students. These students are accustomed to developing mathematical
thinking naturally and gradually in their classroom environments.
Mathematics teachers’ skills for eliciting students thinking is crusial because those
skills as a foundation for teaching to help the students get deeply understanding
about mathematical ideas. Eliciting student thinking is a core teaching practice

269
that is useful in many contexts and content areas (Shaughnessy & Boerst, 2017),
using student thinking to inform the teacher instructional decisions and
interactions with students in ways that support student learning (Teuscher et al.,
2016). Specifically, learning mathematics equips the student with the ability to
develop critical, logical, and abstract thinking skills as well as the confidence to
analyze and solve problems in school and daily life situations through the use of
creative strategies (Nurlu, 2017). As stated by Harel and Soto (2017) that the way
of thinking for each person in terms of making mathematical conclusions and
deductions, both as an individual and as a community member, always develops
naturally and gradually, with cultural and social interventions. Constructivism
views mathematical learning as an active mental construction and understanding
process, where students construct the intellectual, knowledge, and understanding
abilities not given by the teacher (Lui & Bonner, 2016). Therefore, mathematics
teachers must design classes. Mathematics teachers have become ‘designers’, or
act as ‘partners’ in the design of curriculum materials not as a ‘implementers’ of
curriculum materials (Jones & Pepin, 2016). Teachers design and interact with
curriculum resources to prepare and set up their teaching in class. They also work
with colleagues in school or across schools in local, regional, or international
professional development collectives to design and adapt curriculum materials
for their own teaching and that of their colleagues (Pepin, Gueudet & Trouche,
2017). So that students can construct their knowledge through inquiry and
discovery, thereby enabling them to develop mathematical creativity. In addition,
the teacher provides scaffolding to help with the development of meaningful
knowledge. According to the constructivist paradigm, teachers create the context
of mathematics learning and pose the questions that will guide students to
thinking actively and arguing logically. The mathematical learning activities best
suited to this paradigm are probing inquiries and/or discussions; investigations,
explorations, and/or discoveries; learning from peers and groups; real-world
applications; learning games; using manipulations; and visual representations
(Lui & Bonner, 2016).
In learning practice, the teachers’ efforts to elicit student thinking will coincide
with their interpretations and responses to such thinking. Teachers help the
students bring up ideas; they then make interpretations to pose additional
questions to elucidate the students’ thinking process, giving responses that
encourage the students to learn (Shaughnessy & Boerst, 2018). This ability, teacher
noticing, is referred as a professional vision or lens through which teachers come
to view teaching. In this regard, the teacher needs to have the ability to respond
to students’ thinking during the learning experience. Teacher professional
noticing or attention toward children’s mathematical thinking involves three
simultaneous processes: attending to children’s strategies of using their ability;
interpreting student responses, work, and solution methods; and deciding, at that
moment, how to respond based on student understanding.
In the classroom setting, teachers use various methods to help their students
develop mathematical thinking skills. As many as 76% of questions asked by
elementary school teachers (two second grade, one third grade) in Southern
California are four types of questions (i.e., general question, specific questions,

270
probing sequences of specific questions, and leading questions, 67% of

explanations from second and third grade students in Southern California were
true and complete, and 82% of students’ explanations were incomplete,
ambiguous, and unclear. Thus, even though the teachers provided a stimulus to
encourage student thinking, the further direction was needed to make student
responses clear, complete, and unambiguous.
Teachers of mathematics in schools must understand the mathematics being

taught and be cognizant of students’ mathematical thinking processes (Carpenter
& Lehrer, 1999). To this end, their planning and implementation of mathematics
material should open discourse concerning students’ thinking and involve
students’ active participation in learning. Mathematics learning is designed to
incorporate several activities to find concepts, ideas, procedures, or principles.
Thus, the teacher's activities act as a stimulus to facilitate an improvement in
students’ thinking abilities (Pratiwi, Herman & Lidinillah, 2017). However,
because it is the teacher’s responsibility to develop these activities, organizing a
mathematics class is not easy (Anthony, Hunter & Hunter, 2015). The teachers
have to provide challenging mathematical activities that involve the students in
asking, justifying, and reflecting, thereby providing a stimulus for fifth-grade
students to think.
The research question associated with this study takes two forms: viz. first, what
a fifth-grade elementary school teacher uses stimulus for developing students’
mathematical thinking abilities; and second, what is the stimulus
recommendation of a fifth-grade elementary school teacher for developing
students’ mathematical thinking abilities. These formulations also become the
limitation of this study. As the questions indicate the uniqueness of mathematic
teachers undertaking a stimulus for their students’ mathematical thinking, any
discussion related to students' achievement by applying the stimulus is therefore
excluded in this study.
This study illustrates how teacher stimulus brings up students’ mathematical

thinking in elementary school. Furthermore, it serves as a reference point for the
second year of research. Various thought stimuli were implemented to develop
students' students’ mathematical thinking and test their effectiveness.
2. Methodology
2.1 Research Design
The research was conducted from May 2018 to September 2018. The research
involved 9 teachers of grade 5 elementary school from 9 different schools (in phase
1) and 2 of those 9 teachers (in phase 2) in Gunungpati district, Semarang
Indonesia. This study used a descriptive qualitative method and occurred in the
academic year 2018/2019. Research activities were divided into several phases.
The first phase was a discussion group forum with fifth-grade elementary school
teachers, which took place on Saturday, May 12, 2018. Nine teachers participated,
and there was no selection activity in UPTD (Unit Pelaksana Teknis Daerah)
Gunungpati Semarang. The teachers filled out a questionnaire to illustrate the
thought stimulus they had done in class.

271
The second phase involved the research conducted at two elementary schools.
This phase described the fifth-grade elementary school teacher's stimulus for
developing the students’ mathematical thinking abilities that occurred in the field.
The elementary schools involved were Plalangan 01 and Pakintelan 01 public
elementary school in Gunungpati Semarang. Observation of the implementation
of mathematics learning in each of the schools was carried out four times during
face-to-face learning in August–September 2018. The result of the second phase is
a triangulation of the result of the first phase.
The final result involved descriptions of the stimulus form of a fifth-grade

elementary school teacher on mathematics learning in developing students’
mathematical thinking abilities. The third phase was the recommendation of a
thinking stimulus, which was suitable for mathematics learning for fifth-grade
elementary schools. The research phases are presented in Figure 1 below.
First Phase Description about the

Questionnaire stimulus teachers had
Discussion group about thinking done to facilitate
forum (9 teachers of stimulus done students’ mathematical
by the teachers thinking
fifth grade)
Technical
triangulation
Second Phase Passive

participation
Description about
Research at 2 observation
about thinking the stimulus
elementary schools (2
stimulus done by teachers had
teachers)
the teachers done to facilitate
students’
mathematical
Descriptions of the stimulus form of a fifth-grade
elementary school teacher on mathematics learning in
developing students' mathematical thinking abilities
Third Phase Recommendation of thinking stimulus, which

was suitable to be applied in mathematics
learning for fifth-grade elementary schools
Figure 1: The research phase
2.2 Instruments
The research subjects were fifth-grade teachers of state elementary schools in the
Regional Technical Implementation (Unit Pelaksana Teknis Daerah) in
Gunungpati, Semarang. Both methods of sampling including purposive and
snowball sampling techniques were used. For phase 1 activities, research subjects
were determined by purposive sampling, a data source sampling technique with
specific considerations. Nine fifth-grade elementary school teachers were selected

272
from nine different elementary schools, with the following considerations:

(1) In the Teacher Working Group in the Regional Technical Implementation
(Unit Pelaksana Teknis Daerah) Gunungpati Semarang, there are four
clusters (gugus): Dewi Kunthi, Laras Asri, Drupadi, and Srikandi. Nine of
the selected fifth-grade elementary school' teachers are teachers in state
primary schools in the Dewi Kunthi cluster and the Laras Asri cluster.
(2) The school's location is close to the university and needs to be involved in
higher education research activities.
(3) Advice from the head of the Dewi Kunthi cluster.
(4) Nine fifth-grade teachers from nine public elementary schools were
willing and open to be the subjects of research.
For phase 2, the research subjects were determined by snowball sampling.

Snowball sampling is done by searching for subjects that are likely to get
information in phase 2. Using snowball sampling, two fifth-grade elementary
school teachers were selected out of the nine fifth-grade elementary school
teachers involved in phase 1. Both of these fifth-grade teachers volunteered to
serve as research subjects in phase 2.
Data collection methods used in this study were open-ended questionnaires and
passive participation observations. The questionnaires were teacher stimulus
questionnaires in the first phase. Passive participation observations were carried
out by observing teacher learning in the second phase of the study through
participatory observation. The researcher presents in the learning activities
carried out by the two elementary school teachers each during four mathematics
learning meetings in class. With this participant observation, the thought stimulus
data provided by the teacher is clearly visible. The type of participatory
observation of this study is passive participation observation—i.e., the researcher
comes to the place of the observed activity (fifth-grade of Plalangan 01 public
elementary school and Pakintelan 01 public elementary school), but is not
involved in the teaching and learning activity.
2.3 Data Validation

Data are required to meet standards of credibility, transferability, dependability,
and confirmability (Siswono, 2007). In this study, credibility of the data was
carried out by persistent observation of the implementation of learning in
Plalangan 01 public elementary school and Pakintelan 01 public elementary
school. The research also used technical triangulation to validate the data i.e.,
comparing the data from the questionnaire and observation results, as technical
triangulation aims to obtain the data regarding the types of teacher stimulus. For
the other activity, the researcher held discussions with the research team.
Transferability described, in detail, the stimulus types given by the teacher in
mathematics learning. Dependability was carried out by an audit technique that
maintains the honesty and the accuracy of the researcher's perspective. This
research satisfied confirmability because it was based on data exploration of the
truth.

273
2.4 Data Analysis

This study used descriptive analysis to describe the stimulus types given by the
teachers in mathematics learning. Miles and Huberman model was used to
analyze the data. This model involved data reduction, data display, and
conclusion phases. Interview data analysis was performed with data reduction,
data presentation, and conclusion drawing steps. Reducing data means
summarizing, choosing the main things, focusing on the important things, looking
for themes and patterns, and discarding unnecessary ones (Sugiyono, 2016, p.
338). All data information obtained using questionnaires and observation sheets
in phases 1 and were collected, then reduced to obtain the data needed for the
analysis process; data that did not support the analysis process was discarded.
This study's reduction activity was to simplify the questionnaire and observation
sheet results into a suitable arrangement of languages and then transform them
into notes, discarding those found unnecessary. The presentation of data is done
in brief descriptions, charts, relationships between categories, and others. The
reduced data will be organized and arranged in a relationship pattern through the
presentation of data to be more readily understood. The presentation of data will
make it easier for researchers to understand what is happening and plan the next
steps based on what has been understood. The research data will be presented
tabularly to make it easier for readers to understand; specifically, this presentation
includes data classification and identification. Conclusions are drawn using the
results of questionnaires and observations to coalesce the stimuli thought by the
teacher.
3. Research Findings
3.1 Finding of The First Phase
Regarding the Act of the Government Regulation No. 74 of 2008 (Government
Regulations, 2008) mandated that teachers must have a minimum qualification
academic S-1 or D-IV, competencies (pedagogic, personality, social, and
professional) as learning agents, and educator certificates. Therefore, teachers
have to continuously improve their competencies continuously improve their
competencies through various training, scientific writing activities, workgroup
meetings, and workshop discussions, including the teacher working group. The
teacher working group, one of the professional forums for teachers (both class and
subject teachers) in a regency/city/district/studio/school group, is a
nonstructural school organization that is independent, family-based, and has no
hierarchical relationship with any other institutions (Al Rasyid, 2017). The teacher
working group is a forum for the teachers. Working group activities included
exchanging ideas for solving learning problems, sharing learning information,
and discussing the challenge of difficult learning material such that a mutual
solution is needed. Thus, the teacher working group serves as a strategic effort to
improve teacher performance and teaching ability.
Due to the reasons above, this study involved teachers working group to obtain
the data. The first phase of the research was a discussion group forum with nine
teachers of the fifth-grade elementary school in a teacher working group setting.
Teachers were given questionnaires to find out teachers’ stimulus types in

274
thinking development. Based on the questionnaire results, the teacher's stimulus

most often given in mathematics learning is presented in Figure 2 below. Based
on Figure 2, student utilization of technology, self-made manipulative props,
math games, investigations, and adventure activities outside the classroom were
not optimal.
Teacher stimulus percentage based on a questionnaire
100
90
80
70
60
50
40
30
20
10
0
Figure 2: The percentage of teachers’ stimulus by questionnaire
3.2 Finding of The Second Phase

The second phase was conducted in two elementary schools, namely Plalangan
01 and Pakintelan 01. This phase was conducted to confirm the results of the first
phase. The questionnaire result was technically-triangulated with the learning
result observation in the field. Based on field research in Plalangan 01 public
elementary school and Pakintelan 01 elementary school for four occasions of
learning, data on the teachers' stimulus follow. The fifth-grade mathematics
teachers in both schools always used a thinking stimulus that asked guiding
questions. The teacher gave the guiding question to help the students identify
concepts and strengthen their understanding of those concepts.
The Plalangan 01 elementary school teacher taught multiplication and division of
fractions and percentages by providing a stimulus in the form of a song. The song
used is “Menanam Jagung” (translation: “Planting Corn”) by Mrs. Sud (Figure 3a,
b), and the lyrics of “Pecahan” song were changed by the teacher to embed the
concept of fractions (Figure 4a, b).
Ayo kawan Kita Bersama Come on, friend, we’re together

menanam jagung di kebun kita planting corn in our garden
ambil cangkulmu, ambil pangkurmu take your hoe, take your pick
kita bekerja tak jemu-jemu we work not get bored
cangkul, cangkul, cangkul yang dalam hoes, hoes, deep hoes
tanah yang longgar, jagung kutanam loose ground, I plant corn
a b
Figure 3: a) Lyrics of “Menanam Jagung” song, b) Lyrics of “Planting Corn” song

275
Ayo kawan kita bersama Come on, friend, we're together

Membagi angka jadi pecahan Divide numbers into fractions
Atas pembilang, bawah penyebut The top is numerator; the bottom is a
denominator
Janganlah sampai terbalik-balik
Don't turn it upside down
Kali dengan bilangan yang sama akan
Times with the same number will be
jadi pecahan senilai equivalent fractions
Bagi dengan bilangan yang sama akan Divide by the same number will be
jadi pecahan senilai equivalent fractions
a b
Figure 4: a) Lyrics of “Pecahan” song, b) Lyrics of “Fractions” song
Also, the teacher provided a stimulus in the form of problem posing, giving
several numbers in the forms of integers, decimal numbers, and percents (Figure
5).
3;5;6;8;12;15;20;25; 20%; 25%; 30%; 45%; 0.20; 0.25; 0.45
Figure 5: Numbers for problem posing
In learning about addition and subtraction of fractions, the teacher presents

contextual math problems unusually. As a variation in giving the mathematical
problem to develop thinking, the teacher provided several contextual problems
about fractions presented using Plickers application. Plickers stands for Paper
Clickers. Plickers technology is a web-based application that uses synchronization
between a smartphone and a computer or PC. The free Plickers application allows
teachers to get answers from students quickly. At the beginning of learning, the
teacher gives a special paper containing the answers A, B, C, and D. After the
teacher raises the questions and students finish working on them, students are
asked to show the teacher the Plickers paper with the answers they selected above.
After that, the teacher only has to scan the answer paper using the smartphone's
camera. Students’ answers and true mistakes will automatically appear on the
computer screen. Figure 6 below shows some of the teacher's problems in class
and the name of the student who worked on it.

276
Figure 6: Display of mathematical problems using Plickers. The other facilitation

technology used by the teacher for teaching distance, time, and speed units were
PowerPoint and miscellaneous learning videos
The teacher of Pakintelan 01 public elementary school taught distance using the
song “Naik Ke Puncak Gunung” (translation: “Climb up to the Mountain”). The
songwriter was Mrs. Sud, whose lyrics were changed and became the song
“Tangga Ukuran” (translation: “Units Stairs”). The teacher stimulated students'
thinking about changing the units of distance. Original lyrics of the song “Naik
Ke Puncak Gunung” (and then translated as “Climb up to the Mountain”) as well
as the song “Tangga Ukuran” (and then translated as “Units Stairs”) are presented
below (Figure 7a, b; Figure 8a, b, respectively).
Naik-Naik Ke Puncak Gunung Climb up to the top of the mountain

Tinggi-Tinggi sekali.......................2x So high ................ 2x
Kiri Kanan Ku lihat saja Look left, look right, I’ll see
Banyak pohon cemara.............2x Lots of pine trees ...................... 2x
a b
Figure 7: a) Lyrics of “Naik Ke Puncak Gunung” song, b) Lyrics of “Climb up to the
Mountain” song
Naik-naik Tangga ukuran Climb up to the unit stairs

Tinggi-tinggi sekali......................2x So high ...................... 2x
Mili senti desi dan meter deka Milli centi deci and meter deca hecto
hekto dan kilo................2x and kilo ................ 2x
Kalau naik bagi sepuluh If it goes up divided by ten
Turun kali sepuluh.......................2x Goes downtimes by ten ................2x
a b
Figure 8: a) Lyrics of “Tangga Ukuran” song, b) Lyrics of “Units Stairs” song

277
To train the students in understanding distance units, the teacher provided a

stimulus in the problem-posing form. Students in the groups made questions as
well as the answer keys. Also, the teacher gave a stimulus in the form task of
contextual problem posing. The following data in Figure 9 resulted from students’
was the result of students' work on the contextual problem of changing the
distance unit.
1. The distance from Winong village to Kepil village was 7 kilometers. The paved
road is 5,000 meters. How many meters did the rest of the way go from Winong to
Kepil that unpaved?
2. Fadil has a 500 centimeters band. Then, Nofal asks for a 200 centimeters long
ribbon. Fadil's father asks for a 100 centimeters long ribbon. How many meters of
the rest of the band did Fadil have?
3. Mr. Eko has 300 centimeters of wood. Mr. Eko cut the wood into two equal lengths,
and then the mother used one part of the wood as firewood. How many decimeters
of the rest of the wood does Mr. Eko have?
Figure 9: Contextual problem-posing made by the students
3.3 Finding of The Third Phase

Based on the observation in the field, the teachers' thinking stimuli are problem
posing, asking guiding questions, facilitating technology (learning videos,
PowerPoint, and Plickers), and using songs. Based on technical triangulation, the
teachers’ stimuli were problem posing, asking guiding/challenging questions,
facilitation technology (learning videos/PowerPoint/Plickers), and song (Table
3).
Table 3: Triangulation for teacher stimulus based on questionnaire and field research
(Plalangan 01 and Pakintelan 01)
Questionnare Plalangan 01 Pakintelan 01
Open-ended task - -
Problem posing Problem posing Problem posing
Asking questions Asking guiding Asking guiding
(guiding and challenging) questions question
Facilitating technology Facilitating technology -
(Plickers, PowerPoint,
learning videos)
Manipulative prop that has existed - -
- Song Song
Based on the research in the field, the teachers' thinking stimuli are problem
posing, asking guiding questions, facilitating technology (learning videos,
PowerPoint, and Plickers), and using songs. The teachers' teachers’ stimuli are
problem-posing, asking guiding/challenging questions, facilitation technology
(learning videos/PowerPoint/Plickers), and song based on technical
triangulation.

278
4. Discussion
Mathematics learning activities in the classroom carried out by students and
designed by the teacher. Teachers need to encourage learning as a cognitive
process of students. The teacher points out mistakes, mismatches, cognitive
conflicts so that students gain a deep understanding of mathematics. The teacher
encourages representation and encourages students to think about thinking.
Strategies that can be done are demonstration and teaching, connecting and
exploring-giving examples, encouraging students to test ideas, discussing and
using mathematical language, encouraging representation and visualization,
problem posing, encouraging predictions and providing feedback, raises errors
and misconceptions, models and encourages reflection on his thinking, and so on.
The teacher provides a stimulus in the form of tasks, activities, activities to help
students develop their thinking. The first teacher’s stimulus is problem posing.
Problem posing helps the students to understand mathematics (Cai, Hwang, Jiang
& Silber, 2015), and the students' learning styles are changed from passive to
active with its use (Chinese Ministry of Education, 2011). Problem posing
conducted by the teacher in this study was something that can be freely done by
friends i.e., the students created the story problem about the distances (see Figure
9). Students in this research were also asked to formulate the question of the
problem with the question not being stated, and were provided many integers,
fractions, and percentages in order to do so (see Figure 5). This method seemed
effective for enabling students to make math problems using a variety of basic
mathematical operations (adding, subtracting, multiplying, and dividing).
In this study, the two teachers only used free situation problem posing as shown
in Figures 5 and 9. The instruction of the two teachers was "Please create free
questions using numbers in the form of fractions, percent, decimals, and integers
and then answer it". The numbers provided by the teacher are shown in figure 5.
Then, "Make daily life questions about the distance to the students". The results
of the questions made by students are shown in figure 9). Teachers need to learn
and try a semi-structured situation or structured situation problem posing.
Various strategies for problem posing can be conducted by the teacher to develop
students’ thinking; in particular, it is a helpful tool for understanding the thinking
ability of students’ regarding mathematics. By understanding it while posing
problems would able to inform the instructional choices that teachers make when
teaching in this manner (Xu, Cai, Liu & Hwang, 2019). This study's first
recommendation was that the teacher gives the stimulus of problem posing with
another strategy. In other words, making the problem based on the context, and
the calculation given, making the problem based on the solution provided,
making the sub-problem in finishing the bigger problem, and making the question
a “what if” scenario. Based on this research, the results of the teacher's
mathematical thinking stimuli were problem-posing, asking guiding/challenging
questions, facilitation technology (learning videos/PowerPoint/Plickers), and
song.
The teacher played an important role in mathematical learning as a facilitator by

asking the question. Submitting teacher questions to students helped students
develop their level of thinking. Moreover, the teacher's quality of the questions

279
affected the development of students’ thinking. This question was the main
component in the mathematical learning interaction (Hähkiöniemi, 2017) and
influenced the students’ learning results. The questions asked by both teachers in
this study were to guide concept discovery, and these questions subsequently
guided the students’ conceptual understanding.
The questions given by the teachers included those that explored a fact or student
understanding, called factual questions, and those that gave hints or scaffolding,
called guiding questions. Both teachers asked about fractions, distances, times,
and speeds in the form of a short answer, low-level question exploring the
associated fact, rule, and procedure. One of the criteria of guiding questions is that
it asks for a specific answer or for the next step of a solution after facing problem.
The following are examples of teachers’ guiding questions to help students solve
the problem by such criteria (Figure 10).
Use matchsticks to make a triangle pattern as follows
1st Pattern 2nd Pattern 3rd Pattern 4rd Pattern
Investigate
Figurethe10:
relationship
Example ofbetween
problemthe number
used of triangles
for teachers’ and the
guiding number of
question
matchsticks
To answer the problem above, the teacher asks guiding questions. The following
are examples of question and answer activities between the teacher and students
(Figure 11).
Teacher : How to solve the problem?
Student : (silent)
Teacher : Look at the picture above. How many triangles can be made in
each pattern?
Student : Pattern 1 has 1 triangle, pattern 2 has 2 triangles, pattern 3 has 3
triangles, and pattern 4 has 4 triangles
Teacher : Right. Then pay attention to the number of matchsticks in each
pattern. How many matchsticks are used in each pattern?
Students : Pattern 1 there are 3 matchsticks, in pattern 2 there are 5
matchsticks, pattern 3 there are 7 matchsticks, and in pattern 4
there are 9 matchsticks.
Teacher : Good. Then how many triangles can be made in the 5th pattern?
The 6th pattern? 7th pattern?
Student : (draw 5th pattern, 6th pattern, 7th pattern)
Figure 11: The examples of question and answer activities between the teacher and
students

280
Teacher : Yes, the picture is very correct. Okay, now count how many
matchsticks/sticks were used in the 5th pattern? The 6th pattern?
7th pattern? (5th pattern is called 5th term, 6th pattern is called 6th
term, etc.)
Students : In the fifth pattern, there are 11 matchsticks, in the sixth pattern
there are 13 matchsticks, and in the seventh pattern there are 15
matchsticks.
Teacher : Very smart. Now, write down the numbers that show the number
of triangles in sequence. Then write down the numbers that
indicate the number of matches/sticks in sequence.
Student : (writing)
Number of triangles 1 2 3 4 5 6 7 ... n
Number of matchsticks 3 5 7 9 11 13 15 ... 2n + 1
Teacher : Right. Now, consider the relationship between the number of
triangles and the number of matchsticks.
Student : if there are many triangles, where many matchsticks become 2n +
1
The low-level question aimed to test the students’ understanding. Most of the
teacher's questions only needed a short answer and explored the related fact, rule,
and procedure. Both teachers rarely asked a high-level question asking the
students to give the thinking explanation. Much of the research showed that most
of the teacher’s questions were low-level questions and that only a few were a
high-level question (Alkhateeb, 2019). Most teachers asked questions that
measured students’ thinking aspects of knowledge and understanding, and few
asked questions that measured thinking aspects of analysis and evaluation
(Alkhateeb, 2019). In this research, both teachers rarely asked students a question
that required them to explain/elaboration/justification, which is referred to as a
probing question. The percentage of sixth-grade mathematical teachers asking
probing questions ranges from 17% to 42%. Viirman (2015) noted that the teacher
often posed routine inquiries in the form of control questions, asking for facts,
inquiries, and rhetorical questions in the mathematical learning context to engage
students in the learning process.
In this study, the teacher used the song stimulus. The teacher arranged the song
“Naik-naik Ke Puncak Gunung” (translation: “Climb up to the Mountain”) by
Mrs. Sud to introduce the concept of distance units. In this way, the teacher
created the thinking stimulus to make the abstract distance unit concept easier for
students to understand. The use of songs in learning helps students remember
because it is an effective way to store information for a long time (Bahrami,
Izadpanah & Bijani, 2019). Thus, the song can be used to introduce various
mathematical concepts (Noviyanti & Suryadi, 2019). As singing is a play activity
that can be integrated into education and used by the teacher to teach basic
mathematical learning, teaching mathematics to the children can be conducted by
using a singing technique (Rosli & Lin, 2018). Mathematics teachers have used the
chant “Pleases Excuse My Dear Aunt Sally” for helping students learn about the
order of mathematical operation—i.e., adding, subtracting, multiplying, dividing,

281
squaring, and so forth. Moreover, children develop the ability to understand and
manipulate models using simple rhythms and chants.
The teacher’s stimulus in the form of asking a question has various purposes
(Warshauer, 2015). These include serving as a means of discussion interaction
between the teacher and students, and allowing the students to organize
mathematical ideas in finishing a mathematical task. The question is asked
sequentially and is carefully developed and built based on the students’ ideas,
helping evaluate students’ thinking. The stimulus form's recommendation for
developing mathematical thinking ability was asking the probing question that
explains and sheds light on the students’ thinking process. This is because the
probing question required intellectual effort and led to an increase in student
learning to a greater degree than the other type of question. Given that the probing
question can be defined as the question that asks students to explain or justify
mathematical justification (Kosko, 2016), its influence on students’ mathematical
achievements tends to be higher.
Asking the probing question in mathematical learning was recommended to

construct in the inquiry environment because, in that environment, the teacher
can bring up the students’ high-level thinking. The learning environments that
were planned based on the modeling perspectives had a positive influence on the
teacher’s ability to qualitatively ask different questions to explore thestudents’
high-level thinking (Aydogan Yenmez, Erbas, Cakiroglu, Cetinkaya & Alacaci,
2018). The teacher developed a math-talk community by increasing the use of
probing questions, which in turn reduces teacher effort in providing explainations
while increasing support for students’ mathematical autonomy.
Mathematical learning in elementary education must be carried out so that

mathematics becomes beneficial for children. The strategy that can be conducted
is through the use of play and games. Play activity allows children to interact with
the concrete object to build their knowledge. Playing gives children valuable
experience i.e., children experience using language and symbols, create and carry
out the rules, and learn abstract thinking. As a result, children’s cognitive abilities
(attention, memory, and problem solving) increased. The game is one of the play
activities that is useful for developing children’s mathematical knowledge. Games
are exciting, and they provide the structural experience necessary to helping
children to achieve learning purposes (Ramani, Daubert & Scalise, 2019); they are
also socially interactive and utilize children’s interest (Hassinger-Das et al., 2017).
Games facilitate learning for children with a variety of mathematical abilities,
enabling them to interact and learn from each other; in addition, they increase
motivation and build a positive attitude towards mathematics (Ramani et al.,
2019). For this reason, the recommendations of further thinking stimulus for fifth-
grade elementary school students are play and games.
The Teacher Working Group is a forum for elementary school teachers'

professional activities at the cluster or sub-district level, consisting of several
schools. Teachers need to improve their level of professionalism, especially in
terms of developing stimuli to thinking and applying them to classroom learning.

282
Teacher professional improvement programs that need particular attention are

competency improvement through training, in-service training programs, and
increased experience through an internship program or on-the-job training. The
bottom-up training model, which is based on the real needs and problems of
mathematics teachers in the field, involves developing stimulus-thinking problem
posing, asking the guiding question, Plickers/PowerPoint, and Sing a Song; thus,
this training helps teachers develop mathematical activities that facilitate students
thinking mathematically. The intended training can also be carried out
independently through the mathematics teacher community, teacher working
groups, or other programs that do not always depend on central or regional
government programs.
One of the training models available is INNOMATTS. Since 2013, Asikin, Junaedi
and Cahyono (2015) have developed the INNOMATTS model as a training model
for Mathematics teachers. The strategy for implementing INNOMATTS is
threefold. First, INNOMATTS training can be carried out as part of routine
activities scheduled at the Teacher Working Group. A KKG can also carry it out
under the auspices of the Education Foundation. Second, INNOMATTS is
designed and implemented as a cycle model (as an application of the Deming P-
D-C-A/Plan-Do-Check-Act cycle), with activities based on group and individual
work. For example, group-based activities (activities carried out in clusters)
comprise teachers together with other members in one cluster designing (plan)
learning tools, and followed up by implementing (do) learning, one as the other
model teachers observing and observing results are discussed together again
(check). Furthermore, each teacher implements (action) these learning strategies
in their respective schools without being observed by peers, and repeatedly
(according to the program that is designed as a whole). Third, mentoring by tutors
occurs during the training process, both in the cluster and individual activities.
Therefore, INNOMATTS can be applied in the teacher working group setting to
encourage and foster professional development.
5. Conclusion
The in-field stimuli that a fifth-grade elementary school teacher in mathematical
learning uses to develop students’ mathematical thinking abilities were diverse.
Teachers used problem posing, asking guiding questions, facilitating technology
in learning videos/PowerPoint/Plickers, and Sing a Song. A fifth-grade
elementary school teacher would use the stimulus forms to develop students’
mathematical thinking abilities in mathematical learning comprised asking the
probing question, play, and games. Further research based on this study result
(second year) was developing valid, practical, and effective learning tools to
accommodate the thinking stimulus for asking the probing question, problem
posing, and mathematical games in the fifth-grade elementary school.
Furthermore, the results of the second year’s research were widely disseminated
to teachers. Primary school teachers who are part of the teacher working group
also need to improve their professionalism, especially concerning developing
stimulus thinking and applying it in learning. Therefore, the follow-up is the
management of learning in the form of INNOMATTS training. Since Asikin et al.
(2015) state that the INNOMATTS training model can improve the competition

283
and character of mathematics teachers, this model's inclusion would be helpful in

follow-up management.
6. References
Alkhateeb, M. A. (2019). Assessing eighth-grade mathematics teachers and textbook in
embodying thinking levels. International Journal of Instruction, 12(1), 371-388.
doi:10.29333/iji.2019.12125a
Al Rasyid, H. (2017). Fungsi kelompok kerja guru (KKG) bagi pengembangan
keprofesionalan guru sekolah dasar [The function of the teacher working group
(KKG) for the development of elementary school teacher professionalism]. Sekolah
Dasar: Kajian Teori dan Praktik Pendidikan, 24(2), 143-150. Retrieved from
http://journal2.um.ac.id/index.php/sd/article/view/1355/715
Anthony, G., Hunter, J., & Hunter, R. (2015). Supporting prospective teachers to notice
students' mathematical thinking through rehearsal activities. Mathematics Teacher
Education and Development, 17(2), 7-24. Retrieved from
https://files.eric.ed.gov/fulltext/EJ1085900.pdf
Asikin, M., Junaedi, I., & Cahyono, A. N. (2015). Pengembangan model pelatihan
innomatts untuk meningkatkan kompetensi dan karakter guru matematika
[Development of Innomatts training model to improve the competence and
character of mathematics teachers]. Jurnal Penelitian Pendidikan, 32(2), 1-7.
doi:10.15294/jpp.v32i2.5713
Aydogan Yenmez, A., Erbas, A. K., Cakiroglu, E., Cetinkaya, B., & Alacaci, C. (2018).
Mathematics teachers’ knowledge and skills about questioning in the context of
modeling activities. Teacher Development, 22(4), 497-518.
doi:10.1080/13664530.2017.1338198
Bahrami, Z. N., Izadpanah, S., & Bijani, H. (2019). The impact of musical mnemonic on
vocabulary recalling of Iranian young learners. International Journal of
Instruction, 12(1), 977-994. doi:10.29333/iji.2019.12163a
Cai, J., & Jiang, C. (2017). An analysis of problem-posing tasks in Chinese and US
elementary mathematics textbooks. International Journal of Science and Mathematics
Education, 15(8), 1521-1540. doi:10.1007/s10763-016-9758-2
Cai, J., Hwang, S., Jiang, C., & Silber, S. (2015). Problem posing research in mathematics:
Some answered and unanswered questions. In F. M. Singer, N. Ellerton, & J. Cai
(Ed.), Mathematical problem posing: From research to effective practice. New York:
Springer. doi:10.1007/978-1-4614-6258-3_1
Chinese Ministry of Education. (2011). Mathematics curriculum standard of compulsory
education (2011 version). Beijing, China: Beijing Normal University Press.
Government Regulations. (2008). Peraturan pemerintah nomor 74 tahun 2008 tentang guru
[Government regulation number 74 of 2008 concerning teachers]. Jakarta:
Departemen Pendidikan Nasional RI.
Hähkiöniemi, M. (2017). Student teachers’ types of probing questions in inquiry-based
mathematics teaching with and without GeoGebra. International Journal of
Mathematical Education in Science and Technology, 48(7), 973-987.
doi:10.1080/0020739X.2017.1329558
Harel, G., Soto, O. (2017). Structural reasoning. International Journal of Research in
Undergraduate Mathematics Education, 3, 225-242. doi:10.1007/s40753-016-0041-2
Hassinger-Das, B., Toub, T. S., Zosh, J. M., Michnick, J., Golinkoff, R., & Hirsh-Pasek, K.
(2017). Más que diversión: el lugar de los juegosregladosen el aprendizajelúdico
[More than just fun: a place for games in playful learning]. Infancia y
Aprendizaje, 40(2), 191-218. doi:10.1080/02103702.2017.1292684

284
Kosko, K. W. (2016). Primary teachers’ choice of probing questions: Effects of MKT and
supporting student autonomy. International Electronic Journal of Mathematics
Education, 11(4), 991-1012.
Lee, M. Y. (2018). Further investigation into the quality of teachers’ noticing expertise: A
proposed framework for evaluating teachers’ models of students’ mathematical
thinking. EURASIA Journal of Mathematics, Science and Technology
Education, 14(11), 1-15. doi:10.29333/ejmste/92019
Lui, A. M., & Bonner, S. M. (2016). Preservice and inservice teachers' knowledge, beliefs,
and instructional planning in primary school mathematics. Teaching and Teacher
Education, 56, 1-13. doi:10.1016/j.tate.2016.01.015
Ministry of Education and Culture Republic of Indonesia. (2016). Salinan lampiran
peraturan menteri pendidikan dan kebudayaan nomor 21 tahun 2016 tentang standar isi
pendidikan dasar dan menengah [A copy of attachment to the regulation of the
minister of education and culture number 21 of 2016 concerning the content
standards of primary and secondary education].
Ministry of Education and Culture Republic of Indonesia. (2018). Peraturan Menteri
Pendidikan Dan Kebudayaan Republik Indonesia Nomor 37 tahun 2018 [Regulation of
the Minister of Education and Culture of the Republic of Indonesia Number 37 of
2008].
Munter, C. (2014). Developing visions of high-quality mathematics instruction. Journal for
Research in Mathematics Education, 45(5), 584-635.
doi:10.5951/jresematheduc.45.5.0584
National Council of Teachers of Mathematics. (2017a). Supporting the common core state
standards for mathematics. Retrieved from http://www.nctm.org/Standards-and-
Positions/Position-Statements/Supporting-the-Common-Core-State-Standards-
for-Mathematics/.
National Council of Teachers of Mathematics. (2017b). Principles and standards for school
mathematics: Processes. Retrieved from http://www.nctm.org/Standards-and-
Positions/Principles-and-Standards/Process/.
Noviyanti, M., & Suryadi, D. (2019). Conceptualizing mathematical knowledge for
teaching of Indonesian teacher in teaching number sense to early childhood.
Journal of Physics: Conference Series, 1157(3), 032121. doi:10.1088/1742-
6596/1157/3/032121
Nurlu, Ö. (2017). Investigation of teachers’ mathematics teaching self-
efficacy. International Electronic Journal of Elementary Education, 8(1), 21-40.
Retrieved from https://www.iejee.com/index.php/IEJEE/article/view/95/92
OECD. (2019). PISA 2018 results (Volume I): What students know and can do. Paris: OECD
Publishing.
Pepin, B., Gueudet, G., & Trouche, L. (2017). Refining teacher design capacity:
Mathematics teachers’ interactions with digital curriculum resources. ZDM, 49(5),
799-812. doi:10.1007/s11858-017-0870-8
Pratiwi, V., Herman, T., & Lidinillah, D. A. M. (2017). Upper elementary grades
students’algebraic thinking ability in Indonesia. International E-Journal of Advances
in Education, 3(9), 705-715. doi:10.18768/ijaedu.390554
Ramani, G. B., Daubert, E. N., & Scalise, N. R. (2019). Role of play and games in building
children's foundational numerical knowledge. Cognitive Foundations for Improving
Mathematical Learning, 5, 69-90. doi:10.1016/B978-0-12-815952-1.00003-7
Rosli, R., & Lin, T. W. (2018). Children early mathematics development based on a free
play activity. Creative Education, 9(7), 1174-1185. doi:10.4236/ce.2018.97087

285
Shaughnessy, M., & Boerst, T. A. (2018). Uncovering the skills that preservice teachers
bring to teacher education: The practice of eliciting a student’s thinking. Journal of
Teacher Education, 69(1), 40-55. doi:10.1177/0022487117702574
Sugiyono. (2016). Metode penelitian kombinasi (mixed methods) [Combined research methods
(mixed methods)]. Bandung: Alfabeta.
Teuscher, D., Moore, K. C., & Carlson, M. P. (2016). Decentering: A construct to analyze
and explain teacher actions as they relate to student thinking. Journal of
Mathematics Teacher Education, 19(5), 433-456. doi:10.1007/s10857-015-9304-0
Tofade, T., Elsner, J., & Haines, S. T. (2013). Best practice strategies for effective use of
questions as a teaching tool. American Journal of Pharmaceutical Education, 77(7),
155. doi:10.5688/ajpe777155
Viirman, O. (2015). Explanation, motivation and question posing routines in university
mathematics teachers' pedagogical discourse: a commognitive
analysis. International Journal of Mathematical Education in Science and Technology,
46(8), 1165-1181. doi:10.1080/0020739X.2015.1034206
Xu, B., Cai, J., Liu, Q., & Hwang, S. (2019, in press). Teachers’ predictions of students’
mathematical thinking related to problem posing. International Journal of
Educational Research. doi:10.1016/j.ijer.2019.04.005
Warshauer, H. K. (2015). Productive struggle in middle school mathematics
classrooms. Journal of Mathematics Teacher Education, 18(4), 375-400.
doi:10.1007/s10857-014-9286-3

286

Vol. 19, No. 10, pp. 286-303, October 2020
Move to Online Learning during COVID-19

Lockdown: Pre-Service Teachers’ Experiences in
Ghana
Ugorji I. Ogbonnaya
University of Pretoria, South Africa
https://orcid.org/0000-0002-6243-5953
Florence C. Awoniyi
Department of Teacher Education, School of Education and Leadership,
University of Ghana, Legon, Accra, Ghana
https://orcid.org/0000-0001-9806-1837
Mogalatjane E. Matabane
University of Pretoria, South Africa
https://orcid.org/0000-0001-7953-6729
Abstract. The coronavirus pandemic (COVID-19) has brought another

dimension to teaching and learning across the levels of education. The
lockdown imposed in many countries to curtail the pandemic forced many
institutions of learning to shift to the online mode of teaching and learning.
Using a descriptive survey research design, this study explored the online
learning experiences of pre-service teachers at a Ghanaian university
during the COVID-19 lockdown. The study focused on the pre-service
teachers’ preparedness for online learning in terms of their digital literacy
and technological devices used for online learning, their positive online
learning experiences, and the challenges they encountered learning online.
The findings suggested that the pre-service teachers were digitally literate
and mostly accessed online learning using smartphones. Besides, online
learning enabled them to communicate and collaborate actively with their
course mates and lecturers. It was found that the flexibility of online
learning increased the students’ motivation to learn. However, poor
internet connectivity, the high cost of data, erratic power supply, lack of
appropriate devices, inability to effectively manage their time, and family
interruptions were some of the challenges experienced by the pre-service
teachers.
Keywords: Coronavirus; COVID-19; E-learning; online learning; pre-

service teachers

287
1. Introduction
The coronavirus pandemic (COVID-19) has brought another dimension to
teaching and learning across the spectra of educational levels and systems in a
way that was never envisaged (Goh & Sandars, 2020; Tanabe, 2020). To minimize
the loss of teaching and learning time due to the lockdown imposed in many
countries to curtail the spread of the pandemic, many institutions of learning
adopted online modes of teaching and learning (Burgess & Sievertsen, 2020; Rose,
2020). Most of these institutions were unprepared (both in material and human
resources) for the unprecedented online teaching delivery, and most students
were mentally and materially unprepared for the switch to online learning. Many
teachers and lecturers have never been formally trained to teach online, neither
have students been trained or had prior experience of learning online. Hence,
most institutions, lecturers, and students were caught off-guard by the sudden
radical move to online education.
Effective engagement in online teaching and learning requires some necessary

information and communication technology infrastructure (hardware and
software) that most colleges and universities, especially in developing countries,
might not have. It requires teachers’ and lecturers’ knowledge and skills of not
only computer operations but also of the platforms used for online teaching and
learning. However, at many institutions of learning in Africa, some teachers and
lecturers do not have enough knowledge of how to use a computer, and many
might not know how to teach using online platforms. Additionally, most of the
students might not own computers (desktop or laptop) nor smartphones to enable
them to access online learning. Besides, lack of or erratic power supply and access
to the internet could be other challenges many people in some parts of Africa
experience. Hence, meaningful online teaching and learning during the pandemic
lockdown was a serious challenge to many students and teachers in Africa. This
study explored the online learning experiences of pre-service teachers at a
Ghanaian university. More specifically, it explored the students’ preparedness for
online learning concerning their digital literacy and ownership of technological
devices, positive online learning experiences, and challenges encountered when
learning online during the COVID-19 lockdown.
2. Background
Online learning is a complex term and there have been extensive debates about
its standard definition (Halverson & Graham, 2019). According to Watson (2008),
there is no single definition of online learning. In its broadest sense, online
learning is any access to learning through the use of some technology (Kalpokaite
& Radivojevic, 2019; Mahama, 2016; Watson, 2008). Both Watson (2008) and
Mahama (2016) describe online learning as a version of distance learning, which
improves access to educational opportunities for students who cannot attend the
traditional face-to-face classroom tuition.
Enrolments for online learning expanded rapidly in higher education in the last
two decades as the online mode of learning appeals to a diverse population of
students with different needs that traditional face-to-face classes are incapable of
meeting (Thomas, 2010). The growth of online learning not only enables
opportunities for flexible learning environments but is a global initiative to

288
provide quality education for all students, irrespective of their location (Casey,
2008). Proponents of online learning suggest that the quality of online learning is
embedded within the method of delivery, asserting that the quality of instructions
impacts learning outcomes (Figueroa et al., 2020; Mahama, 2016).
Online learning has been very effective for self-regulated learners. According to
Thompson (2010, p. 24), “Online learning is more self-guided.” The ability to self-
regulate one’s learning is critical in online learning as self-regulation allows
students to use various metacognitive and cognitive strategies to accomplish
learning objectives and goals (Greene & Azevedo, 2010; Sealander, 2016). Online
learning allows students to work at a place and time compatible with their
learning needs and enables students to have sufficient time to focus on content
and not on issues like traffic and other problems that may arise in the traditional
classroom environment (Hartini et al., 2020; Thomas, 2010).
Online learning not only offers flexibility in terms of time and place but also
provides students with expanded opportunities to register for credit-bearing
courses at universities and colleges while they are still at secondary school (Shea
& Bidjerano, 2014). Online learning is an excellent platform to allow master
teachers to teach students from different schools and opens pathways to learning
for more rural students from poor socio-economic districts with unqualified and
under-qualified teachers. Thus, teaching and learning online creates educational
opportunities for individuals who may have faced unsurpassable barriers to
study through traditional face-to-face learning (Sealander, 2016; Watson, 2008).
While there are many benefits of online learning, the absence of quality interaction
when learning online can negatively affect some aspects of learning, as social and
emotional aspects of learning are equally important as technical information
(Donlevy, 2003; Traxler, 2018). Contrary to this view, some researchers (e.g. Fogel
& Nehmad, 2009; Ogbonnaya, 2019) argue that the use of social media platforms
such as Twitter, WhatsApp, Facebook, and LinkedIn play a critical role in
providing students with opportunities for interactions and expressions of ideas.
Thus, social media networks play an essential role in reshaping traditional face-
to-face education systems by allowing students to collaborate and share
information through audio and virtual communication. Effective interaction and
collaboration amongst students provided by online learning platforms are in line
with Vygotsky’s theory which focuses on the sociocultural perspective,
suggesting that cognitive growth is highly influenced by society and culture
(Vygotsky, 1962). Thus, as Ferdig (2007) posits, social media networks promote
interaction between learners, and improves active learning in the student-centred
constructivist environment.
While Ferdig (2007) argues that social media allows effective communication
between faculties and students with students getting to know their lecturers
better, he also warns of the likelihood of lecturers losing professionalism as
students delve into their postings and personal profile information. According to
Casey (2008), Straub (2009), and Figueroa et al. (2020), while much is known about
the outcomes and characteristics of students who take online courses, less is
known about their experiences.

289
Objective of the study

The objective of this study was to explore pre-service teachers’ online learning
experiences during the COVID-19 lockdown concerning their preparedness for
online learning in terms of their digital literacy and use of technological devices
for online learning, their positive online learning experiences, and the challenges
they encountered learning online. The study was expedient because the pre-
service teachers of today are the classroom teachers of tomorrow. There is the
possibility that education will mainly be online in the near future, hence the need
to explore pre-service teachers’ online learning experiences to understand how to
help them to learn online and prepare them to effectively teach online in the
future. The pre-service teachers’ experiences might help them understand the
possible struggles of their potential students, thereby helping them to perhaps
accomodate any future students’ challenges. Besides, studies are yet to report the
online learning experiences of pre-service teachers in a pandemic situation such
as that of COVID-19.
3. Methodology
3.1 Research design and sample
This study used a descriptive survey research design which describes the opinions
of the pre-service teachers regarding their online learning during the COVID-19
lockdown. The design involves the collection of data as presently constituted to
describe a phenomenon, without a conscious effort to control any variables
(Creswell & Creswell, 2018; Dempsey, 2018). The participants were the pre-service
teachers (faculty of education undergraduate students) at a university in Ghana.
The university was purposely chosen because it is generally believed to be the
most technologically advanced public university in Ghana and therefore expected
to be more adaptive to the reality of online learning. It is also renowned for its
academic excellence. Consequent to the lockdown in the country due to the
COVID-19 pandemic, the university switched to an online mode of teaching and
learning to complete the academic work for the semester. A convenient sampling
technique was used to select the 300 third year level students who could easily
be reached through some of their online courses. A sample of 147 pre-service
teachers (Table 1) participated in the study. Descriptive statistics (frequencies and
percent) were used to analyse the data from Likert scale type of questions while
thematic analysis (Clarke & Braun, 2013) was used to analyse data from the open-
ended questions.
Table 1: Demographic profile of students

Variables Frequency Percentage
Gender
Male 57 38.8
Female 87 59.2
*Not indicated 3 2.0
Total 144 100.0
Age
15-20 26 17.7
21-25 111 75.5
26 -30 10 6.8

290
31-40 0 0
Over 40 0 0
Total 147 100.0
*Three of the respondents did not indicate their gender
3.2 Data collection instrument

Data were collected using an online survey questionnaire. The questionnaire
comprised 35 items. It was divided into four sections. Section A had two items
dealing with the demographic data (gender and age) of the students. Section B
consisted of eight items that dealt with their digital literacy and devices for online
learning. Section C, which focused on their online learning experiences, had 23
items. The students were expected to respond to the 23 items by expressing their
level of agreement, or otherwise on a four-point Likert scale (strongly agree, agree,
disagree, and strongly disagree). The last section of the questionnaire had two open-
ended questions on the positive experiences and challenges regarding online
learning – these items allowed the students to express their opinions in their own
words. The instrument was developed from items used in online course
evaluations and from some experienced education evaluation researchers. The
items were reviewed and edited by some scholars. The reliability of the
instrument, namely internal consistency, was established using Cronbach alpha.
A value of 0.926 was obtained, meaning that the instrument could be judged
reliable (George & Mallery 2003). The link to the survey questionnaire was sent to
the students via the learning management system.
4. Findings
The findings of this study are presented and discussed according to the themes:
students’ digital literacy, online learning devices, time spent on online learning,
positive experiences, and online learning challenges.
4.1 Students’ digital literacy, devices, and time spent on online learning
Digital literacy
To delineate the students’ digital literacy, they were asked to rate their digital
literacy by choosing from excellent, very good, good, adequate, limited, poor, and not
sure. Out of the 147 students that attended to the rating, 22 respondents,
representing 15%, rated their digital literacy excellent; 47 (32%) rated it very good;
53 (36%) rated it good; 10 (7%) rated it adequate; 10 (7%) rated it limited, and five
(3%) rated it poor. Thus, the findings showed that the students were digitally
literate.
Online learning devices

The students were asked to indicate the devices they used to connect to the
internet for their online learning, the devices they owned personally, and the
devices and online platforms they found the most valuable for their online
learning.
Regarding the devices they used to connect to the internet for their online
learning, 134 (92%) students used smartphones; 74 (51%) used laptops; 10 (7%)
used tablets; and only five (3%) used a desktop computer. It could be seen that the

291
total frequency (223) was above the number of respondents (147). This is an
indication that some of the students used more than one device for their online
learning.
Out of the 146 students that responded to the ownership of the devices item of the
questionnaire, 143 (98%) owned a smartphone; 74 (51%) owned a laptop; 14 (10%)
owned a tablet, and two (1%) owned a desktop computer. Regarding the devices
used to connect to the internet, 83 (56%) used their smartphone for connection; 42
(29%) used mobile phone hotspots; 19 (13%) used Wi-fi routers, and three (2%)
used modems.
The students were also asked to indicate the platform they found most valuable
for their online learning. Seventy-two (49%) of the respondents found both the
Zoom and Sakai learning management systems worthwhile. This was followed
by WhatsApp, valued by 40 (27%) students, while Screencast, Google meet, and
Google classroom were found most valuable by seven (5%) of the students.
Furthermore, students were asked to list the tools they found the most valuable
for their online learning. As many as 58 (39%) valued Forum; Chatroom 51 (35%);
Test and Quizzes 49 (33%); Commons 7(5%); Zoom four (3%); Sakai three (2%),
lecture videos two (1%), and WhatsApp one (less than 1%). One of the
respondents did not value any tool for online learning. The student declared, “I
found none valuable because it is not easy for me to afford the technology and data for it.”
Time spent on online learning

The number of hours of online lectures per week received attention from the 147
students. Twenty-four students, representing 16%, received lectures for one to
two hours per week; 38 (26%) for three to four hours per week; 36 (24%) for five
to six hours per week; 21 (14%) and 28 (19%) for seven to eight hours, and more
than eight hours per week, respectively. Out of the 146 students who responded
to the item on the number of hours spent learning online daily; 73 (50%) spent one
to two hours; 41 (28%) spent three to four hours; 25 (17%) spent five to six hours;
4 (3%) spent seven to eight hours, and 3 (2%) spent more than eight hours learning
online daily.
4.2 Students’ positive online learning experiences

The participants were asked to share their positive experiences of learning online
in an open-ended question. The data were coded for common themes amongst the
responses. Although the authors planned to use predetermined codes, some codes
emerged during the data analysis (Creswell & Creswell, 2018), thus, a
combination of emerging and predetermined codes were used. There was also an
inter-coder agreement where two or more coders agree on codes used for the same
statement. The coding occurred in two phases. The first phase identified
overarching themes in the data. The second phase identified sub-themes within
the overarching themes. In this case, the focus was on four overarching themes:
active learning and collaboration, flexibility, self-regulated learning, and
improved digital communication skills.

292
Active learning and collaboration

While many studies highlighted a lack of collaboration and active learning as a
deficit for online learning (Crosta, 2016), in this study, there was high active
engagement and collaboration among students. One hundred and fifteen (115)
participants, representing 78%, reported that online learning enabled them to
communicate actively with classmates and they were free and comfortable to ask
questions to fellow students and lecturers. Angela stated, “During online learning,
one can easily ask questions without timidity or fear, it encourages me to ask questions
since I won't see any student.” The participants also reported that learning online
allowed them to hear the voices of other students who were mainly quiet during
face-to-face interactions. As George reported, “I got to know many students’ views
on topics that if it were to be the face-to-face class, they wouldn't have contributed.”
Flexibility
The self-pacing nature of online learning increased students’ motivation and
satisfaction. The most frequently reported positive aspect of online learning was
the flexibility of time and place to learn. One hundred and twenty-one (121)
participants, representing 82%, reported that they enjoyed online learning as they
had a choice of places to learn and time compatible with their learning needs.
According to Sam, “Lecture videos sent by the lecturer can be replayed at any time”.
The view was also shared by Lesedi, “I could go back to the chatroom to see all that
we've discussed to get myself abreast of the topic.”
Self-regulated learning
The ability to self-regulate and acquire independent learning skills is crucial at
university (Mahama, 2016; Zimmerman, 2002). While being able to regulate
oneself is not a mental ability or an academic performance skill, it is a critical “self-
directive process in which learners transform their mental abilities into academic
skills” (Zimmerman, 2002, p. 3). In this study, 112 participants, representing 76%,
indicated that they enjoyed online learning because it encouraged them to take
control of their learning and continuously evaluate their progress. According to
Anita, “Online learning taught me that I don’t have to be taught to learn, one could check
many resources from YouTube, and so on.” Thus, the participants in the study
reported online learning as an opportunity to take ownership of and self-direct
their learning.
Improved digital communication skills

In this study, 108 (72%) respondents found online learning improved their digital
communication skills and appreciation of using technology to learn. According to
Mensah, “Learning online has enhanced both my typing and presentation skills. I have
learned how to use various learning platforms.” Rose and Annan had a similar
experience, Rose said “it helped me enhance my presentation skills”, while Annan
said, “online learning has helped with constant communication between students and
lecturer to facilitate learning.”
4.3 Challenges to the students’ online learning

Amidst the positive online learning experiences that the participants in this study
had during the COVID-19 pandemic lockdown were some challenges they
encountered with online learning. The students were asked an open-ended

293
question to indicate the challenges they faced with online learning. Their
responses to the question were analysed to identify common themes that
emerged. The themes identified were: internet connectivity, cost, power
(electricity) outage, devices, IT knowledge, time management, and family
interruptions.
Internet connectivity
The study revealed that most of the respondents (105 - 71%), encountered
challenges in learning online that were related to internet connectivity. The
internet connectivity challenges ranged from a slow internet network connection
to a complete internet network failure. Kofi expressed that “unstable internet
connectivity is one of my biggest problems.” Similarly, Rose said that “the network
wasn't stable” and Kwado noted, “the network the school gave us isn’t working in my
area.” Another respondent expressed how internet connectivity leads to the digital
divide between people in different cities and towns. According to the respondent,
“The internet connectivity at my area is so bad to the extent that sometimes I have to travel
to another town to do my quizzes or submit assignments which also affects my income.”
Cost
The cost of data to engage in online learning was found to be another challenge
experienced by some of the students. Even though Vodafone© provided 5GB of
data monthly to the students and lecturers to support online teaching and learning
during the lockdown, the data were insufficient for some of the students. Hence,
they had to buy more data for their online learning. Kwasi stated, “I'm not able to
afford data anytime, and the data given by the university finishes in a week.” In all, 50
students (34%) found the cost of data to access the internet a challenge they
encountered to engaging effectively in online learning.
Power (electricity) outage

In most developing countries, including Ghana, the power supply is very erratic
(Atta-Obenga & Dadzie, 2020). However, contrary to expectation, only five
respondents (constituting less than 4% of the participants) in this study reported
that power outages was a challenge to their online learning during the COVID-19
pandemic lockdown.
Devices
Some of the respondents (9) did identify having challenges relating to a device to
access online learning. Pointing out the challenge, one student said, “I have faced
many challenges in terms of the digital device to use.” Another respondent opined that,
“I did not have a laptop and a good phone with enough space.” Most of the students
stated that they used only their smartphones to access the internet for their online
learning. As a result of this, one student said, “because I use only the phone for
everything, I sometimes miss information.” Similarly, another student said that “some
of the learning tools such as go-to-meeting app [were] not compatible with my tablet.”
Time management
Time management was a challenge experienced by some of the respondents in
this study. Thirty-three respondents (approximately 22%) stated that they found

294
it challenging to manage their time with online learning. One student said that it
was difficult for her to allocate time for her studies, while another student said
that online learning was time-consuming. Yet another student commented that
different assignments from different lecturers made it difficult for her to manage
her time effectively.
Interruptions from family members

Interruptions from family members and chores at home challenged some of the
students’ online learning. This view was expressed by seven participants as a
challenge they encountered. Akuba said, “Parents don’t even care they still give us a
heap of chores to do, and it’s irritating,” while Kwami said, “interruption from family
members” was a challenge to his learning online during the lockdown.
5. Discussion of findings
It was found that the participants in this study were digitally literate and most
owned smartphones that they used for online learning. Hence, it could be said
that digital illiteracy was not a threat to their online learning. Also, the portable
nature of smartphones and their easy internet access might have helped the
participants spend more time (at least one hour a day) learning online. However,
the use of smartphones by most of the participants for their online learning
probably impacted negatively on the online learning experiences of some of them.
As observed by Morgan (2015), the use of smartphones can distract students from
learning as they can easily switch over to social networks and other non-learning
related websites. Besides, the use of mobile phones for online learning by most of
the participants might have also contributed to some of their internet connectivity
issues and other challenges they experienced with access to online learning
platforms. While most mobile phones have great features for accessing the
internet, some of them might not have the capabilities to access some of the online
learning platforms.
Most participants in this study reported active engagement and collaboration as

part of their positive experiences of online learning. This finding is contrary to the
lack of collaboration and active learning in online learning reported in some
previous studies (Crosta, 2016; Gilbert, 2015). The active learning opportunities
reported by most of the respondents in this study suggest that online learning
eliminates some barriers that limit participation during face-to-face instruction.
As observed in some past studies, online learning allows students to easily
interact and build communities, even when they are a thousand miles away
(Brown, 2001; Tang & Lam, 2014).
The result of this study indicates that 82% of the participants found online
learning flexible. The students reported being able to replay the lecture videos at
times convenient to them and that put them in charge of their learning as they
could work at their own pace. The flexibility of online learning could have
accounted for 76% of the students reporting that online learning made them take
control of their learning. This finding corroborated the view of Casey (2008) that
the most important characteristic of online learning is that it is flexible and
student-centred. This view is also corroborated by Arkorful and Abaidoo (2015),

295
who suggest that students perform better during online courses because they
support student-centred instructional strategies and are flexible with time and
place of study.
Another major positive of the online learning experience found in this study was
students’ improved digital communication skills reported by 72% of the
respondents. The world revolves around communication, and digital
communications have revolutionised and changed traditional communication
both personally and academically (Berry & Fagerjord, 2017; Waisbord, 2019). The
need for students to hone their communication skills and exchange information
meaningfully and on digital platforms is a critical skill in the 21st century. The
students’ improved digital communication skills reported in this study
corroborated with the view of Kelentrić et al. (2017) that frequent digital
communication not only helps students to become better acquainted with the
forms of communication required by various professional and academic
disciplines, but also enables students to think critically and construct new
knowledge.
Poor internet connectivity was one of the major challenges found in this study that
the students encountered in online learning. This corroborated the findings of
some recent studies (for example, Atta-Obenga & Dadzie, 2020; Bekoe et al., 2018;
Mahama, 2016) that poor internet connectivity poses a challenge to internet usage
in some parts of Ghana. The finding also corroborated the view of Narh et al.
(2019) that poor internet connectivity is a challenge to e-learning in Ghana. Four
years ago, Baylon and Antwi-Boasiako (2016) acknowledged that “Ghana also has
a significant urban-rural digital divide, with the majority of the country’s Internet
connectivity (and especially faster fibre-optic connections) concentrated in the
capital city of Accra and other large cities” (p. 2). The findings of this study
confirmed that the digital divide still exists in Ghana, as is the case in most African
countries.
The cost of data for online learning was found to be a challenge to the students’
online learning in this study. Though the students were given some free data, it
was not enough for most of them and some could not afford to buy additional
data for their learning. This agreed with some earlier studies (e.g. Atta-Obenga &
Dadzie, 2020; Baylon & Antwi-Boasiako, 2016; Mahama, 2016) that the cost of
internet access is unaffordable to many in Ghana.
While power outages could be a major challenge to online learning (Atta-Obenga

& Dadzie, 2020), only a few of the participants in this study reported power
outages as a challenge to their online learning. This could be because most of the
participants used smartphones for online learning, so they were able to continue
their learning even when there were power outages – using stored energy in their
phone batteries or power banks. Similarly, a few participants reported challenges
relating to devices for online learning. This challenge might have been because
not all technological devices work well with certain online learning platforms as
affirmed by Orlando and Attard (2015), that “teaching with technology is not a

296
one size fits all approach as it depends on the types of technology in use at the
time and also the curriculum content being taught” (p. 119).
Time management and interruptions from family members were found to be

other challenges the students encountered in online learning. In agreement with
these findings, students’ inability to manage their time effectively when learning
online has been found in other studies to be a challenge to some students (Roper,
2007; Yen et al., 2016). Some of the students’ time management challenges were
not unexpected because they were accustomed to face-to-face learning, where the
time allocated to learning activities is predominantly controlled by their learning
institutions. Similarly, lack of concentration due to distractions from family
members and family responsibilities has been found in some previous studies to
be a challenge to meaningful online learning (Figueroa et al., 2020; Kara et al.,
2019). This could even be more pronounced during the COVID-19 pandemic
lockdown when most family members were together at home over a long period.
6. Conclusion
This study explored the online learning experiences of pre-service teachers at a
Ghanaian university concerning the students’ preparedness for online learning in
terms of their digital literacy and ownership of technological devices, positive
online learning experiences, and challenges encountered while learning online
during the COVID-19 lockdown. The study found that the students were digitally
literate and that they were very aware of their level of digital literacy. Most of the
students used smartphones to learn online and this limited their online learning
activities. It was also found that online learning enabled them to communicate
and collaborate actively with course mates and lecturers. Besides, the flexibility
of online learning increased students’ motivation to learn. However, low internet
connectivity, cost of data, regular power (electricity) outages, lack of appropriate
devices, time management, and family interruptions were some of the challenges
experienced by the pre-service teachers.
7. Implications of the findings and recommendations

Lecturers and students need to keep pace with new online learning
technologies
In a recent study, Ogbonnaya (2019) advocates the upskilling of students and
faculties for effective teaching and learning using educational technologies. This
upskilling is needed now more than ever. Various information and
communication technologies are at the fulcrum of online teaching and learning.
Most of the participants in this study revealed having challenges with the
technology/devices used for online learning. We, therefore, recommend that
lecturers and students be regularly trained to ensure that they are well-skilled on
how to use technology to teach and learn. Lecturers and students need to be
knowledgeable about using various information and communication technology
platforms, not only in the face-to-face teaching and learning mode, but also in
blended and fully online learning modes. This will make it easy for lecturers and
students to quickly adjust to the needs of any teaching and learning mode that
may arise.

297
Need for a rethinking of the education model

To all stakeholders in education, especially in Africa, the COVID-19 pandemic has
underscored the need for a rethink of the teaching and learning models in the
current century. It is evident that the education landscape is no longer what it
used to be in past decades, hence, there is a need for a paradigm shift in our
philosophy of education.
Investment in technology infrastructure and training

The study revealed that most participants had challenges learning online because
of a shortage/lack or inefficient infrastructure to engage in meaningful online
learning. Most of the students accessed online learning via smartphones.
Smartphones may not offer the flexibility and capability needed for meaningful
online learning. Hence, we further recommend that learning institutions and the
government invest more into acquiring relevant infrastructure to enhance online
learning and teaching. These may include high-speed internet devices, computers,
and tablets to enable easy and fast access to online learning materials and
platforms.
Training student on time management

This study showed that some students lacked time management skills. Hence,
there is a need for students to be helped to develop time management
skills/strategies to enable them to effectively self-regulate their online learning.
Providing students with devices and data
Furthermore, we recommend that the government of Ghana and learning
instituions consider finding ways to provide students with devices and sufficient
data for online learning. Providing students with devices and data has minimized
students’ online learning challenges during the COVID-19 crisis in some countries
(Radha et al., 2020). Learning institutions could also make devices part of the
tuition package to ensure that every student has a device upon registration.
Limitations of the study
Just about 50% of the third-year pre-service teachers participated in the study.
Hence, there is the possibility that those who did not participate in the study had
a different experience with online learning from those who participated. Hence,
the findings might have been different if all the students participated in the study.
Besides, the participants were third-year students from one university, the online
learning experiences of students in different levels of study and from different
institutions might differ from the experiences of the students in this study. We,
therefore, suggest that the finding of this study be interpreted considering the
context.
8. References
Arkorful, V., & Abaidoo, N. (2015). The role of e-learning, advantages and disadvantages
of its adoption in higher education. International Journal of Instructional Technology
and Distance Learning, 12(1), 29-42.
Atta-Obenga, L., & Dadzie, P. S. (2020). Promoting sustainable development goal 4: The
role of academic libraries in Ghana. Information & Library Review, 52(3), 177-192.
https://doi.org/10.1080/10572317.2019.1675445

298
Baylon, C., & Antwi-Boasiako, A. (2016). Increasing internet connectivity while combatting
cybercrime: Ghana as a case study. The Centre for International Governance
Innovation and Chatham House.
Bekoe, S., Atiso, K., Ayoung, D. A., Dzandu, L., & Kumangkem, K. K. (2018). Examining
internet usage patterns on socio-economic benefits of marginalised communities:
The case of community information centres in Ghana. Library Philosophy and
Practice (e-journal), 1870, 1-25.
Berry, D. M., & Fagerjord, A. (2017). Digital humanities: Knowledge and critique in a digital
age. Polity Press.
Brown, R. E. (2001). The process of community-building in distance learning classes.
Journal of Asynchronous Learning Networks, 5(2), 18-35.
http://dx.doi.org/10.24059/olj.v5i2.1876
Burgess, S. & Sievertsen, S. S. (01 April 2020). Schools, skills, and learning: The impact of
COVID-19 on education. VOX, CEPR Policy Portal. Retrieved from
https://voxeu.org/article/impact-covid-19-education
Casey, D. M. (2008). The historical development of distance education through
technology. TechTrends, 52(2), 45-51. https://doi.org/10.1007/s11528-008-0135-z
Clarke, V., & Braun, V. (2013). Teaching thematic analysis: Overcoming challenges and
developing strategies for effective learning. Psychologist, 26(2), 120-123.
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed
methods approaches (5th ed.). Sage Publications, Inc.
Crosta, L. (2016). The online learning environment: A personal experience of collaboration.
In S. J. Cranmer, N. Bonderup-Dohn, M. De Laat, T. Ryberg & J-A. Sime (Eds.),
Proceedings of the Tenth International Conference on Networked Learning 2016: Looking
back – moving forward (pp. 1-6). Lancaster University.
Dempsey, P. R. (2018). How LIS scholars conceptualize rigor in qualitative data. Libraries
and the Academy, 18(2), 363-390. https://doi.org/10.1353/pla.2018.0020
Donlevy, J. (2003). Teachers, technology and training: Online learning in virtual high
school. International Journal of Instructional Media, 30(2), 117-121.
Ferdig, M. A. (2007). Sustainability leadership: Co-creating a sustainable future. Journal of
Change Management, 7(1), 25-35. https://doi.org/10.1080/14697010701233809
Figueroa, F., Figueroa, D., Calvo-Mena, R., Narvaez, F., Medina, N., & Prieto, J. (2020).
Orthopedic surgery residents’ perception of online education in their programs
during the COVID-19 pandemic: Should it be maintained after the crisis? Acta
Orthopaedica, 91. https://doi.org/10.1080/17453674.2020.1776461
Fogel, J., & Nehmad, E. (2009). Internet social network communities: Risk taking, trust,
and privacy concerns. Computers in Human Behavior, 25(1), 153-160.
https://doi.org/10.1016/j.chb.2008.08.006
George, D., & Mallery, P. (2003). SPSS for windows step by step: A simple guide and reference.
11.0 update. Allyn and Bacon.
Gilbert, B. (2015). Online learning revealing the benefits and challenges. Paper 3030. Education
Masters. Retrieved from https://fisherpub. sjfc.edu/education_ETD_masters/303
Greene, J. A., & Azevedo, R. (2010). The measurement of learners’ self-regulated cognitive
and metacognitive processes while using computer-based learning environments.
Educational Psychologist, 45(4), 203-209.
https://doi.org/10.1080/00461520.2010.515935
Goh, P. S., & Sandars, J. (2020). A vision of the use of technology in medical education after
the COVID-19 pandemic. MedEdPublish, 9(1), 49.
https://doi.org/10.15694/mep.2020.000049.1

299
Halverson, L. R., & Graham, C. R. (2019). Learner engagement in blended learning

environments: A conceptual framework. Online Learning, 23(2), 145-178.
https://doi.org/10.24059/olj.v23i2.1481
Hartini, T. I., Liliasari, A. S., & Ramalis, T. R. (2020). Implementing analytic mechanics
learning based on multiple representations on GeoGebra Software: In forwardness
to face the Industrial Revolution 4.0 (Mr-Geo. 4ir). Journal of Talent Development and
Excellence, 12(1), 3940-3954. https://doi.org/10.1088/1742-6596/1572/1/012015
Kalpokaite, N., & Radivojevic, I. (2019). Teaching qualitative data analysis software online:
a comparison of face-to-face and e-learning ATLAS. ti courses. International Journal
of Research & Method in Education, 43(3), 296-310.
https://doi.org/10.1080/1743727X.2019.1687666
Kara, M., Erdoğdu, F., Kokoç, M., & Cagiltay, K. (2019). Challenges faced by adult learners
in online distance education: A literature review. Open Praxis, 11(1), 5-22.
https://doi.org/10.5944/openpraxis.11.1.929
Kelentrić, M., Helland, K., & Arstorp, A. T. (2017). Professional digital competence
framework for teachers. The Norwegian Centre for ICT in Education, 1-74.
Mahama, A. (2016). Challenges facing internet connectivity: Perspectives of private cyber
cafes. International Journal of Scientific Engineering and Applied Science, 2(2), 466-474.
Morgan, K. (25 July 2015). The pros & cons of cell phone usage in college. Education Web.
http://education.seattlepi.com/pros-cons-cell-phone-usage-college-1578.html
Narh, N., Boateng, R., Afful-Dadzie, E., & Owusu, A. (2019, August). Virtual platforms:
Assessing the challenges of e-learning in Ghana [Paper presentation]. The Twenty-
fifth Americas Conference on Information Systems, Cancun, Mexico.
Ogbonnaya, U. I. (2019). Adoption and perceived usefulness of social media by pre-service
teachers in Nigeria. International Journal of Interactive Mobile Technologies, 13(6), 52-
67. https://doi.org/10.3991/ijim.v13i06.10299
Orlando, J., & Attard, C. (2015). Digital natives come of age: The reality of today’s early
career teachers using mobile devices to teach mathematics. Mathematics Education
Research Journal, 28, 107–121. https://doi.org/10.1007/s13394-015-0159-6
Radha, R., Mahalakshmi, K., Kumar, V. S., & Saravanakumar, A. R. (2020). E-Learning
during lockdown of Covid-19 pandemic: A global perspective. International Journal
of Control and Automation, 13(4), 1088-1099.
Roper, A. R. (2007). How students develop online learning skills. EDUCAUSE Quarterly,
30(1), 62-65. https://www.vistacampus.gov/system/files/
legacy/37/VISTABlend/documents/how_students_develop_online_learning_sk
ills. pdf
Rose, S. (31 March, 2020). Medical student education in the time of COVID-19. JAMA
Network. https://doi.org/10.1001/jama.2020.5227
Shea, P., & Bidjerano, T. (2014). Does online learning impede degree completion? A
national study of community college students. Computers & Education, 75, 103-111.
Straub, E. T. (2009). Understanding technology adoption: Theory and future directions for
informal learning. Review of Educational Research, 79(2), 625-649.
https://doi.org/10.3102/0034654308325896
Tanabe, J. (2020). Exploring a post-COVID-19 sustainable peace model. Social Ethics
Society. Journal of Applied Philosophy, 6(2), 73-103.
Tang, E., & Lam, C. (2014). Building an effective online learning community (OLC) in blog-
based teaching portfolios. The Internet and Higher Education, 20, 79-85.
https://doi.org/10.1016/j.iheduc.2012.12.002
Thomas, P. Y. (2010). Towards developing a web-based blended learning environment at the
University of Botswana [Doctoral dissertation, University of South Africa].

300
Thompson, D. L. (2010). Beyond the classroom walls: Teachers' and students' perspectives
on how online learning can meet the needs of gifted students. Journal of Advanced
Academics, 21(4), 662-712. https://doi.org/10.1177/1932202x1002100405
Traxler, J. (2018). Distance learning—predictions and possibilities. Education Sciences, 8(1),
1-13. https://doi.org/10.3390/educsci8010035
Vygotsky, L. S. (1962). Language and thought. Massachusetts Institute of Technology Press.
Waisbord, S. (2019). Communication: A post-discipline. Polity Press.
Watson, J. (2008). Blended learning: The convergence of online and face-to-face education.
promising practices in online learning. North American Council for Online Learning.
Yen, C., Tu, C., Sujo-Montes, L., & Sealander, K. (2016). A predictor for PLE management:
Impacts of self-regulated online learning on students’ learning skills. Journal of
Educational Technology Development and Exchange, 9(1), 29-48.
https://doi.org/10.18785/jetde.0901.03
Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory into
Practice, 41(2), 64-70.

301
Data Collection Instrument
Student Online Learning Experience Questionnaire
This questionnaire is designed to obtain information about your online learning due to the
COVID-19 pandemic. There are no right, or wrong answers and your response will be
anonymous, and the information gathered will be used for academic purposes only.
We thank you in advance for taking out your time to complete the questionnaire. It will
take you about 10 minutes to complete.
Demographic data
Gender: Male Female
Age: 15 -20 years 21-25 years 26-30 years 31-40 years Over 40 years
Engagement with online learning

How would you rate your own Digital Literacy?
• Excellent
• Very good
• Good
• Adequate
• Limited
• Poor
• Not sure
Which of the following devices do you use for online learning?

• Desktop
• Laptop
• Tablet
• iPad
• Smartphone
Which of the following devices do you personally own?

• Desktop
• Laptop
• Tablet
• iPad
• Smartphone
How do you connect to the internet for your online learning?

• Wi-Fi
• Modem
• smartphone
• Cell phone hotspot
About how many hours of online lectures do you receive weekly?

1 - 2 hours 3-4 5-6 7-8 years More than 8 hours
hours hours
About how many hours did you spend on your own learning online daily?
1 - 2 hours 3-4 5-6 7-8 years More than 8 hours
hours hours

302
List the online platforms you used for online learning (e.g. Zoom, Screencast, WhatsApp,
Sakai, google meet, etc.)
………………………………………………………………………………………………………
………………………………………………………………………………………………………
Which tools did you find most valuable for online learning (e.g. chartroom, Forum,
commons, collaborate, clickup, test and quizzes, etc.)?
………………………………………………………………………………………………………
………………………………………………………………………………………………………
Statements about online learning

Strongly Agree Disagree Strongly
Agree Disagree
1. I could easily access the Internet as needed for my
studies.
2. I am comfortable communicating online
3. I am willing to actively communicate with my
classmates online
4. I am willing to actively communicate with my
lecturers online
5. I find it easy to set aside time for learning online
6. I could manage my online learning time effectively.
7. I could easily complete my online assignments on
time.
8. I enjoy online learning
9. I enjoy working with other students in online groups
10. I have enough computer skills for doing online
learning.
11. I feel comfortable communicating online
12. I feel comfortable to ask my lecturers questions
during online learning activities
13. I receive a quick response during online learning
activities
14. I feel that face-to-face contact with my lecturers is
necessary to learn
15. I can discuss with other students during online
learning activities
16. I can collaborate with other students during online
learning activities
17. Online learning is as good as face to face class
learning
18. I believe that learning online is more motivating than
face to face learning
19. I believe a complete course can be taught online
without any difficulty.
20. I can pass a course online without any face to face
class lecture
21. I found it difficult to adjust to the online learning
context
22. Online learning encouraged the exchange of ideas
23. Online learning enhanced my learning

303
Explain some of your positive experiences of the online learning

………………………………………………………………………………………………………
………………………………………………………………………………………………………
………………………………………………………………………………………………………
Explain some of the challenges you had learning online

………………………………………………………………………………………………………
………………………………………………………………………………………………………

304

Vol. 19, No. 10, pp. 304-322, October 2020
Current Methods for Assessing the Level of Foreign

Language Proficiency of University Students
Nataliia S. Ivasiv, Mariya S. Kozolup, Olena V. Oleniuk,

Nataliia V. Rubel and Nataliya Y. Skiba
Ivan Franko National University of Lviv, Lviv, Ukraine
https://orcid.org/0000-0001-6243-0709
https://orcid.org/0000-0002-5892-6696
https://orcid.org/0000-0002-6203-9344
https://orcid.org/0000-0003-1924-6580
https://orcid.org/0000-0002-3899-3189
Abstract. The associative technique is a traditional method to estimate a

student’s foreign language proficiency. The aim of this article is to review
the current methods for monitoring and assessing the level of foreign
language proficiency of students; the arguments in favour of choosing the
associative method of testing for knowledge of lexical semantics; and how
assessments are conducted using this method through an associative
experiment. The article is based on semantic analysis and field
construction of the associative concept at the post-experimental stage. The
experiment involved two groups of Ukrainian-speaking students
studying German as a foreign language. Each group (control and
experimental) consisted of 20 people. The results demonstrate that
the assessment of the students’ linguistic and communicative
competencies requires consideration of parameters in multiple aspects:
paradigmatic, syntactic, syntagmatic relations; knowledge of typical
idiomatic and phraseological correlations and all possible logical-
associative relations. The experiment revealed shortcomings to include:
contamination; weak knowledge of phraseology and moderate ability to
establish logical relationships. In order to avoid the above shortcomings,
there is a need to use associative experiment in foreign language learning
and computer programs to achieve traditional associative learning
methods. Lexical-semantic concept analysis revealed different content for
semi-peripheral, peripheral and extreme peripheral zones. The article
also outlines further promising areas of research, including
psycholinguistic and metalinguistic ones. The research is potentially
useful for the development of didactic computer programs, for the
improvement of associative diagnostic methods in foreign language
teaching, etc.
Keywords: associative relations; associative experiment; associative-

semantic field; monitoring; evaluation

305
1. Introduction
The assessment of students’ human achievements has always been impeded by
the difficulty of measuring and determining the relationship between the quantity
and quality of socionic knowledge. Since the second half of the 19th century, there
have been universal criteria, indicators, and scales developed for assessing the
language competencies of future specialists in Europe (Fitzpatrick & Thwaites,
2020). The result of detailed methodological work was the development of six
levels of language proficiency. They have been used since the late 1990s: A1 —
Breakthrough; A2 — Waystage, B1 — Threshold, B2 — Vantage, C1 —
Proficiency, C2 — Mastery. A “third wave” of revisions of the principles and
methods for assessing foreign language competence is currently taking place,
taking into account new digital, communicative and cross-cultural requirements
(Turnbull, 2020). This requires new valid methods for diagnosing the quality of
language proficiency in the context of bilingualism.
In the context of the humanisation of education, leading governments are revising

and transforming traditional test methods that primarily take into account
quantitative characteristics of language proficiency: knowledge of grammar rules,
the volume of personal vocabulary, and the ability to use language units at the
syntagmatic level (McNamara, 2011). Thus, at the Ministerial Conference in
Germany in December 2016, the Standing Conference of the Ministers of
Education and Cultural Affairs (KMK) recognised that a trilateral approach
should be the basis for training foreign language teachers. This approach consists
of three dimensions: electronic resources, language communication, and
intercultural communication. It can lead to a language-competent person
achieving higher-than-expected results (Kultusminister Konferenz, 2017). This
approach correlates with metalinguistics, as communication concepts based on
both approaches can be both intracultural and cross-cultural (universal) (Serrano,
2011). It is also associated with psycholinguistic types of diagnosis which reveal
personal linguistic tools that are built as a result of personality development.
It has become clear that innovative methods for diagnosing foreign language
competencies, as well as modern teaching, implement a bottom-up approach
(Aristizábal, 2018). This corresponds to the learning and representation of
competencies from the native language (natural environment). The diagnostic
process itself is intelligent and includes equal and open queries that the student
can select and formulate himself (Turnbull, 2020). This determines how modern
higher education is monitored, with the prerogative given to self-monitoring and
self-regulation (Chang, 2010; Fabriz, Dignath-van Ewijk, Poarch & Büttner, 2014).
One of the problems with university study of a foreign language is a low or
satisfactory level of spontaneous and improvised (not on given topics) oral speech
(Güzel, 2017). Unfortunately, we found the lowest results for such indicators as:
flexibility, lack of tension, and the ability to switch to extraneous topics within the
discourse. However, students can show good or excellent knowledge in testing,
translation of texts, and knowledge of grammar (Chang, 2011). Obviously, a lack
of deep interiorisation of the material, a lack of knowledge of figurative and
contextual uses, and, as a result, a lack of naturalness are issues here. Since the

306
objective of this study is not the formation but the assessment of foreign language
competence, the aim is to find an effective method of testing foreign language
competences.
The above considerations confirm the relevance of the study of current methods
for assessing the level of foreign language proficiency of university students.
The study intends to review current methods for monitoring and assessing
foreign language proficiency among students; the arguments in favour of
choosing the associative method for testing knowledge of lexical semantics; and
how to conduct an assessment using this method through an associative
experiment. Based on the results of the associative experiment, the associative-
semantic field for one German language token will be constructed in parallel.
The system for monitoring the dynamics of foreign language competencies
consists of general assessment approaches and principles, assessment methods
and a system of expected results. Table 1 presents a broad overview of the variety
of scientific approaches to assessment.
Table 1: Basic scientific approaches to the assessment principles as a subject of

didactics
Assessment aspect Reference to Notes
the source
Application of smart - Program capabilities of in-class control
technologies in teaching and self-diagnostics are analysed.
and assessment
Diagnosis of motivation Njiru (2003) Academic motivation, confidence,
mobility and flexibility, creative ability
to learn something new are
determined.
Monitoring as a didactic Mishra (2005) In addition to diagnostic function,
process monitoring performs didactic
systematising, and repetitive function
in academic data collection.
Types of assessment Noh (2006) Regardless of the type, actual data are
always subject to assessment.
Stages and gradual Mertens (2009) Initial, current, intermediate and stage-
assessment of by-stage control are distinguished.
knowledge quality
Comprehensive Kissling (2012) This involves complexity of using
diagnostics teaching and assessment methods,
which are parallel rather than separate
stages.
Philosophical basis of Bibik (2017) Regularity, complexity, completeness
assessment and objectivity of the obtained data are
considered to be the main indicators of
acquiring the forecast competencies.
Self-assessment Lorenz (2018) Creating open resources for self-study
and self-assessment.

307
Mutual assessment in Bock (2018) This involves team forms of mutual

the course of online control based on network
communication communications.
Monitoring within a Goyal (2017); Assessment through free access to
determined educational Siddiqui, Alam, electronic platforms of a particular
environment Khan and educational institution.
Gupta (2019)
As we can see from the table, evaluation and monitoring correlate as subnotions
and generic terms. Assessment refers to determining the quality of specific
didactic manifestations and results of activities (facts). Monitoring and assessment
also correlate as general and partial (monitoring is carried out through a number
of assessment acts). Assessment of knowledge and competencies performs both
diagnostic and formative functions. Assessment procedures can be organised
externally or by an individual him/herself; it can rely on specific facts, criteria and
indicators.
Researchers consider assessment to be the most controversial component of

higher education due to various assessment-related problems. In his publication,
Purpura (2016) summarised the main problems, contradictions and shortcomings
in so-called L2 (Second and Foreign Language) assessment. The research covers
current approaches to L2 assessment and predicts their prospects in the future. It
also analyses methodological experiences of assessing foreign language
knowledge and concludes that knowledge should be assessed in accordance with
time requirements; and knowledge should be assessed to make decisions and
solve practical problems (Aghababyan, 2016; Aristizábal, 2018; Fitzpatrick &
Thwaites, 2020).
The simplified analysis of the most common assessment types is presented below
in the form of dichotomous oppositions (Table 2).
Table 2: Dichotomous classification of current assessment types for foreign language

competence
Assessment Notes
type
Subjective/ Subjective assessment always takes place when diagnosing
objective knowledge in the humanities (it focuses on flexibility,
expressiveness, and a general impression produced by the speaker).
Quantitative/ Quantitative indicators are easily diagnosed through testing,
qualitative listening and translation, while non-linear means, syntagmatic
range, and communicative efficiency require qualitative
parameters.
Continuous / Continuous types of assessment include self-assessment, current in-
at a certain class control.
time
Formative and Any detailed assessment is also a formative factor. The final types
summarising include all types of academic control (stage-by-stage control,
modular, etc).

308
Relative to the The criteria selected for different language styles may also contain
criterion / obscene components (slang, jargon, individual word usage).
relative to the
norm
Partial / In a partial assessment, certain aspects, such as pronunciation, are
aggregate subject to testing.
Direct / In direct assessment, the teacher perceives the student’s discourse
indirect directly. In indirect assessment, the teacher studies results
assessment presented in the form of written work and project presentations.
By rational Impression-based assessment is more holistic and focuses mainly

criteria / by on nonverbal aspects (speech rate, intonation, lack of tension, etc).
impression
Assessment of Assessment of skill is used at higher levels of language proficiency
performance / or in the creative (artistic) use of a foreign language.
skill
By others / The combination of external assessment and self-assessment is most
self- in line with the modern personality-oriented approach.
assessment
ёAnalytical / Analytical assessment allows you to assess individual parameters
holistic (knowledge of grammar rules, pronunciation, personal vocabulary).
Holistic assessment gives an overall impression of an individual’s
knowledge of the relationship of different parameters and a general
impression of language proficiency.
By skills / by This opposition to assessment correlates with a process and an
activity outcome. For example, a student speaks well but does not achieve
performed communication goals well.
Source: Author’s research
Among the modern methods of testing students' foreign language competence,

the test approach still dominates. However, modern tests are significantly
different from classic ones: they are mostly open, flexible, focused on current
knowledge and meet four criteria: validity, authenticity, reliability and positive
“washback” (Aristizábal, 2018). Moreover, the concept of “washback” is
becoming more common in didactics. It refers to formative and motivational
feedback from the performance of diagnostic tasks.
Currently the tools and conceptual framework for monitoring linguistic education
are well developed. Discussions mainly centre around its effectiveness and
validity: managerial effectiveness when monitoring teaching and learning in
school didactics (Mngomezulu, 2015); and methods of assessing university and
extracurricular management effectiveness (Victoria Primary School, 2018).
As part of a diagnostic experiment in non-linguistic specialties, scientists propose
to test, first, communicative competence. Such assessment contains informational,
organisational, regulatory and communicative components. Assessment criteria
and methods based on these aspects are selected.
Mathematical methods, quality assessment methods and graphical presentation

of the results are widely used in modern knowledge monitoring. The stages of
modern students’ knowledge monitoring, according to Hoover (2009), are
presented in Table 3.

309
Table 3: Stages of modern monitoring of knowledge

Knowledge Explanation
monitoring stage
Criterial Identification of competencies, criteria and indicators of their
manifestation.
Instrumental Development or selection of valid and compact methods over
time and techniques for identifying competencies.
Algorithmic Development of a monitoring program and its planning in time.
Procedural Conducting monitoring taking into account the results of

training in the previous stages.
Presentational At this stage, the assessment results are presented after
mathematical development by means of visualisation (graphs,
charts, digital data).
Progressive This is to determine the dynamics, and positive or negative
changes in language competences according to criteria and
indicators.
Corrective Adjustments are made to the educational process based on the
results of the analysis of dynamics.
Permanent This consists of planning and regularly conducting the next
stages of monitoring or self-monitoring. This stage is
permanent.
Source: Hoover (2009)
The literature discusses the gradual departure from traditional approaches to

testing grammar and vocabulary knowledge (Siddiqui et al., 2019). Testing (closed
and open) is supplemented by elements of constructing spontaneous oral and
written speech, and establishing the volume of learned meanings and associations
(Turnbull, 2020). Assessment of foreign language competencies should be
diagnostic and formative, and the main element of diagnosed skills should be
communicative. This approach allows knowledge to be tested, and conclusions to
be drawn about the acquisition of bilingual status by students, even those who
study a foreign language on their own (Turnbull, 2020).
Modern linguodidactics considers a foreign language in the context of

bilingualism and translanguaging. A truly bilingual person has the ability to
authentically and naturally speak his native and foreign languages (Creese,
Blackledge & Takhi, 2014; García & Wei, 2014). They study issues relating to
epistemological (scientific) approaches to the transformation of linguistic
education in the context of native languages (García, 2019) such as mastering a
foreign language before obtaining a degree (in the context of scientific
propaedeutics). Kaplow (2017) believes that specialised competencies in a foreign
language can be considered valid and sufficient when the native speaker can read
a complex scientific text, argue and verify the facts for authenticity. Competencies
sufficient in a case include a speaker being able to put forward hypotheses, prove
them and draw conclusions. A competent speaker can participate in scientific
discussions and illustrate theory with examples, demonstrate contexts, clarify and
formulate epistemes as basic units of scientific knowledge.

310
This raises the problem of finding the best overall method of assessing foreign
language competence or several methods that could complement each other.
Modern scholars, on the basis of their experience, indicate that associative
experimentation may be such a method. It can be both diagnostic and formative.
According to Khirzoeva (2008), such an experiment diagnoses not only
knowledge of a foreign language, but also flexibility in thinking, the ability to
establish syntagmatic and paradigmatic relations, to feel the peripheral semantics
of tokens and to use linguistic richness to generate coherent oral text. Research on
association experiment began to be developed in the nineteenth century
(Fitzpatrick & Thwaites, 2020). German scholars were the first to propose
associative experimentation as a method of studying and diagnosing language
competencies in the early 20th century (Fitzpatrick & Thwaites, 2020). It correlates
with the theory of neural connections (neurolinguistics), the theory of functional
fields, cognitive linguistics, sociolinguistics, and other theories. The closest
linguistic theory that complements this experiment is mental linguistics.
According to that approach, key concepts together with a set of logical,
metaphorical, associative and other linguistic connections make up the lexical-
semantic field (concept). The lexical-semantic concept can be considered a
common language (national concepts) or a personal (personal concepts)
formation.
3. Methodology
Foreign language proficiency depends on a student’s individual mental
development, the richness of his/her native individual style, and the formation of
thinking based on deep knowledge of the native language. These processes
develop in a linear way through connection of the referent and its verbal
objectification. Such a relationship can be diagnosed in a fast, indirect mode using
the associative method as an element of tests or as a separate didactic experiment.
According to scholars (Fitzpatrick & Thwaites, 2020), such an experiment reveals
not so much semantic connections in students as connections between external
stimuli-irritants, which are instantly verbalised (Khirzoeva, 2008). The
associations' speed and range allow conclusions to be drawn about the quality of
an individual’s knowledge and the degree to which foreign language units have
been internalised.
A simplified dichotomous approach to linguistic professional competencies

reduces the number of criteria to two: knowledge of language units (language)
and mastery of language units (language). While the first level of achievement is
easily diagnosed through tests, translation of written texts and listening
(representative level), the second requires a more personal and irrational
approach. Acquiring foreign language competencies means using them similarly
to the native language to the maximum possible extent. Cognitive learning of the
native language is well diagnosed by constructing concepts in the form of a lexico-
semantic field or, using an associative experiment, an associative-semantic field.
Since the authors do not research natural mastery of a foreign language during
studies for a bachelor’s degree, this article proposes that an associative structure
of the word should be built by analogy with the structure of the concept. The

311
associative structure process will be final in the reflection of the associative

experiment.
The experiment involved two groups of Ukrainian-speaking students studying

German as a foreign language. They were in their first year of a bachelor’s degree
(control group) and the fourth year of a bachelor’s degree (experimental group).
Each group (control and experimental) consisted of 20 people. This choice was
due to the gradual expansion of personal experience in the use of the most
commonly used words over time. The set of tokens selected for analysis during
the associative experiment was mandatory for basic proficiency at both level A2
(first year) and level B2 (fourth year). Such a set of tokens means that the
participants in both the control and experimental groups have approximately the
same knowledge of at least the primary meanings of these tokens. Therefore, the
experiment was designed not so much so that individuals could recognise the
meaning, but to clarify conceptuality (of words as concepts) and the scope of
meaning.
After obtaining the results of the assessment in the associative experiment, the
authors used the semantic analysis method. Such an analysis clarifies the relations
that students have established between tokens. The authors also applied a field
modelling method to the associative concept of individual tokens based on the
results of these relations.
3.1. The Course of the Experiment

An associative experiment assessed foreign language competencies and
determines the coefficient of bilingualism at the Faculty of Foreign Languages for
specialisations where the main language was German. From the thesaurus of the
most commonly used meaning-bearing words (over 5,500), 40 tokens were
selected (nouns, adjectives and verbs in approximately equal proportions) by
random sampling (random machine generation). The authors informed
participants about the experiment and they gave verbal consent to participate in
it. After that, the teacher read the tokens in a random order, and the students had
5-7 seconds to write down the blitz associations. There was no maximum limit for
this; they ranged in number from 2 to 5 for the control and experimental groups.
The second stage of the experiment involved collection of the maximum number
of associations for a limited number of tokens with no time limit. The aim was to
find out the language-associative competencies and levels of students’ personal
language vocabulary for the control and experimental groups. The second stage
also intended to collect lexical associative material to construct an associative-
semantic field for a widely used token in the German language. At this stage, we
offered each student one token and an unlimited amount of time (up to an hour
or more). The student had the opportunity to comprehend and arbitrarily explain
the meaning and essence of the selected associations. At this stage, we clarified
the maximum scope of the individual concept.
The authors selected the respondent’s age category according to the most relevant
neurodidactic indicators. Thus, late adolescence and early preadult age were
relevant to the acquisition of linguistic richness. At this time, a person’s linguistic

312
personality, his verbal tools are formed and language acquisition includes a range
of meanings, uses and images that make up a personal individual style.
The third stage of the research involved concept construction based on all the
maximum possible meanings of a single word and determination of its associative
and subjective meanings. Such meanings have a relatively weak correlation with
their primary lexical and semantic variants, so they illustrate logical and
emotional thinking (emotional intelligence) by type of native language. The
authors applied the free associations method proposed by Sternin (2001) in order
to carry out an inductive assessment on language proficiency on the basis of the
analysis of rare lexical-semantic variants. The researcher proposed to make an
inventory of linguistic and non-linguistic associations, to establish types of
connections and to explain their origin from the point of view of personal
development (formation). All types of associative words and their relations were
taken into account. After data collection, we constructed the associative-semantic
field for the token Fabel (fairy tale). Figure presents the results for that construction
(see Appendix A).
Thus, during the experiment the authors assessed the personal foreign language
competencies of participating students in their early and final stages of study. The
article structured one of the concepts in the mind of a modern student.
4. Results
The analysis of methodological sources allowed a number of preliminary pre-
experimental conclusions to be drawn. Thus, modern foreign language
competencies consist of two key components. The first is a social order and the
second is personal orientation. Both components can have a narrow specialisation
(agricultural translation, technical translation, guide-translator), but must include
intercultural (sociocultural) orientation. The latter is associated with the
acquisition of relevant political, social, artistic, ethical, communication
knowledge. The initial competence involves intercultural communication skills.
A full assessment of foreign language competence should consistently take into
account the following gradation aspects: personal prerogatives — mandatory
program requirements — interpersonal communication — intercultural
communication — intercultural dialogue —dialogue of cultures.
Psycholinguistic indicators of foreign language proficiency result from an

assessment of vocabulary diversity within the areas of discourse, the volume of
speech production, the average or appropriate size of sentences (depending on
speech style), the inclusion of clichés, speech stereotypes, word-parasites (the less,
the better), and the ratio of verbs to the total number of tokens. These indicators
make up so-called “verbal intelligence”. In fact, the psychological indicators are
spontaneity, richness of associations and involvement of words from related
topics, emotional inspiration and personal orientation (Khirzoeva, 2008).
Mentalinguistics (the doctrine of language concepts) correlates with the actual

psycholinguistic approach to teaching and assessing individuals’ knowledge of
foreign languages. A number of subjective, connotative and associative meanings

313
in the mind of a native speaker of a foreign language, along with the denotative
meaning, indicate the relative completeness (sufficiency) of the vocabulary
available to an individual. Students should develop knowledge of foreign words
not in the form of one- or multi-component definitions, but concepts.
A concept is a psycholinguistic individual and group formation that contains all

the paradigmatic, syntagmatic, and possible associative relations of a single
notion. Field construction of the concept helps to distinguish the center, semi-
periphery, periphery and extreme periphery (up to subjective associations that
cannot be explained extrovertedly).
Qualitative and quantitative assessment methods can be applied using

psycholinguistic and mentalinguistic approaches in the analysis of the initial
experimental data. This assessment is based on an associative method for
determining the meanings selected from a foreign language glossary, which is
mandatory for a student at this stage of study. In this regard, scholars note that
results in the study of the humanities are monitored using quantitative data only
initially and in an auxiliary way. The very dynamics of competencies can only be
measured as quality dynamics in the context of other competencies (Hu, 2002).
During the first part of the experiment, students in the control and experimental
groups were offered a number of meaning-bearing words in German. First-order
associates (selected during the first seconds) included one to two, less often three,
lexico-semantic variants, typical syntagmatics, standard expressions (clichés, less
often phraseological units), personal associations, etc.
Before the first part of the diagnostic experiment (selection of blitz associations)
was undertaken, we arranged the types of meaning according to their complexity:
direct meaning — typical syntactics — typical syntagmatics — lexico-semantic
variants — derivational relations — figurative meanings — phraseologically
related meanings — free associations — subjective meanings. Hypothetically, the
degree of associative complexity increases in such a series. Thus, this ranking can
determine fluency level in terms of the vocabulary. We assessed of the varied
associative use of tokens by scoring them from 1 to 9 (according to the number of
components from the above ranking used).
The graph (see Appendix B) shows that a higher rank correlated with less success
in terms of assignment completion in the experimental group. However, an
increase in extremes reflects figurative meanings, the selection of free associations,
and the subjective meaning of a word. This testifies to the moderation of logical
relations and knowledge of typical usages, and proves the validity of the thesis
about the dominance of emotional, subjective and personal word usage. Students
often find associations both within the syntagmatic laws of the German language
and by analogy with the native language. Sometimes this is detrimental to a
normative phraseologically or syntagmatically related meaning, because these
meanings may not coincide in different languages.
Unfortunately, we identified a number of negative trends during the experiment.

They were caused by semantic contamination, literal translation, ignorance of

314
lexical and semantic variants, calquing at the level of words and phraseologism,
etc. The main conclusions after the first part of the experiment (selection of blitz
associations) are as follow:
1. In the control groups, 75% (15 people) gave only 1-2 lexico-semantic variants of
meaning-bearing words.
2. Some associations indicated a literal translation of a word or cliché into German.
For example, students chose the lexico-semantic associations breaking
stereotypes, persuading someone, playing hard-to-get (slang) for the verb brechen
(break). Such analogies do not correspond to German language laws and are, in
fact, calques from Ukrainian.
3. Some associations (approx. 7%) related to paronomasia — a similar, but not
related meaning in terms of word creation: dick (thick), deckel (cover), dichte
(density) etc. These are formal associations that do not correspond to any semantic
or word-forming correlations. Therefore, we need to carefully analyse free
associations, as some of them are not valid.
The results of the second part of the experiment give a generalised picture of the
knowledge and associative abilities of students in the experimental and control
groups. Semantic analysis of these results (establishment of meaning,
paradigmatic, syntagmatic and associative relations) allowed the results to be
presented in the form of a table indicating the mechanism of associations, the
frequency of their manifestation and examples (Table 4).
Table 4: Classification of typical associations found during the experiment on the

example of the token Die Wand (wall)
Type of association Number of students
Examples
and/or its mechanism EG CG
Hypo-/hyperonymic Die Wand – das Gebeude, Wall
18 16
relations – structure
Part of the whole Die Wand – das Haus, Wall –

17 14
(synecdoche) house
Die Wand – die Zerstörung,
Antonymous association 11 7
Wall – ruin
Die Wand – hangen, Wall –

Association by function 13 8
hang up
Die Wand – das Bild, Wall –

Spatial contiguity 16 14
picture
Die Wand wird gebaut (steht),

Predicative association Wall is under construction 14 10
(stands)
Die Wand – das Dach, das
Association by subsection
fundament, Wall – roof, 13 8
relations
foundation wall

315
Die Wand – das Hindernis, der

By similarity (metaphor) Schirm, Wall – barrier, 18 16
obstacle, shield
Die Wand – die Wende, Wall –
Paronomasia (consonance) 11 8
boundary
Die Wand – wandlich, Wall –
Grammatical transposition 10 7
wall-mounted, wall-type
Connotative (evaluative) Stark wie ein Wand – strong
5 -
judgment as a wall
Phraseologically related Sich wie ein Mann erheben –
7 4
association make a stand for
Idiomatically related Die Wand – Die Berliner
6 3
association Wand, Wall – Berlin Wall
Association by property, Die Wand – Die starke Wand,
4 -
feature Wall – strong wall
Die Wand – bauen, zerstören,
Determinant relations 7 4
Wall – build, destroy
According to the results shown in Table 4, the following preliminary conclusions

about the second stage of the associative experiment are made:
1. Students in the experimental group found more associations than students in
the control group;
2. All students in the control and experimental groups found a wide range of
attributive, predicative associations, as well as hypo-/hyperonymy relationships;
“part of the whole” relations; associations by function and by subsection relations.
This indicates flexibility in thinking, even among the first-year students.
3. The students in the control group did not find any paronomastic and
phraseological relations, which indicates less knowledge of vocabulary and
phraseology at the initial stage of learning.
4. Students in both groups showed equally weak (satisfactory) competencies in
establishing relations by similarity (metaphor), associations by subsumption,
grammatical transpositions, connotative (evaluative) judgments, idiomatically
related associations, associations by property (feature) and determinant relations.
This indicates a low level of functional use of lexical units.
5. All students found consistent associations by analogy with their native
(Ukrainian) language. This aspect needs to be addressed didactically, as figurative
and phraseologically related meanings are different in different languages.
Linguistic interference must be weakened in the process of acquiring foreign

language competencies.
In this regard, we recommend improving the content of curricula in terms of

functional linguistics and phraseology by including an associative experiment in
the didactic tools (educational, not just diagnostic). There is a need to intensify the
study of typical syntagmatic, clichéd, phraseological word usages in the German
language.

316
The result of the third part of the experiment involved the construction of an
associative-semantic concept in the form of a field. Figure (Appendix A) shows
the structure and content of this concept for the example of the token Fabel (fable,
plot).
Figure (Appendix B) shows the quantitative and qualitative content of the

associative-semantic concept as summarised by the experimental group. Let us
analyse the structure of the associative field of the token Fabel (fairy tale).
1. Level (core) — the primary meaning, as well as basic lexical and semantic
variants. In the form of synonyms, they can be rendered by two representatives
— a fable, a plot. Overall, 17 out of 20 students showed knowledge of these
secondary meanings.
2. Level (semi-periphery) — the ratio of hypo- and hyperonymy folklore, fiction,
lies, literature. Most students perceived these associates as synonyms, although
they are related generally and partially by presupposition: each fairy tale is
folklore; every fairy tale is a fiction, and so on.
3. Level (periphery) — figurative meanings and related actions (metonymy):
forgery, tell, untruth, transformation, listen, charming, Grimm brothers, legend,
good, myth, curiosity. This zone turned out to be the most productive, which
indicates a high level of metaphorical and metonymic mechanisms in the reflected
pictures of the concept by students in both groups.
4. The level of typical associations (victory, ancient, believe, childhood, seven,
three, past, doubt, hero) is quantitatively limited, which indicates a weak
development of associative-logical thinking with the number of logical or
associative links over 2.
5. The level of occasional associations (outside the concept: fear, childhood,
evening, wolf, comfort, remember, end, joy, mother, country) is quantitatively
open and varies depending on the student. The development and significant
fullness of this outside-the-concept zone indicates the dominance of subjective
meanings and associations.
6. The number of students who selected associates is declining in the direction of
the periphery. In the area of outside-the-concept subject associations, the number
of associations increases. The number of students who selected them also
increases. The selection of phraseologically related meanings, which imply
knowledge of stable expressions, rather than an association between tokens, is
particularly difficult.
5. Discussion
The obtained results correlate with a number of similar studies on foreign and
native languages. Associative experimentation is a part of foreign language
competence development and diagnostics. It is a tool for constructing a concept
or associative-semantic field for the native language (Mubarakshina, Fedorova &
Fattahova, 2016). This testifies to the wide functionality of the associative method
in the humanities.
The presented results and the principles indicate the essence of associative
experimentation and previous discourse confirmed the using sense language
theory and practice by students or those who acquire foreign language skills. The

317
main indicator of this approach is the dynamics of students’ sense of language;

such dynamics are regularly assessed using the associative experimentation
method (Aghababyan, 2016).
The data obtained as a result of the experiment have demonstrated the

possibilities of an associative experiment for constructing universal, national and
personal concepts in the form of an associative-semantic field, which extends from
lexico-semantic variants (core) to the far periphery (the latter contains subjective
associations with the original token). The word association technique is a basis for
exploration of lexicon content and structure, and can be used as a method for
understanding the bilingual brain, and a technique for assessing vocabulary
knowledge, among other research aims (Fitzpatrick & Thwaites, 2020).
Based on the associative experiment results that record and evaluate a student’s
sense of language, this research indicates an effective assessment method for the
development of foreign language competence. For this, the student would learn
the obligatory paradigmatic relations, gain experience in establishing typical
associations during training and communication, and generate new relations
during professional activity (Aghababyan, 2016). Flexible use of connections is
possible in a formative associative experiment, involving the first word-reaction,
the second word-reaction and the whole chain of self-consistent associations, each
following from the previous one. This is how associative-semantic memory is
formed.
As a result of generalisations of experimental data, the research has proved that

the connection between thematic relations and conceptual representations for
particular concepts does not always coincide, since there are individual
differences. However, both vertical and horizontal relations between a word and
its associates play an important role in the development of semantic memory, and
these associations complement each other and contribute to the spontaneous
unprepared generation of a new text (Mukhametzyanova & Shayakhmetova,
2014).
The experiment revealed the negative aspects of the associative method in

assessment, and especially in training. Rose and Rahman (2016) predicted that the
constant use of the associative method leads to semantic interference between
associations (mixing of meanings) and cumulative (directed to a particular aspect)
selection of associations. Negative effects from an associative experiment can be
avoided by encouraging students to change contexts, to select associations in
terms of expansion, rather than deepening subsumptive semantics.
Associative relations between language elements can be used not only as a

methodological tool, but also a way to study semantic processes: connotation,
language interference and individual language modality (individual style). The
cumulative effect of associative fields, units of which stimulate the emergence of
an increasing number of associations in a certain direction, was confirmed by
means of an associative experiment (Rose & Rahman, 2016). This effect can be

318
used in linguodidactics: new tokens will be layered on the knowledge of tokens

of a certain semantic field in the process of applying the associative method.
6. Conclusion
The associative method is the most valid tool for undertaking a comprehensive
knowledge check of lexical material and foreign language competence for a
student assessment. The associative experiment has been demonstrated to be a
universal tool that allows assessment and the formation of foreign language
competencies, while providing valuable material for constructing associative-
semantic fields for particular tokens. This could be a starting point for
constructing lexical concepts in the language consciousness of young students.
The diagnostic part found that students in the experimental group completed the
assignment for the selection of associations 15% better than the control group. At
the same time, we revealed that there were a number of shortcomings of
Ukrainian-speaking students in learning a foreign (German) language. This
applied to both the control and experimental groups. Most of the shortcomings
related to native language analogies and ignorance of typical associations in
German. Thus, further methodological improvements in German language
research are required. The constructed associative-semantic field for the Fabel
token showed that the peripheral zone and the zone of subjective associations are
the most developed, which indicates a sufficient syntagmatic level of word usage
and a developed subjective meaning. At the same time, paradigmatic relations
and remote periphery indicate the need for active work with the vocabulary and
the development of logical relations between denotations. Further research on this
topic could develop software to automate the associative experiment, and could
supplement this type of assessment with others (establishing paradigmatic
relations between words, the use of words in context, etc). Associative methods of
teaching foreign vocabulary and phraseology also need further development.
7. References
Aghababyan, K. H. (2016). Investigation of dynamics of students' sense of language by
word associative experiment method. In R. Draut (Ed.), Humanities in the 21st
century: scientific problems and searching for effective humanist technologies (pp. 55-62).
San Francisco, CA: B&M Publishing.
Aristizábal, G. F. (2018). A diagnostic study on teachers’ beliefs and practices in foreign
language assessment. Íkala, Revista de Lenguaje y Cultura, 23(1), 25-44.
https://doi.org/10.17533/udea.ikala.v23n01a04
Bibik, N. M. (2017). New Ukrainian school: teacher’s advisory book. Kyiv, Ukraine: TOV
“Vydavnychy dim ‘Pleyady’“.
Bock, A. (2018). The transformation of school textbooks into digital educational media. Retrieved
from https://repository.gei.de/handle/11428/291
Chang, M. M. (2010). Effects of self-monitoring on web-based language learner's
performance and motivation. Calico Journal, 27(2), 298-310.
Chang, S. C. (2011). A contrastive study of grammar translation method and
communicative approach in teaching English grammar. English language teaching,
4(2), 13-24. https://doi.org/10.5539/elt.v4n2p13
Creese, A., Blackledge, A., & Takhi, H. K. (2014). The ideal ‘native speaker’ teacher:
negotiating authenticity and legitimacy in the language classroom. The Modern
Language Journal, 98(4), 937-951. https://doi.org/10.1111/modl.12148

319
Fabriz, S., Dignath-van Ewijk, C., Poarch, G., & Büttner, G. (2014). Fostering self-
monitoring of university students by means of a standardized learning journal—
a longitudinal study with process analyses. European Journal of Psychology of
Education, 29(2), 239-255. https://doi.org/10.1007/s10212-013-0196-z
Fitzpatrick, T., & Thwaites, P. (2020). Word association research and the L2 lexicon.
Language Teaching, 53(3), 237-274. https://doi.org/10.1017/S0261444820000105
García, O. (2019). Reflections on Turnbull’ s reframing of foreign language education:
bilingual epistemologies. International Journal of Bilingual Education and
Bilingualism, 22(5), 628-638. https://doi.org/10.1080/13670050.2016.1277512
García, O., & Wei, L. (2014). Translanguaging: language, bilingualism and education.
Basingstoke, UK: Palgrave Macmillan.
Goyal, A. (2017). 8 benefits of adopting cloud-based applications in education. Retrieved from
https://elearningindustry.com/cloud-based-applications-in-education-8-
benefits
Güzel, M. Ç. (2017). Developing oral proficiency and motivation through script-based and
improvisational drama (unpublished master’s thesis). California State University,
San Bernardino, CA.
Hoover, J. J. (2009). RTI assessment essentials for struggling learners. Thousand Oaks, CA:
Corwin Press.
Hu, G. (2002). Potential cultural resistance to pedagogical imports: The case of
communicative language teaching in China. Language Culture and Curriculum,
15(2), 93-105. https://doi.org/10.1080/07908310208666636
Kaplow, I. (2017). „Fachspezifische Fremdsprachenkenntnisse“und Wissenschaftliche
Propädeutik ["Subject-specific foreign language skills" and scientific propaedeutics].
Berlin, Germany: Berlin Catholic University of Social Sciences.
Khirzoeva, Z. M. (2008). Associative experiment as a method of ranking the foreign
language proficiency level. Proceedings of Dagestan State Pedagogical University, 3,
16.
Kissling, E. M. (2012). Techniques and principles in language teaching, third edition (book
review). Retrieved from
https://pdfs.semanticscholar.org/b9fb/e8d83b069c57deec5aa1a8967f085fb7664
b.pdf
Kultusminister Konferenz. (2017). The standing conference’s “Education in the digital world”
strategy summary. Retrieved from
https://www.kmk.org/fileadmin/Dateien/pdf/PresseUndAktuelles/2017/KM
K-Strategie_Bildung_in_der_digitalen_Welt_Zusammenfassung_en.pdf
Lorenz, A. (2018). Eine offene Bildungsressource (OER) ist konsequent eingesetzt eine
Chance für den Hochschulzugang: ein Praxisbericht [An open educational
resource (OER) is consistently used an opportunity FOR university admission: a
practical report]. Medien Pädagogik: Zeitschrift für Theorie Und Praxis Der
Medienbildung (Offenheit in Lehre und Forschung), 32, 28-40.
https://doi.org/10.21240/mpaed/32/2018.10.21.X
McNamara, T. (2011). Authority and language assessment. Language Teaching, 44(3), 39-52.
Mertens, D. M. (2009). Research and evaluation in education and psychology. London, UK:
SAGE Publications Inc.
Mishra, R. C. (2005). Educational research. New Delhi, India: A.P.H. Publishing
Corporation.
Mngomezulu, N. M. (2015). Strategies of monitoring teaching and learning: A school
management team perspective (unpublished master’s thesis. School of Education in
the discipline, Educational Leadership, Management and Policy, University of
Kwazulu-Natal, Durban, South Africa). Retrieved from

320
https://pdfs.semanticscholar.org/ad88/c69af2852be19f4c0ab1c47f961ef4619ad4.
pdf
Mubarakshina, A., Fedorova N., & Fattahova, N. (2016). Associative-semantic field of
"nravstvennost" in modern Russian language. Journal of Language and Literature,
7(2), 280-284.
Mukhametzyanova, L., & Shayakhmetova, L. (2014). Application of associative
experiment in forming the foreign communicative competence. English Language
Teaching, 7(12), 60-64. https://doi.org/10.5539/elt.v7n12p60
Njiru, J. N. (2003). Measuring academic motivation to achieve for high school students using a
Rasch measurement model (unpublished master’s thesis. Faculty of Community
Services, Education and Social Sciences, Edith Cowan University, Perth,
Australia). Retrieved from https://ro.ecu.edu.au/theses/1320/
Noh, H. J. (2006). Policy evaluation. Seoul, Korea: Bupmunsa.
Purpura, J. E. (2016). Second and Foreign Language Assessment. The Modern Language
Journal, 100(S1), 190-208. https://doi.org/10.1111/modl.12308
Rose, S. B., & Rahman, R. A. (2016). Cumulative semantic interference for associative
relations in language production. Cognition, 152, 20-31.
Serrano, R. (2011). From metalinguistic instruction to metalinguistic knowledge, and from
metalinguistic knowledge to performance in error correction and oral production
tasks. Language Awareness, 20(1), 1-16.
https://doi.org/10.1080/09658416.2010.529911
Siddiqui, S. T., Alam, S., Khan, Z. A., & Gupta, A. (2019). Cloud-based e-learning: using
cloud computing platform for an effective e-learning. In S. Tiwari, M. Trivedi, K.
Mishra, A. Misra & K. Kumar (Eds.), Smart innovations in communication and
computational sciences. Advances in intelligent systems and computing, vol. 851 (pp.
335-346). Singapore, Singapore: Springer.
Sternin, I. A. (2001). Concept structure research methodology. In I. A. Sternin (Ed.),
Methodological problems of cognitive linguistics (pp. 58-65). Voronezh, Russia:
Voronezh State University.
Turnbull, B. (2020). Towards new standards in foreign language assessment: learning from
bilingual education. International Journal of Bilingual Education and Bilingualism,
23(4), 488-498. https://doi.org/10.1080/13670050.2017.1375891
Victoria Primary School. (2018). Monitoring and Evaluating Policy. Retrieved from:
ttps://storage.googleapis.com/siteassetsswd/104/downloads/Monitoring%20a
nd%20Evaluating%20Policy.pdf

321
Appendix A
Associative-semantic field for “Fabel” (fairy tale) token

322
Appendix B
Gradation of associative uses of autonomous words in the experimental group

323

Vol. 19, No. 10, pp. 323-337, October 2020
Teaching through Experiential Learning Cycle to

Enhance Student Engagement in Principles of
Accounting
Rohaila Yusof*, Khoo Yin Yin, Norlia Mat Norwani,

Zuriadah Ismail and Anis Suriati Ahmad
Universiti Pendidikan Sultan Idris, Malaysia
https://orcid.org/0000-0001-5304-8970
https://orcid.org/0000-0003-4850-2184
https://orcid.org/0000-0002-6795-906X
https://orcid.org/0000-0002-9721-3402
https://orcid.org/0000-0002-4070-3398
Salniza Salleh
Universiti Utara Malaysia
https://orcid.org/0000-0003-4518-7721
Abstract. This paper aims to investigate structuring classroom activities

in experiential learning cycles to facilitate first-year Business and
Accounting students’ engagement in principles of accounting course.
The levels of student engagement and achievement in understanding
accounting concepts are measured at the end of the course. The content
of fundamental accounting is structured into experiential learning cycles
to provide a practical model for learning accounting concepts. Two
models of students learning are tested in this study are the Kolb
Experiential Learning Model and Burch Engagement Model. A quasi -
experimental study is implemented to test the effectiveness of an
experiential learning model in improving student engagement. Lessons
on Business Accounting is structured into an experiential learning cycle
and delivered to an experimental group to measure the improvement in
students’ engagement. Burch Engagement Model which measures
students’ cognitive, emotional, and physical engagement is applied in
measuring their engagement level. A pre and post-test design are
employed to measure the difference in their engagement level between
the control and the experimental group. The respondents of this study
are a total of 112 students in Principles of Accounting course.
Questionnaires and test sets are the measurement instruments used for
data collection. After an eight-week exposure to experiential learning
strategy, the post -test score for students’ engagement show an increase
in means of the four dimensions of student engagement, with the
* Corresponding Author: Rohaila Yusof, Email: rohaila@fpe.upsi.edu.my

324
highest mean in the cognitive in-class engagement, followed by

cognitive out-of class, physical and emotional engagement.
Improvement in the cognitive score is concluded to be significant across
the cognitive levels of Bloom Taxonomy mainly in understanding,
application, and analysis. The implications of this study involve
improvement of the support system to materialize the experiential
learning strategy.
Keywords: Kolb Experiential Learning Strategy; Students Engagement;

Business Accounting; Cognitive in-class Engagement
1. Introduction
The aspiration of the Ministry of Education in Malaysia through the Malaysian
Educational Blueprint 2015-2025 aims to nurture entrepreneurial graduates with
the attributes of financial sustainability. It is significant to improve the delivery
of basic accounting courses to prepare students with basic financial management
skills to choose and start their initiatives that align with nurture holistic
graduates. The National Entrepreneurship Policy targets 15% of graduates to
become entrepreneurs.Thus, financial skills acquisition need to be strengthened
to equip graduates with venture into their entrepreneurship process. Hence, the
teaching methods in Higher Education Institutions in the field of accounting and
management need to be enhanced. It is inevitable to mention that accounting is a
significant concept in financial management to help these potential
entrepreneurs in the act of planning, organizing, directing and controlling
economic activities.One of the main financial activities in business includes the
process of procurement and utilization of funds of the enterprise (Juneja, 2017),
wherein specific accounting knowledge, skills and values are required.
Unfortunately, reality reveals that many entrepreneurs have not taken this
matter seriously to the degree that sales proceeds is used to fund their
expenditures rather than to prioritize company needs.Indeed, studies on the
failure of entrepreneurs of small and medium businesses also found that 83% of
businesses failed due to cash flow management weaknesses (Arditi, Koksal &
Kale, 2000). This situation indicates the insufficient skill and level of awareness
of financial management among entrepreneurs.
2. Background of the Study

Business Accounting course is one of the many names given to fundamental
accounting courses offered in accounting degree programs to all accounting
majors and non-majors in most HEIs (Higher Education Institutions). It comes
under different names such as Introduction to Accounting, Principles of
Accounting, and other terms that reflect the nature of the course as
fundamentals and basic. Accounting is a field that requires a careful systematic
approach to recording, reporting and analyzing diverse business transactions.
Accounting subjects are often seen as more conceptual than procedural (Wyhe &
Arthur, 1991) ,and an excessive emphasis is on finance rather than accounting
management (Roudaki, 2016). The course syllabus is specifically designed to
provide students with basic knowledge in accounting, namely: technical
classifications, recording transactions, and preparing financial statements. The

325
Course Learning Outcomes (CLO) of the course is for students to prepare a full
set of accounting and probably some additional generic skills incorporated into
the course to meet the overall Programme Learning Outcomes (PLO). However,
past research on students’ achievement shows that students' achievement in the
first accounting course is not convincing and dependents on various factors,
especially prior learning experience (Garkaz, Banimahd & Esmaeili, 2011) and
students' interest and career aspirations (Kamaruddin, 2015). Students’
achievement results from various interacting factors such as intellectual
capacity, quality of delivery, motivation, learning approaches, students' socio-
economic factors, and other factors. Nevertheless, students’ engagement could
be one of the factors determining their retention (Ndege, 2010), skills
development (Kapur, 2018) and student interest (Ghasemi, Moonaghi & Heydari,
2018). Student engagement also includes comprehensive aspects of achievement
besides cognitive,and cognitive engagement inside and outside the classroom. In
this line of thought, Student engagement is a critical factor worth studying,as it
is believed to drive students to work independently.Add to this, engaged
students are expected to be inquisitive and carry their thinking beyond the
classroom (Taylor & Parsons, 2011). A close attempt by the quality assurance
framework to measure student engagement is the compliance of 40 hours SLT
for each credit assigned to a course, applied as the standard measurement in
MQA. Various learning activities are suggested to fill the 120 hours SLT for a
three-credit system, with the biggest portion on non-face-to-face learning. The
SLT concept demands motivation, ability and monitoring of independent
studies to take place among students.
Business accounting course has to be relevant and effective in preparing holistic

graduates. The practical or hands-on method is believed to be the most effective
delivery method for suitable classes. Thus, experiential learning is a method that
could provide relevant in-class experience for better engagement among
students. Fundamental Accounting course content ranges from technical book
keeping skills to financial statement preparation, which requires analytical
skills.The business examples used are often based on textbooks and less relevant
to real business situations. The textbook approach is not effective in engaging
students in learning, because students need to be engaged through meaningful
and challenging activities to see the big picture of accounting skills in solving
real problems. The study by Kraukuer, Serra and De Almeida (2017) found
advantages regarding experiential learning in entrepreneurship learning, such as
greater student engagement and a sense of empowerment. Students can apply
knowledge management theories to their team experiences combining theory
and practice and use them in their work (Pauleen, Marshall & Egort, 2014). The
AACSB Guidelines (AACSB,2013) strongly recommend applications of
experiential learning in the business curriculum. It is more fitting to attain
learning outcomes that require more successful areas or practical abilities such
as strategic decision-making, coordination, leadership, collaboration and many
other skills.

326
Effective delivery methods provide new experiences to students to link existing
knowledge to the desired knowledge. Learning is the process whereby
knowledge is created through the transformation of experience (Kolb, 1984).
Beard and Wilson (2002) stated that when receiving a stimulus, be it an external
or internal one; a form of learning from experience is generated. Figure 1
represents the complex process of perception being interpreted and responded
as an experiential learning process.
Figure 1. The process of perception and experiential learning (Beard & Wilson 2002)
Kolb’s(1984) learning cycle is an experience-based learning strategy in which

each student should be given the opportunity to undergo the four phases of
learning for each topic. Kolb Learning Cycle (1984) originates from a theory that
learning style can be ranked along a continuum. Many studies attempted to
show the relationship between learning styles and achievement, yet there is little
evidence to support teaching according to individual learning styles as
differences and preferences in the way individuals learn exist. Hence, a variety
of experiences in learning provides students with a rich understanding of
knowledge. Such a conception is elaborated and therefore different learning
styles are met through the four cyclical phases namely: concrete /abstract, and
experimental /reflection being organised in two-opposite continuum. Figure 2
reflects the four phases in respect to their continuum.

327
Figure 2. Kolb learning cycle
The Kolb learning cycle model (1984) is applied in learning Principles of

Accounting by providing four different activities with related resources. Firstly,
students need to understand the big picture, 'Why' to stimulate their interest in
the subject matter, 'What' is the significance of this lesson to their everyday life,
'How' to understand the concepts and related theories. In return, students
should apply the newly learned concept and ideas into a situation; that is the
'What if' phase. These four questions represent the learning cycle's internal
structure which implies a pattern for learning new concepts. A combination of
the learning modes from all four learning styles produces the highest learning
level (Harb & Terry, 1995). Among the teaching challenges in a cycle is preparing
suitable materials to provide relevant experience. In another terms, each
quadrant offers a learning experience that suits one out of four learning styles
according to the Kolb Learning Styles Model (KLSM); assimilators, divergers,
convergers, and accommodators. KLSM is based on how students perceive and
process information in learning through two-opposite continuum; feeling and
thinking in the perceiving continuum, while reflecting and doing is in the
processing continuum.
The first quadrant refers to suitable activities to assimilate learning inclination

through feeling and doing, such as motivational stories, group problem solving,
etc. The second quadrant is for those who learn through feeling and watching,
such as textbook reading assignments, seminars, etc. Whereas the third learning
style assimilates who learn through thinking and watching, including field trips,
group discussions, and the like. Finally, the fourth style consists of convergers
that calls for who learn through reflection and doing, which is best supported by
lectures with demonstrations, examples, problems, etc. These activities are not
mutually exclusive; meaning that the same types of activities can provide a
different learning experience. In this regard, educators can give various learning
activities to offer small experiences in class for better student engagement. Table
1 indicates the type of learning activities to provide students with rich learning
experiences (Harb & Terry, 1995).

328
Table 1. Learning activities for each quadrant

FIRST QUADRANT LEARNING SECOND QUADRANT LEARNING
ACTIVITIES ACTIVITIES
Motivational Stories Formal lecture, thinking tone

Simulations Lecture with Visual Aids
Class Discussion Lecture with Programmed Notes
Group Discussion Textbook Reading Assignment
Journal Writing Problem Solving by Instructor
Brainstorming Example Problem from Textbook
Interactive Lecture Professional Meetings
Group Problem Solving Large Seminars
Formal lecture, feeling tone TV Demonstrations
Field Trips Independent Research
Role-Playing Objective Exams
Socratic Lecture Library Searches
Group Experiments Gathering Data
Subjective Tests
THIRD QUADRANT LEARNING FOURTH QUADRANT LEARNING
ACTIVITIES ACTIVITIES
Example Problems Worked by Open-Ended Problems

Students Open-ended Laboratories
Student Prepared Problems
Homework Problems Field Trips Student Presentations
Guided Labs Semester Long Design Projects
Computer Simulations Socratic Questioning
Field Trips Group Discussion
Objective Exams Student Lectures
Laboratory Test Brainstorming
Individual Reports Role-Playing
Computer-Aided Instruction Subjective Exams
Lectures with Demonstrations Think Tanks
Simulations
Group Problem Solving
Group Projects/Reports
Source: Harb and Terry (1995)
Based on four chapters in the accounting syllabus, namely: budgeting,

bookkeeping, cashflow, and profit and loss statement, a comprehensive
approach is presented by utilizing all 4 phases of the Kolb Learning Cycle.
Figure 3 illustrates the cycle for the first chapter, which is budgeting. For the first
quadrant, Concrete Experience (CE), known as the "Why" phase, students are
exposed to business failure due to failed budgeting. Taking the case of Muslim
Awang, the students go through the 'big picture' before answering six reflective
questions in the second quadrant, the so-called the "What" phase. Moving on to
the third quadrant, Active Experimentation (AE), students are provided with
complimentary notes on budgeting before preparing a budget of Muslim plans
to set another branch up in the campus with more students in the last quadrant
(Concrete Experience (CE).).

329
Figure 3. Kolb learning cycle for budgeting
Figure 4 depicts the Kolb Learning Cycle for the second chapter, which is
bookkeeping. The Concrete Experience (CE) or the "Why" phase engages the
students through a news article and role-plays on Pn Husna, who owns a
bakery, and her business partners. For the Reflective Observation (RO)
quadrant, students must answer several reflective questions for all three
scenarios.In this context, students are supplied with notes on bookkeeping being
discussed in class for the "How" phase or Abstract Conceptualization (AC).
Lastly, students are required to prepare bookkeeping for bigger scale bakery
business by Pn Husna.
Figure 4. Kolb learning cycle for bookkeeping

330
Figure 5 respectively shows the Kolb Learning Cycle for the preparation of cash
flow.The use of interactive stimuli for the Concrete Experience (CE) is
manifested through cafe businesses' video. Next, the students are expected to
reflect on four issues surrounding Bob and his newly-opened cafe “The Wired
Cup” for the Reflective Observation (RO) phase. Using the journal articles and
presentation slides in the Abstract Conceptualization (AC), students will gather
the necessary knowledge to prepare a good cash flow to help Bob increase the
revenue while minimizing the costs simultaneously for the last quadrant, Active
Experimentation (AE).
Figure 5. Kolb learning cycle for cash flow
Figure 6 below explains the Kolb Learning Cycle for the fourth chapter, the
profit and loss statement.In the first quadrant or the "Why" process, students are
introduced to the circumstances of a stall owner who does not manage to figure
out where the profit has gone even though their profits are increasing. Moving
on to the next quadrant, namely the Reflective Observation (RO) or the What
step, they are invited to give their opinions and perspectives on the respective
issues based on various thought-provoking questions. In the third quadrant, the
Abstract Conceptualization (AC) or the How to process, they are expected to
read the necessary materials, such as textbook notes and presentation slides in
hope to solve problems in the last quadrant, Constructive Exploration (AE) or
"What If" process.

331
Figure 6. Kolb learning cycle for profit and loss statement
The activities put forward befit the definition of experiential learning which
stands on an interdisciplinary approach based on management, education, and
psychology, and implicating a holistic process of action/reflection built on
experience/abstraction (Kolb & Kolb, 2005). Past research in applying
experiental learning models in entrepreneurship and business education results
in positive learning outcomes achievement, measured by the Students Course
Experience Scale (Yusof et al., 2019).
One of the expected outcomes of experiential learning is student engagement

which is defined as the interaction between time, effort, and other resources
invested by students and institutions to optimize students’ experiences and
increase learning outcomes for their performance development and institutional
reputation (Trowler, 2010). Efforts towards student participation can be seen
through a diversified pedagogical approach to attracting and unburdening
accounting topics, including activity-based courses, sports, internet studies, and
computer simulations that combine to enable students to learn through real-time
accounting circumstances (Jaijaram, 2012). The ability for students to learn by
experience is recognized by AACSB (2013), which defines experiential learning
as one of the meaningful means of engaging students.
4. Methodology
This research is a quasi-experimental study which is used to study relevant
teaching and learning materials development to enhance students’ engagement.
Quasi-experimental research involves the manipulation of an independent
variable without the random assignment of participants to conditions or orders
of conditions. In this research paper, the researchers have implemented Kolb

332
learning cycle which takes eight weeks. 112 first-year students in Business
Accounting field are selected to participate in this study. In hope to meet the
aim of this study, important modifications are done on the course outline to
ensure that the class activities focus on enhancing students' experiential
learning. The T&L tasks are conducted differently where students have to go
around the loop in a clockwise direction in which the treatment group receives
the intervention (Kolb learning cycle) of learning in a process while participants
in the control group go through the usual lecture and demonstration method. A
pre-test questionnaire and a test set are designed and administered to students
from the Business Accounting course, in which the pre-test consists of a test set
with 24 item questions on students’ engagement and a test set of 9 items on
existing knowledge of business accounting. The same set is repeated to students
as a post-test after the intervention. At the end of the cycle that takes eight
weeks, 24 hours of lessons, students' level of engagement is measured using a
24-item scale adapted from Burch et al. (2015) who proposed a conceptualization
of student engagement grounded in A. W. Astin’s Student Involvement Theory
and W. A. Kahn's employee engagement research where student engagement is
built on four components: emotional engagement, physical engagement,
cognitive engagement in class, and cognitive engagement out of class.
Early findings indicate that students put a significantly high amount of time and
energy on experiential learning, which leads to increased engagement in feeling
and cognitive continua. On the other hand, classical instructional approaches are
effective if one finds the exam score to be the final learning indicator. Exam score
and drilling are successful opportunities to improve thinking abilities, such as
analytical skills in accounting. Four main topics in Business Accounting are
delivered through the Kolb Learning Cycle (KLC). The topics are as follows: I
Budgeting, II. Bookkeeping, III. Cash Flow, IV. Profit and Loss. These four topics
are delivered in four different cycles in eight (8) weeks.
5. Findings
The goal of the paper is to create a Business Accounting course in an experiential
learning loop that could offer a realistic model for learning fundamentals in acco
unting for entrepreneurs. It is assumed that as a lesson is prepared, it will
address Why, What, How, and What if that improves students’ learning and
increases their engagement. Each additional loop should be in the target position
to offer deeper awareness and dedication to lessons. This research believed that
experiential learning could contribute to student participation and therefore
improve achievement of learning outcomes. The course helps students to
demonstrate the practical knowledge of learning fundamentals in accounting for
entrepreneurs and improve their application of accounting experience in
entrepreneurship through physical, emotional, and cognitive engagement.
Business accounting content is grouped into four main topics designed into
experiential learning cycles to maximize students' learning experience.
Understanding why the issue is important motivates them to know more. Going
through the quadrant of concrete experience provides students with a big
picture which helps them see the topic's importance. The ‘Why’ quadrant offers
actual cases and circumstances for students. Next,the students learn through

333
their reflections of the big picture through the cases provided. Reflection is
shown to be a good engine of learning. Instructors prepared significant leading
questions to direct students through reflections.
For example, answers such as, "business fail because the owner did not
anticipate steep competitions from other food outlets that would cause him to
lose sales volume, should therefore provide more variations or extend business
hours to be competitive. Students directly relate their perceptions and
information to the lecture. In the meantime, engagement is at an emotional level.
The students' answers show that they even discuss the issue after class. Out-of-
class discussions strengthen cognitive engagement through the "How" quadrant.
The latter is the core concern of the syllabus and resources suggested by the
instructors. Emotionally engaged students find more examples and ask
questions about the new information they have discovered.The 'What if'
quadrant would take the students to what happens if any action is taken, and a
simulation of the predicted outcomes if any changes are made. The fourth
quadrant requires students to gather information from reliable sources. At the
end of the course, students' engagement is measured using Burch's (2015) Model
of Engagement, which has three sub-dimensions: physical, emotional, and
cognitive.
Figure 7. Four phases of the learning cycle.
Measurement of cognitive achievement is done through a test set consisting of

three cognitive ability levels, understanding, application, and analysis. This
study applies four dimensions of student engagement, considering the nature
and limitations of the students' experience to go through in this teaching cycle.
There is a difference in the level of engagement among the students after the
intervention. Simultaneously, there is an improvement in the cognitive score for
all three cognitive levels, understanding, application, and analysis. Findings of
this study conclude that experiential learning activities provide an opportunity
for students to demonstrate practical knowledge in accounting, and students are
physically, emotionally, and cognitively engaged while learning through the
various phases of experiential learning.
Table 1 indicates the difference in means for the four areas of students’
engagement. The largest mean is for physical engagement (mean 4.21, n.d. .42),

334
cognitive in-class engagement (mean 4.20, n.d. .46), cognitive out-of-class

engagement (mean 4.20, n.d. .47), and mental engagement (mean 4.14, n.d. .51).
There is a small increase recorded in the four dimensions of student
participation; cognitive in-class (.37), cognitive out-of-class (.33), physical (.31),
and emotional involvement (.30). There are also educational tools available to
educators, such as cases and challenges with developing teaching and learning
practices. As a consequence, educators using traditional teaching strategies may,
at some stages, use tools that provide an opportunity for students to experience
one or more of the four experiential learning quadrants. The results are
confirmed by Kraukuer, Serra, and de Almeida (2017), who observed greater
student participation and a sense of empowerment. Similarly, Pauleen, Marshall
and Egort (2014) found a complete learning experience by integrating theory and
practice through experiential learning.
Table 2. Domains of students’ engagement

Pre-test Post-test
Domains of student engagement
Mean s.d. Mean s.d.
Cognitive-in class
I am interested in the material I learn in this
3.83 0.79 4.27 0.53
class/course.
I am excited about coming to this
3.73 0.86 4.28 0.52
class/course.
When I'm in the classroom for this
lesson/course, I pay a lot of attention to class 4.01 0.58 4.23 0.43
topics and events.
I use the knowledge and skills I have gained
from this class/course to deal with other 3.72 0.42 4.16 0.34
students studying in the classroom.
I often explain course material to one or more
3.73 0.73 4.12 0.52
students for this class/course.
I've memorized the course materials for
assignments and exams in this class/course. 3.91 0.55 3.92 0.31
This class/course taught me to think

3.82 0.32 4.23 0.46
objectively and analytically.
This class/course helped me learn the
financial expertise of a small company. 3.91 0.71 4.36 0.57
Cognitive-out of class
I exert my full effort towards this
3.82 0.32 4.22 0.43
class/course.
I connect learning in this class/course to
3.74 0.81 4.21 0.52
societal problems or issues.
I reviewed my notes after class in this
4.05 0.61 4.10 0.51
class/course.
This class/course allowed me to learn
challenging real-world problems. 3.86 0.31 4.25 0.42

335
Physical
I am working with passion on assignments for

3.87 0.41 4.10 0.35
this class/course.
I feel positive about the assignments I have
completed in this class/course. 3.91 0.53 4.21 0.45
I work harder than I thought I could meet the

instructor's expectations in this class/course. 3.92 0.43 4.12 0.35
I work harder to produce good products in

this class/course during difficult assignments. 3.91 0.69 4.42 0.53
Emotional
I am enthusiastic about this class/course. 3.81 0.74 4.15 0.42
When I'm in this class/course, I feel energetic. 3.81 0.72 4.18 0.56
I sometimes ask questions or add to a
class/course debate. 3.73 0.83 4.10 0.64
I discussed class/course topics, ideas or
concepts with friends outside of class. 3.85 0.87 4.21 0.53
I have drawn results based on my study of

numerical knowledge (numbers, tables, 3.82 0.97 4.10 0.61
figures, etc.) in this class/course.
I used numerical information to examine a
real-world problem or issue in this 3.87 0.98 4.10 0.63
class/course.
This class/course helped me speak clearly and
3.93 0.83 4.11 0.34
effectively.
This class/course helped me work effectively
3.91 0.97 4.13 0.31
with others.
Table 3. Total difference of means and standard deviation.

Pre test Post test
Domains of student engagement Difference of mean
Mean s.d. Mean s.d.
Cognitive-in class 3.83 0.62 4.20 0.46 0.37
Cognitive-out of class 3.87 0.51 4.20 0.47 0.33
Physical 3.90 0.52 4.21 0.42 0.31
Emotional 3.84 0.86 4.14 0.51 0.30
Table 4 displays the test score on comprehension of the basics of financial

management for entrepreneurs.There is growth in the cognitive score for all
three cognitive stages, understanding (13%), application (25%) and analysis
(15%).
Table 4: Test score on students' knowledge

Types of questions Pre-test score (%) Post-test score (%) Difference (%)
Understanding 70 83 13
Application 35 60 25
Analysis 45 60 15

336
6. Conclusion, limitations and Recommendations

This study attempts to adapt to the Kolb Learning Cycle Model to design lessons
for Business Accounting. Four main topics areorganized into four different
cycles, which provide a different learning experience for the students. Students
participate in the four learning cycles and are offered alternative ways of
perceiving and processing the information. The materials used for the lessons
play an important role in engaging the students. Students will receive current
and popular types of business as stimuli because of their familiarity and interest
in the subject. Awareness of the 'Why' do I have to learn this sub-topic becomes
the driver for students to further understand. The stimulus must be relevant and
current to make students curious enough to do further research and discussions
on the subject matter. Variation of experiential learning programs can offer more
opportunities for participation, such as fieldwork and on-site learning.
Improving students’ engagement, particularly on emotional and cognitive out-
of-class engagement make the learning process more productive. The quadrant
of What is an important reflection in which educators direct students through
their reflections. Answers given by students demonstrate their thinking and the
ability for educators to know their previous understanding of the subject. In the
'How' quadrant, showing students notes and slides allow them to get a better
idea of the relevant topics. The 'What if' quadrant allows them the to apply their
experience from the three preceding quadrants to one major implementation of
practical situations customised to brief problems or case studies.These activities
can be unstructured, where students have to justify their acts.
Business Accounting is an applied functional subject that should be delivered in

a practical way. Creating an environment in the classroom is a way to address
time constraints and opportunities for students to do field study or work-based
learning. The limitations of this analysis are the number of samples involved.
Extending the study to other HEIs would serve to make the results of this study
more effective. Another study area to be discussed will be applying the
experiential learning approach to online learning for more analysis. Well-
planned experiential learning lessons can be provided by online learning
leveraging multimedia tools that are more attractive to the z generation. The
implications of this research is the support a sound framework for the
introduction of an experiential learning approach. Flexible scheduling of classes
will provide enough time for class events. A strong internet connection for
connecting online services as part of the classroom experience is important.
7. Acknowledgement
This study was supported by UPSI Teaching and Learning Innovation Grant
(GPU) 2017-0255-107-01.
8. References
AACSB. (2013). Eligibility procedures and accreditation standards for business accreditation.
Tampa, FL.
Arditi, A., Koksal, A., & Kale, S. (2000). Business failures in the construction industry.
Engineering, Construction, and Architectural Management, 7(2), 120-132.

337
Beard, C., & Wilson, J. P. (2002). The power of experiential learning: a handbook for trainers
and educators. Kogan Page Limited, London.
Burch, G. F., Heller, N. A., Burch, J. J., Freed, R., & Steed, S. A. (2015). Student
engagement: Developing a conceptual framework and survey instrument.
Journal of Education for Business, 90(4), 224-229.
https://doi.org/10.1080/08832323.2015.1019821
Garkaz, M., Banimahd, B., & Esmaeili, H. (2011). Factors Affecting Accounting Student's
Performance: The Case Of Students At The Islamic Azad University. Procedia -
Social and Behavioral Sciences, 29, 122–128.
Ghasemi, M. R., Moonaghi, H. K., & Heydari, A. (2018). Student-related factors affecting
academic engagement: A qualitative study exploring the experiences of Iranian
undergraduate nursing students. Electronic physician, 10(7), 7078–7085.
https://doi.org/10.19082/7078
Harb, J. N., & Terry, R. E. (1995). Application of the Kolb Learning Cycle to Design
Instruction. ASEE Rocky Mountain Section Meeting, Golden, Colorado.
Juneja, P. (2017). Management Study Guide. Retrieved from
https://www.managementstudyguide.com/financial-management.htm
Kamaruddin, R. (2015). Hubungan faktor pendidikan dan persekitaran dengan aspirasi kerjaya
perakaunan dalam kalangan pelajar di negeri Pahang (Master’s thesis). Universiti
Pendidikan Sultan Idris.
Kapur, R. (2018). Factors Influencing the Students' Academic Performance in Secondary
Schools in India. Retrieved from
https://www.researchgate.net/publication/324819919_Factors_Influencing_the
_Students Academic_Performance_in_Secondary_Schools_in_India
Kolb, A. Y., & Kolb, D. A. (2005). Learning styles and learning spaces: Enhancing
experiential learning in higher education. Academy of management learning &
education, 4(2), 193-212.
Kolb, D. A. (1984). Experiential Learning: Experience as the Source of Learning and
Development. Prentice-Hall, Englewood Cliffs, N.J.
Kraukuer, P. V. D. C., Serra, F. A. R., & de Almeida, M. I. R. (2017). Using experiential
learning to teach entrepreneurship: a study with Brazilian undergraduate
students. International Journal of Educational Management, 31(7), 986-999.
Ndege, T. (2010). Students’ Engagement and Student Retention in Moi University,
Kenya. Global Business and Economics Anthology, 2, 247-257.
Pauleen, D. J., Marshall, S., & Egort, I. (2014). ICT-supported team-based experiential
learning: Classroom perspectives. Education+ Training, 46(2), 90-99.
Roudaki, J. (2016). Undergraduate accounting programmes in developing countries: The case of
Iran. Australia: University of Wollongong.
Taylor, L., & Parsons, J. (2011). Improving Student Engagement. Current Issues in
Education, 14(1). Retrieved from http://cie.asu.edu.
Trowler, V. (2010). Student engagement literature review. York: The Higher Education
Academy.
Wyhe, V., & Arthur, G. (1991). The accounting curriculum in higher education: A study in
educational policy. University of Washington: ProQuest Dissertations Publishing.
Yusof, R., Yin, K. Y., Ahmad, A. S., Halim, H., & Norwani, N. M. (2019). Students Course
Experience: Reflections of Outcome- Based Education Implementation in
Accounting Programmes in Higher Education. International Journal of Academic
Research in Progressive Education and Development, 8(2), 228–238.

338

Vol. 19, No. 10, pp. 338-354, October 2020
The Value of Feedback in Primary Schools:

Students’ Perceptions of the Practice
Abatihun Alehegn Sewagegn

University of Johannesburg, South Africa
https://orcid.org/0000-0003-0147-5190
Askalemariam Adamu Dessie

Debre Markos University, Ethiopia
https://orcid.org/0000-0001-8440-8879
Abstract. Feedback is a crucial aspect in assessment that enables

students to learn from assessments received. The perception of the
practice of feedback has an impact on the fruitfulness of the teaching-
learning process. Hence, the main purpose of the study was to assess
primary school students’ perceptions of the practices of feedback. To
attain this purpose, a descriptive survey design was used, and 474
students from two government and two private primary schools
participated. Questionnaires and document reviews were used to collect
data and analysed using quantitative and qualitative techniques. The
results indicated that students favourably perceive the value and
practice of feedback. Most of the students considered feedback as a
provision of marks, grades, and correct answers. Yet, judgmental
feedback using words such as “excellent”, “very good” and “good” used
in the students’ exercise books have little value in showing learning
gaps and as such, this type of feedback is not useful in improving
students’ learning. Therefore, for feedback to be useful, students should
have a positive perception of feedback and take the feedback into
account for future learning to enhance their knowledge and
understanding.
Keywords: feedback; perception; practice; students; value
1. Introduction
Feedback refers to “commentary, verbal or written information that a student
receives about their performance” (Marrs, 2016, p. 7). In the academic
environment, feedback is an essential component and a vital strategy of the
teaching-learning process (Ansari & Usmani, 2018). If there is an assessment in
the teaching-learning process, then there is feedback, and it is given in different
forms to the students. Helping students to learn from their activities, mainly
through encouraging dialogue, is a crucial aspect of feedback (Irons, 2008).

339
Feedback is central to the development of effective educational practice (Sadler,

2010) and the students’ perceptions of the practices of feedback have an impact
on the fruitfulness of the teaching-learning process. The students’ perceptions,
understanding of the purpose, and beliefs about feedback are crucial
determinants for achieving educational outcomes (Ansari & Usmani, 2018). In
the absence of meaningful feedback, good practices are not reinforced, poor
performance is not corrected, and the path to improvement is not identified
(Ansari & Usmani, 2018).
According to Irons (2008) and Sadler (2010), the feedback given should be
constructive to bring about improvement in students’ learning. Constructive
feedback helps learners to become more self-regulated, develop intrinsic
motivation and positive self-esteem, identify and fill gaps in their current and
future understanding, and encourages dialogue between teachers and students
(Irons, 2008; Nicola & Macfarlane-Dick, 2006). Therefore, a key aspect of effective
feedback whether written or oral, is that the students need to learn what is
required of them.
Effective feedback starts a thought process for students as it allows them to

evaluate the quality of their work against that of their teacher or peers and
empowers them to become self-regulated learners which effectively improves
their learning abilities (Fautley & Savage, 2008; Lee, 2006; Moss & Brookhart,
2009; Sadler, 2010). According to Li and De Luca (2014), for feedback to be
effective it should be timely and relate to the focus of the learning; it needs to
acknowledge where the student has been successful and identify where and how
improvement can take place. It also needs to be able to be clearly understood by
the student, allowing them the time needed for improvement. According to
Amua-Sekyi (2016), the feedback given becomes useful when it is provided
quickly enough and acted upon to improve student’s work and learning. The
feedback should be of value to the student by ‘closing the gap’ on their
understanding and providing an opportunity for dialogue to occur (Irons, 2008).
However, research evidences indicated as assessment results are not utilized

effectively for the purpose of providing timely and constructive feedback rather
used for grading purposes (Amua-Sekyi, 2016; Acar-Erdol & Yildizli, 2018; Tan,
Whipp, Gagné & Quaquebeke, 2019). Therefore, this study assessed primary
school students’ perceptions of the practice of feedback. Unlike in other studies,
in this research both quantitative and qualitative methods were used and
students were the primary sources of the data. In other words, feedback should
provide information about how the student did with regards to the desired aim
or outcome and then further highlight how their performance could be
improved (Tan, Whipp, Gagné & Quaquebeke, 2019). Therefore, this study
assessed primary school students’ perceptions of the practice of feedback.
The topic of students’ perceptions of the practice of feedback is an under‐

researched area, and there has been little empirical research published which
focuses on students’ perceptions of the practice of feedback at primary schools in

340
the Ethiopian context. Moreover, to the knowledge of the researchers, in this

study area and topic, no complete research has been done to assess students’
perceptions of the practice of feedback to improve their learning at primary
schools. Thus, based on these issues, the following research questions were
raised.
• How do students perceive the value of feedback on their learning?
• How do students use feedback to improve their learning?
• To what extent do students’ perceptions influence their use of feedback in
their learning?
2. Review of Literature
2.1 The link between assessment and feedback
Assessment and feedback are the main components of the teaching-learning
process, and there is a close link between the two. Hattie and Timperley (2007,
p. 1) noted that “feedback is an essential component of assessment for learning”,
and when it is “used appropriately, can support students’ learning and lead to
substantial learning gains”. Brookhart (2008) and Irons (2008) also stated that
feedback is an essential component of the formative assessment process.
Specifically, formative assessment and formative feedback are closely related
(Irons, 2008). Irons (2008) also added that formative assessment and formative
feedback are compelling and potentially constructive learning tools.
According to Oluwatayo, Irene and Appah (2014), feedback is the central

function of formative assessment, and teachers’ good feedback practices
strengthen the students’ capacity to self-regulate their performance. Feedback on
assessments is an effective and efficient teaching and learning tool that is
powerful in enhancing and strengthening students’ learning (Sadler, 2010).
Quality feedback needs to be relevant to what is being assessed and to the

learning process of the individual student. There are several ways of measuring
the quality of feedback such as the timeline in which it is given (immediacy) and
if it encourages students’ learning and by how much (either too little or too
much) (Irons, 2008). The most effective feedback is that which is given as close to
the completion of the task as possible (Cowan, 2003 cited in Irons, 2008). As
regular constructive feedback can have a positive impact on students’ academic
performance, not providing timely feedback will hamper their progress (Irons,
2008).
2.2 Students’ perceptions on the value of feedback

Student’s perceptions of feedback include what they believe the purpose of
feedback is, how they feel about or value feedback that is given, and their
interpretation of that feedback (Van Der Kleij & Adie, 2020). Students might
often find teachers’ feedback confusing and very challenging to understand;
often they might think they have understood the feedback when they have not,
and even if they do manage to understand it, they may not know how to put it
into practice (Sutton, Hornsey & Douglas, 2011).

341
Due to their perceptions and beliefs about feedback, students may not
understand the importance of it and pay it little attention, focusing on getting
good grades rather than on the comments received (Ansari & Usmani, 2018;
Carless, 2006). If the student does not know how to use the comments received
to improve their work it undermines its effectiveness – this needs to be
considered by teachers and schools when deciding on the format of their
feedback practices (Bevan, Badge, Cann, Willmott, & Scott, 2008 cited in Ansari
& Usmani, 2018).
As Irons (2008, p. 38) stated, “students can be demotivated as a result of

feedback particularly when: they perceive the feedback to be unfair; feedback is
unclear; they don’t understand the feedback; the feedback doesn’t seem to relate
to the work they have done; they don’t receive feedback in time; feedback is
overly critical, and feedback is non-constructive”. Spiller (2014) also noted that
the following are the concerns of students regarding the feedback they are
provided:
• Feedback comments are not understandable;
• Feedback does not point them in the right direction so that they can
improve their work;
• The purpose of feedback is not clear to them;
• The relationship between the comments received and the grade given may
create confusion;
• Feedback is either too much or too little;
• There are no opportunities to talk about the feedback;
• Feedback is not timely and therefore is not helpful;
• The reason why that specific feedback is given is unclear and
• Feedback comments can upset or embarrass a student.
2.3 Value/use of feedback to improve students’ learning

Feedback can only be useful if students understand it and are willing and able to
act on it (Price, Handley, Millar & O’Donovan, 2010). As Marrs (2016) argues, it
is most beneficial when students actively use the feedback they receive. Properly
given feedback enhances the students’ learning and has positive impacts on their
performance. Irons (2008) listed the following positive benefits of feedback from
the students’ perspective:
▪ Feedback is used to learn;
▪ Feedback helps students to more fully understand their learning
requirements and their subject understanding;
▪ Feedback can motivate students in their studies;
▪ Students are encouraged to try new ways of learning and
▪ Feedback can be used to reflect on learning practices and activities.
According to Di Costa (2010, p. 2), the characteristics of acceptable or good

feedback practices are that they “clarify what good performance is; provide
opportunities to close the gap; deliver high-quality information to students
about their learning; facilitate the development of self-assessment (reflection) in
learning; boost teacher and peer dialogue around learning; encourage positive

342
motivational beliefs and self-esteem; and provide information to teachers that

can be used to help shape teaching”.
3. Methods
3.1 Research Design
The main purpose of the study was to assess students’ perceptions of the
practice of feedback to improve their learning in primary schools. To attain the
purpose, a descriptive survey design was used. Both quantitative and qualitative
methods were used in the study.
3.2 Population, Participants and Sampling Techniques

The study was carried out at Debre Markos primary schools. Government and
private second cycle primary schools (grade 5 to 8) students were the population
of the study. Biruhi Tesfa and Dibiza government primary schools and Lingerh
and Selam Academy private primary schools were randomly selected. To
determine the sample size, stratified random sampling was employed. In the
strata, school and grade levels were considered. Finally, the sample sizes for
each school were determined based on the sample size determination guidelines
proposed by Cohen, Manion and Morrison (2018). Thus, the nearest sample
numbers to the guideline were proposed for each school.
Table 1: Sample size from each primary school

School name Total No. students Students
f %
Biruhi Tesfa 358 138 29.1
Dibiza 341 146 30.8
Lingerh 104 99 20.9
Selam Academy 89 89 18.8
Total 892 474 100.0
Note: f =frequency; % = percentage
As indicated in Table 1, 138 (29.1%) and 146 (30.8%) students were selected from
Biruhi Tesfa and Dibiza primary schools, respectively. Besides these, 99 (20.9%)
of the students were selected from Lingerh primary school, whereas 89 (18.8%)
of them were selected from Selam Academy. 474 students were selected in total
from the four primary schools.
3.3 Instruments
To answer the research questions, data from questionnaires and document
reviews were collected. A 5-point Likert scale questionnaire was used for the
quantitative section of the study. The questionnaire (see Appendix 1) had three
sub-scales (students’ perceptions on the value of feedback which had 15 items,
students’ perceptions on the practice of feedback which had 14 items and
feedback usage sub-scale which had 13 items). The questionnaire had 42 items,
including the background information.
For the qualitative aspect, document review was used. The document review
mainly focused on students’ exercise books to assess the content and form of the
teachers’ feedback, whether it was judgmental or constructive.

343
3.4 Data Analysis

To analyze the quantitative data, descriptive statistics (frequency, percentage,
and mean) and inferential statistics (one sample t-test and multiple regression)
value were employed. Moreover, correlation was computed to see the
relationship between students’ perceptions on the power of feedback and
practice. To enhance clarity at the descriptive analysis level, specifically for
percentage, it was decided to collapse the original 5-point Likert scale into three
categories (Agree, Undecided, and Disagree) and (Often, Sometimes, and
Rarely). According to Babbie (2010), collapsing Likert responses is acceptable
when one wants to see a clearer pattern in the findings. The document review
was analyzed qualitatively using words to triangulate the quantitative data.
3.5 Reliability and Validity

Reliability refers to the dependability, stability, and predictability of responses
(Creswell, 2007; De Vos, Strydom, Fouche, & Delport, 2011). To test the
reliability of items in the questionnaire, a pilot study was conducted using 20
students, and Cronbach Alpha was used to analyze the reliability and the results
for the three sub-scales were 0.594, 0.588 and 0.706 for students’ perceptions on
the power of feedback, students’ perceptions of feedback practice, and its usage,
respectively. After revision, the reliabilities of the students’ questionnaire for the
main study were 0.73, 0.71 and 0.756 for the above three sub-scales respectively,
which is acceptable based on George and Mallery’s (2003) standard.
To increase the validity of the questionnaire, three selected experts in education

and psychology helped in reviewing the questions. De Vos et al. (2011)
established that content validity and face validity of the instrument can be
established on the basis of researchers’ and other experts’ judgments.
4. Findings
The findings were organized based on the research questions. In this section,
students’ perceptions on the value of feedback, perceptions on the practice of
feedback and their use of feedback are presented.
4.1 Students’ perceptions on the value of feedback

To determine the students’ perceptions on the value of feedback on students’
learning, 15 items were used, and the analysis was done using frequency,
percentage, and mean as indicated in Table 2.
Table 2: Students’ perceptions on the value of feedback

Responses
No. Items Agree Neutral Disagree
f % f % f % Mean
1 Feedback is helpful to show gaps in 425 89.3 16 3.4 32 6.8 4.32
knowledge and understanding.
2 Feedback rarely provides me with 42 8.9 48 10 378 79.8 2.01
useful suggestions for improvement.
3 I consider that peer feedback is 303 63.9 94 19.8 73 15.4 3.66
essential for learning.
4 Teacher feedback is effective in 422 89 17 3.6 31 6.5 4.37

344
promoting students’ learning.

5 Constructive criticism is needed to 299 63.1 56 11.8 114 24 3.58
know how to improve my work.
6 Feedback given at the end of the 106 22.4 48 10.1 319 67.3 2.32
subject is not useful.
7 The feedback which is given should 442 93.2 15 3.2 17 3.6 4.49
be clear to be helpful.
8 Feedback encouraged me to improve 369 77.8 61 12.9 39 8.2 3.99
my work.
9 Positive comments increase my 381 80.4 39 8.2 46 9.7 4.15
confidence.
10 I thought about giving up when I got 52 11 74 15.6 340 71.8 1.93
negative feedback.
11 I felt demoralized or angry after 134 28.3 85 17.9 248 52.4 2.58
reading negative feedback.
12 Feedback has helped me to reflect on 363 76.6 60 12.7 36 7.6 4.02
what I have learned.
13 I need to receive positive feedback to 366 77.3 35 7.4 68 14.3 3.97
progress.
14 I need to receive negative feedback to 207 43.7 62 13.1 200 42.2 2.99
progress.
15 Teachers’ comments are vital in my 448 94.5 10 2.1 15 3.2 4.66
learning.
As indicated in Table 2, most students perceived that teachers’ feedback is useful

to promote their learning, to reflect on what they have learned, and to identify
the learning gaps. Besides this, according to the perception of students, feedback
should be clear, positive, and constructive to improve their learning and
confidence.
80.4% and 77.3% of students agreed on the value of positive feedback to increase
their confidence and showed their interest in receiving such feedback to improve
their learning, respectively. Even if 28.3% of the students agreed that negative
feedback demoralizes them and affects their future learning, 43.7% of them
perceived that the negative feedback given has the power to help them perform
better in their future learning.
4.2 Students’ perceptions on the practice of feedback

In Table 3, 14 items were included to assess students’ perceptions on the practice
of feedback by their teachers, and the analysis was done using frequency,
percentage, and mean.
Table 3: Students’ perceptions of the practice of feedback

Responses
No Items Agree Neutral Disagree
f % f % f % Mean
1 On the subject that I am learning, I 251 52.9 103 21.7 114 24 3.40
get plenty of feedback on how I am
doing.
2 Teachers give me immediate 227 47.9 81 17.1 163 34.4 3.20
feedback for each assessment.

345
3 There is hardly any feedback on my 93 19.6 67 14.1 311 65.6 3.56

assignments when I get them back.
4 When I get things wrong, I don’t 98 20.7 51 10.8 318 67.1 3.63
receive much guidance in what to
do about it.
5 I would learn more if I received 372 78.4 49 10.3 47 9.9 4.10
more feedback.
6 Teachers’ feedback that comes too 186 39.3 98 20.7 185 39 3.00
late is not useful for learning.
7 Teachers provide enough feedback 307 64.7 62 13.1 93 19.5 3.60
on my work.
8 The feedback mainly tells me how 336 70.9 61 12.9 69 14.6 3.80
well I am doing in relation to
others.
9 Teachers’ feedback is rarely written 289 60.9 102 21.5 72 15.2 3.50
with reference to the assessment
criteria.
10 The feedback given helps me to 392 82.7 35 7.4 40 8.5 4.20
understand things better.
11 The feedback given shows me how 408 86.1 25 5.3 37 7.8 4.20
to do better next time.
12 Once I have read the feedback, I 326 68.8 72 15.2 67 14.2 3.80
understand why I got the mark I
did.
13 I don’t understand some of the 198 41.8 97 20.5 172 36.3 3.10
feedback given.
14 I can seldom see from the feedback 409 86.3 18 3.8 44 9.3 4.20
given what I need to do to improve.
As indicated in Table 3 above, most of the students perceived that the more
feedback they receive from their teachers, the better they learn. The feedback
given to the students helps them to understand things better (82.7%) and shows
how to do tasks better the next time (86.1%). Even if 41.8% of the students
sometimes do not understand some of the feedback given to them from their
teachers, 86.3% of them perceived that they sometimes do understand the
feedback given and what they need to do to improve their learning.
As shown in Table 3, 68.8% and 70.9% of the students agreed that the feedback
given to them indicates why they got the mark they did and how well they are
doing when compared to other students, respectively. Most students agreed that
their teachers provide enough and timely feedback for different assessment tasks
and provide guidance/support on how to do better the next time. However,
34.4% of the students agreed that their teachers do not provide immediate
feedback to them.
Results from document review (students’ exercise books) showed that the
feedback which is given is general and judgmental such as excellent, very good,
good, etc. In addition, teachers provide feedback in the form of marks or
numbers. However, such types of feedback are less powerful in improving
students’ learning and do not clearly show what students did and did not do or
highlight the learning gaps.

346
4.3 Students use of feedback to improve their learning

Feedback was useful when we used the comments to identify learning gaps and
to propose strategies for future learning. Here, 13 items were used to examine
whether students were using the feedback given to improve their learning or
not. The analyses were made using frequency, percentage, and mean. The results
are indicated in Table 4 and 5 below.
Table 4: Students’ use of feedback to improve their learning

No. Items Responses
Rarely sometimes Often
f % f % f % Mean
1 I read the feedback carefully and try 65 13.7 107 22.6 295 62.3 3.70
to understand what the feedback is
saying.
2 I use the feedback to go back over 78 16.4 88 18.6 303 64 3.70
what I have done in the assignment.
3 The feedback given with any 68 14.4 80 16.9 316 66.7 4.00
subsequent assignments helps me.
4 The feedback prompts me to go back 57 12 64 13.5 344 72.5 3.90
over material covered earlier in the
course.
5 I read the comments carefully and 60 12.7 96 20.3 312 65.8 3.80
identify the gaps in my learning.
6 I use the feedback to compare my 84 17.7 127 26.8 254 53.6 3.70
progress with other students.
7 I do not use the feedback for revising 370 78 43 9.1 46 9.7 4.28
my work
8 I acted on suggestions to improve 63 13.3 50 10.5 354 73.9 4.00
my coursework.
9 I have good intentions, but forget 373 78.7 55 11.6 40 8.4 4.26
comments for improvement next
time.
10 I have ignored negative or critical 302 63.7 86 18.1 75 15.9 3.73
feedback.
11 I tend to only read the marks 324 68.3 75 15.8 66 13.9 3.89
compared to other students.
12 After reading comments, I re-work 91 19.2 127 26.8 252 53.2 3.50
any assessment task.
13 I read comments and plan strategies 52 10.9 76 16 333 70.2 4.00
for future learning.
Note: f =frequency; % = percentage
In Table 4, the mean values showed that the students use the feedback given by
their teachers to improve their learning for the future. However, compared to
other items presented to them, the tendency of doing their assessment tasks
again after reading comments is low.
While 12.7% of the students hardly use the feedback given by their teachers,
65.8% of students regularly read their comments and identify the gaps in their
learning for future success. Most students use the feedback given to revise their
work (78%), to improve tasks given (73.9%), and to plan strategies for
improvement the next time (78.7%). A reasonable number of students (53.6%)

347
use the feedback given to compare their progress with that of their peers;
however, using feedback for comparative purposes has little value for learning,
rather, such type of usage undermines the confidence and self-esteem of low
achieving students.
Table 5: One sample t-test

Participants Variable Expected Mean Observed mean SD t-value Sig
Students PPFL 45 57.06 7.12 174.58 .000*

(N=474)
PPF 42 47.89 6.59 158.08 .000*
Usage 39 49.94 7.62 142.32 .000*
Note: PPFL= Perception on the power/value of feedback on learning;
PPF= Perception on the practice of feedback; PP = Perceived practice
The one-sample t-test result shows that the observed mean is greater than the
expected mean in the case of students’ perceptions on the power of feedback to
improve their learning and significant difference was observed (t = 174.58,
158.08, 142.32; p = 0.000). Thus, we can conclude that students have high and
positive perceptions on the power of feedback on their learning. Moreover,
students perceived positively the practice of feedback in their schools. The one
sample t-test also shows that a significant number of students use the feedback
provided by their teachers to improve their learning.
The qualitative data result that was obtained from document review showed
that feedback given to students is in the form of marks and words such as
‘excellent’, ‘good’, and ‘very good’ for correct answers. These have no value in
showing where students might have learning gaps and does not help them with
strategies for future learning.
Table 6: Inter correlation between students’ perceptions and usage of feedback

Usage PPF PPFL
Pearson Correlation
Usage Sig. (2-tailed)
N 472
Pearson Correlation .425**
PPF Sig. (2-tailed) .000
N 472 473
Pearson Correlation .451 ** .388**
PPFL Sig. (2-tailed) .000 .000
N 472 473 474
**. Correlation is significant at the 0.01 level (2-tailed).
Note: PPFL= Perception on the power/value of feedback on their learning; PPF= Perception on
the practice of feedback
The result of Pearson correlation in Table 6 strengthens the above finding. It

indicates that there is a significant relationship between students’ usage of
feedback to improve their learning and their perceptions on the practice of
feedback in the classroom (r = 0.425, n= 472, p < 0.01). Moreover, as indicated in

348
Table 6, students’ perceptions on the power of feedback on their learning has a

significant relationship with their perceptions on the practice of feedback
(r = 0.388, n = 473, p < 0.01).
Table 7: The multiple regressions model summary and analysis of variance

Model Sum of Mean
Squares df Square F Sig.
Multiple R 0.52 Regression 7458.92 2 3729.46 87.84 .000
R Square 0.273 Residual 19912.30 469 42.46
Adjusted R 0.269 Total 27371.22 471
SEM 6.52
Note: SEM= Standard Error of the Estimate; df = Degree of freedom; F = F-test
Multiple regression was employed to assess how much variance in students’

usage of feedback can be explained by their perceptions on the power of
feedback on their learning and the practice of feedback in the classroom (by the
two independent variables). The result of the multiple regressions or the
adjusted R square in Table 7 indicates that 26.9% of the variance in students’
feedback usage is attributed to the variance of the combination of the two
independent variables (i.e. students’ perceptions on the power of feedback and
practice of feedback). Moreover, F-test was employed to determine whether the
result was statistically significant or not. The result showed that the relationship
between the dependent variable and the combination of the two independent
variables (perception on the power of feedback and perception on the practice of
feedback) was statistically significant at F (2,469) = 87.84, p = 0.000.
Table 8: Significance of the predictor variable

Variable B SE Beta t Sig
PPFL .375 .045 .350 8.290 .000
PPF .302 .045 .282 6.676 .000
Note: B =Beta for unstandardized coefficient; SE = Standard Error; t = t-test; PPFL= Perception
on the power/value of feedback on their learning; PPF= Perception on the practice of feedback
The t-test result showed that from the two independent variables (PPFT and
PPF), students’ perceptions on the power of feedback on their learning makes a
major significant contribution to explain the dependent variable when the
variance accounted by all other variables in the model is controlled for (beta =
0.35). Students’ perceptions on the practice of feedback in the classroom also
made a statistically significant (p = 0.000 < 0.01) contribution (beta = 0.282) to the
regression model.
5. Discussion
5.1 Students’ perception on the value and practice of feedback
To provide students with the best opportunity to improve their knowledge and
necessary skills in a required subject, it is essential that teachers are aware of
their feedback styles, nature, and purpose. According to Weaver (2006), the
nature and style of feedback provided by teachers can affect the way students

349
perceive and use feedback in their learning. Thus, the way teachers value and
provide feedback will affect students’ usage and perception of it.
The results of the study showed that students positively perceive the value of
feedback in improving their learning. This result is in line with Weaver (2006)
and recommends that improvement in learning occurs when students perceive
feedback as enabling learning, and not just as a judgment on their level of
achievement. Moreover, most students in this study positively perceived the
practice of feedback in their schools, yet the qualitative data indicated that the
provision of feedback to improve students’ learning is not practiced in the way it
should be. This mismatch between the qualitative and quantitative result
suggests that students might not really understand the nature, form, and content
of the feedback needed to improve learning. Most of them consider effective
feedback as a provision of marks, grades, and correct answers, but it is more
than these.
Research evidence shows that teachers need to review their beliefs and
understanding on the purpose, language, and content of feedback they provide
(Weaver, 2006), as it will then have a better connection with that of students and
help address their main concerns. According to Orsmond and Merry (2011),
student responses to feedback were influenced by their perception of the teacher
giving the feedback. For example, the results of this study suggest that students
considered the provision of marks, correct answers, and praise as effective
feedback to improving learning.
5.2 Students’ use of feedback to improve their learning

Hattie and Timperley (2007) argued that the main purpose of feedback is to
reduce the gap between students’ current understanding and the learning
objectives stated in the subject. Therefore, for Hattie and Timperley, feedback
must help students answer these questions: where am I going, how am I going,
and where to go next? However, as indicated in different literature, all types of
feedback do not assist students to improve their learning. For example, even if
the results of this study indicates that students are perceived as they are using
the feedback given by their teachers to enhance their learning, they did not get
the feedback on different assessment tasks on time and the tendency of doing
their assessment tasks again after reading the comments given by their teachers
is low. According to Amua-Sekyi (2016, p. 4), “when feedback on students’ work
is delayed, it loses its relevance”.
On the other hand, a reasonable number of students (53.6%) in this study use the
feedback given to compare their progress with that of their peers; however,
using feedback for comparative purposes has little value for learning and this
type of usage can undermine the confidence and self-esteem of those students
that are low achievers (Dinnen & Collopy, 2009). Lee (2006) suggested that for
feedback to be effective it must inform the student how to improve; specific
suggested actions must be clearly stated so that the student can act on them.

350
Similarly, Weaver (2006) claimed that students might not use feedback to
improve their learning; some feedback may be limited and not point the way
forward or motivate students to take further action, or they may have
insufficient understanding of the subject matter to interpret comments
accurately. When a teacher corrects all mistakes or copyedits written work, the
student does not get an opportunity to figure anything out and to implement
different strategies in their next steps in learning.
Generally, even if students perceive they are using feedback to improve their
learning, the way they use the feedback provided is not appropriate for their
further learning. This happens because of the nature, content, and purpose of
feedback provided to them and teachers’ misconceptions on the nature of
effective feedback.
6. Conclusion
Most of the students in the selected schools perceived that teachers’ feedback is
effective to promote their learning, to reflect on what they have learned, and to
identify the learning gaps. To improve students’ learning and confidence, the
feedback given should be clear, positive, and constructive. The feedback given to
students helps them to understand things better and shows how to improve on
their future tasks. The results of this study show that the students use the
feedback given by their teachers to improve their learning, however, the
tendency of doing their assessment tasks again after reading the comments
given by their teachers is low. A reasonable number of students use the feedback
given to compare their progress with that of their peers; however, using
feedback for comparative purposes has little value for learning, especially as
such comparisons undermine the confidence and self-esteem of low achieving
students. Students have a high and positive perception on the value of feedback
on their learning. Moreover, students positively perceive the practice of
feedback in their schools. The results also showed that students, to a large
degree, use the feedback provided by their teachers to improve their learning.
The significant relationship observed between students’ usage of feedback and
students’ perceptions on the practice of feedback in the classroom helps improve
their learning. Moreover, the significant relationship between the students’
perceptions on the power of feedback and their perceptions on the practice of
feedback also improves their learning. Judgmental feedback using words such as
‘excellent’, ‘very good’, and ‘good’, were used in the students’ exercise books
which has little value in showing learning gaps and strategies for future learning
and this type of feedback is not effective in improving students’ learning.
Effective feedback is constructive and shows the gaps in understanding and
suggests ways for the students to fill those gaps in the future by themselves.
Therefore, for feedback to be meaningful, students should have a positive
perception of feedback and take the feedback into account for future learning to
enhance their knowledge and understanding.

351
7. References
Acar-Erdol, T., & Yildizli, H. (2018). Classroom assessment practices of teachers in
Turkey. International Journal of Instruction, 11(3), 587-602.
https://doi.org/10.12973/iji.2018.11340a
Amua-Sekyi, E. T. (2016). Assessment, student learning and classroom practice: A
review. Journal of Education and Practice, 7(21), 1-6.
Ansari, T., & Usmani, A. (2018). Students’ perception towards feedback in clinical
sciences in an outcome-based integrated curriculum. Pakistan Journal of Medical
Science, 34(3), 702-709. doi: https://doi.org/10.12669/pjms.343.15021
Brookhart, S. M. (2008). How to give effective feedback to your students. ASCD: Virginia.
Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher
Education, 31(2), 219-233. doi: 10.1080/03075070600572132
Cohen, L., Manion, L., & Morrison, K. (2018). Research methods in education (8th ed.). New
York: Routledge.
Creswell, J. (2007). Qualitative inquiry and research design: Choosing among five approaches
(2nd ed.). University of Nebraska: Sage Publications Ltd.
De Vos, A. S., Strydom, H., Fouche, C. B., & Delport, C. S. L. (2011). Research at grass roots:
For the social science and human service professions (4th ed.). Pretoria: Van Schaik.
Di Costa, N. (2010). Feedback on feedback: Student and academic perceptions,
expectations and practices within an undergraduate Pharmacy course. ATN
Assessment Conference, University of Technology, Sydney.
Dinnen, J. L. D., & Collopy, R. M. B. (2009). An analysis of feedback given to strong and
weak student writers. Teacher Education Faculty Publications, 49(3), 239-256.
Fautley, M., & Savage, J. (2008). Assessment for learning and teaching in secondary schools.
British: Learning Matters Ltd.
George, D., & Mallery, P. (2016). IBM SPSS statistics 23 Step by step: A simple guide and
reference. New York: Routledge.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research,
77(1), 81-112.
Irons, A. (2008). Enhancing learning through formative assessment and feedback. New York:
Routledge.
Lee, C. (2006). Language for learning mathematics: Assessment for learning in practice. New
York: Open University press.
Li, J., & De Luca, R. (2014). Review of assessment feedback. Studies in Higher Education,
39(2), 378-393, https://doi.org/10.1080/03075079.2012.709494
Marrs, S. A. (2016). Development of the student perceptions of writing feedback scale
(Unpublished doctoral dissertation). Virginia Commonwealth University,
Richmond, Virginia.
Moss, C. M., & Brookhart S. M. (2009). Advancing formative assessment in every classroom.
Virginia: ASCD.
Nicola, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated
learning: A model and seven principle of good feedback practice. Studies in
Higher Education, 31(2), 199-218.
Oluwatayo, G. K., Irene, W., & Appah, O. R. (2014). Assessment of university lecturers’
feedback in formative tests. Journal of Educational Review, 7(1), 119-127.
Orsmond, P., & Merry, S. (2011). Feedback alignment: Effective and ineffective links
between tutors’ and students’ understanding of coursework feedback.
Assessment & Evaluation in Higher Education, 36(2), 125-136.
Price, M., Handley, K., Millar, J., & O’Donovan, B. (2010). Feedback: All that effort, but
what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277-289.

352
Sadler, D. R. (2010). Beyond feedback: Developing student’s capability in complex

appraisal. Assessment and Evaluation in Higher Education, 35(5), 535-550.
Tan, F. D. H., Whipp, P. R., Gagné, M. & Quaquebeke, N. V. (2019). Students’ perception
of teachers’ two‑way feedback interactions that impact learning. Social
Psychology of Education, 22: 169–187. https://doi.org/10.1007/s11218-018-9473-7
Spiller, D. (2014). Assessment: Feedback to promote students learning. Teaching Development
Unit, The University of Waikato.
Sutton, R., Hornsey, M. J., & Douglas, K. M. (2011). Feedback: The communication of praise,
criticism, and advice. New York: Peter Lang Publishing.
Van Der Kleij, F., & Adie, L. (2020). Towards effective feedback: An investigation of
teachers’ and students’ perceptions of oral feedback in classroom practice.
Assessment in Education: Principles, Policy & Practice, 27(3), 252-270.
https://doi.org/10.1080/0969594X.2020.1748871
Weaver, M. R. (2006). Do students value feedback? Student perceptions of tutors’ written
responses. Assessment & Evaluation in Higher Education, 31(3), 379-394.
Appendix 1
Questionnaire to be filled by Students
Dear students,
This questionnaire is designed to gather information for the research entitled

“The Value of Feedback in Primary Schools: Students’ Perceptions of the
Practice.’’ Thus, the main objective of this questionnaire is to assess students’
perceptions and practice of feedback to improve students’ learning. So, you are
kindly requested to provide genuine information, which is very important to the
quality of the research.
Your cooperation in this regard will be highly solicited. All of your responses
will be kept confidential.
Thank you in advance for your cooperation.
Part I: - Students’ perceptions on the value of feedback

Direction፡- Please indicate how much you agree or disagree with the following
statements putting a tick mark (√) in the box
Level of agreement 5 = Strongly Agree 3 = Unable to decide

4 = Agree 2 = Disagree 1 = Strongly Disagree
No. Statement Level of Agreement

5 4 3 2 1
1 Feedback is helpful to show gaps in knowledge and
understanding.
2 Feedback rarely provides me with useful suggestions for
improvement.
3 I consider that peer feedback is essential for learning.
4 Teacher feedback is effective in promoting students’ learning.
5 Constructive criticism is needed to know how to improve my

353
work.
6 Feedback given at the end of the subject is not useful.
7 The feedback which is given should be clear to be helpful.
8 Feedback encouraged me to improve my work.
9 Positive comments increase my confidence.
10 I thought about giving up when I got negative feedback.
11 I felt demoralized or angry after reading negative feedback.
12 Feedback has helped me to reflect on what I have learned.
13 I need to receive positive feedback to progress.
14 I need to receive negative feedback to progress.
15 Teachers’ comments are vital in my learning.
Part II: Students’ perceptions of the practice of feedback
Direction፡- Please indicate how much you agree or disagree with the following
statements putting a tick mark (√) in the box
Level of agreement 5 = Strongly Agree 3 = Unable to decide

4 = Agree 2 = Disagree 1 = Strongly Disagree
No Statement Level of Agreement

5 4 3 2 1
1 On the subject that I am learning, I get plenty of feedback on
how I am doing.
2 Teachers give me immediate feedback for each assessment.
3 There is hardly any feedback on my assignments when I get
them back.
4 When I get things wrong, I don’t receive much guidance in what to do
about it.
5 I would learn more if I received more feedback.
6 Teachers’ feedback that comes too late is not useful for learning.
7 Teachers provide enough feedback on my work.
8 The feedback mainly tells me how well I am doing in relation to
others.
9 Teachers’ feedback is rarely written with reference to the
assessment criteria.
10 The feedback given helps me to understand things better.
11 The feedback given shows me how to do better next time.
12 Once I have read the feedback, I understand why I got the mark
I did.
13 I don’t understand some of the feedback given.
14 I can seldom see from the feedback given what I need to do to
improve.

354
Part III. Students use of feedback to improve their learning

Direction፡- Judge to what degree you are using feedback and give your
response by putting a tick "√" mark in one of the boxes under the
alternative scales given.
5 = Always 4 = Often 3 = Sometimes 2 = Rarely 1= Never
No Statement 5 4 3 2 1
.
1 I read the feedback carefully and try to understand what the
feedback is saying.
2 I use the feedback to go back over what I have done in the
assignment.
3 The feedback given with any subsequent assignments helps
me.
4 The feedback prompts me to go back over material covered
earlier in the course.
5 I read the comments carefully and identify the gaps in my
learning
6 I use the feedback to compare my progress with other students
7 I do not use the feedback for revising my work
8 I acted on suggestions to improve my coursework.
9 I have good intentions, but forget comments for improvement
next time.
10 I have ignored negative or critical feedback.
11 I tend to only read the marks compared to other students.
12 After reading comments, I re-work any assessment task.
13 I read comments and plan strategies for future learning.

355

Vol. 19, No. 10, pp. 355-370, October 2020
Digital Collaboration in Teaching and Learning

Activities: The Reflexivity Study on Educational
Digital Empowerment
Irwansyah*
Communication Department, Universitas Indonesia, Jakarta
https://orcid.org/0000-0002-5047-1746
Sofiatul Hardiah
Anthropology Department, Universitas Indonesia, Depok
https://orcid.org/0000-0002-8910-9170
Abstract. This research examines digital empowerment in high schools in

Indonesia. Based on the reflexivity perspective, digital empowerment is
described as the effort to respond to changes in learning systems in
digital-based schools. This research was conducted qualitatively through
observations and in-depth interviews. The participants of this study
included school principals and the teachers of information and
communication technology in three Indonesian public senior high schools
in different cities, namely Banda Aceh, Padang, and Banjarmasin. The
results highlighted the need to understand the context and projections of
the teachers' and student's ability and self-access to information and
communication technology elements, in which appropriate measures can
be formulated in response to the digitisation of the teaching-learning
process in the schools. Self-awareness and information and
communication technology competence shape digital empowerment
efforts in schools based on their respective needs and abilities. As such,
the digital-based collaboration of teaching and learning can be construed
as a form of contextual digital empowerment, which is applied at the
micro-level adapting to dynamic changes in digital competence,
especially in teachers and students.
Keywords: Information and communication technology; digital

empowerment; collaboration; education
1. Introduction
The integration of technology in the learning process encourages teachers and
students to adapt new media in the teaching-learning and communication process
*
Corresponding author: Irwansyah, email: irwansyah09@ui.ac.id

356
(Casal, 2007). Information and communication technology (ICT) are

advantageous in various ways. For instance, it provides a lifetime learning space
(Aubert, 2013), and brings about a change of role in teaching-learning activities,
allows access to internet-based learning materials and information, eliminates
space-time limitations, supports education, and opens opportunities for the
collaboration of teachers and students (Fitriyadi, 2013).
Government policies and the efforts of school management to incorporate ICT in

schools' teaching-learning process (Ekayanti & Irwansyah, 2019) are laudable. The
process improves the attitudes of teachers' and students' towards learning
because self-control and focus are practised for the proper use and application of
computer technology (Teo, 2008). Teachers' competence in interacting with ICT
devices (Weckert & Lucas, 2017) tends to be in line with the level of experience
acquired, boosting their confidence in using technology-related tools.
Likewise, self-confidence in using ICT devices arises from the teachers'

persistence in understanding how to use such devices and ICT competencies of
students (Ekayanti & Irwansyah, 2019). Moreover, ICT allows teachers to balance
the perceptions and competencies of educated ICT students (Jones, 2004), and also
helps teachers improve ICT-based teaching productivity (Sánchez & Alemán,
2011). In addition, teachers and students can exchange understanding around ICT
and competence to shape creative and effective learning outcomes (Birch & Irvine,
2009; Honan, 2008).
The tendency of students to use the internet for personal needs necessitates
integrating digital technology in order to enhance activities and behaviour more
positively (Austin, 2017). Three measures can be applied to support and produce
meaningful changes in the teaching-learning process, which include building
collaborative structures, effective technology use models, and reflection on the
practices and beliefs lived (Ertmer, 1999). Furthermore, autonomy, capability, and
creativity are required to develop the quality of ICT-based teaching and learning.
As such, it is essential to develop training for teachers to refine and hone their
competence and creativity in using ICT tools. This ability is useful for modifying
effective learning material in this digital age (Lowther, Inan, Strahl, & Ross, 2008).
The implementation of the 2013 Curriculum (Hermawati, 2020) in Indonesia,

projects the government's efforts and intention to accommodate the role of ICT to
improve the quality of education (Fitriyadi, 2013). Moreover, the curriculum can
be effectively implemented with the view that teachers and students are ICT-
competent and literate. The integration of ICT in the learning process should
consequently model teachers into facilitators, collaborators, mentors, trainers,
directors, and study partners while providing students with great choices and
responsibilities to experience learning events (UNESCO, 2007). This integration
makes students in many respects, the centre of learning, sharpening their
communication, problem-solving, critical thinking, creativity, adaptive and
reflective skills (Gray, 2003).
There is no doubt that teachers experience obstacles in integrating ICT in learning

due to internal and external factors (Ertmer, 1999, 2003). External barriers relate
to time, support, resources, and training provided by the school, while internal

357
obstacles include attitude, self-confidence, and the willingness to try. However,

these barriers can be overcome by acting on the need for ICT facilities by schools,
offering ICT training to teachers, and collaboration between teachers and students
(Samuel & Zaitun, 2007).
The obstacles encountered by teachers and students in responding to changes in

learning systems are attributed to unpreparedness in terms of ICT knowledge and
practice. These blocks or obstacles call for digital empowerment efforts by
teachers and students to create effective ICT-based teaching and learning
activities. In the context of Indonesia, digital empowerment in education has not
been extensively undertaken or enacted, especially from the reflexivity
perspective. Therefore, this research examines how reflexivity functions in digital
empowerment in Indonesian high schools.
2.1. Digital Empowerment
The concept of empowerment originated as a reaction to the power imbalance in
society (Hage & Lorensen, 2005). It is a multidimensional social learning process
that controls life using knowledge and ability (Payne, 1997; Saleebey, 2006), and
occurs in individual and community contexts. Empowerment also relates to how
people think and consider themselves, their capacities, abilities, and mastery over
what they have (Staples, 1990). The community context shows the psychological
cognition of a group of people in response to challenges and the capacity to
change their social positioning by participating in collective actions and helping
others (Boehm & Staples, 2004). Group empowerment is achieved through
collaborative effort (Hur, 2006).
Information and communication technology (ICT) is rapidly evolving in today’s

society by facilitating connections among different communities (Mäkinen, 2006).
The process of digital integration occurs within the spheres of consequential and
dynamic digital empowerment (Safira & Irwansyah, 2019). The indicators of
empowerment include technology awareness, motivation in technological
devices, technical access to digital technology, competence in using technology,
and constructive participation in creating interactive roles (Kuyoro, Awodele, &
Samuel, 2012).
2.2. Reflexivity
Reflexivity defines the character of all human behaviours with a fundamental
identity that continues to be reproduced and modified over time, along with life
changes. Giddens used the expression "reflexive monitoring" of action to describe
human behaviours that change consistently but continue to reflect essential traits
(Giddens, 1990). The concept of reflexivity emerges from individual responses
based on quick life changes (Cohen-Cole, 2005).
The concept of reflexivity underlines an individual's response to a given situation

with a particular impact on one’s life. The effect occurs in the context of self-
development and the availability of choices for the individual's future (Giddens,
1990). Reflexivity is a process of self-definition based on social and psychological
information in the dynamics of life, resulting in active and creative actions in
response to life changes (Elliott, 2014).

358
The rapid change in people's lives has two value sides. However, there is both an
opportunity as well as a danger due to overlapping situations and choices in life.
Changes cannot be avoided or appropriately addressed due to the
unpreparedness of information and knowledge (Ferguson, 1997). As such,
reflexivity is an essential component for individuals to respond to life changes and
provides an opportunity for individuals to shape themselves and their lives.
Indeed, reflexivity is having the competence to act in the world, reflected through
the formation and reshaping of actions and identities (Ferguson, 2003).
3. Research Methodology
A qualitative method is employed in this study using in-depth interview
techniques for data collection. These techniques are appropriate in studying a
group of individuals' behaviour in response to a phenomenon (Alshenqeeti, 2014;
Baker, 2006). There are two categories of informants in this study, namely school
principals and ICT teachers. Each is represented by one principal, three to six ICT
teachers, and between eight to 12 students. The information provided in this study
explains teachers, students, and school authorities' behaviour regarding
education digitalisation and empowerment. The implementation in this study
was based on three stages: (1) Gathering initial information on high schools that
implemented ICT-based learning. The search for necessary information was not
limited to the sites provided by the search engine (Levene, 2010); (2) Visiting three
high schools identified as research locations, specifically public senior high
schools in Banda Aceh (SMA Lab School), Padang (SMAN3), and Banjarmasin
(SMAN9). The schools selected from institutions that had integrated ICT since the
enactment of the 2013 Curriculum. ICT lessons at the junior and senior secondary
levels were replaced as extracurricular activities in the schools; (3) Conducting in-
depth interviews with the informants to obtain information on several research
questions such as the availability of technology-based facilities, technology-based
learning activities, and digital empowerment efforts in the teaching-learning
process.
Reflexivity provides a perspective on digital empowerment having a variety of

data patterned on the following themes (Giddens, 1990): (1) ICT Technical access,
(2) ICT awareness of teachers and students, (3) ICT motivation of teachers and
students, (4) ICT initiative of teachers and students, (5) ICT competence of
teachers and students, (6) ICT barrier of teachers and students, and (7) Digital
empowerment efforts of teachers and students
The coding process of the primary data was carried out in three stages: open, axial,
and selective coding (Miles & Huberman, 1994). Selective coding shows informant
behaviour repetition, which was grouped as a theme in answering the research
questions (Irwansyah & Triputra, 2016). The data analysis was conducted using a
thematic approach to identify, analyse, and determine a specific social
phenomenon theme (Braun & Clarke, 2006). The data and results were validated
by triangulating qualitative and quantitative data communication between
researchers and supplementing secondary data accessed through online literature
in order to obtain a holistic explanation (Yeasmin & Rahman, 2012).

359
4. Research Findings
Digital technology involvement in the three schools occurred in several forms,
including the availability of digital access, extracurricular facilities, and ICT-based
competitions. The integration of ICT can be seen in teachers' and students' ICT
competencies in the teaching-learning process. Interestingly, teacher-student ICT
competencies were at different levels, based on individual ICT access, motivation,
and initiative. This gap in ICT competencies was addressed by seeking digital
empowerment facilitated by the school authorities and teachers and students
directly in their daily interactions.
4.1. Digital Access at School

Digital access in the context of a school demonstrates its technological awareness
and motivation related to the teaching-learning process, achieved through the
availability of ICT facilities, extracurricular and ICT-based competitions and the
implementation of teaching and learning activities in digital-based classes. Table
1 summarises the information on digital access in the three schools of this study.
Table 1 shows the similarity in the provision of ICT-based facilities, specifically

computer laboratories, and limited internet Wi-Fi access in specific locations, such
as public senior high school (Sekolah Menengah Atas – SMA) Computer laboratories
and Wi-Fi access were used in teaching and learning activities within set times
and rules. The schools implemented technological awareness and motivation
through digital-based activities.
Table 1: Information Technology and Facilities Based Activities in Several Public Senior High
Schools in Indonesia
Facilities SMA Lab School in SMAN9 in
SMAN3 in Padang
Banda Aceh Banjarmasin
Liquid Crystal Display Three computer Two computer
(LCD) in each class; laboratories with 35 laboratories with
Wi-Fi; Computer computer units, a total of 65
laboratory. direct internet computer units;
access on each wifi.id access.
School Facilities
personal computer
(PC), headset, and
study material via
digital video disc
(DVD).
Computer Extracurricular Extracurricular
extracurricular material, namely material, namely
(compulsory for class design, making email
of 10); material on programming, and and e-learning.
how to access the cinematography.
ICT-Based internet, determine
Extracurricular appropriate and safe
social media,
operating video
editing applications,
animation, screening,
and web design.

360
Facilities SMA Lab School in SMAN9 in

SMAN3 in Padang
Types of competitions: Types of Types of
cinematography/ competitions, competition,
short films, and including graphic specifically,
ICT-Based Contest
graphic design/ design and poster design.
poster design. computer
Olympiad.
Use email and Google Unknown Some subject
Forms for teachers,
assignments; E-report including
card. English,
Sociology,
Mathematics, use
Google Translate,
Google, e-books,
and computer
ICT-Based Subject laboratories for
teaching-learning
activities.
Schools only
conduct
computer-based
examinations in
computer
laboratories.
Unknown. Personal activities A teacher
are based on digital manages the
technology school's
students, such as YouTube,
becoming a Instagram,
Other Vlogger/YouTuber, Facebook, and
Blogger, and online Website accounts
seller. and applies
copyright in
producing
content.
Source: Irwansyah, 2020
These facilities provide opportunities and a place for students to learn about
technology in education and develop interests and talents in ICT. The availability
of technology-based facilities provides teachers access to explore their ability to
design and deliver digital-based subject matter. For instance, senior public high
school in Banda Aceh and Banjarmasin, teachers, and students are encouraged to
use email and Google Forms in the teaching-learning process, such as sending
school assignments. The school provides digital access through facilities and
convenes extracurricular and digital-based competitions for students. The
Ministry of Education and Culture of the Republic of Indonesia established the
competition as an alternative to ICT learning in the 2013 Curriculum (K-13). In
carrying out K-13, students communicate the knowledge acquired in the learning

361
process to others, both orally and in written form. This is one approach of utilising
network access to the internet to deepen their understanding (Ahmad, 2014).
Extracurricular materials are provided through Microsoft Office applications,

accessing the internet, via social media, operating video editing applications,
animation, screening, and web design. For this reason, supportive steps are taken
to provide basic ICT knowledge and skills. The school supports students talented
in specific ICT fields to participate in competitions, such as cinematography and
graphic design. The application of K-13 changes the role of teachers from
instructors to facilitators, in guiding ICT-based activities.
4.2. Teachers ICT Competence

Teachers and students have technological competencies having varying degrees
of proficiency which is useful in developing technological devices through related
experience and individual awareness. Furthermore, it aids motivation and related
activities based on digital technology.
The findings from the study show that teachers and students used digital
technology in their education ecosystem, though not all had acquired the
motivation and competence needed to operate technological devices. In SMA
Padang, 85% of teachers were shown to be proficient in operating Microsoft Word.
In SMA Banda Aceh, young civil servant teachers were shown to have good ICT
competency since they selected educators to understand basic ICT knowledge.
Elderly teachers, however, were not adept at using technological devices. Age
barriers and physical inabilities account for this inexperience in the field of
technology.
Subject teachers in SMA Banda Aceh and Padang having ICT competencies use
digital-based classes, especially in the use of computers. Technology is utilised to
provide a varied and enjoyable learning experience and atmosphere for students.
Additionally, they use email technology or Google Drive to send assignments to
students. Similarly, the teachers at SMA Banda Aceh use email and Google Form
to minimise distraction in the learning process and provide convenience and
variety in an atmosphere of learning for students. The school supports digital
motivation by providing e-report-based facilities in making grading much easier.
In SMA Banjarmasin, out of 26 teachers, 16 were proficient in technology tools

and applications. However, ten were not proficient given their older age. In this
school, ICT-competent teachers help incompetent colleagues, given it is not a
requirement for teachers to be proficient in using technological devices. However,
they strive to distribute the technology capabilities of teachers equally.
About 10% of subject teachers with ICT competencies permit students to access
technology devices. The 2013 Curriculum requires students to conduct a critical
analysis of a case in a subject. Here, teachers digitising teaching and learning
activities help students understand the learning material. For instance, teachers
use Google Translate and Google search to translate text conversations or practice
listening displayed on an overhead projector or LCD.

362
4.3. Students ICT Competence

Student competencies in using technological devices are developed and shaped
through their interactive learning experience using the technology each day; for
example, some students own and use devices, such as smartphones and laptops.
On the other hand, students with limited access to digital technology have low
ICT competencies. Consequently, they face obstacles in understanding ICT
material and digital empowerment that is provided and encouraged in schools.
In the three schools, students with ICT competence initially acquired ICT
knowledge while in junior high school. However, their level of proficiency varied
depending on the frequency of interaction with smartphone devices and PCs.
Students with ICT competence typically use technological devices to achieve their
daily needs in learning, entertainment, and talent development.
Notwithstanding, students access Google to find answers to questions and other

subject matter during their break times. For informal learning, they tend to use e-
books accessed via PCs situated in school laboratories or smartphones. Moreover,
they actively search for information on the internet to build social networks with
students from other schools through social media. They also share information
with teachers through digital communication applications in seeking directions
or instructions in their learning.
Students at SMA Banda Aceh and Padang having ICT competencies utilise their
skills in achieving personal development and creative needs, such as graphic
design, cinematography, and uploading to YouTube, selling online, and for
blogging. These students use digital communication applications, such as Line
and Instagram, to build communication networks and working relationships with
school alumni currently studying at universities.
However, there is a gap in ICT competencies between students in SMA

Banjarmasin. About 25% of students do not own or have access to technological
devices, given economic constraints. Teachers conduct assessment among
students on different technological capabilities beginning from their first day at
school. Although some students understand ICT material relatively easily, others
need several in-depth explanatory sessions. The explanatory sessions are
necessary since they cannot practice at home using a laptop or smartphone device.
Students having no or little access to technology devices often have slow learning
rhythms when using ICT.
There is also the transformation of student learning behaviour due to the rapid
development and emergence of advanced technologies, shaping them into digital
generations or being digital naïve. This generation is dominated by digital
devices, especially to access information. For this reason, it tends to have more
significant learning opportunities than previous generations (Prensky, 2001). The
development of digital technology among students and teachers can be further
facilitated through digital access, such as through ICT knowledge, facilities, and
opportunities to practice ICT in daily activities. Students and teachers with
limitations in accessing these facilities face significant obstacles in ICT-based
teaching and learning activities. Therefore, there is a need for increased digital
accessibility for students and teachers to address their digital needs.

363
4.4. Digital Empowerment Efforts in Schools

School authorities, teachers, and students at SMA Banda Aceh, Padang, and
Banjarmasin schools respond differently to digital accessibility and ICT
competence. They put in significant effort towards micro-digital empowerment
and implementation adapted to each school's context and digital needs. The
following table presents a matrix of the types of digital empowerment in the three
schools:
Table 2: Types of Digital Empowerment in School

Types of Digital
SMA Lab School in SMAN
Empowerment SMAN Padang
in School
Digital Form: Face-to-face Form: Digital Form: Training or
Competency training in groups socialisation workshops every
Training for with school computer packaged in seminars year with material
Teachers and teachers as on digital about technology.
Students presenters, and empowerment with Online-based
discussion forum presenters from teacher
with presenters from universities, and assignments and
outside the school. computer training strategies (e-report)
with ICT teachers as facilitated by two
facilitators. computer teachers
with computer
training in a pilot.
Target: Teachers
Targets: Teachers and Target: Teachers
Students
Material: How to
Material: Computer Material: How to use Microsoft
operations and make learning media Word, Microsoft
applications elaborated following Excel, and email.
(Microsoft Word and the character and However,
Microsoft teaching techniques workshops with
PowerPoint), and e- of each teacher; make similar material did
learning. slides in Microsoft not continue
PowerPoint and email anymore because
for some digital-based there were no
subjects chosen by the teacher enthusiasts.
trainee teacher.
Results: The training

of the teachers had Result: Unknown.
increased skills in
Result: The teachers operating technology
already have their sets and their
laptops and can application to make
operate the features learning media and
on the laptop presenting material in
independently to technology-based
create digital learning classes.
media.

364
Types of Digital
SMA Lab School in SMAN
Empowerment SMAN Padang
in School
Transfer of Form: Helps teachers Form: Helps teachers Form: Delivering
Digital use technology with difficulty to information in the
Knowledge ICT devices in the rename a document form of lessons or
and Practice classroom, such as and make competitions from
between laptops and presentations in the internet to the
Teachers and projectors. Microsoft teacher.
Students PowerPoint.
Digital Literacy Form: The school has Form: Appealing Form: Provision of
for Students not deepened digital about digital ethics to information on
literacy, such as students in the class, hoaxes to students
enacting the such as how to deal in the classroom by
information and with hoaxes. Provide a school computer
transaction electronic information about teacher, urged
(ITE) Law in schools. password security of students not to
a student's digital create creative
account. content related to
the government;
information about
copyright in
making creative
content;
Information about
scamming in
activities in the
digital world.
Digital Activity Form: The school Form: The teacher Form: SMA
Assistance for urges parents to pay encourages students Banjarmasin guides
Students attention to the digital to carry out digital the teachers to
activities of children activities positively; monitor learning
at home because the the teacher builds and student
school is not a place students' thinking so activities through
to care for and that they can control the school's
improve children's themselves and take academic website.
behaviour to be better responsibility for Students working
individuals; the themselves in their in the creative field
school conducts a activities in the digital often solicit
sudden raid on world. computer
students' electronic development
devices in class. teachers' opinions
before publishing
their work on
YouTube.
Source: Irwansyah, 2020
Table 2 above presents the types of digital empowerment conducted by the school
authority, teachers, and students in respective schools. Teachers help organise
digital empowerment through competency training for fellow teachers and
students and digital literacy and assistance for students. Empowerment is formed

365
and implemented based on digital access, the schools' financial capacity, the
absorption of digital knowledge, and teachers' and students' needs. The digital
socio-cultural context in each school shapes teachers, students, and the
perspective of authorities' and behaviour towards digital technology and
empowerment. The forms and materials of digital empowerment in each school
are adjusted following the teaching and learning techniques. This adjustment can
be created via a two-way communication process on the competencies, initiatives,
and digital barriers in the teaching-learning process. The communication process
helps to shape teachers' and students' self-awareness with varied digital access
and competencies, though they are in the same learning ecosystem. Self-
awareness drives the teachers and students to share knowledge and practices
integrated into various subjects' through digital collaboration.
4.5. Discussion: Reflexivity Study on Digital Collaboration between

Students and Teachers in Teaching and Learning Activities
The integration of digital technology in the education system requires teachers
and students to facilitate digital-based learning following the introduction of the
2013 Education Curriculum. Practically, this enactment is not similar in all schools
in the context of Indonesia since it varies based on the social-cultural context,
needs, and digital accessibility. For this reason, two situations arise from the
efforts to integrate ICT in learning in schools both in terms of the inequality and
the adaptation of ICT.
There is no doubt that teachers and students are involved in a digital learning
ecosystem (Sarnok, Wannapiroon, & Nilsook, 2019) and experience gaps in
adapting ICT, and carrying out direct and two-way digital collaboration actions
(Lopes, Oliveira, & Costa, 2015). Moreover, they assist each other in the use of ICT
tools and share knowledge on digital literacy. This collaboration arises from their
reflection on ICT access and competencies (Kopaiboon, Reungtrakul, &
Wongwanich, 2014).
According to reflexivity, humans make observations on primary individual

behaviour in order to adjust and establish their fast-changing life systems; these
changes in human behaviour occur consistently. Humans respond to situations in
order to gain the development of self-abilities that are both useful now and in the
future (Giddens, 2006). People with reflexivity ability exhibit dynamic and
creative behaviour in order to meet their needs and societal changes (Elliott, 2014).
Students at SMA Banda Aceh, Padang, and Banjarmasin reflected on the gaps
faced in ICT competency during the learning process (Kopaiboon et al., 2014). The
ICT competency disparity between teachers and students; however, hindered the
process of knowledge delivery in learning sessions (Samuel & Zaitun, 2007). They
respond to this challenge by sharing ICT knowledge on how to operate related
devices and applications. For instance, when teachers had difficulty connecting a
laptop to the projector, in order to create slides using Microsoft PowerPoint, or
displaying slides on the projector screen, they were assisted by students with ICT
knowledge and competencies.
The initiative of students to share knowledge and help operate ICT tools enabled
the teachers to achieve good performance in applying such devices during

366
learning. Mentoring helps teachers adapt their abilities to the needs of teaching
(Cherian, 2007). Superior competence in a particular field and implementation is
normally through direct collaboration between the teacher and the mentor. In the
context of digital collaboration, students act as mentors to guide teachers in the
transition to ICT-based teaching in the classroom. Therefore, self-evaluation forms
reflective teaching.
Self-evaluation (Dewey, 1933; McNamara, O'Hara, Boyle, & Sullivan, 2009); and
mentoring (Cherian, 2007) are methods used by teachers, and students to engage
in responsive teaching and learning. Self-evaluation requires teachers and
students to assess such practices. However, it needs to be persistent in teaching-
learning experiences to produce new perspectives and teaching-learning
practices, such as integrating digital technology in classroom learning
(McNamara et al., 2009).
Some ICT teachers in SMA Padang and Banjarmasin having useful ICT
competencies, practice reflexivity by conducting digital literacy to students.
Digital literacy was initially developed in the 1990s when information became
more easily compiled, accessed, and disseminated through information
technology networks (Bawden, 2001). An ICT teacher at SMA Banjarmasin,
initialised as informant RD, discovered that about 60% of students lacked the
understanding of data security and digital privacy. Therefore, the digital literacy
material was shared among students on the avoidance of hoaxes, data privacy on
the internet, data security on digital student accounts, content copyright, and
scamming. According to informant RD, in creating email passwords, the use of
birthdays, home addresses, or parents' names should be avoided in order to
minimise password hacking. Students accessing online content such as songs,
videos, and images on YouTube were also asked to pay attention to copyright and
were advised not to trust strangers on the internet without having a transparent
and verifiable background.
Digital collaboration is a form of digital empowerment that is due to teacher and

student reflexivity towards ICT competencies and the need to conduct teaching
and learn more responsively (Lopes et al., 2015). Moreover, it helps teachers to
understand student behaviour and the different ways of learning to accommodate
their responsibilities concerning digital culture and student pedagogy (Ramanan
& Mohamad, 2020). It also models students to be individuals having sensitivity to
the differences surrounding them, aside from empowering them to become active
and creative in fast-evolving digital technology (Bruno & Canina, 2019). In many
respects, students are directly involved as digital empowerment agents with
meaningful learning experiences (Giddens, 2006).
Accordingly, schools should be transformed into learning communities that bring

together learning activities, direction, and cooperation between teachers and
students (Bransford, Brown, Cocking, Donovan, & Pellegrino, 2000). From the
findings of this study, it was shown that teachers and students upheld digital
collaboration as community-centred digital empowerment. As such, the
surrounding ecosystem provides space that allows practising digital skills
interactively and cooperatively.

367
5. Conclusion
This study examined digital collaboration between teachers and students
concerning digital empowerment. The results show that self-reflection in teachers
and students helps to shape ICT awareness and supporting initiatives. In other
words, the critical sustainability of digital empowerment through digital
collaboration in the teaching-learning context. However, in the context of
Indonesia, future studies should develop an analysis unit to gain a much broader
understanding of digital empowerment.
This study has both academic and practical contributions. Academically, it

motivates further studies on digital empowerment in education from elementary
to secondary schools. Advanced qualitative research may be used to focus on
measuring instruments, unit of analysis, the geographical and socio-cultural
context of informants, and analytical perspectives.
Practically, this research may help to motivate the government to initiate and
implement digital empowerment policies for the education system in Indonesia.
This study has explained about digital empowerment that focuses not only on the
provision of sophisticated digital access but also on how the schools authority,
teachers, and students receive and integrate digital technology in the learning
ecosystem. The implementation of these policies can be based according to the
community's socio-cultural context in various regions in Indonesia.
6. Acknowledgement
We are grateful for the support of research funding from PDUPT DIKTI No. NKB-
1519/UN2.R3.1/HKP.05.00/2019; NKB-132/UN2.RST/HKP.05.00/2020; NKB-
2731/UN2.RST/HKP.05.00/2020
7. References
Ahmad, S. (2014). Problematika kurikulum 2013 dan kepemimpinan instruksional kepala
sekolah. Jurnal Pencerahan, 8(2), 98–108. Retrieved from
http://www.jurnal.unsyiah.ac.id/JPP/article/download/2158/2111
Alshenqeeti, H. (2014). Interviewing as a data collection method: A critical review. English
Linguistics Research, 3(1), 39–45. https://doi.org/10.5430/elr.v3n1p39
Aubert, N. B. (2013). Documentation of ICT usage for Maya development: A case study from
Aguacatán, Guatemala. Noragric: Norwegian University of Life Science.
Austin, C. G. (2017). Personal connections in the digital age, second edition. In
Consumption Markets & Culture (Vol. 20).
https://doi.org/10.1080/10253866.2015.1135538
Baker, L. M. (2006). Observation: A complex research method. Library Trends, 55(1), 171–
189. https://doi.org/10.1353/lib.2006.0045
Bawden, D. (2001). Information and digital literacies: A review of concepts. Journal of
Documentation, 57(2), 218–259.
Birch, A., & Irvine, V. (2009). Preservice teachers' acceptance of ICT integration in the
classroom: Applying the UTAUT model. Educational Media International, 46(4), 37–
41. https://doi.org/10.1080/09523980903387506
Boehm, A., & Staples, L. H. (2004). Empowerment: The point of view of consumers.
Families in Society: The Journal of Contemporary Social Services, 85(2), 270–280.
https://doi.org/https://doi.org/10.1606/1044-3894.314
Bransford, J. D., Brown, A. L., Cocking, R. R., Donovan, M. S., & Pellegrino, J. W. (2000).

368
How experts differ from novices. In J. D. Bransford, A. L. Brown, R. R. Cocking,

M. S. Donovan, & J. W. Pellegrino (Eds.), How People Learn: Brain, Mind, Experience,
and School (pp. 31–50). https://doi.org/10.17226/9853
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research
in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Bruno, C., & Canina, M. (2019). Creativity 4.0. Empowering creative process for digitally
enhanced people. Design Journal, 22(sup1), 2119–2131.
https://doi.org/10.1080/14606925.2019.1594935
Casal, C. R. (2007). ICT for education and development. In Info (Vol. 9).
https://doi.org/10.1108/14636690710762093
Cherian, F. (2007). Learning to teach: Teacher candidates reflect on the relational,
conceptual, and contextual influences of responsive mentorship. Canadian Journal
of Education, 30(1), 25–46. https://doi.org/10.2307/20466624
Cohen-Cole, J. (2005). The reflexivity of cognitive science: The scientist as model of human
nature. History of the Human Sciences, 18(4), 107–139.
https://doi.org/10.1177/0952695105058473
Dewey, J. (1933). How We Think: A Restatement of the Relation of Reflective Thinking to the
Educative Process. Massachusetts: D.C. Heath and Company.
Ekayanti, S. R., & Irwansyah. (2019). UTAUT in communication technology of learning
management system. 2018 International Conference on Advanced Computer Science
and Information Systems, ICACSIS 2018, 253–258.
https://doi.org/10.1109/ICACSIS.2018.8618172
Elliott, A. (2014). Concepts of The Self (3rd ed.). Malden, MA: Malden, MA: Polity Press.
Ertmer, P. A. (1999). Addressing first-and second-order barriers to change: Strategies for
technology integration. Educational Technology Research and Development, 47(4), 47–
61. https://doi.org/10.1007/BF02299597
Ertmer, P. A. (2003). Transforming teacher education: Visions and strategies. Educational
Technology Research and Development, 51(1), 124–128.
https://doi.org/10.1007/BF02504522
Ferguson, H. (1997). Protecting children in new times: Child protection and the risk
society. Child and Family Social Work, 2(4), 221–234.
Ferguson, H. (2003). Welfare, social exclusion, and reflexivity: The case of child and
woman protection. Journal of Social Policy, 32(2), 199–216.
https://doi.org/http://dx.doi.org/10.1017/S0047279402006967
Fitriyadi, H. (2013). Integrasi teknologi informasi komunikasi dalam pendidikan: Potensi
manfaat, masyarakat berbasis pengetahuan, pendidikan nilai, strategi
implementasi, dan pengembangan profesional. Jurnal Pendidikan Teknologi Dan
Kejuruan, 21(3), 269–284.
https://doi.org/https://doi.org/10.21831/jptk.v21i3.3255
Giddens, A. (1990). The Consequences of Modernity. Cambridge: Polity Press.
Giddens, A. (2006). Sociology. Cambridge: Polity Press.
Gray, T. (2003). Rich ICT learning experiences: What do they look. Enabling E-Learning, 1–
8. Retrieved from http://elearning.tki.org.nz/Media/Files/Rich-ICT-learning-
experiences-What-do-they-look-like2
Hage, A. M., & Lorensen, M. (2005). A philosophical analysis of the concept
empowerment; the fundament of an education-programme to the frail elderly.
Nursing Philosophy, 6(4), 235–246. https://doi.org/10.1111/j.1466-
769X.2005.00231.x
Hermawati. (2020). Teachers' efforts in understanding the factual, conceptual, procedural,
and metacognitive assessment using the revised 2013 curriculum. International

369
Journal of Learning, Teaching and Educational Research, 19(5), 186–199.

Honan, E. (2008). Barriers to Teachers Using Digital Texts in Literacy Classrooms. Literacy,
4(21), 36–43. https://doi.org/10.1111/j.1467-9345.2008.00480.x
Hur, M. H. (2006). Empowerment in terms of theoretical perspectives: Exploring a
typology of the process and components across disciplines. Journal of Community
Psychology, 34(5), 523–540. https://doi.org/10.1002/jcop
Irwansyah, I., & Triputra, P. (2016). Indonesia gastronomy brand: Netnography on virtual
culinary community. The Social Sciences, 11(19), 4585–4588.
https://doi.org/10.3923/sscience.2016.4585.4588
Jones, A. (2004). A review of the research literature on barriers to the uptake of ICT by
teachers. British Educational Communications and Technology Agency (Becta), 1(June),
1–29. Retrieved from
http://dera.ioe.ac.uk/1603/1/becta_2004_barrierstouptake_litrev.pdf
Kopaiboon, W., Reungtrakul, A., & Wongwanich, S. (2014). Developing the quality of ICT
competency instruments for lower secondary school students. Procedia - Social and
Behavioral Sciences, 116(October), 1802–1809.
Kuyoro, A., Awodele, O., & Samuel, O. (2012). ICT: An effective tool in human
development. International Journal of Humanities and Social Science, 2(7), 157–162.
Retrieved from
http://www.ijhssnet.com/journals/Vol_2_No_7_April_2012/17.pdf
Levene, M. (2010). An introduction to search engines and web navigation. Hoboken, New
Jersey: John Wiley & Sons, Inc.
Lopes, I., Oliveira, A., & Costa, C. J. (2015). Tools for online collaboration: Do they
contribute to improve teamwork? Mediterranean Journal of Social Sciences, 6(6 S4),
511–517. https://doi.org/10.5901/mjss.2015.v6n6s4p511
Lowther, D. L., Inan, F. A., Strahl, J. D., & Ross, S. M. (2008). Does technology integration
"work" when key barriers are removed? Educational Media International, 45(3), 195–
213. https://doi.org/10.1080/09523980802284317
Mäkinen, M. (2006). Digital empowerment as a process for enhancing citizens'
participation. E-Learning and Digital Media, 3(3), 381–395.
https://doi.org/10.2304/elea.2006.3.3.381
McNamara, G., O'Hara, J., Boyle, R., & Sullivan, C. (2009). Developing a culture of
evaluation in the Irish public sector: The case of education. Evaluation, 15(1), 101–
112. https://doi.org/10.1177/1356389008095492
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook
(2nd ed.). London, UK: SAGE Publications, Inc.
Payne, M. (1997). Modern social work theory. London: MacMillan Press.
Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9(5), 1–6.
Ramanan, B., & Mohamad, M. Bin. (2020). Validating a model of change readiness among
Malaysian school teachers: A structural equation modeling approach. International
Journal of Learning, Teaching and Educational Research, 19(2), 79–93.
Safira, M. R., & Irwansyah, I. (2019). The social humanism factor in digital empowerment
in Indonesia. 2019 International Conference on Advanced Computer Science and
Information Systems (ICACSIS), 409–416.
https://doi.org/10.1109/ICACSIS47736.2019.8979911
Saleebey, D. (2006). Strengths perspective in social work practice. Boston: Allyn & Bacon.
Samuel, R. J., & Zaitun, A. B. (2007). Do teachers have adequate ICT resources and the

370
right ICT skills in integrating ICT tools in the teaching and learning of the English
language in Malaysian schools? The Electronic Journal of Information Systems in
Developing Countries, 29(1), 1–15. https://doi.org/10.1002/j.1681-
4835.2007.tb00196.x
Sánchez, J. J. C., & Alemán, E. C. (2011). Teachers' opinion survey on the use of ICT tools
to support attendance-based teaching. Computers & Education, 56(3), 911–915.
Sarnok, K., Wannapiroon, P., & Nilsook, P. (2019). Digital learning ecosystem by using
digital storytelling for teacher profession students. International Journal of
Information and Education Technology, 9(1), 21–26.
https://doi.org/10.18178/ijiet.2019.9.1.1167
Staples, L. H. (1990). Powerful ideas about empowerment. Administration in Social Work,
14(2), 29–42. https://doi.org/10.1300/J147v14n02
Teo, T. (2008). Pre-service teachers' attitudes towards computer use: A Singapore survey.
Australasian Journal of Educational Technology, 24(4), 413–424.
https://doi.org/https://doi.org/10.14742/ajet.1201
UNESCO. (2007). The UNESCO ICT in education programme. Bangkok: UNESCO, Thailand.
Weckert, J., & Lucas, R. (2017). Professionalism in the information and communication
technology industry. In Professionalism in the Information and Communication
Technology Industry. https://doi.org/10.26530/oapen_459997
Yeasmin, S., & Rahman, K. F. (2012). "Triangulation" research method as the tool of social
science research. Bup Journal, 1(1), 154–163. Retrieved from
http://www.bup.edu.bd/journal/154-163.pdf

371

Vol. 19, No. 10, pp. 371-387, October 2020
“This is why students feel lost when they go into

teaching practice”: English Language Teachers’
Views on their Initial Teacher Education
Sue Garton
Aston University, Birmingham, UK
https://orcid.org/0000-0002-7421-0858
Abstract. The last 20-25 years have seen a significant shift in the views
about what teachers need to know to be able to teach. This shift has led to
new developments in the theory of second language teacher education
(SLTE) and a growth in research in this area. One area of research concerns
the attitudes and expectations of those learning to become teachers. While
most studies in this area focus on teacher education programmes in BANA
countries, this article looks at data from student teachers studying in Russia
and Uzbekistan. The study employed a quantitative and qualitative
research design, using a researcher-designed on-line questionnaire.
Through snowball sampling, data from 161 students and recent graduates
in the two countries were collected, analysed, and compared to investigate
the content of SLTE programmes. The study identified what the novice
teachers felt were the strengths and weaknesses of their programme, and
what changes they would like to see. Results showed that while the
respondents were mainly satisfied with their methodology, and theoretical
linguistics courses, they felt the need for more practice, both teaching and
language practice. The data also revealed that, in Uzbekistan in particular,
the idea of global English struggles to take hold as native-speaker models
remain the norm. The implications of the study underline the need for
SLTE to explicitly link theory to practice and to promote the idea of
varieties of English, rather than focus on native-speaker norms.
Keywords: language teacher education; Russia; Uzbekistan
1. Introduction
The rapid spread of English as a global language in the last 25-30 years has been
accompanied by the reform of school language curricula away from grammar-
focused teaching towards the development of communicative competence.
Numerous studies have focused on the issues that such a shift has raised, and
especially the challenges for language teachers (see, for example, Copland, Garton
& Burns, 2014). Graves and Garton (2017) note that there is often a gap between
the principles of communicative curriculum reform and actual classroom practice,
and identify both ideological and practical reasons for the gap. These reasons

372
include socio-cultural appropriacy, clashes with teachers’ beliefs and experience,

large classes, lack of appropriate materials, and the impact of national
examinations. However, perhaps the biggest obstacle to the successful
implementation of curriculum reform is the lack of teacher training (Rahman &
Pandian, 2018) or provision of training that does not facilitate a real
understanding of new methods of teaching (Hardman & A-Rahman, 2014).
At the same time, changes have taken place in second language teacher education
(SLTE) with a shift in views around three fundamental questions (Johnson, 2016):
what do teachers need to know? What do teachers need to be able to do? How
best can they learn? In particular, there has been a move away from the idea of
teacher education as a set of prescribed behaviours to be followed to teacher
learning as social practice (Freeman, 2020; Johnson, 2016)
These two interlinked phenomena have led to a rich and diverse field of research
in SLTE with studies on, for example, teacher identity, reflection, and observation
and feedback, to name just a few (see Walsh & Mann, 2019, for a comprehensive
overview of research into English language teacher education). Another area is
that of the attitudes and expectations of novice teachersi concerning their training
programme, and their perceptions of its effectiveness.
As Johnson (2016) notes, the spread of English and the increased demand for
English teachers has led to ever greater numbers seeking teacher education in so-
called BANA countries (Britain, Australasia, and North America) (Holliday 1994).
However, this mobility has given rise to concerns about the extent to which
teacher education programmes prepare novice teachers for the contexts in which
they teach (see, for example, Barnawi & Le Ha, 2015). As Johnson (2016, p.130)
asserts,
“understanding the extent to which the language learning settings in
which language teacher education takes place adequately prepares
language teachers for any and all instructional contexts is an under-
researched yet highly relevant emerging area of debate.”
Johnson appears to be mainly concerned with how SLTE in BANA countries

prepares teachers to teach when they return to their local contexts, and to date,
the majority of studies on the attitudes of novice teachers concerns those studying
in BANA countries (see, for example, Copland et al., 2017; Faez & Valeo, 2012).
Far fewer studies have examined the attitudes and expectations of novice teachers
who have studied locally. Moreover, the few existing studies are concentrated on
a limited number of countries such as Turkey (see, for example, Akcan, 2015) and
Spain (Martínez Agudo, 2017) Yet, in the context of widespread curriculum
reform and the ever-increasing demand for English teachers in public state school
systems, it would seem important to research whether SLTE programmes in a
variety of contexts outside BANA countries prepare language teachers
adequately.
This article, therefore, focuses on teachers from two countries of the ex-Soviet
Union whose SLTE programmes have not so far been widely studied: Russian and

373
Uzbekistan. Part of a larger project aimed at strengthening language teacher

training programmes in these countriesii, the article presents data from 161
students and graduates to investigate the content of their training programmes.
The research questions guiding the study were:
1. What do novice teachers perceive as the strengths of their SLTE programme?
2. What do novice teachers perceive as the weaknesses of their SLTE programme?
3. How can SLTE programmes better prepare teachers for the language
classroom?
2. Literature review
Formal programmes in higher education institutions are still the main form of
language teacher education (SLTE) around the world and maintain an important
“professional gate-keeping function” (Wright, 2010, p.262), determining who will
teach, particularly in state education systems.
Richards (2008) identifies two issues that he maintains shape SLTE. The first is the
development of the knowledge base of teaching and our understanding of what
teachers need to know. Freeman (2020) identifies two problems associated with
this knowledge base: one is how theory becomes practice and the second is who
defines what is valued as knowledge, what Freeman calls ‘positionality’. For the
most part, practices in BANA countries have defined SLTE (Freeman, 2020).
The second issue identified by Richards (2008) is the external pressures brought
about by globalisation and the spread of English as the international language.
This spread has led ministries to formulate new national policies for language
curricula and language teacher education. As the role of English changes in the
world, so too does what the knowledge base needs to address, particularly
concerning what is taught and who is teaching (Freeman, 2020).
Traditionally, SLTE has been grounded in linguistics and applied linguistics with
a clear separation between the theoretical knowledge gained in classes in the
institution and the application of that knowledge during a practicum (Wright,
2010). This was a one-size-fits-all model, based on the assumption that general
theories could be translated into practice whatever the context (Johnson 2016).
It was in the 1990s that the view of language teacher education began to change
(see Wright, 2010, for a detailed overview). The publication of Freeman and
Johnson’s (1998) reconceptualisation of SLTE marked a paradigm shift from a
behaviourist, knowledge transmission view of teacher education to a
constructivist approach, which considers the nature of teacher learning. SLTE was
viewed as “a dialogic process of co-constructing knowledge that is situated in and
merges out of participation in particular socio-cultural practices and contexts.”
(Johnson, 2016, p.122)
Despite the shift in the nature of SLTE and the academic debates around it, there
remain doubts about the impact that these developments have had on actual
teacher education programmes and whether teachers are equipped for the
realities of the classroom (Farrell, 2015).

374
Hennissen, Beckers and Moerkerke (2017) cite several studies from different
disciplines showing the difficulties that pre-service teachers have in making the
connections between the theory they have learned on their courses, and the
demands of the practicum, including difficulties in linking their own beliefs and
ideas about teaching to the practicum. Whilst not specifically about SLTE,
Hennissen et al.’s (2017) study implies that teacher education programmes are still
not taking a constructivist approach. They conclude:
“Pre-service teachers apply theory, as offered in curriculum courses at the
teacher education institute, only to a limited degree in educational
practice. Theory is part of teacher education, but it is not embedded in
teaching practice and not anchored in the actions of pre-service
teachers.[…] pre-service teachers receive information they cannot
transfer, because they lack relevant experiences. This is a feed-forward
problem.” (Hennissen et al., 2017, p.314)
Within SLTE many programmes include what are considered to be practical

courses, such as methodology, but they are often ‘knowledge about’
methodology, rather than ’knowledge how’ to teach (Richards, 2008). Such
theoretical approaches can leave teachers unsure as to how to bridge the gap from
the course to the classroom. This was the case, for example, with the Malaysian
primary school teachers in Hardman and A-Rahman’s (2014) study. The teachers
were aware of suitable activities for children, such as songs and games, but they
did not know how to use such activities in a language classroom. Even more
recently, studies such as Yin (2019) are still finding limitations in how SLTE
prepares teachers for the reality of the classroom. Yin researched pre-service
teachers in Korea as they tried to apply what they had learned in their theoretical
courses to their practice during the practicum. Her results showed that, while the
trainees felt that they were prepared in the classroom skills that they needed, they
were not ready to deal with the realities of the classroom. A similar result was
noted in Vietnam by Canh (2014) who found that, during the practicum, student
teachers tended to imitate their cooperating teacher rather than trying to apply or
adapt what they had learned on their programme. Canh (2014) concluded that
student teachers were not prepared for the transition from their programme to the
realities of the classroom.
These studies were concerned mainly with the practicum and novice teachers’
experiences and practices. A related area of research focuses on novice teachers’
views about their formal preparation and in particular what they see as the
strengths and weaknesses of their programmes and how to improve their learning
experiences.
A number of studies in this area have been carried out in BANA countries.
Copland et al.’s (2017) study, for example, found that the most popular courses
for TESOL master’s students in the UK were Methodology, Second Language
Acquisition, and Teaching Practice. Li and Tin (2013, cited in Copland et al., 2017,
p. 9) looked at the general perceptions of students on a Master’s programme in
TESOL in New Zealand. Their participants identified several strengths of their
programme, including applied linguistics knowledge, opportunities for
reflection, and the realisation that learning to teach is an on-going process. The

375
weaknesses they found were a lack of teaching practice and the fact that the
programme did not consider the socio-cultural context to which the students
would be returning. Focusing on a different type of programme and context, Faez
and Valeo’s (2012) participants were enrolled in a Canadian TESOL certificate
programme preparing teachers to teach ESOL in Canada. The trainee teachers in
this programme found the practicum to be the most useful aspect and suggested
that it should increase in duration. Least useful were the theoretical elements (SLA
and theoretical linguistics), mainly because theory was not linked to practice.
A more limited number of studies have been carried out on SLTE in non-English
speaking countries. One exception is Turkey, where three studies found similar
results (Akcan, 2015; Coskun & Daloglu, 2010; Seferoğlu, 2006). Similar to Li and
Tin (2013, cited in Copland et al., 2017), the trainee teachers in Coskun and
Daloglu’s (2010) study saw the theoretical aspects of their programme as a
strength. However, they also lamented the lack of opportunities for practice, both
in courses on the programme and during the practicum. The lack of connection
between theory and practice was also identified by participants in Seferoğlu’s
(2006) study. Whilst the trainee teachers were generally positive towards the
methodology courses, they perceived a lack of opportunity to implement theory,
and they called for more micro-teaching and opportunities to practice. Akcan’s
(2015) participants were generally positive about their programme and, like the
trainees in Faez and Valeo (2012), they saw the practical teaching experience as a
strength, but they also expressed the need for more practice. They stated that there
was too much theory and not enough practice on the programme. They put
forward several suggestions for improvement, including specific areas of
classroom practice such as more on classroom management, and dealing with
learning disabilities, behavioural problems, and lack of motivation in learners;
better language improvement courses; use of videos of real classrooms;
participation in exchange programmes such as Erasmus; participation in on-line
fora to interact with teachers from other countries.
In a different context and with slightly different results, Martínez Agudo (2017)
looked at TESOL education in Spain. Generally, the participants in this study were
satisfied with the teaching skills, pedagogical knowledge, and practice they
obtained, and they felt prepared to teach. The biggest weakness was the lack of
emphasis on English proficiency, and participants felt there was a lack of balance
among linguistic, pedagogical, and managerial competences. They called for more
specialist subjects related to English as well as for opportunities to teach abroad
and to interact with native speakers. As with the other studies mentioned above,
overall satisfaction with the practical aspects of the programme did not exclude
calls for more practice and less theory. Lack of consideration of context was
another common theme.
Despite the numerous calls for context-appropriate approaches and methodology,

together with calls to move away from native-speaker norms, SLTE still seems to
struggle to adapt to such needs. In a number of the studies cited above, student
teachers refer to the context-appropriacy of their TESOL programmes. Various
studies have noted the lack of fit between BANA-based teacher education and

376
local realities when teachers return home (see, for example, Chowdhury, 2003;
Barnawi & Le Ha, 2015). This possible mismatch raises the question as to whether
teacher education outside BANA countries better reflects the local context.
So far this article has given an overview of developments in the theory and
principles of SLTE. It has also presented a summary of key findings in previous
research on the attitudes and expectations of student teachers on programmes in
both BANA and non-BANA contexts. In the next section, the current situation in
Russia and Uzbekistan is briefly outlined.
3. Teacher education in the Russian Federation and Uzbekistan

Teacher education in the countries of the Ex-Soviet Union was generally
characterised by teacher-centred approaches, at least until the 1990s. Aydarova
(2014) notes that until that time, in Russia, how language teacher knowledge was
conceptualised was indicative of a view of the ‘teacher as knowledge producer’ in
which teachers were also seen as subject specialists and scientists.
In the early 2000s, Russia signed the Bologna Declaration, leading to reforms in
the higher education system and teacher education. Whilst the Bologna Process is
not explicitly concerned with teacher education, its principles are in line with the
constructivist views of SLTE outlined in the previous section. The most recent
curriculum plan for language teacher education at the Bachelor level in Russia
dates back to 2011 and is based on a 4-year programme. In an analysis of the effects
on the Bologna Process on teacher education in Russia, Aydarova (2014) outlines
some criticisms of the most recent plan. These include an increase in the number
of independent study hours, seen as problematic “due to a lack of material
provision for independent work and an absence of a culture of individual
responsibility” (Aydarova, 2014, p. 71). The number of electives, well over half the
subjects, is also seen as an issue leading to a lack of consistency in the knowledge
base that teachers have.
Uzbekistan did not sign the Bologna declaration, but it does adhere to many of its
principles in formulating its educational reforms. As in Russia, initial teacher
education consists of a 4-year bachelor’s degree. In 2013, a new bachelor’s
curriculum, developed by a partnership between the Ministry of Education, the
British Council and the Norwich Institute for Language Education, was
introducediii. The purpose of the reform was to improve both the pedagogical
practice and the English language proficiency of teachers (see, for example,
Gulyamova, Irgasheva & Bolitho, 2014; Isamukhamedova, 2016, for detailed
descriptions of the new curriculum and a comparison with the old one).
Independent research into the outcomes of this reform is not yet available.
However, anecdotal evidence suggests that it may not have been as successful in
bridging the theory-practice gap as hoped and, similar to reform attempts in other
countries, the new ideas and approaches may not be widely finding their way into
EFL classrooms. Hasanova and Shadieva (2008) note the highly bureaucratised
system in Uzbekistan, together with the lack of resources as challenges faced by
both schools and higher education institutions. Although Hasanova and Shadieva

377
(2008) were writing before the most recent reform, it would seem some of the
obstacles remain, and Isamukhamedova (2016) notes there were considerable
limitations on what aspects of the degree programme could be changed.
4. Methodology
As mentioned in the introduction, this study is part of a larger project. The part of
the study reported in this article uses both quantitative and qualitative data, but
is primarily a qualitative study situated within a social-constructivist paradigm
(Cresswell & Poth, 2016). The study used a researcher-designed on-line
questionnaire to ascertain perceptions of the content of SLTE programmes in
Russia and Uzbekistan, together with the attitudes and opinions of those who
have experienced them. The survey was provided electronically through Survey
Monkey. It combined open and closed questions and was distributed through
local project partners using snowball sampling (Taherdoost, 2016). Thus, whilst
the survey was initially distributed amongst students and graduates from the
participating institutions, they were also asked to distribute amongst their wider
networks to obtain a broader view.
The survey items drew on the literature on survey design (see, for example,
Dörnyei & Taguchi, 2009) and were constructed in close collaboration with local
partners to ensure greater local relevance. The survey was distributed in English,
but project partners advised on phrasing and appropriate expression to ensure
mutual understanding as far as possible. Even so, it cannot be discounted that
words such as ‘practicum’ or ‘teaching practice’ may have been interpreted
differently by different respondents. Questions focused on four different areas
relating to:
1. demographics (gender, country/place of work, language(s) spoken;
2. qualifications and experience (qualifications obtained, years of experience,
languages taught, experiences overseas);
3. content of the training course (modules, the practicum);
4. opinions of the training course (strengths, weaknesses, proposed changes).
These were not separate sections in the questionnaire, but rather questions were
given in order of importance of the information for the study on the assumption
that response rates would drop off towards the end of the survey. For example,
country, place of work, and questions concerning the content of the programme
and the respondents’ opinions were the first questions, while questions on gender
and experience came at the end.
A range of closed and open-ended items was used to gain both quantitative and
qualitative responses. The introduction to the questionnaire explained the
purpose and that all responses were anonymous. No personal details were
collected and researcher contact details were given if participants had any
questions or wanted further information. The final data set includes 115
responses from Uzbekistan and 46 from Russia.
The closed questions are presented as raw numbers and percentages to show
general trends. The open questions were coded and categorised to identify the

378
main themes, and themes are included in the discussion where there are at least
five responses from one of the countries. The analysis was also carried out by
country to investigate where there were notable differences, enabling
comparisons to be made. Given the snowball sampling and the different number
of responses from each country, the study does not aim at generalisability. Rather,
it intends to give a snapshot of SLTE in particular settings at a particular time.
5. Findings
This section presents the key findings from both the closed and open questions. It
starts with a brief overview of the experience of the respondents and the subjects
that are offered in their programmes. The majority of the section then focuses on
the responses to the open questions about the successful aspects of the
programmes, what was missing, and what the respondents would change. For
each question, the data are presented first in tables showing the main themes
together with the numbers and percentages of responses. The tables are followed
by examples from the data which add explanatory power to the numerical
elements. The examples are reproduced exactly as they were written by the
respondents. The numbers in the tables do not correspond to the overall numbers
of responses as many responses contained more than one theme, while other
responses were unique.
The majority of questionnaire respondents in both countries were already

working, with just 19% saying that they were not currently teaching. Of those who
were working, 70% had between one and five years’ experience. This means that
most participants had some understanding of how relevant their programme was
to actual classrooms.
To ascertain the content of current teacher training provision, a list of 17 common

subjects was given for respondents to say which were covered in their
programmes. The wording of the question (which of the following areas do
you/did you cover on your teacher training courses?) was designed to avoid
words such as ‘module’ or ‘course’, which may have different meanings in
different contexts. Therefore, the answers do not give any indication as to whether
the subject was a whole course, just a few classes, or even one class. There were
44 responses from Russia and 96 from Uzbekistan for this question.
Of the 17 subject areas, seven were common to more than 50% of the respondents
in both countries (see Table 1 - percentages are rounded to the nearest whole
number.). Language teaching pedagogy/methodology was the most common
with over 90% in both countries saying it was part of their course. Teaching
practice and Psychology were also relatively common to both countries. However,
there are notable differences amongst the other areas.
Table 1. Courses covered in the programmes

Country RF Uz
N= % N= %
Language Teaching 41 93% 87 91%
Pedagogy/ methods
Teaching practice 35 80% 62 65%

379
Psychology 33 75% 64 67%

Linguistics 36 82% 53 55%
Literature 31 70% 52 54%
Culture 27 61% 50 52%
Second language acquisition 22 50% 52 54%
Government standards/ 33 75% 44 46%

requirements
Materials 24 55% 41 43%
Knowledge about the target 23 52% 36 38%
language
Special educational needs 22 50% 19 20%
Motivation 22 50% 27 28%
While a total of 12 subject areas were common to more than half the Russian
respondents, that number was only seven for Uzbekistan. Moreover, the
percentages in the latter were relatively lower. This result would seem to imply
that there is more consistency across teacher training programmes in Russia.
Given Aydarova’s (2014) observations that Russian teacher education is
traditionally highly regimented and based on continuity, coherence, and
systematicity, it may be that there is more consistency in the knowledge-base of
Russian language teachers in the recent reforms of SLTE than Aydarova (ibid)
believed. However, the numbers are far from implying a standardised experience.
More difficult to interpret is the apparent variety in coverage in programmes in
Uzbekistan. It may be that there is a greater number of electives on the
programmes or that the curriculum is not being implemented consistently.
Further research is needed here, but there would appear to be a certain
fragmentation in the knowledge base of novice teachers in both contexts.
Interestingly, there were two subject areas that no course covered: classroom
management or learner strategies. These gaps could be an indication that the very
practical aspects of language teaching are lacking (see below).
There were 78 responses from Uzbekistan 34 responses from Russia to the open
question about what was felt to be the most successful aspects of the programme.
Answers were varied but covered four main themes (see Table 2):
1. Methodology, both in general and specific aspects of teaching
2. Learning the language or about the language
3. Theoretical educational courses/topics
4. Theoretical linguistics courses/topics
Table 2. The most successful aspects of programmes

Theme/country Ru Uz
N= % N= %
Methodology 21 62% 45 58%
The language 7 21% 28 38%
Theory - 7 21% 16 21%
education
Theory - 4 12% 13 17%
linguistics

380
Methodology, or methods of teaching, was by far the most common answer in

both countries, with 58% of the Uzbek respondents and 62% of the Russian
respondents listing this aspect as most successful. Whilst most answers did not
give details, simply writing ‘methods’ or ‘methodology’, some were more specific,
for example:
New approach (student-centred, skill-oriented) (Uz)
Content based learning, language teaching methodology (Uz)
Lesson plans, teaching skills, usage of different types of assessment (Ru)
New technologies, planning lessons, teaching speaking (Ru)
The second most common theme was learning the language or learning about the
language, although this was more of a strength for the Uzbek teachers (38%). In
particular, they listed learning about different aspects of language such as
grammar, lexis, and phonetics, as well as opportunities to practise.
Knowledge about the target language (Ru)
Study of vocabulary, lexis grammar (Uz)
English practice (Uz)
Theoretical linguistics courses, such as lexicography and stylistics, and what

might be called theoretical educational courses, such as psychology and pedagogy
were also mentioned. However, the numbers were much smaller (between 12%
and 21%).
The question concerning what was missing from the programme elicited 30
responses from Russia and 78 from Uzbekistan. There was a wide range of very
specific answers, the majority of which could loosely be grouped around four
themes, but with different responses from each country (Table 3)
1. Language practice
2. Aspects of methodology
3. Education content
4. Linguistic content
Table 3. Elements missing from the programme

Theme/country Ru Uz
N= % N= %
Language 4 13% 32 41%
Methodology 13 43% 14 18%
Education 4 13% 5 8%
Linguistics 1 3% 7 9%
While methodology was given as one of the successful aspects, it also figured
strongly in what was missing from programmes. In particular, respondents
picked out specific aspects of how to teach that they did not learn. For example:
Some modern approaches and methods should be added to teacher training
courses such as learner-centred teaching. (Uz)
Material design sessions and assessment (Uz)
How to deal with groups of students with mixed abilities (Ru)
Learning to use the whiteboard and corresponding software (Ru)

381
One Uzbek respondent felt that the focus of methodology on the course was too
narrow:
Some training programmes are based only on the principles of the
communicative approach in language teaching (Uz)
These answers would seem to indicate that programmes may be giving students
a good overview of methods and approaches to language teaching, but they are
lacking in the more practical, classroom-based aspects of teaching. It would also
seem that teacher training programmes outside the BANA countries are adopting
western approaches.
Respondents identified a number of content areas that were lacking. Seven Uzbek
respondents wanted more theoretical courses, with the history of language,
lexicography, and stylistics all mentioned. Intercultural communication, teaching
young learners, and special education needs were the more educational content
courses requested.
By far the most common missing element in Uzbekistan concerned language

practice, which was identified by 32 of the 78 respondents, with 16 using the word
‘communication’ in their reply. Answers concerned communication in general as
well as different aspects of language proficiency, especially speaking. For
example:
communication (speaking) (listening) (Uz)
oral speech practice (Uz)
Moreover, 13 of the answers specified that the language practice should be with
native speakers:
Communication with native speakers (Uz)
Live talks with Englishmen (sic) was missing from the programme (Uz)
Connected to this, both Russian and Uzbek respondents felt that the involvement
of native speakers and international experts was missing from their training
experiences:
Training by other (international) specialists who can share their
experience; more lessons with native speakers (Uz)
observation of a teaching/learning process in a target language country
(Ru)
Finally, respondents were asked what they would change in their teacher training
programme if they had the opportunity, with 72 responses from Uzbekistan and
35 from Russia. To an extent, the responses mirrored what was identified as
missing from the programme. However, there were important differences, and
there was also a more limited range of answers, which would seem to indicate a
clear set of priorities for these novice teachers. The majority of responses from
both countries can be categorised into four main themes (Table 4):
1. Internationalisation
2. Methodology
3. Teaching practice
4. Language practice

382
Table 4. What is missing from programmes

Theme/Country Ru Uz
N= % N= %
Internationalisation 5 14 21 29
Methodology 4 11 19 26
Teaching practice 13 31 7 10
Language practice 3 9 14 19
What might be termed ‘Internationalisation’ covered different aspects. These

included international exchange programmes, visits/practice overseas, as well as
more communication and exchanges with native speakers or international
experts. Some typical comments included:
Add more classes with a native speaker professional in the field of TEFL
(Ru)
I would send teachers to the country whose language they are teaching, to
practise (Uz)
I would add the opportunity of participation in international programms
(Ru)
Practice with native speakers (Uz)
Generally, the Russian responses focused more on in-country initiatives, while

there was a greater call to be able to go overseas from the Uzbek teachers. This
may be because Uzbekistan is currently undertaking an extensive programme of
overseas training for its language teachers, and so this is a more realistic
possibility than perhaps it is for the Russian teachers.
The second area was around methodology and a more practical approach to
teacher education. Some teachers made quite general comments about the
methodology aspect of their programme and what they would like to change. For
example:
Not enough attention was given to methodology – this is why students
feel lost when they go into teaching practice (Uz)
I would try to raise training participants’ awareness about different
approaches in language teaching (Uz)
Add more practice oriented subjects (Ru)
However, some respondents also identified very specific areas of English

language teaching as needed, most of which are very practical, and classroom-
based. For example:
Learning how to use more English in English lessons (Ru)
Teaching by the help of multimedia would be more interesting and useful
(Uz)
Add more specific topics such as assessment, target language, time
management and content-based learning. (Uz)
Overall, the data raise potentially interesting issues around teaching practice and
the practicum. Only three replies from Uzbekistan and five from Russia indicated
this aspect as successful on their programme, and even fewer indicated it as
something that was missing (just one from Russia and one from Uzbekistan).

383
More responses indicated it as an aspect that teachers would like to change, but
these are still fewer than may have been expected, given the findings of previous
research outlined above. Comments included:
Pay more attention to the school practice (Ru)
More practice at schools less theory (Ru)
I would give more time to teaching practice and specialist subjects (Uz)
The final area that respondents would like to change concerns language practice,
especially speaking practice. Russian respondents called for more hours of
English in general, while the Uzbek teachers also identified the aspects they
wanted to improve, especially speaking and listening.
spend more time on the language practice (Ru)
more speaking and listening (Uz)
Make much hours of speaking clubs (Uz)
One slight caveat here is that it was not always possible to distinguish whether
‘practice’ referred to teaching practice or language practice. However, there were
enough specific calls for more of each to conclude that these are both areas where
change is desired.
5. Discussion and Implications

Perhaps unsurprisingly, methodology figured strongly in all aspects of this study.
It was seen as a strength of the programmes, but there were relatively frequent
calls for both more on methods of teaching and more practical approaches to it.
Various specific practical areas such as techniques for teaching young learners,
using technology, or managing classes were identified. These results are in line
with previous studies (see, for example, Coskun & Daloglu, 2010; Faez & Valeo,
2012; Martínez Agudo, 2017; Seferoğlu, 2006) and it would seem that, from a
student’s point of view, programmes and courses can never be too practical.
However, the theoretical underpinnings of practice are an important part of the
knowledge base of teaching, and, while theoretical courses were seen as a strength
of the programmes by some respondents (see also Li & Tin, 2013, cited in Copland
et al., 2017), it seems likely that they fail to make the links to practice. As one
Russian participant put it:
Too much theory and not enough exercises or sample tasks which can help
teach according to this theory. (Ru)
Teaching practice and the practicum, whilst mentioned by respondents

particularly as something to change, were nevertheless given less importance than
in previous studies (Akhan, 2015; Faez & Valeo, 2012). While there were calls for
more school-based practice, this was not among the most commonly identified
areas, nor was teaching practice seen as a strength or as something that was
missing from the programmes. These data are difficult to interpret without further
research, but one explanation could be that participants see school-based teaching
practice as something separate from their institutional programmes and therefore
not within the scope of the questions. If this is the case, then it indicates a
significant disconnect between formal classes and the practicum.

384
Like the Spanish respondents in Martínez Agudo’s (2017) study, Uzbek

respondents, in particular, were concerned with language proficiency and
opportunities to develop their language skills. The current study also found a
strong desire for contact with native speakers and experience overseas, which is
similar to Faez and Valeo’s (2012) findings and implies that native speaker
competence is still seen as the benchmark, even in these days of English as a
Lingua Franca. This reflects Cameron and Galloway’s (2019) assertion that,
despite the recognition in academic circles that English classrooms need to reflect
multilingual norms and how English functions globally, a bias towards native
speaker norms is still strong in TESOL.
The results of the study have several practical implications for SLTE, which
formed the basis of the response of the project in which this study was embedded.
(1) Methodology and specialist courses should have a sound theoretical basis, but
the theory needs to be explicitly linked to practice. The project prepared a
series of very practical content coursesiv based on the idea of loop input
(Woodward, 2003). Novice teachers, therefore, experienced first-hand the
type of learner-centred teaching, as well as the specific strategies and
techniques that they could use in their own teaching.
(2) To address concerns about language proficiency, and increase opportunities
to use the target, as well as increase practical input, the principles of content
and language integrated learning (CLIL) (Coyle, Hood & Marsh, 2010) can be
used in language preparation courses. The project prepared an English
language improvement course based on CLIL principles with the dual aim of
improving both English proficiency and practical teaching skills. This means
that all the content of the course is related to the SLTE programme, so that, for
example, all reading and listening texts were on topics related to language
teaching.
(3) Novice teachers need to be exposed to the idea of English as a Lingua Franca
and the acceptance of varieties of English outside so-called native-speaker
varieties to relieve the pressure of achieving the often-unattainable goal of
native-like proficiency. In the project, debates around varieties of English and
ELF, as well as research into language acquisition, were introduced in the
Methodology and Age Appropriate Pedagogy courses. Students were
presented with the arguments against both the feasibility and desirability of
native-speaker norms in ELF contexts and the value of their own experiences
as proficient speakers of English was promoted to build confidence.
(4) Opportunities for practical adaptation/implementation of theories, and
reflection on them, need to be built into all aspects of teacher training, through,
for example, the use of creative assessments. Whilst making changes to the
practicum or teaching practice was beyond the remit of the project,
opportunities for practical applications were built in to all the new courses.
These took the form of, for example, preparing lesson plans or designing
materials with a specific theoretical rationale, designing and evaluating uses
of software, and so on.
(5) The content of SLTE programmes needs to reflect the realities of classroom
teaching and educational cultures of the local context and not unquestioningly
follow models imported from BANA countries. All materials in the project

385
were prepared by teams of teacher trainers from the three project partner
countries (including Ukraine) to ensure that all materials were appropriate to
the local context in both content and approach. The final results can be
described as a balance between transmission and constructivist approaches
which is more appropriate to the contexts for which they were prepared.
6. Conclusion
This article has investigated the content of initial teacher training programmes in
Russia and Uzbekistan, and has identified some trends that merit further research
There are, however, a number of limitations to the research reported here. First of
all, the data are only from two countries and the number of participants is
relatively small. Moreover, snowball sampling was used, so findings cannot be
generalised. It would be useful to carry out much larger-scale quantitative studies
to generalise findings. The research instrument used also has its limitations.
Although the open questions allow the teachers’ voices to be heard to an extent,
interviews would have enabled a deeper and more detailed understanding of the
issues. Interviews would also have allowed for clarification of unclear or
ambiguous responses. Finally, this study did not collect data from teacher
educators. Such data would be useful to enable comparisons between the
perceptions of the novice teachers, and those of their teacher educators.
Whilst it may be the case that no programme can ever be too practical from a
novice teacher’s perspective, it is also important not to sacrifice theory to practice.
Ultimately, sound theoretical underpinnings will give teachers the knowledge
base they need to make informed decisions during their careers. Training
programmes, therefore, need to focus on bridging the gap between theory and
practice, whilst managing trainee teachers’ expectations. Although SLTE has been
moving in this direction, at least in principle, for over 20 years, it seems there is
still some way to go.
7. References
Akcan, S. (2015). Novice non-native English teachers’ reflections on their teacher
education programmes and their first years of teaching. PROFILE, 18(1), 55-70.
https://doi.org/10.15446/profile.v18n1.48608
Aydarova, O. (2014). Universal principles transform national priorities: Bologna Process
and Russian teacher education. Teaching and Teacher Education, 37, 64-75.
https://doi.org/10.1016/j.tate.2013.10.001
Barnawi, O., & Le Ha, P. (2015). From western TESOL classrooms to home practice: a case
study with two ‘privileged’ Saudi teachers. Critical Studies in Education, 56(2), 259-
276. https://doi.org/10.1080/17508487.2014.951949
Cameron, A., & Galloway, N. (2019). Local thoughts on global ideas: pre- and in-service
TESOL practitioners’ attitudes to the pedagogical implications of the globalization
of English. RELC JournaL, 50(1), 149-163.
https://doi.org/10.1177/0033688218822853
Canh, L. V. (2014). Great expectations: The TESOL practicum as a professional learning
experience. TESOL Journal, 5(2), 199-224. https://doi.org/10.1002/tesj.103
Cresswell, J. W., & Poth, C. N. (2016). Qualitative inquiry and research design: Choosing among
five approaches. Thousand Oaks, CA: Sage.

386
Chowdhury, M. R. (2003). International TESOL training and EFL contexts: The cultural
disillusionment factor. Australian Journal of Education, 47(3), 283-302.
https://doi.org/10.1177/000494410304700307
Copland, F., Garton, S., & Burns, A. (2014). Challenges in teaching English to young
learners: Global perspectives and local realities. TESOL Quarterly, 48(4), 738-762.
https://doi.org/10.1002/tesq.148
Copland, F., Viana, V., Bowker, D., Moran, E., Papageorgiou, I., & Shapira, M. (2017). ELT
master’s courses in the UK: students’ expectations and experiences. ELT Research
Papers, 17.04. London: British Council.
Coskun, A., & Daloglu, A. (2010). Evaluating an English language teacher education
program through Peacock’s model. Australian Journal of Teacher Education, 35(6),
24-42. https://doi.org/10.14221/ajte.2010v35n6.2
Coyle, D., Hood, P., & Marsh, D. (2010). Content and language integrated learning.
Cambridge: Cambridge University Press.
Dörnyei, Z., & Taguchi, T. (2009). Questionnaires in second language research: Construction,
administration, and processing. New York, NY: Routledge.
https://doi.org/10.4324/9780203864739
Faez, F., & Valeo, A. (2012). TESOL teacher education: novice teachers’ perceptions of their
preparedness and efficacy in the classroom. TESOL Quarterly, 46(3), 450-471.
https://doi.org/10.1002/tesq.37
Farrell, T. S. (2015). Second language teacher education: A reality check. In T. S. Farrell
(Ed.), International perspectives on English language teacher education (pp. 1-15).
London: Palgrave Macmillan. https://doi.org/10.1057/9781137440068_1
Freeman, D. (2020). Arguing for a knowledge-base in language teacher education, then
(1998) and now (2018). Language Teaching Research, 24(1), 5-16.
https://doi.org/10.1177/1362168818777534
Freeman, D., & Johnson, K. E. (1998). Reconceptualizing the knowledge‐base of language
teacher education. TESOL Quarterly, 32(3), 397-417.
https://doi.org/10.2307/3588114
Gulyamova, J., Irgasheva, S., & Bolitho, R. (2014). Professional development through
curriculum reform: the Uzbekistan experience. In D. Hayes (Ed.), Innovations in
continuing professional development of English language teachers (pp. 45-62). London:
British Council.
Graves, K., & Garton, S. (2017). An analysis of three curriculum approaches to teaching
English in public-sector schools. Language Teaching, 50(4), 441-
482. https://doi.org/10.1017/s0261444817000155
Hardman, J., & Rahman, N. A. (2014). Teachers and the implementation of a new English
curriculum in Malaysia. Language, Culture and Curriculum, 27(3), 260-277.
https://doi.org/10.1080/07908318.2014.980826
Hasanova, D., & Shadieva T. (2008). Implementing communicative language teaching in
Uzbekistan. TESOL Quarterly, 42(1), 138-143. https://doi.org/10.1002/j.1545-
7249.2008.tb00215.x
Hennissen, P., Beckers, H., & Moerkerke, G. (2017). Linking practice to theory in teacher
education: A growth in cognitive structures. Teaching and Teacher Education, 63,
314-325. https://doi.org/10.1016/j.tate.2017.01.008
Holliday, A. (1994). Appropriate methodology and social context. Cambridge: Cambridge
University Press.
Isamukhamedova, N. (2016). Curriculum for preparing teachers of English in Uzbekistan:
now and then. Humanising Language Teaching, 18(6).
https://old.hltmag.co.uk/dec16/mart02.htm

387
Johnson, K. E. (2016). Language teacher education. In G. Hall (Ed.), The Routledge handbook
of English language teaching (pp. 139-152). London: Routledge.
https://doi.org/10.4324/9781315676203
Martínez Agudo, J. (2017). What EFL student teachers think about their professional
preparation: Evaluation of an English language teacher education programme in
Spain. Australian Journal of Teacher Education, 42(8), 62-76.
https://doi.org/10.14221/ajte.2017v42n8.5
Rahman, M. M., & Pandian, A. (2018). A critical investigation of English language teaching
in Bangladesh: Unfulfilled expectations after two decades of communicative
language teaching. English Today, 34(3), 43-49.
https://doi.org/10.1017/s026607841700061x
Richards, J. (2008). Second language teacher education today. RELC Journal, 39(2), 158-177.
https://doi.org/10.1177/0033688208092182
Seferoğlu, G. (2006). Teacher candidates’ reflections on some components of a pre-service
English teacher education programme in Turkey. Journal for Education in Teaching,
32(4), 369-378. https://doi.org/10.1080/02607470600981953
Taherdoost, H. (2016). Sampling methods in research methodology; how to choose a
sampling technique for research. International Journal of Academic Research in
Management, 5(2), 18-27. https://doi.org/10.2139/ssrn.3205035
Walsh, S., & Mann, S. (Eds.). (2019). The Routledge handbook of English language teacher
education. London: Routledge. https://doi.org/10.4324/9781315659824
Woodward, T. (2003). Key concepts in ELT: Loop input. ELT Journal, 57(3), 301-304.
https://doi.org/10.1093/elt/57.3.301
Wright, T. (2010). Second language teacher education: Review of recent research on
practice. Language Teaching, 43, 259-296.
https://doi.org/10.1017/S0261444810000030
Yin, J. (2019). Connecting theory and practice in teacher education: English-as-a-foreign-
language pre-service teachers’ perceptions of practicum experience. Innovation
and Education, 1(4), 1-8. https://doi.org/10.1186/s42862-019-0003-z
This project has been funded with support from the European Commission. This
publication reflects the views only of the author, and the Commission cannot be held
responsible for any use which may be made of the information contained therein.
iNovice teacher will be used to refer to students studying on a pre-service SLTE programme
or who have recently completed one. Programme will be used to refer to the overall
curriculum leading to a teaching qualification, and course will be used to describe a
subject within a programme, e.g. Methodology.
iiThe original project also included institutions from Ukraine. However, only 14
questionnaire responses were obtained from Ukraine and almost half of the
respondents taught in a university context. Therefore, responses have not been
considered here
iiihttps://www.britishcouncil.uz/en/teach/reform-project
iv These were Age Appropriate Pedagogy, Course Design and Evaluation, Foreign
Language Teaching Methodology, Technology and Language Teaching.

PUBLISHER International Journal of Learning, Teaching and
Society for Research and Knowledge Management Educational Research
Port Louis
Republic of Mauritius
www.ijlter.org The International Journal of Learning, Teaching
and Educational Research is an open-access
Chief Editor journal which has been established for the dis-
Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the
Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER
welcomes research articles from academics, ed-
Editorial Board
ucators, teachers, trainers and other practition-
Dr. Fatima Zohra Belkhir-Benmostefa
ers on all aspects of education to publish high
Dr. Giorgio Poletti
quality peer-reviewed papers. Papers for publi-
Dr. Chi Man Tsui
cation in the International Journal of Learning,
Dr. Fitri Suraya Mohamad
Teaching and Educational Research are selected
Dr. Hernando Lintag Berna
through precise peer-review to ensure quality,
Dr. Charanjit Kaur Swaran Singh
originality, appropriateness, significance and
Dr. Abu Bakar
readability. Authors are solicited to contribute
Dr. Eglantina Hysa
to this journal by submitting articles that illus-
Dr. Mo'en Salman Alnasraween
trate research results, projects, original surveys
Dr. Hermayawati Hermayawati
and case studies that describe significant ad-
Dr. Selma Kara
vances in the fields of education, training, e-
Dr. Michael B. Cahapay
learning, etc. Authors are invited to submit pa-
Dr. Bunmi Isaiah Omodan
pers to this journal through the ONLINE submis-
Dr. Vassiliki Pliogou
sion system. Submissions must be original and
Dr. Meera Subramanian
should not have been published previously or
Dr. Muhammad Kristiawan
be under consideration for publication while
Dr. Wahyu Widada
being evaluated by IJLTER.
Dr. Som Pal Baliyan
Dr. Reem Khalid Abu-Shawish
Dr. Froilan Delute Mobo
Dr. Mohamed Ali Elkot
Dr. Anabelie Villa Valdez
Mr. Teody Lester Verdeflor Panela

Vol 19 No 10 October 2020

Uploaded by

Copyright:

Available Formats

You might also like

Vol 19 No 10 October 2020

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 19 No 10 October 2020

Uploaded by

Copyright:

Available Formats

IJLTER.

Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Aims and Objectives

The main objective of this journal is to provide a platform for educators,

Indexing and Abstracting

The International Journal of Learning, Teaching and Educational

We are very happy to publish this issue of the International Journal of

The International Journal of Learning, Teaching and Educational

We are grateful to the editor-in-chief, members of the Editorial Board

The Editorial Board is composed of renowned people from across the

We will endeavour to ensure the reputation and quality of this journal

Editors of the October 2020 Issue

Active Learning in Economic Subject: A Case Study at Secondary School ................................................................... 19

Rethinking Privilege in Teaching English in Japanese Higher Education .................................................................. 100

International Journal of Learning, Teaching and Educational Research

Using Systematic Synthetic Phonics to Accelerate

Jia Rong Yap

Mellisa Lee Lee Chin

Abstract. Studies focusing on the strategy of phonics in Malaysia have

Keywords: early literacy; English language learners; indigenous children;

©2020 The authors and IJLTER.ORG. All rights reserved.

However, despite the Malaysian government’s substantial investment in revising

©2020 The authors and IJLTER.ORG. All rights reserved.

Teaching of Literacy have acknowledged the centrality of SSP in accelerating the

©2020 The authors and IJLTER.ORG. All rights reserved.

type of phonics instructions, defined as teaching practices that are designed to

2.2 Systematic Synthetic Phonics

2.3 Systematic Synthetic Phonics Programme

©2020 The authors and IJLTER.ORG. All rights reserved.

Pseudowords are a pronounceable combination of graphemes that have the

Table 1: Summarised details of phases in a stage in SSP

©2020 The authors and IJLTER.ORG. All rights reserved.

Figure 1. An overview of the Systematic Synthetic Phonics programme

©2020 The authors and IJLTER.ORG. All rights reserved.

training) caused an effect on the dependent variable (the children’s reading

Table 2: Participants’ background

3.2 Phonics Training Procedure

©2020 The authors and IJLTER.ORG. All rights reserved.

3.3 Training Scope and Sequence

3.3.1 Systematic Synthetic Phonics (SSP) Training

3.3.2 SBELC Phonics Training

©2020 The authors and IJLTER.ORG. All rights reserved.

Productive Letter-Sound Test (PLST). This test measured the participants’

©2020 The authors and IJLTER.ORG. All rights reserved.

Table 3: Free Sound-Isolation Test

Table 4: Summary of words formed from stages A – E

©2020 The authors and IJLTER.ORG. All rights reserved.

3.5 Data Analysis

©2020 The authors and IJLTER.ORG. All rights reserved.

4. Findings and Discussion

Table 5: Descriptive statistics

Table 6: Independent samples t-test

4.2 The Relative Effect of SSP and SBELC Phonics Training

©2020 The authors and IJLTER.ORG. All rights reserved.

Table 7. Paired samples descriptive statistics for the experimental group

Table 8. Paired samples t-test for the experimental group

©2020 The authors and IJLTER.ORG. All rights reserved.